Rate limitting for backing up & restore to gmail account

[daniellefisla]

1. To make the gyb.py script work (main branch) I had to insert the sleep() call and use batch size of 5 to pull about 40K messages

def callGAPI(service, function, soft_errors=False, throw_reasons=[], retry_reasons=[], **kwargs):

time.sleep(2) # sleep for X second(s) to avoid hitting the rate limit --batch-size 5 works

this is a hack, should be some parameter.

2. for restore, I got errors with fractional timestamp precision, so I had to chage the below to make this work:

FROM:

def convert_timestamp(val):
    """Convert Unix epoch timestamp to datetime.datetime object."""
    # yuck, we aren't actually storing timestamps in the GYB
    # database. I blame the original developer :-)
    # return datetime.datetime.fromtimestamp(int(val))
    return datetime.datetime.strptime(val.decode('UTF-8'), '%Y-%m-%d %H:%M:%S')

TO:

def convert_timestamp(val):
    val = val.decode('UTF-8')  # Assuming `val` is a bytes object that needs decoding.
    # Check if the datetime string contains fractional seconds.
    if '.' in val:
        # If yes, parse with fractional seconds.
        format_str = '%Y-%m-%d %H:%M:%S.%f'
    else:
        # If no, parse without fractional seconds.
        format_str = '%Y-%m-%d %H:%M:%S'
    return datetime.datetime.strptime(val, format_str)

to handle both values with and without factional values.