[pivotal] Use client-side rate limiting to avoid request throttling

Shortcut's REST API has an advertised rate limit of 200 requests per
minute. If 200 or more requests are made within one minute, API
clients should expect to receive responses with HTTP status code 429
until enough time has past for the total number of requests made by
their API token over the course of the last minute to have dropped
below 200.

This commit introduces rate limiting on the client (this importer)
side, such that users of this pivotal-import tool should never be
throttled by Shortcut's API.

Since the importer is single-threaded and the rate limiting is handled
by an algorithm operating entirely within the Python process' memory,
the script should never pause for more than the max_delay set and
should never throw an exception related to the rate limit delay being
exceeded. It has been configured to throw an exception in case these
assumptions do not hold, so that errors are not passed by silently.

The rate has been set to 195 requests per minute to keep us
comfortably below the rate limit threshold (which is 199 inclusive)
without wasting too much possible capacity.

The max_delay has been set to 70 seconds to ensure (1) in a case where
the max number of requests are made within 1 second, we have a full
minute to regain capacity on the Shortcut API side without being
throttled by the 200 reqs per minute limit, while also (2) adding a
buffer of 10 seconds (1 min == 60 seconds, 60 + 10 == 70 seconds) to
account for possible mismatched clock times between the client running
this script and the Shortcut API server accepting requests.

Co-authored-by: Toby Crawley <[email protected]>

pivotal-import/lib.py

@@ -14,11 +14,43 @@
 import os
 import logging
 
+from pyrate_limiter import Duration, InMemoryBucket, Limiter, Rate
 import requests
 
 # Logging
 logger = logging.getLogger(__name__)
 
+# Rate limiting. See https://developer.shortcut.com/api/rest/v3#Rate-Limiting
+# The Shortcut API limit is 200 per minute; the 200th request within 60 seconds
+# will receive an HTTP 429 response.
+#
+# The rate limiting config below sets an in-memory limit that is just below
+# Shortcut's rate limit to reduce the possibility of being throttled, and sets
+# the amount of time it will wait once it reaches that limit to just
+# over a minute to account for possible computer clock differences.
+max_requests_per_minute = 200
+rate = Rate(max_requests_per_minute - 5, Duration.MINUTE)
+bucket = InMemoryBucket([rate])
+max_limiter_delay_seconds = 70
+limiter = Limiter(
+    bucket, raise_when_fail=True, max_delay=Duration.SECOND * max_limiter_delay_seconds
+)
+
+
+def rate_mapping(*args, **kwargs):
+    return "shortcut-api-request", 1
+
+
+rate_decorator = limiter.as_decorator()
+
+
+def print_rate_limiting_explanation():
+    printerr(
+        f"""[Note] This importer adheres to the Shortcut API rate limit of {max_requests_per_minute} requests per minute.
+       It may pause for up to {max_limiter_delay_seconds} seconds during processing to avoid request throttling."""
+    )
+
+
 # API Helpers
 sc_token = os.getenv("SHORTCUT_API_TOKEN")
 api_url_base = "https://api.app.shortcut.com/api/v3"
@@ -30,6 +62,7 @@
 }
 
 
+@rate_decorator(rate_mapping)
 def sc_get(path, params={}):
     """
     Make a GET api call.
@@ -43,6 +76,7 @@ def sc_get(path, params={}):
     return resp.json()
 
 
+@rate_decorator(rate_mapping)
 def sc_post(path, data={}):
     """Make a POST api call.
 
@@ -59,6 +93,7 @@ def sc_post(path, data={}):
     return resp.json()
 
 
+@rate_decorator(rate_mapping)
 def sc_put(path, data={}):
     """
     Make a PUT api call.
@@ -73,6 +108,7 @@ def sc_put(path, data={}):
     return resp.json()
 
 
+@rate_decorator(rate_mapping)
 def sc_delete(path):
     """
     Make a DELETE api call.

pivotal-import/pivotal_import.py

@@ -509,6 +509,7 @@ def main(argv):
 
     cfg = load_config()
     ctx = build_ctx(cfg)
+    print_rate_limiting_explanation()
     process_pt_csv_export(ctx, cfg["pt_csv_file"], entity_collector)
 
     created_entities = entity_collector.commit()