Improve the retry strategy: Support the Retry-After header & exponential retries

[rhamzeh]

Checklist

• I agree to the terms within the OpenFGA Code of Conduct.

Describe the problem you'd like to have solved

Retry-After is a standard header used by APIs to indicate when the SDK can retry.

The SDKs should:

• Honor this header on 429s
• Expose this header value in the error when received
• Fallback to exponential retry when this header is not available (e.g. not sent by the server on 429s or on e.g. 500s)
• Drop support for retrying based on X-Rate-Limit-Reset (currently only .NET SDK supports that), though still expose it in the logs

Current State

SDK	Retries on	Default Num Retries	Max Num Retries	State
JS	429s, 500s	3	15	Does not consider headers. Implements exponential backoff, with the following algorithm 2^loopCount * 100ms and 2^(loopCount + 1) * 100ms
Go	429s, 500s except for 501	3	15	Does not consider headers. Implements exponential backoff, with the following algorithm 2^loopCount * 100ms and 2^(loopCount + 1) * 100ms
.NET	429s, 500s except for 501	15, being reduced to 3 in an upcoming sync	15	Honors the X-Rate-Limit-Resetheader, treats it as number of seconds in the future. Does not fall back to exponential backoff.
Python	429s, 500s except for 501	3	15	Does not consider headers. Implements exponential backoff, with the following algorithm 2^loopCount * 100ms and 2^(loopCount + 1) * 100ms
Java	429s, 500s except for 501	3	15	Does not consider headers. Does not implement exponential backoff. Adds a fixed 100ms delay to each retry
CLI	follows the Go SDK	follows the Go SDK	follows the Go SDK	follows the Go SDK

Describe the ideal solution

Retry On

• For Write:
  • Retry on 429s, falling back to exponential backoff
  • Retry on 5xxs (except 501 not implemented) only if the Retry-After headers are sent - do not fall back to exponential backoff
• For all others:
  • All 429s, and >=500 (except 501 not implemented)

Max Allowable Retries

15

Default Number of Retries

SDKs: 3
CLI: 15

Retry Parameters

1. If Retry-After header is found, use it

   1. if it is an integer, treat it as the number of seconds from now to retry, if it is <1 from now or >1800 from now (aka >30 min) - assume it is invalid and continue
   2. if it is a date, parse it but if it is <1 from now or >1800 from now (aka >30 min) - assume it is invalid and continue

2. If neither header is found, use exponential backoff but we'll add some jitter, so the retry is a random number between

   1. 2^loopCount * 500ms and 2^(loopCount + 1) * 500ms
   2. if the result of (a) is > 120s, cap it at 120s which should happen between the 8th and 9th retry

That means:

• if retry-after header was returned and is valid, we’ll use it - so if it says in 4 min all good
• if retry-after header was not returned, we will retry at:
  • 500ms
  • 1s
  • 2s
  • 4s
  • 8s
  • 16s
  • 32s
  • 64s
  • 120s ← at this point is is >4min since initial call
  • 120s
  • 120s
  • 120s
  • 120s
  • 120s

Alternatives and current workarounds

No response

References

• Use http rate limiting headers to decide when to retry requests go-sdk#156
• Document the standard Rate Limit headers that the SDK implements openfga.dev#641
• Incorrect interpretation of x-ratelimit-reset for Okta dotnet-sdk#76

Additional context

No response