A practical retry budget for OpenAI-compatible gateway clients

Retries are useful only when they are bounded. In a small coding-agent test, a request budget of three attempts—one immediate retry, then waits of 250 ms and 1 s—handled transient 429 and 502 responses without turning a failed call into a long queue.

The important detail is to keep the retry policy at the client boundary. The gateway should receive the same request shape on each attempt, while the client records the status code, elapsed time, and model identifier. Do not retry authentication failures or malformed requests. For a concrete OpenAI-compatible setup, APIClaw documents the base URL and client configuration in its developer documentation.

Suggested policy

1. Retry 429, 500, 502, 503, and 504, subject to a total deadline.
2. Use exponential backoff with jitter rather than a fixed sleep.
3. Stop after three attempts or when the deadline is reached.
4. Log the final failure separately from intermediate attempts.

A flat-rate gateway changes the accounting question, but not the engineering constraint: an unbounded retry loop can consume the daily request allowance quickly. Measure successful completions and total attempts separately. APIClaw describes its request-metered tiers and daily reset on the flat-rate API page.