A retry policy is a probability model, not a vibe. Given a per-attempt failure rate for
429 rate_limit_error, 529 overloaded_error, 500 api_error or a dropped
stream, this tool computes the exact odds of succeeding within n retries, the expected and
worst-case wall clock the schedule burns, the retry at which you cross your hard deadline, and the
probability of blowing that deadline. It then inverts the same equations to recommend the minimum
retry count for your SLO and the largest delay cap that still fits the budget.
Generated from the inputs above. MAX_RETRIES and API_TIMEOUT_MS are read as
environment variables by the wrapper below; the equivalent Anthropic SDK client options are
max_retries and timeout.
Retry i (indexed from 0) waits a nominal exponential delay, clamped by the cap:
d_i = min(cap, b × m^i) for i = 0 .. n-1
Jitter then transforms each d_i into a random variable. Because the distributions are
uniform, the expectations are closed-form — no simulation needed:
d_i, worst case d_i.U(0, d_i): E[delay] = d_i / 2, worst case d_i.d_i/2 + U(0, d_i/2): E[delay] = 3·d_i / 4, worst case d_i.min(cap, U(b, 3·prev)) seeded with prev = b:
the worst case follows min(cap, 3·prev_max), and the expectation is the mean of the
cap-truncated uniform, [(cap² − b²)/2 + cap(3prev − cap)] / (3prev − b) when the cap
bites, else (b + 3prev)/2. The recursion is propagated through the mean, which is exact while
the cap is inactive and a close approximation once it binds.
When retry-after is honored and the server supplies it, the header value replaces
d_i entirely for every retry — jitter and multiplier stop mattering, and the whole schedule
becomes n × retry-after of pure waiting.
Treat attempts as independent Bernoulli trials with failure probability p. You make one
initial attempt plus n retries, so n+1 attempts total. The call fails only if every
attempt fails:
P(success within n retries) = 1 − p^(n+1)
E[attempts made] = (1 − p^(n+1)) / (1 − p) (= n+1 when p = 1)
The expected-attempts formula is the truncated geometric sum Σ p^k for
k = 0 .. n: attempt k happens exactly when the previous k attempts all failed,
which has probability p^k.
The same p^k weights price the schedule. Attempt k costs latency L and
happens with probability p^k; delay d_i is only paid if attempt i failed,
which has probability p^(i+1):
E[T] = Σ(k=0..n) p^k · L
+ Σ(i=0..n-1) p^(i+1) · E[d_i]
worst T = (n+1) · L + Σ(i=0..n-1) d_i^max
Worst case assumes every attempt fails and every jittered delay lands at its upper bound — the number your timeout configuration actually has to survive.
Walk the cumulative timeline and find k*, the last attempt that finishes at or before the
deadline D. If all n+1 attempts fit, the deadline is never the binding constraint and
the breach probability is zero (you simply exhaust retries first). Otherwise you breach exactly when the
first k*+1 attempts all fail:
P(deadline exceeded) = p^(k*+1) if k* < n
= 0 if the whole schedule fits inside D
If even the first attempt cannot finish inside D (L > D), the breach probability is 1.
To hit a target success probability S, solve 1 − p^(n+1) ≥ S. Taking logs and
flipping the inequality (because ln p < 0):
n ≥ ceil( ln(1 − S) / ln(p) ) − 1
The largest safe cap comes from the worst-case equation. The delay budget is
Sdelay = D − (n+1)·L; if that is negative, no cap value helps — you must cut
retries or latency. Otherwise worst-case total delay is monotonically non-decreasing in the cap, so a
bisection over cap finds the largest value with
Σ min(cap, b·m^i) ≤ Sdelay. If the uncapped schedule already fits, no cap is
required at all.
L is a single figure for every attempt. A request that fails at the connect stage is much
cheaper than one that times out after ten minutes; model those separately.408, 409, 429, and all 5xx responses, plus connection
errors, using exponential backoff. Non-retryable client errors — 400,
401, 403, 404, 413 — should never enter a retry loop.max_retries is 2 and the request
timeout is 10 minutes. Units differ by SDK: Python and Ruby take seconds,
TypeScript takes milliseconds, Go takes a time.Duration, Java a Duration,
C# a TimeSpan.timeout × (max_retries + 1). That product, not the timeout alone, is what your caller
experiences — size it against the worst case above.retry-after. Rate-limited responses tell you how many seconds
to wait; honoring the header beats any locally computed backoff. Rate-limit responses also expose
x-ratelimit-remaining-* headers you can use to throttle before you get rejected.overloaded_error means the API is temporarily at capacity. Back off,
spread load over time, or shift some traffic to a less contended model.max_tokens requests
precisely because it avoids hitting the request timeout in the first place.