Retry with backoff and jitter

A retry heals a short fault only if it waits longer each time and varies the wait, because callers that retry together keep a struggling dependency down.

Use it when#

A call fails for a reason that may pass: a dropped connection, a deploy, a brief overload.
The call is safe to repeat.
Many callers talk to the same dependency, so their retries add up.

How it works#

The caller waits a base delay after the first failure and doubles it after each one, up to a cap: 100 ms, 200 ms, 400 ms. Jitter picks a random point inside that window, so a thousand callers that failed together do not return together. A retry budget caps retries at a share of normal traffic, say 10%, and stops when the budget is spent.

Without backoff the arithmetic is cruel. A database that slows for 20 seconds, behind callers that retry 3 times at once, receives up to 4 calls for every request. The slowdown ends and the load does not.

What it costs#

Every retry is extra load on a dependency that is already struggling. Backoff softens it and a budget bounds it, but neither removes it, and retries at several layers multiply: 3 attempts at each of 3 layers is 27 calls for one request. Retry at one layer, usually the one closest to the user. Pair retries with a circuit breaker for the case where the fault does not pass.

In Quorum#

Both Designs below put the same short timeout on a Link into an undersized Database. Only the retry policy differs.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 3.50k Users the Simulation solves this Design to Apdex 0.13, p99 5.5 s, 58.7% errors and $1.80/s Burn.

Where	Setting	Value	Why
App Server	Instance size	large	A large App Server never runs out first, so the Database link is where the pressure shows.
Database	Instance size	micro	A micro Database is too small for these Users, which stands in for a slow dependency.
Link App Server → Database	Timeout	1 s	A Database call that takes a second is already stuck, so the caller gives up and frees its slot.
Link App Server → Database	Retries	naive	Naive retries send each failed call again at once, up to three more times, which loads a Database that is already behind.

Research this Design needs: Timeouts and retries.

With Naive retries, each failed call is sent again at once. The caption shows the damage as errors and a long tail, because the Database gets more calls than the Users sent.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 3.50k Users the Simulation solves this Design to Apdex 0.27, p99 2.5 s, 53.9% errors and $1.80/s Burn.

Where	Setting	Value	Why
App Server	Instance size	large	A large App Server never runs out first, so the Database link is where the pressure shows.
Database	Instance size	micro	A micro Database is too small for these Users, which stands in for a slow dependency.
Link App Server → Database	Timeout	1 s	A Database call that takes a second is already stuck, so the caller gives up and frees its slot.
Link App Server → Database	Retries	backoff	Backoff waits between attempts and keeps the extra calls inside a small budget, so a struggling Database gets room to recover.

Research this Design needs: Timeouts and retries, Backoff with jitter.

With Backoff, retries wait and spread out, and the extra calls stay inside a small budget. Apdex rises and the p99 falls against the first Design, though the Database is still too small and still drops requests. Backoff limits the harm, and only capacity fixes the cause. Backoff needs Backoff with jitter on top of Timeouts and retries.

The Retry storm incident shows the failure, and Stop a retry storm walks through the fix. The glossary defines retry with backoff and jitter and the thundering herd.