Why Did Our Retry Make the Incident Worse?

Retries are additional requests. During an outage, every caller may create them at the same time.

Multiplication across layers

If three layers each try three times, one user action can cause up to 27 downstream attempts.

Retry with a budget

Retry only transient, safe operations. Use exponential backoff, jitter, a total time budget, and a cap on attempts.

Recovery needs room

Circuit breakers and load shedding can stop callers from consuming the small amount of capacity a dependency needs to recover.