Metastable failure
A failure that keeps itself going after its trigger is gone. A short slowdown pushes the system into a bad state, and a feedback loop, most often retries, keeps the load above capacity, so the system stays down until someone cuts the load. Bronson and others named this class of failure in 2021.
Reliability
In Quorum#
The Retry storm Incident slows the Database to 30% of its capacity for 20 s. Links that retry at once send up to 1 + f + f² + f³ calls per call when a share f fails, they carry that failure share into the next Step, and the Database queues what it cannot take; the Incident ends only when that backlog drains. Backoff with jitter, Circuit breakers, and Priority load shedding each break the loop, and naive retries into a Database make this Incident 6 times as likely as backoff does.
Read next#
Retry with backoff and jitter, Circuit breaker, Load shedding, Thundering herd