Thundering Herd

A thundering herd happens when a large number of clients or processes are all waiting on the same event and then act at the same moment. The sudden spike can overwhelm a backend that would have handled the same total work if it had arrived gradually.

Why it matters

The herd often arrives exactly when the system is weakest. A server restarts, every client reconnects within a second, the cold server collapses under the load, and the cycle repeats. Monitoring shows a sawtooth pattern of recovery followed by failure.

Example

A mobile app polls an API every 60 seconds. After a 10-minute outage, thousands of devices that were failing now succeed in the same minute because their timers are aligned to the moment of failure. The backend receives roughly ten minutes of deferred work in a few seconds.

Common fixes:

• Add random jitter to timers and reconnects, for example 60 seconds plus a random 0 to 15 seconds.
• Use exponential retry with backoff.
• Ramp traffic back gradually after maintenance instead of switching it all on.
• Coalesce identical requests so only one fetches the data (see cache stampede).

Common mistakes

• Setting cron jobs, token refreshes, or cache expiries to the exact same second across a whole fleet.
• Blaming capacity when the real problem is synchronization.
• Testing recovery only with a warm cache and idle server.

Related

• Cache stampede
• Jitter
• Load shedding
• Rate limiting

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.