Chaos engineering is experimenting on a system by introducing failures such as stopped servers, added latency or blocked network links, to see whether it behaves as expected. The approach was popularized by Netflix's Chaos Monkey tool, which terminated production instances at random.
Why it matters
Redundancy and failover only count if they work. Many teams find through real outages that a backup node was misconfigured or an alert never fired. Controlled experiments surface these gaps during working hours, with people ready, instead of at 3 a.m. They also test monitoring, runbooks and on-call response. A rehearsal event of this kind is often called a game day.
Example
An experiment written as a hypothesis:
Hypothesis: if one of three web nodes is stopped, error rate stays below 1%
and an alert reaches on-call within 3 minutes.
Action: stop web-2 for 10 minutes (staging first, small blast radius)
Abort if: error rate above 5%
Result: errors 0.4%, alert arrived in 2 min; health check removed node as intendedCommon mistakes
- Starting in production with no safeguards. Begin small and in staging.
- Running experiments with no steady-state metric to compare.
- Skipping the fix. Findings must lead to changes.
- Surprising the team without agreed abort conditions.
Related
Get notified when this term updates
Plus occasional practical tips on uptime and monitoring, nothing else. Unsubscribe anytime.
यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.