Tail Latency

Tail latency refers to the slow end of the response-time distribution, typically measured as the 95th, 99th, or 99.9th percentile. Most requests may finish quickly while a small fraction takes many times longer.

Why it matters

Averages hide the tail. If the median is 80 ms but p99 is 3 seconds, one request in a hundred is painfully slow. That matters more than it sounds: a single page load often makes dozens of requests, so the chance that a user hits at least one slow call is much higher than 1 percent. In systems that fan out to many backends, the slowest backend sets the pace for the whole response.

Example

p50:  82 ms
p90:  140 ms
p99:  3100 ms
max:  9800 ms

Common causes of a long tail are garbage collection pauses, lock contention, cold caches, noisy neighbors on shared hosts, retries after a timeout, and a slow database replica. Useful mitigations include hedged requests (send a duplicate after a delay and use the first reply), tighter timeouts, and load shedding.

Common mistakes

• Alerting on average response time only.
• Averaging percentiles across servers, which is mathematically meaningless. Merge histograms instead.
• Setting timeouts so long the tail has no ceiling.

Related

• Percentile
• Latency
• Apdex
• Golden signals

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.