System design · interactive
Hedged requests and tail latency
Most requests to this backend take 15–40 ms, but about 1 in 10 hits a slow server and takes half a second. If you wait a little and then send a backup copy to another server, you take whichever answers first.
0400 ms800 ms
| p50 | p95 | p99 | |
|---|---|---|---|
| no hedging | 30 | 508 | 631 |
| hedge at 60 ms | 30 | 91 | 98 |
Extra requests sent: 6 of 40 (15%). Grey: the slow original, abandoned. Green: the backup that won.
Hedging at 60 ms cuts p99 from 631 to 98 ms for 15% extra load.
Strategy
1 2 Send a second copy only when the first is slower than usual. Most requests never need one.
Too early doubles load; too late saves nothing.
3 4
In production
Tools and technologies
Hedging is built into some RPC frameworks.
Not tied to one language — use these names when you search docs or design a system.