Skip to content
Optimal Lab

System design · interactive

Rate limiting: three algorithms, one burst

Every API caps how often one client can call it: here 10 requests per 10 seconds. Three common ways to count give different answers on the same traffic. Watch a burst that straddles a window boundary.

Second 0: quiet.

t = 0s

Fixed window

count 0 / 10 · peak 20 in 10s

Count per 10-second window. Reset to 0 on the boundary.

0s■ allowed ■ rejected · dashed = window boundary30s

Sliding window

estimate 0 / 10 · peak 10 in 10s

This window's count plus the overlapping part of the last one.

0s■ allowed ■ rejected · dashed = window boundary30s

Token bucket

10 tokens left · peak 13 in 10s

10 tokens, +1 each second. A request spends a token.

0s■ allowed ■ rejected · dashed = window boundary30s
  1. 1

    The limiter sits in front of the API (NGINX, Envoy, an API gateway) and identifies the client by API key, user id or IP.

    t = 0s · 0 arriving

  2. 2

  3. 3

  4. 4

  5. 5

In production

Tools and technologies

Rate limits live in the gateway in front of your API, with counters in a shared store.

Not tied to one language — use these names when you search docs or design a system.