Operating systems · interactive

The memory hierarchy

Fast memory is small and expensive; big memory is slow. Real numbers like 1 ns and 100 µs are hard to feel, so switch to human scale: pretend one CPU cycle takes one second.

CPU register ~1 KB1.0 seconds
L1 cache 64 KB3.3 seconds
L2 cache 1 MB13 seconds
L3 cache 32 MB50 seconds
RAM 16–512 GB6 minutes
NVMe SSD read TBs4 days
Datacenter round trip —19 days
Hard disk seek TBs1.1 years
Internet round trip (EU↔US) —8.5 years

Bars use a log scale: each step right is 10× slower.

If a cycle were a second, fetching from RAM is a 6-minute wait, a disk seek over a year, and a round trip to another continent 8 years.

Show times as

Step 1 of 4 · read left to right

1
Caches hide RAM's slowness
The CPU keeps recently used data in small, fast caches. A hit costs a few cycles; a miss costs hundreds.
Locality decides your hit rate.
2
Storage is another world
An SSD read is a thousand RAM accesses; a spinning disk seek, a hundred thousand.
Why databases cache pages in memory.
3
The network is slowest of all
Even a fast datacenter round trip costs more than an SSD read. Cross-continent trips are limited by the speed of light.
Batch calls, keep data close.
4
Design around the gaps
Cache hot data a level up, read sequentially, avoid round trips in loops. Knowing these numbers lets you estimate before you build.
“Latency numbers every programmer should know.”

In production

Tools and technologies

Every performance decision sits on these numbers.

Not tied to one language — use these names when you search docs or design a system.