System design · interactive

Scaling up vs scaling out

Traffic grows. You can buy a bigger machine (vertical) or add more machines behind a load balancer (horizontal). Each vCPU handles about 250 requests a second. Prices are illustrative.

1,000 requests / second

Vertical

4 vCPU

$100/mo · capacity 1,000

One machine fails: 100% down

Horizontal

2 × 4 vCPU + LB · $220/mo

One machine fails: 50% of capacity lost

At low traffic, one bigger box is cheaper and far simpler. Just keep a standby for failures.

Traffic1,000 rps

Step 1 of 4 · read left to right

1
Scale up first
A bigger instance needs no code changes. Databases in particular go a long way vertically.
Simple beats clever until it doesn't.
2
The ceiling and the single point of failure
There's a largest machine, the top sizes cost extra, and one box means one failure takes everything down.
Upgrades usually mean downtime too.
3
Many small, identical servers
A load balancer spreads requests. Losing one costs a slice of capacity, and you can add servers without downtime.
Keep headroom for a node failure.
4
Servers must be stateless
Sessions, uploads and caches move out to shared stores so any server can handle any request. State is what's hard to scale out.
Then autoscaling becomes possible.

In production

Tools and technologies

Clouds make both kinds of scaling a configuration change.

Not tied to one language — use these names when you search docs or design a system.