System design · interactive

Bulkheads

A service has 20 worker threads. The payments provider starts hanging for 10 seconds per call. Catalog calls are fast and have nothing to do with payments. Watch what happens to the catalog.

0scatalog 20/20
1scatalog 20/20
2scatalog 20/20
3scatalog 20/20
4scatalog 0/20
5scatalog 0/20
6scatalog 0/20
7scatalog 0/20
8scatalog 0/20
9scatalog 0/20
10scatalog 0/20
11scatalog 0/20

Red: threads stuck waiting on payments. All 20 threads are shared.

160 catalog requests failed

Each hanging payment call holds a thread for 10 s. After 5 s every thread is stuck on payments and the catalog, which is perfectly healthy, goes down too.

Thread pools

Step 1 of 4 · read left to right

1
Slow is worse than down
A dead dependency fails fast. A hanging one holds threads, connections and memory until they run out.
Resource exhaustion spreads failures.
2
Separate pools per dependency
Like a ship's watertight compartments: a flood in one doesn't sink the rest.
Thread pools, connection pools, semaphores.
3
Also by tenant or traffic class
Separate clusters or queues for big customers, batch jobs and interactive traffic.
Cell-based architecture is bulkheads for whole stacks.
4
With timeouts and circuit breakers
Timeouts bound each call, breakers stop calling a broken dependency, bulkheads cap the damage.
Resilience4j has all three.

In production

Tools and technologies

Resilience libraries implement bulkheads as separate pools.

Not tied to one language — use these names when you search docs or design a system.