Skip to content
Optimal Lab

System design · interactive

Load balancing strategies

Pick a backend, skip unhealthy hosts, respect queue caps, and shed load when the waiting room at the load balancer is full — then compare round robin, least connections, and weighted routing.

Clients

4 new / tick

Load balancer

round robin · health-aware

Server A

2/4 in flight · 1/tick

Server B

1/4 in flight · 1/tick

Server C

1/4 in flight · 3/tick

Tick 1 · routing: placed 4. 0 waiting total.

Scenario

All backends up, generous queues — compare how each strategy uses fast Server C.

Strategy

Tick 1 · route (1/12)

Same scenario, all strategies

  • Round robin (playing): failed 0, peak LB queue 0, placed 24
  • Least connections: failed 0, peak LB queue 0, placed 24
  • Weighted (by speed): failed 0, peak LB queue 0, placed 24

This run — placed 24, peak slow-server backlog 6.

  1. 1

  2. 2

    Each arrival goes to a healthy backend with space. Overflow waits at the load balancer. The queue drains first on the next routing step before new bursts land.

    LB waiting room0

    This step: 4 placed

    • → A
    • → B
    • → C
    • → A
  3. 3

  4. 4

In production

Tools and technologies

Load balancers sit between clients and your app tier regardless of how the backends are written.

Not tied to one language — use these names when you search docs or design a system.