Tools · interactive

Model routers

A router sits between your app and many model providers behind one API. It can fall back when a provider is down and pick a cheaper model when the task is easy. Twelve requests; provider A has an outage in the middle.

#1 simplebig
#2 complexbig
#3 simplebig
#4 simplealtafter 1 fail
#5 complexaltafter 1 fail
#6 simplealtafter 1 fail
#7 simplealtafter 1 fail
#8 complexbig
#9 simplebig
#10 simplebig
#11 simplebig
#12 complexbig

provider A down for #4–#7

big: Big model, provider A10 credits
alt: Big model, provider B12 credits
small: Small model1 credits

0 errors · 128 credits

When A fails, the router retries on provider B. No errors, but B costs more: 128 credits.

Routing

Step 2 of 4 · read left to right

1
One API, many models
Your app calls one endpoint with a model name. The router handles each provider's keys and formats.
model: "provider-a/big"
2
Fall back automatically
On a 5xx, timeout or rate limit, retry the same request on the next model in the list.
models: ["a/big", "b/big"]
3
Route by difficulty or cost
Send classification and short answers to a small model, reasoning to a big one. Test that the small model is good enough first.
Often most of the savings.
4
Log which model answered
Fallbacks change output quality and cost. Record the model used per request so evals and bills make sense.
Pin versions where behaviour matters.

In production

Tools and technologies

Routers and gateways put many providers behind one API.

Not tied to one language — use these names when you search docs or design a system.