AI Engineering · interactive

Open vs closed models

A closed model is an API you pay per token. An open model is a file of weights you run on GPUs you rent by the hour, busy or not. Slide your traffic and its peaks, watch the two bills cross, then weigh the things that aren't on the bill: privacy, control, quality and who's on call.

10,000 requests a day · peak 5×

$1,980 vs $21,520

hosted APIopen weights on rented GPUs

Frontier API, per month: $1,980
Open 70B on 4 GPUs: 2 replicas, 3% busy: $21,520

Frontier API is cheaper here: $1,980 against $21,520 a month.

Requests a day

Size your traffic

Requests a day, tokens per request, and how spiky it is. Peaks matter for GPUs: you pay for enough to serve the busiest minute.

Peak traffic, as a multiple of the average

700 in + 300 out per request · 116 tokens/s on average

Price the hosted API

Per-token prices, so the bill is a straight line: double the traffic, double the bill. Idle time costs nothing. You get the provider's best model the day it ships.

$3/M in · $15/M out · quality 90% → $1,980 a month

Price your own GPUs

Enough replicas for the peak, at least 2 so one can fail, each rented by the hour. Most of the day they sit partly idle. Add the time of whoever runs them.

2 replicas × 4 GPUs × $3/h × 730 h = $17,520 · + $4,000 of someone’s time · busy 3% · quality 84%

Find the break-even

With steady, heavy traffic the GPUs are busy and self-hosting wins. With spiky or small traffic, you pay for idle machines and the API wins. A cheap hosted tier moves the crossing far out.

1,000/day$198 · $21,520
10,000/day$1,980 · $21,520
100,000/day$19,800 · $39,040
1,000,000/day$198,000 · $293,080

No break-even below 10 million requests a day: the API stays cheaper.

Weigh what isn't on the bill

Cost rarely decides alone. Tick what applies to you.

Data can't leave our infrastructure→ open
Model must not change under us→ open
We'll fine-tune heavily→ open
We need the best quality available→ hosted
No one to run GPUs→ hosted
Traffic is spiky or tiny→ hosted

0 for open weights · 0 for a hosted API · bill favours hosted

Many teams do both: a hosted frontier model for the hard requests and an open model for the bulk. Prices are typical published rates.

In production

Tools and technologies

Open weights come from a few labs and run on a few serving stacks; comparison sites track price and quality across both kinds.

Not tied to one language — use these names when you search docs or design a system.