Skip to content
Optimal Lab

AI Engineering · interactive

Fine-tuning

Fine-tuning keeps training a model on your own examples. It is the best way to lock in a format or a voice, a slow way to teach a new skill, and the wrong way to add facts. Pick a goal, choose how many examples and how many passes, and compare the result with a good prompt and with retrieval.

Always answer in our JSON schema · 200 examples × 3 epochs

  • Instructions82%
  • Few-shot93%
  • RAG82%
  • Fine-tuned96%
on your task
96%
general ability (was 90%)
86%
01213510epochs
training loss 0.25held-out loss 0.49

Fine-tuning wins for this goal: 96% against 93% for the best prompt or RAG. Fewer epochs would do as well with less forgetting.

Goal

1

Decide what should change

Fine-tuning changes how the model answers, not what it knows. Format, tone and a narrow skill are good goals; a changing body of facts is not.

A shape the model has to reproduce every time.

2

Collect examples

Each example is an input and the exact answer you want. A format needs tens; a style, hundreds; a skill, thousands. Quality matters more than count: the model copies mistakes too.

Examples

most of the way there by about 40 examples for this goal

3

Train for a few epochs

One epoch is one pass over the examples. Watch the held-out loss: when it starts rising while training loss keeps falling, the model is memorizing your examples instead of learning from them.

Epochs

200 × 3 × 300 tokens at $25/M = $4.50 to train

4

Check the task, and everything else

Score it on held-out examples of your task. Then score it on a general benchmark: narrow training pushes other abilities down a little with every epoch (catastrophic forgetting).

task 96% · general 90% → 86%

5

Compare with prompting and RAG

Always try a good prompt first: it needs no training and changes in a minute. RAG beats fine-tuning on facts. Fine-tuning pays off when it wins on quality, or when it lets you drop a long prompt from every request.

scoreper 1M requestsInstructions only82%$3,450Few-shot prompt93%$5,250RAG82%$7,950Fine-tuned96%$4,313

The few-shot prompt carries 600 extra tokens on every request and RAG 1,500; the fine-tuned model carries none, but costs a little more per token.

Scores are a teaching model of the usual pattern; prices are typical hosted rates.

In production

Tools and technologies

Hosted APIs fine-tune from a file of examples; open models fine-tune with a few libraries.

Not tied to one language — use these names when you search docs or design a system.