Skip to content
Optimal Lab

AI Engineering · interactive

Training: loss and gradient descent

Every model, from this line to a large language model, learns the same way: guess, measure how wrong the guess is, work out which way is downhill, and take a small step. Watch it fit a line, then change the step size.

Step 0 of 40

y = 0.00·x + 0.00

0510

Loss over steps (log scale)

model error per point best possible line

Start with w = 0 and b = 0: a flat line. The loss is 35.93. Press Play to train.

Learning rate (step size)

About right: the loss drops fast and settles.

loss 35.93
  1. 1

    The model is y = w·x + b. Two numbers, w and b, are all it can learn. A large language model has billions of them.

    w = 0.00, b = 0.00

  2. 2

  3. 3

  4. 4

In production

Tools and technologies

Every framework trains models with this same loop; they compute the gradients for you.

Not tied to one language — use these names when you search docs or design a system.