Skip to content
Optimal Lab

AI Engineering · interactive

The context window, turn by turn

A model has no memory between calls. Every turn, your app re-sends the system prompt, the tools, the whole chat and any retrieved notes, and it all has to fit. Play a long chat and watch the window fill.

Turn 1 of 10

3,670 / 8,000 tokens

  • System prompt 400
  • Tool definitions 900
  • Turn 1 520
  • 3 retrieved notes 1,050
  • Room for the answer 800

What the model can see of the chat

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10

this call $0.013

Turn 1: 3,670 of 8,000 tokens. Every turn re-sends the whole chat, so the prompt keeps growing.

When it doesn't fit

  1. 1

    The system prompt and the tool definitions go first, every single call.

    400 + 900 = 1,300 tokens

  2. 2

  3. 3

  4. 4

  5. 5

  6. 6

In production

Tools and technologies

Every hosted model bills per token and caps the window; prompt caching and trimming are the levers in any stack.

Not tied to one language — use these names when you search docs or design a system.