AI · interactive

Long context vs RAG

Context windows are now huge. Why not paste every document into each prompt instead of building retrieval? Grow the document set and compare cost, speed and accuracy per question. Figures are illustrative, at an example price of $3 per million input tokens.

50 pages ≈ 30k tokens · window 200k

	tokens	cost	latency	accuracy
Whole corpus in context	30.0k	$0.090	1.4 s	91%
RAG: top 8 chunks	4.3k	$0.013	1.0 s	86%

At 10,000 questions a day: long context $900/day vs RAG $129/day

Small corpus: put it all in context. It's simple, accurate, and cheap enough. RAG would add machinery for little gain.

Document set50 pages

Step 1 of 4 · read left to right

1
Small and stable: just use the context
A handbook or a few contracts fit easily. No index to build or keep fresh.
Prompt caching makes repeats cheap.
2
Long prompts cost on every call
You pay for every token, every time, and latency grows with length. Models also miss facts buried mid-context.
“Lost in the middle.”
3
Past the window, retrieve
RAG sends only the few chunks that matter. Cost stays flat as the corpus grows; quality depends on retrieval.
Evaluate retrieval recall separately.
4
Often both
Retrieve generously (whole documents, not tiny snippets) into a large context. Permissions and freshness also favour retrieval.
Measure on your own questions.

In production

Tools and technologies

Long context and retrieval are both standard features now.

Not tied to one language — use these names when you search docs or design a system.