Skip to content
Optimal Lab

AI Engineering · interactive

RAG, step by step

The model does not search your docs. You split them, retrieve a few chunks, and paste those chunks into the prompt. The model only sees that prompt.

User

asks

Your app

builds the prompt

Doc index

3 chunks

LLM

sees only the prompt

Run retrieval to watch the question travel.

1

Ask a question

Retrieval starts from the question, not from the whole corpus.

What is the capital of Pakistan?

2

Split the docs into chunks

A sentence keeps one fact. A whole section glues facts together, so the match gets blurrier.

  • Handbook

    The capital of Pakistan is Islamabad.

  • Handbook

    Karachi is the largest city by population.

  • Cookbook

    Soak the rice, parboil it, then layer it for biryani.

3

Score each chunk against the question

Higher means this chunk is more like the question — the same cosine idea as the embeddings session.

Embeddings and similarity search shows this step on its own.

  • The capital of Pakistan is Islamabad.0.94
  • Karachi is the largest city by population.0.38
  • Soak the rice, parboil it, then layer it for biryani.0.06
4

Keep the top matches

k is how many chunks go into the prompt. Too few and you miss the fact. Too many and you add noise.

  • The capital of Pakistan is Islamabad.
  • Karachi is the largest city by population.
  • Soak the rice, parboil it, then layer it for biryani.
5

Paste them into the prompt

This is the whole input the model gets. If a fact is not in here, the model was not given it.

Answer using only the notes below. If they do not say, say you do not know.

- (Handbook) The capital of Pakistan is Islamabad.

Question: What is the capital of Pakistan?
6

Answer only from that prompt

The model is not searching again. It writes from the notes you just pasted.

The capital of Pakistan is Islamabad.

In production

Tools and technologies

RAG pipelines chunk documents, retrieve vectors, and inject context — usually with off-the-shelf pieces.

Not tied to one language — use these names when you search docs or design a system.