Skip to content
Optimal Lab

AI Engineering · interactive

Zero-shot and few-shot prompting

Ask a model to sort support tickets and it answers, but in its own words and by its own idea of each label. Add a few worked examples and the format locks in and your team's conventions carry over. Pick the wrong examples and they bias every answer.

The prompt · about 49 words

Classify the support ticket as billing, bug or how-to.

Ticket: I was charged twice this month.
Label: billing

Ticket: The app crashes when I open settings.
Label: bug

Ticket: I can't find where to change my password.
Label: how-to

Ticket: I can't find the export to CSV option.
Label: how-to

All eight tickets

8 of 8 filed correctly. A billing, a bug and a how-to example. The how-to example teaches the team's convention: “can't find” is a question, not a bug.

Examples in the prompt

8/8 right · 8/8 readable

1

Write the instruction

Say what to do and list the allowed answers. Zero-shot prompting stops here: no examples, just the instruction.

Classify the support ticket as billing, bug or how-to.

2

Add worked examples

Each example is an input with the answer you want. They can sit in the prompt like this, or as earlier user and assistant turns. Click to add or remove; the order counts.

3

The model continues the pattern

Each label gets a score. The model's own guess comes first. A similar example pulls toward its label; so does every example, a little, and the last one most of all.

I can't find the export to CSV option.

  • billing0.25
  • bug1.15
  • how-to1.55
  • own guess
  • a similar example +0.6
  • label count +0.15 each
  • last example +0.2

Click a ticket in the picture to follow it here.

4

Parse the reply

Your code files the ticket only if the reply is exactly one of the labels. With no examples, the model picks its own wording.

reply: “how-to” → filed as how-to

The team files it as how-to.

5

Score every ticket

Run the same prompt over a test set and count. Examples fix the format first, then the edge cases. One-sided examples make it worse.

  • Zero-shot2/8 right
  • One example5/8 right
  • One of each8/8 right
  • All six8/8 right
  • Only bug examples3/8 right

Each example costs prompt tokens on every call. Today’s instruction-tuned models need fewer examples than GPT-3 did, so use a few varied ones that cover the hard cases, balanced across labels.

3 labels · scores are a teaching model of the label biases measured by Zhao et al. (2021)

In production

Tools and technologies

Few-shot examples are plain prompt text; tools help you pick them and measure whether they help.

Not tied to one language — use these names when you search docs or design a system.