AI Engineering · interactive

Reflection and self-correction

Ask a model to check its work and try again, and it will. Whether the second try is better depends on what it checks against. Revise a function against eight tests with only self-critique, with the test results, and with the error output.

slugify: 8 tests · Self-critique only

input → expected	draft	r1	r2	r3
“Hello World” → “hello-world”	✓	✓	✓	✓
“ trim me ” → “trim-me”	✗	✗	✗	✗
“Café au lait” → “cafe-au-lait”	✗	✗	✗	✗
“a--b” → “a-b”	✗	✗	✗	✗
“C++ & Rust!” → “c-rust”	✗	✗	✗	✗
“” → “”	✓	✓	✓	✓
“Already-slug” → “already-slug”	✓	✓	✓	✓
“Many spaces” → “many-spaces”	✓	✓	✓	✓
passing	4	4	4	4

A ringed cell changed in that round.

Self-critique only 4.5Run the tests 7.4Tests + error output 7.7

Self-critique: 4 → 4 passing. Without a signal from outside, revising is mostly reshuffling.

Feedback each round

Write a first draft

The model writes a function that turns a title into a URL slug. It looks right and passes the easy cases.

“Hello World” gave “hello-world”
“ trim me ” gave “-trim-me-”
“Café au lait” gave “caf-au-lait”
“a--b” gave “a--b”
“C++ & Rust!” gave “c----rust-”
“” gave “”
“Already-slug” gave “already-slug”
“Many spaces” gave “many-spaces”

Get feedback

The feedback decides everything. Re-reading your own work finds little; running it against tests finds exactly what's wrong.

Self-critique: “check your answer and improve it”.
Run the tests: which of the 8 fail.
Tests + error output: each failing input, with expected and actual output.

Critique the draft

The model writes down what it thinks is wrong. This is the reflection: only as good as the evidence in front of it.

Looking at my function again, it lowercases, replaces spaces with hyphens and should handle punctuation. I'll tidy the regex to be safe.

Revise

It rewrites the function. Each failing case may get fixed; each passing case may get broken by the change.

Chance a failing test gets fixed: 12%
Chance a passing test gets broken: 8%
This run, round 1: +0 fixed, −0 broken

Repeat, then stop

Loop until the tests pass or a round limit is hit. With no outside check there is no clear point to stop, and more rounds can make it worse.

Rounds

4 → 4 of 8 passing

Huang et al. (2023) found models can’t reliably correct their own reasoning without outside feedback. Reflexion (Shinn et al. 2023) and coding agents work because the tests, the compiler or a tool supply it.

In production

Tools and technologies

Reflection works when the loop has a real check in it: tests, a compiler, a validator or a tool result.

Not tied to one language — use these names when you search docs or design a system.