Reflection in AI agents: critique and revise

Overview

Reflection is an agent pattern in which a model looks back at its own work, judges what is wrong with it, and uses that judgement to produce a better version. The simplest form is a loop: draft, critique, revise, and repeat until the result is good enough or a limit is reached. More advanced forms let an agent learn from failed attempts across tries, by writing short lessons and reading them before the next attempt.

Reflection works best when the critique is grounded in something real. A model that only re-reads its own answer and asks "is this right?" often agrees with itself, or changes a correct answer into a wrong one. A model that sees a failing test, a validation error, a compiler message or a separate judge's score has concrete evidence to act on. Most of the value of reflection comes from that feedback signal.

An everyday comparison is writing an important email. You write a draft, read it back, notice it sounds too sharp, soften two sentences and fix a missing date. Better still, you ask a colleague to read it, or you check the attachment actually opens. Reading your own draft helps a little. A second pair of eyes or a concrete check helps much more. Reflection in agents follows the same rule.

What it is

Reflection is a family of designs. They differ in who critiques, what evidence the critique uses, and whether lessons are kept.

Variant	How it works	Feedback source
Self-refine	The same model drafts, critiques and revises in a loop	The model's own judgement
Evaluator-optimizer	One model generates, another evaluates and gives feedback	A separate judge prompt or model
Tool-grounded critique	The output is checked by tools such as tests, search or validators	External, verifiable signals
Reflexion-style memory	After a failed attempt, the agent writes a lesson and uses it next time	Task outcome plus self-written notes
Rubric-based review	A critic scores the output against explicit criteria	A checklist or rubric
Human feedback	A person reviews and comments; the model revises	A human reviewer

All of them share three parts: a generator that produces work, a critic that evaluates it, and a stopping rule that decides when to finish.

Where it came from

Several 2023 papers shaped the pattern.

Self-Refine (Madaan and colleagues, 2023) showed that one model could generate an output, give itself feedback and refine the output over several rounds, improving results on tasks such as code optimisation, dialogue and text rewriting, without extra training.
Reflexion (Shinn and colleagues, 2023, presented at NeurIPS 2023) had agents turn task feedback into written reflections stored in memory, then use them in later attempts. On the HumanEval coding benchmark it reported 91 per cent pass@1, compared with 80 per cent for GPT-4 at the time.
CRITIC (Gou and colleagues, 2023) let models check their outputs with external tools such as search engines and code interpreters before revising.
"Large Language Models Cannot Self-Correct Reasoning Yet" (Huang and colleagues, 2023) found that, without external feedback, models asked to review their own reasoning often failed to improve and sometimes got worse.

That last result is the key caution. In December 2024 Anthropic's guide "Building effective agents" described the evaluator-optimizer workflow, in which one model generates and another evaluates in a loop, as useful when there are clear evaluation criteria and iteration measurably helps. Reflection is now a standard building block in coding agents, which run tests and fix failures, and in writing and extraction pipelines with judges.

How it works

Step 1: Generate a first attempt

The generator produces a draft: code, an answer, a plan, a summary or structured data.

Step 2: Gather evidence

Before critiquing, collect whatever objective signals exist. Run the unit tests. Validate the JSON against a schema. Check that every cited source is in the retrieved documents. Compile the code. Count words. These signals are far more reliable than the model's own impression.

Step 3: Critique

A critic, either the same model with a different prompt or a separate model, reviews the draft with the evidence and a rubric. A good critique is specific: "Test test_refund_rounding fails because amounts are rounded before tax; the instruction asked for after." A vague critique, such as "could be improved", gives the generator nothing to act on. See LLM as judge for how to design judges.

Step 4: Revise

The generator receives the draft, the evidence and the critique, and produces a new version. It is usually best to ask for targeted changes rather than a full rewrite, so that correct parts are not broken.

Step 5: Decide whether to stop

The loop ends when all checks pass, when the critic's score reaches a threshold, when a maximum number of rounds is reached, or when a round makes no improvement. Without a stopping rule, reflection can loop, cost a lot and drift.

Step 6: Keep lessons, if useful

In Reflexion-style designs, after a failed attempt the agent writes a short lesson, such as "check for empty input lists before indexing". The lesson is stored and included in the prompt for the next attempt on similar tasks. This is a simple form of agent memory.

Key concepts and terms

Term	Meaning
Reflection	An agent reviewing and improving its own work
Generator	The component that produces drafts
Critic	The component that evaluates drafts and gives feedback
Self-refine	Draft, self-critique and revise with one model
Evaluator-optimizer	A loop where one model generates and another evaluates
Reflexion	A method that stores verbal lessons from failures for later attempts
Grounded feedback	Critique based on external checks such as tests or validators
Rubric	Explicit criteria used for scoring
Stopping rule	The condition that ends the loop
Self-correction	A model fixing its own errors, reliable mainly with external feedback
Regression	A revision that breaks something that was correct

A worked example

This scenario is illustrative. A team uses a coding agent to fix bugs reported in a payments library with 340 unit tests.

The agent receives a bug report: refunds of 0.01 euros are rounded to zero.
It writes a patch. The harness runs the tests: 338 pass, 2 fail.
The critic step receives the patch and the failure output. It notes that one failing test expects rounding after tax, and the patch rounds before.
The agent revises only the rounding order. Tests run again: 339 pass, 1 fails, a test for negative amounts.
The critic points out that the new code does not handle negative refunds. The agent adds a guard. All 340 tests pass.
The loop stops because all checks pass. It would also have stopped after five rounds.
The agent writes a lesson for its memory: "In this library, rounding always happens after tax." On a later, similar bug it applies this from the first attempt.

A human reviewer still reads the final patch before merging. Reflection made the patch better; it did not remove the need for review.

Strengths and limitations

Strengths:

Improves quality with no extra training.
Very effective when objective checks exist, such as tests, validators or compilers.
Separate critics can catch issues the generator misses.
Stored lessons can improve later attempts on similar tasks.
Easy to add to existing pipelines.

Limitations:

Without external feedback, self-critique is unreliable and can make answers worse.
Each round adds cost and latency.
Critics can be biased towards their own style or towards longer answers.
Revisions can break parts that were already correct.
Loops without clear stopping rules can run long and drift.
Stored lessons can be wrong and spread errors if not reviewed.

How it compares with alternatives

Approach	How it improves quality	Best for
Reflection	Critique and revise after a draft	Tasks with checkable criteria
Tree-of-thoughts	Explores and scores several branches	Problems with many possible paths
Best-of-n sampling	Generates several answers and picks the best by a scorer	When a good scorer exists and drafts are cheap
Plan-and-execute	Plans the whole task up front	Multi-step tasks with tools
Human-in-the-loop review	A person corrects the output	High-stakes outputs

Practical guidance and best practices

Ground critiques in external checks wherever possible: tests, schemas, retrieval, calculators.
Use a separate critic prompt, and ideally a different model, from the generator.
Give the critic an explicit rubric and ask for specific, actionable feedback.
Ask for targeted edits rather than full rewrites.
Set a maximum number of rounds and stop when a round brings no improvement.
Re-run all checks after every revision to catch regressions.
Measure whether reflection actually improves your metric; remove it if it does not; see evals.
Review stored lessons before they become permanent memory.

Common mistakes

Asking the model "are you sure?" with no evidence and treating the answer as verification.
Letting the critic and generator share the same prompt and blind spots.
Running many rounds without checking whether quality is improving.
Accepting a revision without re-running the checks that the previous version passed.
Using reflection on simple tasks where the first answer is already good, which only adds cost.
Storing every lesson forever, so memory fills with noise.

Frequently asked questions

What is reflection in AI agents?

It is a pattern where an agent critiques its own output or past attempts and uses that critique to produce a better version, usually in a loop with a stopping rule.

What is the difference between Self-Refine and Reflexion?

Self-Refine improves one output through rounds of self-feedback and revision. Reflexion turns the outcome of whole attempts into written lessons stored in memory and uses them in later attempts.

Can models reliably correct their own mistakes?

Not on their own. Research has shown that without external feedback, self-correction of reasoning often fails. Reflection works much better when critiques use tests, validators, tools or a separate judge.

What is an evaluator-optimizer loop?

It is a workflow where one model generates output and another evaluates it against criteria and gives feedback, repeating until the evaluator is satisfied or a limit is reached.

How many reflection rounds should I use?

Usually few: two to three often capture most of the gain. Measure on your own tasks and stop when extra rounds no longer improve the metric.

Does reflection replace human review?

No. It raises quality before review, but high-stakes outputs still need a person to check them.

Glossary

Reflection: reviewing and improving one's own work.
Generator: the model that drafts.
Critic: the model or check that evaluates.
Rubric: a list of criteria for evaluation.
Grounded feedback: critique based on objective checks.
Stopping rule: the condition that ends a loop.
Regression: a change that breaks working parts.
Lesson: a short note stored after a failure.
Self-correction: fixing one's own errors.

Summary

Reflection lets an agent draft, critique and revise, and in some designs learn lessons across attempts. It grew from 2023 work such as Self-Refine, Reflexion and CRITIC, alongside evidence that self-correction without external feedback is unreliable. Its real power comes from grounded feedback: tests, validators, tools and separate judges. Use clear rubrics, targeted edits, regression checks and strict stopping rules, and measure whether each extra round actually helps.

It belongs to Patterns in my stack, under Agentic AI. Others in the same group:

ReAct loop Plan-and-execute Tree-of-thoughts Orchestrator-workers Multi-agent systems Human-in-the-loop LLM-as-judge Guardrails