Human-in-the-loop
Human-in-the-loop designs keep people in control of AI agents through approvals, edits, reviews and escalation. Learn where to place checkpoints and how to avoid rubber-stamping.
Overview
Human-in-the-loop, often shortened to HITL, is the practice of designing an AI system so that people take part in its decisions at chosen points. For AI agents, this usually means the agent pauses before certain actions and waits for a person to approve, edit or reject them. It also covers people reviewing samples of finished work, answering questions the agent cannot resolve, and taking over when the agent escalates a case.
Agents need this because they act, not just talk. An agent that can send emails, issue refunds, change records, deploy code or move money can do real harm with one wrong step, and models still make mistakes, misread instructions and can be manipulated by malicious content. Placing people at the right checkpoints lets a team use agents for routine work while keeping control over what matters most.
An everyday comparison is a bank's large payment process. A clerk can prepare any payment, but a payment above a set amount needs a second signature from a manager before it leaves the bank. Small payments flow through quickly; large ones get a human check. Good HITL design for agents works the same way: most actions run automatically, and the risky ones wait for a signature.
What it is
Human-in-the-loop is a design approach with several forms. They differ in when the person is involved and how much they control.
| Form | What the person does | Typical use |
|---|---|---|
| Approval gate | Approves or rejects a proposed action before it runs | Payments, deletions, external messages |
| Edit before execution | Changes the agent's proposed action or draft, then approves | Emails, contracts, code changes |
| Clarification | Answers a question the agent asks when information is missing | Ambiguous requests |
| Escalation | Takes over a case the agent cannot or should not handle | Complaints, edge cases, low confidence |
| Review after the fact | Checks samples of completed work | High-volume, low-risk tasks |
| Plan approval | Approves the agent's plan before any steps run | Multi-step tasks with side effects |
| Stop control | Halts the agent at any time | Any autonomous system |
A related idea is "human-on-the-loop", where a person monitors the system and can step in, but does not approve each action. The right mix depends on risk.
Where it came from
The term comes from control engineering and simulation, where a human operator is part of the control loop, as in flight simulators and process control. In machine learning it came to mean people labelling data, correcting predictions and guiding training, as in active learning, and later in reinforcement learning from human feedback.
For language-model agents, human-in-the-loop became central as agents gained tools. Agent frameworks added explicit features for it. LangGraph can interrupt a graph, save its state and resume it with a person's input. The Claude Agent SDK routes undecided tool calls to an approval callback and has a planning mode that never edits without approval. Google ADK 2.0 workflows and Microsoft Agent Framework workflows can pause for human input and resume from checkpoints, and Pydantic AI can mark tools as needing approval.
Regulation also pushes in this direction. Article 14 of the EU AI Act requires high-risk AI systems to be designed for effective human oversight. People overseeing them must be able to understand the system's limits, stay aware of automation bias, interpret outputs, decide not to use or to override them, and stop the system safely. GDPR Article 22 also gives people rights around decisions based solely on automated processing that significantly affect them.
How it works
Step 1: Classify actions by risk
List every tool or action the agent can take and rate it. Reading public data is low risk. Drafting an internal note is low. Sending an email to a customer is medium. Refunding money, deleting data or changing access rights is high. Irreversible and external actions deserve the most care.
Step 2: Choose a checkpoint for each class
Low-risk actions run automatically, with logging. Medium-risk actions may need approval only above a threshold, such as a refund over 100 euros or a message to more than 10 recipients. High-risk actions always need approval, and some may need two people.
Step 3: Pause and persist
When the agent reaches a checkpoint, the system stops, saves the full state (the conversation, the plan, the proposed action and its inputs) and creates a review task. Because a person may answer hours later, the state must live in durable storage, not in memory. See long-running tasks and Temporal.
Step 4: Present a clear decision
The reviewer sees what the agent wants to do, why, with what data, and what will happen if they approve. Good review screens show the exact action, the key evidence, the agent's confidence, and simple choices: approve, edit, reject or escalate.
Step 5: Resume
The system resumes from the saved state with the decision. On approval, the action runs. On edit, the edited version runs. On rejection, the agent receives the reason and can try another path or stop.
Step 6: Learn and adjust
Record every decision. If reviewers approve 99.5 per cent of a certain action unchanged, consider lowering its checkpoint. If they often correct another, improve the agent or tighten its controls.
Key concepts and terms
| Term | Meaning |
|---|---|
| Human-in-the-loop | A person takes part in decisions at defined points |
| Human-on-the-loop | A person monitors and can intervene, without approving each action |
| Approval gate | A pause where a person must approve before an action runs |
| Interrupt and resume | Pausing an agent, saving state and continuing later |
| Escalation | Handing a case from the agent to a person |
| Automation bias | The tendency to over-trust automated suggestions |
| Rubber-stamping | Approving without real review |
| Risk tiering | Grouping actions by potential harm to set controls |
| Four-eyes principle | Requiring two people to approve high-impact actions |
| Confidence threshold | A score below which the agent escalates |
| Audit trail | A record of proposals, decisions and who made them |
A worked example
This scenario is illustrative. An online retailer deploys a customer service agent that can look up orders, issue refunds and send replies.
- The team tiers actions. Looking up orders and drafting replies are automatic. Sending a reply is automatic for order-status questions and needs approval for complaints. Refunds up to 50 euros are automatic; refunds over 50 euros need a team lead.
- In week one the agent handles 4,000 conversations. It proposes 310 refunds, of which 64 are above 50 euros and go to review.
- The review screen shows the order, delivery tracking, the customer's message, the proposed amount and the policy rule the agent applied.
- Team leads approve 51, edit 9 (mostly reducing amounts where only part of an order was damaged) and reject 4 (duplicate claims).
- The team adds a rule for partial damage and a duplicate-claim check. In week four, edits fall to 3 out of 58.
- To fight automation bias, one review in ten hides the agent's proposed amount, so the lead must decide independently. Their decisions are compared with the agent's to measure real agreement.
Strengths and limitations
Strengths:
- Keeps people in control of risky, irreversible or external actions.
- Catches model errors and manipulation before they cause harm.
- Builds trust, so organisations can adopt agents step by step.
- Produces labelled data on where the agent is right or wrong.
- Supports legal and governance requirements for oversight.
Limitations:
- Adds delay; a task can wait hours for a reviewer.
- Costs staff time and does not scale indefinitely.
- Reviewers can rubber-stamp, especially when the agent is usually right.
- Poorly designed review screens hide the information needed to decide.
- Requires durable state and resumable workflows, which add engineering work.
- Too many prompts cause approval fatigue.
How it compares with alternatives
| Control | How it works | Trade-off |
|---|---|---|
| Human-in-the-loop | A person approves or edits at checkpoints | Strong control, slower and costly |
| Full autonomy with guardrails | Automated checks block bad actions | Fast, but misses novel errors |
| LLM as judge review | A second model checks outputs | Scales, but shares model weaknesses |
| Human-on-the-loop monitoring | A person watches dashboards and can stop the agent | Low friction, slower to catch single errors |
| Narrow permissions with OAuth for tool access | The agent simply cannot do risky things | Safe, but limits usefulness |
Practical guidance and best practices
- Tier every action by reversibility, cost and reach before deciding where people approve.
- Put approval gates on irreversible, external and financial actions first.
- Use thresholds so routine cases flow and only significant ones wait.
- Persist full state at every pause so work can resume after hours or restarts.
- Design review screens that show the exact action, evidence and consequences in one view.
- Counter automation bias with sampling, blind reviews and reviewer training.
- Record every decision and use the data to adjust thresholds and improve the agent.
- Always provide a way to stop the agent immediately, and test it.
Common mistakes
- Asking for approval on everything, which causes fatigue and rubber-stamping.
- Asking for approval on nothing, then adding it after an incident.
- Showing reviewers only the agent's summary, not the underlying data.
- Holding paused state in memory, so restarts lose pending approvals.
- Letting the agent ask a person and act in the same step without waiting.
- Never reviewing whether human decisions actually differ from the agent's.
- Treating human review as a substitute for narrow permissions and guardrails.
Frequently asked questions
What does human-in-the-loop mean for AI agents?
It means the agent is designed so that people take part at chosen points: approving risky actions, editing drafts, answering questions, taking over escalated cases or reviewing samples of completed work.
Which actions should need human approval?
Start with actions that are irreversible, external, financial or affect access and personal data. Use thresholds so small, routine cases can run automatically.
How do agents pause for a person?
The framework interrupts the run, saves its full state to durable storage and creates a review task. When the person decides, the run resumes from the saved state with that decision.
What is automation bias?
It is the tendency of people to trust automated suggestions too much, approving them without real checking. It is a known risk in human oversight and is named in the EU AI Act's oversight article.
Is human-in-the-loop required by law?
For some systems, yes. The EU AI Act requires effective human oversight for high-risk AI systems, and GDPR gives rights around solely automated decisions with significant effects. Requirements depend on the use case and jurisdiction; see AI governance.
Does human review make an agent safe?
It greatly reduces risk but is not enough alone. Combine it with narrow permissions, guardrails, sandboxing, logging and testing.
Glossary
- HITL: human-in-the-loop.
- Approval gate: a pause awaiting a person's decision.
- Escalation: passing a case to a person.
- Interrupt: stopping a run at a checkpoint.
- Resume: continuing from saved state.
- Automation bias: over-trust in automated output.
- Rubber-stamping: approval without real review.
- Risk tier: a class of actions with shared controls.
- Audit trail: a record of decisions and actors.
- Four-eyes principle: two-person approval.
Summary
Human-in-the-loop design keeps people in control of AI agents by placing approvals, edits, clarifications, escalations and reviews at the points where errors would matter most. Tier actions by risk, gate the irreversible and external ones, persist state so runs can pause and resume, give reviewers clear evidence, and actively counter automation bias. Agent frameworks now support interrupts and approvals directly, and laws such as the EU AI Act require effective oversight for high-risk systems. This page is general information, not legal advice.
Related
It belongs to Patterns in my stack, under Agentic AI. Others in the same group: