Agent harness

Overview

A language model on its own does one thing: it reads some text and writes some text back. It cannot open a file, run a test, search the web or remember what it did an hour ago. An agent harness is the software that surrounds the model and gives it those abilities. It sends the model a task, reads the model's reply, notices when the reply asks for a tool, runs that tool, feeds the result back and repeats until the work is done. Along the way it decides what the model is allowed to see, what it is allowed to do, how much it may spend and when it must stop.

The name comes from the idea of a harness on a horse, or a test harness in software: equipment that connects raw power to useful work and keeps it under control. Two agents built on the same model can behave very differently because their harnesses differ. One may give the model good tools, clean context and clear stopping rules. The other may flood it with noise, let it loop for ever or let it delete things it should not touch.

An everyday comparison is a professional kitchen. The chef is the model: skilled, creative, able to decide what to cook. The kitchen is the harness. It provides the knives and ovens (tools), the order tickets and prep notes (context), the rules about who may use the deep fryer (permissions), the fire doors (sandbox) and the budget for ingredients (limits). A great chef in a chaotic kitchen still produces poor service. This page explains what a harness contains, how each part works, and how to think about harness design alongside model choice. It builds on the ReAct loop, function calling and context engineering.

What it is

A harness is a runtime, not a model and not a single library call. Most harnesses contain the same set of parts, even when they name them differently.

The model provider supplies intelligence. The harness supplies everything else: the environment, the guard rails and the bookkeeping. In practice the harness also carries a large system prompt that explains the tools, the house rules and the expected working style, so the boundary between "prompt" and "software" is blurred.

It helps to separate three layers. A model API answers one request. An agent framework, such as LangGraph or the OpenAI Agents SDK, gives building blocks for writing a loop and wiring tools. A harness is a complete, opinionated runtime that already has a loop, tools, permissions and context handling, ready to run a task end to end. The lines are not sharp, and many products sit between the last two.

Where it came from

The word "harness" has long been used in software testing for the code that drives a program under test. Its use for agents grew as language models became good enough at tool use that the surrounding software became the main lever on quality.

Early agent experiments in 2022 and 2023 were simple loops: prompt the model to think, choose an action, observe the result, and repeat. Research then showed that the interface between the model and its environment mattered a great deal. The SWE-agent paper, presented at NeurIPS 2024 by researchers at Princeton, introduced the idea of an agent-computer interface: commands, file viewers and feedback formats designed for a model rather than a human. Its central finding was that a carefully designed interface improved an agent's ability to fix real software issues, with the model held fixed. OpenHands, an open platform for software agents that was first released as OpenDevin, was presented at ICLR 2025 and packaged a sandboxed runtime, tools and agent loops into a reusable system.

Commercial and open coding agents then made the term common. Block released goose, an open-source agent under the Apache 2.0 licence, in January 2025. Google released Gemini CLI as an open-source terminal agent under Apache 2.0 in June 2025. OpenAI's Codex CLI is open source under Apache 2.0, written in Rust, and runs commands inside a platform sandbox. In September 2025 Anthropic renamed its Claude Code SDK to the Claude Agent SDK, explaining that the harness behind Claude Code could power many kinds of agent, not only coding ones. In November 2025 Anthropic's engineering blog published guidance on harnesses for long-running agents, describing an initialiser agent that writes a feature list and a coding agent that makes incremental progress, commits to git and keeps a progress file for the next session. In December 2025 the Linux Foundation formed the Agentic AI Foundation, with MCP, goose and AGENTS.md as founding projects. At the time of writing, providers also offer hosted harnesses, where the provider runs the loop and the sandbox for you.

How it works

The loop

Every harness has a loop at its centre. It can be written in a few lines:

state ← [system_prompt, tool_definitions, user_task]
repeat
    reply ← model(state)
    if reply contains no tool calls: return reply
    for each tool call c in reply:
        d ← permission(c)                    (allow, ask or deny)
        r ← run_in_sandbox(c) if d = allow, else refusal message
        state ← state + [c, r]
    if tokens(state) > θ: state ← compact(state)
    if steps > S_max or cost > C_max: stop and report
until done

cost ≈ Σₜ ( p_in · n_in,t + p_out · n_out,t )

Here state is the list of messages sent to the model, θ is the token threshold at which the harness shrinks the history, S_max and C_max are the step and cost limits, p_in and p_out are the prices per input and output token, and n_in,t and n_out,t are the tokens read and written at step t. The cost line shows why long loops get expensive: each step re-sends the growing history, so input tokens dominate unless the harness caches or compacts.

Anthropic describes the same loop in plainer words as gather context, take action, verify the work, and repeat. Verification can be a rule (tests pass, the file parses), visual feedback (a screenshot) or another model acting as a judge.

Tools

The model never runs anything itself. It writes a structured request, usually JSON that matches a schema the harness published, and the harness executes it. Good harnesses offer a small set of sharp tools: read a file, edit part of a file, search, run a shell command, fetch a web page. Tool design is interface design. Clear names, tight schemas, helpful error messages and outputs that are short enough to read all change results. Many harnesses also load extra tools through the Model Context Protocol, so the same agent can reach databases, ticket systems or browsers without custom code.

Context management and compaction

The model's context window is finite, and quality tends to fall as it fills with stale material. The harness decides what each call contains. Common techniques are: keep tool outputs short; clear old tool results once they are no longer needed; load files only when the agent asks for them; and compact, which means replacing older turns with a summary. Compaction can run on the client or, with some providers, on the server. For example, at the time of writing Anthropic's API offers server-side compaction in beta, either on demand or when input reaches a token threshold, and a separate context-editing feature that clears old tool results by rule. The trade-off is simple: a summary saves space but can drop a detail a later step needs, so good harnesses keep critical facts in files rather than relying on the summary alone.

Permissions and hooks

A permission system sits between the model's request and the execution. Typical modes are "ask before anything risky", "auto-approve edits inside the project" and "plan only, change nothing". Rules can allow read-only tools, require approval for network access and deny writes to secrets. Hooks are developer callbacks at fixed points in the loop. In the Claude Agent SDK, for example, a PreToolUse hook runs before a tool call and can allow, deny or rewrite it, a PostToolUse hook sees the result, and PreCompact runs before history is summarised. Hooks are where teams put audit logging, policy checks and human-in-the-loop approvals.

Sandboxes

Because the model can be wrong or manipulated through prompt injection, the commands it runs should be isolated. Codex CLI, for instance, uses Apple's Seatbelt on macOS and bubblewrap with seccomp on Linux, with network access off by default. Other harnesses run each session in a container or a microVM. See sandboxing for the isolation options.

Memory and sub-agents

Within one session, memory is just the context. Across sessions the harness needs durable notes: a project instructions file, a progress log, a task list, or a store of facts. See agent memory. Sub-agents are child loops with their own fresh context. The parent hands each one a focused job, such as "search the codebase for every caller of this function", and gets back only a short answer. This keeps the parent's context clean and lets independent work run in parallel.

Budgets and stopping

Agents fail expensively when they never stop. A harness sets limits on turns, tokens, wall-clock time and money, and defines what success looks like, such as "all tests pass" or "the user approved the plan". When a limit is hit, the harness should stop cleanly and report what was done and what remains. See cost and latency budgets and long-running tasks.

Key concepts and terms

A worked example

This is an illustrative scenario, not a description of any real company.

A small team wants an agent that fixes failing tests in a 40,000-line Python repository overnight. They start with a bare loop: the model gets a shell tool and the instruction "make the tests pass". On the first night the agent runs the full test suite 30 times, each run printing about 8,000 tokens of output. By step 25 the context is full of old test logs, the model forgets which failures it already fixed, and the run ends at the step limit with three of eleven failures fixed.

They redesign the harness, keeping the same model.

1. Tools: they add a run_tests tool that accepts a test path and returns only the failing test names and the first 40 lines of each traceback. A typical result shrinks from 8,000 tokens to about 900.
2. Context: they clear tool results older than ten steps and set compaction to trigger at 70% of the window, with a custom summary prompt that must keep the list of fixed and remaining failures.
3. Memory: an initial step writes progress.md with all eleven failures. After each fix the agent ticks one off and commits to git.
4. Sub-agents: for each failure, a sub-agent searches the codebase for the relevant code and returns a short summary of the call path.
5. Permissions and hooks: writes are allowed only inside src/ and tests/; a PreToolUse hook refuses any command containing rm -rf or git push; network access is off.
6. Budget: 150 turns and a fixed spending cap; on reaching either, the agent must write a final report.

On the second night the agent fixes nine of eleven failures in 96 turns, flags two as needing a human decision, and the morning report lists the commits. Nothing about the model changed. The difference came from smaller tool outputs, cleaner context, durable progress notes and clear limits.

A minimal hook of the kind used in step 5, written with the Claude Agent SDK for Python:

import asyncio
from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions, HookMatcher

BLOCKED = ("rm -rf", "git push")

async def guard_shell(input_data, tool_use_id, context):
    command = input_data["tool_input"].get("command", "")
    if any(b in command for b in BLOCKED):
        return {
            "hookSpecificOutput": {
                "hookEventName": input_data["hook_event_name"],
                "permissionDecision": "deny",
                "permissionDecisionReason": "Destructive or publishing commands are not allowed",
            }
        }
    return {}  # allow; normal permission rules still apply

async def main():
    options = ClaudeAgentOptions(
        max_turns=150,
        hooks={"PreToolUse": [HookMatcher(matcher="Bash", hooks=[guard_shell])]},
    )
    async with ClaudeSDKClient(options=options) as client:
        await client.query("Read progress.md and fix the next failing test.")
        async for message in client.receive_response():
            print(message)

asyncio.run(main())

Strengths and limitations

Strengths:

• Turns a text-only model into a system that can act, check its work and recover from errors.
• Improves results without changing the model, often more cheaply than switching to a larger one.
• Centralises safety: permissions, sandboxes and hooks apply to every tool call.
• Makes long tasks possible through compaction, durable memory and sub-agents.
• Produces traces that support debugging, evaluation and audit.
• Lets teams swap models behind a stable interface when the harness is model-agnostic.

Limitations:

• More moving parts mean more ways to fail: bad tool schemas, lossy summaries, runaway loops.
• Harness behaviour is often tuned to one model family; moving to another can need re-tuning.
• Compaction and clearing lose information, and the loss is not always visible.
• Permissions that ask too often train users to click "approve" without reading.
• Sandboxes reduce, but do not remove, risk from prompt injection through tool outputs.
• Costs grow with every step because the history is re-sent; budgets need active management.

How it compares with alternatives

The practical question is not "harness or not" but how much of the harness you want to own. Teams often start with an existing harness, then replace parts, such as the tool set or the permission policy, as their needs become clear.

Practical guidance and best practices

1. Design tools for the model: short, structured outputs, clear errors and one obvious way to do each job.
2. Keep the context lean. Return summaries rather than raw logs, clear stale tool results and load files on demand.
3. Write important state to files. A task list and a progress log survive compaction and session restarts.
4. Use sub-agents for search-heavy or parallel work, and have them return short answers, not transcripts.
5. Start with strict permissions and loosen them deliberately. Deny writes to secrets and production by default.
6. Run commands in a sandbox with network egress off unless a task needs it.
7. Put policy in hooks, not in the prompt alone. A prompt can be ignored; a hook that returns "deny" cannot.
8. Set budgets for turns, tokens, time and money, and make the agent report when it stops.
9. Define "done" with a check the harness can run, such as tests, a linter or a schema validation.
10. Trace every call and decision, and evaluate the harness and model together rather than separately.

Common mistakes

• Blaming the model for failures caused by noisy tool outputs or a polluted context.
• Giving the agent one giant "run anything" tool instead of a few well-described ones.
• Relying on compaction alone for critical facts instead of writing them to durable notes.
• Running with full permissions "just for testing" and forgetting to tighten them.
• Treating tool outputs as trusted, so injected instructions in a web page or file steer the agent.
• Having no step or cost limit, so a confused agent loops until someone notices the bill.
• Changing model and harness at the same time, so nobody knows which change helped.

Frequently asked questions

Is the harness more important than the model?

Both matter, and they interact. A stronger model tolerates a weaker harness better, but research such as SWE-agent showed that interface design alone can change results substantially with the same model. In practice, teams usually get large gains from harness fixes before they need a bigger model.

What is the difference between a harness and an agent framework?

A framework gives you components to build a loop. A harness is a complete runtime with the loop, tools, permissions and context handling already decided. Frameworks offer more freedom; harnesses offer a working system sooner.

What is compaction and when should it run?

Compaction replaces older turns with a summary so the conversation fits the context window. Run it before quality starts to drop, not at the last moment, and use a summary prompt that keeps the facts later steps need.

Do I need a sandbox if I review every command?

Yes for anything long-running or unattended. Reviews get tired and rushed, and a sandbox limits damage when a bad command slips through. Sandboxes and approvals work best together.

How do hooks differ from instructions in the system prompt?

Instructions ask the model to behave; hooks enforce behaviour in code. A hook can block a tool call regardless of what the model wants, log every action or require approval.

Can one harness work with different models?

Many can, especially open-source ones that support several providers. Expect to adjust prompts, tool descriptions and limits, because models differ in how they use tools and how verbose they are.

What should a harness log?

Every model call with token counts, every tool call with its inputs and outputs, every permission decision and every compaction. This is the material for debugging, cost analysis and audit.

Glossary

• Harness: the runtime that wraps a model and runs it as an agent.
• Agent loop: the cycle of model call, tool call and result.
• Tool schema: the structured description of a tool's name and inputs.
• Compaction: summarising older context to save space.
• Hook: code that runs at a fixed point in the agent's lifecycle.
• Permission mode: the policy for which actions need approval.
• Sandbox: an isolated place where the agent's commands run.
• Sub-agent: a helper agent with its own context window.
• Progress file: a durable note used to resume work across sessions.
• Budget: a limit on steps, tokens, time or money.
• Trace: a record of everything the agent did.

Summary

An agent harness is everything around the model that makes it an agent: the loop that alternates thinking and acting, the tools it can call, the context it sees, the permissions and hooks that control it, the sandbox that contains it, the memory and sub-agents that extend it, and the budgets that stop it. Harness design is a first-class engineering task. The same model can succeed or fail depending on tool outputs, context hygiene and stopping rules. Start from a proven harness, keep context lean, write state to files, enforce policy in code, sandbox execution, set clear limits and measure the model and harness together.

Related

It belongs to Frontier topics in my stack, under Agentic AI. Others in the same group: