Agent Harnesses

An agent harness, also called an agent scaffold in some research and evaluation contexts, is the software system that enables a language model to operate as an agent. It processes inputs, assembles context, calls the model, routes tool requests, records events, applies controls, and returns or persists results. The harness is distinct from the model itself, the execution sandbox, and higher-level orchestration that schedules repeated work.

This separation makes agent behavior easier to analyze. The model proposes actions; the harness determines how proposals become tool calls and observations; the sandbox limits where code or commands execute; a durable session preserves events; and an external workflow or loop decides when another run should begin. Real systems may package these components together, but their responsibilities remain different.

Definition and boundaries

The model produces responses from the context it receives. The harness manages the repeated model-and-tool interaction. A session is the durable record of messages, actions, observations, and state that may outlive one harness process. Tools and connectors expose operations or information. A sandbox is the isolated environment in which generated code or commands run. Orchestration selects tasks, schedules sessions, and coordinates work above the harness.

The boundary is functional rather than commercial. A framework, command-line agent, hosted service, or custom application may all contain a harness. Conversely, a model API by itself is not a complete harness, because it does not necessarily provide tool execution, durable state, permissions, recovery, or outcome handling.

Core components

Input and context assembly

A harness transforms a task into the context sent to the model. It may combine system instructions, user input, tool descriptions, files, retrieved records, prior events, and summaries. Context management determines what is included, omitted, compacted, or loaded on demand. These choices affect both task performance and the risk of exposing irrelevant or sensitive information.

Model-call loop

During execution, the harness calls the model, interprets its response, performs allowed tool requests, returns observations, and repeats until a stopping condition is met. This reason–act–observe pattern appears in systems derived from ReAct and many later agent implementations. The harness must also handle malformed requests, model refusals, rate limits, timeouts, and partial tool failures.

Tool routing

Tool interfaces translate model output into operations. Clear names, descriptions, schemas, error messages, and response shapes help the model select and use tools correctly. Validation occurs before execution, and results should distinguish success, failure, uncertainty, and partial completion. Model Context Protocol formalizes prompts, resources, and tools as distinct server primitives, although using MCP does not determine the rest of the harness architecture.

State and event records

State may include conversation events, task status, files, checkpoints, tool outputs, external identifiers, and resource usage. Durable event records support recovery, audit, debugging, and context reconstruction. They also allow the harness process to be stateless or replaceable while the session persists.

Execution environment and controls

The execution environment provides filesystems, processes, network access, and other resources. Sandboxes and containers can enforce boundaries around these capabilities. Permissions determine which tools and destinations are available, while outcome handling records changes and maps errors or completion back to the surrounding workflow.

Agent-computer interfaces

The agent-computer interface is the set of actions, observations, commands, and environmental affordances through which an agent uses a computer. SWE-agent demonstrated that purpose-built interfaces can materially affect software-engineering performance. This makes interface design part of agent behavior, not merely an implementation detail.

An effective action interface exposes operations at a useful level of abstraction. Very low-level tools may consume context through repetitive calls, while overly broad tools may hide consequential behavior. Observation design has a similar tradeoff: complete raw output can overwhelm the context window, but aggressive summarization can remove evidence needed for the next decision.

Tool descriptions should state preconditions, effects, failure modes, and important limits. Schemas should make invalid requests difficult to express. Responses should return identifiers and evidence that later steps can inspect. Evaluation is needed because an interface that appears intuitive to a human may not work reliably for a particular model.

Long-running work

Long-running agents must preserve continuity across failures, context limits, and fresh sessions. A common approach divides the goal into incremental tasks and records progress after each meaningful change. Handoff artifacts can describe completed work, current state, tests or other evidence, unresolved questions, and the next bounded action.

Context compaction and context reset address different problems. Compaction summarizes older events so a session can continue with less context. A reset starts a fresh context, which can reduce accumulated confusion but loses unstored information. Both depend on durable state if the agent must resume accurately.

Harnesses should not rely only on the model to declare that a large task is complete. Premature completion can be reduced by explicit task inventories, independent checks, repository or external-system inspection, and a surrounding workflow that compares outcomes with the original goal. Recovery should distinguish an interrupted action from an action that completed externally but failed before its result was recorded.

Evaluation and verification

An agent harness runs the agent; an evaluation harness runs tasks or trials, records traces and outcomes, applies graders, and aggregates results. Keeping the distinction clear helps teams test changes to prompts, tools, models, and harness components without confusing the system under test with the system measuring it.

Evaluation may examine final outcomes, intermediate actions, policy compliance, cost, latency, and recovery behavior. For open-ended work, several graders or human review may be needed. Generator–evaluator separation can help when the evaluator has independent criteria or evidence. It is less useful when both components repeat the same assumptions.

More scaffolding is not inherently better. Anthropic’s long-running application experiments found that planner, generator, and evaluator components can help under some model and task conditions, while later or more capable models may make parts of that structure unnecessary overhead. Harness design should therefore be evaluated against current tasks and models rather than treated as a permanent stack of roles.

Security and containment

Agent-generated actions should run under structural controls. Credentials and sensitive resources should be isolated from code the agent can inspect or alter. Sandboxes can restrict filesystem access, processes, network destinations, and resource use. Tool permissions can narrow the operations available for a task.

Containment works best through overlapping defenses. Model-layer safety, prompt instructions, tool validation, sandbox boundaries, credential brokers, egress controls, monitoring, and human approval address different failure modes. No single layer establishes that unattended execution is safe.

External content can contain instructions that conflict with the user’s goal or system policy. A harness should label untrusted data, limit its authority, and prevent retrieved content from silently expanding permissions. Consequential actions such as publishing, sending messages, moving funds, or changing production systems should require evidence and authority outside the model’s own assertion.

Design tradeoffs

A small transparent harness is easier to inspect, but complex tasks may require durable sessions, specialized tools, recovery logic, and evaluation. Each component should have a demonstrated purpose. Added layers increase latency, token use, and the number of failure boundaries.

Model-specific assumptions can improve performance while reducing portability. A tool schema, compaction method, or planning routine tuned for one model may behave differently after a model change. Observability helps identify these shifts, but collecting every trace can create cost and privacy concerns. Useful telemetry should connect actions to decisions, evidence, and outcomes.

Interoperability remains incomplete. Protocols can standardize tool discovery and invocation while leaving session state, permissions, identity, evaluation, and recovery to each implementation. Clear component boundaries make those differences easier to manage.

Open questions

• Which harness abstractions remain stable across model generations and task domains?

• Which state belongs in the session, the task record, the execution environment, or an external workflow?

• How should partial completion and uncertainty be represented to downstream systems?

• Which agent-computer interfaces generalize across models rather than overfitting one system?

• How much trace evidence is enough for debugging and audit without excessive cost or data retention?

• When does an independent evaluator justify its additional complexity?

• How should least-privilege permissions adapt during long-running work?

Related topics

• Loop Engineering

• Agent-First Design

• Agent-Oriented Developer Workflows

• Agent-ready business systems

• Human-In-The-Loop AI Workflows

Further reading

• Building Effective AI Agents — Anthropic

• Effective Harnesses for Long-Running Agents — Anthropic

• Harness Design for Long-Running Application Development — Anthropic

• Scaling Managed Agents — Anthropic

• Demystifying Evals for AI Agents — Anthropic

• Writing Effective Tools for AI Agents — Anthropic

• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

• ReAct: Synergizing Reasoning and Acting in Language Models