Part 2 of a three-part sequence on AI systems thinking. Part 1: From Prompts to Loops.

The core argument

The winning AI products will not just have better models. They will put models inside better work environments.

By 2026 this stopped being a contrarian opinion and started being the consensus of people who ship agents for a living. The frontier models are close enough in raw capability that, as one comparison of the leading coding agents put it, the tool's architecture and workflow fit matter more than the model score in most practical scenarios. When the model is a near-commodity, the product is everything wrapped around it.

That wrapper has a name now: the harness. This essay is about what's in it, why each piece exists, and what an enterprise should actually evaluate when it buys or builds an agent system.

Why prompts are too small an abstraction

A prompt is a single instruction to a model. It was the right abstraction when the interaction was a single turn. It is the wrong abstraction for work, because work is not a turn — it is a loop that runs over time, touches real systems, fails in the middle, and has to recover.

Watch what happens when you try to scale a prompt up into a real task. You can write a beautiful prompt and get a clean function call and a clean result. Then, as engineers who have studied production agents describe it, you run it for 45 minutes on a real codebase, and things quietly fall apart — context overflows, permissions too loose or too annoying, tool results truncated without the model knowing. None of those failures are prompt-quality problems. No amount of prompt-craft fixes a context window that overflowed at minute 30. The prompt was never the unit that needed engineering. The loop was.

This is why the engineers closest to the metal keep arriving at the same counterintuitive ratio. After studying the Codex CLI source, one concluded that building an agent is 5% "call the model in a loop" and 95% context management, tool execution, sandboxing, and error handling. The prompt is the 5%. The harness is the 95%. If you are buying or building on the strength of the 5%, you are evaluating the wrong thing.

What a harness actually is

It helps to be precise, because "harness" is becoming a marketing word. A useful 2026 constitutive definition pins it down: a harness is a runtime layer with four necessary and sufficient elements: an agent loop, a tool interface, context management, and control mechanisms. Strip any one of those out and you don't have a harness — you have a generator, a tool wrapper, or a guardrail by itself.

But the four-part definition is the minimum. The harness that ships value in an enterprise has more layers than that, and the easiest way to see them is to follow a single tool call through the system.

flowchart TB M([Model proposes an action]) M --> P{Permission gate} P -->|denied| D[Reject / ask human] P -->|allowed| T[Tool interface executes] T --> S[Sandbox / execution environment] S --> R[Structured result:<br/>exit code, output, timing] R --> CM[Context manager:<br/>compact, truncate, inject] CM --> MEM[(Memory / state)] CM --> L[(Logs / provenance)] CM --> M R --> EV{Eval / test check} EV -->|fail| M EV -->|pass| HR{High-stakes?} HR -->|yes| HReview[Human review gate] HR -->|no| Commit[Action committed] HReview --> Commit D --> M

Every box in that diagram is a place where a naive "model in a loop" silently breaks, and every box is therefore a place where a real product competes. Let me name the layers the diagram implies, because this is the actual surface area of the product:

  • Tools — the actions the model can take, each with a typed interface.
  • Permissions — gates that decide whether a given action is allowed before it runs.
  • Context management — the machinery that keeps the loop coherent across many steps without overflowing the window.
  • Memory and state — what persists across steps, sessions, and restarts.
  • Logs and provenance — the record of what the agent did and why.
  • Evals and tests — automated checks that catch bad actions and bad outputs.
  • Human review — the gates where a person approves before something irreversible happens.
  • Recovery — what the system does when, not if, something fails.

These eight are the harness. They are also, not coincidentally, what you'd build around a new employee: access, permissions, onboarding context, a memory of past work, an audit trail, code review, sign-off on big decisions, and a way to undo mistakes. The harness is the management system for a non-human worker. That framing — managing the AI rather than just prompting it — is the whole skill, and it's why I think of harness construction as compound engineering: each layer you add compounds the reliability of every layer beneath it, the way good tests compound the value of good types, which compound the value of good interfaces.

Compound engineering: why the layers multiply

Here is the part that the "just use a better model" crowd misses. The harness layers are not additive. They're multiplicative.

A model that's 90% reliable per action is, naively, about 35% reliable over a ten-step task (0.9¹⁰). That's useless. But put an eval check after each step that catches half the errors and feeds them back for correction, and the effective per-step reliability climbs; add a permission gate that makes the worst errors impossible rather than merely unlikely, and you've truncated the bad tail entirely; add recovery so that the failures that do slip through are contained and reversible, and the system as a whole becomes something you can trust with real work — even though the underlying model never got better.

flowchart LR A[Raw model<br/>90% per step] -->|10 steps, no harness| B[~35% task success] A --> C[+ structured tool results] C --> D[+ eval/test feedback loop] D --> E[+ permission gates<br/>make worst errors impossible] E --> F[+ recovery/rollback] F --> G[Production-grade<br/>task success] style B fill:#e74c3c,color:#fff style G fill:#2c7,color:#fff

This is why harness-only changes — no model swap — produce results that look like a generational model upgrade. LangChain published a case study in which pure harness engineering moved their coding agent from rank 30 to top 5 on Terminal Bench 2.0 with no model swap, through structured verification loops and context injection. Same model. Better environment. Twenty-five places of improvement on a competitive benchmark. The model was never the bottleneck.

Coding harnesses: the reference implementations

The most mature harnesses in existence today are coding agents, because code is the domain where the loop is tightest (write, run, observe the error, fix) and the feedback is cheapest (compilers and tests are free oracles). They're worth studying as reference architectures even if you never write code, because they show what a complete harness looks like.

Claude Code is the most-documented example. Anthropic describes it as providing the tools, context management, and execution environment that turn a language model into a capable coding agent — note that the model is one item in a list of four. Its architecture decomposes into seven functional components: user, interfaces, the agent loop, a permission system, tools, state & persistence, and an execution environment. The permission system is the part most demos skip and the part that matters most: rather than "the agent has filesystem access," each tool has its own permission gate that checks a rule pipeline before anything executes. The model never touches the file system directly; it asks, and the harness decides.

Its context management is equally telling. Because the context window is the binding resource constraint, Claude Code runs five distinct context-reduction strategies before every model call — a five-layer pipeline that exists because no single compaction strategy handles all types of context pressure. That is pure harness engineering, invisible to the user, and decisive for whether a 45-minute task survives.

Codex (OpenAI's CLI agent) converged on a strikingly similar shape from an independent codebase — the loop, the context manager, the tool registry, the approval system. The convergence is the signal: two teams solving the same problem independently arrived at the same layers, which means those layers are intrinsic to the problem, not arbitrary design taste. Codex also added programmable lifecycle hooks — inject deterministic scripts at SessionStart, PreToolUse, PostToolUse, and other loop events to enforce guardrails, audit actions, and customize behavior without relying on prompt-level trust. That phrase — without relying on prompt-level trust — is the entire philosophy of the harness in six words. You do not ask the model nicely. You build a gate.

OpenHands (formerly OpenDevin) is the leading open harness, an open-source agentic developer environment that, like SWE-agent, relies on container-based isolation to contain arbitrary execution. It's the reference for teams that want to inspect and own every layer.

Two architectural lessons generalize from all three. First, structured tool outputs are non-negotiable: raw stdout isn't enough — you need exit codes, timing, truncation indicators, and clear success/failure signals, because the model has to understand what happened to decide what's next. Second, safety is table stakes, not a feature: any agent that can execute shell commands needs approval flows, sandboxing, and audit logging.

A notable 2026 entrant makes the "harness, not model" thesis explicit in its design. Flue, a TypeScript framework from the Astro team, is built on the equation Agent = Model + Harness, positioning the harness layer as the primary design concern. Where most frameworks focus on model orchestration, Flue centers on giving agents a secure, durable execution environment: sessions, tools, filesystem access, a built-in sandbox, and structured deployment targets — treating sessions, tools, skills, sandbox, and filesystem access as first-class primitives, not afterthoughts. Its design pitch — that agents like Claude Code and Codex broke the mold because you give them a task, not a pre-defined series of steps, and trust them to complete it using the context and tools you provide — is the loop thesis from Part 1, productized.

Orchestration frameworks: coordinating multiple loops

Coding harnesses run one loop well. Orchestration frameworks coordinate many loops — multiple agents, or one agent across a complex branching workflow. This is a different layer of the stack, and the 2026 field has sorted itself by a single axis: how much control the framework gives you over the loop versus how much it does for you.

flowchart TB subgraph Control["Control vs. simplicity tradeoff"] direction LR LG["LangGraph<br/>graph + durable state<br/>most control"] CA["CrewAI<br/>role-based crews<br/>fastest to prototype"] OS["Vendor SDKs<br/>OpenAI / Claude / ADK<br/>lowest friction"] FL["Flue / Mastra<br/>harness-first,<br/>TS-native"] end LG <-->|migration path| CA CA -->|outgrow role model| LG

LangGraph is the high-control end: it models an agent as a directed graph with conditional edges, and has matured into a durable execution engine with first-class human-in-the-loop support. Its distinguishing feature is built-in checkpointing with time travel — durable state that survives process crashes, plus the ability to rewind. You reach for it when one workflow needs cycles, branching, retries, durable checkpoints, or a real human approval step. The cost is boilerplate: a simple ReAct agent might be 40 lines elsewhere and 120 in LangGraph — you pay in boilerplate for what you gain in control.

CrewAI is the fast-prototyping end: role-based crews where you define each agent's role, backstory, and goal, then assemble them into a crew with a set of tasks. The code reads like English, and you can get from install to a working three-agent crew in about ten minutes. The tradeoff is exactly what you'd expect from a higher abstraction: lighter durability primitives, and when you need to debug a failure in a five-agent pipeline, the abstraction becomes opaque. The well-documented pattern is that teams that start with CrewAI for prototyping often migrate to LangGraph when they need production-grade state management.

The rest of the field fills in the corners: the OpenAI Agents SDK uses an explicit handoff model but is model-locked; the Claude Agent SDK and AWS Strands trade fine-grained orchestration for simplicity; Mastra is the TypeScript-native option, built TS-first with tool inputs, state, and stream events all strongly typed; and AutoGen/AG2 (now folded toward the Microsoft Agent Framework) specializes in conversational multi-agent patterns.

But here is the thing to internalize before you spend a week choosing: the framework is rarely the deciding factor. Two independent 2026 analyses landed on the same blunt conclusion. One: the framework debate is largely a distraction — the gap between a good agent system and a bad one is almost never the framework; it is the eval pipeline, the observability setup, and the failure recovery logic. The other warns that most projects labeled "we need agents" are actually DAG workflows in disguise — they need a well-structured chain with a few tool calls, not CrewAI or LangGraph, because agents add non-determinism, debugging complexity, and cost. The first architectural question is not which framework — it's do I even have a loop, or a fixed pipeline pretending to be one.

Durability layers: surviving the real world

There is one layer that sits underneath all the orchestration frameworks and quietly determines whether an agent can run in production at all: durable execution. It answers a question the model layer can't: what happens when the process crashes at step 40 of a 60-step job?

The reference technology here is Temporal, and the pattern is worth understanding even in the abstract because it's the same pattern every durability layer implements. Temporal is a durable execution runtime: you write what looks like ordinary code, but every step is recorded in an append-only event history. If a worker crashes, a new worker picks up the event log and replays the workflow from the beginning — skipping already-completed activities and resuming exactly where execution stalled. The crucial efficiency, and the reason it's safe for expensive LLM calls: replay does not re-execute activities — it re-runs the coordination code using recorded outputs, so API calls, database writes, and LLM calls are not repeated.

sequenceDiagram participant W as Workflow (agent loop) participant H as Event history (durable) participant A as Activities (LLM, tools, writes) W->>A: Step 1 — LLM call A-->>H: record result W->>A: Step 2 — tool call A-->>H: record result Note over W,A: 💥 Worker crashes at step 3 Note over H: history survives W->>H: New worker replays H-->>W: steps 1–2 returned from log (not re-run) W->>A: Step 3 resumes exactly where it stalled

This stopped being niche in 2026. Temporal raised a $300M Series D at a $5B valuation in February 2026, and the customer list tells the story: OpenAI, Replit, and Lovable build their agents on Temporal. The signal, as that write-up puts it, is that durable execution has moved from a niche infrastructure concern to a core requirement for production AI systems. Alternatives implement the same idea with different mechanics — Inngest checkpoints the output of each step.run() call, Restate and others vary the journal — but the principle is universal: record every step, replay on failure, never repeat side effects.

The durability layer also turns out to be a security and governance layer, which is the bridge to Part 3. A well-designed durable workflow enforces a discipline where the AI step produces an action plan, not direct writes; the workflow then enforces gates and permissions — only calling the ERP posting if policy thresholds pass and the right approval token exists; and every run stores evidence artifacts: retrieved document IDs, extraction confidence, policy rules applied, decision summaries. The model proposes; the durable workflow disposes, gates, and records. That is the harness philosophy expressed at the infrastructure layer.

What enterprises should evaluate when buying or building agent systems

Pull the threads together and you get a buyer's checklist that looks nothing like a model benchmark. If you're evaluating an agent product or platform — or deciding what to build — these are the questions that actually predict whether it survives contact with production. They map directly onto the harness layers.

On capability and tools. What tools can the agent call, and is each tool independently gated, or does "access" mean blanket access? In mature harnesses, each tool has its own permission gate that checks a rule pipeline before anything executes. Ask to see the gate.

On context management. What happens at minute 45? Ask the vendor how context is compacted and whether tool results can be truncated without the model knowing — because token efficiency matters enormously; without intelligent truncation and compaction, agents hit context limits after a few commands.

On observability. Can you reconstruct, after the fact, every action the agent took and why? This is the layer that LangGraph treats as first-class — it's designed to emit traces at every node transition — and it's the difference between a debuggable system and a black box. If you're in a regulated industry, your framework choice is driven by observability first.

On evals and recovery. Does the system have an eval pipeline and a defined failure-recovery path, or does it assume the happy path? Recall that across multiple practitioners, the eval pipeline, the observability setup, and the failure recovery logic — not the model, not the framework — separate good agent systems from bad ones. Build those three things and you're ahead of 80% of teams shipping agents today.

On durability. Does a crash at step 40 lose the whole job, or resume? With 40% of enterprise apps expected to embed AI agents by end of 2026, the ones that aren't built on a durable foundation risk remaining experiments or facing disastrous failures in production.

On the determinism budget. How much non-determinism can your use case tolerate? An internal analyst tool can absorb occasional re-runs; a customer-facing billing workflow needs near-deterministic behavior — which means constrained graphs, strict tool schemas, and timeout policies. Your tolerance for non-determinism directly constrains which architectures are even viable.

On lock-in. Is the orchestration and the harness logic something you own, or something a vendor owns on your behalf? This is the question that decides who controls your roadmap when the model layer commoditizes underneath you — and it's a strategic question, not a technical one.

Notice what's absent from that list: "which model." Not because the model is irrelevant, but because at current capability levels the model is the input you can swap, and the harness is the asset you keep. Frameworks and models will churn; the convergence is already visible, with the major orchestration frameworks all racing toward agents deployed as always-on services with durable state, managed hosting, and enterprise-grade security. The differentiator that persists across that churn is the quality of the work environment you build around whatever model you plug in.

The product was never the model

Here is the whole argument in one line: when models converge, the environment around the model becomes the product.

The harness — tools, permissions, context management, memory, logs, evals, human review, recovery, and the durable substrate underneath — is not plumbing you build reluctantly on the way to deploying a model. It is the thing you're deploying. The model is a remarkable, increasingly interchangeable component inside it. The teams that internalize this build agents that survive 45 minutes on a real codebase, a crash at step 40, and a prompt-injected document at step 12. The teams that don't ship a beautiful demo and a production incident.

Which raises the question that the demo never asks and Part 3 is entirely about: once the harness lets the loop reach real systems — real credentials, real databases, real money — what stops it from doing real damage? The answer is not a better model. It's a security architecture built on the assumption that the loop will, eventually, be turned against you.

Next: Least Agency: Zero Trust for AI Agents →

References