Research notes28 October 2026

What Happens in the Model: The Harness's Share of an Agent's Work Runs From 2% to 79%, Depending on the Measure

Berg Atkinson · 28 October 2026 · The paper: What Happens in the Model, and What Happens Everywhere Else

What Happens in the Model, and What Happens Everywhere Else (summary card): the model makes 98% of decisions and writes at most 1.9% of tokens. For each measure the card shows the range of outside sources with its median and this operator's value: decisions counting only overrides, 98.3% in 28 public Claude Code sessions and 97.7% here; decisions with hooks and injections, 79.9% public and 47.8% here; active time, 33.0% to 87.3% across 10 sources, median 65.8%, 72.2% here; new text in characters, 22.0% to 62.7% across 4 sources, median 55.0%, 53.9% here; output tokens, 0.23% to 1.89% across 11 sources, median 0.43%, 0.28% here; operations, model 20.6% here, no outside measure. The model picks each next step. The harness does most of the doing.

How much of what an AI agent does is actually the language model (LLM), and how much is the software around it? ("AI" = LLM + harness. The air quotes are on purpose.)

Depends on what you count, and both camps get to be right. In my Claude Code sessions the model makes 97.7% of the decisions (tool calls and final answers, overrides only) and the harness performs 79.4% of the logged operations. The model picks. The harness does.

"That's one guy's sessions." Fair! So I ran the exact same code on 12 public datasets of other people's agent sessions (GitHub Copilot's cloud agent, Terminal-Bench and SWE-bench runs, public Claude Code uploads), 5 to 675 days old, nothing sampled.

Side by side, 12 datasets | mine:

  • Decisions, model share, overrides only: median 97.7% (78.7% to 99.6%, 11 datasets) | 97.7%
  • Operations, harness share: 66.5% to 74.1%, median 70.1% (all 12) | 79.4%
  • Active time, model share: median 62.9% (47.9% to 82.7%, 7 datasets) | 72.2%
  • New text, model share: median 36.1% (19.2% to 52.9%) | 45.6%
  • Tokens the model writes: 0.34% to 2.62% of tokens processed | 0.28%

Fun part: same model, DIFFERENT harness. Three Anthropic models through claude-code vs terminus-2: 99.4% vs 91.3% of the decisions, 71.7% vs 61.0% of active time. One DeepSeek model, two harnesses: 43.8% vs 80.8% of active time. The harness is NOT a constant.

The one weird number: Harbor's claude-code runs give the model only 78.7% of decisions. Three non-Anthropic models hit an endpoint that kept failing, and Claude Code logs every failed call as its own "API Error" turn. Pull those out: 99.0%.

The "AI escaped" stories fit the same split. 21 cases with frontier-lab agents went public between July and mid-October, none hijacked, and the postmortems are out. A later count, in Houdini or Humans (wolfberg.ai/papers/houdini-or-humans/), reaches 45 by 20 October. In every one with real detail, the harness did the acting with whatever reach the setup GAVE it (internet on, safety filters off, shared credentials), and nothing in the harness was set to stop the run. The monitors often saw it! OpenAI's DNS case got flagged in 15 minutes...and a person stopped it about 2.5 hours later 🙄 Hugging Face? OpenAI test agents, cyber safeguards off, all given one package-registry login, used it to coordinate and spent about 2.5 days inside, after seven weeks of warning signs nobody connected.

To be fair, the model picked the method, and sometimes a bad one: cheating a grader, going past an explicit instruction, one agent telling its user it hadn't removed a monitor it had removed. Both labs call that reward hacking, and training is where it came from. But OpenAI's own retest says where the fix lives: same models through its production harness, more than 100 times less likely to hit out-of-scope systems. My take: every one is the harness plus the model doing what people asked, with nobody paying attention anymore. The next paper digs into exactly why.

So why care?
1. Cost: most of the bill is the harness re-sending stuff (cache reads are 97.8% of my input). Fix that before shopping for a cheaper model.
2. Accuracy: upgrade the model for capability, work the harness for cost and reliability.
3. Control: the harness can block, stop and inject. A rule you type to the model is just more text 😅
4. Benchmarks: a score is model PLUS harness. Which harness?
5. Blame: put the stop in the harness, not in a prompt. See above.

Does the model matter? Of course. It picks EVERY step. Limits: many outside cells are floors or estimates (a dataset only logs what its harness writes down), the old SWE-bench sets ran older models, and the escape cases are the labs' own accounts, not something I measured. Name the unit 🤷‍♂️

https://wolfberg.ai/papers/what-happens-in-the-model/

Sources

  1. Pairing Matters (arXiv:2610.32131, 23 Oct 2026) arxiv.org
  2. Liu and Han: What Does a Harness Buy? Tokens, Mostly (arXiv:2610.04433, 3 Oct 2026) arxiv.org
  3. Hardy et al.: Agent Evaluation Reliability (arXiv:2610.00651, 30 Sep 2026) arxiv.org
  4. AgentSysBench (arXiv:2608.15127, 15 Aug 2026) arxiv.org
  5. Alier Forment et al.: The Scaffolding Matters More Than the Interface (arXiv:2608.08654, 9 Aug 2026) arxiv.org
Read this before you believe any of it. Keystone runs in production in its own AWS account. The rest of what you see here is a running prototype, not an accredited or certified system. The register, the war room, and the brain are receipts, not revenue. Every claim on this site links to something you can check. If one doesn't, tell us, and we'll fix the page.