So What The Hell
Is A Harness?
DeepSeek V4 Flash is 90× cheaper than Opus 4.8 and good enough for most coding work. I still haven't switched. Neither have about 8 out of 10 devs I know. The reason isn't the model — it's the seven layers of software wrapped around it, which almost nobody can name and nobody benchmarks.
The Switch I Keep Not Making
DeepSeek V4 Flash 0731
$0.28
per 1M output tokens
Claude Opus 4.8
$25.00
per 1M output tokens
Input is $0.14/M on a cache miss and $0.0028/M on a cache hit — so on a long agentic session, the real gap runs wider than 90×.
On July 31st, DeepSeek shipped DeepSeek-V4-Flash-0731 — a 284B mixture-of-experts model with 13B active parameters and a 1M-token context window. It scores 82.7 on Terminal-Bench 2.1, 54.4 on DeepSWE, 70.3 on Toolathlon. It beats DeepSeek's own V4-Pro preview on all nine published benchmarks, which is a 13B-active model beating its 1.6T sibling.
It is not better than Opus 4.8. It lost to Opus on all nine of those same benchmarks, and I want to be precise about that because the interesting claim doesn't need the exaggeration. It is good enough. For most of what I do on an average day — refactors, test scaffolding, boilerplate, chasing a bug through four files — the gap between good enough and best is worth a lot less than 90×.
So by any rational reading I should have moved my daily driver to an open harness running Flash. I haven't. I asked around, and roughly eight out of ten developers I know haven't either. When I push them on why, almost everyone lands on the same shapeless sentence:
“I tried it. It just feels worse.”
That sentence is the whole post. Something real is being detected and nobody can name it, so it gets filed under vibes and the switch never happens. The thing being detected is the harness — and once you can see its parts, “feels worse” resolves into about seven specific, fixable engineering decisions.
The Model Does Almost Nothing
Strip away the scaffolding and what's left is smaller than people assume.
A language model takes a sequence of tokens and predicts the next one. That is the entire job. It has no memory between calls, no ability to read a file, no way to run a command, no concept that your repository exists. It cannot even choose to stop — it emits a token that means stop, and something else has to notice.
Every single thing that makes a coding agent feel like an agent lives outside the weights. The file it opened, the test it ran, the error it recovered from, the decision it remembered from twenty minutes ago — all of that is orchestration code written by humans, wrapped around a function that predicts tokens.
That wrapper is the harness. And the reason the word keeps getting used without being defined is that it isn't one component. It's a stack.
assemble_context() # layer 1 — what does it get to see?
→ call_model() # the commodity part
→ parse_tool_calls() # layer 2 — what can it do, and how do failures read?
→ check_permissions() # layer 5 — is it allowed?
→ execute()
→ feed_results_back() # layer 2 again — shaped for recovery, or just "Error"?
→ decide_continue() # layer 3 — loop, stop, or spawn a subagent?
→ compact_if_needed() # layer 4 — what survives the boundary?
→ checkpoint() # layer 7 — can this turn be undone cleanly?One line in that block is the model. Eight are the harness. When two agents running identical weights behave differently, the difference is in those eight lines, and it compounds every turn.
The Seven Layers
Ordered by how much they differentiate harnesses in practice, not by how much attention they get. Each one has a failure mode you have almost certainly hit.
Context assembly
What enters the window, in what order, and what is deliberately withheld?
Does it build a repo map up front, or read files lazily as it goes? Is your conventions file loaded once at session start or re-injected every turn? Are tool results truncated, summarised, or dropped? Is the prompt laid out so the KV cache actually hits on turn two, or does every turn re-pay for the whole prefix? This is the largest single differentiator between harnesses and there is no public benchmark for any of it.
Tool design
Not which tools — what granularity, and what happens when one fails?
An edit tool that requires an exact string match forces the model to prove it read the file. One that rewrites the whole file lets it hallucinate the other 400 lines. A failed call that returns 'Error' teaches the model nothing; one that returns the three closest near-matches lets it self-correct on the next turn. Tool descriptions are prompt engineering that happens to live inside a JSON schema, and most of them are written like API docs instead.
The loop
When does it act, when does it stop, and what runs in parallel?
Turn structure, stop conditions, whether independent tool calls are batched into one round trip or serialised into ten. Whether the harness can spawn subagents with their own clean context, and whether those subagents can spawn more. What happens when you hit escape halfway through a twelve-step edit — does the work unwind cleanly, or do you get a half-applied refactor.
Context lifecycle
What survives when the window fills up?
Every non-trivial task eventually exceeds the context window. What happens at that boundary is a design decision: hard truncation, rolling summarisation, a written scratchpad the agent maintains itself, or an external memory file it re-reads. The choice determines whether hour two of a task is as good as hour one. A 1M-token window does not solve this — it moves the wall, it doesn't remove it.
Permissions and sandbox
What can it do without asking?
This one sets the autonomy ceiling for everything above it. Allowlists, sandboxing, which commands need confirmation, whether the agent gets network access. It reads like a security feature, and it is, but it's also a capability feature: an agent that cannot execute cannot verify, and an agent that cannot verify is guessing.
The verification loop
Can it run the tests, read the output, and try again?
Write code, run it, read the failure, fix it, run it again — unattended, until it passes. This is the entire line between a tool that generates code and a tool that delivers working code. It sounds trivial and it is not: it requires the sandbox from layer 5, error output shaped by layer 2 into something the model can parse, and enough of layer 4 that the agent still remembers what it was trying to do on attempt four.
Recovery
What happens after a bad edit lands?
Checkpoints, undo, whether the harness can diff its own damage. Every agent produces bad edits — that isn't the failure mode. The failure mode is a bad edit that can't be isolated and reversed, so you abandon the whole session and start over. Cheap, reliable undo is what makes it safe to let the agent take big swings in the first place.
Three Pieces Of Evidence
If the harness were a thin wrapper, none of the following would be true.
One: the leaderboard flips depending on the wrapper. Claude Code leads SWE-bench Verified. Codex leads Terminal-Bench. Broadly the same class of frontier models underneath both. If the model were the deciding variable, one tool would win both. Instead the ranking inverts based on which benchmark better matches each harness's design assumptions.
Two: some bugs follow the wrapper, not the weights. OpenCode produces reformatting artifacts across multiple different underlying models. A defect that survives a model swap is not a model defect — that's just elimination. It's a diff-application or context-assembly bug, sitting in layers 1 and 2, and no amount of model upgrade will touch it.
Three: token consumption differs by 3–4× on identical work.Community head-to-heads consistently put Claude Code at roughly three to four times Codex's token use for the same task. That is not the model being hungrier. That is a context assembly strategy — how much gets loaded up front, how aggressively results are pruned, how the cache is laid out. Pure layer 1, and it lands directly on your bill.
| Observation | Naive reading | What it actually localises to |
|---|---|---|
| Claude leads SWE-bench, Codex leads Terminal-Bench | Different models are better at different things | Layers 1 & 3 — context strategy and loop design match different task shapes |
| OpenCode reformats code across several models | Those models are sloppy | Layer 2 — edit-tool granularity and diff application |
| Claude Code burns 3–4× Codex's tokens | Claude models are verbose | Layer 1 — how much context is assembled per turn, and cache layout |
| “It works great for an hour then loses the plot” | Context window too small | Layer 4 — compaction strategy, not window size |
Every Model Score Is A Harness Score
This is the part that made me want to write the post, and it's hiding in DeepSeek's own release notes.
Go back to those V4-Flash numbers from the top. 82.7 on Terminal-Bench 2.1. 54.4 on DeepSWE. 68.7 on DSBench FullStack. Genuinely impressive, and the jump from the April preview's 7.3 on DeepSWE to 54.4 is one of the largest post-training gains anyone has published this year.
Now read the caveats that shipped with them. Several of those evaluations were run on DeepSeek's own internal harness. The Terminal-Bench result depends on a “minimal mode” component they have not released. Two of the DSBench evaluations use internal datasets. None of this is fraud and DeepSeek isn't hiding it — it's stated plainly. But follow it through.
This isn't a DeepSeek problem. Every lab publishing agentic benchmarks has the same structural issue, and the incentive runs one direction: your scaffolding is tuned for your model, so your model looks best inside it. The industry standardised on reporting the model and omitting the harness, which was defensible when harnesses were thin and is not defensible now.
The practical consequence for you is narrower and more useful: a benchmark delta between two models tells you very little about what you'll experience, because you will not be running either one inside the harness that produced the number. The only test that predicts your experience is your work, your repo, your harness.
The Hierarchy Nobody Names
Six levels of agentic coding. The interesting thing is which transitions are model jumps and which are harness jumps.
The L2 → L3 jump is the one that matters. Same weights on both sides of it.
L0 to L2 was a model story. Autocomplete became useful because models got better at short completions, and inline editing became useful because they got better at instruction-following. Real capability gains, driven by weights.
L2 to L3 is not. Put the same model behind an inline editor and behind a terminal agent with a verification loop, and you get two different products. Nothing about the weights changed. What changed is that the second one can execute, observe the result, and try again — layers 5 and 6 — which converts a generator into something that closes its own feedback loop.
This is why the DeepSeek price gap doesn't automatically win. Moving from Opus to Flash is a move along one axis. Moving from a strong harness to a weak one can knock you back down a level on a different axis, and level beats price. A 90× saving on an L2 experience is a bad trade against an L3 one, which is exactly what those eight developers are detecting when they say “it feels worse.”
L3 to L4 is a harness jump too, and mostly an unsolved one — it needs layer 4 to be genuinely good, because async delegation means nobody is watching when the context boundary hits.
The Moat Moved And Nobody Announced It
For three years the defensible thing in AI was weights. That is ending in public. DeepSeek is shipping near-frontier open models at 1/90th the price, and the gap between best and good-enough keeps narrowing while the price gap keeps widening. If your product's advantage was model access, you are on a clock.
The layer that is not commoditising is the seven above it. And here is what makes that interesting rather than merely true: unlike weights, the harness layer is mostly open. OpenCode is MIT-licensed with 171k stars and works across 75+ providers. The protocols underneath — MCP, the tool-definition conventions, the agent file formats — are open specifications. Nobody needs a GPU cluster to contribute to the part of the stack that is currently deciding outcomes.
Most of the market is stuck at L2, and it isn't stuck for lack of model quality. It's stuck because harness work is unglamorous: no benchmark number, no launch tweet, no scaling-law chart. It's error message design, cache layout, compaction heuristics, and undo. It is exactly the sort of unfashionable engineering that ends up being where the value is.
The cheapest model on earth doesn't matter if the thing running it can't recover from a failed edit.
I'm still not switched. But I know what I'm waiting for now, which is better than a vibe — I'm waiting on layers 1, 4 and 6. When an open harness closes those, the 90× stops being theoretical and I'll move the same week.
Keep Reading
Hey DeepSeek, I Fixed Your Branding
The same model, from the other direction — how DeepSeek-V4-Flash-0731 got a name that hides everything interesting about it.
Enterprise Agentic AI: From Chatbot to Autonomous Agent
What these levels look like inside an organisation, and what has to be true before you can trust an agent with a real workflow.
Shashank Padala
Founder, Kirak Labs · AI Product Leader
AI Product & Transformation Leader with 8+ years building production LLM systems. Previously led GenAI integration into an internal content-authoring platform at a Fortune 500 enterprise, serving millions of employees globally — an AI assistant embedded in the CMS that surfaced grounded, cited insight from engagement and support-ticket data to inform what the team published next.