An agent's real skill isn't decided by the weight of the model, but by the structure wrapped around it. We saw that in hard numbers this week. We had the same model write our posts, and before we changed the structure, 40 of 50 published pieces contained factual distortions. After the change, errors were caught before the very first publish. The model didn't change by a single character. What changed was the harness.
This post is a straight record of how that went. What harness engineering and loop engineering actually are, why papers on this are pouring out right now, and how a small team can put them to work in practice — told through our own case and three papers.
The harness is everything outside the model. Google Cloud AI Research's RRSI paper from this year defines it like this: "The capability of an LLM agent is heavily amplified by the harness — the prompts, control flow, tools, memory, and context management surrounding a fixed backbone model" (arXiv:2609.24972). The point is that even with the same model, the output changes completely depending on what you show it, what order you assign the work in, how you inspect the results, and where you route failures.
Think of a car: the model is the engine, and the harness is everything else — the transmission, the brakes, the dashboard, the seatbelts. Even with identical engine output, a botched transmission makes the car slow, and without brakes it crashes. For the last few years everyone was racing on engine horsepower, but the papers coming out this year show the center of gravity shifting noticeably toward the harness.
Let's look at just three.
| Paper | Core claim | What caught our eye |
|---|---|---|
| NeoHorse-1 (arXiv:2609.08183) | The execution traces a live routing harness leaves behind are themselves the raw material for the next round of training | The harness is an execution device and an observation device at once |
| SoL-Pi (arXiv:2609.20519) | Running harness improvement as an automated research loop cut token usage by 44.7–49.0% while holding the score drop to within a few percent | Harness improvement itself can be turned into a loop |
| RRSI (arXiv:2609.24972) | Harness self-improvement overfits. It needs a regularization principle | A loop needs brakes |
The NeoHorse-1 team's observation is especially interesting. A deployed harness records, turn by turn, which capabilities were demanded, which model tier was selected, and what interaction followed. That record is itself evidence of "what the model can and can't do right now," and feeding that evidence back into the next round of training is the starting point of recursive self-improvement (RSI). Build a good harness and you don't just improve execution — the system gains an eye for observing itself.
SoL-Pi goes a step further. Researchers from NVIDIA, NTU, and MIT handed the harness-improvement work itself to the AI. A base harness runs the tasks, a research AI pores over the execution traces and proposes improvement candidates, and capability and efficiency gates filter those candidates. The harness that combines the four mechanisms that survived this loop (action fusion, online context compression, observation packing, and evidence-preserving abridgement) is SoL-Pi, and on a 51-task benchmark it held roughly 94% of the original harness's score while cutting recorded token traffic by nearly half (44.7–49.0%). It traded away a little score to save a lot of cost — and it never touched the model at all.
But RRSI throws cold water on this. A system that appears to keep improving as you iteratively patch the harness has, in many cases, simply memorized its training tasks. Scores climb sharply inside the training distribution, but take it to an out-of-distribution benchmark and the gains shrink or vanish. So the paper puts regularization on harness evolution. It permits only small changes at a time, rejects changes without justification, and penalizes changes that add complexity. In human terms: "don't just memorize the answer key — check whether you actually understood the principle by trying a different problem."
Enough theory. Here's our story.
This blog picks the latest papers each day and auto-publishes explainer posts. About a month after the whole thing — paper selection, writing, publishing — had been running without a human touch, we ran a retroactive fact-check on all 50 published posts using two models from different vendors. Since the model that wrote a piece can't see the beam in its own eye, the inspector was always a model from a different company.
The result stung. 40 of the 50 got flagged. These weren't catastrophic fabrications, but the details were off. A post that said there were 11 benchmarks when counting the table in the paper's body gives you 10. A post that flatly declared "the real cost is memory" when the paper actually named three bottlenecks — compute, storage, and bandwidth. A post that bundled three scores from three different experiments as if they were the achievement of one.
Digging into the cause, it was a harness problem. The writing prompt was only feeding in the paper's abstract. An abstract is a trailer. Write a movie review off just the trailer and you'll get the broad plot right but start getting the scene descriptions wrong. The model wasn't dumb — we'd given it too little to work with.
So we fixed the harness in three places.
First, we injected excerpts from the paper's full HTML into the writing stage. Instead of the trailer, we made it read the actual film.
Second, we set up a dual independent verification gate right before publishing. Two models from vendors different from the writing model each cross-check the piece against the source and render a verdict, and it only publishes if both pass. If either flags it, we incorporate the notes, revise, and verify again.
Third, if it still can't pass after revision, we hold publication and report to our admin channel. We chose to skip that cycle rather than push out a half-baked piece.
The very first publish run on the fixed harness paid off immediately. On the draft explainer for a new paper, the verification gate caught an "overstated measured range." The draft said both experimental tracks were measured out to five stages, but cross-checking the source showed the measurement range differed per track. The gate handed this note to the revision stage, and only after the corrected draft passed re-verification did it publish. A human never stepped in once.
We put the existing 40 posts through the same loop. We incorporated the notes, revised, and re-published only the pieces that passed re-verification. Pieces that still couldn't pass after two revisions came down. From a reader's point of view nothing looks different, but every post on this blog right now is a piece that has passed two external inspectors.
In that story, the loop matters as much as the harness. A verification gate isn't a door that renders one verdict and stops — it only works if you build it as a cycle that feeds failures back for another attempt.
There are four things we always put into a loop.
A termination condition. A loop must end. Our revision loop runs at most two rounds. If it still can't pass verification after two revisions, that piece is held. A loop without a termination condition isn't improvement — it's wandering. RRSI putting a cap on revision count and adoption criteria on harness evolution follows the same principle.
A fallback path. Decide in advance where to go when the primary path is blocked. Our writing stage automatically switches to a backup model when the primary model hits a limit. We've also seen what happens when you don't build this. An earlier pipeline once mistook a model's rate-limit notice for the body text and tried to publish it, coming up empty three days in a row. A fallback isn't a luxury — it's basic plumbing.
Resumability. A long batch job must be able to restart from the middle. The retroactive audit of 50 posts took several minutes per piece, so we logged the number of each processed post to a file and made re-runs pick up only what was left. If the structure re-runs from scratch because the network dropped at the 40th piece, that loop isn't fit for real use.
Recording failures and rolling back state. A failed cycle doesn't get erased — it gets logged, and the state is reverted. Our publishing worker claims a paper before it starts, and if publishing fails it releases the claim so the next cycle can retry that same paper. Failure must not lock up the system.
Contrast a weak loop and a strong loop, one line each:
The sentence got longer, but every new word blocks one specific failure scenario.
You don't need some grand framework. At its core, even our gate is a few dozen lines of code that "asks another model in a fixed verdict format, parses the answer, and branches." When starting, order it like this.
One warning. Per RRSI's lesson, don't be unconditionally happy that the gate pass rate is climbing. The system may just have learned how to look good to the inspector. Splitting our inspectors across two vendors and forcing a source cross-check is precisely to prevent overfitting to one inspector's taste.
ONTLE is a service that dispatches AI project managers, so the principles above are the product itself. The whole flow — intaking a customer request, processing it, producing deliverables, and reporting — is woven out of harnesses and loops. When a request stalls, a monitoring loop notices the holdup; deliverables go out only after passing verification; and a failed task returns to a retry path along with its record. This blog's publishing pipeline is a scaled-down version of that structure, and also a testbed where we try out new mechanisms first.
Models will keep getting better. But whichever model shows up, what you can reliably build with it ultimately comes down to whoever designs the harness and the loop. That's what we learned fixing those 40 posts this time.
A prompt is one component of the harness. Harness engineering designs the whole system — on top of the prompt, it covers control flow, tool wiring, memory, context management, verification gates, and failure paths. In our case too, the key to the fix wasn't the prompt wording but the input material (the full paper) and the verification structure.
Verifying with a model from the same family risks sharing the same blind spots. And tuning to a single inspector leads to overfitting to that inspector's tendencies — the very risk RRSI points out. We use a rule that a piece must pass the verdicts of two different-vendor models to publish.
Yes. The core gate is code at the level of "ask another model in a fixed verdict format and branch on the parsed response." You don't even need to go all the way to an automated revision loop at first. Just standing up a single pre-publish verdict starts filtering out the detail distortions that ran at 80% for us.
SoL-Pi ran harness improvement as an automated research loop and found mechanisms that hold the score at around 94% while cutting token traffic by 44.7–49.0%, and NeoHorse-1 laid out a path for using harness execution traces as material for the next round of training. That said, as RRSI shows, unbounded self-improvement overfits, so in practice you have to design in brakes — adoption criteria and revision limits — alongside it.