SoL-Pi is a study that automatically improves the "harness" itself — the execution scaffold a coding agent uses to call tools and manage context — that wraps around a coding agent. Instead of hand-tuning it, the authors repeatedly scaled an automated research loop across many environments and kept only the improvements that held up. The result: on EdgeBench, an evaluation of 51 tasks, recorded token traffic dropped by 44.7–49.0% and API cost fell by about a third. This doesn't come for free, though — the average score came in roughly 5–6% lower than the existing strong harness (Pi) (details in the "Results and what they mean" section below).
Earlier coding AI was mostly an "autocomplete" assistant. You typed code and it predicted the next line, so one prediction amounted to one unit of work. Recent coding agents are a different animal entirely. Without someone watching over them, they carry out unattended, around-the-clock exploration of a codebase — reasoning, using tools, and taking the feedback from execution results to judge again.
The catch is that an agent's work has stretched from "a single prediction" into "a long, continuous trajectory." Reasoning → tool use → feedback → reasoning again chains out over dozens or hundreds of steps. To use a cooking analogy: where the old job was "hand me the next ingredient," now you're handing off the whole thing — shopping, cooking, tasting, cooking again. Across that long process, the text (tokens) flowing back and forth explodes.
That's why token efficiency becomes the key bottleneck. In particular, to scale up Recursive Self-Improvement (RSI) — an agent iteratively improving itself — in a big way, the cost of one loop has to be low enough to run it many times. If each pass is expensive, you simply can't run the self-improvement loop often.
The limitation of prior approaches is that the focus of improvement sat mostly on the "model" side. This paper works not on the model but on the harness layer that wraps around it. The view is that, even with the same model, a well-built harness can do the same job with far fewer tokens.
In one line, the method is this: "Let an automated research loop, not a human, discover the harness on its own."
The approach draws inspiration from RSI but aims it at the harness layer. The authors scale an auto-research loop across increasingly numerous and diverse environments, actually running harness candidates (harness rollouts). The "diverse environments" part matters. A trick that only works in one situation becomes useless the moment you step outside it.
As an analogy, rather than training one cook in a single kitchen, you send them into many different kitchens to work under all sorts of conditions, then keep only the "habits that work in any kitchen." Gimmicks that only pay off in one kitchen get filtered out, and only reusable improvements survive. The paper says the improvements obtained this way transfer beyond their development setting, pushing automated harness discovery closer to production-level outcomes.
Four mechanisms survive this selection process to the end and make up SoL-Pi. The abstract lays out each axis as follows.
| Mechanism | Area it covers | Plain-language gloss |
|---|---|---|
| Action execution | Executing actions | How the agent actually runs tools and commands |
| Context compaction | Compressing context | How a lengthening conversation/history is squeezed down to save tokens |
| Observation handling | Handling observations (results) | How observations such as tool-execution results are dealt with |
| Delegated reading | Delegated reading | How read work is split off and delegated |
One caveat here. The abstract states what domain each of these four axes covers, but it does not describe how each mechanism's implementation actually works in detail. The "plain-language gloss" column above is a conceptual paraphrase inferred naturally from the axis names; the exact algorithmic details have to be checked against the full paper.
Intuitively, all four axes are different facets of a single question: "where, in a long trajectory, can you stop tokens from leaking?" Execution style, history compression, result handling, read delegation — these are the classic chokepoints where tokens balloon when an agent works for a long time.
The core evaluation was done on the 51-task EdgeBench. The comparison baseline is Pi, an existing strong harness. The models used were GPT-5.6 Sol and Opus 5.
The gains and their price are as follows.
To sum up the significance, the core message is: "give up about 5–6% of the score in exchange for a large cut in tokens and cost." It isn't that there's no performance loss at all, but the claim is that the margin is small enough to be acceptable as the price of the savings. What's especially striking is that the savings are expressed on an hourly basis. If you assume an agent running nonstop through the night, hourly savings accumulate into a structural difference in operating cost. Multiply by 24 hours a day and a full month, and it becomes clear why harness-layer efficiency is a precondition for scaling up.
Another point worth noting is the transfer of the improvements. Because these mechanisms survived across many different environments, the claim is that they get reused in other situations beyond the specific setting they were developed in. That's closer to "the improvements you discover remain as assets and keep getting used" than to "tune it well once and you're done."
To be honest about it, there are several things the abstract alone can't settle.
In short, the direction and the skeleton of the results are clear, but judging reproducibility and generalization requires evidence from the body of the paper beyond the abstract.
ONTLE is a "dispatched AI project manager SaaS." Because an AI PM watches over projects, uses tools, and makes judgments on its own, autonomously and over long stretches in place of a human, its problem space overlaps exactly with the "long-running, unsupervised agent" this paper deals with.
Three takeaways.
The harness is the cost structure. For a product like ONTLE, where an AI stays resident and keeps working, there's room to cut cost significantly — without changing the model — just by reworking the execution scaffold that wraps the agent (how context is compacted, how observations are handled, how reading is delegated). The result of nearly halving tokens while giving up only a small (about 5–6%) score connects directly to operating margin and pricing competitiveness. That said, whether the performance drop you accept here stays within our own service quality bar has to be verified separately.
Context management sets the ceiling on autonomy. The longer an AI PM works, the more the conversation, history, and tool results pile up and the harder tokens explode. Among SoL-Pi's four axes, context compaction and observation handling are especially important design points for a product that bolts on knowledge via RAG and deals with long project histories. What you keep and what you squeeze down is exactly what determines how long you can keep running autonomously.
The "automatic discovery of improvements" methodology. Rather than having a human tune the harness piece by piece, the approach of running it across diverse real environments and adopting only what survives maps directly onto operating an AI PM that gets dispatched to many projects. In effect, you keep only the operating habits that work across many sites as the standard, and filter out the gimmicks that fit only one site.
That said, adopting it means carrying the above limitations with us. Our domain differs from EdgeBench, so the right order is to measure both the savings and the accompanying performance drop together, at small scale, on actual ONTLE workloads. The abstract's figures are "grounds for an expectation," not "a guarantee for our environment."
No. The focus of this study is not the model but the harness layer wrapping around it. Using the same models (GPT-5.6 Sol and Opus 5 in the abstract), it improved the execution scaffold to cut tokens and cost.
It does fall. The abstract describes the performance as "comparable," but the body tables (§3.1) show the average score dropping from 44.833→42.003 on GPT-5.6 Sol (about a 6.3% drop) and from 44.756→42.224 on Opus 5 (about a 5.7% drop). It should be understood not as "no performance loss" but as "savings bought at the cost of a small score drop."
Action execution, context compaction, observation handling, and delegated reading. The abstract states only the area each axis covers; the concrete implementation has to be read in the body of the paper.
The abstract says API cost was cut by about a third, and presents the hourly savings as estimated at roughly $8.75–13.50 versus native Codex and Claude Code, and about $4.36–5.71 versus Pi. These are estimates that can shift with the comparison baseline and usage patterns.