You can think of an AI agent as two parts. One is the model that plays the role of the brain — the large language model (LLM) that understands what you say and produces text. The other is everything that lets that brain actually get work done: how the prompts are composed, in what order the steps are taken, which tools get called and when, and what gets remembered. Bundle all of that together and you get what's called the harness.
Here's one interesting fact. Leave the brain untouched and just refine the harness, and the very same model gets noticeably smarter. So lately there have been attempts to let the agent fix the way it handles that harness all by itself.
The catch is that when you let it loose like this, the agent often comes back as a "smart fool." Its scores on the practice problems climb and climb. But the moment it runs into a new problem that wasn't in the practice set, its ability drops off a cliff. It didn't learn the underlying principles — it memorized the answer key. In machine learning, this is called overfitting. Google's RRSI research puts a leash on exactly this overfitting. The result: performance held up even on previously unseen problems, and as a bonus, costs went down too.
What does "fixing your own harness" actually mean? The agent keeps proposing candidates that tweak its own harness bit by bit, picks the good ones, and folds them in — over and over. This is recursive self-improvement, or RSI for short. The brain (the model) is never touched. Only the know-how for using that brain grows on its own.
Overfitting happens inside this automatic evolution. Within the tasks used for evolution, scores jump dramatically. We call tasks of the same character as those evolution tasks "in-distribution," and new tasks of a different character "out-of-distribution." When the skill gained in-distribution carries over to out-of-distribution tasks, it shrinks or disappears entirely. The agent has picked up tricks that only work on this particular test, not fundamentals that work anywhere.
The heart of RRSI (Regularized Recursive Self-Improvement) is planting "regularization" inside the self-improvement process itself. Regularization is a long-standing machine learning technique for keeping a model from memorizing its training data — and here, that same idea is carried straight into the process by which the harness evolves.
Evolution runs as a back-and-forth between two roles: a proposer that brings new changes, and a selector that decides which of them to keep. RRSI regularizes both.
Start with the proposer. Its most important mechanism is an edit budget that shrinks over time. There's a cap on how many edits can be made at once, and as evolution proceeds, that cap is progressively tightened. The approach is borrowed from annealing — heating metal red-hot and then slowly cooling it to make it hard. The logic goes like this. Early on, you leave room for big changes so the agent explores the solution space broadly (the hot state). Then you gradually narrow the scope of change so it converges stably onto one good solution (the slowly cooling state). If you let it rip out ten places at once from start to finish, you can't tell what actually worked, and changes that happened to fit the practice tasks by coincidence slip in easily. That's why it's pushed toward small, careful edits the further along it gets.
There's one more piece. By consulting the evolution history so far, directions that have already been tried many times are pushed to the back, and changes not yet attempted are brought to the front. This is a safeguard against digging deeper and deeper in one direction only to end up snugly fitted to the practice tasks.
Now the selector, which weighs each incoming proposal on more than one axis (paper §3.3). The main check is a generality screen — but it works by hunting for leakage, not by foreseeing the future. It inspects whether a proposal smuggled in task-specific shortcuts: a hard-coded task name, a memorized gold answer, logic tailored to one benchmark. Edits that lean on those telltale signals get caught here. In other words, it is a check for leakage in the proposal itself, not a device that predicts in advance how a proposal will perform on unknown tasks. Alongside this, the selection also weighs evaluation noise — how much of a reported gain is real rather than measurement jitter — and the proposal's performance against its token cost. And among changes already folded in, an efficiency pass periodically cuts the ones that are too trivial, too costly, or no longer useful.
Here's an example. Say the proposer puts forward this edit: "When solving a coding task, don't dump in the whole file — pick out only the relevant functions." The generality screen asks: did this edit smuggle in a shortcut tied to one specific task — say, hard-coding which functions to grab for a problem it has already seen — or is it a general rule that applies to any coding task? If it carries no such leakage, it clears the check. Later, if that edit only drives up costs while its benefit fades, the efficiency pass strips it out. Keep cycling through new attempts, a leakage check, and an efficiency check, and evolution naturally comes to prefer reusable, general-purpose mechanisms over test-specific tricks. The fact that pruning keeps the harness lean and uses fewer tokens is a practical bonus.
RRSI was validated across eight benchmarks spanning coding, agentic workspaces, and engineering design. Here are the headline results from each setting, broken out. One term before you read the table: a token is the smallest unit AI handles when reading and writing text (roughly a word fragment), and policy tokens specifically are the tokens an agent spends deciding what it will actually "do."
| Where the gain came from | Reported figure |
|---|---|
| Tasks used for evolution (in-distribution) — coding task family | Up to +14.1 points |
| 5 out-of-distribution benchmarks — agentic workspace (JobBench) | Up to +4.7 points |
| Token efficiency — vs. unregularized evolution | 30% fewer policy tokens |
These three numbers are each a best result from a different task and setup. So it would be a mistake to line them up in one sentence as "it preserved this much while gaining that much." The in-distribution gain and the out-of-distribution gain come from different tasks to begin with, so they aren't things you can compare side by side.
Even so, taken together the direction is clear. Cut overfitting with regularization, and the agent improved a lot on practice tasks, kept some of that improvement on unseen tasks, and actually used fewer tokens. Raising scores, generalizing to new problems, and cutting costs are not mutually exclusive. That is this paper's message.
That said, it's worth reading honestly that the out-of-distribution gain, even at its "maximum," is nowhere near the in-distribution one. It's a signal that regularization reduces overfitting but doesn't eliminate it entirely. Since the two figures come from different tasks you can't draw firm conclusions, but it's also no basis for the overblown expectation that "we regularized, so overfitting is gone."
Good results — but before putting the idea into a product, you need to look at what concrete challenges it turns into in the field.
The strength of the regularization is ultimately something a human tunes. The annealing budget, the leakage bar, the pruning threshold — these are dials a person turns. Crank them too tight and useful changes get cut away too; leave them too loose and overfitting comes right back. A backbone dependency also remains. The paper checked whether the gains transfer to a separate backbone not involved in evolution, but it only goes as far as validation between a handful of backbones. Whether the same gains hold on a wider variety of models is still open. One more thing: the very judgment the critic makes — clean edit versus leakage-tainted one — can become yet another bias. How accurately the critic draws that line is itself subject to validation.
It's precisely because of these limits that in a field like ours at ONTLE, we tame the idea rather than transplant it as-is. ONTLE assigns each client company its own AI project manager (PM) agent — the agent in charge of taking in a request and driving it through analysis, development, and delivery. What we sell isn't the model itself; it's a harness well-assembled to handle a client's repetitive work. So "field-grade overfitting" isn't someone else's problem.
There's an important distinction here. A PM agent's harness splits into two kinds of capability. One is shared capability used by many clients together — things like the flow of taking in a request, analyzing it, and breaking it into tasks. The other is capability tailored to one specific client. Mix the two up and your judgment gets muddy.
Let's walk through a concrete scene. Company A requests, "Write our weekly report automatically, formatted to our template." So we refine report-writing logic specific to Company A. This is a tailored capability. The edit isn't bad just because it doesn't work for other clients — it was for Company A alone in the first place. The problem is when that logic fits only precariously to Company A's particular table format. If the report falls apart the moment Company A tweaks its template next quarter, that's overfitting even within Company A. So for a tailored capability, we run it against other Company A cases that weren't used for tuning — a different week, a different department's report, say — and check whether quality and cost hold up. It's the same thinking as the paper verifying performance out-of-distribution.
Conversely, an edit like "pull in only the relevant materials when writing a report" is on the shared-capability side. This one shouldn't be validated on Company A's cases alone. Only when it holds up attached to Company B's and Company C's work does it earn the right to be promoted to a shared capability. Every time we attach a new automation, we first sort "is this for one client, or a fundamental for everyone?" and set up a gate that validates it against the matching cases.
Finally, logic and context that only pile up and become useless get pruned periodically. In a staff-augmentation service where tokens are operating cost, this pruning is what protects quality and margin at the same time.
Of course, this is a direction to apply, not a guarantee. The limits — that a human has to tune the regularization strength, and that the critic's line-drawing can be another bias — carry over to the field intact. How much to leave to automatic review and where a human steps in is something to calibrate with each client.
RRSI's implications are bigger than a single paper. To an industry prone to getting swallowed up in benchmark-score races, it points out that setting the direction of growth matters as much as the speed of growth. A higher score doesn't mean a more useful agent. Only when an agent grows smarter in a way that doesn't collapse in front of unfamiliar work and gets lighter the more it's used does that smartness finally earn its keep in the field. What a self-improving agent needs may be not a stronger engine, but a proper leash.
No. RRSI keeps the brain (the backbone model) fixed and evolves only the harness on top of it, so it isn't tied to any particular model. The paper checked whether the gains transfer to a separate backbone not used in evolution. But it only goes as far as validation between a handful of backbones, so it's too early to declare that it works the same on any model.
The code and a project page are public, so follow-up verification and reproduction are possible. The paper is available on arXiv and HuggingFace (see references below). Being public means there's room to check the numbers rather than take them on faith, which adds to its credibility.
If anything, the appeal is stronger for the cost-sensitive side. RRSI uses pruning to keep the harness lean, cutting policy tokens by 30% versus unregularized evolution — and in a service where tokens are operating cost, that translates straight into margin. That said, it does take effort for a human to tune the regularization strength (the budget, the bar, the thresholds), so you should factor in the capacity for that calibration.