MILO is a framework that automatically discovers an AI agent's "harness" instead of leaving it to be hand-tuned by people. Going one step further, it also evolves the search strategy used to find that harness—and that is the paper's central idea. On three long-horizon benchmarks—Terminal-Bench 2.1, PaperBench, and DeepSWE—the harness MILO discovered beat all 8 well-engineered harnesses and all 6 search methods it was compared against.
A modern AI agent can be split into two parts. One is the model (the LLM) itself; the other is the harness that wraps around it. The harness is the software layer that actually executes the actions the model proposes in a real environment, and that manages the scaffolding—prompts, tools, context, memory—along with the control flow. Whether to spin up a sub-agent, intercept a tool call, verify a result, or decide when to stop: all of that is the harness's job.
Here's the interesting part: the exact same model scores very differently depending on the harness. Nobody swaps the model; only the scaffolding changes—and both the solve rate and the token cost that comes with it can shift substantially. That gap is the whole motivation for automating harness design.
And that's where the trouble starts. Building a good harness is still close to artisanal handwork. Commercial harnesses like Claude Code or Codex take teams months to build—engineers pore over failure cases, tweak heuristics, and iterate on a handful of designs over and over. Worse, harnesses are model-specific. A harness tuned for one model often doesn't suit another, so every time the model changes you're back to tuning from scratch. The idea behind automated harness discovery (AHD) is to automate exactly this grind.
AHD is usually framed as an LLM-driven evolutionary search: an LLM mutator proposes a new harness, a scorer grades it, and the result steers the next round—an iterative loop. But the paper points to four weaknesses in existing approaches.
The insight behind MILO (Meta-evolutionary Island Orchestration) is this: don't just evolve the harness—evolve the way you find the harness, too. It's easiest to grasp as an organization running several independent research teams.
First, the island structure. Each island starts from a different initial harness (a seed) and grows in isolation. Each island has its own dedicated mutator agent, so that mutator's particular biases stay boxed inside that one island. Islands share successful ideas with one another, but they don't compete for survival. Thanks to this design, the best harness can actually emerge from what looked like the weakest seed—you don't disband a team just because its early scores are bad, so slow-blooming directions get to survive.
Second, hierarchical lineage memory. The entire path from seed to candidate is recorded as a tree, including the candidates that were rejected. Successful edits tell you what worked; failed edits prune that direction and push the search toward bolder structural changes instead. Using failure as negative evidence is the clever bit: when you've burned hours on a scoring run that went nowhere, you at least recoup the cost as "don't go there" information.
Third, the mutator agent doesn't just patch a few lines of prompt—it rewrites the entire harness. In doing so it consults the global search history, the parent harness's failure traces, and feedback about that parent's specific weaknesses.
Fourth, an orchestrator agent coordinates strategy in the outer loop. When an island stalls, it diagnoses that island, swaps out the mutator, adjusts the memory, and rewrites the curriculum (which tasks to evaluate against). Grafting lineages together and splitting them into branches (speciation) are also the orchestrator's calls. In other words, each team researches its own way, while the lab director re-engineers the very approach of any team that has ground to a halt.
The whole thing runs as an inner loop and an outer loop. In the inner loop, a parent is selected from an island's passing candidates, the mutator rewrites the harness wholesale using the failure traces and lineage memory, and the resulting child is scored. A child is accepted only if it advances the Pareto frontier across three axes—accuracy, tokens, and latency. Concretely (§4.2, Eq. 7), acceptance is defined by non-dominance plus a hypervolume increase: the child must not be dominated by the existing candidates and must enlarge the frontier. One candidate dominates another only when it is no worse on every axis and strictly better on at least one—dominance is a condition over all axes at once, not a single-axis comparison. So a child isn't disqualified simply for costing more: a more expensive child can still be accepted if its higher accuracy keeps it off the dominated set, and a cheaper one can be accepted when it holds accuracy while spending less. When progress stalls, the orchestrator in the outer loop steps in.
Performance was measured on two kinds of backbone: Opus 4.8 (a frontier model) and gpt-oss-120b (an open-weight model). The figures below are for Opus 4.8.
| Benchmark | Gain over MILO's initial harness | Best gain from prior search methods |
|---|---|---|
| Terminal-Bench 2.1 | +12.0% | +4.5% (GEPA) |
| PaperBench | +28.3% | +18.3% (Meta-Harness) |
| DeepSWE | +10.3% | 0% (no gain) |
MILO beat the best prior search method on all three benchmarks. On DeepSWE especially, prior methods couldn't take a single step past the initial harness, while MILO pushed it up by 10.3%.
The absolute scores matter too. On Terminal-Bench 2.1, MILO scored 86.1 ± 2.0%, edging past the official leaderboard's No. 1 (83.8 ± 2.3%)—and it did so while using 26% fewer tokens than the initial harness. Accuracy and cost improved at the same time, which reads as evidence that filtering candidates on a Pareto multi-objective criterion actually worked.
MILO isn't limited to strategy-level discovery like finding harnesses; it can also drive instance-level search that digs deep into a single problem. The paper points to cases where it tightened the best known upper bounds on unsolved math problems in EinsteinArena. It lowered the Erdős minimum-overlap bound from 0.3808586 to 0.3808568, the first autocorrelation inequality from 1.50274365 to 1.50274360, and the third autocorrelation inequality from 1.45081 to 1.44889. The numbers are tiny, but the point is that it actually set new records on open problems.
To be honest, there are things worth flagging. First, cost. The whole setup runs multiple islands plus an orchestrator on top of the premise that scoring one harness takes hours, so the total compute to run MILO is far from trivial. When the paper talks about token efficiency (26% savings over the initial harness), that's the operating cost of the discovered harness—it needs to be kept separate from the cost of the MILO search itself.
Second, the scope of the results. The figures above come from specific benchmarks and specific models (Opus 4.8, gpt-oss-120b). Given that a harness is the kind of thing that has to be re-tuned per model in the first place, it's hard to conclude from the abstract alone that the same size of gains would reproduce on other models or other task distributions.
Third, the math records are, in raw numerical terms, very small. They're convincing as a demonstration of the method's generality, but that's a different story from the magnitude of practical impact.
Fourth, reproducibility and operational stability. Since both the mutator and the orchestrator are agents (LLMs), how stably the search converges, and what happens on an unlucky run, are things you'd have to read the body of the paper more closely to judge.
ONTLE works by dispatching an AI PM to each customer project, managing everything from intake to service launch. The way that PM operates—in what order it takes in, reviews, and advances requests, which tools it uses when, and when it stops and hands a decision back to a human—is, in effect, a bundle of rules that amounts to a harness.
Let's follow one scenario all the way through. Say a new customer comes in and hands us an integration job of a completely different character from anything our past customers needed. Under today's approach, the assigned person drops the existing PM flow onto it as-is and runs it, and only after watching the PM stall somewhere odd or burn through too many tokens do they hand-edit the prompts and branching rules. That fix is a patch tailored to that one customer and that one model, so for the next customer it's back to square one.
Drop a MILO-style approach into this scene and three things change. First, you run several variants of the PM flow concurrently, like isolated islands—and instead of keeping only the one that's pulling ahead, you also keep the variants that settle in slowly. Next, instead of deleting a failed flow, you leave a failure trace in memory—something like "the customer asked back three times at this branch"—so the next mutation steers clear of the same trap. Finally, if your acceptance criterion is a Pareto trade-off of "solve rate, tokens, response latency" rather than accuracy alone, then a flow earns its place whenever no surviving flow beats it across all three axes at once—so a cheaper, faster flow that holds the solve rate survives, and so does a flow that clearly improves the solve rate even if it costs more. Being expensive, in other words, is not grounds for automatic elimination. ONTLE's model routing already leans toward weighing cost against performance at each stage—chat, intake, execution, revision—so evolving the harness itself on top of that same scale could be the next step. The picture is one where the flow-tuning a human used to repeat for every customer gets handed off to a search loop that remembers its failures.
No. A prompt is just one part of a harness. On top of prompts, a harness includes scaffolding like tools, context, and memory, plus control flow such as spawning sub-agents, intercepting tool calls, verifying results, and deciding when to stop. The paper argues that prior methods that only touch prompts or skills (GEPA, A-Evolve, and so on) miss most of the search space, whereas MILO rewrites the entire harness.
It's designed for exactly the opposite effect. If islands compete for survival, everything converges quickly onto the few that score well right now, and diversity collapses. By growing each island in isolation and exchanging only successful ideas, each mutator's exploitation bias stays trapped in one island and search diversity is preserved. The paper reports a case where, thanks to this, the best harness came out of the weakest seed.
They're a case showing that MILO's search framework applies not only at the strategy level (harnesses) but also to single-problem optimization (instance-level). Tightening the best known upper bounds on unsolved problems in EinsteinArena (for example, Erdős minimum-overlap from 0.3808586 to 0.3808568) suggests the same evolutionary structure generalizes to other domains.
The principle is portable, but you have to weigh the cost first. To score a single harness properly you have to run it against a real bundle of tasks, and that takes time and compute. The token savings the paper cites (26% over the initial harness) is the operating cost of the discovered harness, not the cost of the search itself. A realistic approach is to start with a small set of tasks and a clear multi-objective criterion (accuracy, tokens, latency), then scale up the search gradually.