The Robot Already Knew How. We Just Didn't Know How to Tell It.

There's a humanoid—a human-shaped robot—that learned by watching people move. It pushes boxes, lifts them, carries them. It's even pretty good at pinching small objects with its fingertips. Then a new order comes in: "Tip that box over on its side." The robot freezes.

The strange part is that the robot can already do every motion tipping requires. Pushing, leaning, shifting weight—it has all the parts. What it's missing is the instruction sheet that says "string these motions together like this." The body was ready. The words weren't there.

A robot-learning method called InterEvolve turns the other way here. It doesn't make the body smarter. It makes the words you hand the body smarter. The already-trained body (the controller) stays frozen, and the task is recast as a "reward program"—a bundle of rules spelling out what counts as doing well—so that only the instruction sheet gets revised. Rather than sending a seasoned pro who's mastered the fundamentals back to school, you just hand them a new work order.

Freeze the Body, Change Only the Instructions

Let's be honest, though. The body wasn't fully ready out of the gate. Existing humanoid controllers watched only their own body, not the box in front of them. So "move the box left" and "move it right" collapsed into the same motion as far as the controller could tell. In tracking experiments, this body-only model almost always lost the box (8% success rate).

The fix wasn't to re-teach the whole body. The part in charge of the body stays frozen, and only a part that reads the target gets layered on top. This is where the term "residual" comes in—it means you don't rebuild what already exists; you learn only the difference and add it on top. The key is that this is a one-time job. Once you've built a body that can see the box, no amount of new tasks pouring in will make the body re-learn.

Now, with the finished body set aside, you edit only the instructions. A reward program has three pieces: what to reward, when to advance to the next stage, and what numbers to tune the two with. If you want to switch from lifting to pushing, you don't choreograph joint trajectories from scratch. You just adjust the reward for where to move the target and the stage conditions.

So who edits the instructions? Two loops interlock. In the outer loop, an LLM (a language model that works with human language) changes the program's structure—which rewards and stages to use. In the inner loop, a numerical optimizer tunes the numbers that go with that structure. Why split them? Because even a good structure fails when the numbers are wrong. So every proposal gets its numbers tuned before it ever reaches judging. It's a division of labor: creative decomposition goes to the language side, dogged number-fitting to the optimizer.

The judging criteria stay locked. Once a task is given in words, the success criteria (the verifier) are set once and fixed. This blocks the agent from quietly loosening the criteria to make the task easier. Replacement is cautious too. A new candidate competes against the current best across the same situations, and the top candidates are each re-evaluated three times. A swap is approved only when the improvement exceeds the run-to-run noise—the point is to refuse to credit a lucky one-off as real skill. Programs that survive this are summarized into a "skill library," so the next task doesn't start from a blank page.

Why the Success Rate More Than Doubled

Two metrics before you read the table. SR (success rate) is the fraction of trials where all of a task's criteria and physical constraints hold simultaneously in a single attempt. Fall over or miss the goal, and that attempt fails. Earned is the average fraction of criteria passed—but there's a trap. Just standing still passes nearly half the criteria on its own (see Inaction's Earned of 49.1% below). So Earned inflates easily. The real headline metric is SR. Below are averages across eight task families.

A reward a person writes naively tops out at 8% success. It means that even when the body holds the capability, a hand-written instruction sheet barely draws it out. Even an agent-written draft plus numerical tuning reaches only 34.6%—still under half. Only once "evolution"—editing the structure based on execution results—is added does it climb to 86.5%.

This leap from 34.6% to 86.5%—about 2.5x—is the paper in one line. What's worth noting isn't the size of the number. It's that a motion people couldn't easily think of was found jointly by the language model's proposals and evolution's selection. Rotating a box in place wasn't in the training data. The evolved program pulled it off by using the body in a new way. Some of the skills discovered this way in simulation were then performed by a real Unitree G1 robot on its own, using only cameras and sensors.

Underneath that "why" is a clear stance. What this work chased wasn't the ability to copy the right answer exactly, but the robustness to finish the job whatever the situation. A high success rate means that, even if it can't perfectly mimic a prescribed trajectory, it commits fewer catastrophic failures like falling over or losing the box entirely. The tracking-task numbers reveal this; the details are in the appendix. One editorial note: the more you send AI into the real world, the more this robustness matters than the decimal places of a benchmark score.

Where the Limits Are, and What It Means for Us

To avoid overstating things, let's draw the line first. The controller is fixed, and InterEvolve only re-surfaces capabilities the body already has. It can't create an ability that simply isn't in the body's repertoire of possible motions (its motor repertoire). For a task outside that range, you'd have to grow the body itself.

Let's also be clear about where evolution runs. Evolution happens in simulation. The real-robot results are a program picked there and replayed. "Done autonomously on real hardware" and "evolution happened on real hardware" are completely different claims, and the latter isn't true. That success is defined by the verifier is also a double-edged sword: who sets those criteria, and how, governs the interpretation. The objects handled were boxes and small items, too, so generalization to a wider range of shapes and materials can't be asserted from these results alone.

Here's why this design reads beyond robotics. The idea of "searching over the task's instruction sheet itself" touches the same nerve as the question we wrestle with at ONTLE: how far can you trust an AI agent?

Let's follow one scene. A customer asks ONTLE's AI project manager to "move next week's delivery date up." It arrives mid-project—something the original procedure never anticipated. Here, the one principle we share with InterEvolve is this: the language model proposes, but execution and judgment belong to a server-side deterministic layer the agent can't touch—a layer that processes strictly by set rules, with no room for improvised judgment. This blocks the agent from waving things away with "Done" or quietly loosening the bar for "complete" to suit itself.

For that, "finished" has to be written as conditions, not words. A delivery-date change is complete when, say, the related process schedule is updated to the new date, the materials' arrival date is set ahead of it, and the customer has accepted the change—all true at once. Until these conditions hold, no one can declare "complete."

But there's one difference between the robotics research and our own shop floor that can't be glossed over. InterEvolve runs candidate instruction sheets through a simulator countless times to verify in advance that "this way of handling it is better." There, "adopt only when the improvement beats the noise" holds—because you can repeat it. But each real customer request gets no such retry. A single execution is the actual outcome. So what we verify repeatedly is the way of handling things, while what we can do for a real request is confirm completion through the conditions above. These two have to be kept separate. Which leaves a plain question: is our "complete" really written as conditions a human can't touch?

Appendix: Notes on Numbers and Terms

A collection of the detailed numbers and terms left out of the main text.

Tracking task — low error doesn't equal high success. On the task of following a prescribed reference trajectory exactly, the dedicated tracker ULTRA has the lowest error (body 15.68cm, object 21.45cm, 66% success). Yet InterEvolve's controller comes close even without evolution, at 60% success (27.20/30.80cm), and once you add a tracking reward and evolve, it reaches 72% (19.80/24.92cm)—surpassing the dedicated tracker. ULTRA has the lower error, but InterEvolve has the higher success. The reward the agent discovered was "drift correction anchored to the target." The controller sees only coordinates around its own body (egocentric coordinates). So it slowly slides relative to the world at large (drift) without being able to see it itself. Once it used the box in front of it as an anchor to fill this blind spot, the final deviation of the torso anchor (the root) and the box dropped by nearly half (Figure 5 in the paper).

Ablations — which piece drove the results. Remove the target's state or the body-to-target distance signal, and the controller tracks its own body plausibly but rarely succeeds. Train the same network from scratch without freezing it, and it barely works. Conversely, the more you grow the residual on top of a frozen body, the better the scores. It amounts to numerical confirmation of the division of labor: the frozen part carries the fundamentals while the residual learns the interaction. It went as far as handling six boxes at once, and delicately pinching small objects with the fingers.

Cost — fully evolving one task family takes 2.1 hours on a single GPU and 0.23M LLM tokens. That it's larger than a fixed program (0.6 hours, 0.03M) is because it runs five rounds, tuning the numbers of several proposals each round. Capping the round count pins down the ceiling on cost clearly.

Terms

• Rollout: a single "attempt" in which the controller is run all the way through in simulation from a given initial condition. Both loops evaluate across many initial conditions in parallel, so every decision stands on many attempts rather than a single stroke of luck.
• FB (forward-backward) motor model: a device that, when fed "what to reward," pulls out "how to move." Because the behavior rule (the policy) is fixed as a single thing, a change in reward requires no re-training of the policy—only a fresh computation of the corresponding signal. This is why the body doesn't need re-training for each task.
• Latent prompt and the "bank": the latent prompt is the intermediate signal for turning a reward into behavior. After training and before evolution begins, a collection of "body-target state" samples (the bank) is drawn once from past runs mixing many tasks and targets, and then fixed. The current stage's reward is scored against these samples, and the latent prompt is formed as a weighted sum tilted toward higher reward. Because the reward reads context on the fly, this signal is recomputed at every control step.
• Demonstration discriminator: a device that judges whether the robot's behavior looks as natural as human demonstration data. It's used alongside residual training to steer toward human-like motion instead of contrived movement.
• Retarget: the work of refitting human motion-capture data onto the robot's body. They retargeted to a Unitree G1 (rubber hands) and an Inspire-hand version, applying OmniRetarget for the dexterity-demanding hands.
• Numerical optimizer (CMA-ES): an evolution-based search method that nudges a bundle of constants bit by bit to find better values (Covariance Matrix Adaptation Evolution Strategy).
• Budget: one task family is counted as five rounds, and each round uses 192 tuning rollouts per valid proposal plus 192 rollouts to confirm the top candidates.
• Data and bodies: pre-trained on whole-body manipulation data for large objects (OMOMO) and whole-body grasping data for small objects (GRAB). Four box types were used, and for each target 50 evaluation clips were held out with the remaining 3,866 used for training. All controllers were run in Isaac Lab.
• How it differs from other approaches: methods that use references, goal states, or skill labels directly request the desired motion/state/action name. The Eureka line has an LLM write the reward function but re-trains a policy for each candidate. Language to Rewards optimizes LLM-written rewards with MPC; MotionDisco evolves motion with an LLM and trajectory optimization and then trains a tracker separately; ROSETTA builds multi-stage reward programs from language preferences. InterEvolve parts ways with all of these by searching over the task instruction sheet itself while keeping the controller fixed.

FAQ

References

• Paper: InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation
• Authors: Zhuo Lin, Sirui Xu, Liuyu Bian, Yu-Xiong Wang, Liang-Yan Gui (University of Illinois Urbana-Champaign)
• arXiv: https://arxiv.org/abs/2610.02196
• HuggingFace: https://huggingface.co/papers/2610.02196