The heart of this paper is a division of labor. The AI (a coding agent) writes the code that drives the robot during a development phase, and the robot simply follows that code during the execution phase. There is no neural network inside the execution loop. Instead, code reads the state of the robot, the environment, and the task directly from camera images and joint sensors, stores those readings in variables, and then decides every action through code logic based on nothing but those variables.
For this to work, one precondition is required: values like the 3D shape of each object, the camera position, the robot's geometry, and the table height must be given in advance and never change. The benchmark publishes these values, so the code only has to figure out "which object is where right now" from the image.
The authors call this approach Code-Only-as-Policy (COAP), and report a 70.24% success rate with no model at execution time across the 42 bimanual manipulation tasks in RoboDojo.
Today's mainstream robot policies share one assumption: "keep a model inside the control loop." A VLA (Vision-Language-Action, a neural network that takes images and language and outputs actions) maps observations straight to actions. An agent harness (a structure that manages model calls and tool execution) asks a vision-language model (VLM, a model that understands images and text together) "what should I do next?" at every moment.
The discomfort of this design shows up in three places at once. First, you can't see what the model looked at to pick that action. A VLA's decision is buried in its weights, and even a harness's model, while it may surface reasoning text, offers no real control over the actual selection process. Second, calling a model at every step is slow and expensive, and even identical inputs come back with randomness mixed in from call to call. Third, learning a new task means collecting data and retraining all over again, and teaching one more task often chips away at performance on earlier ones—so-called forgetting.
Start with a simple scene: picking up a single cup. COAP writes down two things in code. One is the state variables—the cup's position and orientation, both arms' joint angles, and which step it's currently on. The other is the action rules—"move the fingertip to the cup's position, close the gripper, lift." At execution time the robot reads the cup's position from the image to fill the state variables, and the rules pick the next action by looking at those variables. If the cup sits somewhere else, the rules stay the same. All that changes is the variable values.
This distinction is what sets COAP apart from other approaches. If you view a policy as "a function from state to action," the question is how exposed that function is. In a VLA, the function is hidden inside weights. An agent harness exposes part of it as text memory, but the core model is still a black box. COAP pulls the entire function out into code. State lives in code variables, rules live in code logic. Both are visible, so no model is needed at execution time.
The authors liken this structure to a Turing machine. The changing state corresponds to the tape where symbols are written down, and the fixed rules correspond to the machine's transition rules that read and write the tape. Swap out the tape and the same rules can process new input—just as the same code runs unchanged in a new situation where only the object layout has changed. They name it the Embodied Turing Machine.
State splits into three parts, and its shape is the same no matter the task.
Perception, too, is not some special neural network but ordinary code written with NumPy and OpenCV. It runs in roughly three steps.
The camera position, robot geometry, object shapes, and table height are constants that don't change from episode to episode—part of the rules. What changes is only "which object is where," and that's the state. Tasks requiring millimeter precision take one more measurement with the wrist camera just before grasping.
Because every task uses the same shape of state, a module written for one task is reused as-is by another. The code is a layered structure that depends only downward. Task code calls per-object code, which calls manipulation code like pick, place, push, and pour, which in turn calls perception and motion code. The upper layers decide "what" (ordering, task-specific judgments, recovery conditions), and the lower layers handle "how." As a result, 83% of the code a new task runs comes from the shared library.
A new task can extend the library in three ways.
The third matters because of compatibility. For an existing task that doesn't pass the new argument, a condition ensures its execution trajectory stays identical within the range of inputs covered by unit tests. That said, since this condition is limited to the tested input range, it does not fully guarantee that an extension will never break an existing task. The design intent is to narrow the scope so that only tasks using the new argument change.
The idea of writing a policy in code is not itself new. Lining it up against the lineage the paper surveys makes the difference clear. The Code-as-Policies / ProgPrompt / VoxPoser family has a model write code that calls perception and control APIs. PyRUA-Lean has the model write a bit of code while the episode runs, and Skill2Real assembles a fresh program from stored skills at every decision step. They share two traits: the model writes a program during execution, and the state is supplied by an external, learned perception model.
There are also approaches that write a program just once offline—RHO and EmbodiedSWE—but here too the state comes from outside the code (RHO from a learned perception model, EmbodiedSWE from the ground-truth values the simulator provides). These perception models are black boxes, so the code can't control their uncertainty.
COAP's difference is that the code measures the state itself. If a measurement is wrong, the coding agent fixes the measurement code, and the state grows more accurate over time. RoboRSI, a self-improvement line of work on the robot side, puts a "block it if performance drops" gate on a loop of agents with different roles, but decisions during execution are still made by a model. Even a change that passes the gate merely steers a black box, so the behavior that was tested and the behavior that actually executes can diverge. Because COAP's state is explicit code, "the behavior you tested is the behavior that executes," and you fix the policy the way you fix software.
The benchmark is RoboDojo. Its 42 bimanual tasks span five axes: Memory (6), Precision (8), Open instruction (8), Generalization (12), and Long-Horizon (8). Each task is evaluated over 50 layouts with three seeds. For Generalization tasks, half the layouts are randomized, bringing in new objects, new room and table appearances, and up to 20 distractors. All numbers are produced by the official evaluation client, judge, and summary scripts.
The policy's input is three RGB camera feeds (one head + two wrist), joint-sensor values such as joint angles, end-effector pose, and gripper command, plus the instruction text. Success rate (SR) is the fraction of episodes where the whole task succeeded, and Score is the benchmark's progress score. What this design measures is "how well the policy maintains and measures state, and acts accordingly." The Memory and Precision tasks in particular target state maintenance and measurement accuracy directly.
One thing must be made clear here. COAP's code library is developed offline. A Claude Opus 5.5 coding agent writes and debugs code inside a closed self-improvement loop with no human intervention. It runs under fixed rules like "no tricks that only work in a single episode" and "no referencing ground-truth state at test time." Development does not use the benchmark's test set, and the finished code runs as-is across all episodes, including the test layouts.
The improvement loop turns in four steps.
This adoption rule filters out large regressions on the validation set, so improvements mostly accumulate. That said, as Appendix A.2 notes, the regression check only excludes large drops and does not catch every change, so it does not guarantee that each step causes no performance loss at all.
In Table 2, COAP's overall success rate is 70.24% and its progress score is 75.45. Among the comparison baselines, the one with the highest overall success rate was PhysicalRSI (agent + VLA), an agent-harness-family method, at 31.38%. The gap between the two is about 38.9 percentage points. The "38.9% over state-of-the-art" cited in the abstract and body refers to this overall success-rate gap.
Broken down by dimension, the success rate and the "gap over the best baseline" are different numbers. Separating the two gives this:
| Dimension | COAP success rate | Gap over best baseline |
|---|---|---|
| Memory | 89.9 | +39.6 |
| Precision | 75.1 | +41.6 |
| Open | 64.1 | +33.1 |
| Generalization | 65.7 | +35.2 |
| Long-Horizon | 56.4 | +13.0 |
A caveat: the "best baseline" on each row may be a different method from row to row. Don't lump it together as having beaten one competitor on every front.
Here's what the experiments showed. The dimension with the largest gap is Precision (+41.6), followed by Memory (+39.6). Memory tasks are solved only by "carrying information seen earlier through to later steps." COAP writes that information straight into task state variables, so a fact from an earlier step doesn't vanish before the next decision. By contrast, Long-Horizon has the narrowest gap at +13.0.
To add my own hypothesis here: it seems to be because, the longer a task runs, the more measurement errors and recovery failures pile up at each step. One wrong position estimate throws off the next action, and that deviation accumulates into later steps. In other words, the longer the task, the more the code logic has to withstand this accumulation. That said, this is an interpretation drawn from the gap pattern, not a conclusion the numbers prove directly.
To sum up, this result shows that model calls during execution can be eliminated. The development cost and the viability of real-world deployment must be assessed separately, since this achievement came entirely from a coding agent polishing per-task logic offline. The authors suggest that because this code isn't tied to any specific episode, a single policy can reliably produce successful trajectories across many episodes—trajectories that could serve as training data for VLAs and agent harnesses.
The weakness the authors admit outright is generalization. They state plainly that COAP is weakest mainly at generalization, because it requires a small amount of per-task code logic (adding that agent-harness-family methods also use task-level memory, and some rely on episode-level logic). And they specify that COAP's ceiling ultimately rests on "how accurately state is represented and how robust the code logic is." Meaning: if the measurement is wrong, the decision is wrong.
Adding my own observation: the preconditions under which this method works are fairly strong. The camera position, robot geometry, object shapes, and table height must be given in advance and must not change, so that the perception code can recover poses via virtual rays and shape-matching. The benchmark publishes these values, but robots in the field often don't know an object's shape. How well the assumption that "state can be measured accurately" holds up against lighting, occlusion, and never-before-seen objects is also hard to settle from this scope alone. The more you move toward contact-rich manipulation of deformable shapes, the looser this assumption may become.
When carrying COAP's lesson over to business automation, there's a boundary to draw first. Values like an assignee or a due date can be read cleanly by code, but the intent packed into a message requires interpretation. Not everything can be moved into code.
Take one narrow case: "find tasks with an empty assignee and a near due date, and reassign them." Here, "assignee vacancy" and "time left until the deadline" can be read precisely from tickets and schedules by code and stored in variables. Then a rule like "deadline near + assignee empty → send a reassignment alert" can be hardened into code, so there's no need to ask a model "who should do what" every time. Conversely, a judgment like "does this reassignment fit the team's current situation" requires reading a message's context, so it stays the model's job.
The grain of the "reducing repetitive work" that ONTLE aims for lives right here. Measurable, repeated decisions settle down into verified code, while expensive model inference is spent only on the boundary that code can't yet handle. To be clear, though, this is a direction arrived at by viewing COAP's demonstrated principle through ONTLE's design lens—not a claim that it's implemented this way today. The state of natural-language work isn't measured as cleanly as a robot's object pose, so the realistic starting point isn't "all code" but a hybrid that hardens the measurable parts into code first.
In the end, the practical criterion converges on one thing: which states and which decisions to move into code. Push the things that are measured clearly and happen repeatedly down into code first, and leave the rest to the model.
You have to check the preconditions first. COAP runs on the condition that values like object shapes, camera position, and robot geometry are published in advance. Robots in the field often don't know such shapes. So rather than porting the whole thing, it's more realistic to harden the repetitive tasks whose state you can measure cleanly into code first, and leave the rest to the model.
At execution (test) time, no model like a VLA or VLM is needed. But during the offline development phase that builds the code library, a Claude Opus 5.5 coding agent writes and debugs the code. The authors also see that because the code built this way reliably produces successful trajectories, it can conversely be used to churn out training data for VLAs and agent harnesses. It's "model-free at runtime," not "free all the way through development."