Picture yourself walking over to a shelf and grabbing a cup. In those few seconds, the video in front of your eyes changes in two ways at once. As you step forward, the whole field of view slides past; as you reach out, the shape of your hand shifts in one corner of the frame. For a person these feel like a single motion, but to a machine they are two changes of entirely different character.
Ropedia W0 is a model that predicts both changes together. Give it a single opening frame and an instruction — "move like this" — and it paints a first-person video of what the world will look like after that movement. A model that draws out the consequences of an action as video like this is called a world model.
Until now, world models have mostly stuck to the tame cases: a fixed arm in front of a table, picking things up with a gripper. The longer situations where the viewpoint shakes as you walk and several fingers handle an object delicately — where locomotion and manipulation overlap — got sidestepped. That overlap is exactly where Ropedia W0 stakes its claim. Handling human hands, two-finger grippers, and multi-jointed humanoid hands — several differently shaped bodies — within one model while jointly predicting walking and manipulation is, as far as the authors know, a first.
Ropedia W0 is not a model that decides "what to do next" on its own. It takes three inputs: a single frame of the scene right now, a short line of text describing the task, and the motion information — "this is how I move." The output is the future video you would see if that motion were actually carried out.
The model that actually drives the robot is a separate thing. It's usually called a policy: it looks at the current scene and decides the next action — "okay, move the arm this way." The world model sits one step before that. Once the policy picks "how to move," the world model predicts "then what will I see." When a policy that's trying to grab a cup floats a candidate move — "reach the hand a little further forward" — the world model draws out the picture in front of the eyes that would follow.
The hard part is how you write that "this is how I move" into the model.
A human hand has many joints; a two-finger gripper does nothing but open and close; a multi-jointed robot hand is yet another structure. When the body differs, even the same "grasp" gets written out in completely different numbers. So every time you added one more body, you had to lay a new pipe to feed that body's motion into the model.
Prior work has gotten over this wall in more than one way, and some approaches already route everything through a single rendering interface rather than a per-body pipe. Where Ropedia W0 differs is in what that shared interface carries: whatever the body, the motion gets rewritten into the same single picture — one that keeps each body's own geometry — so no body-specific module or extra adaptation step has to be bolted on when a new body shows up.
Motion is split into two strands and written as a picture.
The viewpoint change produced by walking is written as a camera pose (a term that bundles the camera's position and orientation). Where the camera goes, how far, and which way it looks tells you how the frame will slide in the next instant. The trick here is to write the amount of travel in meters. Whether it's a person walking or a robot moving, the sense of distance — "a meter is a meter" — never wobbles.
Hand manipulation is written as a skeleton overlaid on the image. It lays a picture on top showing where the hand joints are right now. Whether it's a human hand joint, a robot finger, or the opening of a gripper, each body's own connective structure is drawn as-is.
The upshot is that every differently shaped body gets translated into the same format: one camera pose plus one skeleton drawing. Because the input format is identical, you can feed expensive robot video and cheap human video into one model together.
What that shared format actually lets you reuse shows up in the training. The model first learns, from human first-person video alone, the common relationship "when the camera and hand move like this, the scene changes like that" (pre-training). Then it mixes human and robot data at equal probability, layering on the robot's own appearance and joint structure (mid-training). Because both stages use the same camera-skeleton format, the sense of "motion → outcome" learned from human video carries straight over to robot prediction. The geometry that differs from body to body stays right there in the picture.
Laid side by side along "how motion is expressed" and "what you have to do to apply it to a new body," three prior approaches and Ropedia W0 look like this.
| Approach | How motion is expressed | What it takes to apply to a new body |
|---|---|---|
| DreamDojo | Latent action — an abstract motion signal estimated from video | Swap that signal for robot-specific actions when used on a robot |
| Cosmos3 | Shared backbone, but motion is projected per domain | Lay a new dedicated projection path for each domain |
| AnyWorld | Reuses the source human skeleton as a condition | Does not use the target robot's kinematics (joint connectivity) |
| Ropedia W0 | Camera pose + on-image skeleton drawing | Draw only the new body's geometry into the same format |
Note on scope: AnyWorld has not been validated in settings where locomotion and manipulation overlap.
The authors report state-of-the-art (SOTA) results across five benchmarks and publish the per-benchmark quantitative numbers in Tables 3–5. For instance, under DreamDojo's 1% training setting, In-lab PSNR (higher means the predicted video is closer to the original) reaches 23.028dB for the W0 A14B model, ahead of DreamDojo 14B's 18.983dB. Below is the direction of the reported gains, too.
The headline result is that scaling up human video improved robot prediction along with it. Adding more human pre-training video didn't just improve prediction on human video — it also raised transfer performance to robots (transfer performance as a function of human-data scale is shown in Figures 6–8). That cheap, abundant human first-person video lifts the prediction of expensive, scarce robot data is the paper's core claim.
The authors also demonstrated usefulness along three lines: (1) policy evaluation — scoring a policy in advance by rendering it as video, (2) model-based planning — drawing out the outcomes of several actions and picking among them, and (3) world-action modeling. In the last of these, rather than bolting on a new action-only component, they took the backbone trained via video prediction (the model's shared body) and used it directly as the starting point for learning actions. The view is that the representation built up by drawing video is also good footing for learning to act.
Generation speed has crossed into the practical. After distillation — moving a large model into a small, fast one — the student model runs minute-scale interaction at 21.3 frames per second. That's not an offline predictor you run through once; it's fast enough to handle an interactive simulator with a human in the loop, or even real-time VR interaction.
The hidden center of gravity in this work is not the model but the data. There's plenty of raw material. Combining public and self-shot human footage with public robot data, they gathered 10,000 hours of human and 3,000 hours of robot video (the dataset is named Xp-Data). The problem is that these materials come in wildly different specs. There's a lot of public human first-person video, but it has no meter-scale camera trajectory, the hand poses are messy, and the formats are all over the place. Abundant, but unusable as-is.
The thing that gave them the most trouble was pinning a hand skeleton onto the image accurately. For robots, you can derive the skeleton via forward kinematics — computing the fingertip position from joint angles. But in public robot video, this computation often disagrees with the actual hand or gripper, and camera calibration is frequently missing. So people hand-marked keypoints (major points like joints) on the images and manually corrected the skewed projection. Forward kinematics is only a first-pass estimate; the skeleton actually used for training was the hand-fitted one.
In the evaluation data the authors newly built (Xp-Teleop), they deliberately left in some failed grasps and the process of recovering from them. The point is to see whether a world model merely mimics the successful scenes or can also draw out things that go wrong. With data that holds only successful scenes, it's hard to adequately learn or evaluate "what the frame looks like when things go wrong."
The data pipeline and the model's hardware, tools, and numeric specs run long, so they're folded away below.
Xp-Data consists of 14 public human datasets + 6,000 self-collected hours (a four-fisheye-camera rig) + 13 public robot datasets. Raw video goes through five stages — collection → video filtering (a vision-language model removes heavy blur, exposure problems, and excessive shake, plus segments where neither camera nor hands move) → multimodal annotation → quality inspection → training-clip assembly — and the sub-tasks that pass are standardized into 81-frame, 16fps clips.
Annotation tools: camera trajectories are taken as-is when calibration exists, otherwise estimated to meter scale with in-house SLAM and DA3, with Grounding DINO used to find dynamic regions such as hands and people and exclude them from the trajectory-consistency check. For hand alignment, humans use MANO keypoints recovered via WiLoR and projected, while robots use URDF-based forward kinematics plus manual correction. Action descriptions are three-tier captions — task, sub-task, and atomic action. For error verification, captions are generated by Gemini → verified by Qwen → resolved by humans; hands are cross-checked between MediaPipe and a fine-tuned MMPose 2D detector; robot keypoints are cross-verified with two in-house fine-tuned detectors.
Xp-Teleop: a new benchmark collected with a whole-body teleoperation rig — a Unitree G1 humanoid fitted with two 6-DoF multi-jointed hands, assembled from off-the-shelf hardware and the Unitree open-source framework. It has 27 long takes (first-person, 30fps, about 1 hour), with two operators performing walking and hand manipulation simultaneously. RGB-D, camera pose, projected skeleton, robot state, language, tactile, and head IMU are synchronized, and it is cut into 543 human-annotated atomic segments (30 skill classes, median 18 segments per take). Per Table 1, each take averages 118.6 seconds, a walking range of 1.48m, and 3.6 sub-tasks on average. Evaluation is run on five cross-embodiment benchmarks (DreamDojo, Ψ_0, ω-HOME, Xp-Teleop, AgiBot) plus a held-out set of human first-person video, under zero-shot, few-shot, and full fine-tuning; comparisons are against DreamDojo, Cosmos3, OSCAR, RynnWorld-Teleop and others, with an appendix comparison against BWM on WorldArena.
Model and training: the backbone is a latent video diffusion transformer. Camera conditioning is encoded as a 6-channel Plücker-ray map per pixel; conditions that align to pixels (camera, skeleton) are fused by adding them to the video tokens, while information with no pixel correspondence, such as a wide-field-of-view skeleton, is injected via cross-attention. The camera and skeleton projection inputs are zero-initialized so they don't disturb the pre-trained backbone early in training. Scale comes in a dense 5B model and a Wan2.2-I2V-A14B-based MoE (about 27B total, 14B active). Training runs in two stages — pre-training on human first-person video → mid-training that mixes human and robot at equal probability — both using the same camera-skeleton format. After that, Causal Forcing builds a block-wise 4-step generator, LongLive stretches it for long streaming, and then it is distilled.
What the unified representation actually carries over from human to robot is "where the hands and joints go in space." The skeleton input, by contrast, doesn't directly mark how firm the contact is or how hard the grip squeezes. And making the format identical doesn't erase the difference in motor capability between a human arm and a robot arm. How far a representation stripped of force and contact can support planning and control — up to which tasks — remains a question to be pressed further against the concrete numbers from the policy-evaluation and model-based-planning results mentioned above.
The price of the unified representation was paid by human hands. That public robot video's forward-kinematic projection often went wrong, requiring substantial manual keypoint annotation and correction, flips around to mean this format wobbles without people propping it up. The cost of making specs line up fell on humans, not on the model architecture. How much that cost runs when you add one more robot is hard to gauge from the current material alone.
Physical correctness is a separate matter. Drawing a plausible video and that video being physically correct are two different things. Related work also keeps separate benchmarks for visual quality, physical consistency, and usefulness in actual control. How trustworthy a plausible-looking prediction is for robot control needs to be pressed further against the concrete numbers on the policy-evaluation and planning side.
Don't over-trust the size of the evaluation set, either. Xp-Teleop's 27 takes, about an hour, is a set held out for evaluation rather than training. It's clearly a good yardstick for long-horizon locomotion-manipulation, but the diversity of one evaluation set doesn't fully stand in for future generalization.
From here on, this is a hypothesis, not a verified conclusion.
ONTLE goes into field settings where the tools, the stack, and the team differ from project to project, gathers the scattered work information into one place, and reads out how things are currently playing. There's one idea to borrow from this paper: leave the raw data in each field as varied as it is, but unify the higher-level representation used for inference into one. The point isn't that "integration gets easier" — it's that information from different tools gets interpreted against the same standard.
Let's narrow it to one scene. A customer asks to move the delivery date up. To read what ripple this request sends through the schedule, you have to see the issue tracker's tickets, the repository's commits, and the calendar's milestones at a glance. But the three look nothing alike. Just as Ropedia translated different bodies into "camera pose + skeleton," you translate these three into a shared format of "work status (waiting, in progress, done)" and "dependencies between tasks."
That said, status and dependencies alone won't tell you "when it slips." You have to fit information like the work remaining on each task and each owner's available capacity into the same format, so that "if I pull this task in, where do the tasks that follow it slip" can be drawn out on one board. And this "fitting-in" isn't free. In the paper, too, the unified representation didn't line up by itself — people had to hand-correct the skeleton. Which work status to read an issue as, which task to tie a commit to — the quality of that alignment becomes the ceiling on prediction quality.
Finally, one question remains that Ropedia doesn't warrant for us outright. Just as human video lifted robot transfer, does accumulated past-project history actually support delay prediction for a new project with thin data? That has to be checked separately with ONTLE's own data — measure whether a predictor trained on past projects beats a human's rough hand-estimate on a new project with no history.
It means handling bodies with different shapes and joint structures — human hands, two-finger grippers, multi-jointed humanoid hands — within one model, crossing between them. The fact that all five benchmarks this work uses for evaluation are "cross-embodiment" benchmarks is precisely to measure whether the same model can predict across changing bodies.