PhysBrain 1.5 is a piece of research that packs three jobs into a single model: understanding a physical environment, generating actions, and predicting the next scene. It starts from a general-purpose AI that handles both images and text (a vision-language model) and trains all three of these in the same way. The result is an 8B model (one with 8 billion tunable parameters) that scored a 72.5 average across 28 benchmarks designed to test whether it understands situations that involve using a body — the top result in the open-source (freely downloadable and usable) category. It stands shoulder to shoulder with top-tier commercial models like GPT-6-Astra and Gemini 3.6 Flash.
Before you read on, though, there is one distinction worth nailing down. The 72.5 measures how well the model "understands." There is a quantitative metric for action ability too — Appendix B.3 shows that during pretraining, the perplexity of action-token prediction (a measure of how well it guesses the next piece, lower is better) dropped from 17.12 to 5.07 within the training distribution, and from 12.39 to 6.57 even on RoboDojo, which was held out of training. But this is only an "offline" metric — predicting the next action on pre-recorded data. Whether a real robot can execute those actions in a closed loop (moving, then seeing the result of that move, and repeating) to actually complete a task was not verified this time around. In short, the model has shown its understanding and even its action prediction in numbers, but its ability to actually use a body to finish a job is still at a stage that needs further verification. Hold onto this distinction while you read the rest, and the picture gets much sharper.
Until now, most large AI models have been good at "seeing and talking." They describe photos, answer questions, summarize text. But actually "doing" something in the real world is a different dimension. Humans live inside a loop: we see with our eyes, reach out and intervene, and then look again at the scene that intervention changed. You see a cup, grab it, and look again at where it ended up.
The limitation of earlier approaches was that they handled the three links in this loop separately. There was one model for understanding a scene, another for generating motion, and another for imagining the next scene — the eyes, the hands, and the imagination unable to talk to one another. PhysBrain 1.5 tries to tie all three into a single training framework.
Another practical difficulty is data. Records of robots actually moving are expensive to collect and scarce. Videos of humans handling objects, on the other hand, are everywhere. This paper takes those cheap, abundant human videos as the main fuel for training.
The most important insight is "translate every kind of information into the same language." The language here is a sequence of tokens. A token is a small chunk of information — just as a language model breaks a sentence into words to handle them, here even motion and scenes get broken into chunks.
PhysBrain 1.5 turns all three of these into strings of such chunks.
Once all three are turned into chunks, you can train everything with the one method a language model was already good at: looking ahead and guessing the next chunk. Just as it guesses the next word in a sentence, it guesses the robot fingertip's next position or the next chunk of the next scene. By optimizing all three at once, a single brain learns to speak, move, and imagine with the same grammar.
Training happens in two stages.
The first is pretraining. This is the stage for grabbing a big-picture feel before the real training begins, and here the signal that teaches body-using ability comes solely from human videos. Not robot logs, but videos of humans handling objects, are the textbook. At this point it uses a unit called a "task-centric episode." Centered on a single task, it pairs up — into one bundle — what is being done and why and where, the motion reconstructed from the video, and the scene that follows. It resembles watching a cooking video and learning, as one whole, "that hand moved like this, and then the pot was placed like that."
The second is fine-tuning. The model, having grabbed a broad feel from pretraining, is refined with three kinds of data: human demonstrations, logs of real robots moving, and experience gathered in a virtual environment (simulation). In effect, the feel captured from video is tuned down to the fingertips with precise data from real robots and virtual environments.
The key numbers are gathered in the table below.
| Item | Detail |
|---|---|
| Model size | 8B |
| Number of benchmarks | 28 (understanding body-using situations) |
| Average score | 72.5 |
| Standing | Best open-source, on par with top commercial models |
| Open-source best achieved | 14 of 28 |
There is one point as important as the numbers. Even while learning robot motion, the model's original general abilities — image understanding and language — did not collapse. Adding a new capability while existing ones degrade is a common trap in this field, and the model held that balance. This is especially valuable when putting it into a real product.
Action and prediction abilities are presented together through offline quantitative metrics and demonstration scenes. The action-token prediction perplexity in Appendix B.3 fell steadily during pretraining (17.12→5.07 within the training distribution, 12.39→6.57 on the held-out RoboDojo), and to this are added demonstrations of generating fingertip trajectories and drawing the next scene. But these numbers are an offline metric — predicting the next action on pre-recorded data — not a success rate of a real robot moving in a closed loop to complete a task. Notably, when drawing the next scene, it produces the same scene matched in three layers at once: color (RGB), sense of distance (depth), and where the robot itself is on the screen (robot-mask). In other words, it does not merely imagine the color, but draws depth and its own position along with it.
The distinction raised in the lead is the crux of this section. The 72.5 average is a score on an "understanding" benchmark, and action prediction has been confirmed as far as the offline perplexity metric (Appendix B.3). But how accurately a robot actually moves in a closed loop to grasp and move things — that is, a number measuring task success rate — is outside the scope of this release.
One more thing. Leaning pretraining solely on human videos scales well, but the human body and the robot body are different. How far the gap between a human hand and a robot gripper has been bridged is hard to judge from this release alone. Mixing real robot logs and virtual environments into fine-tuning reads as an attempt to narrow that gap, but just how large its effect is, again, is not given in numbers.
To sum up: the possibility of "tying it all into one framework" and the strength of its understanding were shown convincingly, and the offline metrics for action prediction improved too — but the closed-loop success rate of actual manipulation and the scope of generalization remain open questions to be confirmed in the main text and in follow-up verification.
ONTLE is a service that dispatches an AI project manager to each client project. It may seem far removed from robotics papers, but one lesson this research leaves lands squarely on how we validate our own product: "being good at understanding" and "actually getting it done" are separate things, and must be evaluated separately.
PhysBrain 1.5 clearly proved an understanding score of 72.5 and showed action prediction through offline metrics, but its ability to let a real robot finish a job in a closed loop is unverified. The same goes for an AI PM. The ability to accurately grasp a client's request (understanding) and the ability to carry that request through to actual task completion (action) must be measured by different yardsticks. For example, just because a PM smoothly summarized a request like "sort out last week's delayed tasks" is no guarantee that it properly finished reassigning owners and updating the schedule. That is why ONTLE QA looks at two axes separately: whether the request was properly understood (understanding accuracy) and whether the promised work was actually handled (task completion rate), each measured on its own. The habit of not concluding "this product works" from a single benchmark score — this paper confirms it once more.
As of this release, it presents offline quantitative metrics for action-token prediction (the perplexity drop in Appendix B.3) and demonstrations of fingertip trajectories and next-scene prediction — but it does not present numbers on whether a real robot manipulates things in a closed loop to complete a task. Think of it as having shown "the direction in which it could move."
Records of robots actually moving are expensive to collect and scarce, whereas videos of humans handling objects are abundant. The order is to grab a basic feel with cheap, plentiful resources, then refine it with precise robot and virtual data.
This research states that it retained its existing image and language understanding abilities even while gaining the new capability. Avoiding the common trap where a new feature pushes out old ones is a practical strength.