A robot arm is about to grab a cup off a table. Before it closes its grip, it has to make a few calls. Is the cup to the left of the plate or the right? How many centimeters from the fingertips? Where should it be set back down? For a person that's a blink. But today's vision-language models (VLMs — models that handle images and text together) make these calls by writing them out as prose. "It's on the left." "About 2.4 meters." "At point (0.37, 0.52)." Here the coordinate (0.37, 0.52) expresses a location on screen as horizontal and vertical fractions.
And yet the answer already lives inside a very narrow set of options. Left or right. One of a handful of distance bands. A single point on the image. Even so, the model writes that answer out one token at a time, autoregressively (predicting the next token from the ones before it), and then we read the sentence back to pull the number out of it. MJev-Spatial changes this behavior. Instead of composing the answer as text, it scores every one of a set of predefined candidates and picks one.
A multiple-choice exam makes the loss obvious. The answer is one of ①–④, but instead of ticking a bubble on the answer sheet, you're made to write out a long sentence: "The answer is 3, and the reason is…" Writing it takes time (generation cost), and the grader then has to read that sentence back to recover the number (parsing cost).
It isn't only about cost. The model doesn't just learn the spatial answer. It also learns the tone and the stock phrasing of the sentence wrapped around it. Writing that has nothing to do with spatial judgment bleeds into the training signal.
There's a speed constraint too. When a robot is picking something up or an agent is moving through a scene, a slow decision leaves it stuck in place. This is where the authors invoke the interaction rate (the decision cadence a robot or agent has to keep up with to avoid stalling). What's needed isn't an answer deliberated out slowly, but a fast, automatic judgment — something closer to what psychology calls System 1 (the thinking that comes out immediately, without conscious deliberation).
Spatial tasks produce roughly three kinds of output: pick one of several options, a single number (distance, angle, count), or a single 2D point on the image. All three can be written as a finite list of candidates. Options are finite to begin with. Numbers and coordinates become finite once you fix a range and resolution and chop them into bins — this is called discretization (binning a continuous value into a few intervals and treating them as discrete "cells"). So all three tasks collapse into one rule: "pick one of the candidates."
To pick, you have to score each candidate. The usual way is to bolt on a new classification output layer (a classification head) for exactly this. MJev-Spatial doesn't bolt anything on. It turns the question, the image, and each candidate into their own bundles of numbers, then computes a relevance score for how well each candidate fits the question and image. This computation borrows, as-is, the comparison machinery the backbone (the pretrained model it's built on) already used. So it adds not a single new parameter. The gain doesn't come from some extra component bolted on for performance — it comes from redirecting an existing capability, one that sat idle while the model was busy generating answers, toward spatial judgment instead.
Going deeper — where do the scores come from A transformer's attention turns each token into two vectors, a query (what it's looking for) and a key (what it offers up), then matches them to measure relevance. The process described above as "turning things into bundles of numbers to produce a score" is exactly this. The state at the position where the answer is decided becomes the query, and each candidate's state becomes a key. The matched value is that candidate's score, and the scores across the currently valid candidates are compared and turned into probabilities (softmax). MJev-Spatial doesn't build these query/key transforms from scratch. It takes the pretrained projections as initial values but doesn't leave them untouched — it fine-tunes them for the spatial task. The only things frozen are the vision encoder and the merger; the language-side parameters it borrowed are trained along with everything else (the readout layer that reads off the scores gets its own learning rate). In other words, it adds no new classification component — but that does not mean the existing parameters go untrained.
Technical note — why attention's projections, not the final layer When matching candidates, it uses attention's query/key projections rather than the transformer's final MLP layer. The final MLP layer tends to push the representation toward "next-word prediction," which degrades the ability to match meanings against one another (the authors cite observations from FreeRet). So candidates are compared earlier, at the projection stage specialized for comparison, before that transformation happens.
Distance and coordinates have too many candidates. Lay out everything from one centimeter to tens of meters, or every point in the image, all at once, and the list becomes unmanageable. So the model picks in two or three passes, in a coarse-to-fine fashion that narrows the range from a wide interval to a tight one.
Back to the cup. We have to pin its location on screen as a coordinate. First, divide the screen into 3×3 — nine cells. The model picks the one cell holding the cup. Only that cell is cropped out of the original again, divided into nine cells again, and one cell is picked again. Three picks in, this effectively pins one cell in a 27×27 grid, and the center of the final cell becomes the coordinate. The reason each pass re-crops from the original rather than from the previous fragment is to keep the errors introduced by cropping and downscaling from piling up pass over pass.
Distance works the same way. Asked how many meters apart two objects are, pass one picks a wide distance band — or picks "exactly 0" or "out of range." Pass two picks one of the finer cells that band is split into, and returns that cell's midpoint as the answer. Once wide, once narrow. Two choices get you to a continuous distance value.
MJev-Spatial parts ways with existing spatial models at three points.
| Aspect | Conventional approach | MJev-Spatial |
|---|---|---|
| Output | Tries to "generate" spatial answers better | Bundles options, numbers, and coordinates all into one "pick a candidate" |
| Scoring | Trains a separate output layer for candidate scores | Borrows the backbone's pretrained comparison machinery as-is (0 added parameters) |
| Refinement | Generates coordinates as text | Supervises region selection as classification; re-crops the chosen cell from the original to serve as the next pass's evidence |
With an aggregate score of 65.64%, it was the highest among the eight models compared, and it took first place on seven of twelve benchmarks. Against the untouched autoregressive original (the same backbone, Qwen3.5-4B), it was better on all twelve.
The aggregate score is an average. So while it supports saying "best on average," it can't tell you from that one number which tasks it won big on and which it fell behind on, or by how much it trailed on the five benchmarks where it missed first place.
On speed, across all twelve benchmarks it was 2.5× faster than the fastest autoregressive competitor and 5.6× faster than the same-backbone original. The two multiples use different baselines. The 2.5× is the more conservative figure, measured against the fastest competitor, while the 5.6× is measured against its own original. Both come out of a design that generates not a single answer token and runs the computation just once per decision (one forward pass).
Looking only at the results, it's tempting to read this as "autoregressive generation, as a format, was itself a burden on spatial tasks." But this model didn't only change the format. It was retrained with a new objective — classification — and it also brought in the coarse-to-fine structure and crop re-encoding. The fairer reading is that in the space left by stripping out the format, task-fitted training and structure were added.
One piece of evidence shows up in the ablation (an experiment that changes one part to isolate its effect). When length was discretized more coarsely — that is, when the cells were made bigger — there was a region where prediction error dropped even though resolution went down. Why? Finer cells shrink the error of the value each cell stands for, but they increase the number of candidates the model has to tell apart, which makes "picking" harder. The two forces push against each other, and past a certain point the "too many candidates to pick well" side wins. Higher resolution, in other words, doesn't automatically bring higher accuracy.
This raises a design problem. Candidate spacing, the number of passes, the time budget per decision, and the error the task tolerates — these four can't be set independently and have to be tuned together. Since chopping finer doesn't simply help, the optimum has to be found empirically for each domain.
"0 added parameters" also needs to be read carefully. Adding no parameters is not the same as adding no computation. More candidates mean more things to score, and a task with many passes, like coordinates, re-encodes a crop at every pass. So while "no extra components" is true, whether the overall computation stays light is a question to re-examine per task.
The limitations the authors state outright touch on exactly this. If a coarse pass is wrong, the region containing the true answer drops out of the candidates entirely. The geometric error bound on point prediction is a guarantee only "when the correct cell was picked" — if an earlier pass is wrong, it's moot. The quantization error bound on numbers is likewise a guarantee about the codebook (the candidate list), not about whether the model actually gets it right. In short, they don't hide the chronic weakness of coarse-to-fine structures: an error at an upper pass propagates downward.
Even in head-to-head experiments under matched conditions, the authors don't claim this approach is "always" better than a dedicated classification head. Their position is "competitive and cheap to train, but the trade-offs cut different ways depending on the task." The scope of application is also clear. This method runs cleanly only when the output can be expressed as a finite set of candidates. If the question is open-ended and needs free-form prose, or the candidates can't be enumerated in advance, you can't use this framework as-is. The range and spacing of the numeric codebooks (length, angle, count) are choices fitted to a particular task distribution too, so moving to another domain means redesigning and re-validating them.
The candidate list used to pick a number is called a codebook. It's laid out per value like this.
| Value | Candidate structure |
|---|---|
| Length / distance | Pass 1: 30 positive bins at log1p spacing + exactly 0 + out-of-range → Pass 2: split the chosen bin into a linear 4 or 8 cells. The chosen cell's midpoint is the answer |
| Angle | 30-degree positive bins + 0 + out-of-range, 32 cells per bin |
| Count | 32 coarse groups × 32 integers per group, exactly reconstructing 0–1023 |
To handle short dimensions and far distances together in one meter-based codebook, the pass-1 distance bins are spread out at log spacing (log1p). This codebook is saved with the checkpoint, so training labels, bin text, and inference values never drift apart from one another.
When there are several of the same object and all must be pointed at, high-probability cells are grouped together to stand for one object's location. It's a way of unfolding the 3-pass classifier into a tree, with no retraining. If pass 1 finds an object plausible in both the top-left and the bottom-right, the path forks in two and each branch descends further. Each final cell carries the product of the probabilities picked along the way to it, forming a kind of spatial probability map, and cells that rise above a set threshold relative to a peak are grouped to return one representative point per object. How many to return is decided by the number of groups, so there's no need to separately train a "that's enough" signal.
The evaluation compared eight models across twelve spatial benchmarks. The metrics are the aggregate score (an average over accuracy-type measures) and inference latency (the time to answer once). The backbone is Qwen3.5-4B, and the training data is 12.3 million question-answer pairs, converted into classification format from five existing spatial datasets (RefSpatial, SAT, SenseNova-SI, VSI-590K-image, VST-P). By task: 7.1 million multiple-choice spatial QA, 3.5 million point localization, 1.6 million length estimation, 120K angle estimation, and 38K counting. These numbers are for the original QA before unfolding into passes. Since point and numeric tasks unfold into two or three passes, the number of "decisions" the model is actually supervised on is larger than this.
One more thing from the ablations. When decoding a number, taking the expected value (conditional mean) over the fine cells within the chosen coarse bin actually increased error in a 72K-scale experiment and was not adopted; instead, the midpoint of the argmax-chosen cell is used.
Technical note — how training was run Heterogeneous spatial data varies in sentence length and in the number of image crops, so batches end up with empty space (padding) and the workload swings from GPU to GPU. The authors reduced this waste with length-aware global batching and load balancing across GPUs — leaving the optimizer's batch composition unchanged — and raised training throughput by 30.8%. That 30.8% is a training-speed gain, not accuracy. They also supervise every coarse-to-fine pass with teacher forcing (feeding in the ground truth ahead of time and training all passes together) in a single forward pass, while averaging the per-pass loss for each task so that a 3-pass point problem doesn't carry more weight than a 1-pass multiple-choice one.
Beyond spatial judgment, whenever the possible answers to a task collapse into a few predefined options, you can ask the same question: "Is this really something worth writing out as a sentence and then reading back to extract?"
ONTLE's request routing is one such scene. Say a request comes into Slack: "Could you rush me just two banner mockups for the detail page by next Monday?" The AI PM has to read this, classify it as a design task, and hand it to the right persona and the design queue. The destinations are already finite — a handful of fixed categories — yet right now an LLM typically writes its classification out as a sentence and we pull the category out of it to route. That has exactly the shape of the inefficiency MJev-Spatial pointed out in spatial tasks.
That said, this far it's a hypothesis, not an applied result. Testing it comes with one condition: the model has to be set up to read scores stably over the category candidates — whether by training with a classification objective as in the paper, or by some other method that produces candidate probabilities consistently. And the pitfall the paper admits carries straight over. The moment the first-level classification misroutes "design" as "data fix," every subsequent sub-classification is meaningless. The upper-pass-error-propagating-downward problem shows up in routing just the same, so when the top-level classification's confidence is low, you have to pair it with a safeguard that hands off to a human.
So if we were to look into this, the right move — before sketching an extension that breaks urgency, too, into multiple passes — is to compare one thing first: request routing. Process the same batch of requests both with the current generate-and-parse approach and with the candidate-scoring approach, and measure (1) latency per decision, (2) the rate of misroutes to the wrong queue, and (3) the rate that had to be re-reviewed by a human. The effects have to be read separately. Not composing an answer sentence cuts latency, and with the parsing step gone, errors from broken formatting sending things to the wrong queue go down. But the classification-accuracy problem — picking the wrong category outright — remains. On top of that, we'd need to see how much the low-scoring requests overlap with the ones that actually landed in the wrong queue. Knowing that overlap is what lets us set the review threshold — "below this confidence, send it to a human" — as a number.
This implementation uses Qwen3.5-4B as the backbone. An earlier prototype was built on Molmo2-ER with LoRA and a separate decision head, but the current model switched to Qwen3.5-4B and trained the full language parameters. The core idea — reusing pretrained query/key projections — isn't tied to a specific model, but performance on other backbones is outside what this paper reports.
No. Unfolding multiple paths to build a map is a mechanism for pulling out several points at once, not for recovering from a pass-1 mistake. Single-path inference has no undo, so if the coarse pass is wrong, the true region drops out of the candidates and the guarantees of the later passes lose their meaning.
No narrative justification comes out. The output is just the one chosen candidate and a probability obtained by comparing the valid candidates against one another. That is, instead of an explanation of "why," you're left with a number for "how confident." How that probability is used in operations is covered in "The ONTLE Angle" above.