You ask an AI to "pull together last week's client-meeting schedule." It picks one tool to start from among the ones available — calendar lookup, CRM lookup, mail search, document generation. But depending on the order you listed those four lines in the prompt, the same request calls the calendar yesterday and mail search today. The request hasn't changed. The tools haven't changed. The only thing that changed is the order the lines were written in.
What we're dealing with here isn't the kind of job that delivers a long explanation. It's the kind where choosing one option quickly, judging whether a condition holds, and assigning a score — where selection and judgment are the whole point. The paper calls models that skip the long reasoning and hand back a result immediately "System 1 models," named after the snap, intuitive judgments people make. And it's exactly in that moment of the snap answer that something as trivial as ordering shakes the result, as in the scene above. Jeva, from a team at the National University of Singapore (NUS), is an attempt to remove that wobble at the structural level.
Most of today's publicly available System 1 models work like this: the request and the candidates (the options) are strung together into one block of text, fed to a language model, and the answer is read off the scores the model assigns to each option or from a dedicated output head. "Score" here means the logit — the raw number before it's turned into a probability. The approach is convenient. The trouble is that it fits poorly with the very nature of a "decision."
The paper gathers in one place the four properties a decision problem implicitly demands.
On the surface the four properties look like separate demands, but they share a single root: expressing meaning, comparing candidates against each other, and judging whether an answer exists are all tangled together inside the same prediction step. Jeva's design starts by pulling these three apart.
First, a single shared encoder turns the request and each option into vectors separately. A vector is a bundle of numbers that captures a text's features. When it turns an option into a vector, it includes neither the current question, nor the progress state, nor the other options, nor the option's index number. So an option's representation stays fixed no matter what the request is. That fixedness is the precondition for caching. Picture a restaurant that prints its menu once and keeps using it regardless of who walks in. There's no reason to reprint it every time.
Having fixed the option representations, it seems you could just score each option on its own and be done. But then the process of comparing options against each other disappears. Jeva adds a lightweight "decision block" that reads over the fixed options and nudges only the "request-side state" little by little. The request state is the request's internal representation, updated as it looks over the candidates.
Why this nudging matters becomes clear by comparing two scenarios. Say the router has only two usable tools, A (mail search) and B (general-purpose document generation), and the request "summarize the minutes from the last meeting" comes in. Of the two, B is the better fit. Now two things can happen. One: leave A and B as they are and only shuffle the order they were written in the prompt — the judgment must not change. Two: actually integrate a new C (a dedicated minutes-summarization tool) — and here the story is different. Once C enters, the very criterion for "who handles this request" shifts to "can it deal with minutes," and on that axis even the ranking of A versus B may have to be reconsidered. Because the whole candidate set has changed.
If you score options completely independently, the second case is impossible: adding or removing other options never changes the relative ranking of two options. Jeva adjusts the request state to fit "the entire current candidate set," so it holds its judgment when only the order changes and re-weighs the ranking when the set changes.
Order invariance is preserved through the aggregation method. When the decision block gathers the options, it uses a symmetric aggregation that isn't affected by order, so shuffling the options merely shuffles the output probabilities along with them — the values don't change. And since every option is handled by the same scoring method, the output side doesn't need to grow as the option count rises.
"Not applicable" is split off into an entirely separate mechanism. Relative ranking among options alone can't answer "is there even a single acceptable option at all?" Adding the same value to every score leaves the competition among options unchanged, yet "does an answer exist or not" ought to change. With a "coverage gate" that uses a learned threshold and temperature, Jeva first splits the probability between "none" and "the candidate set," then resolves the competition among options within that. Temperature is the value that controls how sharp or flat a probability distribution is made. This one decoupled structure handles all three outputs: picking (Choice), verifying (Noul), and scoring (Score).
There are broadly two currents for solving this problem. One encodes the request and candidates together as text and reads from answer tokens or a decision marker; the other encodes the query and candidates separately to gain reuse and efficiency. Jeva sits between them. It encodes options separately for reuse, but before scoring it first adjusts the request state to the entire candidate set. The difference is sharp along three axes.
| Approach | Representative models | Candidate reuse | Set interaction | "None" separation |
|---|---|---|---|---|
| Joint encoding | SemIf, JevK5, InternDecision, Re, Flex, KEV, Laya | No | Yes | Limited |
| Separate encoding | DPR, ColBERT, CLM | Yes | No | Not addressed |
| Jeva | — | Yes | Yes | Separated via coverage gate |
In effect, Jeva fills in the reuse the first current gave up and the set interaction the second current missed. It also connects to the symmetric-aggregation ideas of set-oriented neural networks that handle sets without order (Deep Sets, Set Transformer, Perceiver). Its "none" handling shares a concern with tasks that ask about answerability, like SQuAD 2.0, but Jeva differs in nailing "none" down as "the absence of a valid candidate" and separating it from uncertainty.
Start with the promise in the title. Running the same 600 Choice items in original, reversed, and random order — aligning the options by their identity and then changing only the order — they measured whether the decision wobbled. Jeva never wobbled once. The difference between the probability distributions produced by reordering (TV distance, the distance between two distributions) averaged about 3.17e-08, essentially computational error. More telling is that two numbers match: the original accuracy and the proportion correct across every tested order are both 76.67%. It means order added no information to the result. To be precise, though, what the structure mathematically guarantees is that, under exact arithmetic and fixed weights, each option's probability value itself doesn't wobble with order (which is why the TV distance isn't exactly 0 but stays at the level of floating-point error). The final choice stays the same only when the top-scoring option is unique; with an implementation that picks the first option on a tie, or with floating-point error, the choice can diverge (Proposition 3, Appendix B.5). In this experiment there was not a single such flip. Other models, by contrast, flipped their decision about one time in five.
| Model | Flip rate | All-order accuracy |
|---|---|---|
| Jeva | 0.00% | 76.67% |
| Jev-1.13.0 | 17.00% | 66.67% |
| InternDecision-4B | 22.67% | 64.50% |
| JevK5 | 26.17% | 47.83% |
Its ability to flag "none" was also in front. Measured on 3,080 items from BANKING77 — 16 options, with 588 answer-absent cases mixed in — Jeva's AUROC (higher is better) was 98.66%, well ahead of KEV-BF16 (93.61%), InternDecision-4B (84.27%), and JevK5 (79.44%). While holding the rate of wrongly calling a valid option "none" to 5.02%, it still caught 93.88% of the genuinely absent answers. For jobs where saying "I can't" matters more than forcing a call to the wrong tool, this number is the crux.
Scalability was examined by growing the option count. On 500 ToolRet items Jeva processed every request and posted an aggregate accuracy of 78.40%, while JevK5, which recursively narrows the options, managed only 64.00%. Across 83 queries where options were grown from 32 up to 2,048, the number of queries answered correctly at every size was 58 for Jeva versus 48 for JevK5, and only 7 for CLM. The InternDecision family hit a cap on registered options and processed only some requests on the large bundles. The promise that candidates can grow without rebuilding the structure is confirmed in the numbers.
Reuse speed was measured by how much latency drops when candidate computation is cached (prior answers are not cached). On MLDR the p50 latency fell from 2,102.0ms to 75.1ms, and on Typed Decisions, which shares candidates, it dropped from 93.1ms to 46.6ms, skipping 99.2% of candidate encoding. The decision didn't change either, and the maximum probability difference was 0.00736. That said, all of these latencies come from an H200 accelerator.
On accuracy itself, it led in 7 of 8 slices. On the sole one it trailed, Typed Decisions, it lost by a hair, and on type fidelity it was narrowly behind on just one metric, Score RMSE. The details are gathered in the supplementary table below.
| Item | Jeva | Comparison |
|---|---|---|
| BANKING77, MASSIVE (8 and 16 options) | 90%+ | — |
| Mind2Web cross-task / website / domain | 79.99% / 62.25% / 66.23% | — |
| Typed Decisions | 78.30% | InternDecision-4B 78.70% |
| Noul Brier / MAE (lower is better) | 0.0184 / 0.0939 | lowest on each |
| Score MAE / RMSE | 0.2315 / 0.3222 | RMSE only trails InternDecision-4B's 0.3208 |
These numbers are different metrics from different experiments, so they shouldn't be read together as one combined achievement. Each comes from different data and conditions.
The constraints the authors acknowledge help gauge the study's scope. Each text gets its own length budget from its own encoder, but that doesn't make an individual text's length limit disappear. Because scores can change as the state is adjusted to the set, there's no guarantee that adding an option leaves the final choice over (candidates, none) unchanged (Appendix A.3). Reuse isn't free either: only the candidate-side computation is cached, while state encoding and decision reading run anew on every request. So for a service where the option set is completely fresh every time and there's little to reuse, the caching gain shrinks. The evaluation was reported with a Qwen3.5-4B backbone fixed to a single decision block, so how the four properties and the efficiency shift on a larger or smaller backbone can't be settled within this scope. Jeva's strength is better read as guaranteeing the four properties together, rather than dominating on average accuracy.
ONTLE is a service where an AI PM takes repetitive work off people's hands. Back to the opening scene. When a user says "pull together last week's client-meeting schedule," the PM has to pick one from the list of registered tools and integrations. That "which tool handles this request" is exactly a Choice decision.
A router that leans on token scores can pick a different tool for the same request when the order the tool list was written in changes. But this is a risk that could show up given that structure — not something we've measured in operation. Jeva-style decoupling blocks this risk at the structural level, because order never enters the result at all. Conversely, attaching a new integration is a different situation from reordering. The candidate set itself has changed, so it's actually correct for the choice to change, and Jeva is designed to re-weigh the ranking in that case. The important thing is not to conflate the two cases.
"None" also mustn't conflate two judgments. One is the judgment that "there's no tool to handle this request at all" — a case like an external system not yet integrated. The other is the approval judgment of "is it OK to actually run that tool." What the coverage gate handles is the former — making it say "this can't be done right now" instead of forcing a call to a plausible but wrong tool. Execution approval is handled separately by ONTLE's principle of approval before external sending and execution.
The effect is right to treat as a hypothesis to confirm with numbers, not a declaration. If you run the same request with only the tool-list order changed (original, reversed, random) and measure the flip rate and probability difference, you learn whether order sensitivity is actually a problem and whether Jeva-style decoupling removes it. Whether caching the tool and document lists — nearly fixed per client — keeps the decision the same while cutting latency is measured the same way. For a product that keeps adding integrations, the fact that you don't need to rebuild the output structure as options grow would be especially valuable, and that too is something to confirm by measurement.
In the end, whether to consider Jeva first comes down to two things: do candidates repeat across requests so there's a lot to reuse, and must "no answer" be clearly distinguished from "confusable"? If both are "yes" and the work has an ever-growing set of options — tool routing is precisely that — it's the first design to put on the table. Conversely, for work where the labels are fixed and you need a stable score per option, order invariance and scalability don't pay off much.
The decision block reads over the fixed options and corrects the request state twice in a residual manner. A residual update is adding a correction to the existing value rather than overwriting it wholesale. The paper shows as propositions that this design mathematically guarantees three things. That even with a single decision block the ranking of two options can flip depending on the set (Proposition 1 — with the independent-scoring softmax of Equation 2, the denominator cancels and this is impossible); that the coverage gate's "none" probability differs from a "normalize by the number of candidates" scheme (Proposition 2, Equation 6 — provided the final scores of existing candidates are fixed, adding more options below the existing maximum doesn't shake the "none" probability, unlike a flat softmax that uses the candidate count as denominator and dilutes "none" as candidates grow; in actual Jeva, though, adding a candidate changes the request state and the existing scores together, so the "none" probability can change — Appendix A.3); and that the structure guarantees order invariance (Proposition 3). That said, what Proposition 3 guarantees is that the output probability distribution is invariant to order, and invariance of the final choice follows, under exact arithmetic, when the top-scoring option is unique — with an implementation that breaks ties by the first option, or with floating-point error, the choice can differ (Appendix B.5).
Training used Qwen3.5-4B as the backbone, learning only the two projections and the decision head (13.37M parameters) with a single decision block and rank-64 LoRA. LoRA is a lightweight method that trains only a small set of added weights rather than the whole model. The curriculum proceeds in the order of type decisions, intent selection, verification, reading and reasoning, and large-scale candidate retrieval.
In the open-set evaluation, we opened a workaround for those models too — giving them a None option directly, or having an auxiliary question of "does a candidate exist" correct the candidate probabilities. For within-model comparisons we fixed the checkpoint and query order, and all latencies were measured on H200.
They're the same structure read differently. Choice picks one of the options; Noul verifies true/false but answers with the probability of the positive-side option; Score produces a score as an expected value against a numeric rubric. Only the reading criterion differs — the input fed into the encoder does not grow separately per output.
No. For serving, the LoRA is merged and fixed, and that model is used to precompute the option memory. What the cache targets is this precomputed candidate side, and previously produced answers are not cached.