Why are LLM agents slow? One big reason is that they decide their next action by spelling it out one character at a time, every single time. A token here is the smallest unit a model works with when it handles text. Even at the moment an agent decides to go "up one square," it generates those two words from scratch. Same for "down one square," same for "left one square." JevSpawn takes direct aim at exactly this — the cost of writing actions out.
When you think about it, it's a strange waste. The alternative actions an agent weighs usually share the same frame and differ by just a few values. One direction, one number. And yet it rewrites the whole sentence from the top each time. So why not fix the shared frame in place and just pick the slots that change from a set of predefined candidates? The shift is from writing (generating) to choosing from options by probability (selecting).
The idea itself isn't new. "Jev-style models" got there first. The paper's comparison point, TypeSafe Jev, doesn't answer a question in sentences — it assigns a probability to each of a fixed set of options and returns them directly. Choosing over a finite set of choices is fast. The catch is that the application has to compose and feed in that list of options ahead of time. For an autonomous agent, that's fatal. What it can do has to be read out of natural-language instructions on the fly, and it changes with every new observation that comes in.
So the real hard problem narrows down to this: keep the speed of finite choice, while letting that action list change mid-interaction. That's the point JevSpawn solves.
Boiled down to one sentence, the fix is to pull the action's blanks (the fields) into the policy itself. The policy is the rule by which the agent chooses its next action. Instead of receiving it pre-made, the model declares those blanks on its own after reading the task description, and revises them from feedback.
A kitchen analogy makes the difference sharp. TypeSafe Jev is a chef who only works once the customer hands over an order slip that says "pick spicy or mild." Fast, but useless without the slip. JevSpawn is a chef told, "open the fridge, look inside, and compose the possible options yourself." It reads the task instructions and the rules of interaction, then declares the names and meanings of the blanks itself, along with the finite set of values each one allows given the prior choices. Fill in all the blanks and you have one executable action.
Four mechanisms hold this chef up.
First, parallel spawning. From one state, it fans out several action candidates at once. Which combinations to look at first is guided by the joint probability — the likelihood that several choices occur together. In the paper's setup, it picks one parent and fans out up to four different actions.
Second, feedback-based branch selection. It runs each candidate once in an independent copy of the parent environment. Then it re-scores the branches based on the observations that come back. In other words, it can change its mind after trying.
Third, representation revision. Based on the feedback that accumulates, it rewrites the declaration itself. The list of possible actions changes mid-interaction — the very point where it parts ways with existing Jev-style models that are locked to pre-fixed blanks.
Fourth, recovery from preserved branches. When it hits a dead end, it doesn't walk the whole path again. It backs up to another road it didn't take but set aside at an earlier fork, and resumes from that point. This saves the cost of regenerating the earlier path.
One thing to be clear about here: there's no additional training. The pretrained parameters are left untouched. Text is generated only when composing a new declaration, revising one, or producing a final answer; the iterative search runs on a finite policy.
The secret to the speed is two kinds of "sharing."
One is action-structure sharing. Rather than rewriting the complete description each time, it fills in only the missing slots. Exactly as in the paragraphs above.
The other is prefix sharing. A prefix is the leading context that several candidates share. If many fairy tales all begin with "Once upon a time," it's like reading that opening once and then only reading the parts that come after. Since the common opening isn't recomputed per candidate, the input tokens to process drop from the cost of "including the prefix every time" to the cost of "processing the prefix once."
The comparison was run under controlled conditions. The policy model was fixed to a single model, Qwen3.8-27B. JevSpawn and all seven competing baselines used the same model. Randomness was stripped out with temperature 0 and seed 42, and the total budget was capped at 300 seconds. So this isn't "which method is absolutely fastest" — it's a relative comparison under identical conditions.
The competitors' approaches fall into three groups. Some reduce when and where the model gets called (call orchestration), some reduce the amount of information passed between steps (information reduction), and some raise the output generation speed itself (decoding acceleration). JevSpawn aims at a different axis from all three — action generation itself. That said, the idea of "choosing actions instead of writing them" isn't the first of its kind. The paper also grounds itself in prior work, like the finite choice of TypeSafe Jev seen earlier, or SGLang, which skips deterministic generation paths. JevSpawn's contribution is combining this idea with an adaptive action space and branch search, making that list change mid-interaction.
There's just one thing to watch when reading the results: the unit of measurement differs per task. Navigation-type tasks use success rate (0 to 1), while puzzle and numeric games use benchmark rewards or scores. You can't lay 0.96 and 305.12 side by side and compare them.
The most interesting result is here. Normally, raising the score costs more time. But on the two navigation tasks, both got better.
| Task (success rate) | JevSpawn | Best baseline | Latency: JevSpawn / fastest baseline |
|---|---|---|---|
| Maze | 0.96 | FoldAgent 0.52 | 40.91s / 47.91s |
| Grid | 0.95 | FoldAgent 0.73 | 40.58s / 61.44s |
Latency is the total end-to-end (E2E) time from request to final response. Maze went from 0.52 to 0.96, nearly double; Grid from 0.73 to 0.95, roughly 1.3×. The score jumps substantially, yet the time is actually shorter. Quality and speed are usually two metrics you have to trade off — getting more of one means giving up the other, and the best edge of that trade-off is called the Pareto frontier. On these two tasks, though, JevSpawn broke that trade-off and took both. It means there's essentially no method that's better on both quality and speed.
There's a result on the other side, too. LightsOut reward comes in at 0.61, ahead of AgentPrune's 0.34; Nullify at 0.21, over HiAgent and FoldAgent's 0.09; and 2048 at 305.12 points, leaving LLMCompiler's 5.00 far behind. But 2048's latency is 169.94s, much longer than the fastest 33.51s. This is the territory where it paid in time for the score it gained.
And it didn't win every task. AgentPrune is still the strongest on PPNL, RushHour, and Sokoban. JevSpawn's membership in the Pareto frontier on Sokoban is shaky; the paper states directly that this reflects a situation where it's 1.50s faster than LLMCompiler (122.49s vs 123.99s) but scores lower (0.150 vs 0.350) — a case where speed and quality trade off.
They also ran a variant that keeps the structure as-is but swaps only the finite-scoring part for the external TypeSafe Jev API. The results are mixed. The external-API variant beats Qwen's own scoring on some tasks but trails on others. And once you include the API communication, it takes 1.4–2.1× the time of in-house scoring. The takeaway: an external interface isn't always a quality advantage, and on speed it's actually a loss.
On throughput (pure generation speed excluding prefill), it hits 544.35 on PPNL and 580.05 on LightsOut — the highest figures in the table. Prefill is the preparation stage where the model reads in the input context all at once before producing an answer. The interesting part is that the text throughput for Maze and Grid shows up as N/A. Not because it's slow. These two tasks run on finite evaluation from start to finish and barely generate any text. More than the numbers, the key is "where did it strip text generation out entirely?" That's also why speed exploded on navigation.
At this point an honest picture is needed. JevSpawn has no universal switch you just flip on or off. The good setting flips to the opposite from task to task. Whether it's a usable technique or not comes down to whether you can tune that "flip" to your own task.
Search width is a double-edged sword. Width refers to how many branches to fan out at once. Narrow the width to 1 and Maze success rate sinks to 0.16, Grid to 0.58. So you're tempted to widen it. But crank the width to 10 and Maze gets fully solved — while Nullify's reward is cut in half and RushHour's latency doubles. Wide search becomes poison depending on the task.
The same budget flips the result depending on how you split it. Below is a four-action budget divided differently as "number of parents × actions per parent."
| Split | Maze | Grid |
|---|---|---|
| 1 × 4 | 0.88 | 0.94 |
| 2 × 2 | 1.00 | 1.00 |
| 4 × 1 | 0.12 | 0.75 |
Maze is solved perfectly at 2×2 and collapses at 4×1. Yet that very 4×1 extreme split actually raises the 2048 score. It's hard to show more clearly that the formula differs per task.
Whether to fix or revise the declaration, whether to confine observations within a branch or share them across branches — all of it depends on the task. Still, there's one consistent signal amid the chaos. Sharing observations across branches lowered latency on all eight tasks, even though the scores split by task. That's one default worth leaving on.
The time budget and the number of rounds collide head-on. Raise the search rounds from 36 to 108 and the 2048 score jumps from 315.16 to 1102.24. The more it thinks, the better it solves. But put a 300-second deadline on it, and at 108 rounds, 85 percent of the 2048 runs time out. Deeper thinking clashes with the real-world latency budget.
Finally, the assumption underneath this whole method: that "actions share a shape and have finite values." The paper's tasks are grid movement, puzzles, and numeric games, so the assumption fits well. Whether the same gains hold on tasks where actions are open natural language (free-form writing, code authoring) or the argument space is effectively infinite is hard to assert from these experiments alone. The fact that all results came from a single model, Qwen3.8-27B, also remains a homework problem for generalization.
The reason to pick through conditions this carefully is to gauge how we'd realistically plug this technique into our service. Let's follow one concrete scene all the way through.
A customer on an in-progress project asks to move the delivery date up. The AI PM has to decide the next move. Should it re-run the collection work against the revised schedule, first pull a list of affected deliverables, ask the customer back which scope to cut, or notify the internal owner of a reassignment? In the current structure, this decision is drawn down as one long stretch of reasoning text. It's slow, and once it commits to a chosen path and that path goes wrong, the cost of backtracking is high.
Hold JevSpawn's framework up to this scene and the point that changes is clear. The PM's "next move" is in fact a finite, repeating-shape action. It breaks into a few slots: a target (which task, which deliverable), an operation (re-collect, list, query, notify), and a recipient. Instead of writing it out at length in natural language each time, if you declare candidates for these slots and choose by finite probability, there's room to make the same decision more lightly. The narrow action tools ONTLE already uses — a design where word matches action and the server executes deterministically — fit the definition of "actions that share a shape and have finite values" exactly.
That doesn't mean we can lift the paper wholesale. The snag is the second mechanism, branch simulation. The premise of "try candidates in independent copies and then choose" only holds when that execution has no side effects and is reversible. Grid movement can be run in a copy as many times as you like, for free. But a good share of the PM's real actions — external integration calls, queries going out to the customer — have side effects and can't be floated lightly. You can't email the customer four times in a copy. So this structure is better applied first only to reversible actions, like collection-result previews or internal state transitions, while side-effecting actions still go through an approval gate.
Step back from here and the direction this paper points in touches the industry's larger current. Early on, everything from reasoning to action was left to the LLM's free generation. JevSpawn steps back one pace from that discourse and separates things: reversible, deterministic actions go to finite choice, and the parts that need open generation go to text generation. The ONTLE-style application is the same hybrid. The decision of which next move to pick goes to a finite policy; the sentences to send the customer go to text generation.
One line does have to be drawn, though. "A finite policy is cheaper" is a conclusion drawn from grid and puzzle tasks, not a guaranteed fact on our tasks. So the first step of adoption has to be measurement, not expansion. Here's the proposal.
Only when time and success rate actually improve in this small comparison do the numbers from the earlier experiments carry over into our real-world judgment. Until then, "cheaper" is a hypothesis, not a conclusion.
Not guaranteed as-is. The paper's numbers are values controlled to strip out randomness and compare the methods themselves. In operation, temperature, model, and inputs differ every time, so the right move is to re-measure in your own environment before putting it into service.
Fair point. Fanning branches out in parallel is a way of folding more compute into wall-clock time. You have to remember the paper measured total latency under a fixed condition of 4 H100s. If your available hardware is tight, the latency gain from widening the search can shrink.
The abstract lists a GitHub address (https://github.com/Hoyant-Su/JevSpawn). For implementation details and reproduction, see that repository and the paper's appendix.