You've probably told a chatbot "find me a good restaurant nearby," then a few turns later changed your mind: "actually, make it a quiet cafe instead." How much should the chatbot hold onto, and what should it throw away? A human knows "restaurant" is a dead request. Yet an LLM agent (an AI that carries out tasks on your behalf) will often cling to a request you already dropped and bake it into the final answer. The researchers call this failure intent drift — the phenomenon where stale intent, left over from a conversation that has moved on, contaminates the result.
This is the first work to measure that failure properly. It quantifies the problem with IntentFlux, a benchmark that turns verifiable tasks into controlled conversations (here, a benchmark means a standard set of exam problems for measuring performance), and recovers part of the loss with StateForge, which explicitly pins down "the requirements that are valid right now" just before generation.
Real conversations don't arrive in one clean package. The user asks for something, later fixes one value, retracts one constraint, and finally says "okay, go with that." Across this flow, the agent has to figure out on its own which fragments are still live requirements. When a fragment that has already been replaced or withdrawn keeps influencing the final answer or a tool call, that's intent drift.
It's worth clearing up a common misconception here. Intent drift is not "the model forgetting the conversation." If anything, it's the opposite. It's a failure where the model vividly remembers and actively uses information that should have been discarded, and gets the result wrong because of it. It's misuse, not amnesia.
Earlier work on multi-turn (back-and-forth) conversation evaluation has brushed up against this problem. There's research showing that drip-feeding information in fragments causes models to lose the task (Laban et al., 2025), and an evaluation that adds argument reveals, revisions, and task switches while keeping the original grader (Tack et al., 2026). But these only check whether the agent "eventually followed the changed task" — they don't isolate the failure where stale intent ruins the answer from other multi-turn errors. Measuring that cleanly requires two things at once: you have to know for sure what the final intent is, and you have to design the task so that misusing stale information actually changes the outcome.
The idea behind IntentFlux is simple. Take tasks that already come with a ground-truth grader (code, math, SQL, data-to-text, summarization, tool use), and decompose each task into a conversation in which the intent changes. The key is that the original grader is preserved as-is. No matter how tangled the conversation gets, the correctness criterion is still the one from the original task — so if the score drops, you can say it's because of the conversation structure, not because the task got harder.
The way stale information is planted is clever. There are two kinds; let's start with examples.
That last condition matters. Because only constraints that directly affect grading — not ones that are merely verbally off — are retained, if the agent is still clinging to a stale decoy it will inevitably show up in the score. It turns "did the agent use the stale state or not?" into something testable via whether the task succeeds.
The measurement design is clean, too. The same final task is compared across two paired conditions: (1) a clean condition where it's handed over directly in a single turn, and (2) a drift condition where it has to be reconstructed from a conversation in which the intent changes. The gap in full-correctness rate (the fraction solved perfectly against the true current intent) between the two conditions is the Intent-Drift Gap (IDG). To rule out the possibility that simply having more turns is the cause, they also include a control group that only adds turns without any intent revision.
The prescription is StateForge. As the name suggests, it forges the state. It separates the job of solving the task from the job of managing intent: a separate tracker model estimates the "state that is valid right now" by filtering out discarded intent items and any conclusions derived from them, then explicitly appends that valid state to the existing conversation history and hands both to the execution agent. It subtracts discarded information from the state — it does not strip the conversation history out of the execution model's full input. In effect, it puts a fresh note of "the requirements that are valid right now" on the desk before the model starts composing the answer.
I'll cite only the numbers stated in the abstract and the body. The key results are threefold.
| Experiment | Condition | Mean task score |
|---|---|---|
| 627-task calibration pool | easy → hard | 0.476 → 0.384 |
| General-Test (125 tasks) | base multi-turn → StateForge | 0.367 → 0.467 |
First, the more stale information piled up, the worse the score got. Across 627 tasks, raising the difficulty so that the conversation carried more discarded and withdrawn information dropped the mean score from 0.476 to 0.384. That said, this experiment raised the "stale-information load" as a whole rather than pulling apart variants, decoys, and length one by one — so while you can say "more stale information means worse performance," this number alone can't tell you how much each factor contributed.
Second, the drop came not from the conversation being long but from the intent changing. Across all eight recent LLMs, for the very same final task, the full-correctness rate was clearly lower when it had to be reconstructed from a conversation (drift) than when it was handed over in a single turn (clean). The control group that only added turns without any intent revision, by contrast, performed similarly to the single-turn case. Because the grading criterion was pinned to the original task and a length control was examined alongside, you can say fairly strongly that this drop comes from the intent change, not from task difficulty or turn count.
Third, it was fixed — but not completely. Let's address the natural objection first: "couldn't you just summarize or compress the conversation history?" The paper tested two external aids that only shrink the history (rolling summarization, two-stage compression), but the loss wasn't recovered reliably. Compression reduces "what's left," but it doesn't solve "which of what's left is still valid." StateForge takes on exactly that validity judgment explicitly, lifting the General-Test mean from 0.367 to 0.467. And here a further diagnostic is interesting. Even when the estimated state was replaced by injecting the ground-truth final intent directly, performance rose further but still did not recover to the single-turn clean score. That means even when you tell the model the valid requirements perfectly, if the messy conversation history is still sitting right in front of it, the model still wavers. State-estimation error was an "important part" of the loss, but not all of it.
There's one practical finding, too. The role of managing state could be handled by a model far smaller than the main one solving the task — the paper reports that state-keeping can be "distilled" down to a smaller tracker. It's a signal that when the role is narrow, a lightweight model can be enough.
The authors are candid. IDG only captures "the total loss from reconstructing the final intent out of a conversation" — it doesn't separate out which of variants, decoys, or length is the culprit one by one. The fact that both the simulator playing the user role and the judge grading the answers were handled by a single model (DeepSeek-V4-Flash) also leaves questions about validity. There's a quality-control figure showing that in a post-hoc check the annotated final intent could be recovered for 95% of General-Test tasks, but how closely the simulator-generated conversations resemble the texture of real user revisions is a separate matter. Above all, StateForge is a partial mitigation. Even when the ground-truth state is supplied, a residual gap to the single-turn case remains, and the nature of that residual is still an open question.
This is exactly the failure that ONTLE's AI PMs run into all the time. Picture the request board.
On Monday a customer writes, "hook up an Excel collector, based on Naver Shopping." On Wednesday they add, "actually, not Shopping — switch it to Cafe24 order data." On Friday they retract a constraint: "let's drop that daily notification for now." When the PM moves into execution over the weekend (creating dev tasks, deploying), the valid spec is "collect Cafe24 orders, no notification." But an agent that read the entire long conversation history wholesale might resurrect the first turn's "Naver Shopping" (a Variant) or the already-retracted "daily notification" (a Decoy) and spin up the wrong tasks. From the customer's side it becomes "I clearly changed that — why did you go with the old one again?" and trust breaks.
This research changes that scene at two points. First, right before execution, it explicitly establishes the "currently valid requirements" state, strips out the stale conclusions derived from them, and only then generates tasks. Rather than simply summarizing the request board's comment thread, it folds it into an active spec that reflects every replacement and retraction. Second, that state tracking can live in a small module separated from the execution agent. The finding that state-keeping can be handed off to a smaller tracker suggests that, within ONTLE's company-wide model routing standard, a design that dedicates a lightweight model to the narrow role of state-keeping can be cost-sensible.
We also take to heart the warning that a residual gap remains even when the ground-truth state is supplied. Even if the PM hands over a perfectly organized valid spec, the execution stage can waver again if it's still hugging the entire conversation log. So ONTLE won't stop at establishing the valid spec; it will shape its next AI PM architecture toward narrowing the context handed to the execution agent itself around the valid state.