"Just give me a short answer."
You can nail it down like that, and the LLM still leads with "Well, this is a problem where you add two numbers…" before getting to the point. Flip the off switch, or sternly order it not to reason — same result. The paper arXiv 2610.11765 gives this behavior a name: Thinking Inertia. You want only the answer, but the explanation trails along like momentum.
One thing is worth settling up front. This paper does not peer into what the model computed internally. It measures only the explanation that is visibly emitted on screen (whether before the answer or after it). So from here on, every mention of "it stopped / failed to stop reasoning" is about text printed on the screen — not about what happens inside the model. "The explanation disappeared on the surface" and "the computation stopped inside" are different questions, and all this work can speak to is the former.
For the last few years, the key lever for pushing LLM performance has been "make it think more before answering." From chain-of-thought (a prompt that makes the model write out its reasoning step by step before giving an answer) to test-time scaling that spends more tokens on reasoning. The more tokens spent on intermediate computation, the higher the accuracy across many tasks. The price is cost. Reasoning tokens inflate both response time and billing. So, in the opposite direction, research on reducing reasoning has also picked up.
But what, exactly, counts as "turned thinking off"?
Prior work leaned on three proxies (a proxy is an indirect signal you use when you can't measure the thing you actually want directly). (1) Did you switch the prompt to no-think mode? (2) Is there no separate reasoning block like <think> in the response? (3) Is the output short? The paper argues all three break easily. Turn off thinking mode and the model can simply pour its reasoning into the ordinary answer field. A single short output might have the core reasoning compressed inside it. Conversely, a long output might be filler with no substance.
It's like an exam proctor saying "don't show your work, just write the answer," and a student jotting the calculation in tiny letters in the margin next to the answer box. Grade by the answer box alone and you'd mark it "answer only," but the reasoning is right there in the margin. The old metrics look only at the answer box; this paper reads the writing in the margin too.
So the location of measurement shifts. Where the old metrics judged by surface conditions (what prompt was it, is there a <think> block, is the length short), this paper judges by the question-relevant content the response actually reveals. Seen this way, prior work that claimed to "reduce reasoning" by cutting length or toggling a switch may in fact have only changed the form of the reasoning and moved it into the ordinary field. It amounts to redefining no-thinking not as a model setting, but as a visible response behavior.
Here's one number to hold on to. Even in the thinking-off setting, DeepSeek-V4-Flash (284B) kept visible reasoning in 99.9% of its free-form responses. The switch was down, yet virtually all of them still exposed their work. To understand why that line comes out, we first have to see how "exposed reasoning" was counted.
The starting point is simple. Any LLM response is split into two pieces: the final answer, and all the text that came before it. The latter piece is called the pre-answer trace. The reasoning hides right there, in the pre-answer trace.
But a non-empty pre-answer trace isn't all reasoning. To the same question "What is 237 + 568?", a model could answer any of these:
There are three metrics, and each looks at a slightly different place. ETR checks whether the whole response held to "answer-only output," EIR peels off the pre-answer trace to see whether there's reasoning, and QRel. uses the pre-answer trace by default but, when the answer comes first and the explanation follows, folds that trailing text into the relevance judgment too (Appendix E).
Empty-Thinking Rate (ETR) — is the whole response answer-only? If the output strictly contains only the answer, it scores 1. If a single line of explanation is attached, before the answer or after, it's a violation. Among the examples above, only A passes. D fails: even though the answer came first, reasoning is tacked on afterward. Had we looked only at the pre-answer trace, D would have been miscounted as "the pre-answer is empty, so it passes." That's exactly why ETR looks at the whole response.
Question Relevance (QRel.) — how closely does the revealed explanation stick to the question? It measures, on a continuous scale, how semantically close the question-relevant text revealed alongside the answer is to the question. The pre-answer trace is the default, but when the answer comes first and the explanation follows, that trailing text is also included in constructing the visible text (Appendix E). One caveat the authors nail down, though: this is "relevance," not "reasoning." A relevant remark with no reasoning, like B, can also score high. So QRel. stays a supporting signal.
Explicit Inference Rate (EIR) — is there genuine reasoning in the pre-answer trace? This is where B and C part ways. An identity-masked LLM judge is handed only the pre-answer trace and asked to decide whether it's actual reasoning. If it engages the question through a process — deduction, calculation, comparison — it counts as reasoning, even if short, shallow, or even wrong. C qualifies here.
Overlay the three metrics and the responses that "looked answer-only" split into three kinds: truly answer-only (A), relevant but not reasoning (B), and reasoning-exposed (C). A distinction that a single proxy would have smeared together.
For QRel., an instruction-aware embedding model (Qwen3-Embedding-4B) is given the instruction "find text containing reasoning relevant to answering the question," then encodes the question as the query and the revealed question-relevant text (including explanation appended after the answer, Appendix E) as the candidate document, computing cosine similarity (a measure of how aligned the semantic directions of two texts are). The encoder uses last-token pooling with a maximum sequence length of 32,768. EIR's primary judge is GPT-5.5 at temperature 0, forced to pick one of five labels: boilerplate/off-topic, paraphrase-only, relevant but non-reasoning, explicit reasoning, or unclassifiable. Only "explicit reasoning" scores 1. All three metrics are aggregated over the full output set — an empty pre-answer trace counts as 0 for QRel. and is included in EIR's denominator too.
The measurement is built on three axes: models, intervention methods, and benchmarks. There are six models, from small to frontier-grade, and to avoid judge bias, verdicts were cross-checked with models from different families.
Six intervention modes. They escalate the strength of the no-think pressure from weak to strong. It runs from the baseline with thinking on (Think-On), through the switch turned off (Think-Off), up to a mode that keeps it off and strongly demands "answer only" (Strict). The final one, M6, is different in kind. Instead of asking via instruction, it pre-fills the start of the response so the model continues straight from the answer. Rather than telling it in words, it draws the starting line over on the answer's side.
Six benchmarks. The crux is the answer space (the range of answers the model can choose from). They're arranged so the answer space widens progressively — boolean (yes/no only), then multiple choice (pick from candidates), then free-form (write it yourself with no candidates).
| Type | Answer space | Example tasks |
|---|---|---|
| Boolean | Narrowest (yes/no) | BoolQ, StrategyQA |
| Multiple choice | Medium (pick a candidate) | MMLU, MMLU-Pro |
| Free-form | Widest (generate directly) | GSM8K, MATH |
The results involve two kinds of comparison. One is across task types — how reasoning exposure differs across boolean, multiple choice, and free-form. The other is the same question with only the format changed — whether the model behaves differently when the same content is asked as free generation versus a format that shows candidates. Keeping these two apart in advance keeps the results that follow from getting tangled. The metrics are task accuracy (Acc.) plus the three above.
Six models: Qwen3 4B / 32B / 235B-A22B (to observe change by scale within one family, transparently controlling on/off locally), DeepSeek-V4-Flash (284B), and OpenAI and Google frontier models. The selection criterion was "models you can turn on and off with an explicit switch, without inferring whether reasoning happened from output length or format." Intervention modes: M1 Think-On (official thinking interface on), M2 Think-Off (only enable_thinking off, no output constraint), M3 CoT (off, with "think step by step"), M4 Soft (off, a short lead-in allowed but no explanation), M5 Strict (off, answer only with no reasoning), M6 Prefix (`The answer is \boxed{` pre-filled at the start of generation). Cross-validation re-ran with Claude Opus 4.8 in Appendix D to confirm the conclusions aren't tied to one particular judge.
Read the findings in order — switch → answer format → accuracy — and they connect into a single story.
First, the no-think switch is not a reliable control. The 99.9% from DeepSeek-V4-Flash (284B) seen earlier is the representative figure for this finding. The switch was down, yet nearly every free-form response exposed reasoning. Having an off button doesn't mean it turns off.
Second, reasoning exposure is governed more by "answer space" than by the switch. This is the heart of thinking inertia. Native or strict mode, EIR was lowest for boolean, middling for multiple choice, and highest for free-form. The wider the answer space, the more the model pours out its work (comparison across task types).
The same direction showed up within a single question too. The same question's reasoning exposure varied by answer format. When the question was factual or in a format showing candidates, "answer only" came far more easily than in open-ended generation. The authors' observation is that this is a pattern not explained by question difficulty or answer length alone (comparison of format changes on the same question). The two comparisons point to the same place: what stops the reasoning is the format of the answer itself, more than an instruction to "stop."
Why? From here it's hypothesis, not observation. Two lines of explanation are possible. One: given choices, the model's burden of inventing answer candidates itself drops, so it unfolds less reasoning. The other: internal computation stays the same, and only the channel for pushing it out to the output closes. This paper didn't look inside, so it can't decide between the two. What measurement confirmed goes as far as "change the format and the behavior changes"; why the model behaves that way remains an open question.
Third, in free-form, a trade-off surfaces. The more strongly you demand answer-only, the higher the "answer-only" compliance and the lower the reasoning exposure — but accuracy can drop. Across five LLMs with thinking fully off, a strict answer-only prompt pushed average ETR from near zero up to about 40%, at the cost of lowering average accuracy by about 15 percentage points (the bundled figures for free-form tasks, thinking off, strict mode). "Answer-only, accuracy unchanged" is not free. On top of that, tasks the model solves well didn't expose less reasoning either. The ability to reason well and the ability to stop reasoning are separate things.
Cost should be read carefully. The intro said reasoning tokens inflate cost, but what this paper directly measured is "the amount and kind of explanation revealed on screen" and accuracy. It did not measure output token count, response time, or internal computation. So don't jump straight from "less explanation = less cost." There is room for less explanation to mean fewer output tokens and thus lower cost, but that connection is yours to measure directly in the field.
The measurement framework itself carries dependencies. EIR leans on an LLM judge. The authors defended with identity masking, a five-label rubric, a cross-family judge (Claude Opus 4.8), and human verification (five annotators reached, on average, 98.8% binary agreement with the primary GPT-5.5 judge on the explicit-reasoning label). Even so, a judge-based metric isn't perfectly neutral.
Prescription is the next assignment. This paper showed that thinking inertia exists, but how to fix a model so it "stops when told to stop" is a problem beyond measurement. It only put numbers on the trade-off; training and decoding techniques that bend that curve favorably are left as follow-up work. And the figures come from six benchmarks and six models, so whether they generalize to other domains or a broader model family can't be asserted within this scope.
All you need is to pick the responsible team, and the model writes out its whole rationale for the assignment: "This request relates to the payment flow, and therefore…" It's a familiar scene at the triage (sorting-and-routing) stage, where incoming work is quickly split and handed to the right person. Usually you want this stage to run fast and cheap, so you leave the model in no-think mode or instruct it to "give the classification result only, no explanation."
This paper flags two cautions for that common sense. If the triage output format is open, like "summarize in one sentence and assign an owner," the model keeps exposing its process even with the switch down. And if you force answer-only, classification quality can drop — just as in free-form — and misclassifications can rise. In trying to build a fast path, you end up adding rework that humans have to undo.
So we rework the design like this. Instead of leaving triage as free summarization, we give owner candidates as choices and make it a closed format that falls back to "needs review" when ambiguous. The point is to narrow the answer space so less inertia arises. And we measure before and after side by side — misclassification rate, processing time, reassignment rate. We check the logs for whether closing the format really made it faster, and whether misclassifications rose in exchange. Whether the trade-off the paper saw on free-form benchmarks takes the same shape in our triage is something you only know by measuring it directly like this.
This design and verification is exactly what an ONTLE AI PM does. Define the owner categories and the exception criteria (what counts as ambiguous and gets escalated to a human), separate the fast path (closed format) from the deep path (open-ended with thinking on), and confirm the effect of the change in numbers. Rather than forcibly breaking thinking inertia, the move is to route around it with an interface where less inertia arises, and verify the result with metrics.