An AI assistant reading a German quarterly report is, in its own head, muttering in English. The text in front of it is German; the thinking is English. That small misalignment is where this paper starts. And it shows, with two dots over a single letter, just how much that misalignment can shake your trust in an AI's answer.
So: if you take a retrieval-based AI whose question, source documents, and answer are all German, and make it think in German too — what happens?
Let me give away the ending. Aligning the thinking language to German produced more accurate answers than forcing it to think in French. The interesting part comes next: on benchmarks this model is actually better at French than at German, and German still won. At least inside this experiment, the gain looks like it came not from "language skill" but from alignment — the material you read and the language you think in being the same. That said, even that gain never got past the model's native English reasoning.
Today's "reasoning models" (models that write out a long chain of thought before committing to an answer) work through a long deliberation before they conclude. That process lifts accuracy, and it opens a window onto why the model answered the way it did. The catch: most of these models were trained to think in English (some in Chinese). For users whose first language isn't English, that's only half the tool.
A prior study (Qi et al., 2025) made a sharp observation first: telling a model to think in a specific language lowers its accuracy — and it dropped even when that instructed language matched the language of the question. But what that study looked at was a "short prompt" situation, with the model holding nothing but the question. Other multilingual retrieval studies mostly varied the language of the question or the documents; none of them isolated the language the model thinks in inside a retrieval setting.
Real services, though, are a different story. A great many services built for non-English users run on RAG (retrieval-augmented generation — the model first retrieves and reads external documents, then answers grounded in them). Here, most of the text the model examines isn't the question — it's the retrieved "target-language material." That's where this paper's question comes from. In a RAG setup where both the question and the documents are entirely German, is a German-thinking instruction still a liability? Or does matching your thinking to the language of the material help this time? And can you tease apart whether any effect is due to "alignment" or to "language skill"? This is the first study to pull out reasoning language as a single variable and test it inside RAG.
There's reason to expect German thinking to help. For a model, a language is a channel for feeling out the world, and every word carries that language's particular grain. When you have to interpret piles of German material, thinking in German keeps your reasoning pressed right up against the material. Think in English and you've slipped in an extra layer of translation.
The paper's opening figure compresses all of this. Answering a German question, the model that thought in English skated over the umlaut on a river's name (the two dots placed above a vowel) and produced the wrong answer, "Sudask"; the model that thought in German kept "Südask." Two dots separated right from wrong. In Korean terms, it's like 검토 ("review") and 검수 ("inspection") getting mashed into each other by a single garbled character during translation.
To measure the effect cleanly, you have to design three things: where you measure it (the stage), how you find and read material (retrieval), and how you grade (evaluation). Let's take them one at a time.
A good experimental stage has to satisfy two conditions at once. First, the material must be in a single non-English language. Second, the model must be unable to solve the task from memory alone, forced to lean on retrieval. The second one matters most: if the model can answer from knowledge it already has, you can't isolate the effect of "reading retrieved material and reasoning over it."
So the stage they chose is Aventuria, the fictional world of the German TTRPG (a tabletop role-playing game where a group builds a story together with rules and dice) "The Dark Eye." It has been documented in German sourcebooks with unusual density since 1984, yet it's a world too narrow and niche for today's AIs to have picked up through pretraining. The three sourcebooks used as the knowledge base run to roughly 1.13 million tokens combined (a token is the smallest unit a model processes text in) — far more than a model can read in one go. So answering requires retrieval.
There's a clever device built in here: French slips in as a control. The model wasn't trained to reason in French, and above all, French is a language different from the German material. That lets you separate "the alignment effect that comes from the material and the thinking being in the same language" from "sheer skill at a given language." If the effect were purely a matter of skill, French — with the higher benchmark score — ought to at least match German.
The RAG machinery itself is standard. A question comes in, the model thinks, then either calls the retrieval tool or produces a final answer.
The heart of the design is "structure preservation." Instead of blindly slicing the book into ten-page chunks, each piece gets a tag that tells it where it belongs — like "Chapter 3, Section 2." A book naturally has a chapter-section-subsection hierarchy, and that structure is carried straight into the retrieval unit (the chunk). That way even a short paragraph titled "Market" doesn't lose its way. On its own, you can't tell which city's market it is; but carry it around together with its parent heading (→ the city of Al'Anfa) and the context comes alive. This "German-language structure" is the key ingredient for the alignment effect that shows up later.
Retrieval runs in two stages. The first converts a sentence's meaning into coordinates (numbers) the AI can work with (this is called an "embedding") and scoops up a broad set of candidates with nearby meaning. Fast, but coarse. So in the second stage, among the candidates the first stage pulled in broadly, only the ones genuinely close to the right answer get carefully re-ranked (this is called "reranking"). Because the first stage gets unusually confused by fictional German material, this second-stage filter is especially handy here.
The workhorse model was a lightweight one, small enough to fit on a single GPU. Feed in long material and repeat retrieval, and a large model's costs become unbearable. The large model was used only as a grader and as a "solve-it-without-retrieval" baseline.
There are two evaluation sets. One is 585 single-pass questions, answerable from a single section. The other is 30 multi-hop questions that can't be solved from one source and require crossing all three books. Here a "hop" means a step you take, jumping across sources, in the search for an answer.
Grading was split in two as well. Single-pass answers were scored 1 to 5 by the larger model (this is called LLM grading); multi-hop answers were scored blind by three experts who know Aventuria, without knowing which setup produced each answer. When comparing two setups, they paired up the questions and ran a statistical test, and they corrected for the illusions that arise from making many comparisons.
Finally, it's only fair to be clear about what this design does and doesn't measure. With the question, material, and answer all pinned to German and only the reasoning language moving, the one thing this experiment takes aim at is: "given the evidence, how does the answer change when you change the language you think in?" Also, the reasoning language wasn't achieved by touching the model's internals — it was steered purely through prompting, by planting the target language in the system instruction (the preamble that briefs the model on its role and rules) and in the first sentence of the chain of thought. That limitation is baked in from the start.
First, let's check the premise. Did the model really not know this world? It did not. The no-retrieval setup stayed stuck in the low 1s out of 5 on both single-pass and multi-hop, and every retrieval setup cleared it by a wide margin (table below). The large model didn't do any better. Rather than inventing things out of nothing, it hallucinated by dragging in entities that really do exist elsewhere in Aventuria and misapplying them. The few answers it got right without retrieval were common sense, not Aventuria-specific knowledge (mangroves grow on prop roots — true in the real world too). The assumption that "the model doesn't know this domain" was confirmed this way, by measurement.
Now the core. Here's the scorecard by setup.
| Setup (thinking language · reasoning) | Single-pass, 585 Qs | Multi-hop, 30 Qs |
|---|---|---|
| No retrieval (baseline) | 1.335 / 1.323 (small · large model) | 1.189 |
| No reasoning | 3.903 | 2.778 |
| Unconstrained (free thinking, in English) | 3.718 | 3.011 |
| Forced English | 3.670 | 3.000 |
| Forced German (same language as the material) | 3.701 | 2.889 |
| Forced French (a different language) | 3.491 | 2.778 |
(Scores out of 5.)
First, alignment won. The German-thinking setup (3.701) edged out the French-thinking setup (3.491) by 0.21 points. Question by question, German won on 182 and French on 138. Both are non-native languages the model was forced into, with the same data and the same system. The only difference is that German matches the language of the question and the material. Yet this model's average benchmark is higher for French (71.8) than for German (68.2). If this were a skill issue, French should have at least tied — and it didn't. At least inside this experiment, it reads most naturally to say the gain came from "alignment with the material's language," not from "skill." The multi-hop set preserved the same ordering (2.889 vs. 2.778), but with only 30 questions there, the honest reading is just that the direction agreed.
That Südask from earlier is exactly what this 0.21 points looks like up close. It may read like an abstract gap in a number, but inside it is the force of keeping your thinking fastened to the language of the evidence — enough to hold onto even two little dots.
Second, the richer the structure, the smaller the cost of forcing a language. Switching from uniformly chopped chunks to chunks that preserved the section structure, the unconstrained setup that just thinks in English rose the least, and the German-thinking setup rose the most. With uniform chunks, German thinking was last within its group; with section chunks, it even pulled ahead of English thinking. The setup that made the most of the German-language structure turned out to be the one thinking in that very language.
Third, it still didn't beat English. On single-pass, unconstrained English (3.718), forced German (3.701), and forced English (3.670) are a statistical tie. German thinking only "reached the level of" the model's native English thinking; it didn't pull ahead. On multi-hop, forced English (3.000) is actually a touch above forced German (2.889). Steering through prompts narrows the gap but doesn't flip it. The authors see the real fix not in prompt manipulation at inference time, but in the multilingual capability itself — a model built from the ground up to think in the target language.
Fourth, reasoning isn't free. On single-pass, the setup with reasoning turned off entirely came in first overall at 3.903. For a question you can crack in one shot, a long chain of thought mostly just adds noise and drags in irrelevant tangents. The reverse held on multi-hop: no-reasoning dropped to last among the retrieval setups (2.778). Reasoning only earns its keep on questions that demand weaving several sources together.
The two datasets show only the two extremes — "harmful at one hop, helpful at multi-hop" — and everything in between is still a blank. So the direction this observation points in is clear: "routing" that decides, case by case, whether to turn reasoning on based on a question's complexity. Simple lookups served fast without reasoning; composite analyses that weave several pieces of evidence served with reasoning on — where exactly to draw the line is left as future work.
Finally, the authors doubted their own grader. One author hand-regraded 50 answers (five per setup) and compared: exact agreement 66%, within one point 90%, off by more than two points 10%. Meaning: enough to settle the "ranking" of setups, but not enough to split a razor-thin gap like 3.718 vs. 3.701. On top of that, the disagreements leaned one way. Of the five cases off by two or more, four were answers a human scored 3–4 that the LLM grader gave the lowest score of 1.
The authors name several limitations themselves. One model, one language pair (German–English/French), reasoning-language control resting entirely on prompts, a multi-hop set as small as 30 questions, a grader that — as we just saw — isn't perfect, and retrieval pinned to a single method. In particular, finding the boundary line for "when to turn reasoning on and off" would require a benchmark that grades questions by hop count, and the authors state outright that no such data exists yet.
The size of the alignment effect (0.21 points out of 5) deserves a balanced read too. The direction is consistent and statistically significant, but read alongside the authors' own confession that "a razor-thin gap is beyond the grader's resolution," the honest view is that the alignment payoff is "clear, but not large."
Earlier, on the scene of "Südask" getting mashed into "Sudask," I added that in Korean it's like 검토 ("review") and 검수 ("inspection") getting tangled. This is exactly the problem that shows up when ONTLE's AI PM answers grounded in a client's Korean material. When a client asks, "Pull together the requirement-change history from last quarter and the exception clauses we agreed on at the time," the PM retrieves and reads and weaves together Korean documents — meeting minutes, requirement specs, decision logs — and answers. A textbook monolingual RAG.
What this paper gives that scene isn't a "lesson" but "items to go verify." Let's narrow it to three things to measure directly in Korean.
One, does matching the thinking language to Korean actually help? The moment you read Korean material while thinking in English, a layer of translation wedges in. A single particle flips the meaning (예외는 "the exception" vs. 예외도 "even the exception"), and an ending splits settled fact from mere plan (합의됐다 "it was agreed" vs. 합의하기로 했다 "they agreed to agree"). Let 검수 완료 ("inspection complete") and 검토 완료 ("review complete"), or 반려 ("rejected") and 보류 ("on hold"), get mashed once in translation, and the whole credibility of the PM's reply is shaken. So compare, side by side on the same Korean material, setting the reasoning language to Korean versus leaving it in English. This isn't a conclusion that "Korean is uniquely fragile" — it's a hypothesis to measure first.
Two, does passing the document structure along make it more accurate? Instead of chopping meeting minutes into uniform pieces, carry the "agenda → decision → exception" hierarchy along with the chunks, and see whether the alignment effect comes through more strongly.
Three, when do you turn reasoning on? Split lookup questions ("when was this clause agreed?") from synthesis questions ("weave the change history and the exceptions together and analyze the impact") by hop count, and test whether you can set a threshold for turning reasoning on and off.
Brace for one thing in advance. Even if you do all three, lifting Korean quality all the way with a single line of prompt will be hard. If it goes the way this paper showed in German, the last inch is likely to be decided by the multilingual capability a model was built with from the start — whether it was shaped to think in Korean. But even this stays a forecast, not a verdict, until it's confirmed directly in Korean.
When you do run the verification, follow this paper's method too. Don't lean on LLM grading alone for the scores — especially not for close calls — run a blind evaluation by people who know the domain in parallel, and don't assume "our domain is already known to the model"; measure retrieval dependence first. Just remember that even a large model without retrieval fabricated plausible entities.
Because it isn't a study aimed at Korean. For the experiment to hold up, it needed monolingual, non-English material the model couldn't solve from memory, and a German TTRPG sourcebook simply happened to fit the bill. So whether the same result carries over cleanly to Korean is something this paper doesn't answer. You have to measure it yourself, on your own data.
The processed German knowledge base itself is private, but the tool that builds that knowledge base from the original PDFs, along with the QA benchmark and the system code, are released. Anyone who has the original sourcebooks can run the tool, obtain the same knowledge base, and reproduce the results. Flip that around and it's also an accessibility wall: without the originals, you can't match it exactly.