When you fuse several specialist AIs into one, the knob routing methods usually turn is "which expert gets to teach this problem." Letting each expert's feedback push harder or softer isn't new in itself — prior work such as Open-MOPD already adjusted how much each domain's training counts. What this paper adds is a specific way to set that strength: a batch-by-batch correction driven by the standard deviation of each domain's feedback — and it shows that dialing in just this adjustment improves performance. Across three sizes of Qwen3.5 (9B, 4B, 2B), the authors show the existing label-routing approach couldn't even match the single best specialist, and they propose DN-MOPD, which normalizes feedback strength per domain this way.
Post-training a language model with reinforcement learning (RL) produces a specialist that digs one well very deep. Math is trained by verifying answers, code by executing it, instruction following (IF from here on) by checking constraints — each in a separate pipeline. The catch is that users want a single model that can do math, code, and instruction following all at once. You can't keep three separate experts around and swap them in by situation, so you have to merge them into one.
The established fix is Multi-teacher On-Policy Distillation (MOPD). It works like this. The student model first writes its own answer. If the problem is math, the math expert — if it's code, the code expert — looks at every token the student wrote and hands it a signal: "here's how much I would have used this word." This signal (the gap between the teacher's and student's log probabilities) tells the student which direction to move.
And here's the trap. The routing rule only decides who teaches; it says nothing about how hard that teacher's feedback moves the shared student parameters. Since the three experts come out of three different RL pipelines, the raw magnitude of their feedback is all over the map — and MOPD just adds it up as-is.
Think of it as three tutors taking turns with one student, each at a different volume. The math tutor speaks softly while the Korean (instruction-following) tutor is shouting. From the student's side, regardless of who's actually right, they keep drifting toward whoever's loudest. In the authors' own measurements, the IF expert's feedback was spread several times wider than the overall signal in the very first training batch, while the math feedback was much narrower. As a result, in the early 4B student, even with equal weighting, the IF loss dominated — making up the overwhelming majority of the total gradient.
The outcome was bad, as you'd expect. Label-routed MOPD failed to beat the strongest single-teacher student at any of the three sizes, and it barely transferred the math expert's strengths. Under an 8K evaluation budget only a fraction of the math gains survived, and at 16K the gains over the starting point vanished entirely.
DN-MOPD (Domain-Normalized MOPD) prescribes something simple: turn the loud voices down and the quiet ones up, so every tutor's volume matches the room's average.
Concretely, for each batch it measures how spread out each domain's feedback is (the standard deviation) and multiplies by a scale factor of "overall average spread ÷ that domain's spread." A domain with small spread (math) gets amplified; one with large spread (IF) gets attenuated. The scale factor is bounded above and below, so even extreme cases don't get over-corrected.
Two things in this design are worth noticing. First, it never touches the sign. A signal that was "recommending" a token is still a recommendation — only its strength changes — and a "suppressing" signal keeps its direction too, because only positive scale factors are applied. Second, it leaves routing alone. Who teaches, and how many problems each domain is assigned, stay exactly as they were; only the strength is adjusted. MOPD is just the special case where every scale factor equals 1.
The clever part is that this costs almost nothing. It calls no extra teacher model and builds no new learned router. It reuses the log ratios already produced during rollout scoring to compute a handful of standard deviations and multiplies them into the existing signal. It adds a single line of computation.
The scale factors this rule actually pulled out in the Qwen3.5 experiments reveal the character of the prescription. Math feedback was amplified across all sizes, while code stayed essentially unchanged (a scale factor near 1) at the smaller sizes. The IF factor, by contrast, sat pinned at its lower bound in most batches. In other words, for IF this was less normalization than it was clipping at the bounds.
DN-MOPD lifted the average score across six public benchmarks above MOPD at every size and under both evaluation budgets. The gains were larger under the tighter 8K budget than under the 16K one. The improvement held across three student seeds and two answer-length limits, and all the paired 95% confidence intervals cleared zero.
The tables tell a consistent story: at every size, the six-task total rose from label-routed MOPD to DN-MOPD (each measured against an independently trained expert pool per size). The key point is that the biggest recovery came in exactly the area MOPD had missed most — math. The AIME scores, for example, climbed under DN-MOPD toward the math expert's own level.
The control experiment the authors ran here is what sharpens the conclusion. Running a fixed-weight control shows that most of the gain came not from "turning math up" but from "turning instruction following down." Lowering the loud voice mattered more than raising the quiet one. They also found that using fixed weights close to the ones DN-MOPD measured gave comparable performance at 9B and 4B. The practical advantage of DN-MOPD is that it derives those weights automatically from batch statistics, so you don't have to hand-search a weight per size.
In one sentence: when you merge experts, you have to decide not only who teaches but also how hard to let that feedback push.
A few things to be honest about. First, this is a result from one specific expert pool in the Qwen3.5 family. The abstract and the excerpts available don't guarantee the same imbalance shows up in other model families or arbitrary domain mixes (earlier Qwen3 results used a different teacher pool and protocol, and the paper only notes they're handled in a separate appendix).
Second, the domains are limited to three — math, code, and instruction following. With far more domains, or fuzzier boundaries, it's not clear from these excerpts how stable the per-batch standard-deviation measurement would remain.
Third, a limitation the authors make explicit: matching the standard deviations doesn't pin the batch loss magnitude in place or make the per-domain gradient sizes perfectly equal. The fact that the IF factor is mostly stuck at its lower bound suggests this method "clipped" IF more than it "normalized" it. Full equalization is actually blocked by the clipping.
Fourth, while it does beat the single-teacher student, some of those confidence intervals include zero. So it's premature to claim that fusion is always reliably better than a single specialist.
ONTLE is a service that dispatches AI project managers. Since a single PM agent has to combine many abilities at once — scheduling, document writing, data lookups, communication — this paper's concern hits close to home.
In particular, when you fuse several specialized models or specialized prompts into one agent, it's easy to worry only about "which expertise to call in which situation" (routing). This work raises the next question: when you combine signals from different sources and each signal's strength differs, the loudest one quietly ends up dominating the direction of learning. A situation where the signal that wins is merely the high-variance one rather than the high-quality one can recur in any system that mixes multiple tools, experts, or feedback sources.
For anyone running a RAG or multi-agent ensemble, the practical lesson is this: when you take a weighted sum of signals from multiple sources (retrieval scores, tool responses, evaluator feedback), actually measure the value distribution (the spread) of each source. It may look like you're adding things up fairly, but a difference in variance could mean one side is swallowing the result. An approach like DN-MOPD that auto-corrects from batch statistics is worth keeping in mind, since it spares you from hand-tuning a weight every time you add a source. That said, these numbers are strictly the outcome of one particular LLM-distillation setup, so before porting any of this into our own pipeline, the first step is to re-measure the variances on our own data.
MOPD decides only which teacher does the teaching (routing); DN-MOPD adds a single line of computation on top that rescales each teacher's feedback strength to match its domain's spread (standard deviation). If every scale factor is 1, DN-MOPD is identical to MOPD.
The fixed-weight control showed that most of the gain came not from amplifying math feedback but from reducing instruction-following (IF) feedback. In fact, the IF scale factor stayed pinned at its lower bound in most batches.
Almost not at all. It needs no extra teacher model, no extra teacher calls, and no learned router; it reuses the log ratios already produced during rollout scoring to compute a few standard deviations, then multiplies them into the existing signal.
It used independently trained expert pools for each of three Qwen3.5 sizes (9B, 4B, 2B) and was confirmed across six public benchmarks (AIME25/26, LiveCodeBench v5/v6, IFEval/IFBench), three student seeds, and two evaluation budgets (8K and 16K). Generalization to other model families or to a larger number of domains is outside this paper's scope.