A tiny 0.5B specialist that picked up math reasoning through reinforcement learning can hand that skill off to a model many times its size — a 14B model (B is for parameters, meaning the model has a billion numerical knobs to turn). And it can do so to the point where the larger model eventually overtakes its teacher. This paper lays out just how high and how fast that "skill transfer" happens, framed as power laws (rules in which, as one quantity grows, another changes with it at a fixed proportion). The crux: given the sizes of teacher and student plus a single number for the teacher's skill, you can predict two things — the peak performance you will reach and the early transfer speed — before running a single training step. The caveat: the experiments are confined to one model family, Qwen2.5, and one domain, math reasoning.
Today's reasoning models are pushed to higher task-specific skill through reinforcement learning (RL). The trouble is that this process is expensive. If a model family comes in five sizes, from 0.5B to 14B, then in principle you have to redo RL at every size. Each run burns fresh power, time, and data.
The detour that emerged to avoid this is on-policy distillation (OPD). You take one expert (the teacher) grown once through RL, and the remaining sizes (the students) inherit the ability through token-level guidance from that teacher (a token is the smallest unit a model reads and writes — roughly a fragment of a word). The key is in the word "on-policy." The student is not made to copy the teacher's model answers verbatim. Instead, the teacher grades, token by token, the work the student produces on its own (a rollout).
To put it in analogy: this is not the coach who loops a swing video and makes you mimic it exactly, but the coach who watches the swing you actually took and points out, on the spot, "your wrist broke here." The former forces a form fitted to the coach's body; the latter gives corrections fitted to yours. That difference starts to matter when the student model is in a different weight class from the teacher.
But until now this distillation was a "you only know once you run it" affair. There was no law describing how the outcome shifts with teacher and student size. The authors' question is simple: before running OPD, can you predict the outcome from just the scales of teacher and student plus the teacher's score?
The starting point is borrowed from earlier work. When optimizing a reward model trained with PPO, there was an observation that performance (the gold reward) moves regularly if you plot it against the KL divergence from the initial policy. This paper applies the same frame to OPD.
Two coordinates are central here. One is the gold score (G) — the accuracy on a held-out test set, i.e., actual skill. The other is training progress, which measures how far the student has drifted from its starting state. This is gauged by the square root of the token-level reverse KL divergence (d). KL divergence is a measure of how different two probability distributions are; here it is used as a "magnitude of change" indicator for how much the model has shifted relative to where training began. KL itself tends to grow like the square of a distance, so you have to take a square root once to get a clean axis that is proportional to "how far it actually moved."
The regularity the authors found in this coordinate space is the backbone of the piece. Early in OPD — whether the teacher is weak or strong, whether the student is big or small — there is a common stretch in which the gold score climbs almost linearly with d. They call this the "useful-transfer" regime.
Why linear? A walking analogy makes it easy. Take one very small step (a parameter change) and skill (the gold score) grows honestly in proportion to the step size. But the distance opened up from the starting point (KL) grows in proportion to the square of the step size. So if you plot the score against 'KL (distance squared),' the relationship is hard to see, but if you plot it against 'the square root of KL (the distance itself),' a clean straight line emerges early on (a local-geometry argument, Appendix D).
This linear stretch has a slope (the transfer speed) and an endpoint where the straight line gives out (the authors call this coordinate d_transfer). Crucially, this endpoint is not the same as the peak: the paper separates d_transfer from the location of the best performance (d_peak), noting that the score can keep climbing gently even after the linear region ends. The two quantities the authors actually fit as power laws — in the student parameter count, the teacher parameter count, and the teacher's measured gold score — are the early transfer speed (the slope) and the peak performance (the highest gold score reached). This is the machinery that makes "predict before you run" possible.
The experiments were run on five sizes of the Qwen2.5 family (0.5B–14B) on math reasoning problems. The teacher-student combinations number 25, spanning weak→strong, same-base, and strong→weak.
Every model first learned basic instruction-following on a portion of Dolci-SFT, and the teachers were made by taking them through GRPO reinforcement learning on a 14.8K training set mixing GSM8K and MATH. Evaluation used a 6.3K test set. For comparison, alongside the baseline (Vanilla-OPD) they included Delta-OPD, which uses the teacher's RL-induced policy change as the reward, and off-policy distillation (OffPD), which copies the teacher's demonstrations.
What makes all of these results more meaningful is that they were predictable from a single set of power laws. For instance, the 'weak→strong' transfer in which a small specialist teaches a larger student actually worked — across every observed weak→strong pair, the student's peak score surpassed the teacher's own. The important part is that even how much it happens and how fast could be computed from nothing more than the teacher and student parameter counts and the teacher's score.
The results show an interesting interaction. Bulkier students like the 7B and 14B did better with bigger teachers too. The 7B student's peak rose from 72.7% to 81.9% as the teacher grew, and the 14B student's from 77.7% to 87.3%. But for a small student like the 0.5B, a teacher far above its own weight class was actually poison. The 0.5B student peaked at 40.8% with a 3B teacher, then slid backward to 39.0% with a 7B teacher and 37.7% with a 14B teacher.
The fitted law compresses this phenomenon. Peak score benefits from a larger teacher only up to roughly the student's own scale. And comparing teachers with the same gold score, the smaller teacher taught better. In other words, a teacher's score alone does not fix its "teaching value." This peak law held to within one accuracy point even for the largest held-out scale (the combination deliberately left out when fitting the law).
The findings on design choices are practical too. Delta-OPD had a steeper early slope in 15 of the shared pairs and a higher observed peak in 12, and that advantage clustered in the weak→strong pairs. There is a reverse direction as well: inserting an SFT phase that copies the teacher's demonstrations before distillation (an off-policy cold start) degraded OPD performance more the larger the weak→strong gap. This is circumstantial evidence lining up with the reading that the value of on-policy guidance lies in "correcting the student's own work."
Finally, one result that defies intuition. The "bootstrapping" chain — raise a mid-sized model with a small teacher, then make that the teacher to raise a bigger one — never once beat direct transfer distilled straight from the smallest specialist. This held even though the intermediate teachers scored higher gold. It dovetails with the earlier law that "a higher-scoring teacher is not a better teacher."
The scope is narrow. The experiments are confined to one Qwen2.5 family and one domain, math reasoning (GSM8K+MATH). Whether the same laws hold for code, general dialogue, or other model families is not something this paper answers. That is also why the paper's title carries the phrase "same-family."
Also, while the early useful-transfer regime is regular, the dynamics afterward are messy. Slowing improvement, saturation, and regression appear mixed together with no common functional form. Nine of the 25 trajectories broke away from the early linear prediction within the observed range. In the regression regime, the gold score falls even as the proxy score the teacher provides keeps rising — a textbook sign of over-optimizing a proxy metric. In short, "when to stop" still leans on observation rather than prediction. The value and speed of the peak were captured by law, but the location of the peak (the d value) is erratic enough that the authors did not force a law onto it.
ONTLE dispatches an AI PM to every client project. Behind that PM run models split by role — one for intake, one for execution, one for revision. Pull out one scene: the moment you have to tune the PM's "execution model" to a new client's domain (say, the request-form classification and processing rules of a particular industry).
The naive approach so far is to RL the large model used for execution directly on that domain. It is expensive, and it repeats per project. The alternative picture this paper shines a light on is this: compress the domain expertise into one small, cheap model via RL (that is the teacher), and have that specialist correct the large execution model's own work, letting only the ability flow across. The result that students overtake teachers in weak→strong transfer promotes "teach a heavyweight worker with a single small specialist" into an option worth running as a head-to-head experiment first.
This is where the paper's "laws" shine. They can turn operational decisions — teacher selection, cost allocation, pipeline design — from gut feel into data-grounded prediction. That said, because the experiments are tied to one domain, math, we hold this for ONTLE's own work data as a hypothesis to validate, not a settled directive. So for a first project there are three candidates to run side by side: (1) the existing approach of RL-ing the large execution model directly, versus small-specialist→large-model distillation; (2) whether the law that "at equal scores, a smaller teacher wins" also holds in our domain (cutting the teacher's weight class near the execution model versus overshooting it); and (3) bootstrapping that stacks intermediate models as stepping stones, versus direct transfer that goes straight from the small specialist. The fact that the peak law held to within one point even at held-out scales gives hope that much of this comparison can be narrowed down by prediction, without full training.
As with the golf-swing analogy in the body, it is a method where the teacher corrects, on the spot, what the student solved on its own. The copying method (off-policy distillation) has the student follow the teacher's finished solution verbatim. This difference matters especially when student and teacher are in different weight classes.
No. The condition is "at the same gold score." Comparing two teachers with identical scores, the one with fewer parameters transferred better — it does not say lower performance is fine. The point is: don't judge teaching value by a teacher's absolute score alone.
The paper does not guarantee it. The experiments are confined to one Qwen2.5 family and one domain, math reasoning. The "same-family" in the title spells out that scope. Generalization to other domains or families is an open question that needs further validation.
Partly. The slope and peak value of the early transfer regime are captured by law, but the exact location of the peak (the KL-square-root coordinate d) is erratic. After the peak, improvement, saturation, and regression are mixed with no common pattern, so the stopping point still has to lean on observation.