When you teach a small language model one new skill after another, two things go wrong at once. The skill you are trying to add doesn't take hold (acquisition failure), and the things the model used to be good at slip away (catastrophic forgetting). The smaller the model, the harder this tug-of-war pulls.
LG AI and KAIST's SynthSelf plays what looks like a counterintuitive move here. The correct examples a small model is meant to learn from get rewritten — by a model one size up from the same family — into a form the small model already finds familiar. The small model only trains on the result. Why rewrite a correct answer instead of handing it over as is, and why farm that rewriting out to a larger model rather than letting the small model do it itself? Those two questions are the spine of this piece.
Continually stacking new skills onto a model that has already finished training is called continual post-training. The idea is to tailor a model to a particular industry, or to let it learn a little at a time over a lifetime, the way a person does. The obstacle is the acquisition-versus-retention pull described above. In particular, when you feed new data through supervised fine-tuning (SFT, which has the model copy the correct answer verbatim), performance on the original tasks tends to drop hard — catastrophic forgetting. It's a long-reported phenomenon.
SDFT (self-distillation fine-tuning) sidestepped this by changing the learning target. Instead of the raw correct answer, it trains the model toward a version the model has rewritten in its own style. The content stays; only the phrasing is moved into a frame the model finds familiar. This reduces forgetting, and in practice SDFT beat prior methods on both acquisition and retention.
The catch is in the premise. For SDFT to work, the model needs enough in-context learning (learning from examples shown in the prompt) and instruction following to carry out the rewrite. Large models have both; small models are weak at both. So when you ask a small model to do the rewriting directly, quality collapses. The paper measured it: data rewritten by a small model dropped to 60% average accuracy, and the chain-of-thought (CoT, the step-by-step reasoning that leads to the answer) sometimes vanished entirely (8.2% in the tool-use domain). The method spins its wheels precisely on the small models that matter most for low-latency, on-device use.
SynthSelf's fix takes the rewriting out of the small model's hands.
Step 1: the larger model rewrites. The rewriting is handed not to the student itself but to the next size up in the same family. The paper calls this model the aligner. Rather than telling a rookie who can't yet write well to fix their own prose, picture a senior colleague one rung up — same house style — leaving the content alone and just smoothing the sentences. If the senior is too far away — a giant model from a different family — its habits differ enough that the writing ends up feeling foreign to the student. So you pick the minimally larger model within the same family.
Step 2: filter by the correct answer. Instead of generating data on the fly during training, you build it offline ahead of time. Building it in advance lets you check that the rewrite didn't break the correct answer and keep only what passes (rejection sampling). Small models are especially vulnerable to low-quality training material, so this filter is decisive.
Step 3: train the small model on the filtered data.
Results first. These are averages across three student sizes (0.5B, 1.5B, 3B, where B means a unit of one billion parameters).
| Metric (avg) | Base model | SFT | SynthSelf |
|---|---|---|---|
| Acquisition (new skills, 4 tasks) | 34.2 | 52.2 | 56.4 |
| Retention (prior skills, 14 tasks) | 42.8 | 40.5 | 41.7–41.8 |
The new skills rose the most. At 56.4, it clears both SFT (52.2) and the untouched starting point (34.2). Retention lands at 41.7–41.8 — above SFT (40.5), but short of the pre-training 42.8. Forgetting was reduced, not eliminated. The mitigation the title promises is exactly this much — the loss is softened, not erased.
The core is one thing: the rewrite stays inside the range of phrasing the student already finds familiar. Each model has expressions and solution styles it favors; the paper calls this its distribution, and calls staying inside it being in-distribution.
An example makes it concrete. Put three answers to "what is 2 plus 3?" side by side.
In the rewrite, both the answer 5 and the "add them" reasoning are unchanged. All that changed is the vessel holding them. It was moved into the "→ the sum is 5" format the student normally uses. What persists is the fact of the answer and the reasoning; what shifts is the frame of expression.
Why does this shake the original skills less? Train on the textbook as is, and the student has to absorb not just the new knowledge but an unfamiliar answer format all at once. The extra burden of adapting to that format, piled on top, is what you could say drags down the things it used to do well. But this is an interpretation spun out of a correlation the paper observed, not a proven cause. What the paper actually measured is a correlation: the more the distribution shifted before versus after training, the worse both forgetting and acquisition were.
Two experiments back this correlation. One: varying the source and type of the data every which way, the pattern held — the larger the shift, the larger the forgetting. Two: fixing a single corpus (OpenR1-Math-220k) and varying only the target text into three forms. The original with all of its reasoning (large shift), just the final solution (middle), and the model's own solution verified against the correct answer (small shift). The less the shift, the more math acquisition went up and the less forgetting there was in other areas like coding.
At this point it's natural to suspect it's just "borrowing" from the larger model. The paper set up a control group to look into exactly that: a distillation that uses the same aligner but, instead of rewriting, just answers the original question (Off-PD, On-PD). Head to head, SynthSelf raised acquisition by 5.7–16.9%p more, while retention was similar across the three methods. But reading this gap as isolating "the effect of rewriting in the student's style" overstates it. The decisive difference is in the input — SynthSelf's aligner receives the correct example and rewrites it, while the control only receives the original question and answers. The paper attributes this acquisition gain to the correct example injecting knowledge along the way, and notes that all three methods keep the prior distribution similarly intact. So the source of the gain is less the rewriting itself than the knowledge injection from the correct example. One more thing: stronger but more distant aligners (Qwen3.5-27B, Gemma-4-31B) produced more accurate data, yet acquisition was the same or lower. Closeness to the student, not the aligner's strength, explained the result better.
The selection criterion comes in one sentence: a model close enough that the student feels "this looks like something I wrote," while still able to give better guidance than plain copying of the correct answer. The point that satisfies both closeness and guidance quality is the model "one size up in the same family." In practice, the 7B, 14B, and 32B of the Qwen2.5 family were closest to the student, and after factoring in cost, the final experiments used 7B (SS-1) and 14B (SS-2).
How the paper measures closeness as a number, and how the experiments were set up, are gathered in the technical appendix at the bottom. No harm in reading it after you've grasped the results.
The most interesting tension in this method lives in the Step 2 filter. Filtering out samples the aligner rewrote badly raises the quality of the training data. Indeed, remove the rejection sampling and performance dropped; and even with less data after filtering (a 17.1% reduction in tool-use), it raised acquisition by 4.2%p on average over SFT with fewer examples — a signal that the gain comes from quality, not quantity.
But the same mechanism cuts the other way. "Discard what the aligner wrote incorrectly" also means "the data in areas where the aligner is weak drops out wholesale." The weaker the aligner is on a topic, the more samples of that topic get eliminated, and the student ends up learning that topic less. A filter meant to protect quality quietly narrows the scope of what's learned. So you shouldn't just watch the filter's pass rate — you have to look, by type, at what kinds of samples are being dropped.
The medicine results show the other end of this tension. On medicine, SynthSelf's improvement over the control was small. The authors reckon the aligner already knew almost all of that knowledge, so the rewrite had little to add. If the aligner knows too little, that data drops out at the filter; if it knows too well, rewriting adds nothing. The value is largest when there's a moderate gap — which yields a practical hypothesis: this method may be most useful when the aligner knows the relevant knowledge only halfway. Still, that's a guess drawn from a single domain's result, not a hardened adoption criterion.
The premise is also clear: there has to be a larger aligner in the same family. The authors judge the real-world constraint to be small, since most major open-weight families — Qwen2.5, Qwen3.5, OLMo 2, Llama 3, Gemma 4 — come with a size lineup. Tuning an out-of-family model to match the student's style is left as future work.
The title says you keep teaching "continually," but what the experiments actually showed is the result of teaching once. Building the data requires running the large aligner once, and the paper doesn't answer how the prior skills change if you repeat this process every time the workload grows. Chase the acquisition score alone and you miss the forgetting that quietly accumulates across repetitions. The promise of "keep teaching while mitigating forgetting" was confirmed for one round, but whether it holds across many rounds is still an open question.
ONTLE's work is running small businesses' repetitive tasks on small models, and at the center of that is tool calling (invoking fixed operations like internal API lookups, bookings, or placing orders). That makes this paper not a curiosity but something to put straight on the test bench.
Picture a scene. A client asks to "move next week's order up to this week." The model has to understand this and call the order-change function with the right arguments. If you SFT directly on company data to build a low-latency model specialized for this kind of tool call, it learns order handling but loses general conversation and reasoning — so it wavers on any request slightly outside the scope. And if you call a giant cloud model for every request, the whole point of low latency is gone.
SynthSelf aims at the space between. Instead of feeding company data as is, you have a model one size up in the same family (if the deployed model is 1.5B, something around 7B) rewrite it "in the way the deployed model finds familiar, with the correct answer intact," and train only on what passes answer verification. You set up the verification like this.
Whether this design can stand rests on three things. Is there a larger model one size up in the same family (without it, the whole thing falls apart). Is this a task where answer preservation can be checked automatically — an order call is a good fit since whether it's right can be verified by machine, but for work that needs human judgment, like consultation, summarization, or writing, you first have to define what "answer preservation" even means. And did you budget the cost of calling the large model for data generation (serving can be left to the small model, but the aligner's inference is spent once at the training stage).
There are two metrics for closeness. NLL measures how naturally the student would be likely to continue writing a given text; lower is closer. KL measures how far apart two generation styles have drifted; smaller is closer. The Qwen2.5-family aligners had the lowest average KL — closer to the student even than the baseline of copying only the correct answer (gold SFT). Compared against SDFT with prompts matched identically, SynthSelf still led by 18.96%p on acquisition and 1.68%p on retention. It also saved compute: alignment compute dropped 69.9%, and data-generation inference cost dropped 62.9–76.9% in FLOPs (the range varies with how it's aggregated).
The comparison was run along two axes. Below are the methodological baselines. Here, on-policy refers to a scheme where the model learns on the spot from data it generated itself during training.
| Method | Learning target | Weakness on small models |
|---|---|---|
| SFT | Copy the correct answer verbatim | Heavy forgetting |
| DFT, ASFT | Skew the target formula to mimic the on-policy effect (ASFT reduces the skew more) | Acquisition doesn't even clear SFT (DFT drops sharply on medicine) |
| SDFT | The model rewrites it itself | Rewrite quality degrades |
| Off-PD, On-PD | The same large aligner answers the original question (no rewriting) | No correct example as input, so knowledge injection is missing |
| SynthSelf | A large same-family aligner rewrites it in the student's style | Needs the large aligner once |
| Item | Content |
|---|---|
| Student (base) models | Qwen2.5-Instruct 0.5B, 1.5B, 3B |
| Aligners | Qwen2.5-7B (SS-1), 14B (SS-2) |
| Acquisition eval (4) | Science (SciKnowEval Chemistry L-3), tool use (ToolAlpaca), medicine (MedMCQA), science mix |
| Retention eval (14) | Instruction following, common sense, knowledge, code, math across the board |
| Distribution-forgetting correlation experiment | Train Qwen2.5-3B-Instruct on 23 math datasets with 2.5M completion tokens → measure the drop on 5 unrelated benchmarks (GPQA-Diamond, ARC-C, MMLU-Pro, HumanEval, MBPP) as forgetting |
Using the older Qwen2.5 family as the base model rather than the latest edition was to reduce data contamination and measure with "genuinely new data," and switching the medicine eval to MedMCQA was a choice to avoid LLM-graded evaluation and improve reproducibility. Because the experiments are confined to the Qwen2.5 family and 0.5–3B, behavior on other families or larger students can't be settled from this alone.
Standard distillation has the student imitate the output of a large teacher model as is. SynthSelf's goal isn't to imitate the teacher — it's to convert the correct answer, left intact, into a form the student can take in more easily. The direction of the copying is reversed, in a sense.
No. This contrasts with SDFT, the comparison target, which only became a fair, strong baseline once its skip-token setting for dropping format tokens was separately optimized.
Yes. It uses greedy decoding (picking only the highest-probability token), so that rewriting the same sample yields the same result every time.