Researchers from Meta Superintelligence Labs, UW–Madison, Stanford, and NYU take a hard, direct look at the hidden cost of reinforcement learning (RL) post-training. The headline finding runs against conventional wisdom. Take a pretrained "base" model that never went through post-training, bolt on a lightweight execution harness (a thin layer that helps the model's default behavior along with a simple prompt and a few conventions), and on agentic tasks it catches or even overtakes the post-trained model's solution coverage — provided the attempt budget (how many tries you allow per problem) is generous. The authors roll this loss up into a single number they call the "Sharpening Tax," and go a step further with PTGS, a sampler that trims the tax by tuning the sampling temperature (the knob that controls how varied the generated answers are) to match each task's difficulty.
RL post-training is the key step that elevates an LLM into a "reasoning model" or an "agent." And it comes with a long-running debate attached. Does RL genuinely create abilities that weren't there before, or does it merely amplify a handful of "profitable" behaviors the base model already carried while suppressing the rest? The latter view is known as the sharpening hypothesis.
Most of the evidence so far has come from math and coding, and it pointed one way. RL is effective at raising single-shot accuracy (pass@1, the probability of getting it right in a single try). But the ability to land at least one hit across many attempts — the coverage of solutions (pass@K, the probability of at least one success in K tries) — actually shrinks. The catch is that math and coding make a weak testbed for settling this debate. Pretraining data is saturated with math and code, so the base model already holds plenty of solution strategies, and grading often passes an answer as long as the final result is correct — meaning a garbage derivation that stumbles onto the right answer still collects the points.
So the authors picked a harsher testbed: agentic tasks. The ability to call tools to spec, take in feedback from the environment across multiple turns, and carry a long thread without letting it unravel. These behaviors are rare in pretraining data and are generally thought to be learned during post-training. If conventional wisdom holds, agentic tasks should be exactly where post-training "cuts new paths that weren't there before." This paper pokes at precisely that assumption.
First, three yardsticks need to be kept apart. The probability of getting it right in one try is pass@1 (accuracy); the probability of at least one success in K tries is pass@K (coverage); and the probability of succeeding on all K tries is consistency. pass@1 measures efficiency, pass@K measures "the range you can solve if you throw enough attempts at it," and consistency measures reliability.
What post-training does to a model's behavior snaps into focus with a darts analogy. The post-trained model is a player who has drilled a single throwing form thousands of times. When the board sits in its familiar spot, the darts cluster tightly around the bullseye (high pass@1, high consistency). But move the board to an unfamiliar corner and the player, locked into that practiced form, keeps hammering the same old spot and rarely reaches the corner. The base model, by contrast, is an amateur with no fixed form. Any single throw tends to miss, but the throws scatter widely — so over many attempts it occasionally lands in a corner the pro would almost never reach.
Why does post-training harden into "a single form"? It comes down to how RL works. RL refines a model by pushing up the probability of action paths that tend to succeed (reward) and pushing down the probability of the rest (penalty). So the picture the authors examined lines up with the darts story. Post-training drives per-task success probabilities toward the extremes, splitting tasks within the experiment into "always solved" and "never solved." Note that "never solved" here means the paper classified a task as "always fail" when it logged zero successes across 128 attempts per task — not that the success probability is exactly zero, or that it would fail given infinite tries. Thanks to this sharpening, the model solves what it can with few attempts (sampling efficiency) and nails the same task reliably every time (consistency). The cost is plain. A task that has crossed over to the "unsolvable" side rarely comes back within the given attempt budget. In effect, the model throws away the benefit of widening coverage by trying many times in parallel.
This is where the harness comes in. It's a thin support layer — dataset-agnostic and model-agnostic — that lays a loose set of tool-calling and parsing conventions on top of a simple system prompt. Its job is to prop up the places where a base model fumbles because it can't get the format right. Interestingly, put that same harness on a post-trained model and performance actually drops (more on why in the FAQ below).
The sharpening tax boils this loss down to one number. It's a diagnostic that measures how much post-training eroded the "solve more as you add attempts" ability — that is, test-time scalability (how much the score keeps climbing as you add attempts). The authors report that the tax can be estimated from just a few attempts (rollouts — a single pass in which the model produces an answer), and that it tracks well with both the tax measured over many attempts and with other performance metrics.
The paper's prescription is PTGS (posterior-tempered group sampling). The key move is to stop using the same temperature for every task. It estimates each task's difficulty in a Bayesian way (updating the difficulty estimate as it sees attempt outcomes), then raises the sampling temperature on hard problems to explore more broadly (softening) and lowers it on easy problems to concentrate (sharpening). Instead of pinning one temperature across all tasks, it varies the temperature per problem. It's also practical in that it drops straight into common RL algorithms like PPO and GRPO.
Start with the scale. The study evaluates 14 open-source checkpoint pairs (four families — Gemma-4, Ministral-3, Qwen2.5, Qwen3.5 — ranging from 3B to 35B effective parameters, the figure that denotes model size) across three agentic benchmarks (BFCL v4 multi-turn, WebShop, ACEBench), for 42 combinations in total. For a fair comparison, the post-trained models' "thinking" mode was turned off.
How much the harness helps the base models shows up in the average over eight base models (w/o = without the harness, w/ = with the harness).
| Benchmark | pass@1 (w/o → w/) | pass@32 (w/o → w/) |
|---|---|---|
| BFCL | 5.59 → 15.63 | 15.19 → 49.13 |
| WebShop | 6.32 → 9.73 | 39.48 → 49.15 |
| ACEBench | 44.24 → 42.63 | 89.25 → 87.69 |
On benchmarks where the base models especially struggled — BFCL and WebShop — the harness lifts both pass@1 and pass@32 substantially. On ACEBench, where the base models were already doing fine, it barely moves the needle or dips slightly. The takeaway: the harness props up what the model couldn't do, but it doesn't push what already worked any harder.
The main event — the crossover, the point where the two curves cross and the base model pulls ahead — appears once the attempt budget is stretched to 128 per task. At small K the post-trained model leads, but as the budget grows the harnessed base model's curve climbs more steeply and overtakes it in most settings. On WebShop at 128 attempts (rollouts), the base gemma-4-31B clears 85%, while the same model's RL post-trained version stalls at 56%. The larger the model, the earlier the crossover happens.
Applied to RL training in two agentic environments, PTGS paid less sharpening tax than the fixed-temperature baseline, solved more tasks under repeated sampling, and raised single-shot accuracy (pass@1) on top of that. Coverage and accuracy are usually a trade-off — gain one, give up the other — so lifting both at once is what makes this sampler appealing.
A few points deserve an honest airing.
First, the exact training recipes behind the checkpoints aren't public. The authors themselves admit they use "RL post-training" loosely. Other post-training methods like SFT and DPO also sharpen the distribution, so the analysis doesn't hinge on any one recipe — but it's hard to isolate "the effect of pure RL alone."
Second, these results come with thinking mode off for the sake of a fair comparison. Peak performance with reasoning turned on could look different (the authors do address that case in a separate figure).
Third, the base model's crossover is a story told at a large budget of 128 attempts per task. In a real product, the luxury of firing 128 attempts at a single request is rare. Whether the crossover happens within a realistically usable budget varies from setting to setting.
There's a more important caveat. pass@K counts "at least one success in K tries." Even with broad coverage in hand, if you have no mechanism (a verifier or grader) to pick which of those many attempts is the correct one, you can't ship a single answer to a product. An edge in coverage is not the same as deployable performance. Widening the range and picking the right one out of that range are two separate homework problems.
Suppose ONTLE's AI PM gets an integration request it has never seen before — an unfamiliar job with no clean template to fall back on. The common trap here is to push the one path you're most confident in, all in a single shot, and then close the ticket as "not feasible right now" the moment it fails. This paper explains why that reflex is dangerous. The better an agent has been drilled (sharpened) on domain data, the more likely it is — when faced with a rare, non-standard request (the long tail) that its practiced moves don't crack — to keep repeating the same move and shove the task into the "unsolvable" bin.
Put differently: the execution strategy should fork on task difficulty. For familiar, repetitive requests, pushing hard along one sharp path is the right call — finish it in one shot, cheaply, and consistently. For genuinely novel requests, the opposite: instead of insisting on a single silver bullet, carve out a budget to try several different approaches, and the odds of success go up.
So far ONTLE has routed requests by "size," sorting them into s/m/l tiers to decide the processing path. This paper suggests we should extend the axis of that branch from "size" to "difficulty" or "uncertainty." Sharp for easy requests, wider exploration for hard ones — much the way PTGS varies the temperature from task to task.
The sharpening tax, meanwhile, is a useful diagnostic in its own right — though from here on this is a working hypothesis we'd have to validate ourselves. When we fine-tune a PM agent to a customer's domain, we shouldn't stop at "average accuracy went up." We need the habit of asking, with just a handful of attempts, "did it lose the power to eventually crack rare, non-standard requests?" A model that looks better on average may in fact be getting stuck more often than before on the tricky minority of requests. The tax metric puts a number on exactly that blind spot.
No. A base model's single-shot accuracy (pass@1) is far below a post-trained model's. The base only comes out ahead when you crank up the number of attempts and prop up the formatting with a harness. In a single-shot setting with neither that attempt budget nor a mechanism to pick the correct answer from many, the post-trained model is still the better choice.
It's less an overturning than an extension. The accuracy-versus-coverage trade-off was first observed in math and coding. This paper shows the same phenomenon in agentic tasks and argues that the agentic setting is a harsher testbed for the sharpening hypothesis. Abilities like tool-calling and multi-turn interaction are thought to be learned during post-training — so if the tax shows up even there, that's all the stronger evidence.
The harness is designed to prop up the spots where a base model fumbles on formatting. But a post-trained model already has tool-calling and instruction-following in its bones, so that scaffolding acts as interference rather than help. It's the same story as the familiar observation that the better-tuned a model is, the more that fiddling further with the prompt can cost more than it gains.