There's an AI that gets work done by chaining several tools together. Every Monday morning, it digs through a handful of internal systems to produce a weekly status report. Then the tool fetching the data suddenly stops responding. At the same moment, the boss who's receiving the report cuts in: "This time give me the quarterly running total, not last week." The tool is dead, and the goalposts have moved. What is the AI supposed to do in that moment?
We call this kind of AI an "agent." Unlike a chatbot that answers once per question, an agent calls a tool, looks at the result, and decides its next move on its own. The trouble is that real-world tools aren't as well-behaved as chatbot questions. They time out, they deny permissions, and users change direction partway through. A human operator wouldn't panic at any of this. An agent is a different story.
Agent-Reflex, from a team at Renmin University of China and the National University of Singapore, tackles this mismatch head-on. The idea is simple: pull rules like "when you stumble this way in this situation, recover like this" out of records of failed tasks, and plant them in the model before deployment.
The training environments that raise these agents have almost none of these variables. Real-world tools wobble, but the training ground is far too tidy.
Here, an "environment" means the bundle of tools, data, and working conditions used for a given task. One internal issue tracker, one operational database — each is an environment. Existing training environments assume these tools always return clean values. So the agent gets deployed without ever once practicing how to detect a failure and back out of it.
Two kinds of wobble are missing. One is the errors tools throw. The other is the user cutting in — the kind of correction where someone says "no, use this instead" and redirects the task. Existing user simulators, in particular, stay stuck in rigid templates and capture almost none of the varied corrections or phrasing a real user brings.
And this isn't just a small-model problem. Figure 1 in the paper shows that even top-tier models like GPT-5.2 and Claude-4.5-Sonnet see their success rates swing sharply depending on whether there's no noise, whether the user cuts in, or whether a tool throws an error. Even frontier-class models wobble when the environment does. Small models have it worse.
The ways people have approached this fall into a few camps. The side where the model critiques itself mid-task and fixes course runs into a problem: the critique comes from the very model that got it wrong, so it can't filter out plausible-but-false errors. The side that retrains on failure records reshaped to include recovery finds that the recovery stops at that one task and never survives as a rule to pull out next time. The side that mass-produces more solvable tasks and environments adds environments but has no mechanism to turn the failures inside them into recovery knowledge. Agent-Reflex is closest to that last camp, but with a difference: it turns failure not into raw material for growing environments, but into reusable rules that accumulate per environment. (Representative work from each camp is in the appendix.)
The heart of Agent-Reflex is that it treats reflection not as something to wait for a model to be born with, but as a skill you can extract from failure and plant. Let's trace how a single failure becomes a rule, and how that rule changes behavior.
A calendar-lookup tool stops responding (a timeout). One agent simply gave up on the task — that's a failure record. But another record that hit the same timeout waited a moment, called once more, then moved on to the next step — a success record. Line the two up side by side and the fork is obvious. Out of this comes one rule: "On a transient timeout, wait briefly and retry exactly once. But do not apply this to errors that return the same result no matter how many times you call — like a permission denial or an input error — where you should immediately look for another path."
What sets this rule apart from a bare "retry on error" is the tags it carries: when to use it (a transient timeout) and when to stop (permission or input errors). Extract rules only from errors that were actually observed, not invented, and stack them up, and you get a "skill library" holding 1,170 rules across 557 environments. Records left behind by multiple models — not one — were mixed together so the rules wouldn't skew toward one model's thinking habits.
Having the rules in hand isn't the end. The deployed model can't consult this library. So the rules have to be steeped into the model itself. It's taught three ways. Roll the model back to the point of failure and have it continue on its own while consulting the rule (self-reflection); if the original plan itself was wrong, reset the environment to its starting state and have it re-plan the task from scratch (skill recovery); and convert a rule into a user correction message, slipping it in as if the user had cut in midway (user-centric reflection). In all three, the rule is used only "for reference" and never left in the training data. That way the model learns not "how to fix things while looking at the rules" but "how to recover on its own without them."
Finally, the method draws more recovery out of the model's own runs. It has the model solve the same task many times, in a clean environment and in a noisy one, then overlays two kinds of signal. "Did the task succeed all the way through" tells you the outcome but only roughly points to where things went wrong. "Did the action in that moment match the recovery rule" pinpoints exactly what to fix but can diverge from the task's overall success or failure. Go back to that timeout. The run that chose "wait and retry" instead of "give up" scores higher under grading that was shown the recovery rule in advance. By exactly that score gap, the choice of the model that didn't see the rule gets pulled toward retrying. The coarse outcome signal and the fine-grained action signal correct each other.
Every training method starts from the same small model (Qwen3-8B). You have to match the starting point for the comparison to be fair. Two abilities are measured: the strength to carry long tasks through to the end across different tool environments, and the strength to hold up when noise is mixed in. The comparison set is same-size models from the reflection and environment-scaling camps, plus, for reference, commercial frontier models and an open model 29 times larger.
Let's also be clear about what this comparison doesn't measure. Operational metrics like inference cost and response latency aren't here. Success rate measures only "can it do it," not "how cheaply and quickly does it do it." (The benchmark list and averaging formula are in the appendix.)
This piece started from "the moment the tool is dead and the goalposts moved." Let's look at that score first. On AgentNoiseBench, which measures noise handling, Agent-Reflex's overall score is 29.1 (per the abstract). It cleared the same-size environment-scaling and reflection baselines. But it's still a long way from GPT-5.2 (55.8) in the reference table.
Let's read it honestly. It did raise noise robustness — true. That its absolute level falls short of the frontier — also true. The team itself notes in the appendix that AgentNoiseBench is out-of-distribution for this method (a kind of situation not seen during training). It's a signal that the idea of planting recovery through training is starting to work, not a declaration that the noise problem is solved.
Now to ordinary long tasks with little noise. Here the story is different.
| Model (all starting from the same small model) | Long-horizon tool use, avg. |
|---|---|
| Qwen3-8B (baseline) | 40.2 |
| NoisyAgent-8B | 43.4 |
| Reflexion-8B | 44.7 |
| EnvScaler-8B | 50.9 |
| Agent-World-8B | 53.6 |
| Agent-Reflex-8B | 58.6 |
| Reference — Qwen3-235B-A22B | 57.7 |
| Reference — GPT-5.2 | 64.4 |
| Reference — Claude-4.5-Sonnet | 72.6 |
| Reference — Gemini 3 Pro | 72.8 |
The small model narrowly edged out the 29x-larger Qwen3-235B-A22B (57.7) on the aggregate score. There's still a gap to the commercial frontier, but among same-size models it stands a head taller. Benchmark by benchmark it leads too: VitaBench 17.5% (Reflexion-8B 6.8%), ACEBench 78.0% (NoisyAgent-8B 52.0%), τ²-Bench 62.1% (Agent-World-8B 54.6%).
It's worth looking at where the gains came from. ACEBench measures two things separately. Accuracy on a single individual action is 85.0%, roughly tied with the baseline. The share of turns in which every tool call within a turn was correct (turn-level accuracy), by contrast, opens up a wider lead at 73.3%. That said, this 73.3% is per-turn accuracy, not the rate of completing multiple turns to the end, so you shouldn't read the number itself as direct evidence of long-task recovery. Read it as: the relative gain was larger in filling a whole turn flawlessly than in the accuracy of each individual action.
Don't fold the two scores into one. 58.6 is ordinary tool work, 29.1 is noise handling — they're different benchmark bundles.
The limitation the authors acknowledge is generalization in noisy environments. On top of that, there's a point I want to press harder.
Just because a task succeeded, can we conclude that the recovery rule that produced it was correct too? The rules are extracted by an analysis model from the contrast between success and failure records. But some recovery action may have just happened to work in that situation.
Take the same timeout situation as two records. One only retried, and the server happened to come back, so the task finished. The other hit another block on retry, rerouted to a different tool, and finished. The outcome label on both is "success." But what finished the first one may have been luck, not the recovery action. Here "when does the rule apply" (scope) and "did the task succeed because of the rule" (causation) become different questions. To isolate causation, you'd have to compare, in the same situation, doing the recovery action against not doing it, side by side. Without that contrast, you can't separate a lucky success from the real effect of recovery. The question of how you guarantee rule accuracy runs straight into the fact that this method leans entirely on the quality of the analysis model and the verifier.
There are two practical constraints as well. The pipeline is heavy — building 557 environments, collecting records with 9 models, extracting 1,170 rules, then two-stage training on top. Reproduction cost isn't small. That said, code, models, and data are released, so there's little need to build from the ground up. And the rules are indexed per environment. Reuse within the 557 topics is a strength, but how well it carries over to an entirely new kind of tool ecosystem is hard to settle from these results alone.
ONTLE is a service that embeds an AI project manager on a dispatch basis. First, something to make clear. The paper planted recovery by retraining the model, but ONTLE's recovery criteria aren't retraining — they're designed at the level of operational logic and prompts. We borrow the idea, but whether the same effect carries over has to be verified separately.
Say ONTLE's AI PM takes on that Monday report. The issue tracker stops with a rate limit (429, an error that briefly blocks you for sending requests too often), the operational DB lookup times out, and the boss changes the aggregation basis. The key is that which errors to retry automatically and which changes to ask a human about differs from client to client.
Let's follow one case to the end. For a 429, wait a moment and retry. If it's still blocked, fall back to a cached past snapshot — but always label it "this figure is as of such-and-such date." Up to here, it's automatic. But the instruction "switch to the quarterly running total" is different. Widen the lookup range and cost grows, and the permissions on the data being touched change too. A change that moves cost, permissions, and data scope — that's the line that calls for human confirmation. The line is drawn not simply by "because the plan changed," but by what is changing.
So the first step is one thing. Limited to a few of your own core tools, measure two numbers: compared to the existing approach, "the rate at which, after a recovery, the task was actually carried through to the end," and "the rate at which wrong fallback data was used." Look only at completion rate and you'll miss the cases where it hastily filled in with wrong data. Both numbers have to come out together before the rest of the design means anything.
The stage names the paper gives are recovery-rule mining (RSD), turning rules into behavior (ReSFT), and refining from one's own runs (ORPO). The three training strategies are denoted self-reflection (SR), knowledge recovery (KR), and user-centric reflection (UR).
The representative work and limitations of the three existing streams are as follows.
| Approach | Representative work | Limitation |
|---|---|---|
| Reflecting mid-run | Reflexion, Self-Refine, ExpeL | The critique comes from the very model that got it wrong, so it can't filter out plausible errors |
| Training on failure records | Agent-R, ReflectRL, LoongReflect, NoisyAgent | Recovery stops at that one task and doesn't survive as a rule to pull out next time |
| Growing practice environments | EnvScaler, ScaleEnv, Agent-World, EnvACE / Simia, GenEnv | Environments grow but there's no mechanism to turn failure into recovery; some invent feedback, risking hallucination |
Evaluation benchmarks. Long-horizon tool use is five: BFCL-v4, ACEBench, ToolSandbox (200 stratified tasks), τ²-Bench (Barres et al.), and VitaBench; noisy interaction is AgentNoiseBench (measured on top of τ²-Bench and VitaBench's tool/user noise settings). The overall score is the simple average of the five benchmark averages, and the sub-averages for BFCL-v4 and ACEBench follow each benchmark's official scoring. The comparison set includes the reflection family (Reflexion, ExpeL, Agent-R, NoisyAgent), the environment-scaling family (Simulator, TOUCAN, AWM, EnvACE, EnvScaler, Agent-World), trajectory-level RL algorithms, reference frontier models (GPT-5.2, Claude-4.5-Sonnet, Gemini 3 Pro), and Qwen3-235B-A22B.
Environment construction. Following the Agent-World approach, a research agent equipped with search, browser, coding, and OS tools builds a per-topic database, and a coding agent creates the tools and unit tests. Only tools that compile, pass the tests, and satisfy state-transition constraints and validation survive. Each environment consists of a database, a validated tool set, and a verifier (a scoring mechanism that judges whether a task finished properly), with 60 verifiable tasks attached per topic. Because long trajectories are hard to evaluate on rule-based metrics alone, an analysis model and a verifier together label each trajectory as success or failure.
Data collection. Running agents across 9 public benchmarks collected 177,700 tool return values, of which 24,510 were errors. These were grouped by their surface form into eight "error families" and mixed probabilistically into each environment's tool returns, with the eight families balanced so that even rare errors accumulated enough to learn recovery from. Trajectories were collected from 9 models spanning 5 families and 4B to 284B.
Training objective. It combines outcome-based reinforcement learning with token-level reflection-guided distillation (on-policy distillation). The key is that when rolling out a group, clean and contaminated samples are normalized together rather than separately. That way genuine failures aren't split off into a separate contrast group and instead stay within the on-policy distribution. The score gap between the version that saw the rule (teacher) and the version that didn't (student) is converted into a confidence-weighted OPD objective (Wu et al., 2026, the SEED method). Gradients don't flow through the teacher branch, only the student branch, so it optimizes to raise the student's score on the tokens the teacher endorsed.
The reason to want a small model is on the deployment and operations side. But this paper doesn't directly measure inference cost or response latency, so "it's favorable on cost" can't be put forward as this study's conclusion. The Qwen3-235B-A22B used for reference is an MoE (mixture-of-experts) structure that activates only 22B of its 235B total parameters, so even a 29x larger total size doesn't necessarily mean operating cost spreads that far apart. What this paper actually showed is two things: as Figure 1 shows, big models wobble too when noise is mixed in; and once a small model learns a given environment's recovery, it gets ahead of same-size models and reaches a level that contends with the reference table's Qwen3-235B-A22B (on success rate).
From MCP, the standard conduit by which agents connect to external tools. The team took 557 widely-used MCP server specifications from Smithery as environment topics. That this conduit returns noise like timeouts and service errors in real deployment is exactly the reality this paper targets, and evaluating in the standard MCP format is also close to real deployment conditions.
No. The rules are used only "for reference" during training and then discarded — the goal is to make the model recover on its own without them. There's one exception. User correction messages are information that's actually visible on screen at deployment too, so those alone are left in the input.