The longer someone has been at a job, the more "last time I did it this way and it worked" tricks they accumulate. But if a trick came from a single success, applying it wholesale to the next job that merely looks similar can wreck things instead. That's what happens when one experience gets mistaken for a rule.
AI agents are no different. Without retraining the model, you can make an agent work better just by writing down tricks like "this is how you use this app" or "run this experiment in this order." We call that trick document a skill — something like a procedure manual kept outside the model and pulled up when needed. And one cycle in which the agent accumulates experience and revises that manual is called an epoch. The commonsense expectation is that skills get better with each epoch.
The problem is that this expectation often fails to hold. You'd think learning from failure and patching accordingly would only help, but in practice it's common for the result to end up worse than before you touched it. Like the veteran above, the agent hastily nails down a single experience as a rule. So what does this mean when we carry it over to our own work? I'll come back to that question later with an ONTLE example.
EviSkill, proposed by a team at Jilin University (Yan Zhou et al.), sets one simple rule for this. Every time it revises a skill, it re-runs the exact scene that went wrong under the proposed fix, and only adopts the fix when behavior actually improves. Validated across three benchmarks and six LLMs, it achieved the highest success rate on most combinations. What stands out in particular: in conditions where other automatic-revision methods actually regressed below the initial skill, EviSkill posted the top score on the spot.
EviSkill targets two holes in existing automatic-revision methods. One is that the justification for a fix disappears; the other is that a rejected batch of fixes gets thrown out wholesale. Following how each of the three steps below plugs these holes also makes it clear why the existing methods regressed.
Step 1 — Hold onto the justification with evidence cards. Existing methods turn an observation straight into "fix it this way." That erases why this sentence was changed and what situation justified the change. Ask half a year later "why is this step here?" and there's no way to answer. EviSkill first organizes an observation into an evidence card. A single card holds a tracking identifier, the execution span that supports the fix (which task, which epoch, from which step to which step), and the proposed correction. When a fix is created, it links to the card it rests on. The justification stays with the fix. Cards don't pile up endlessly: a card that has outlived its usefulness ages out, and a card that has been repeatedly rejected is excluded when new fixes are created.
Step 2 — Validate by replaying only the problem scene. This is where EviSkill diverges from other methods. Other work extracts plausible lessons from execution traces and folds them straight into the skill. But a fix born from experience doesn't always guarantee improvement. The paper's preliminary study shows this empirically. When fixes drawn from experience were applied to the original skill and the task was re-run, some improved performance — but some left it unchanged or actually made it worse. So EviSkill inserts one step before adoption. Using the span information on the card, it restores the state and conversation history up to just before that scene, then re-runs only that span under the revised skill. Rather than running the whole task from scratch, it replays just the scene that got stuck. Then an evaluator (an LLM assigned to judge) compares the original scene with the replayed scene and renders one of three verdicts. If the intended correction holds with no new failures, accept; if a defect shows but it's worth refining, hold; otherwise, reject. A held fix is revised once and then replayed and re-evaluated.
Step 3 — Re-examine across epochs. Passing the replay doesn't mean it's confirmed on the spot. EviSkill runs skills in two copies. One is the conservative confirmed skill, updated only when validation performance rises; the other is the trial skill, which tentatively layers on replay-backed fixes and watches them over multiple epochs. Think of it as keeping a clean master copy separate from a still-experimental draft.
This two-copy structure plugs the second hole — batch discarding. Existing methods throw out a rejected batch of fixes wholesale in a given round. Even if a usable correction is mixed in, it disappears along with the rest. But the paper's preliminary study shows that even when a whole batch was rejected, keeping just a selected subset of fixes from within it raised performance above the previously passing skill. So even when a batch is rejected, EviSkill replays each fix inside it individually, checks whether it works on its own without leaning on other tentative fixes, and salvages only the ones that pass. When the set number of epochs ends, only the confirmed skill is frozen for actual use, and any tentative fix that never got confirmed is dropped.
SkillOpt, the closest experience-driven method, has only a single gate: "accept only fixes that raise the validation score." EviSkill keeps that gate but adds layers in front of it — the replay check and the re-deliberation of rejected fixes.
The evaluation stage is three interactive benchmarks: AppWorld (168 tasks), which performs tasks via app APIs; ScienceWorld (211), which handles science experiments; and ALFWorld (134), which solves household tasks in a text environment. All three datasets split train/validation/test with no overlap — the training set revises skills, the validation set decides adoption, and the test set gives the final evaluation. The single metric is task success rate (%).
For the backbone — the LLM that actually produces the agent's behavior — six models from the GPT and Qwen families were used. There are two groups of baselines. NoSkill, which uses no skill, and LLM Skill, which uses the initial skill as a fixed input, are the "non-evolving" baselines; Trace2Skill, EvoSkill, SkillGrad, and SkillOpt are the "skill-revising" competitors.
The first takeaway is that EviSkill was generally the best. Of 18 model-dataset combinations, it was top on 14 and within the top two on 16, and it beat NoSkill on every combination (an average of +17.93%p over NoSkill). That said, this average bundles combinations with wildly different starting points, so it shouldn't be read as the performance of a single setting. As a standout example of its edge, on ScienceWorld with Qwen3.5-9B, EviSkill hit 37.91%, beating that setting's strongest baseline (SkillOpt at 22.27%) by 15.64%p.
The second takeaway carries more weight: revising a skill is no guarantee it beats the initial version. On a recent GPT-family model run on ScienceWorld, the initial skill scored 76.78%, while Trace2Skill dropped to 65.40%, EvoSkill to 54.50%, and SkillGrad to 67.30% — all worse. Worse, in fact, than doing nothing at all.
Why does this happen? A lesson drawn from a single run easily mistakes the circumstances specific to that situation for a "rule." Once that lesson is nailed into the manual, every subsequent task gets solved through that biased rule. The particularity of one case becomes the default for the whole. Like the "curse of knowledge," where someone who already knows the answer can't imagine where a beginner gets stuck, a skill learned from one scene injects a wrong bias into the entire system while blind to anything outside that scene. That's why hasty learning cuts deeper. In the same setting EviSkill reached 83.41%, and overall it improved on the initial skill in 17 settings and tied in one — because the gates of validating fixes against behavior and keeping confirmation conservative are exactly what filter out this trap.
Removing either mechanism lowered performance. Averaged over the three GPT backbones, removing replay validation dropped average accuracy on the three benchmarks (AppWorld, ScienceWorld, ALFWorld) by 4.56, 2.85, and 2.49%p respectively. Performance also fell when the cross-epoch re-deliberation was removed. These ablations support that each mechanism contributes to the result — though they don't establish that the two effects are independent or that there's no overlap between them.
Start with what the authors acknowledge directly. On AppWorld, the edge over competing methods is less consistent. Run that recent GPT model on AppWorld and EviSkill's 94.64% is not far from LLM Skill's 94.05%, and on Qwen3.5-9B with AppWorld, EviSkill's 43.45% trails SkillOpt's 47.62%. Rather than a flaw in the method itself, this is better read as a signal that some environments — like tasks solved by composing APIs — make it hard to cleanly isolate a correction's effect through behavior replay. That reading is mine.
The scope of what's measured is also worth noting. Since the metric is test-task success rate, it measures "is this fix a procedure improvement that actually works?" well, but whether that improvement generalizes to entirely different kinds of tasks the benchmark doesn't cover is hard to assert from this number alone. The train/validation/test split has no overlap, so there's no peeking at the test tasks themselves — but that is a separate question from "extending to kinds never seen before."
Cost is another thing to weigh separately before real-world adoption. EviSkill replays a scene for each fix, compares across epochs, and even re-runs rejected fixes. That's a lot more re-execution than a method that settles with a single estimate. The paper includes an efficiency analysis, but actual cost can vary with the environment you use it in. Finally, the fact that the evaluator, extractor, and editor are all LLMs means the structure has an LLM judging "did this replay make the correction hold?" — so how far the evaluator's own bias gets filtered is hard to tell from this metric alone.
Let's come back to the question we opened with — "what does this mean for our work?" Say ONTLE's AI PM is running the "register a new client account" procedure and gets stuck at the business-registration-number verification step for a certain client.
An experience-driven method would immediately nail the lesson "add a business-number format check before this step" into the manual. But if that failure was actually due to that one client's input form, this fix forces an unnecessary step on other clients, and yet another fix piles on top of it. One client's special circumstance becomes the company-wide default.
The EviSkill way takes four steps. First, leave the stuck scene as evidence — which client, which task, which step, and what got stuck. Next, before overhauling the whole manual, replay just that scene through the fix procedure. If running it under the revised procedure passes with no new problems, the fix itself has held "for that scene," at least.
But the key is not stopping here. That pass is only the result of replaying one client's scene; whether the fix is good for every client is still unknown. So run it again against a validation set that includes other client cases. This is where "was it that one client's circumstance?" separates from "is it a common improvement that works for everyone?" Local replay (rolling back one stuck scene) and scope validation (weighing it across many clients) are different steps in just this way. Finally, promote it to the official procedure only when the metric rises in that validation.
Two benefits show up plainly in this process. A fix born from one client's special situation has to pass through a layer of separate validation before it spreads straight into the procedure for every client (closer to "it only moves up if it passes" than to "it's fully blocked"), and when someone asks "why is this step here?" you can always answer by pointing back to the original scene.
The question EviSkill raises is less about technique than about attitude. It's easy to stack up the quantity of fixes, tacking one more line onto the manual every time a failure appears. But checking whether that line actually works by rolling it back at the scene of the problem, leaving the justification for where the fix came from, and re-deliberating only the substance even after a rejection — securing that quality of fix is a hassle. What EviSkill's experiments showed is that many fixes that skip the hassle often slide into a spot worse than fixing nothing at all. Not the number of revisions, but whether each revision passed validation, is what decides a skill's fate.