You run a command. A red error appears. You trace the cause and fix one line of code. You run it again. This time it breaks somewhere else. You scroll back through the logs, change a setting, and run it once more. Whether it's development or data cleanup, real work almost always gets to the finish line through this same loop of "check, fix, run again."
The screen where all of this happens is the terminal: the screen where you type text commands to operate a computer. And the full record of this work, with commands, results, and the next commands strung together one after another, is called a "trajectory." For a person, it's closer to the work log you keep while solving a problem.
Researchers at Alibaba's AMAP and the University of Science and Technology of China (USTC) zeroed in on this trajectory. What has to change to take an AI agent that works in a terminal beyond short demos and make it capable of real, multi-hour jobs? Their answer: when you teach an agent, what matters isn't "which tasks it managed to solve" but "which trajectories it learned from." And most of the training trajectories available today are far too short. So the researchers built TerminalHorizon, an engine that automatically churns out long trajectories.
Let's be clear about one thing first. A long record doesn't mean there's a lot to learn from it.
A short trajectory shows how to type commands and call tools. But it can't capture how an agent actually finishes a job: gathering clues, making judgment calls, taking feedback and revising its strategy, turning intermediate results into a final deliverable. In long tasks, these behaviors show up not in one or two steps but across many stages.
That said, longer isn't automatically better either. Trajectories grow long for two different reasons. One is pointless retries, where the record just piles up because the agent repeats the same mistake or wanders in circles. The other is genuine dependency, where the trajectory grows because what was observed in an earlier step changes the next decision, so the steps interlock. The former is noise to filter out; the latter is exactly the capability you want the agent to learn.
There's an even nastier pitfall. When you generate environments automatically, the environment can end up pre-solving the very core task the agent is supposed to tackle. Say you set the task "implement a function that sums the refund amount per order." If that summation function already exists inside the environment, the agent's job shrinks to maybe changing one numeric constant. It looks complex on the surface, but none of the intended exploration, implementation, or verification actually happens. The trajectory may look long while having no substance.
So the question to ask when looking at the data isn't "is this trajectory long?" but "why did it get long?" Is it genuine dependency, or is it because the environment was under-built or the agent floundered? TerminalHorizon forces this distinction starting at the data-creation stage.
The person who sets a problem is bad at spotting its gaps, because they already know the answer. A teacher who writes an exam can't filter out the questions where "the answer is obvious just from the passage." Those holes only surface when you have someone who doesn't know the problem try to solve it first.
TerminalHorizon uses exactly this approach. The task isn't tested by the side that created it, but by a third-party agent seeing it for the first time (an "independent challenger"). Then it traces backward through the segments that got "solved too easily" and rebuilds the environment. The engine runs in two stages. Let's follow the refund-summation task we raised earlier all the way through.
First, a task is created. Three agents work on it in turn. A planner picks a topic that will draw out behaviors not yet covered. A deep-research agent digs through real documents, code repositories, and issues to collect on-the-ground needs and constraints, plus the basis for "what counts as done." A build agent assembles all this into an executable task. The assembled task contains the public instructions, the execution environment the agent will manipulate, a reference solution, and a hidden grader (the mechanism that automatically judges whether the answer is correct). The agent solves it by feeling its way through the environment, seeing only the public parts.
Once the task is built, the reference solution and the grader are both hidden, and the independent challenger is given only the public instructions and asked to solve it. That solution record is then scrutinized. Did errors occur? Did information leak that shouldn't have been visible? Does the grading diverge from the intent? And, most importantly, is the core task that was intended to begin with still alive inside the task?
Back to the refund-summation function. If the challenger passed without writing the function itself, that's a signal the environment is already providing that function. The researchers trace where such "solved too easily" segments came from: whether it's the source material laid down at the start, or the behavior of a pre-built tool. (The paper calls this "horizon gap analysis." The horizon refers to the length it takes to finish a task, that is, the task's breath.)
Here you must not conflate two cases. If the trajectory ended short because the agent solved it efficiently and well, leave it alone. Only fix the cases where the environment pre-solved it so there was nothing left to do. In the refund-function case, you strip out the completed function the environment was holding so the agent has to write and debug it itself, while leaving the public spec that defines that work. The fixed task is then run again by the challenger to confirm that the restored work now actually enters the solving process.
A task that passes isn't the end but the starting point. An evolution agent grows the task while keeping the same goal. There's one rule: a newly added requirement must not pass under the existing solution. It has to invoke additional analysis, implementation, or verification.
Let's bring out the refund-summation function again. On the function that cleared Stage 1, we pile on a requirement to handle exceptional inputs: orders where the refund amount is empty, or where out-of-range values like negatives are mixed in. The earlier solution only handled normal inputs, so as-is it won't pass. The agent runs the function it wrote earlier, sees with its own eyes where the exceptional inputs cause it to blow up, then decides what to fix. The earlier execution result determines the next fix.
There are four directions for growing it, and all four can be sketched out with this one function.
Whichever direction, the added work is designed to lean on the earlier work. The values handled are also made to change on each run (the paper calls this "instance mutation," a device to keep memorized answers from working). In this way the same behavioral pattern extends across multiple stages within a single trajectory. A passing extended task then becomes the parent of the next extension.
The flow has four stages: generate tasks, filter and select the solution records that come out of them, train the model on those records, and evaluate performance with benchmarks.
Training used supervised fine-tuning (SFT, the most basic form of learning, where you show the model answer and have it follow along), picking a strong "teacher model's" solutions and having a smaller "base model" imitate them directly. Transplanting a large model's solutions into a small model this way is called "distillation." The result is TerminalHorizon-27B.
Two trajectories were sampled per task, and after passing them through three filters (Appendix F), the final 2,852 were kept (named TerminalHorizon-3K). Before training, decontamination was also performed to keep evaluation problems from mixing into the training data.
Whether the data is actually long and varied shows up in the numbers. The domains span 25 specialized fields, including chips, physics, and astronomy, all cast in a single terminal format. What about length? When 300 tasks were sampled from each of several open-source synthesis methods and run with the same teacher and the same settings for comparison, TerminalHorizon trajectories had an average interaction length of 52.8 turns, with the top 10% at 98 turns, far exceeding the comparison group. The median of the "longest unbroken chain of actions" that continued without being cut off by terminal feedback was 25 turns. It means that over the long haul, actions and feedback interlocked without fragmenting.
Let's lead with the conclusion. TerminalHorizon-27B, trained on this data, beat the base model Qwen3.5-27B on all 12 benchmarks. The margin was especially wide on coding tasks that directly handle the terminal. On Terminal-Bench 1.0, 2.0, and 2.1 it rose by +8.7, +14.2, and +13.85 points respectively. A signal that what it learned from long trajectories made it better at terminal tasks themselves.
Practical automation, on the other hand, is a different story. The AutomationBench pass rate jumped from 5.5% to 17.2%. The increase is clearly large. But the fact that it landed at 17.2% has to be taken at face value. The rate of completing these tasks from start to finish is still low. It means long practical tasks are that hard, and it's not the stage to call it "usable now" based on the size of the gain alone.
Reasoning ability wasn't eroded. MATH-500, GSM8K, AIME, and KOR-Bench all held steady and even rose slightly. The agentic abilities were boosted without sacrificing basic reasoning. That said, MATH-500 and GSM8K already have base scores near the ceiling, so the gains themselves mean little.
The picture gets sharper when you line up the data volume against another method. Terminal-Universe-27B, using the same base model and the same execution harness, scores 52.8% on Terminal-Bench 2.0, while TerminalHorizon-27B scores 55.8%, ahead by 3.0 points. But the number of trajectories used in training differs. Terminal-Universe used 32.0k, TerminalHorizon 2,852, less than a tenth.
The observation here goes only as far as "a better score came from fewer trajectories." Beyond that, read it carefully. Trajectory count alone doesn't compare total training volume or data production cost. So the right reading isn't "fewer is always fine," but something more like "the density of long trajectories substituted for a good part of the trajectory count."
To sum up, there's one thing this experiment can say clearly. Selecting long trajectories and having the model imitate them raised terminal-task performance, and basic reasoning held. What it hasn't yet separated out is also clear. This model is a distilled model that filters and imitates a strong teacher's solutions, so the credit for the higher scores can't be cleanly split between data design and teacher skill. Whether it was length, diversity of behavior, or teacher quality that drove the gains hasn't been teased apart yet either. Separating that would require an additional control comparison against short trajectories made with the same teacher.
On top of the limitations the authors list, there are points worth flagging from a reader's standpoint.
No matter how refined, a synthetic environment can't fully capture the messiness of reality. The design of plugging gaps with an independent challenger is clever, but both the teacher that sets the task and the independent challenger solving it for the first time are LLMs. The grader that decides success or failure, however, is a separate hidden verifier attached per task, a mechanism that automatically judges correctness and is not an LLM. The LLM judge that inspects the trajectory (Appendix F) is yet another separate thing, and it receives neither the final reward nor the success/failure. Still, if the LLMs that create and solve the tasks share the same blind spot, that blind spot isn't filtered out and remains in the data as-is. On top of that, the constraint that "it must be executable and gradeable" can itself skew the data toward the kinds of problems that get cleanly judged by a grader. Whether the across-the-board improvement on benchmarks is really the result of transferred "long-trajectory experience," or just the effect that usually comes with high-quality, high-difficulty data, also needs more scrutiny to tell apart.
There are also things scores can't measure. A benchmark only measures success on a fixed set of tasks; it doesn't guarantee the endless variation of real operations or long-duration stability. The device meant to fill that gap is the RSI (recursive self-improvement) cases. Each task is given 12 hours of research time, and they watch whether the model stays with the solution it first came up with, or keeps experimenting and revising its method. Tasks that require pushing the score all the way up, like the appendix's "2048 strategy optimization," are used here.
What to watch for here isn't "how many times it retried." The key is whether the earlier result actually changed the next action. A simple retry runs the same strategy again with only the conditions slightly changed. A feedback-driven revision reads the score or failure signal from the previous run and, on that basis, twists the strategy itself and reflects it in the next run. The two may look similar in length but differ in nature. Since this is a device meant to look qualitatively at this "ability to work for a long time" that doesn't reduce to an average score, it's best to take its interpretation qualitatively as well.
Let's narrow it to a single scene. A small-business owner hands it to ONTLE's AI PM: "Last month's supplier reconciliation records don't match the order data from our own store. Find the cause and reconcile them." This isn't a one-line command. You pull data out of both systems and comb through it, form a hypothesis about where things diverge, write and run a reconciliation script, go back to the source and dig when a number looks off, and produce the reconciled result as a report the owner can open. It's a long task with many interlocking stages.
The heart of this job happens in the middle. You run the reconciliation script and find that one supplier's March reconciliation amount spikes well above the order total. A well-trained agent doesn't stop here. That mismatched number changes what it investigates next. It newly digs into whether refunds were reflected in the reconciliation, whether orders straddled two months at the date boundary, whether units got mixed up. The result of discovering the mismatched number determines the next thing to investigate. This is exactly the "genuine dependency" the paper talks about.
ONTLE's AI PM already takes on and performs these kinds of multi-stage jobs. What we take from here isn't the result model but the method. So we set, from the start, the question to validate with a pilot: "Does a model trained on short trajectories miss these anomaly signals more often?" To check this, the eye of evaluation has to move from the final number to the process. In jobs where intermediate results steer the next stage, like reconciliation, it means you have to look at the record of whether the agent, when it met an anomaly signal, actually checked its basis, and whether it changed its judgment when needed.
But merely counting "did it fix something in the middle" isn't enough. That would dock even the cases where it solved efficiently in one shot as "didn't fix anything, so bad." This pitfall is identical to the Stage 1 distinction in the paper. So the first pilot is narrowed to the single job of reconciliation. Instead of throwing the task out and only looking at "pass/fail," we start by inserting into the evaluation pipeline a procedure where a separate agent seeing the task for the first time solves it first, and we trace back "where it got solved too easily."
Because they fixed the training method to the simplest option. SFT leaves behind only the result of "imitating the model answer as-is." It's effectively leaving out the element, like in reinforcement learning, of the model changing its own strategy mid-training, which reduces the room for the cleverness of the training technique itself to creep into performance. That said, using SFT doesn't mean the single variable of "data quality" gets cleanly isolated for observation. Teacher quality, trajectory length, data composition, and training token volume are all tangled together, and this paper's experiment doesn't prove them apart either (exactly the limitations noted in the body's "Results" and "How Far Can You Trust It"). Fixing SFT merely bundles the one axis of "training technique" simply; it doesn't also mask the data-side variables.
Long trajectories have more tokens to process per item, so the per-item cost is higher. This paper's rebuttal is that it cut the number of items. The per-item cost rises, but the count drops to under a tenth, leaving room to offset it. That said, the total training volume and production cost weren't publicly compared, so the actual trade-off has to be weighed in your own environment.
The principle carries over to any task where the environment is auto-generated. But there's one prerequisite: the task has to be executable and automatically gradeable in order to be solved by an independent challenger and have the "too easy spots" sifted out. In areas where grading is ambiguous (writing quality, for instance), the same framework is hard to apply as-is.