An AI that operates a computer the way a person does is called a "computer-use agent" (CUA). Teaching one requires a huge amount of software it can actually click through, and getting that software has always been the bottleneck. SoftwareGym flips the premise: it automatically mass-produces the very software used for training. Using nothing but the operating skills learned inside these synthesized environments, a single model raised its success rate on the real-app benchmark OSWorld from 41.39% to 56.97%. Here "success rate" means Pass@3 — the share of tasks solved at least once across three attempts. And it got there without using a single one of OSWorld's apps, tasks, or action traces during training.
Teaching a computer-use agent is a lot like teaching someone to drive. You can't learn to drive by reading a textbook. You have to get out on a real road, turn the wheel, hit the brakes — only then does it stick. CUAs are the same. Skill grows only once you accumulate records of actually pressing buttons, opening menus, and saving files inside running software — records this field calls "trajectories." The catch is that securing this "road" is the bottleneck.
Until now there were essentially two routes, and both had sharp limits.
The first is to pull existing desktop and mobile apps straight into the training ground. Because programs like GIMP, VLC, and Thunderbird are real, their states are realistic and their workflows are complex. But the cost of turning a single app into a training environment that "reproduces identically every time you run it" is high. That keeps the number of apps you can handle small, and repeatedly training on the same app carries the risk of overfitting — memorizing "tricks that only work in that one app." OSWorld, Windows Agent Arena, and AndroidWorld belong to this family.
The second is to build software environments on top of a browser. They're easy to deploy and easy to keep state under control. The downside is that everything rides on the same foundation — browser, HTML, DOM — so it's hard to capture the structure native apps have or the deep workflows where state carries across many steps. In other words, interaction stays shallow.
The authors argue that a good CUA training environment has to satisfy three things at once: screens and operation styles diverse enough (diversity), trajectories that are complex yet run all the way through (reliability), and a way to keep growing the environment with little manual effort (scalability). The two existing routes couldn't fill all three at the same time. Loosen things up for diversity and layouts converge to look similar within the same category; guarantee reliability and you have to check, one by one, whether every specified feature actually works end-to-end across the whole screen flow — which, done at scale, piles up a verification burden no human team can carry.
SoftwareGym's choice is neither "borrowing" nor "faking it with a browser." It synthesizes native desktop apps from scratch with a verification loop built in, widening the distribution of software itself.
SoftwareGym is an automated pipeline in which several AI agents split up the roles. They design software, implement it, and verify it by clicking through it themselves. Synthesis flows in three stages. Let's follow one feature through it: "rename a file, and after you close and reopen the app, that name is still there" — an ordinary feature every app has.
Stage 1: Design. Given only a software category and some reference material, the pipeline produces a detailed functional specification and several UI designs. For our example feature, this is where it pins down, in writing, where the user edits the name, what action triggers the rename, what should appear on screen afterward, and how to confirm the name persisted on reopening. Fixing that "how to confirm" during the design stage is the crux. Because the functional spec doubles as the test criterion, the goal being built and the goal being graded are the same from the start. The same design stage also enforces diversity. Even for the identical "rename" feature, it produces several variants with different menu placements and screen structures, so the model doesn't get used to a single look.
Stage 2: Implementation and evaluation. The app is actually implemented according to the design received. But code that looks plausible is no guarantee it really works when clicked. So the evaluation stage separately tests the running app. In our example, a testing agent launches the app, actually renames a file, closes and reopens the app, and confirms on screen whether the name remains. If it doesn't pass, the agent records where it got stuck and sends it back to the implementation stage to be fixed, cycling through this repair loop until every case passes or the allotted version budget is spent.
Stage 3: Task synthesis. Finally, it creates the tasks an agent will solve in this app. This is where an "evidence gate" kicks in. Only features that were actually clicked and confirmed to work in the previous stage are eligible as task material. Features that couldn't be tested or that failed are dropped from the task pool. The filtered features are grouped by difficulty: easy tasks become a single goal, while hard ones become multi-step jobs where each stage's result is the starting point for the next. When shown to the user, they're rewritten as natural request sentences that hide answer hints like coordinates, operation paths, and internal identifiers.
🔧 Implementation box — looking inside - Design: FunctionDesigner expands a feature in four steps (a baseline of features from a mature product → adding specialized features like multi-condition search, batch operations, history, setting persistence, and input validation → workflows that stitch together cross-module state, dependencies, and failure recovery → formalizing it all into an "implementation-and-evaluation contract"). The contract captures the user entry point, the user action, the expected state change, the visible GUI response, and the pass checkpoints. - Diversity machinery: StyleDesigner reproduces the same feature in several styles that differ in product positioning, information structure, interaction, and visual identity. Variants must differ along at least two layout dimensions, and each design is converted into a layout tree of windows, regions, panels, and controls so similarity can be measured by tree-edit distance between trees. If the minimum distance falls below a threshold, the least-distinguishable design is regenerated. IconDesigner constrains icons to differ structurally, not just in color. The UI technology is Avalonia, and all generated XAML is compiled to confirm it can actually run. - Implementation and evaluation: Coder implements the Avalonia app, and DevCoder audits and supplements it against the feature manual (source-level verification up to here). EvalTaskGenerateAgent creates the test cases and GUIAgent executes them with mouse and keyboard, leaving behind the trajectory, checkpoint evidence, final state, failure location, and repair suggestions. After each evaluation round, a visual-fidelity score between the implementation and the UI blueprint is computed, and parts that fall below the threshold are singled out for repair.
It helps to separate three levels here so you don't misread the numbers that follow.
What it trained on. The base model is Qwen3.5-9B. Here "9B" means a scale of 9 billion parameters, which is on the small side by today's standards. On top of it, the authors ran multimodal SFT (supervised fine-tuning) using only trajectories gathered from SoftwareGym apps. Not a shred of OSWorld's apps, tasks, or trajectories was used.
Where it was evaluated. Two benchmarks. One is OSWorld: 244 tasks across 8 domains — Chrome, GIMP, Calc, Impress, Writer, Thunderbird, VLC, VS Code. The other is the authors' own SoftwareGym-Bench: one app per category across 12 categories, with 10 manually verified tasks per app, for 120 tasks total.
What each evaluation shows. The two measure different things. OSWorld looks at "transfer to real apps not used in training," while SoftwareGym-Bench — which competes within the same 12 categories — looks at "skill within the learned categories (in-domain)." One misconception is worth heading off here. It is not the case that OSWorld is a benchmark whose categories themselves don't overlap with the training categories. Per §4.2 of the paper, 121 of the synthesized apps fall into the 8 categories that correspond to OSWorld. Saying transfer holds does not mean the software categories don't overlap; it means operating skill carried over to real apps never once used in training. So you shouldn't add the two scores together and read them as one achievement. SoftwareGym-Bench in particular has traps planted on purpose. Each task includes protected objects you must not touch and distractor elements designed to confuse, forcing the agent to judge what to fix, what to ignore, and what to preserve — rather than mechanically following a fixed sequence. It measures the ability to read and discern state, not just to reproduce operations.
Results. On OSWorld, this 9B model improved over the base across every domain, with overall Pass@3 rising from 41.39% to 56.97% — up 15.58 points. Since nothing from OSWorld was used in training, this is direct evidence that operating skill learned in the synthetic environment carried over to real apps. In the same table (Table 1), the model climbs to a level comparable with open-weight baselines far larger than itself.
On SoftwareGym-Bench (Table 2), the same 9B model rose from 25.8% to 40.0% — up 14.2 points. It surpasses Qwen3.7-Plus and comes out ahead of several considerably larger specialized CUAs. That said, it still trails large foundation agents like GPT-5.5 (73.3%) and Gemini 3.1 Pro (70.8%) by a wide margin. So this result should be read not as "beat SOTA" but as a small model, through synthetic-environment training alone, carving out a range that overlaps with models above its weight class.
The scale and reliability of the synthesis are backed by numbers too. Across 182 apps, the average GUI task pass rate is 91.5%. By Figure 4, across the 121 apps in the 8 aligned categories, GUI evidence covers all 49 major modules and 78.8% of the concrete native features. In short, the generated software isn't a glossy shell — it's packed with features that actually click.
📎 Reproduction notes. For synthesis, a Codex CLI agent running on GPT-5.5 handled design, implementation, and evaluation-case generation, and each app was refined through up to 3 versions (with early termination once all cases pass). GUI evaluation of generated apps was performed by Seed 2.1 Pro on an Ubuntu 22.04.3 virtual machine, with up to 50 steps per case. The final model evaluation conditions were temperature 1, an interaction budget of 100 steps, and 3 independent rollouts per task. Separating the design/implementation models from the evaluation model was meant to reduce the bias of the builder grading its own work.
What transfers — a hypothesis. The authors conclude the model learns "transferable GUI interaction." Spelling out what carries over as a hypothesis about specific abilities: ① finding the feature you want on an unfamiliar screen, ② tracking how an app's state changed across multiple operations, and ③ confirming the result properly persisted after saving or rerunning. These three are abilities that cut across apps rather than procedures specific to any one app, which can explain the transfer to OSWorld.
Open question — what produced the gains. Whether this transfer owes to layout diversity, to verification quality, or simply to the amount of training is not yet teased apart. You'd need an experiment that isolates the three factors one at a time before you can say "what the source of the effect is."
Open question — real-world messiness. SoftwareGym builds software that gets cleanly verified via design contracts and checkpoints. That cleanliness can be both a strength and a weakness. Real work software is full of slow networks, broken states, ambiguous error messages, and the inconsistency of legacy UIs. In that case, the points you can verify narrow too. Whether this model holds up amid network latency, or in situations where it has to recover after an operation fails once, has to be evaluated separately.
A design constraint. The synthesis targets are standalone desktop applications based on Avalonia. That's a choice made to avoid the shallow interaction of browser access, but in return it ties the generatable software to this technology stack and the desktop paradigm. Whether web-only workflows or mobile-specific gestures are naturally covered by this pipeline isn't confirmed within the excerpt's scope.
Dependence on stronger models. Synthesis mobilizes top-tier models like GPT-5.5 and Seed 2.1 Pro as designers and evaluators. That is, it's closer to a form of distilling a stronger model's judgment to grow a small one. Whether this approach can exceed the performance ceiling of the evaluator model itself, and whether failure types the evaluator can't see remain baked into the data, are points worth examining. Still, the design of splitting the design/implementation/evaluation roles to reduce self-grading bias partly eases that concern.
ONTLE is a service that attaches an "AI PM" so that small businesses and solo founders can shed the repetitive work for which hiring more people is hard. An AI PM is an agent that directly operates the client's own work tools to handle a set of recurring tasks.
Let's narrow to one scene. ONTLE attaches an AI PM to some small business. That company uses an inventory-management admin it built itself. Its menu structure, its button placements, its save flow — all unique to this company. For the AI PM to take on a job like "classify newly received items, don't touch duplicate SKUs, and export the end-of-month inventory report," it has to get its hands used to exactly that admin. But training data for this admin doesn't exist anywhere in the world. And the path of just scraping the client's software to train on is blocked by PII and confidentiality issues.
The principle to borrow from SoftwareGym narrows down to one thing: "evaluate the data to be modified and the data to be preserved together." The very design by which SoftwareGym-Bench uses protected objects and distractors to force a judgment of "what to touch and what to protect" is exactly the same shape as the work instruction "don't touch duplicate SKUs." It's only an agent that reads state and knows what must be preserved — not one that mechanically follows a fixed sequence — that can lighten repetitive work without accidents.
The application hypothesis is this: instead of scraping and using the client's admin directly, start from that category and its functional spec, synthesize structurally diverse "practice admins," and train on those first. Then verify, in the actual client environment, that inventory edits, duplicate preservation, and report generation work properly. Learn in the practice ground, confirm in the real match. ONTLE's place is not to replace people but to fill the hands a company can't hire, and the gap between the synthetic environment and the real one must ultimately be closed by this final verification.
They're running programs. Each feature is implemented with everything from the entry point through state saving and screen response, and goes through GUI verification where it's launched and clicked. The grounds for this is that the average GUI task pass rate across 182 apps is 91.5%. That figure does not mean every feature works perfectly, however.
No. OSWorld's apps, tasks, and trajectories were not used in training at all — only in evaluation. That's why the rise from 41.39% to 56.97% reads as evidence of "transfer to apps it never learned."
No. On SoftwareGym-Bench the 9B reached 40.0%, ahead of Qwen3.7-Plus and several large specialized CUAs, but it's still a wide gap from GPT-5.5 (73.3%) and Gemini 3.1 Pro (70.8%). The point isn't across-the-board superiority but that "a small model, through synthetic-environment training alone, carved out a range that overlaps with models above its weight class."
It automates design, implementation, evaluation, and repair with minimal human input and generates apps in batches. That said, keep in mind as a cost that synthesis mobilizes top-tier models like GPT-5.5 (Codex CLI) and Seed 2.1 Pro. Detailed resource requirements are covered in Figure 5 of the paper.