Picture an AI agent working through a long task. It calls tools dozens of times, and when it runs out of room to keep the record, it compresses everything so far into a short summary. The moment it finishes compressing, it reruns the exact command that just failed.
Why does this happen? The summary keeps the big picture of "what was done." But it drops two things. One is "why that command failed." The other is "which source this conclusion came from" — the thread that connects a claim to its evidence. Lose the first, and the agent repeats the same mistake. Lose the second, and it writes a plausible sentence in the report while pinning it to the wrong source.
REMORY targets these two gaps. At the moment of compression, it reads both the original record and its summary to build a single "auxiliary memory." The human-readable summary stays as is, and alongside it sits a memory that only the model reads. So the flow goes like this: original record → (human summary + model-only auxiliary memory) → next action.
To give the result up front, it splits two ways. The wasted effort of recalling the same command or repeating the same failure went down. And the ability to connect a claim to its proper source in a report went up. On the other hand, it did not dig up new content that wasn't there before. The memory helped with "connecting properly," not "finding more."
The longer the task, the more observations, tool outputs, and decisions pile up. The context window (the input space a model can see at once) is fixed. So at some point the past has to be squeezed down. The /compact command in Claude Code or Codex CLI does exactly this. It turns the earlier conversation into a text summary to free up room, and that summary becomes the checkpoint for the work ahead.
So the input length moves like a sawtooth. It accumulates, drops sharply at compression, and accumulates again. The problem lives in that valley — right after compression. The summary preserves the broad flow, but there's no guarantee it will support every decision still to come. On top of that, whether information that matters now will still matter an hour later is something you can only know when you get there.
This is where the study's premise sits. Pre-selection of the "just add these three sentences to the summary" kind doesn't work. This is a situation where you can't pick in advance what will turn out to be useful.
Existing approaches to long context fall into two broad camps. One stores the past as human-readable text and retrieves it on demand when needed. The other squeezes long text into a bundle of numbers that only the model reads.
REMORY overlaps the two. It leaves the text summary and retrieval untouched. On top of that, it lays down one more layer: a numeric memory built by looking at the summary and the original together. When precise past detail is genuinely needed, the agent still pulls it out with ordinary tools like file search. What REMORY fills is the moment just before a retrieval call is made — the exact instant the model picks its next move. It's not trying to remove retrieval; it fills in the share that can be handled before you ever have to reach for it.
Let's be precise about what this memory is. What the network produces isn't words but a bundle of numbers (continuous vectors) that the model reads directly as input. It doesn't rewrite the past into text; it sits in the model's input slots in numeric form.
There's one rule. The working model that actually does the job is never touched. The learned values inside that model (its weights) stay fixed, and only the network that builds the memory learns. Learning flows through the frozen working model and only toward the memory side.
Training has two stages. Stage 1 is a reconstruction drill. Even after a passage of the original text is turned into memory, the model is taught to bring that passage back. The point is to make the memory hold the original information. Stage 2 divides the roles. The side that has only the summary and the memory is made to produce the same next response as the side that saw the full record. Since the summary is already given, the memory naturally orients itself toward the gaps the summary couldn't fill. So the memory is allowed to overlap the summary in content, and it doesn't need to reconstruct every missing fact. There's no reason for the memory to shoulder information the summary already provides. That's where the "RE" in the name comes from — it lays only the "residual" on top of the summary.
- Compression unit: one block of up to 1,024 original positions produces 64 soft memory tokens. Hierarchical compression bundles the final memory into 4,096 slots or fewer while preserving the original order. - When passing to the working model, the soft memory tokens are concatenated after the summary. Gated cross-attention is used so the memory network can reference the summary while generating the memory. - Architecture: DFlash is applied to Qwen3.8-27B, and DFlash-2 using group dynamic convolution to GLM-5.3-Flash. The memory network's trainable parameters are 1.94B and 1.502B respectively. - Stage 1 data: the text fields of MMFineReason-Full-2.3M. The teacher is instructed to repeat the original text verbatim, and the student makes the same prediction with that passage replaced by soft memory. - Stage 2 data: reasoning and coding trajectories. It includes prompts from Open-PerfectBlend with precomputed summaries and Qwen-generated responses, following a curriculum ordered by preceding context length. - Optimization: reverse KL from student to teacher. The student, holding only the summary and memory, is made to imitate the next-token distribution of the teacher that saw the full history. The loss spans the whole vocabulary and the entire response, with a slight extra emphasis on earlier positions. Only the memory parameters are updated.
The experiments split into two questions. Does the memory actually learn the information the summary dropped? And did things improve simply because more room was granted?
Source attribution is measured with SummHay — a benchmark that tests the ability to find insights relevant to a question from a pile of long documents and connect them to their sources. Four conditions are compared: full context (raw), summary only, summary++ (the summary plus the same amount of extra text), and summary + REMORY.
The key is summary++. It fills the same amount of room REMORY uses, but with "generated text." This condition blocks the objection that "it only got better because you gave it more room."
Agent performance is measured with BrowseComp, Terminal-Bench 2.1, AutomationBench, and JobBench. For each comparison, the execution environment, prompt, tools, budget, generation settings, and compaction policy are all fixed within the same working model, and only the presence of memory is varied. Both sides can retrieve the precise past, and the only intervention is adding up to 4,096 slots at compression time. When a gap shows up, the cause can be narrowed to the memory.
- SummHay: all 92 queries across ten collections, evaluated with Qwen3.8-27B. Answer generation uses no tools or retrieval; the ground-truth insights and citations are used only for scoring. Scoring is by GPT-5.6-Sol. Coverage and joint score are averaged over all 621 reference insights; citation F1 is averaged over the retrieved insights. - Context window: Qwen 262,144 positions, GLM 1 million positions. - Task counts and runs: BrowseComp's 1,266 and Terminal-Bench's 89 are run once with both actors; AutomationBench's 600 and JobBench's 65 are run five times with Qwen. - Judging: Terminal-Bench and AutomationBench use task verifiers; BrowseComp and JobBench use the GPT-5.6-Sol judge. - Cost basis: figures derived by pricing actor and summary tokens at September 2026 non-cached supply rates and counting residual positions as input. Original feature encoding, memory network execution, and tool costs are excluded. In other words, it's generation cost by token price, not total execution cost.
Start with behavior. Scores rose across all four benchmarks, and the improvement held up even on a stronger working model with a larger context window (GLM-5.3-Flash is a 320B-scale model with a 1-million context window).
| Benchmark | Working model | Score change | Runs |
|---|---|---|---|
| AutomationBench | Qwen | 35.5 → 45.3 (+9.8) | avg of 5 |
| JobBench | Qwen | 33.4 → 41.0 (+7.6) | avg of 5 |
| Terminal-Bench | Qwen | 71.9 → 76.4 (+4.5) | 1 run |
| BrowseComp | Qwen | 74.0 → 77.0 (+3.0) | 1 run |
| Terminal-Bench | GLM | 84.3 → 87.6 (+3.3) | 1 run |
| BrowseComp | GLM | 84.9 → 89.0 (+4.1) | 1 run |
What stands out more is how it rose. Repetition — the same tool producing the same output again within a task — dropped by 17.9–29.8%. Failed commands and tool errors dropped by 25.3–49.7% (repetition is counted excluding planning and polling; errors count failed commands and explicit tool errors). An agent that has lost the past to compression tends to make the call it just made again, or repeat the same failure, and that wasted effort shrank noticeably. Cost also fell in every reported comparison except one, JobBench ($3.09 → $3.21).
There's also a comparison that isolates just the "first move" right after compaction. Across 488 paired next-actions drawn from 100 Qwen BrowseComp traces, with the summary and resident instructions held fixed, a judge model compared them — and the memory side won 345 (70.7%), tied 21, and lost 122. That said, this is a preference over the "proposed next move," not a measurement of what that move would actually have produced.
Now for source attribution. This is the real finding of the study. Comparing summary-only against summary + REMORY on SummHay:
| What's measured (summary only → summary + REMORY) | Change |
|---|---|
| Content recall (coverage) | +0.40 — essentially flat |
| Source attribution (citation F1) | +4.04 |
| Joint score | +3.55 |
The reading is simple. The memory didn't surface new insights that weren't there. Instead, it properly connected already-found claims to their evidence. Even when narrowed to the 553 insights retrieved under both summary conditions, citation F1 rose from 57.31 to 61.41.
So was it the extra room that helped? No. summary++, which fills the same room with generated text, stayed almost glued to summary-only.
| Representation | Joint score | Mean input positions |
|---|---|---|
| Full context (raw) | 43.41 | 96,872 |
| Summary only | 39.84 | 1,393 |
| summary++ | 39.78 | summary only + extra text |
| Summary + REMORY | 43.39 | 5,024 |
Even with the same space, filling it with ordinary text yielded nothing; it had to be filled with learned memory for source attribution to rise. And summary + memory used just 5.2% of the input positions that full context uses, yet came right up against raw on the joint score.
First, what the results don't tell us. The goal of Stage 1 training is to reconstruct the original text verbatim, and the goal of Stage 2 is to retain information useful for the next action. Reviving every fact and retaining only the facts needed for the very next move are different things. The signal the current numbers show is the latter — that behavior and source attribution improved.
This distinction matters. The introduction said the memory "fills in why a command failed," but what was actually measured is the behavior of fewer failures, not evidence that the failure reasons are inside the memory. Telling the two apart requires going one step deeper: directly inspecting whether information like the "cause of failure" from the pre-compression record survives in the memory, or deliberately masking that information and watching how behavior right after compression changes. There's still this much of a gap between performance improving and understanding how the insides actually work.
The authors draw some lines too. The frontier scores in Figure 1 are published reference values under their original execution environments and protocols, not controlled comparisons. The "first move" preference result above is a preference over proposals, not actual outcomes, and the cost is on a token-generation basis, so the cost of running the memory network is excluded.
The conditions for applying it aren't trivial either. The method requires training a separate memory network of 1.94B and 1.502B scale for each working model, and that training has to pass through the working model. So it's hard to drop onto a fully black-box API whose internals you can't touch. It's a design that favors those who serve the weights directly. Since all the numbers come from two specific working models and specific benchmarks, whether it carries over to other models and domains is still an open question.
Let's draw the line first. Most of ONTLE's models run on external APIs (Gemini via Vertex, Claude). We can't layer a technique that presumes weight access directly on top of those. So what follows isn't a feature to bolt on today, but one design judgment worth testing at ONTLE. The core is this: preserve the task state you can't afford to lose at compression time separately from the summary.
Start with work where a mispinned source is immediately obvious. Say ONTLE's AI PM sweeps ten competitor landing pages and writes a proposal from that material. Each claim needs its source attached exactly — "this pricing policy is from company A's page, that benefit line from company B's page." But once compaction runs, the summary keeps the trunk — "surveyed 10 competitors, drafted proposal" — while which sentence came from which page gets blurred. This is precisely the spot this study improved.
The repetition side is the same. Suppose the same PM was calling a scheduled-publish API and it failed, but compaction dropped that failure message. Right after, it tries the same call for the same reason again. In a product that handles the repetitive work of a small business short on hands, this one wasted call is just delay and cost.
So what ONTLE would test isn't reproducing the research technique. We already know what information drives the repeated waste: the URLs already done, and the failed calls and their causes. The question narrows to one thing: does carrying this "what's already been done" separately from the summary earn its keep?
The experiment is set up like this. Build a path that records finished URLs and failed calls with their causes on a separate channel, independent of compaction. Then run the same task with and without this separate record. Four metrics to watch: repeated calls, error recurrence, task success rate, and total cost. To separate it from the effect of simply granting more room, also run a condition that fills that room with plain text, as a control. Cost has to be written down separately for token value and for total execution value, so the comparison is fair.
Nail down the decision criteria in advance. If repeated calls and total cost drop while task success rate holds, that's grounds that an "already done" channel earns its keep even on top of an external API. If no such improvement shows up, the conclusion isn't "one summary is enough." It's "with the record and delivery method chosen this time, usefulness couldn't be confirmed." The door stays open for a different record, a different method.
No. Blocks are hierarchically re-bundled to keep the final memory at 4,096 slots or fewer. The original order is preserved, but there's a cap on the total.
No. The teacher that saw the full record only plays the answer key during training. In actual production, the next response comes from the summary and memory alone.
Making the context window bigger means carrying more of the past verbatim, which eats a lot of input positions. REMORY reached a comparable joint score with far fewer input positions. It's not an approach that grows the room, but one that keeps only the usefulness of the past in less room.