Picture a meeting where one attendee reopens the entire set of past minutes from the top every single time. The more meetings pile up, the more there is to re-read. Another attendee does the opposite: at the end of each meeting they distill the essentials into their head and show up to the next one carrying only that summary. They never flip back to the originals, so they travel light, but details can slip away during the distilling.
MemT-1.0 is an AI that aims to be the second attendee. Rather than re-reading entire past conversations every time, it presses that content into a "memory" inside the model and answers by consulting that memory alone.
Today, large language models have two main ways to remember the past. Either they stuff prior conversations or documents back into the input (long context), or they retrieve relevant fragments from an external store and paste them in (retrieval-augmented generation, RAG). The premise is the same in both: to remember, you have to re-read the source.
That premise turns into a cost as the record grows longer. A transformer holds onto the intermediate results of past tokens (the small units a model processes text in) as a "KV cache" and reuses them, and this cache swells in proportion to the length of the record. For a 128K-token record, you have to keep all of it sitting in memory. Latency and storage cost rise together. RAG lightens the resident burden, but it adds the work of searching for what to pull out each time and pushing those fragments back in.
MemT-1.0 asks a different question. Just as reasoning and multimodality have been absorbed from external tools into the model itself, memory too should be a native capability of the model. The authors' earlier model, Metis, first floated this idea, and MemT-1.0 is the follow-up that inherits its structure and changes how memory is stored and retrieved.
The mechanism fits in one sentence: when new information comes in, the model edits its internal memory with that information, and when it answers, it consults that memory.
One misconception needs clearing up here. This "memory" is not a human-readable summary. It is a numeric state that gets nudged and updated as information arrives. Think of a bundle of numbers that shifts as it soaks up the conversation.
Memory splits into two layers. One is the state that changes moment to moment as the conversation unfolds — the content that actually accumulates in this particular conversation (local memory). The other is the processing scheme that captures "how to remember"; once training ends, it is fixed and does not change during a conversation (hyper memory). The former is refilled from scratch for each conversation; the latter is the rule the model carries into every conversation.
The structural backbone is an ordinary causal language model (the usual kind that predicts the next word by looking only at what came before) with memory blocks laid on top. In some layers the memory block runs a memory attention alongside the original attention (the computation that picks which words in the current sentence to focus on). The key parameters of the original layers are left untouched. Existing abilities stay as they are, and memory is added through a separate path.
The memory block carries two homework problems: how to edit memory stably with incoming information, and how much memory to hold.
The first is handled by a token-level update mechanism (GDN-2). Every time a token comes in, three gates regulate what to keep and what to change. They trim old content, write in new content, and gradually dim the whole. Repeated adding and erasing can make the memory values drift to one extreme and blow up, so the authors refined the update operation to prevent this divergence (the math is set aside at the end of the post).
The second is handled by a capacity-expanding mechanism (Mixture-of-Storage). Memory is divided into several slots, but all slots are used together both when writing and when reading. This grows the memory capacity used during a conversation and the memory parameters obtained through training in tandem. The detailed behavior is likewise set aside at the end.
Performance was examined along four axes. Before reading the numbers, it helps to draw one distinction. The two released models sit on different backbones (the base model before the memory block is laid on). The gains from post-training the backbone itself and the gains produced by native memory are separate numbers. Mixing them wrongly credits the backbone with memory's accomplishments.
Memory. Across 11 memory benchmarks, it was pitted against two kinds of baseline. One is the "parametric memory" approach that stores information in the model's weights or state — the same family as MemT-1.0 — and the other is RAG, which pulls the source text from outside and re-reads it. MemT-1.0 beat the same-family parametric-memory baselines on average. This result came under the condition that it draws on past information at inference without looking at the source text. Against RAG, it did not come out ahead but held its own at a comparable level.
General capability. Specializing in memory tends to chip away at ordinary task performance. After adding the memory function, the authors separately checked general-task performance with an evaluation they call the "noise setting," and general capability was largely preserved.
Efficiency. They compared against the full-context approach while increasing the record length. What is being measured here is the cost of handling the same length, not the accuracy of the answer.
| Measured axis | Condition | Result |
|---|---|---|
| Inference speed | Nano, 128K record, vs. full context | About 1.86x faster |
| Resident storage size | Nano, 128K record, vs. full-context KV cache | About 66x smaller |
Both figures are from the condition where Nano went head-to-head with the full-context approach on a 128K record. §6.5 of the paper measures the 512–256K range and reports that when the record is short it is generally slower than full context, becomes comparable around 32K, and is about 1.20x, 1.86x, and 3.99x faster at 64K, 128K, and 256K respectively. The efficiency advantage grows as the record lengthens, and below a certain length it is a net loss instead.
Agents. They looked at whether putting procedural experience — "how did I solve a similar problem in the past" — into memory makes an agent perform better. On the coding benchmarks SWE-bench Verified and SWE-bench Multilingual, they compared agents with procedural experience added against those without. They looked not only at average success rate but also at consistency across repeated runs. The same problem wobbles in outcome every time you run it, and they checked whether that wobble shrank. The experience-equipped side raised both metrics and was competitive even with the approach of feeding the experience directly as text. It is a signal that memory handles more than simple information recall — it also guides execution that stretches across multiple steps.
MemT-1.0's biggest strength is that the resident state stays a constant size even as the record grows longer. And that is exactly where the sharpest question arises. If the vessel is fixed in size, then as you keep pouring information in, what stays and what overflows?
Imagine a project where a single decision was reversed three times. A fixed memory faces three competing demands at once in this one scene.
| Demand on memory | How it breaks under fixed capacity | What to check separately |
|---|---|---|
| Know the latest conclusion now in force | Overwriting toward the latest loses the reason for the change | Does it recall the latest decision without error? |
| Distinguish the change history — why it changed | Trying to hold all the history fills the vessel fast | Does it distinguish earlier decisions from the latest one? |
| Trace back the rationale of each decision | Compression smears the rationale | Does it link each decision to its source rationale? |
The three demands are a trade-off within fixed capacity. A common failure looks like this: it nails the latest conclusion exactly, but cannot explain why it changed to that conclusion. The paper shows as far as the fact that the resident state stays constant. Whether all three demands are kept together as reversals accumulate cannot be settled from the summary result alone; it only surfaces when you measure each of the three items on the right of the table.
There are also gaps beyond the published numbers. §6.6 and Figure 9 of the paper compare storage cost head-to-head with DenseRAG, but a direct head-to-head figure for inference speed against RAG is not published. How robustly it keeps up when facts change mid-stream matters especially in real use, and the authors set that case study aside in Appendix G. And the fact that memory is an internal state of the model is double-edged. Unlike a RAG fragment you pull out and read from outside, memory pressed into the model is hard for a person to peer into and see what it currently holds.
The authors also left follow-up tasks of their own: a loop in which what is learned from memory flows back into the weights, and work on filling knowledge gaps with synthetic data. Both, they write, are still at the stage of closing problems one at a time.
ONTLE is a service that dispatches AI project managers. Suppose an AI PM takes on a three-month project for a client. Meetings are held daily, the point of contact changes, and a request arrives to go back to the approach discussed earlier instead of the one settled last week. It is the archetypal setting where long interactions pile up endlessly.
Here the application hypothesis worth testing narrows to one: does the result that putting procedural experience into memory reduces run-to-run wobble carry over into consistency on recurring judgments? For example, once "this client wants reports in a specific format" enters memory, the risk of producing a different format each time in the same situation should drop.
Rather than taking the hypothesis on faith, it is better to set up a round and check it. Mix report-format changes and decision reversals together into a three-month project record, and measure three things separately: how well it adheres to the report format the client wants (format compliance rate), whether it recalls the last-finalized decision without error (latest-decision recall), and whether it can tie that decision back to the source minutes it came from (source linkage). You have to watch whether the latter two hold steady while the first rises — only then can you judge whether to put it on a long project.
For source rationale that needs an audit trail, it is more realistic to also keep a design where that record is stored separately as something you can pull out and read, apart from the internal-state memory.
MemT-1.0 is still a research-stage architecture, and even its core efficiency figures come from a specific condition. It is less a technology to plug into a product right now than a signal that the direction of structurally reducing the memory cost of long interactions is starting to be demonstrated. If you deal with long projects, it is well worth taking the three questions above and testing it on a small scale.
Repeatedly adding and erasing per token can make the memory state's values drift to one side and diverge. The authors wrote the update operation in a symmetric form so that the adding side and the erasing side stay balanced. This suppresses values snowballing out of control (details in Appendix C of the paper).
The name brings Mixture-of-Experts (MoE) to mind, but the behavior is closer to the opposite. A typical MoE selects only some experts per token. MoS has every memory expert participate every time. Each expert holds an independent memory state, and a read router and a write router separately decide how to read and how to write. The two routers move over a shared "expert address matrix," but they learn a different projection for reading than for writing. Even when the same drawer number is used, the hand that puts things in and the hand that takes them out are each optimized.
Two were released: the lightweight Nano and the larger Flash. In the table, "B" means billion and "M" means million.
| Model | Total params | Active params | Dynamic memory | Backbone |
|---|---|---|---|---|
| MemT-1.0-Nano | 36B | 3B | 10M | Qwen3.6-35B-A3B |
| MemT-1.0-Flash | 288B | 17B | 336M | Post-trained DeepSeek-V4-Flash |
Active params are the portion actually used in one inference pass, dynamic memory is the size of the memory state updated during a conversation, and the backbone is the base model before the memory block is laid on.
The paper also reports the gains from a stage where only the backbone was post-trained, before native memory was attached (Figure 2 of the paper makes clear this figure is after backbone preparation without native memory). What post-training mainly raises is the reliability of solving problems it only occasionally solved, and a good part of the remaining gap is a shortage of information.
That this gap is filled by information is confirmed separately. When relevant facts were supplied as context for a hard knowledge question, accuracy jumped from 27% to 91%. This is the fact that "supplying information raises performance." That native memory can take over that supply is the next claim, and the two should be read apart.
Two models, Nano and Flash, are available (HuggingFace link below). But since the architecture is still at the research stage, it is safer to re-measure efficiency and quality in your own environment before putting it into production.
Because a single inference pass doesn't use all the parameters — it selects and uses only some. Nano uses only 3B of its total 36B, and Flash only 17B of its 288B, in actual inference. It is a design that keeps the overall scale large while paying the compute cost only for the active portion.
No. Internal-state memory is hard for a person to peer into for what it currently holds. In audit and regulatory areas where you later have to trace a decision's rationale all the way back to the source, you still need a design that keeps a pull-out-and-read record on the side, like RAG.