DeepSeek-V4.1-Flash: A 1M-Token MoE Model That Cuts KV Cache to a Quarter

DeepSeek-AI has released DeepSeek-V4.1-Flash, a new model purpose-built for long-context agentic work. The headline is a single idea: it's a 552B-parameter multimodal MoE model that handles up to one million tokens, yet it compresses the KV cache that must always sit in high-speed memory (HBM) down to 890 bytes per token—roughly a quarter of the previous model's footprint. And despite that, the authors claim performance actually got better.

This post sticks strictly to what the abstract states, laying out what the model solves, how it does it, and what it means for real products.

The problem this paper tackles

How people use LLMs is shifting. It used to be that a person asked a short question and the model wrote a long answer. Now the dominant pattern is the long-horizon agent. You load in entire documents, whole codebases, and multi-turn conversation histories, and the agent keeps referencing all of it as it works. As the abstract puts it, workloads are becoming increasingly "input-heavy."

Two costs become the problem here.

The first is prefill—the stage where the model reads the entire input once before it starts generating an answer. The longer the input, the more expensive this computation gets. A lot of research has gone into reducing long-context compute, but the abstract points out that prefill remains computationally expensive.

The second is the KV cache. Think of it like a librarian's scratch notes. To avoid recomputing information about tokens it has already read, the model jots that information down on scratch paper (the cache). The longer the context, the thicker this stack of notes gets. The question is where to keep them.

• Things you consult often go on the desk (HBM, high-speed memory). Fast, but the space is very expensive and cramped.
• Things you consult less often go to the storeroom (SSD or host memory). Roomy, but it takes time to fetch them.

The abstract says a large KV cache strains HBM and SSD capacity as well as data transfer bandwidth. In the end, compute, storage, and bandwidth form the three burdens that keep deployment costs from dropping further—the core bottleneck. This paper takes direct aim at it.

The core ideas

DeepSeek-V4.1-Flash attacks the bottleneck along three main fronts.

1. An architecture that works differently when reading versus writing (CED)

The model uses a Causal Encoder-Decoder (CED) architecture, and it's an MoE (Mixture-of-Experts) design. Think of MoE as holding all 552 billion parameters but waking up only a subset of experts for each token—like owning an entire encyclopedia but opening only the few relevant pages for each question.

The clever part of CED is that it wakes up different amounts of the model at different stages. According to the abstract, during the decode stage—where answers are generated—it activates 16B parameters per token, but during the prefill stage—where input is read in—it activates only 8B. Work lightly while skimming input, work heavily when actually producing the answer. It's a design aimed squarely at the prefill-is-expensive problem noted earlier, and the abstract states it substantially improves the cost efficiency of agentic work.

2. Compressing and reusing the scratch notes (CSA2 + FP4)

To shrink the KV cache, two techniques are combined.

• Cross-layer KV cache reuse: the model is made of many layers, and each layer creates its own notes. Here, within a scheme called Compressed Sparse Attention 2 (CSA2), the layers share and reuse those notes. Instead of each layer writing the same note redundantly, they pass it around.
• FP4 KV caching: the numbers in the notes are written at lower precision (FP4, 4-bit floating point). Think of it as writing the same content in a more compact shorthand.

The result of combining the two: the global KV cache that must always sit in HBM drops to 890 bytes per token. The abstract states this is roughly a quarter of the level in the predecessor, DeepSeek-V4-Flash. The volume of notes that have to stay on the desk has been cut to a fourth.

3. Trimming the notes kept in the storeroom too (SWA Bounded Replay)

Not just the desk—the storeroom got cleaned up as well. A dedicated deployment optimization called SWA Bounded Replay reduces the persistent KV cache that must always remain in SSD or host memory to about one-eighth of DeepSeek-V4-Flash. The SWA in the name appears to refer to the Sliding Window Attention family, but since the abstract doesn't describe the internal mechanics in detail, the accurate takeaway here is simply the result: a deployment technique that cuts the notes piling up in the storeroom to one-eighth.

Beyond this, the abstract only mentions that it streamlines the DeepSeek-V4 architecture and introduces a few efficient architectural extensions. The specifics aren't in the abstract.

Results and implications

Here's a summary of the gains the abstract spells out.

The most striking part is the "smaller yet better" combination. Compressing something usually costs you performance, yet the abstract claims that despite a far smaller KV cache footprint, it delivers substantially better performance than the baseline. It also reportedly shows strong performance across both text-based and multimodal agentic scenarios. Pretraining was done on a 45T-token multimodal corpus, followed by comprehensive post-training. The model checkpoints are publicly available on HuggingFace.

That said, the abstract contains no concrete benchmark numbers—no indication of which datasets gained how many points, or against which baseline and under what conditions the comparison was made. So "substantially better than the baseline" is best taken as a qualitative statement.

Limitations and open questions

Judging from the abstract alone, the following questions remain.

• There are no supporting figures for the performance gains in the abstract. Exactly what "substantially better" means, and on which tasks, has to be checked against the paper body and evaluation results.
• Precision-lowering compression like FP4 can produce subtle losses on certain tasks (precise numerical reasoning, exact quotation from long code, and the like). The abstract only says performance is good overall and doesn't mention whether there are regimes where the compression is fragile.
• The concrete behavior and reproduction conditions of CED, CSA2, and SWA Bounded Replay can't be pinned down at the abstract level.
• Figures like 890 bytes/token, 1/4, and 1/8 are all relative comparisons against DeepSeek-V4-Flash, the company's own prior model. There is no direct comparison with models from other families in the abstract.
• FP4 KV caching and CED's asymmetric activation presuppose matching inference kernels and hardware support. Whether you can enjoy these benefits as-is in a typical serving environment needs separate verification.

These details have to be confirmed against the paper body and the released checkpoints.

The ONTLE take

ONTLE is a SaaS that dispatches AI project managers. An AI PM is a textbook long-horizon agent. It carries project records, past meeting notes, issue history, and team context over a long span, making judgments across many stages. This overlaps exactly with the "input-heavy work" the abstract points to.

Here's what this research suggests for the product.

• The cost of a long-context agent doesn't come from just one place. The abstract flags prefill compute, storage capacity, and transfer bandwidth together as core bottlenecks. The point that the KV cache strains HBM, SSD, and bandwidth all at once explains why a service like an AI PM—one that keeps accumulating context—gets more expensive over time. If the structure is such that the burden grows as conversations lengthen, then cache compression is directly operational cost savings.
• The idea of handling prefill and decode separately is sound. Just as CED works lightly when reading input and heavily when producing an answer, a real service can save cost by treating the stage of reading in a massive body of project documents differently from the stage of actually making a judgment. It also aligns with the strategy of deploying different-sized models for the retrieval/summarization stage versus the final generation stage in a RAG pipeline.
• A 1M-token context and multimodal support widen the options in product design. Instead of chopping documents into tiny pieces, it becomes genuinely feasible to load relevant material whole and reference it. That said, "you can load it" and "loading it is best" are different things. From ONTLE's perspective, RAG that carefully selects only the context you actually need still wins on both cost and accuracy in many cases.
• Releasing open checkpoints has real value for tracking the technology. Being able to verify these efficiency trends directly—rather than relying solely on a commercial API—matters for any service wrestling with its own infrastructure cost structure.

In short, this paper is a case of pushing the "longer context, cheaper" direction along the specific axis of KV cache compression. For anyone running long-horizon agents as a product, it's a reminder that, before any performance benchmark, "how do we shrink the memory footprint" is itself the service's margin.

References

• Paper: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
• Authors: DeepSeek-AI, Anyi Xu, B. Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, and others
• arXiv: https://arxiv.org/abs/2609.19969
• HuggingFace: https://huggingface.co/papers/2609.19969