DeepSeek-V4.1-Flash · the question
That memory is the KV cache. For every token it has ever read, a serving system keeps a little vector of keys and values so the next word can look back at everything. Reading more meant remembering more — and remembering is what actually breaks the bank.
DeepSeek's V4.1-Flash is a 552-billion-parameter model built around one obsession: grind that memory down to 890 bytes per token — a 437-fold reduction from DeepSeek-V1 — while the model got better, not worse.
Report Figure 1(b): 437× smaller per-token global KV than V1, 4× smaller than V4-Flash. The 388,800 B and 3,560 B figures are derived (890 × 437 and 890 × 4); only 890 is stated directly.
Four levers got it there. Each one is its own little world — pick whichever itch you want to scratch first.
where the pain lives
The KV cache isn't one problem. It's the same bytes hitting three different physical limits, over and over, at serving time:
The live cache for every concurrent request sits in GPU high-bandwidth memory. Full → fewer concurrent users → higher cost per token.
Caches worth reusing (agent turns repeat their prefixes!) are saved for later. V4 kept them resident 72h+. SSDs fill up.
Every miss means hauling megabytes from SSD to host to HBM. Once sparse attention had cut the compute, moving the cache became the bottleneck.
Prior DeepSeek models had already made computing over long context cheap — sparse attention, the trick of reading only a chosen slice of past tokens instead of all of them. The report's opening move is noticing that the leftover pain has moved — from compute to storage and transport. Everything that follows attacks those two.
Agentic workloads make it worse. Long-horizon agents fire tool calls constantly. Every turn re-sends a growing pile of context — input-heavy, prefix-repetitive, and mostly prefill rather than generation. So V4.1-Flash also activates only 8B params per token while reading, versus 16B while writing. That asymmetry is the CED deck's story.
the shape of the answer
Every trick in the paper squeezes one of four multiplicatives. Together they compound into that 890:
1 · Compute less per token. Don't run all 40 layers over the prompt. The top half projects its memory from the bottom half's output. Prefill FLOPs ≈ halved.
2 · Store once, not forty times. Layers share one global KV cache instead of each keeping a copy — and share their sparse selections too.
3 · Keep four bits. Quantize the main cache to FP4 (E2M1 + one E4M3 scale per 16 channels). Storage halves again, accuracy barely moves.
4 · Stop saving what dies young. Most of the SSD cache was sliding-window state that's only useful for minutes. Drop it — reconstruct on demand.
Lever 1 → CED deck. Levers 2 → CSA2 deck. Lever 3 → FP4 deck. Lever 4 → Bounded Replay deck.
choose your path
Also waiting when you want them: (the quantization safety argument), (FLOPs, kernels, the effort dial), and (45T tokens, an optimizer zoo, millions of sandboxes).
An explorable reading of the DeepSeek-V4.1-Flash technical report (Sep 2026). All benchmark numbers, hyperparameters, and architecture facts are from the report; figures marked derived are computed from its stated ratios. Napkin-diagram aesthetic intentional.