On 25 October 2026, Memory Tax: Dissecting the Net Value of Agent Memory (arXiv 2610.34073) appeared, by researchers at the University of Illinois Urbana-Champaign, Tsinghua University, Renmin University, the University of Waterloo and Verdent.ai. It asks a question the debate about agent memory often skips: what does a memory system add over plain retrieval from the raw history?
What was measured. Eleven systems of four kinds (flat stores, tiered lifecycles, graphs, trained policies) were run against a plain retrieval baseline on LongMemEval-s, PersonaMem-v2 and MINT-Eval, each with three language models. On LongMemEval-s, none of the eleven systems scored above two-round retrieval with any of the three models. The authors call the loss the memory tax: questions retrieval answers and memory does not. Its counterpart is the memory benefit: questions only memory solves. A system pays off when the benefit outweighs the tax.
Chart: LongMemEval-s, Table 1 of He et al. (2026). Each dot is one system with one model, measured against plain two-round retrieval; with GPT-5.6 Luna on a sample of 20 percent of the questions.
Where the tax comes from. Two sources are well documented. Summarising loses information: in a sample, one system had no memory unit at all for 38 percent of the facts. And preserved information gets harder to find, because it is fragmented, diluted or rephrased. The authors also note that several widely used benchmarks favour plain retrieval, which leaves memory little room to show its benefit there.
The remedy. The paper's simplest proposal: keep a light path back to the source, search the raw history next to the memory and rank both result lists together. In their experiment this lowered the tax across several systems.
The limits of the study. The authors state them themselves: the choice of systems is representative, not exhaustive, and further tuning of individual systems might improve their scores. The benchmarks are static, offline and in English. With the largest of the three models, LongMemEval-s ran on a stratified sample of 20 percent of the questions to limit cost.
What the brain does. Neuroscience has described this principle for decades. In the complementary learning systems account, the hippocampus stores single episodes quickly while the neocortex slowly learns what is general (McClelland, McNaughton and O'Reilly, 1995). Under the transformation hypothesis, the detailed and the generalised version of a memory can coexist, and the task decides which one is retrieved (Winocur and Moscovitch, 2011). The brain does not replace the episode with its summary; it keeps the summary beside it.
How ZenBrain does it. In ZenBrain the episode stays when consolidation turns it into a fact, and the fact stays linked to its episode. A recall searches episodes and facts together and ranks every hit on one scale from 0 to 1. It is the paper's remedy built into the architecture rather than added as a separate path.
Sources: He et al. (2026), arXiv:2610.34073v1 · McClelland, McNaughton, O'Reilly (1995), Psychological Review 102(3), doi:10.1037/0033-295X.102.3.419 · Winocur, Moscovitch (2011), Journal of the International Neuropsychological Society 17(5), doi:10.1017/S1355617711000683 · ZenBrain on GitHub (Apache 2.0).