A retrospective note from 2023, written as the first wave of large language models reshaped expectations. It records the moment the long-standing question — what would it mean for a system to remember — found a sharper form.
A new capability, an old gap
By 2023 the question we had been circling for years stopped being abstract. Large language models could read, summarise and reason over text with a fluency that earlier systems never had. And yet, asked to hold a conversation across days, or to keep track of what a person had told them last month, they were strikingly forgetful.
The standard remedy arrived quickly: embed everything into vectors, store the vectors, and retrieve the nearest ones at query time. Retrieval-augmented generation worked, and it still does for many tasks. But using it daily made something clear that is easy to miss in a demo. A near-perfect retrieval is not a memory.
What a vector store cannot do
Picture a mind that worked the way a vector index works. It could surface passages that sounded similar to your question. It could not tell you when it learned something, or notice that two facts contradicted each other, or strengthen what it used often and let go of what it never touched. It could not connect an idea encountered on Tuesday to one encountered in March unless the words happened to be similar. It could not sleep and come back with the day's experience reorganised.
It would be an excellent search engine over its own past. It would not be a system that knows things.
The gap is structural, not a matter of scale. Adding more documents to the index does not produce any of the missing behaviours, because none of them are retrieval. They are operations on how knowledge is organised over time: forgetting curves, consolidation, association, the revision of a belief when new evidence arrives. A flat store has no place to put them.
Where to look for the missing structure
If the missing piece is structure, the interesting question becomes: which structure? Here the practitioner's instinct and the literature point the same way. Human memory is not one system but several specialised ones — a distinction between concrete experience and abstract knowledge that goes back to Tulving; an exponential forgetting curve, documented by Ebbinghaus in 1885, that turns out to be a feature rather than a flaw; consolidation during sleep; association by co-activation.
None of this is new science. What seemed underused was the idea of taking it seriously as an engineering specification — not as a metaphor for marketing copy, but as a set of mechanisms a system could actually implement and be measured against.
That reframing is what we carried into the following year. The aim was not to make a model larger. It was to give a system the structure of memory: layers with different roles, principled forgetting, consolidation as a scheduled process rather than an afterthought. We did not yet know whether it would hold together. Writing it down was the first step toward finding out.