Structuring memory

I’ve written before about the importance of LLM memory as a mechanism for preserving state over time. The core idea is simple: loading a model’s context with relevant details avoids repetition and improves performance. But what’s the best way to actually structure those details?

What does “memory” even mean in this context? The word implies something the model has internalized, but that’s not quite right. What we’re really talking about is a set of durable artifacts that can selectively be brought into the model’s context window when they’re relevant to the task at hand. Sometimes that looks like a personal detail the model “remembers” about you (e.g., your preferred programming language, your dietary restrictions). But it can also be institutional knowledge, instructions for a workflow, orientation for a codebase, or any other useful reference that helps the model do its job.

So we know broadly that relevant context improves performance. What people are actively exploring is how to structure that context. There are many approaches in the wild right now: Agent skills rely on collections of markdown files. Mem0 uses a structured store with a graph-based variant for relational reasoning. And then there’s Beads, the project that motivated this post—a persistent issue tracker that lives within git and is designed for agent use.

Beads is Steve Yegge’s answer to a problem he encountered after extensive vibe coding. The system stores issues in a .beads/ directory, versioned and branched alongside regular code. It uses hash-based IDs to prevent merge collisions in multi-agent workflows, tracks dependencies between issues, and gives agents a flat, queryable structure to navigate.

Two things make it stand out. First, it’s designed explicitly in opposition to the pile-of-markdown-files approach. Yegge argues that hierarchical plans trigger pathological recursive behavior in agents, wherein they try to decompose everything endlessly. Issues are flat, concrete, and dependency-linked, which turns out to be a better fit for how agents actually operate. Second, it was frankensteined together through vibe coding against agent hallucinations and tool call patterns. It wasn’t designed from first principles or derived from a benchmark. It emerged from sustained, messy interaction with agents, shaped by whatever actually worked.

And anecdotally, it works well. Other developers have reported similar results, and my own experience has been positive. This pattern—a weird, bespoke tool that seems overengineered but is oddly effective—got me thinking about the structuring question more broadly. Should we all be feverishly vibe coding convoluted architectures for storing and retrieving domain-specific information?

Based on the recent research, the answer is probably not. But the reasoning is worth understanding, because there are some key architectural lessons that apply across contexts.

A log of everything that ever happened sounds comprehensive, but it quickly becomes overwhelming. Worse, it poisons the memory store. If you don’t give the agent tools to distinguish between what went well and what went poorly—between what’s still relevant and what’s been superseded—it ends up with a pile of raw experience and no way to learn from it.

Dedicated processing—in which the model reflects on and structures its own experience—helps with this. The Evo-Memory benchmark demonstrates that evolving memory increases agent performance, with improvement strongly tied to how similar the current task is to past ones. When the agent can reason about, prune, and reorganize its own memories, it solves problems with fewer steps and makes better use of past experience.

Letta’s work on skill learning takes this a step further. After completing a task, agents reflect on their trajectories—what they attempted, what worked, what failed—and generate reusable skill files that capture approaches, common pitfalls, and verification strategies. This “sleep-time compute,” where raw experience is transformed into abstracted insights between runs, yielded a double-digit improvement on complex benchmark tasks. Adding failure analysis to the reflection process contributed an additional double-digit gain, because diagnosing what went wrong is more informative than confirming what went right.

MemGPT explores the idea of memory hierarchy, borrowing from operating systems to manage different tiers of memory across a limited context window. The same pattern keeps showing up: short-term precise context, medium-term structured records, long-term abstract principles. These kinds of architectures allow the agent to traverse prior experience without freshly reassessing what is and isn’t important every time.

There are many sophisticated, structured approaches to storing memory and reference information—Zep uses temporal knowledge graphs, Mem0 uses structured stores and graph memory, Beads uses issues. Each has real engineering merit. But in many cases, this sophistication might be overkill.

Letta’s filesystem benchmark is the sharpest evidence. A filesystem with grep and search_files scored 74.0% on the LoCoMo benchmark, beating Mem0’s graph memory at 68.5%. The explanation Letta offers is revealing: “The quality of an agent’s memory often depends more on the underlying agentic system’s ability to manage context and call tools than on the memory tools themselves.” Agents are effective at filesystem operations because they’ve encountered them extensively in training data. Specialized memory APIs that models haven’t been trained on are harder to use correctly, regardless of their sophistication.

Anthropic found something similar when testing harnesses for long-running agents. Their solution for maintaining continuity across sessions wasn’t a knowledge graph or a specialized memory layer—it was a JSON feature list file, scoped to one feature at a time, with a structured startup ritual that reads progress files and checks what needs work. The feature list serves double duty: it’s both memory (what has been done, what remains) and behavioral constraint (work on this next, not everything at once). Organization shapes behavior as much as it stores information.

It’s worth emphasizing that, while these designs are relatively straightforward, they aren’t naive. Filesystems and JSON align with what agents are trained to understand and manipulate, so they work well precisely because they fit the model’s existing capabilities. Wrapping information retrieval in specialized tooling adds complexity, and it might actually degrade performance if the agent can’t figure out how to interface with your system.

More than logging or structuring, forgetting may be the most important component of an effective memory system. In ablation studies of the Titans architecture, the adaptive forgetting gate was the single most impactful component. ReMem, Evo-Memory’s best-performing method, actively discards 10-37% of stored experiences. Mem0 includes explicit DELETE operations for when new facts contradict old ones. Every successful system I’ve found has a mechanism for ejecting information.

Not everything that gets stored initially stays useful over time. At best, outdated information becomes cruft that clutters retrieval. At worst, it’s actively counterproductive—imagine an outdated architecture diagram persisting in a repository and continually steering a coding agent off course. A mechanism to eject outdated, incorrect, or irrelevant information is key to maintaining performance as the memory store grows.

Within these principles, there is a daunting amount of flexibility. There are millions of ways you could imagine setting up an agent file store, just like there are millions of ways to organize your own filesystem. Some patterns are starting to emerge from the research, but there’s nothing approaching a best practice or an out-of-the-box tool for managing contextual information over long time horizons.

Beads points toward a future that feels increasingly likely, where a blend of first principles and strange agentic patterns—the kind of structures that emerge from sustained interaction with models rather than from theory—inform the shape of tools. The research supports some of what makes Beads work (flat structures over hierarchies, alignment with existing agent capabilities) while challenging other impulses (elaborate retrieval systems, unbounded accumulation).

For now, an organized set of structured files seems as good as anything else. Getting the organizational principles right matters far more than the specific technology you use to implement them. That’s a useful finding, because it means you can start building effective memory systems today with tools you already understand. We can focus on the harder design questions: what should be remembered, at what level of abstraction, and when should it be forgotten. Those are the problems that will need solving across domains, and we’re only beginning to develop good answers.