Agent Topologies and Optimistic Indexes

Single-agent memory is now, mostly, a solved problem. Hierarchical narrative summarization works well enough: you compress conversation history into rolling summaries, preserve the most recent context window, and let the agent maintain coherent state over time. Knowledge graph memory is the competing approach, storing entities and relationships rather than narrative prose. The honest answer is that both are right, they are optimized for different access patterns, and what you actually want is many indexes on the same information, each serving a different kind of query.

This is what human memory does. We do not have one memory, we have episodic, semantic, procedural, emotional, spatial. The same event gets indexed multiple ways simultaneously, and retrieval pulls from whichever representation best matches the query. The fact that we experience this as a unified “memory” is a convenience, not the underlying architecture.

But this is all single-agent. The open problem is multi-agent to multi-agent memory. Where does shared state live? Who owns what? When two agents both update a view of the world, how do conflicts get resolved? Narrative memory does not trivially extend here. You cannot merge two agents running summaries without losing information or introducing incoherence. Knowledge graphs have better merge semantics, but they still do not tell you where the authoritative record lives.

As you build multi-agent systems in practice, you realize quickly that the memory and coordination primitives most agent frameworks offer are inflexible and limiting. The possibility space of LLMs goes well beyond coding agent teams, but if we’re really going to have dozens of agents on dozens of machines running around the clock - how are they going to manage all that information?

Just as agent orchestration is rediscovering the actor model, agent memory is rediscovering databases.

The framing of “individual vs shared memory” is the wrong question. When we elected to store information in normalized databases and use indexing to enable diverse access patterns, it allowed us to stop quibbling over filing system structures. We can do the same for agent memory. It doesn’t matter if you have one, few or many agents. What matters is how you store, index and query information.

We can be smarter than sticking every bit of feedback in AGENTS.md or MEMORY.md. Think of the kind of questions you want to ask and the actions you wish you could take in your agentic loops, then make them readily available. Every request should map, ideally, into a single command for the agent to execute that instantly produces the result. Every additional command is effectively a “cache miss”1 that could be fixed by better structure of context.

If you know how data will be read and how it will be written, you can even design an agent to maintain the right indexes for those perspective. Agent memory isn’t really about shared vs. private memory, it’s about programming with agentic feedback loops to prepare information before you need it2. Each agent is scoped by the queries and mutations it is capable of, its identity is a byproduct of that scope.

As designers, our job is to notice gaps and think “how should I address this? Via a skill, a hook, a prompt, a custom agent harness, a database query, a script, a program?”

So, by indexing information we avoid cache misses. When we ask new questions, we notice the need for new indexing strategies and create agents to implement them. For me, currently, this looks like an always-up-to-date ‘recent activity’ in my notes charting the arc of my thinking over the past week. Or, tracking the evolution of ideas over time with links to the pivot points in my notes. Or, automatically researching any videogame I mention, with screenshots3.

Great, but we can do even better.

Agents can create indexes that have not yet been explicitly asked for, anticipating what will become a bottleneck for the humans (and agents) further downstream. This requires a meta-loop: reasoning not just about how to answer current queries, but about what indexes are missing from the current information environment. It is like a proactive query planner, running continuously, asking: “What question will the human ask in an hour that we currently cannot answer in under a second?”

An agent that does this well removes friction before the user ever notices it existed. That is the goal, not a chatbot. The best infrastructure is the infrastructure you never think about, because it never fails you.

Agent abstractions are mostly theatre. Regardless of vendor, subagents, system prompts, two identical agents placed in different information environments will behave completely differently.

What we are actually designing when we design a multi-agent system is the information topology: who has access to what, which indexes are maintained by which processes, how views are projected and kept consistent. The agent as a concept starts to recede. What remains is a set of skills, a set of prompts, a database, and some processes.

Think about it, what is the real difference between subagents, agent teams, parallel sessions and memory systems..? They’re all just different ways of slicing up context windows!

When you think about it that way, agent design becomes information environment design. The interesting questions are not about agent internals, instead, we should focus on boundaries, feedback loops and index maintenance.

As agents recede as a concept, what remains looks kinda like microservices..? Modular processes that maintain subdomains, communicate through well-defined interfaces, and compose into larger systems. Everyone will run their own agent topology, optimized for their own workflows and access patterns.

Then we will want to connect those topologies to each other’s networks, to existing services, to internet infrastructure. Then the design challenge becomes: how do you handle shared vs. isolated vs. partially overlapping information environments, across organizational boundaries?

The internet is already a massive shared index we all query and contribute to. Pages get indexed before you search for them. Links create a graph that search engines traverse. The architecture of the web is the architecture of a distributed index, and the things we built on top of it (search, social, e-commerce) are all applications of that indexing capability. The next layer will be agent networks running continuous projections over that same infrastructure, with richer semantics and more dynamic feedback loops.

TLS, HTTPS, and the same-origin policy were what made the web socially viable. They provided just enough trust to let you run code from a stranger’s server, make payments across organizational boundaries, and build services on top of other services without requiring bilateral trust agreements for every interaction. Agentic networks need equivalent primitives, and we do not have them yet.

The moment you want to connect your agent network to someone else’s, you are in the territory of federated identity, delegated authorization, and information boundary enforcement at scale. We have most of the other pieces. Authorization and trust models that work at agent-scale are what will hold this back4.

The set of questions that can be instantly answered is bounded by the indexes that exist. Before Google it was not normal to look up the answer to any and every question, then it was.

A similar shift is underway now, and the implications run deeper than faster search. You can now tailor your information environment to your needs by using agents to build indexes to support other agents that support you. This has massively compounding benefits but relies on making tacit, contextual, organizational knowledge explicit - which is arguably harder than indexing the public web. I believe that well-designed agent topologies and the right maintenance strategies may actually make this tractable5.

What will we be able to think that we simply could not before by virtue of such systems? What if knowledge work is mostly an indexing problem? Rather than answers, perhaps we need better questions?