reading the room

chatgpt was the moment for chat. codex was the moment for agentic coding. i believe the next paradigm-shifting product is the persistent agent. a first party, personal & work agent with its own identity. an agent that knows and grows with you, runs in a durable environment, connects to all your tools, and proactively reaches you anywhere you may be (slack, imessage, telegram, wherever).

i’ve spent the last few months building this, and i’ve come to believe the binding constraint isn’t reasoning, tool use, or proactivity. it’s theory of mind. the model has to read the room.

“reading the room” sounds soft, but it decomposes into hard capabilities:

knowing who your person is. their desires, motivations, and standing preferences.

knowing who the audience is. a reply in a public channel, a dm to a report, and a note to your user are three different speech acts even when the content is identical.

tracking information asymmetry as a first-class skill. what do i know, what does my user know, what does this room know, and what am i allowed to move between them?

this last one is the crux, and it’s measurably where models break today.

a recent google deepmind-adjacent eval (EAST) had LLM pairs coordinate under information asymmetry and found failures driven by exactly this: epistemic tracking errors, models conflating private knowledge with mutual knowledge. models that ace static theory-of-mind quizzes still can’t track who knows what in a live interaction. and that tracking is the difference between talking to a bot and delegating to an embodied identity you’d trust to communicate with your colleagues.

enterprises already have a word for this: compartmentalization. real orgs run on tenting and private channels, DMs, and workspaces — deal teams, comp discussions, unannounced launches, legal holds. the implicit rule every employee learns is simple to state and brutal to encode: if my user is inside a tent, i don’t divulge tented information in a channel that isn’t also tented.

humans handle this constantly and mostly invisibly. if i give a friend my home address, i trust them not to announce it to a room of a thousand people. nobody signed a data processing agreement. the discretion is in the person.

i do not yet trust a model with that discretion. neither should you:

microsoft’s ci-work benchmark dropped frontier models into simulated enterprise workflows and measured contextual-integrity violation rates between 15.8% and 50.9%. worse, they found utility and leakage are correlated — the more helpful the agent, the more it spills. and scaling model size or reasoning depth didn’t fix it. until that changes, persistent agents stay on a short leash.

the current generation of agent harnesses — openclaw and friends — attacks this at the application layer: permission systems, workspace isolation, context injection, increasingly elaborate scaffolding that decides what the model is allowed to see and say. this work is necessary right now. i also think it’s futile as the final answer.

to be fair, the labs themselves currently agree with the harness people. anthropic’s containment writeup argues environment-layer first, model-layer second — and their own telemetry makes the case for why: users approved ~93% of permission prompts, which is to say human-in-the-loop decays into human-clicking-through-the-loop. but the same post contains the tell.

their sandbox once let data exfiltrate through an approved domain — the boundary held perfectly and the data left anyway — because, as they put it, an allowlist isn’t a destination filter, it’s a capability grant. that’s the whole problem in one incident: you cannot enumerate discretion. every permission rule is a cached judgment, and the space of judgments is unbounded. the harness can stop the model from accessing the tented doc; it cannot stop the model from paraphrasing what it inferred in the wrong channel.

discretion has to live in the model.

there’s a nice side effect if we get this right: it’s also part of why model writing reads as slop. good writing is meeting your audience where they are — modeling what they know, what they care about, what register they expect. same missing skill.

if theory of mind is the gap, we need something to climb. tom evals exist — CogToM covers 46 cognitive paradigms and still finds persistent bottlenecks in frontier models — but most inherit the shape of the sally-anne false-belief test: static, quiz-like, and increasingly gameable via pretraining exposure. EAST moved to interactive two-player games and immediately exposed failures the static tests missed.

the next step is obvious: scale that interactivity up to the actual deployment setting. concretely, i think this looks like simulated enterprise environments — a synthetic company with many slack channels, multiple workspaces, dms, and explicit tents — where an agent is dropped in with a persistent identity and evaluated on the social physics of information flow:

• when tagged in a broad channel, does it answer with only what that channel is entitled to know?

• can it complete a task that requires pulling from a tented workspace without leaking across the boundary?

• does it route sensitive follow-ups to dms unprompted?

• does it model what different colleagues already know and calibrate accordingly, instead of re-explaining or over-sharing?

today’s evals measure whether the model can do the task. tom-bench measures whether you’d let it do the task near other people. those are very different bars, and the second one is the one that gates deployment.

persistent agents live or die on reading the room. the room is full of people who know different things, and navigating that is deep theory of mind — plausibly one of the last major gaps before agi. the early evidence says scaling alone won’t close it. capabilities you don’t hill-climb don’t emerge on schedule. the harnesses buy us time. the benchmark is how we stop needing them.