I need a place to organize my thoughts and stop sending email to myself so I will try to do this here...
The efficient organizational unit in the agent era is n=1. Or at most 2-3 people with clearly delineated ownership boundaries. Not a team. Not collaborative chaos. Individual sovereignty, augmented by AI. This isn't a prediction about remote work or the gig economy. It's about the fundamental economics of coordination in a world where one person with agents can do what used to require a…
There's a simple question that tells you whether you can trust an LLM for a task: Can you close the loop? If you can verify the output — through tests, experiments, iteration, or just looking at it — you're fine. Use the model. Ship fast. Iterate. If you can't, you're in dangerous territory. That's when you need to think harder about what the model actually knows versus what it's pattern-matching.…
LLMs don't do real causal inference. They pattern-match. They confuse correlation with causation. They fail when you perturb the examples they've memorized. But that doesn't mean they're useless for causal work. You just have to know where to deploy them. The punchline: treat LLMs as powerful interfaces to causal workflows, not as reliable engines of causal inference. They're good at hypothesis…
Last time I covered the research on whether transformers can learn causal structure: Part 1: The theoretical and empirical case that they can Part 2: The reality check — their world models are less coherent than they appear Now the practical question: what do you do with this information? Three Principles for Deployment 1. Don't Trust Benchmark Performance A model that scores well on your test set…
Last time I covered two papers showing transformers can learn causal structure — both theoretically and empirically. If that were the whole story, we could relax. But a third paper complicates the picture significantly. The Myhill-Nerode Test Vafa, Chen, Rambachan, Kleinberg, and Mullainathan asked a different question in "Evaluating the World Model Implicit in a Generative Model." Instead of…
Last time I wrote about whether transformers can escape the Lucas Critique — the idea that models trained on historical correlations fail when policy changes the underlying system. That post raised a deeper question: Can transformers learn causal structure at all, or are they just very good at pattern matching? Two recent papers make a surprisingly strong case that yes, they can. The Theoretical…
In 1976, Robert Lucas wrote a paper that broke macroeconomics. His argument was deceptively simple: if you build models by fitting historical correlations, those models will fail the moment policy changes because it changes the underlying data generation process the model was built upon. The reason? People adapt. When the Federal Reserve changes interest rate policy, consumers and businesses…