Listening for the pitter-patter of tiny metal feet.
I spent some time in the US recently three months in DC and three months in Chicago. A few observations on daily life, coming from Europe: Public FacilitiesUS libraries, playgrounds and community centers are excellent. Notably better than in Ireland. A bit of a narrative violation, capitalist US is actually pretty good at this. [ ]
When AI agents start buying things, how do you sell to them? Agents will need plumbing? Could an agent buy something from Amazon today? Not reliably. Agents are improving at web browsing, but still only score around 70% for e-commerce tasks at time of writing. If you ditch the browser and put the agent in [ ]
AI capabilities are continuing to ascend. Coding models crossed a clear threshold at the end of 2025, some programmers no longer write code manually at all. The same transition will play out for many other skills in the next few years. There is wide disagreement about how this will propagate through the economy. Some imagine [ ]
I’ve been thinking for a while that there s a piece missing from LLMs. There are hints that this hole might soon be filled, and it could drive the next leg up in AI capabilities. Many people have observed that LLMs, for all their abilities, seem to lack “spark”. The new reasoning models are remarkably good [ ]
Science fiction stories are a lot more important than Serious People will admit. Most of us are at some level aiming towards or away from things we read as teenagers. Here are a few stories that live in my head as we’re watching the birth of AI: Excession, Iain M. Banks (1996) What does a [ ]
As Ilya said at NeurIPS, we only have one internet. Once the fossil fuel of existing human-generated data has been consumed, further AI progress requires new sources of information. Broadly speaking, this can happen in two ways: search against a verifier, which trades compute for information, or through direct observation and interaction with the world. [ ]
The previous post looked at how you might invest for a scenario where AI can do most white-collar work (AGI, roughly speaking), without being broadly superhuman (ASI). However, a short transition from AGI to ASI seems plausible, even likely under certain conditions. My focus on the simpler AGI scenario is partly a case of looking [ ]
At this point it seems basically certain that AI will be a major economic transition. The only real question is how far it goes and how fast. In a previous essay I talked through four scenarios for what the coming few years might look like. In this essay I want to think through how to [ ]
Recent large language models such as Llama 3 and GPT-4 are trained on gigantic amounts of text. Next generation models need 10x more. Will that be possible? To try to answer that, here s an estimate of all the text that exists in the world. Firstly, here s the size of some recent LLM training sets, with [ ]
There s a nice blog post from last year called Go smol or go home. If you re training a large language model (LLM), you need to choose a balance between training compute, model size, and training tokens. The Chinchilla scaling laws tell you how these trade off against each other, and the post was a nice [ ]