Heya and welcome back to Five Things AI!
Costs, cracks, challengers, towers and mirrors. This week’s Five Things is basically a group therapy session for anyone building on AI.
Benedict Evans wants us to admit we do not actually know where token prices land, someone else is convinced the whole thing is an engineering disaster that will never scale, and Mira Murati just casually dropped a 975-billion-parameter open model as if to say “hold my compute.” Meanwhile agents keep quietly rewriting the tower of Babel one commit at a time, and a philosopher gently reminds us that the tool we think we are wielding is actually wielding us. My honest read across all five: the model layer is commoditizing, the value is moving up the stack, and the only real question is whether you are building the thing on top or becoming the thing underneath. Grab a coffee. This is a good one. I know, it was about time… :)
Clearly, the situation today is transitory. On the supply side, a trillion dollars or more of data centre capex is coming down the pipe (and plenty more semiconductor capex behind that), inference efficiency continues to improve very quickly, and new models are far more (or far less!) efficient in their token use. On the demand side, although the market has been capacity-constrained since 2022, the crunch in the first half of this year has been driven by sudden product-market fit in really just one use case, software development, and that’s actually a pretty small field (imagine if we had product-market fit for a consumer use case with hundreds of millions of DAUs - today’s infrastructure couldn’t support it at any price). We don’t know what the next use-cases to scale will be, nor when that would be, nor what their token needs would be.
This is a fascinating discussion and one that is very timely as many companies are currently finding out, that AI is not for free. But once a company depends on AI, it needs to continue to spend.
The problem with generative AI, in the industry’s own jargon, is that it does not scale. The cost of growing from, say, a thousand users to a million is a key factor that venture capitalists examine when they evaluate start-ups. They want to see that the cost of adding each new user decreases over time, so that the company can support millions of users and make increasing profits. This is achieved partly through the careful engineering of computer systems that can efficiently handle more users who want to post photos, hail Ubers, or stream music.
With generative AI, the work of building efficient, scalable systems has not been done. And the problem is exacerbated by the ever-larger generative-AI models, which have grown from 175 billion parameters in 2020 to more than 1 trillion today, according to independent estimates (the actual sizes of the models powering products such as Claude and ChatGPT are secret). The large in large language model should not be a selling point.
I disagree with this article. While there are multiple gigantic challenges the need to be faced, growing AI will lead to new solutions for exactly these challenges. We tend to forget how quickly technology evolved in the lat 30 years and I do not see why the pace of development should slow down - on the contrary: AI can help us tackle these challenges even quicker.
Thinking Machines Lab, the company led by Murati, released its first AI model on Wednesday—and did it with “open weights,” meaning others can modify it with their data. Called Inkling, the model has 975 billion total parameters, making it far smaller than estimates of the most advanced closed-source models from rivals such as OpenAI and Anthropic.
“We trained it to be a broad, balanced foundation model: strong across many domains, flexible enough to adapt. Inkling is not the strongest overall model available today, open or closed,” the company said.
Thinking Machines’s push into the decentralized ecosystem of open-weights AI models comes amid a broader industry backlash against the “walled garden” approach of frontier labs such as OpenAI and Anthropic.
I don’t know much of this launch is substance vs. hype, but Thinking Machines really sounds promising. Hopefully this will stir up the AI ecosystem a bit more.
As I said many times before: agents do not feel pain, only humans do. Agents now let us act in parts of the system where we would previously have needed other people and in code bases where the people would have revolted.
When I look at some vibecoded scaled-up projects the codebases become Babel not because nobody can communicate, but because nobody needs to. Every developer has a tireless translator that can explain a corner of the tower and make whatever local alteration they ask of it. The changes keep landing, even as the architectural language that would let the humans reason about them together disappears.
It really is fascinating. I can build platforms on the software stack I want and if I understand enough about what I want to build, coding agents will help me build better software than I could ever do myself.
I confess that I am astounded by how blithely some insist that it is all as simple as learning to use AI well, as if we had not just undergone a nearly 20-year, society-wide experiment showing that a so-called “tool,” say a smartphone or a social media platform, will (mal)form even the most vigilant and virtuous user into its own image and shape. This is the blindness at the heart of modern technological hubris. It is the firm but misguided conviction that our “tools” exist entirely outside of us and thus, if taken up with requisite skill, can be “safely” deployed.
But AI is not a tool in this sense, it is an environment which envelops the user and works on us from the inside out while we naively think that we remain unchanged by our use so long as we are using it carefully and intentionally. The care and intentionality is beside the point, and our confidence in such vigilance probably works against us in the long run.
This really is food for thought. I’ll discuss this with my AI…
We all look at the prices various AI lab demand for their brilliant new frontier models. Ben Evans analyzed what’s happening right now and how it will evolve. Here’s my bet and it is simpler than the whole essay: costs come down, capabilities converge, and within a year or two we stop obsessing over token pricing the same way we stopped obsessing over the price of a database query. LLMs become a commodity. Evans spends a lot of words on “we don’t know” and he is right to, but the base case he keeps circling back to is the one that matters for anyone actually building: the model layer gets cheap and undifferentiated, and everyone is training on the same science and the same data and landing in roughly the same place. That is not a tragedy. That is the setup.
The mobile data comparison is the part people should sit with. Traffic went up by orders of magnitude, it turned into a trillion-dollar industry, and the carrier stocks went nowhere because all the value got captured further up the stack. Selling tokens is selling bits. It is an opaque unit that maps to no use case and no ROI a CFO can sign off on, so it gets bundled and forgotten. If you are building right now, the takeaway is not “which model is cheapest this quarter” - it is that the margin lives in the tooling, the proprietary data, the workflow, the trust. Every SaaS company is a database wrapper. In a couple of years every serious AI company is an LLM wrapper, and that is a compliment, not an insult.
For those of us building in Europe this is quietly good news. If capability converges and the value moves up into product, data and trust, then where you run inference stops being a capability trade-off and becomes a choice you make on sovereignty, cost and control. The frontier stops being the moat. What you build on top of it, and who you build it for, becomes the whole game. I would rather bet on that world than on the one where two or three giant minds run half of everything and set the terms - and every dynamic Evans lays out points away from that outcome anyway.
If you missed last week’s edition of Five Things AI, you can read it here:
That’s it for Five Things AI this week! 🤖
— Nico
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.