Hey there! Welcome to Platform Weekly. Your weekly top-up of platform engineering tokens. Every week, we round up what the community is building, arguing about, and quietly panicking over.
Plus… It's your last chance to fill in the State of AI in Platform Engineering survey!
Software is and has always been priced by capacity. The servers you bought, seats you licensed, or the engineers you hired. All of it is paid based on the ability to do work, irrespective of whether the work actually happened or not. Tokens price something else. They price attempts. A developer pre-AI who manually tries five approaches when coding themselves costs exactly the same as one who nails it first time. That’s because you bought their time, not their effort. While an agent that tries 5x costs five times as much.
Foundational tech often arrives underpriced. Railways, bandwidth, cloud, all sold cheap enough to make adoption. Add an exec AI mandate above all that and you get one of the funniest and trickiest trends of the last 12 months - tokenmaxing. Devs burning tokens as fast as they can think (or not thin;), partly to learn and partly because visible AI usage pretty rapidly became a proxy for being good at your job. Lol.
Well, that bill has well and truly arrived.. Orgs that never thought about the cost of a thought are suddenly trying to forecast it, attribute it and cap it, all at the same time. That’s AI tokenomics.
A sneak peek at the most recent data (survey is still live! Answer it) from our upcoming State of AI in Platform Engineering report:
36% have centralized billing or FinOps oversight behind their AI spend
21% enforce rate limits or token quotas
16% distribute costs to individual teams
A quarter either don’t track token spend at all, or don’t know whether anyone does
Which brings me to the report recently put out by our own Sam Barlien and Pankaj Gupta of Broadcom. Their answer is an obvious but simple one. You don’t need a spreadsheet or a new fancy tracking tool. You need to think about spend as a platform component.
We identify this in the audit and attribution section of agent observability in the ADP.
Attributes every token call to a team, a path and a model, so all spend has a clear owner
Watches spend velocity rather than spend totals, because the rate of change Itself is often the most important thing.
Enforces hard limits in the platform, so a runaway loop doesn’t blow you up, cost-wise.
Produces its own audit trail, which is also the same evidence regulators will ask for anyway
Cost needs to stop being for finance to report on and become something the platform enforces instantly and by design. If your platform cannot tell you which team spent what, on which model, this morning, that is your gap to close asap.
And then the boring part;) And we know the boring stuff is the important stuff. Default idle workloads to zero. Cache what you have already paid to compute. Route cheap work to cheap models. Not so sexy, but ya do what ya gotta do.
This is the FinOps discipline platform teams already have, but pointed at probably the only resource in the world that decides by itself how much it costs and how much it wants to spend. And guess what? It always wants to spend more.
And that’s a wrap on this week! As always, this newsletter is a community project. So if you have anything awesome to share from the cloud-native world, send it my way.
Stay crunchy 🥐
Luca
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.