AI ROI (Part 1)

Hi, it’s Greg and Taylor. 👋 Welcome to Supercompanies, our weekly newsletter about building the world’s most valuable companies.

We all know AI ROI is elusive and hard to measure most of the time - even if you work at an AI-enabled startup where it’s obvious AI is doing meaningful work.

If you’re six to twelve months into your AI program and frustrated that you can’t produce convincing ROI numbers, you’re not failing. You’re probably just looking for evidence at the wrong level and at the wrong time.

Here’s how I think about the actual arc of AI ROI measurement, based on what we’re living through at Section and seeing across dozens of other companies.

- Taylor

Share

The first six months is just about getting your people to build the AI habit. They’ll experiment, some of what they build will be useful, but a lot of it will be what I call exploratory automation. It works, it’s cool, but the problem it solves wasn’t that big to begin with.

But that’s what you want at the beginning. You need people to develop the habit of reaching for AI, to experience the cycle of building something, watching it break, fixing it, and eventually getting it to do real work. You can’t skip this phase, and you can’t rush it.

If you’re looking for financial evidence here, you’re either naive about the adoption curve or you’ve been oversold on the timeline. The right move is to set inference budgets, protect experimentation time, and resist the urge to measure.

Somewhere around the six-month mark, the managers who are genuinely engaged in AI transformation should start making specific, concrete observations about how their team’s work has changed.

At Section, I noticed this with my Head of Data. Last year, a lot of our 1:1s were spent prioritizing his time between internal data projects and the development of our analytics platform.

But as his AI use ramped, our 1:1s changed. He supported our migration from Iterable to Hubspot and built the core data foundation for our analytics platform at the same time. And we never discussed prioritizing between two. He also shared explicit examples of certain work that would have been impossible without Claude Code. .

I couldn’t track any of that in a dashboard. I noticed it because I’m a people manager, and I was paying attention to my 1:1s and how my teams worked.

I tell every head of AI right now: your managers and department heads are your ROI measurement system for this phase. Set up regular meetings with them and ask for their observations on how work is changing. It won’t yet be things like “we cut our agency” (yet) - it’ll be more like “we’re able to do full media pitching cycles without the agency’s support.”

This is when the financial evidence begins to emerge, but it’s still at the team level, not the org level.

Now you’re looking for savings. Probably not via layoffs - it’s a bit more subtle than that. It looks like someone quitting, and the team lead saying they don’t need additional headcount. Or your contractor budget slowly shrinking as fewer hours are needed to generate work. Or agencies or vendors being cut.

Two watch-outs here:

First, your team won’t always attribute this directly to AI. You’ll have to ask questions to understand why they don’t need a backfill, in a way that doesn’t spur them to request the backfill.

Second, sometimes your contractor budget won’t shrink automatically – people will automatically reallocate that budget elsewhere. So implement checks that force managers to re-evaluate and re-articulate why they need new headcount or even existing headcount or contractor budget.

At Section, we require every manager to get re-approval for budgeted contractors or hires when they want to make them - even if it’s already in the budget. Partly because we want to know if we still need them, and partly to understand if the existing team has found more capacity (and taken on more work) without explicitly saying so.

I see two mistakes working with the enterprise. The first is trying to skip to phase three in month six. Execs get impatient and want to see KPI movement and board-ready numbers before the team has fully built the AI habit. When they don’t find it, they conclude that AI isn’t working and start cutting budgets.

The other mistake is staying in phase one forever - celebrating the habit without ever asking whether the automations people built are actually valuable. There’s a moment, usually around month six, where you need to shift from “any experimentation is good” to “okay, which of these experiments are promising and should be scaled?” That transition is uncomfortable because you’ve been rewarding people for building things and now you’re asking them to justify what they built. But it’s necessary, and it’s every manager’s job to drive it.

It’s obvious, and really hard to voluntarily do, but it’s the hack to AI ROI that works: a pre-emptive hiring freeze. It’s a bet that your team will adopt AI and find the value - at least out of self-interest, to get all the work done without working even more hours. But if you pull this trigger, you need to combine it with a serious and sustained effort to support the team to work with AI - access to a best-in-class LLM with full capabilities turned on, lots of training, hackathons, etc.

Share

If I were presenting on AI ROI to a board right now, here’s what I’d say:

In the first year, the evidence is qualitative, directional, and manager-driven. We can point to specific workflows that changed, specific capacity that was created, and specific work that wouldn’t have been possible without AI. We can compare our inference costs to headcount costs and show that the investment is modest relative to the value our managers are observing (and to the cost of new hires).

In the second year, the evidence becomes more quantitative. We’ll see it in headcount efficiency, vendor/agency and consultant rationalization, in the team-level metrics that drive the business. We’ll be able to draw the first thin line from AI investment to dollars saved.

And yes, efficiency is not the only source of AI ROI. AI will enable some teams to create new capabilities, products and services, and hopefully revenue. That’s AI ROI Part 2 - but for now, be demanding but realistic about how you measure your efficiency gains.

Have a good week,

Greg and Taylor from Section