You Are Measuring Innovation Wrong

I came across a post on Substack recently comparing the success rates of Frank Sinatra, Babe Ruth, and Pablo Picasso. Like many things that circulate online, the underlying statistics become increasingly debatable the more closely you inspect them. Babe Ruth’s numbers are objective. Sinatra’s depend on how you define a hit. Picasso’s are almost impossible to verify.

The graphic stayed with me anyway, because it captured something I think many organisations still misunderstand.

We spend far too much time judging the success rate of individual ideas, and not nearly enough time judging the quality of the system that produces them.

Years ago, I used to keep a running list on my website of the ideas and initiatives I’d worked on. Some became successful products. Some evolved into entirely different businesses. Some quietly disappeared. Others never made it beyond an experiment.

I included all of them because I thought they told the story honestly. The response surprised me though… People would point to the projects that hadn’t worked and ask why there were so many failures. To me, that was like criticising a venture capitalist because not every investment became a unicorn, or questioning a pharmaceutical company because most of the compounds it screened never reached patients. It fundamentally misunderstood what innovation is.

Innovation isn’t a sequence of guaranteed successes. It’s a portfolio of investments made under uncertainty. That distinction matters because it changes what we should be measuring. Instead of asking whether each individual idea succeeded, we should be asking whether the organisation consistently makes better investment decisions over time.

Every profession built around innovation already understands this, even if we don’t always describe it in those terms.

Pharmaceutical companies don’t expect every compound to become a medicine. Engineers don’t expect the first design to survive unchanged. James Dyson didn’t build one vacuum cleaner; he built 5,127 increasingly informed prototypes. Even a can of WD-40 carries the evidence of iteration in its name. Nobody working in these environments believes every attempt should succeed. They believe every attempt should improve the quality of the next decision.

Failure, in other words, isn’t the objective. Learning is.

The problem is that businesses often evaluate innovation as though every initiative should justify itself in isolation. If a project doesn’t generate a commercial return, it is labelled a failure. If an experiment doesn’t become a product, the investment is questioned. That’s a poor way to think about almost any form of innovation.

Most meaningful breakthroughs are built on foundations that nobody ever sees. We celebrate the product that transformed a company, but not the abandoned concepts that sharpened its direction. We applaud the strategy that worked, but not the alternatives that revealed why it was the right one.

The visible success is only the final chapter. The invisible investment is what made it possible.

This is one of the reasons I’ve always believed Product is fundamentally an investment discipline rather than a delivery function.

Every feature competes for scarce engineering capacity, scarce capital, and scarce organisational attention. The role of Product isn’t simply to find good ideas. It’s to improve the average quality of the bets the organisation eventually places.

That’s why I often say the most valuable word in Product is no.

Not because saying no is inherently virtuous, but because every unnecessary project consumes resources that can no longer be invested elsewhere. Discovery, customer research, prototyping, commercial modelling, and technical validation all exist for the same reason: to eliminate weak ideas before they become expensive ones.

Ironically, organisations that are perceived as moving fastest are often the ones rejecting the greatest number of ideas. They simply do so before engineering becomes involved.

By the time a team commits months of development effort, dozens of alternatives should already have been explored and discarded. To someone looking in from the outside, that can resemble indecision. In reality, it is disciplined capital allocation. The best Product teams aren’t trying to build more than everyone else. They’re becoming increasingly selective about what deserves to be built at all.

Artificial intelligence changes the economics of experimentation, but it doesn’t change the underlying principle. The cost of exploring ideas has collapsed. Teams can produce prototypes in hours rather than weeks, synthesise customer feedback almost instantly, and investigate strategic options that would previously have required significant investment.

That sounds liberating, but it creates a different constraint… When experimentation becomes cheap, judgement becomes the scarce resource. Generating ideas is no longer the competitive advantage. Selecting the right ones for further investment is.

In many ways, AI makes Product judgement more valuable than it has ever been.

This way of thinking has become even more apparent through my work in Product Due Diligence. The best investors rarely ask whether every attempted initiative succeeded. Instead, they’re trying to understand whether management consistently places intelligent bets.

No experienced investor expects every initiative to succeed. Markets don’t work like that, and neither does innovation. What matters is whether the organisation has built a repeatable system for allocating resources under uncertainty. That is a far more durable competitive advantage than any individual product or feature.

Ironically, I worry far more about organisations that proudly tell me they’ve never failed. Sometimes that’s true (spoiler alert: it’s never true). More often, it means they’ve never taken meaningful risks.

They’ve optimised for certainty instead of learning. They’ve become highly effective at delivering incremental improvements while quietly avoiding the decisions that might genuinely change the trajectory of the business.

When I’m assessing an organisation, evidence of thoughtful experimentation is usually reassuring. The absence of experimentation rarely is, because the real cost isn’t the initiatives that failed - it’s the opportunities you never pursued because the organisation became uncomfortable with uncertainty.

Looking back, I almost wish I’d kept that running list of successful and unsuccessful projects on my website. At the time, I removed it because people interpreted it as a scoreboard. They assumed every unsuccessful initiative represented poor judgement, rather than recognising it as evidence that meaningful experimentation was taking place.

Today, I have a different view. I’m instinctively more sceptical of organisations that appear never to have failed. Either they’re extraordinarily gifted, or they’re placing far too few bets.

The companies that consistently outperform over long periods aren’t those that avoid mistakes. They’re the ones that have built a better system for deciding what deserves investment, what deserves another iteration, and what deserves to be abandoned.

Individual successes matter, of course - they’re the stories that appear in case studies and conference presentations - but they’re usually just the visible outcome of an invisible portfolio of decisions that nobody else ever gets to see.

Perhaps that’s what we should be measuring instead.

When I’m conducting Product Due Diligence, I’m rarely trying to determine whether a company has been consistently successful. I’m trying to determine whether it has built a repeatable capability for making increasingly intelligent investment decisions.

Markets change. Technology changes. Competitors change. No management team can predict every outcome correctly. What separates exceptional businesses isn’t that they’re always right. It’s that they continually improve the quality of the bets they choose to place.

Over time, I’ve found myself returning to the same set of questions…