Design Experiments Are More Important Than Ever, and They Don’t Require a Lab

I’m about to say something that would irritate my old academic advisors and half the people I used to work with.

You don’t need nearly as much scientific rigor as you think to run a design experiment.

That’s something I’ve learned after talking with 260+ designers. Experimentation is increasingly becoming a bigger part of what design does, whether that’s running AI experiments in your process or laying out a hypothesis so other people can actually see the risk you’re weighing.

But the word “experiment” makes a lot of designers freeze, so they avoid it entirely.

So here’s the reframe. If the word “experiment” makes you tense up, just call it a bet. That’s what many design leaders do, and I’ll show you exactly what it looks like.

I understand the freeze, because I went to grad school and teach designers how to work with data. I’ve watched whole classrooms shut down the moment A/B testing comes up.

Minimum detectable effect. Statistical power. Sampling bias. A/B versus multivariate testing. P-values. These are terms most designers never expected to run into, and unless you’re running enterprise-wide research studies, you don’t.

Some organizations (and researchers) require that amount of scientific rigor, but it often does more harm than good. It convinces designers that to “really” experiment you need a dedicated research org, specialized A/B testing software, a user base big enough to power a test, and a week to let it run.

But most designers don’t need to understand laboratory experiments. They need to understand the experimental mindset, what design leaders call strategic curiosity, and the willingness to try something.

If you’ve ever worked at a startup, you’ve seen this up close. Startups don’t have all the data. They don’t have clean lab conditions, reliable metrics, or much certainty at all. Everyone’s wearing three hats, so the experimental mindset is usually on full display.

It often looks like a stripped-down version of the Lean Startup loop: build, measure, learn. Say I kept watching users trip over the same spot in the prototype, and I wondered whether cutting that section would fix it.

Build: work with an engineer, or vibe code, a rough but functional version in an afternoon.

Measure: push it live and watch whether anything actually changes.

Learn: keep it, kill it, or try the next version.

There aren’t three layers of approval. There are no laboratory conditions or scientific rigor. But that move, taking an observation and turning it into a small testable bet, is one of the things designers at big organizations can learn the most from.

There are two reasons this matters more now than it did a few years ago.

The first is hiring. AI experimentation has become something managers screen for, especially for Senior+ design roles. You might not have shipped a real AI project, but they want to see that you’re at least curious about working with AI.

Whether it’s pulling meeting summaries out of Copilot and doing something useful with them, or spinning up a rough PR with generated code instead of waiting weeks on engineering. The specific experiment matters less than the fact that you ran one.

The second is defending your judgment once you’re inside. When a PM can generate a passable mockup in thirty seconds, the question you’ll get asked is why yours is better. That’s where the slower work of running a real test earns its keep.

If your answer goes past “users seemed frustrated” and into what you set out to measure and what the result would mean for the business, you’re not defending taste anymore.

You’re showing your work. That’s experimentation, and it comes straight out of strategic curiosity.

One design leader I interviewed put it bluntly.

“You need curiosity if you still want to call yourself a designer. If you don’t have that curiosity to explore? Well, you’re better served being in another field.”

Head of Design, Security SaaS

Now that building the screens is the easy part, strategic curiosity is one of the few things left that can’t be commoditized. But curiosity on its own is a trait, not a method, and a trait won’t tell you how to develop it. That’s the job the experimental mindset does.

So here’s what “calling it a bet” actually looks like.

A good bet answers three questions, and none of them require a lab.