Does Participatory Design "Work"?

We live in interesting times. It’s possible to tell AI to make all kinds of things and have them even work - but it seems like the question of “should we” make the things often gets lost in the buzz of excitement. Products and software DO NOT MATTER unless people actually value them. Sure, AI can use some tools but AI, at least for now, does not determine what matters to humans.

Jan Schmiedgen and team at the University of Potsdam looked at over 400 projects and discovered something you’ve probably experienced: most people see participatory design having impact, but only a small minority actually measure it. It’s not laziness. Most organizations genuinely don’t know how, or don’t have the resources.

The problem is structural. Most organizations’ measurement systems were built for predictable, linear work: invest X, get Y output. But participatory design doesn’t work that way. It creates what Jeanne Liedtka calls “enabling mechanisms,” the conditions that make better outcomes possible rather than directly producing measurable results.

People think of measurement like following a recipe. Participatory design is more like learning to cook. An outcome is possible to measure and compare when everyone follows the same recipe (though as we are well aware from the Great British Bakeoff, having a recipe is no guarantee of success- timing, context, skills, and whether the tent is too hot all can create a range of outcomes). On the other hand, evaluating whether someone has learned to cook is way more complicated.

Both Liedtka and the Potsdam team agree: Stop looking for one perfect metric. Instead, think about five different types of impact that happen at different timescales and organizational levels.

Liedtka studied 22 projects and found that when you combine deep user research with diverse teams, specific things change: teams develop shared understanding across their differences, they get better at reframing problems, and they improve at testing assumptions. These are the immediate impacts you can observe.

What to measure:

• How long teams spend understanding the problem before jumping to solutions

• How many times teams reframe their initial problem definition

• Diversity metrics for project teams (not just demographics, things like functional backgrounds, organizational roles, time at the organization)

• Number of solution options generated before converging

Schmiedgen’s research shows practitioners track “design thinking activities: projects launched, people trained, coaches developed. Looking at this across your organization can tell you whether participatory practices are spreading or staying isolated in a single team.

This is the most measurable category, which is why organizations gravitate here. Remember, though: “quality” means quality from the perspective of the person using the product, not the designer’s.

What to measure:

• Customer satisfaction scores

• Usability testing results

• Specific feedback on campaigns or features

• Time to achieve user acceptance or adoption

• Referrals

Participatory design doesn’t just produce good solutions, it changes how safe it feels to try things, fail small, and learn fast. This is incredibly valuable but nearly impossible to capture using traditional measurement.

What to track:

• Ratio of experiments to full implementations (higher is often better to avoid heavy investment in the wrong thing)

• Average cost per experiment (lower suggests more willingness to test)

• Speed of pivoting when something doesn’t work

• Qualitative team assessments: “Do you feel safe proposing wild ideas?”

• Concepts finished

• Projects launched

• Number of ideas in development

This is sophisticated and hard to measure, but potentially the most valuable. Liedtka’s cases show participatory design building what researchers call “dynamic capabilities.” They include the ability to sense opportunities, act on them quickly, and redistribute resources as needed.

What might indicate adaptability:

• Time required to respond to major market shifts or big changes in circumstance for the people you serve

• Variety in your solution portfolio, exploring multiple approaches

• Breadth of stakeholder networks the team can activate

• How often teams successfully reframe problems mid-project

Participatory design’s lasting impact often shows up in relationships and capabilities that outlive specific projects. Diverse teams bond and keep working together, rather than diverge. People learn to solve problems collaboratively.

How to assess this:

• Network analysis: who’s connecting with whom before and after projects?

• Instances of cross-functional collaboration outside formal projects

• Evidence that teams independently use participatory methods on new work

Mature organizations eventually move beyond traditional metrics to story-based impact assessment.

Since participatory practices constantly reframe challenges, metrics themselves need constant adjustment. A team looking at educational software might have started with a goal like “improve the registration process” and ended up working on “how to help students feel confident in their career decisions,” which requires completely different measures.

Narrative approaches capture what numbers can’t: why a project succeeded, how stakeholder engagement changed outcomes, what enabled teams to navigate uncertainty. These stories provide context that makes the numbers meaningful.

When to use stories instead of (or alongside) metrics:

• The objective got fundamentally reframed during the project

• The most valuable outcome wasn’t what you initially planned

• Cross-team collaboration or relationship building was critical to success

• Leadership needs to understand how value was created, not just that it was

One interesting note from the research was how much more prevalent so-called Design Thinking approaches are in for-profit business than in nonprofit or government contexts. While it’s great when anyone making things actually does so in collaboration with their audience or clients, we should note that there are ways participatory design can end up being dangerous.

When, for example, we’re trying to understand people to exploit their weaknesses in order to further our business goals, it’s pretty gross. We’re in a bit of a UX-driven hellhole at the moment in our world, where things are very easy and convenient in service of making money for giant corporations with very limited power for individuals regarding needs that don’t lend themselves to purchase (though it feels like companies are trying to find a way to capture almost everything, even community, love, health, and creativity) . You can buy things that match your preferences to an almost disturbing degree, but the kind of intangibles that make life actually meaningful, deep connection with other people, opportunities to develop skills and contribute, and personal opportunities to innovate are all discouraged in a world where most of your agency orients around what you can buy.

It’s not usually that UX and Product people go out to find ways to prey on us, but we do have the option to say no when it becomes clear that’s where things are headed!

Check out the story-based interviewing guide for more hands-on methods of developing a participatory design practice.

and now…

Mercy Corps Senior Project Manager, Marketing

U.S. Green Building Council Digital Product Owner

Mozilla (Firefox) Director of Product Management, Generative AI

Ghost Product Designer

DuckDuckGo Director of Product Management

Omidyar Network Head of Communications

National Domestic Workers Alliance Senior Director of Product

ProPublica Director, Product Engineering

Cinder Founding Marketer

Transform Finance Values-Led Business Project Lead