Reading · Products Users Love

I’ve sat in on hundreds of user interviews over the years, run by trained researchers, by product managers wearing a researcher hat for the afternoon, by whole teams crowded into a single call. What I find really interesting is the half second before the participant’s answer, when I can already tell whether the question just decided what the answer is going to be.

We like to think of user research as a way of getting at the truth. Someone uses our product, we ask them about it, and what comes back is research data. But a lot of what comes back isn’t the user’s experience so much as a version of it, shaped by how and what we asked, who was in the room, and what we needed to hear that week. All of this happens because asking good questions under time pressure, in front of stakeholders, while having a million thoughts about the ways we could solve the problem for the user is genuinely hard, and there’s decades of research on exactly why.

In this piece I want to go through four of the most common ways this happens in practice: leading questions, future based questions, participants designing the solution instead of describing the problem, and the pressure of simply being watched. For each one I’ll cover what the research says, what it tends to look like in a real interview, and what’s actually helped in practice.

✨ In this post I’ll cover ✨:

Leading questions, and the words that do the deciding for us
Why "would you use this?" tells us almost nothing
Why users design buttons instead of describing problems
The pressure of being watched: social desirability and demand characteristics

Leading questions, and the words that do the deciding for us

A leading question is one that contains its own answer. “How easy was this to use?” already assumes ease. “Wasn’t that frustrating?” already assumes frustration. Most researchers know not to ask these in their obvious form. What’s harder to catch is how often leading language sneaks in through word choice alone, with no change to the structure of the question at all.

The clearest demonstration of this comes from a well known research study, where authors showed participants films of car accidents, then asked how fast the cars were going when they hit each other, except the verb changed across groups: smashed, collided, bumped, hit, or contacted. The verb alone shifted people’s speed estimates by close to 9mph between the most and least intense framing. A week later, the people who’d heard “smashed” were also more likely to report having seen broken glass that was never in the film.

That second part is the detail I find most useful for product teams. The leading word didn’t just nudge an opinion, it appears to have reshaped what people believed they’d actually witnessed. While this is not a UX finding, it’s a strong illustration on the fact that language doesn’t just describe what happens in someone’s mind, it can actually edit it.

In an interview room, this is the moment you’ve run out of good questions, the participant has gone quiet, and you fill the silence with something like “so that part felt pretty smooth, right?” With the best intention, but it’s what happens when you need to keep the conversation moving and nothing better comes to mind in the moment.

What’s actually helped me here is less about discipline and more about preparation. I keep a list of neutral probing questions visible during every interview, things like “tell me more about that,” “what were you expecting to happen there,” “walk me through what you were thinking.” When you run dry mid interview, which happens to everyone, you reach for what’s already in front of you instead of improvising something that quietly supplies the answer. It’s a small thing, but having the resource at hand removes the moment of pressure where leading questions tend to slip in.

Why “would you use this?” tells us almost nothing

The second pattern is asking people to predict their own future behaviour. Questions like: Would you use this feature? How would this fit into your routine? Would this be valuable to you? These questions feel reasonable to ask, since we genuinely want to know if something will land. The problem is that people are measurably bad at answering them, even if they intend to be as helpful and honest as possible.

There are two separate bodies of research that are linked to this. The first is the intention-behaviour gap, a well established finding in psychology that stated intentions are a weak predictor of what people actually do. People aren’t lying when they say they’d use something. They’re answering sincerely about an imagined version of themselves in a future context they can’t fully simulate.

The second, which I find even more directly useful for product questions, comes from economics and marketing under the name hypothetical bias. When researchers compare what people say they’d pay or value for something hypothetically against what they actually pay or value in a real transaction, the hypothetical number is reliably inflated. One meta analysis of 28 studies found people overstated hypothetical value by about 35% on average compared to real value, and other reviews have found gaps several times larger depending on the context. The mechanism seems to be psychological distance: the further a question sits from an actual moment of choosing, the less predictive the answer is.

Put those together and “would you use this feature” sits in about the worst possible spot. It asks people to predict future behaviour they can’t reliably forecast, about a value judgment they’re likely to overstate, with nothing real attached to the decision.

My approach here is less about banning the question outright and more about discipline in how it’s used. I try to default to asking about past and present experience: what do you currently do, how did you handle this last time, walk me through the last time this came up or the last time you used a specific feature. If a future based question does slip into a session, which happens, the fix isn’t to throw the data away, it’s to be careful in synthesis. I wouldn’t treat that answer as evidence for a roadmap decision. At most it’s a soft signal worth noting, not something to build a recommendation on.

Why users design buttons instead of describing problems

The third pattern shows up when a participant stops describing their experience and starts proposing the fix. “You should just add a button that does X.” It’s tempting to write this down as a feature request, especially because it sounds so actionable. The problem is that the suggestion usually tells you more about the tools the person already knows than about the problem they’re actually trying to solve.

This connects to anchoring, the well documented tendency for an initial piece of information to shape everything that follows. When you ask someone what would help, their imagination anchors on the interface and concepts they already know, not on some unconstrained ideal solution. It’s the same idea behind the old, possibly apocryphal line about people asking for faster horses instead of a different way to travel altogether.

The theory behind this is well established in research that actually won a Nobel Prize. In one of the experiments, participants spun a wheel rigged to land on either 10 or 65, then were asked to guess the percentage of African countries in the United Nations. People who saw the wheel land on 10 guessed around 25%. People who saw it land on 65 guessed around 45%. The number had nothing to do with the question, and participants knew it was arbitrary, and it still pulled their estimate toward it. A later study found the same pattern in professional real estate agents, whose property appraisals tracked an arbitrary listing price even when they denied being influenced by it at all, which tells me expertise doesn’t make anyone immune to this. While the effect of anchoring in interviews and solution design was not tested, it is worth understanding this type of bias to be able to avoid it when possible.

What’s worked for me when this happens isn’t shutting the suggestion down. It’s treating it as a door into the actual problem instead of an answer to write down. When someone says “you should add a button for that,” I ask why they’d need it, and how they currently manage the task without it. Both questions usually surface the real friction underneath the suggested fix, which I’ve found is often a different problem than the one the button would solve.

The pressure of being watched: social desirability and demand characteristics

The last pattern is less about the wording of any single question and more about the room itself. There are two effects that come into play here. Demand characteristics, describes the way participants pick up on cues about what the researcher is hoping to hear and start steering their answers to match. Social desirability bias is the separate tendency to give answers that present yourself favourably, regardless of what the researcher seems to want.

In practice these two stack on top of each other constantly. In a user session where there is one participant and five interviewers, the participant isn’t just trying to be honest anymore, they’re trying not to embarrass anyone, not to seem ungrateful, not to be the person whose feedback upsets someone watching. Research on what’s called the bystander effect in interviews, has found that the mere presence of additional people changes what participants are willing to disclose, even when those people stay silent the entire time. Other work has found that more people in the room increases vague or noncommittal answers generally.

Two things have helped here. The first is the same prepared script from earlier, extended to the whole team: agreeing in advance on a shared set of strong, neutral questions means that when someone on the call feels the urge to jump in because the conversation has stalled, they have something better than an improvised leading question to reach for. The second is structural. Where possible, I’ll set up sessions so observers aren’t visible to the participant at all, watching from a separate space or with cameras off (or through research tools that allow observers without showing it to the user), and any question from the team gets relayed through the lead researcher rather than asked directly. Even if it’s a small logistical change, removing the visible audience removes a real source of pressure, and it keeps the number of voices the participant has to manage down to one.

What this adds up to

All these patterns discussed above usually come from the genuine difficulty of staying neutral in real time, under time pressure, in front of people who have a stake in the answer. Knowing the mechanism behind each one doesn’t make it disappear, but it does change what you reach for in the moment it’s about to happen: a prepared question instead of an improvised one, a past experience instead of a hypothetical, a question about the underlying problem instead of a note about the proposed fix, and a room designed to lower the stakes instead of raise them.

🚀 Hi! I’m Andreea, an academic HCI researcher turned UX researcher who helps founders build products users actually love. I conducted hundreds of research studies for tech companies seeking clarity and I started this newsletter to share real examples and stories from my experience, so teams can do better research.

If you found this article helpful, I would be super grateful if you shared it with others.

I also love great discussions and debates, so I’m excited to hear your thoughts in the comments.

The Difference Between What Users Say and What Counts as Data

Comments