Customer-interviews: How to stop collecting stories about stuff that doesn't really matter

I’m too embarrassed to calculate how much time I’ve spent in discovery interviews collecting beautifully detailed stories about problems that, it turns out, don’t really matter.

If there’s one thing I learned over the years, it’s that I want to build products that address problems (or wants, or needs - but usually problems) that are Hair-on-Fire (interviewee is aware of the problem and “begging for” a solution) or latent-yet-important (interviewee is not aware of the problem, or accepts it as a hard-fact-of-life, but seeing it solved is delightful).

I’ve been using Teresa Torres’ story-based interview technique for some years. The technique is great at adressing the “perfect self” or “abstraction problem”. By asking interviewees about a concrete story that happened in the past, we can deduct conclusions about their real behavior (allbeit: in the past, not the future).

The technique works, but I bumped into the same problem over and over. I spent countless hours listening to fascinating, vivid stories, only finding out at the very end of the interview “oh, this actually only happened once in the last five years”; or “No, we already switched to (software X) and now it’s totally solved”.

My fix is small. I bracket every story with two importance signals: one before, one after. It makes it far more likely for me to walk away with an unsolved Hair-On-Fire problem.

(For uncovering latent needs, I use Tony Ulwick’s job mapping exercise & ethnography)

The standard story-based prompt is some version of “tell me about the last time you did X.” As outlined above, the stories are interesting, but they might revolve around a pain point, want, or need that isn’t all that important (or worse: that’s already been addressed).

So I anchor on importance first. Before I ask for any story, I make the interviewee tell me what’s painful or wished-for in the area I care about. Two prompts I rotate through:

“What’s the most difficult or painful thing about [phase of the job to be done]?”

“If you had a magic wand and could wish for anything when it comes to [phase], what would you wish for?”

Then I ask for the story:

“Please tell me about the last time you encountered that. What happened step by step, who else was involved, and what was difficult, problematic, or could have been better?”

Concrete example. My current ICP is job seekers looking for jobs at AI-forward tech companies. The job to be done is “find a fitting job.” The phase I’m researching is “preparing for interviews.” A session opens like this:

> Me: What’s the most painful thing about preparing for interviews?

> Them: [pain X].

> Me: Tell me about the last time you ran into pain X. Walk me through what happened.

Now the story I’m about to hear is anchored. I know it’s about something they themselves just told me was painful. I’m much less likely to spend the next 15 minutes nodding along to a one-off annoyance.

When opportunities (pain points, wants, needs) come up during the story, I do two things:

1. Check it’s still unsolved. People loooove talking about problems they used to have. The story is satisfying because it’s resolved. But if they’ve already worked around it - bought a tool, changed a process, made peace with it - it’s not a real opportunity. So I ask: “Is this still a problem for you, or have you already found a way around it?”

2. Score it 1 to 5. “On a scale of 1 to 5, how painful or urgent is this?” I write the number next to the opportunity in my notes.

The 1-to-5 score sounds crude. It is crude. That’s the point - it forces the interviewee to commit to a number, which is much harder to do at a 4 or 5 than it is to nod along when I empathise with their pain. The 2s and 3s become very obvious very fast.

I want hair-on-fire problems. The score tells me which of the things I just heard about actually qualify.

1. 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝗽𝗿𝗼𝗳𝗶𝗹𝗶𝗻𝗴 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀 that will allow you to segment interviewees later (first 5 minutes). These should allow you to “fill in” the attributes that you think might be important to describe your ICP (e.g. role; years of experience; tech stack; maturity of company they work at; etc. )

2. 𝗟𝗲𝘁 𝘀𝗶𝗹𝗲𝗻𝗰𝗲𝘀 𝗳𝗮𝗹𝗹.

   I have a natural tendency to fill silences, or ask follow up questions too quickly. Sometimes it’s best to let silences fall - most interviewees will start to fill them, thereby going just a bit deeper below the surface.

3. 𝗣𝗮𝗿𝗮𝗽𝗵𝗿𝗮𝘀𝗲 𝗶𝗻𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗹𝘆

   When you feel like your interviewee isn’t going beneath the surface, you can try paraphrasing them slightly incorrectly. If you paraphrase them correctly, they’ll say “yes”, and you’ll have captured your own words. If you get it wrong, they will need to correct you: “No, it’s (their own words)”.

Here’s an example interview guide for the MyNewTechJob case study (I dove into this case study here and here):

45-minute story-based interview. Surfaces a single concrete pain, want, or need from the job-search journey, then goes deep on the last time it happened.

1. Profiling questions

Short, factual. Set context - don’t go deep. Capture answers into the “Profiling information” section of the transcript file.

- Current role?

- Years of experience in current role (and total)?

- When you hear “AI-native” talent, does that mean more about how you _use_ AI day to day, or about people who _build_ AI products?

- Desired company maturity? (startup / scale-up / Mittelstand / larger)

- Desired company industry?

- Primary interest in a role? (e.g. meaningful work, flexibility, autonomy, learning, peers, compensation, stability)

- Remote, on-site, or hybrid?

2. Opener - the painful step

> I want to talk about looking for a new job, which I understand is something you are currently doing, or have recently done. This can include a lot of steps - like understanding what kind of work you want to do and at which company, finding roles you’re excited about, applying, preparing and doing interviews, negotiating offers, and so on. In this entire flow, can you tell me what was the most difficult or painful step, and what was difficult or painful about it?

**They should highlight something painful before we move on.** If they deflect (”everything’s been fine”), probe gently for the _most_ painful step even if relatively small. If they surface multiple, pick the one they sound most emotionally engaged with and park the others.

3. The story

> Now think back about the last time you ran into that pain point. Talk me through what happened step by step. Who else was involved, what exactly happened, and what was difficult, painful, or what could have been better?

Spend most of the 30 minutes here. Probe for specifics:

- What happened first? Then what?

- Who else was involved?

- What did you try?

- What workaround did you land on, if any?

- How did you feel during it?

Push for concrete details over generalities. “What happened last week” is worth more than “what usually happens.” If they drift into abstractions, bring them back to the specific instance.

4. The intensity & “already been solved” check for every opportunity

For every pain point, want, or desire that is mentioned, ask:

- Does this still persist, or has it been solved?

- If it still persists, how painful/important is it on a scale of 1 to 5?

5. Wrap (if time)

- Anything in the job search I didn’t ask about that you think is important?

Don’t pitch or test solutions. Stay on pain. The goal is understanding, not validation.