5 signs your user interviews are giving you bad data

Interviewing users is one of the most frequently used qualitative user research methods by UX researchers. At UX studio, we conduct interviews for nearly all of our client projects. Based on our experience, we’ve compiled this checklist for validating high-quality data formulated after your interviews. Let’s see the list!

1. When interviewees poorly match the target audience  

Everything starts with recruitment. Before recruiting, capture the team's impressions of a typical user and create  an assumptive user persona, then screen participants against it to make sure you’re talking to people close to that profile.

In your screener, ask questions that reveal whether a candidate truly belongs to your target segment: their context of use, prior experience with the product category, their habits and demographic characteristics. This filters out participants who might give you enthusiastic but ultimately irrelevant answers, which is one of the most common sources of low-quality interview data.

To do this well, translate your assumptive persona's key factors into screener questions:

1. Ask about prior knowledge and already used solutions: are they a complete novice or an experienced user? 
2. Ask about context: when and where would they use a product like this? 
3. Ask about their usual behaviour: are they actively experiencing the issue your product solves? 

Anyone whose answers don’t align with your persona's profile should be filtered out before the interview stage.

The goal is to make sure that by the time you sit down with a participant, you already know they represent a real segment, not just someone who was available and willing. This way we can guarantee that the selected participants are worth the time and effort to talk to. Later on during the actual interview we are focusing on the rest of the important parts of the persona, like the motivations and pain points and larger context.

2. When your interviews feel like an echo chamber  

One of the most common ways to get unreliable interview data is by leading participants with your questions, often without realizing it.  

Start with the right mindset and attitude. A good facilitator is open, curious, empathetic, but above all, objective. This is one of the natural advantages UX researchers have: they're not building or selling the product, so their only goal is to understand whether it genuinely serves users’ needs.  A good interview should be like a conversation. The facilitator asks the questions and listens, letting the participants take their time answering. They never judge what people say.

Good interview questions are simple and open-ended, giving participants space to openly share their opinions and feelings.  Avoid questions that are too broad, and instead probe specific details from multiple angles.  

3. When you know what users do, but not why  

One thing that sets a skilled UX researcher apart is the ability to go below the surface and uncover the root causes and motivations behind user behavior. Without that depth, you risk designing solutions for the wrong problem.   

Context really matters. It’s not enough to know that a user abandoned a checkout flow or switched to a competitor. You need to understand what led them there, what they were thinking, what they were trying to achieve, and what got in the way. Context and timing both shape behavior, and none of that shows up in surface-level answers. 

So, it’s best to start with a broad, open-ended prompt that invites storytelling rather than yes/no answers:

"Tell me about the last time you bought something online."

Then follow up with targeted probes that push past the what and into the why:

• "What was challenging about that process? Why did that feel difficult?"
• "How much time did you spend on it? Why did it take that long?"
• "What would have made it easier? Why does that matter to you?"

The goal is to keep asking why: not in a blunt, repetitive way, but by following each answer with genuine curiosity. A useful rule of thumb is to go at least three levels deep before moving on. 

4. When patterns aren't emerging across your sessions  

A golden rule in interviewing is to keep having sessions until you feel like you can predict the interviewee's answer. 

In numbers:

• For a discovery phase, exploratory interviews around 10 should be enough.
• In later phases of the product development or with a mature product, even 5 interviews should be sufficient enough to get valuable insights. 
• This means that after 5-10 sessions, you see clear patterns and should be able to comfortably  predict what interviewees are going to say.  

If you can’t, it’s a sign that you’re either not talking to the right people, or not asking the right questions. 

Preparation is what makes a small sample size work. Before any structured interview, UX researchers write a script in advance: a defined set of questions asked consistently across all participants. This doesn’t mean the conversation is rigid. The order can shift naturally depending on how the session flows, but every important topic gets covered with every participant.

This consistency is what makes it possible to notice the common motivations and pain points. Without it, you're not really running the same study across sessions but having a series of loosely related conversations, which makes it much harder to spot patterns or draw conclusions with confidence.

5. When you can't find quotes to support your insights  

After facilitating the interview sessions and grouping the findings to create insights, UX researchers are creating reports to  present them to stakeholders. It’s really important that these reports contain tangible, actionable statements and recommendations, because we want those findings to shape the product development’s direction. We want the discovered users’ needs, motivations and paintpoints to be met in the product.

One way to catch attention, get everyone  on the same page and convey user needs in these reports is to include actual, specific quotes from the users.

Quotes do several things at once. They make abstract insights tangible, they give stakeholders a direct window into user experience, and they add credibility to your recommendations. A finding like "a seamless checkout process is extremely important for the users" lands very differently when it's followed by:

"The last time I was online shopping I got so frustrated by the fact that I needed to type in my address after I’ve already given it as a billing address that I did not buy anything in the end."

If you reach the synthesis stage and can't find strong quotes to back up your insights, the problem usually traces back to the interview itself. Closed or leading questions produce short, shallow answers, and short, shallow answers don’t make compelling quotes. Open-ended questions that invite storytelling and emotional honesty are what generate the kind of responses worth highlighting in a report.

How to set yourself up for quotable moments:

• Ask questions that prompt participants to describe real past experiences, (not hypotheticals)
• Give participants space and silence: some of the best quotes come after a pause
• Take verbatim notes or record sessions so nothing gets paraphrased away
• Flag strong quotes in the moment, so synthesis is faster and nothing gets lost

The goal is to walk away from every session with at least a handful of specific, vivid statements that could stand alone in a report and make a stakeholder immediately understand why something needs to change.

Conclusion

Interviewing might seem straightforward:  you're just talking to people, asking questions. But as this post shows, getting genuinely useful data from user interviews takes careful preparation, practiced skill, and no small amount of experience. Recruitment, script writing, facilitation, and reporting are all equally important, and a weakness in any one of them can compromise the whole.

That’s exactly why UX researchers exist. Each step in this process is something they’ve refined across dozens of projects and hundreds of sessions.

We hope this post helps you approach your next round of user interviews with more confidence  and sharper eyes for the signs that something might be going wrong.

If you'd like more hands-on support, or need an experienced researcher to take the lead on your next project, reach out to us. We'd love to help.

_{Credits
This blog post was written by Fruzsina Fejes, UX researcher
Editing by Dr. Johanna Székelyhidi, marketing manager and copywriter}