How to make superior product decisions using mixed methods research

• 92% of product decision makers say data and analytics are critical to business success.
• Mixed methods research is essential when risk is high and product clarity is low.
• Maria Kamynina says mixed methods research can save you time in the long run.

In a modern startup, there are research opportunities everywhere you look. Product analytics, customer success conversations, customer reviews, heat maps, rage clicks. And that’s before you even get to things like surveys and interviews with users and prospects.

That’s a lot of data — which is great! — because 92% of product decision makers say data and analytics are critical to the success of their business. And yet, most startups only spend 20% of their time on product discovery. In many cases, teams will pick one of those data sources, take a few insights, and dive head first into feature development.

To understand the problem with this approach and how mixed methods research is a better way to go, I caught up with ex-Zalando and -Bloom user insights lead, Maria Kamynina. You can watch our full session here.

“I remember this tale of the elephant and the blind,” says Kamynina. “People look at different parts of the elephant and one thinks the trunk is a snake, another thinks that the body is a wall. With mixed methods research, we aim to see those different parts of the elephant so that we can compare them and build a more holistic picture that gets closer to the truth.”

This 1986 ad from The Guardian newspaper sums it up perfectly.

When to use mixed methods research

Now, just because you have lots of data sources available, it doesn’t mean you have to use them all for every product decision.

Mixed methods isn’t about cranking up research time and procrastinating until the cows come home. It’s about giving you reliable data so you can make superior product decisions with confidence.

Let’s look at two scenarios where mixed methods research is essential.

When risk is high and product clarity is low

“I really love this framework by one of the research leaders in the field, Jeanette Fuccella,” says Kamynina.

Fuccella maps out a simple matrix with risk running along the X-axis and problem clarity on the Y-axis. This gives one of four outcomes:

1. Ship it and measure — If your risk is low and problem clarity is high, you’re safe to ship your new feature or improvement and track the results.
2. Design heavy — If risk is high but problem clarity is also high, you should allow plenty of time for design exploration and iterations.
3. Research light — If risk and problem clarity are both low, you should spend a little bit of time on research to avoid totally stabbing in the dark with your solutions.
4. Research heavy — If risk is high and problem clarity is low, you need to go big on research to increase problem clarity before you ship anything.

“The more we move on the risk axis,” says Kamynina, “the better it is to do research and the more research you should do. If your decisions involve high cost, then it’s better to spend another week doing interviews or validating your interview data with other methods than building your own thing.”

When existing data is limited

Another good time to lean on mixed methods research is when your existing data sources are limited. This is particularly useful at early-stage businesses without legacy data — or even startups trying to find product-market fit.

“You need to start with a hypothesis of who your users are,” says Kamynina. “Your hypothesis around your target users needs to be specific enough so you know where to find them. Then, when you start building, you’re either like, ‘Oh, this is my user,’ or you might be like, ‘No, this person doesn’t seem to have a need for my product’”.

Research can help you in two key ways here:

1. Discover what you could build that would meet the needs of your target users
2. Explore other user segments that would benefit from what you’ve already built

By applying mixed methods research in the early stages of your business, you can steer your product in a more profitable direction based on genuine user needs.

Two surprising benefits of mixed methods research

Some of the benefits of mixed methods research are obvious. You have more data to work from, you can make superior product decisions with confidence, and you can build better and more profitable products.

But if you’re still on the fence about whether mixed methods research is for you, these two surprising benefits should encourage you to give it a try.

1. You save time on research (yes, really!)

It may seem counter-intuitive, but mixed methods research can save you a ton of time.

Here’s how Kamynina explains it: “Let’s say you want to understand a big question like ‘Why are people leaving the app?’ If you just jump into interviews, it will take you a long, long time to come up with the answer. But if you spend your due diligence on data analytics and really investigate the problem, it will narrow down your qualitative questions so you spend less time and effort doing that.”

All of a sudden, you’re able to cut the time of your research calls from one hour to 20 minutes each. This means you’ll have less data to pick through at the end of the process, because everything is more focused. But it may also make participants more willing to talk in the first place, giving you more and better data to work from.

2. Your user interviews are way more insightful

Imagine that you’re trying to hire a new product manager. You’re about to have an interview with a candidate, but you haven’t seen their CV or cover letter. You didn’t ask any screening questions during the application process, and you haven’t even had a chance to stalk them on LinkedIn!

So you have to go into the interview totally uninformed. Would you make it work? Sure. But would you get as much out of it as you could have if you’d seen all that stuff up front? Certainly not.

The same applies to user research. You could just go into an interview with a user and ask all your questions there. But if you run a short survey first, you can refer back to a participant’s answers and dig into things in more detail.

Final thoughts

By combining different research methods and data sources, you can make more informed decisions about the direction of your product. This is especially useful in situations where risk is high and product clarity is low, or at young companies with little-to-no existing data.