Qualitative Research — You Asked, We Answered
This month, we presented the webinar, Fueling Your Product Strategy with Qualitative Research. You can view the accompanying blog post as a primer to learn about how you can better extract insights from your research. We received so many great questions on the topic and have included answers to the top questions down below. Thank you all for engaging in our webinar!
How do you capture data when you’re not conducting interviews, instead you’re doing internet research?
The methods we spoke about in the webinar can still apply when you’re doing secondary research (internet search) instead of primary research (interviews). You want to make sure you have strong documentation practices so that you can pull out codes, themes, and insights, and be able to connect them to sources. When doing secondary research, you will not have the same degree of insights because secondary research is already filtered by another source. You won’t necessarily be able to get a good idea of customer emotions and frustrations, and your perspective may be biased by the keywords you are searching with.
Where does our value of the customer come into play in this process? (i.e., Customers that pay more will receive more value versus one that does not, even if they are asking for the same thing)
As mentioned during the webinar, customer value can be a weighted code during your research. Customers who experience greater frustration may also be the ones who are more likely to pay for your product or feature. However, we do not recommend qualitative interviews to be the only tool or input you use to decide pricing and customer value. You may want to run quantitative analyses with your market research team or counterparts to understand “willingness to pay” data.
Best ways to find people to interview for research? Offer incentives? Can you provide any guidance or best practices for engaging or establishing a user group if your organization doesn’t have a standing user group/forum for your product?
There are plenty of ways to find user groups – both paid and free. There are social media groups that may represent your user base. You may be able to find them in your own peer networks as well. If you’re having trouble finding enough participants, don’t forget to ask users for recommendations. There are also many paid services that can give you access to users at a fairly low cost.
Is there a recommended number of users you should have in a user group (i.e., 5% of users)?
We do not have a recommended number of users in a user group. However, we do recommend running more interviews and tests with fewer users or customers in each round of tests. Running more rounds of tests or interviews enables you to find new insights and validate new assumptions you get over time. We recommend having about five users per round or test.
Post research, have you created a cohort of the larger group to help you build (as advisors) and go to market?
It is common for some companies to create advisor councils of their more important customers, whether they are buyers, influencers, or users themselves. There are pros and cons to this. On the pro, a council will give you regular and direct feedback on your product. On the con, council members tend to be from your largest clients and are not necessarily representative of your total addressable market (TAM). You may be building custom features instead of addressing a widespread market need.
I have a question on the point, “Executives do not understand the importance of a formal product process.” The “formal” product process of the startup I work in has the product team developing new product/features. Sales are asked (by the CEO who is the product person as well) to sell it!! The new products and features are NOT developed by doing any research. Also, there is no Product Strategy or Business Strategy! How can a startup start with this qualitative research to create a process and alignment for the end goal of winning deals and increasing revenue?
Let me be clear upfront – qualitative or quantitative research is not going to replace a product process. Also, it is possible that your company does have a strategy and may just be bad at communicating it. I would start by asking your leadership team what they are optimizing for and why, and see if you can get a better understanding of possible un-validated assumptions. You can start your qualitative research by testing those assumptions. We cannot stress enough that qualitative research must have a “lens” or perspective to drive the discussion first. In this case, the lens may be formed by a better understanding of what your CEO is optimizing for. How are they making prioritization decisions and why? If there is really no strategy behind any of this thinking, and your questions don’t help your team get started, it may mean a risky future for this start-up. In the best-case scenario, as the Product Manager, you may be able to influence your executives into building a strategy. In the worst-case scenario, you may decide to leave for other opportunities.
Loved the webinar! What would you suggest as the best way to capture customer feedback on a demonstration of a proof-of-concept (POC) for a new solution (an audience of 30 people)?
If you’re running a proof-of-concept demonstration with a group of 30 people at the same time, you are unlikely to get all of the audience to relate their feedback to you. There are always going to be issues requesting great feedback in such a social setting for many reasons: people are unlikely to go “deep” in public, and in the context of a POC, people have a natural tendency to want to “please” the presenter. You will need to set the stage properly to make it okay for people to speak freely and give you negative feedback. Make sure to call on people to describe their reactions in detail. Ask questions of how they would use the product and the more specific you can get the answers to be, the better. At this stage, since you are past the “problem space” and are in the “solution space” you should be testing to see whether or not the solution you are presenting is solving the problems you had identified before. We also recommend recording this session and focusing in on audience members so that you can see their emotional/facial/and body reactions to your demonstration.
My products are available to the general public and we don’t facilitate site visits. What has been the best way that you have found to capture voice of the customer (VoC) without site visits?
As we discussed on the webinar, site visits are not the only way to get in front of customers. If your product is publicly available, then its customers should also be fairly accessible. Find a few customers willing to talk to you and conduct 1:1 interviews over Zoom.
Are there data mapping templates or a program you recommend?
- There are no templates for coding or theming as they are very unique to your market, product, and question/lens.
- Empathy mapping, Jobs-to-be-Done, and other UX research tools may help you when gathering customer data.
- Capturing data: I use either a voice recorder or video and voice recorder, such as Zoom provides.
- Coding: Use a word processor such as MS Word to first convert all recordings to writing and then use the comments tool to add codes. These are then searchable.
- Theming: For a small project, I recommend a mind mapping software such as Mindjet or SimpleMind. There are also free versions available such as FreeMind. For a large project, you can check out Nvivo or similar types of programs.
- Final Documentation: Basics such as MS Word or PowerPoint work well here. Whichever is standard in your organization.
These questions were answered by Rina Vernovskaya and Colleen O’Rourke.
Watch the Webinar On-Demand
About the Author
CEO of 280 Group
Rina is the CEO of 280 Group. She began her career at McKinsey & Company as an analyst on the credit portfolio team, offering trading recommendations to hedge against credit and interest rate exposure. She then joined MetLife’s Global Leadership Development Program, focusing on employee recognition programs and removing roadblocks to support intrapreneurship in the Latin America Office of Innovation. Rina joined 280 Group with a focus on bringing transformative change to organizations by arming product people with the right tools and skills to do great product management. Her job running the company at 280 Group is made easy with an amazing team aligned on continuous learning and development and determined to go above and beyond to always delight their clients.