In my previous post I talked how my mindset for processing user feedback changed after launching a SaaS product as a single founder. In this post I’d like to talk about the process I developed for interacting with users and using feedback to guide product decisions.
It might seem controversial (or not), but I’d claim that user feedback rather than data should be your main source for understanding how to improve your product.
I’ve worked on teams and seen many companies in which analytics data is the main driver for product decisions. The problem with this approach is that analytics frame product issues from the perspective of the developers / business, instead of the perspective of the user / customer.
The Problem with Data Driven Product Decisions
Let me know if the following process sounds familiar to you:
- Key Performance Indicators (KPIs) are determined for the product. Some examples: conversion rates, click-through rates, time on page, feature usage rates, daily active users, retention, churn, etc.
- Data is collected to determine how KPIs change over time
- That data is used to identify and frame the issue that needs resolving. For example: “Not enough people are adding products to their carts. How can we increase that number?”
- A hypothesis is formed on why people don’t add enough products to carts by analyzing the product page.
- The hypothesis is implemented and tested by split testing it versus the previous version.
- The test ends once a winning variation is determined and the cycle starts again.
This is a repeatable, proven process that works, and is used by many software companies as the main process for improving product performance. I’m not saying you should stop using it, however there are a couple of problems with only using data to make product decisions.
Data only gives you a partial understanding of product performance
The first problem with this process is that data only gives you a partial understanding of product performance. It tells you how your product is performing, but not why it is performing that way.
With enough experience and data, user experience designers / product managers / software developers / marketers can come up with an informed guess as to why product performance is the it is and try to address it. But it is still guesswork, and the main reason you’ll find most attempts to improve KPIs using this method end up being inconclusive, or only have a marginal impact.
The second part is that by focusing on KPIs, the framing of the problem revolves around what the company is trying to achieve as opposed to what the customer / user is trying to achieve. Understanding the latter is much more productive for figuring out how to improve the user experience and help users achieve their goals, resulting in a better product.
How To Make The Most Out Of User Feedback
User feedback can come in two ways – solicited and unsolicited, and it’s important to use both for informing product decisions.
Solicited user feedback includes surveys, polls, lifecycle Emails asking for feedback, and more. Unsolicited feedback comes in through your contact form, the website chat or customer service center. I’ll be going over the different ways we collect user feedback at Martial Arts on Rails in future posts.
I’ve found unsolicited feedback to be the most useful in most cases, as it’s both topical and timely. When you solicit feedback, it’s not necessarily at the exact time a user is feeling pain using your product (hopefully it’s not that common), and since you don’t know what they might be having issues with you have to keep your questions more generic (i.e, “what would help you make a purchase decision?”).
On the other hand, unsolicited feedback is usually the result of a user encountering an issue, so it’s both timely and highly specific. This is the best time to understand what the user was actually trying to do, and to improve the product to better answer their needs.
We still solicit feedback, and it’s especially important with new users who might not yet have a good sense of the capabilities of the product. It gets the conversation started, which then facilitates the flow of specific feedback to occur.
Much of our best feedback, however, comes in as either issue reports or questions about how to use the product in a specific way. If you are not using customer service inquiries as an opportunity to improve your user experience, you are missing out on an extremely valuable resource.
This is another topic that I would like to expand on in a future post, as customer service interactions are now our best source for understanding product performance and driving product decisions.
Data Informed Customer Driven Product Decisions
The approach I outlined above is not necessarily a one size fits all solution for every company and product. Some companies are notorious for not interacting directly with users and using data as the main driver for decision making (*cough* Google *cough*), and are doing very well in spite (or because) of that.
I would make the case that if your goal is to make better products, rather than optimize your resources, using user feedback rather than data to drive product decisions would usually result in better products. Data can then be used to confirm product if decisions had a positive effect. In my opinion, this would always be the better process for early stage companies and less mature products.
- Use data to understand how your product is performing
- Use user feedback to drive product decisions to improve performance
- Rinse, repeat
In my next post I’ll focus on our customer service interactions and how they drive our product development cycle.
What is your process for making product decisions and changes? Let me know in the comments below!
To know when the next article is published, please subscribe to new articles using your Email below or follow me on Twitter.