Feedback Collection: Don’t Over-rely on it, Product Managers

How to Use Customer Feedback (and How Not to Use it)

In response to customer complaints that it was becoming confusing to figure out whether they were viewing the DVD-rental or on-demand portion of the Netflix site, the company’s CEO decided to split these two services into separate businesses — and charge for each individually.

The result was an updated Netflix subscription that offered streaming content only, and a new standalone DVD-rental service called Qwikster. And it all started because some customers complained, and an executive wanted to respond to that feedback.

But with streaming subscribers suddenly no longer able to order DVDs by mail, and the fact that paying for both services now represented a 60% price hike, more than 8 million Netflix customers canceled their subscriptions within just a few months. And the company’s stock price fell by 77%. Netflix’s CEO quickly apologized and recombined the DVD and streaming services for subscribers. It was a lesson that cost Netflix millions of dollars. 

In this post, we’ll discuss both the importance of feedback collection for building successful products — and why product teams should be careful not to become overly dependent on this data.

First, though, let’s define our terms.

What is customer feedback?

Customer feedback has a much broader meaning for a Product Manager than it has for consumers. When we buy a product or sign up for a service, we think of feedback as posting a review online or calling the customer service number to ask (or beg, or scream) for help.

But for a Product Manager, feedback can and should include a wider range of inputs than unsolicited comments from your product’s users.

Common feedback collection methods

Your product team’s sources for customer feedback collection might include:

• Surveys
• Interviews with representatives of your user or buyer personas
• Product analytics (meaning your product’s actual usage data)
• Focus groups trying and reacting to your product’s prototype or MVP
• Customer advisory boards
• Stories from your customer success team about interactions they’ve had with users
• Customer reviews online (for own product and those of your competitors)

Why Is Feedback Important?

When your product is in its pre-launch development phase, your company’s goals, plans, and expectations for the product remain entirely theoretical.

It’s only after your product hits the market that you can actually test those theories. Only when real users get the chance to buy and use the product — or choose not to buy it — will you gain the important insights that inform your product team’s next moves. Those insights, for example, can include the fact that:

• Users find your product too complicated or frustrating.
• Buyers perceive the product as too expensive.
• The tools or features you assumed would get those most usage… don’t. Customers spend their time on functionality your team viewed as mere nice-to-haves in the product.
• Users love everything about your product and don’t have any complaints or requests. (Good luck with this one. )
• Customers have created different user experiences or pathways to completing tasks with your product than the ones you anticipated they would.
• Customers have found use cases or are able to complete tasks your team hadn’t even intended — and those customers are thrilled with your product because of it.

Why should you gather customer feedback?

Feedback collection is important because it helps Product Managers add important data to their research into what’s working with their products and what isn’t working. That’s one reason to include it in your product management prioritization mix.

Another reason to develop an intelligent feedback collection and analysis process is that it can give you a competitive advantage. When craft.io surveyed product professionals all over the world for our 2023 State of Product Management report, we found that more than 1 in 4 product teams — 27% — have no formal method of gathering customer feedback. These companies are, in effect, building and updating their products in a vacuum.

And as you can see from the graph taken from our report, 42% of those product teams that do gather customer feedback don’t have a reliable process to centralize, digest, and analyze it. Those teams are missing out on the benefits of their users’ insights as well.

So, those are some of the reasons you should collect, centralize, and review customer feedback as part of your ongoing product research.

But that doesn’t mean feedback should drive your product decisions or priorities. Here’s an inspiring product management quote on the topic from Steve Jobs:

 “Some people say, ‘Give the customers what they want.’ But that’s not my approach. Our job is to figure out what they’re going to want before they do.”
– Steve Jobs

Jobs was spot on, of course. Imagine how much the world would have missed if the Apple product team had waited for the market to tell them that what mobile phones needed were high-bandwidth web browsing capability, video players, and a way to find the nearest Starbucks.

The Risks of Over-relying on User Feedback

Did the Netflix customers who complained about the confusing website navigation want a more intuitive experience when looking for shows and movies either to stream or to rent by mail? Yes, of course. But how many more customers did those who complained actually represent?

And to what degree were those unhappy with the current website willing to have their Netflix service changed to make navigating the site easier? Were they willing to lose the DVD or on-demand components of their service altogether?  

These questions bring us to some of the common pitfalls Product Managers face when they rely too much on what their customers tell them. Here are a few reasons you should weigh your user feedback against other important factors — even your own intuition.

1. It inaccurately weighs the loudest people.

According to research published in ScienceDirect, only about 44% of people respond to online surveys. If you’re trying to learn how users feel about your product by sending questionnaires, you need to remember that whatever answers you glean from the users who do respond, this method will always represent what fewer than half of your users actually think.

And that illustrates one of the challenges of collecting and studying user feedback. It can give you a skewed sense of what customers think about your product because: 

• Only a fraction of your users will respond to surveys, agree to interviews, or join your focus group or Customer Advisory Board.

• The people who take the time to post reviews or submit comments to your customer success team might simply be the types of people who believe they can affect a product by submitting feedback.

• Some people just want to complain.

As you can see, these self-selecting categories of people won’t necessarily represent a larger sentiment about your products. Their feedback can give you new insights, but you’ll always want to weigh this feedback against other factors.

2. Customers and prospects often can’t tell you what they want.

As the Steve Jobs quote above makes clear, people rarely have the vision to tell companies how to build the solutions that will make their jobs or lives better. That’s why Product Managers like you are so valuable.

One of Coca-Cola’s rare missteps was reacting to surveys that indicated customers wanted a sweeter version of the soda. That led to New Coke, which proved to be one of the biggest failures in consumer product history.

Your role as a Product Manager is to take many inputs — the details you’ve learned about your user persona’s goals and challenges, market research, and yes, customer feedback — and use those insights to develop an innovative solution that will move customers to buy. But you don’t want to simply ask potential users what they want and then build a product based on their response. Your customers rarely know exactly what they want until they see it.

3. Your users’ goals aren’t always your company’s goals.

Finally, if you act on every piece of user feedback, you will almost certainly be working in many cases against your business objectives.

Imagine this: Your customers tell you they want an overhauled, more modern user experience in your software product — but your goal is to increase new users this year.

If you devote development resources to completely rebuilding your product’s interface, you’ll have fewer resources to add the new functionality your research indicates could lure customers in a new market segment.

Again, our overarching advice here is to welcome, compile, and analyze user feedback — but to remember that it’s only one of several important sources of data that should inform your product strategy.

The Smart Way to Collect Feedback and Apply it to Your Product Strategy

So, if you’re one of the 27% of product organizations with no feedback collection process, or you’re among the 42% with a process that’s messy and unreliable, what should you do?

First step: Find an end-to-end product management platform that includes built-in product feedback collection and analysis.

 

Such a solution, like the one from craft.io, lets you easily pull in any existing feedback from your other apps or data sources, connect feedback to specific items or features, and weigh competing types of feedback in terms of how they support your business objectives.

Without a centralized, intuitive feedback portal connected to your roadmap and strategy, you’ll be leaving a big hole in the road to your product’s market success.

 

Try the craft.io feedback portal free