Planning Your First Survey

In the previous chapters you learned why user/customer feedback is important and the different types of surveys that you can send out. In this chapter, I will cover exactly how you can plan out your very first user feedback survey. While it might be tempting to go out and blast a few surveys to your list right now, it is important to brainstorm with your team to come up with the best questions to ask in order to achieve the maximum results and get the actual data that your team is seeking.

Have a strong hypothesis

Here’s Hiten Shah’s advice on building a hypothesis for customer feedback questions,

When creating a problem hypothesis start by describing the group of people you are targeting and what problem you think they have. It’s a simple format:

“[Group of people] have a problem [their problem]”

“At KISSmetrics, we built a mobile app for people that use Google Analytics. Here’s our hypothesis:”

Google Analytics users have a problem monitoring key business metrics on their mobile phone. You can take the idea for a new feature or a product iteration and create a hypothesis out of it. When we wanted to improve our real-time view in the KISSmetrics product, we started with a hypothesis: New and existing KISSmetrics customers have a problem debuging their implementation and viewing their users in real-time.

From that, we can learn that it is extremely vital to have a hypothesis in place before sending out any customer feedback survey.

Have a strong goal/end result

The point of a user feedback survey is for you to gain actionable feedback and to gain insights on data that you cannot get from just using analytical tools. Before coming up with a question, you need to know the end goal of the question.

For example, if you are sending out a survey question regarding the best product feature for your company to build next, you should know that your goal is to narrow down a list most requested feature and only work on the features that the majority of your users want. The best approach for this would be to brainstorm the survey questions ahead of time with your team.

Ideally, your survey should be built around accomplishing a single goal. What’s the main thing that your company is trying to figure out now? A laser-focused set of questions will produce higher quality results that are easier to analyze and put into action. Think of the survey questions as an unsolved puzzle set. You want to be able to fill in and complete the puzzle by asking customers the correct questions that will help you plug in the holes.

Do we need to segment our list to survey?

You might have a large customer database or a large email list. It is vital at this point that you and your team figure out whether or not you need to narrow down the list by segmenting them. For example, if you are sending a NPS survey, you might only want to target the users who have been using your product for at least 3 months.

Another reason why you should segment your list for different surveys is because there are a lot of different type of respondents. There are a lot of what we call, “unqualified users”. You can’t really avoid this. As your list grows bigger and your trial sign ups goes higher, you will get a lot of people that sign up for your app, but never use it or never convert. Using user feedback surveys is a great way to segment these people out, but in the beginning you should segment your current list of subscribers and customers for your first survey.

Yet by definition, there’s nothing you can do to convert unqualified visitors because they aren’t looking to buy what you’re selling. So, unless you’re testing out a new market, it doesn’t benefit you to survey someone who doesn’t need, want, or use your service. Not only would their answers offer no valuable data, but obtaining them would be a waste of resources that could be better spent elsewhere.

So this goes back to segmenting your list of qualified users based on the goals that you set. Let’s go back to the example where you want to send a customer feedback survey out in order to figure out the best feature to add to your product roadmap. For this specific case, it would not make any sense that you send out this survey to users who are subscribed to your blog, but not an active customer. A qualified user in that case will be someone that’s already paying for your product and getting value of it. Those feedback are the ones that matter and make an impact.

In that situation, another good segment of users will be people who churned. The ones that stopped using your app or did not come back to purchase again after 60-90 days. Asking them what features or products that they want to see next and then actually offering those products in the near future will delight them and reactivate them.

More things to consider before sending out your initial survey is to actually come up with a long term survey plan. How often will you be sending out these surveys? A mix of open ended questions in combination with NPS and website feedback widget is the best way to get the best out of everything. Tweaking the survey questions and sending different follow up surveys based on the respondent’s response is another great approach. Discuss with your team internally on the best practices.

In summary, what you want to find out and who you survey are inextricable—it’s critical you receive feedback from those who are directly affected by whatever issue you’re trying to solve or question you’re trying to answer. I know you might be asking, what questions should I be asking my customers? Don’t worry about that for now. In the later chapters, I will cover all of those in detail. For now, you can just start planning with your team ahead of time and prepare to blast those surveys.