3 Survey Mistakes That Will “Spook” Any Researcher
Sure, we might get a little spooked at haunted houses or when we’re alone in the dark, but nothing keeps us up at night like the thought of getting survey data back and realizing that our own human error ruined what would have been perfectly great insights. The scariest part about bad data is that it not only keeps us from getting the insights we need, it also wastes precious time and resources in the process.
To keep you from getting scared away from research by bad data, we’ve created a whole survey methodology blog series to help our customers and employees stay on the cutting edge of survey building and research best practices. But even after you understand the basics of creating a good survey, you’ll need to make sure that the way you target and distribute your survey also helps you get the best data possible.
Since this is the season for spooking, we’ve gathered a few common survey and research blunders that tend to really freak out researchers. You’ll also find the solutions you need to avoid them.
Not Paying Close Attention to Question Wording
Even the best researchers let bad questions slip through the cracks. You may not realize it, but the way you word or phrase your questions could be leading respondents to answer a certain way.
Here’s an example:
How would you rate the career of legendary outfielder, Joe Dimaggio?
While this may seem like a well-phrased question, the word “legendary” might sway people to rate give him a higher career rating.Here’s a better version of this question:
How would you rate the career of baseball outfielder, Joe Dimaggio?
See the difference? by replacing the work “legendary” with “baseball” you don’t sway your respondents to a certain answer. If you want accurate survey data, keep your language neutral and avoiding leading questions.
Another one of the most common survey blunders happens when survey builders try to ask two questions at once—we call these double-barrelled questions. See, when it comes to survey questions, two for the price of one isn’t always a good deal.
Here’s an example:
How would you rate the newest Star Wars and Star Trek movies?
See the problem here? You force your respondents to give one rating to two different things. Instead of trying to package the questions into one, break them into separate questions so that the resulting data tells you something about each item you’re asking about.
Here’s what the questions should actually look like:
How would you rate the newest Star Trek Movie?
How would you rate the newest Star Wars Movie?
Failing to Target Your Surveys
Have you ever received a survey asking you to rate an experience you have never even had? Or how about a survey that asks about a product you know nothing about? If you’re using a panel provider to distribute your survey, targeting the right people becomes a little easier. But panels aren’t the only way to avoid sending irrelevant surveys to respondents.
By using a platform that allows you to deliver profile-based or behavior-based targeting—whether that tool is a panel management tool or a website experience tool— you can engage particular segments of respondents with questions that are actually relevant to them.
Take a website feedback survey, for example. If you’re an ecommerce company and you want to know why customers abandon their carts, you might trigger a survey to site visitors when they leave your site without buying anything. But unless you’re careful, this is could actually damage your data. You don’t want to intercept site visitors with a survey if they didn’t put anything in their cart—you only want to target those customers who placed items into their digital shopping carts but didn’t complete a transaction.
Believe it or not, lots of researchers or marketers don’t think through the delivery of their survey to realize that they need to target more than individual characteristics. As you’re thinking about delivery, be sure you’re targeting based on customer characteristics and behavior.
Not monitoring contact frequency and opt-outs
There’s nothing worse than receiving too many emails from a brand or business. But hey, that doesn’t apply to you, right? You probably only send survey requests every few months. Well, you’re not the only person in your organization who wants insights. What if, during the course of a week, your Client Success team sends out a regular NPS survey to customers, Sam from Marketing solicits event feedback and Julie from the Support team sends follow-up emails to any customer who called in for help? If you decide to send your survey during that very same week, there’s a high likelihood that some of your customers are getting bombarded with emails from your company, and they’re going to be irritated.
Even worse, if you aren’t all streamlined on one research platform, your customers or respondents have no easy way to opt-out of all this communication.
So how do you avoid this? If multiple people within your organization need insights, consider investing in a panel and respondent management tool that can help you manage contact frequency, target respondents and control global opt-outs. With a tool like this, you can automate the way you interact with your respondents so you can keep them loyal and happy.
If you want to be sure you won’t be spooked by bad data, check out our survey methodology blog series for tips on designing better surveys.