Written by Scott Smith, Ph.D. December 31, 2012
Creating online surveys is as much an art as it is a science.
It involves attention to detail in the design and flow of your survey. Creating an effective survey that yields actionable insights can be difficult.
Effective survey design and flow gives power to your research. But great questions are the foundation for great research.
There are fundamental best practices for creating survey questions that all researchers must know.
Here are 4 tips for creating surveys that work.
Do you remember taking the SAT or ACT? It’s a long and boring process.
Your average survey respondent can start to feel that way about 15 minutes into a survey. Fifteen minutes is a good upper-limit for most surveys.
When a survey is too long, three bad things can happen:
Rather than asking respondents a basic yes or no question, use scales that measure both the direction and the intensity of opinions.
This is critical for research.
Someone who “Strongly Supports” a decision is very different from someone who only “Slightly Supports” it.
Scales extend the power of analysis from basic percentages to high level analyses based on means and variance esti- mates (like t-test, ANOVA, regression, and the like).
Use scales whenever you can. You will get more information from each question.
Every survey response, option, question, or answer is coded as a numeric value that is reported as a percent of responses or as a mean, median, range, etc.
These values are the basis for analysis.
Values must be coded consistently. Generally, we assign the highest value to the best outcome (ie “Strongly Agree” that customer service is responsive) and then move down from there.
For simplicity, keep your scale direction consistent throughout your survey. This makes it easier for respondents to answer and for you as a researcher to conduct your analysis.
If scales have the same scale of points, you can quickly compare responses to different questions. For example, if a survey asks respondents to rate a series of statements from Strongly Disagree to Strongly Agree, the responses are given these values:
Standard scaling helps managers to quickly understand customer service ratings by simply looking at averages.
For example, once managers understand that a 5-point agreement scale is being used, they could be given the mean results for the following customer evaluation (agreement) statements:
Since all the statements are positive and the values are scaled consistently, a higher mean reflects better results in that area. A manager can look at these means and quickly identify the 2.08.
We see that customer service representatives are prompt and polite, but they don’t seem to know what they’re talking about. As a result, overall satisfaction with customer service is perhaps much lower than it could be.
You can reverse scales (or word questions negatively) to encourage respondents to read more carefully.
However, if you use reversed scales or negative wording for some items, be sure to recode the scales so that all scales point in the same direction. This will allow you to quickly compare multiple areas of customer service. (You can do recodes easily in Qualtrics.)
The simplest solution is just to keep all scales consistent throughout every survey.
Respondents are more likely to help you if they see something of positive value for them.
Value offerings can range from a very general alttruistic appeal for their help to a very specific offer of an economic incentive. For instance, with a customer feedback survey, you can explain that feedback will help improve customer service.
Here are some quick examples:
So what do you think? What are some tips you’d give to fellow researchers? Leave a comment below and let’s discuss.
Basic Marketing Research Volume 1: Designing Your Study
Ever wish you could get straight to the point when it comes to marketing research? Well here’s your chance. Get “Basic Marketing Research Volume 1: Designing Your Study” written by Scott Smith Ph.D., along with monthly research tips delivered directly to your inbox. Just type in your email address below.
Get the book FREE