How to label response scale points in your survey to avoid misdirecting respondents
Have you ever been following navigation instructions while driving and wished that an instruction or street sign would’ve been more clear or specific? Maybe you couldn’t tell in time which lane led to that “slight left” you were supposed to take, and that miscue put you on the wrong route.
Similarly, improperly labeled response scales can misdirect respondents when answering your survey questions.
We’ve previously discussed the importance of labeling every point on your response scales. But how exactly should you label them? Unfortunately, few researchers give their scale labels much thought, let alone seek out best practices recommended by survey methodologists.
Here are two examples of response scales that, on the surface, may seem equally viable:
Though they may seem similar, two key differences make the first example much less appealing. First, the endpoints of the response scale in the first example are not maximally differentiated—we want to separate respondents at the ends of the scale as much as we validly can. The second example does this better.
The second difference in the first example is the presence of a response option that is not mutually exclusive of the others—the ‘Important’ option. Given that intensity increases on either end of the midpoint, we can infer that ‘Important’ is intended to convey less intensity than ‘Very important’ and more intensity than ‘Slightly important.’ But technically speaking, ‘Very important’ and ‘Slightly important’ are both sub-classifications of ‘Important,’ meaning that this wording could confuse some respondents and hurt your data accuracy.
How do you avoid this problem? By explicitly labeling each scale point with an appropriate modifier.
Here’s an example of a seven-point bipolar scale where, through the use of appropriate modifiers, all responses are mutually exclusive of one another.
Finally, to ensure that you get the highest-quality data possible, keep these four guidelines in mind as you practice labeling response scales:
- Ensure each scale point is verbally labeled
- Use response options that correspond to the subject of the question
- Use consistent language across the entire response scale
- Use balanced scales
For more tips on how to design response scales, check out “Three Tips for Effectively Designing Rating Scales.”
Free eBook: The Qualtrics Handbook of Question Design
December 22, 2020
Salt & Straw CEO Kim Malek: ‘Community is the power behind our flavors and our philosophy’
December 22, 2020