Optimal Scale Labels: Why They Matter
In our last post, we discussed how many scale points to use and when. We also noted that you’ll get the best data when you verbally label every point.
So, how should you go about labeling them? There are three key best practices:
- Map the response options to the question
- Use consistent language for the entire rating scale
- Use balanced scales
Map the Response Options to the Question
It’s always a good “best practice” to match the response option labels to the subject of the question. For example, when asking about satisfaction, include the idea of satisfaction in the response options like we do in the example below:
Following the example above will save you from a common mistake people make—using consistent response scale labels for many questions. The most common example of this is the agree-disagree scale.
Here are some examples of what NOT to do:
These are examples of biased questions. They are written to take advantage of the same response scale and don’t include the subject in the scale labels. This may produce less valid data.
Use Consistent Language for the Entire Rating Scale
Using consistent language for the entire rating scale makes you less likely to confuse your respondents. This means that you should avoid mixing adjectives in the response scale.
For example, don’t ask about satisfaction and then include synonyms such as ‘pleased’ or ‘content’ in the response scale. If you’re asking about satisfaction in the question, use satisfaction in the answers. Using uniform language for each level of the scale will yield the most comparable data.
Use Balanced Scales
Using evenly balanced response scales is another way you can avoid potentially biasing responses. In all of the response scales above, the range of positive and negative options is equal on either side of the midpoint, numerically and conceptually.
An example of an imbalanced response scale for the satisfaction question might look something like this:
The response options above dramatically weight the scale toward positive responses in two important ways. First, it includes four scale points for positive options and only one for a negative option. Second, it covers less detail in the conceptual space of ‘dissatisfaction’ that your respondents might experience. Doing this will bias responses and reduce your data quality.
When you’re writing response options, you can improve your chance of producing high quality data by:
- Verbally labeling all scale points
- Using response options that map to the subject of the question
- Using consistent language across the entire rating scale
- Using balanced scales
Check back next Wednesday for more from the Qualtrics Survey Methodology Series.
May 21, 2020
How the Utah State Board of Education is winning funding and addressing the COVID-19 crisis
May 8, 2020
The well-being question we most definitely should be asking students, parents and teachers
September 18, 2019