Use Balanced Response Scales, Even When the Results Could Hurt
As a child, maybe you had one parent or mentor who was the “good cop” and one who was the “bad cop.” When you got in trouble, or received a bad grade in school, you probably preferred to tell the good cop, right? That person was likely to be more lenient and a little less critical.
The tendency to avoid criticism isn’t unique to childhood. For many organizations, a satisfaction rating of “good” just isn’t good enough, and as a result, they may try to tip their survey response scales in their favor to elicit more “good” responses. Unless you’re just looking for a pat on the back instead of real feedback to help your organization improve, using unbalanced response scales is a bad idea. No matter how painful it may be to receive poor satisfaction ratings, trying to bias a response scale toward either end of the spectrum leads to inaccurate, and therefore potentially misleading, data. Many researchers already know and avoid this intentional bias in their projects.
But there is another way that the same bias can creep in, and this is the desire to understand one end of the rating scale more than the other. For example, a company may believe that any customer service rating below “Extremely Satisfied” or “Very Satisfied” is effectively “Bad” and they may consider writing a response scale that looks something like this:
The problem with this approach is that this scale effectively collapses half of the response scale and doesn’t allow respondents full freedom to report how they really feel. It is possible for a customer to have an experience that leaves them feeling “Extremely Satisfied” but it is also possible for them to feel “Extremely Dissatisfied.” The response scale above is biased toward the positive (satisfied), so the data this company receives will also be biased and inaccurately represent its customers’ feelings. Part of the reason this can be biased is that it shifts the midpoint of the response scale (which is commonly interpreted by respondents as “average” on the scale) from a neutral option to a positive option.
On the other hand, a very large social networking site recently asked a sample of its users the following question:
Many managers might argue that even a rating of “Neither good nor bad for the world” would be characterized as a bad rating for a brand. But this organization made the decision to collect the most accurate data that they could by using an evenly balanced response scale that covered the entire range of possible responses. This a real-world example of a company who used response scales correctly, even if the responses they received were “bad.”
While it might be painful for employees of this company to learn that many of their users believe that their company is “Very bad” for the world, using the correct approach helped ensure that they collected accurate data to help them make real improvements.
Similarly, if you want to collect accurate and useful data, remember that your response scales should cover the entire range of possible attitudes about your subject. Collecting unbiased data is much more helpful in the long run, regardless of whether or not you care equally about both sides of the response scale, or how painful receiving “bad” feedback is now.