How to Get the Truth When Asking Survey Questions about Sensitive Topics
Imagine you are responding to a survey and you’re asked about a sensitive topic, such as drug use, sexual behavior, racial attitudes, or your income. You might be reluctant to tell the truth and instead respond with what you think is a “socially acceptable” answer. This tendency is called “social desirability bias,” and if you’re not careful, it can bias your data, too.
Strategy 1: Validate against known data. In some cases, you might be able to validate the survey responses and figure which of your respondents are not being truthful. Income, for example, it is a topic that is difficult to measure for many reasons, but one is that most people feel some anxiety about reporting values that they perceive to be too low or too high because being perceived as either very poor or very wealthy can be embarrassing. This leads to inaccurate estimates that don’t match the actual distribution of incomes in the population. But if you are able to get access to bank statements or other gold-standard records of income or wealth, you can check to see who is truthful and who is not.
But it can be difficult to get gold-standard data to benchmark your survey data against. For example, you’ll likely never be able to validate responses about sexual behavior or racial attitudes, because there is no gold-standard data to measure these topics.
So then what else can you do to avoid social desirability bias?
Strategy 2: Ask the respondents to answer for other people instead of themselves. People tend to view themselves and their loved ones favorably. Also known as the “Lake Wobegon Effect,” humans have a natural tendency to overestimate their abilities and good qualities. In survey research, often see something similar to the Lake Wobegon Effect when asking respondents about sensitive topics for which they might be tempted to give a misleading socially acceptable answer. One way to get around this is to ask people to answer the socially desirable question about other people.
For example, in a recent study that we conducted at Qualtrics, we interviewed a large number of parents about their child’s technology usage––but for certain potentially socially desirable questions we asked the parents to evaluate their child’s friends rather than their own child. As expected, parents consistently rated their own kid’s behaviors as being better than their kid’s friends. However, research indicates that the true value of the estimate is likely to be closer to the parent’s evaluation of their child’s friends. One key point to keep in mind about this strategy is that it works best for topics that the respondent can reasonably evaluate about other people.
In my next post, I’ll highlight two more strategies that you can use to help reduce socially desirable responses.
Want more handy tips on research methodology right now? Check out Three Reasons Why Shorter Surveys are not Always Better and Six Ways to Pretest Your Survey.