Survey Speeding, Part 1: Is It Harmful or Harmless?
If you’ve ever looked at completion times for individual questions or an entire questionnaire, you’ve probably noticed that there can be a lot of variation. Some respondents are slower than you expect, some are in your expected range, and some are faster than you expect. If the fast completion times trouble you in particular, you’re not alone.
Survey Speeding explained
When people respond too quickly, researchers commonly consider it an indicator that the respondents did not carefully and thoughtfully engage with the question or questionnaire. They may surmise that the responses are low-quality or invalid as a result.
These are not baseless concerns. Some research indicates that when respondents are notified that they’ve responded in a shorter time frame than the minimum amount of time it takes to read the question, they slow down and provide better quality data. Other research indicates that slower response times are associated with less satisficing behavior. ,
Consider other factors
The evidence isn’t completely clear-cut; other research indicates that setting minimums for completion times can be counterproductive, and that deciding to exclude fast respondents may not be worth it.
Before you label all your fast respondents as “speeders” and throw out their data, consider other predictors of higher response speeds, such as education and age. Your speeders may primarily be younger and more highly educated individuals, and you could be introducing a demographic bias into your data by removing them.
Additionally, there may be little reward associated with excluding fast respondents. Research on the effects of removing fast respondents from the data indicates that there is no discernable effect in terms of improved data quality or even differences in the marginal distributions of variables.
I recommend giving those speeders a second chance. Don’t simply assume that their data are bad and throw them out of your dataset. First, verify the effect that they are having on your variables of interest by looking at the data with and without them included. Chances are they aren’t doing as much damage as you think.
Check out Survey Speeding, Part 2: Designing Surveys to Avoid Harm