3 Reasons Why Shorter Surveys are Not Always Better
Shorter surveys generally have higher completion rates and take less time to complete. As a result, many survey designers attempt to make their surveys as short as possible. However, in many cases, asking fewer questions might actually hurt your ability to get the insights that you need from your data. In this post, I’ll discuss three reasons to think twice before shortening your survey:
Reason #1: Asking too few questions hurts your data
It’s generally a good idea to ask as many questions as you need and no more. Often the drive to write short surveys can leave researchers with partial insights that don’t adequately address their research needs. While it is certainly true that unnecessary questions should be avoided and questionnaires should be kept as short as possible, it is also important to keep your data needs at the center of your survey development. If you don’t ask enough questions to address your needs, then you’re wasting your time and your respondents’ time.
Reason #2: Grid and matrix questions may confuse your respondents
Another way that researchers try to “shorten” surveys is by taking many questions and forcing them to fit into a grid or matrix question. Many researchers think that this is an easy way to take 20 questions and make them into one, and who wouldn’t rather answer one question than 20?
But attempting to gather a lot of data in one grid or matrix question is actually ineffective; first, because you are still asking respondents to make the same number of evaluations and responses, and second, by forcing your questions into a grid it becomes much more difficult to clearly ask for the information you want using optimal scale labels.
This means that your grid question is likely going to take more mental energy for your respondents to answer than the 20 well-designed independent questions. This is likely to result in lower engagement with your survey and lower quality data due to things like respondent misinterpretation, straightlining, speeding through your survey and misunderstanding of your response options.
Reason #3: Asking about more than one thing at a time biases responses
Another common mistake that researchers make when trying to write shorter surveys is asking about multiple things in the same question. To avoid confusing your respondents and ending up with useless data it is best to only ask for one evaluation at a time.
For example, instead of asking “Should the government spend less money on the military and more on education?” first ask whether the respondent thinks that the government should spend less money on the military and then ask a second question about whether they think more money should be spent on education.