Writing survey questions seems deceptively easy. We all ask dozens of questions in our daily lives without much careful thought and planning, which leads many people to think that designing survey questions can be just as simple. But what people forget is that we depend heavily on contextual clues to interpret the meaning of questions in daily life – context that doesn’t exist for most survey questions.


It is surprisingly easy to confuse, frustrate or exhaust the mental energy of your respondents, and when this happens they are likely to stop engaging with your surveys and start short-cutting the response process – also known as “satisficing” – or opting out of your surveys altogether.


So how can you avoid frustrating your respondents?


This week we will highlight three mistakes you might be making in your survey design that confuse your respondents and hurt your data:


Mistake #1

Ambiguous words. Ambiguity in your questions can hurt your data in two ways: First, you don’t know how different respondents might be interpreting the data differently, which means you don’t know if the data collected from those respondents is actually comparable. Second, ambiguity makes it more difficult for respondents to figure out what you are asking, driving them to shortcut the cognitive response process and simply click any response that seems reasonable. Like we suggested in last week’s post, it is best to use simple and direct questions that leave little room for differences in interpretation.


Mistake #2

Double-barreled questions. These questions are the result of asking about multiple things in the same question. Sometimes you will want to ask several things about the same topic; for example, you may want to know how satisfactory and easy a transaction was for a customer. However, putting multiple subjects in the same question is a great way to get bad data. It is best practice to pick a single focus for each question. Key indicators of double-barreled questions include the words ‘and’ and ‘or’ – if either of these are present in a question it is worth looking more closely to be sure it isn’t double-barreled.


Mistake #3

Negations. Adding complexity to a question is almost never a good idea. Negations add a step in the mental process and make it much more difficult for your respondents to understand the question. Examples of negation include negatives, such as “students should not be required to take a comprehensive exam to graduate” and double negatives like “it is not a good idea to turn homework in on time.” There is almost always a way to phrase a question smoothly that doesn’t involve negating a statement. Making your questions as simple as possible is always best practice.