When your survey only meets your needs and doesn’t respect the needs of your survey respondents, it will be a frustrating experience for both of you. Here’s how you can think about survey design to make your surveys more enjoyable for respondents.
Keep your surveys to under 15 minutes
Your average survey respondent will start to feel bored about 15 minutes into a survey. Fifteen minutes is a good upper limit for most surveys.
Here’s what happens when your survey is too long:
- Survey Respondents Drop Out: They simply quit taking the survey. It costs money to find respondents, and a high drop-out rate can not only cost you more, but it can influence the quality of your results. Having a reward for completion can reduce drop-outs, but you can’t stop it completely.
- Survey Respondents Stop Paying Attention: Remember your primary school classmate who just filled in random bubbles during a test? He grew up. If your survey is too long, he might do it again.
- You Anger Your Customers: Don’t ask your customers to kindly take the time to answer your survey and then disrespect their time with an overly long survey. The best way to collect quality data is to keep your surveys short, simple, and organised.
Use images where appropriate
Humans usually respond more favourably to images than to text, and often understand concepts more quickly with an image or an icon. Breaking up long blocks of text with images, header bars, and icons make reading your survey more pleasant.
Your software platform should offer question types that include graphics such as smile/frown faces, graphic sliders, and the capability to upload your own images or video.
Images should add meaningful context to a question but should not bias responses by making one option seem more attractive than another.
Avoid matrices and grids
A grid or matrix question format asks respondents to give a rating along a scale for a number of different variables. While it may look like a neat and effective format, it asks the respondent to do quite a bit of work and could actually negatively impact your data.
When confronted with a grid of checkboxes or radio buttons, it’s not surprising that some respondents disengage from so many possibilities. They also need to match up the correct row with the correct column for each answer, which adds to the cognitive load of the task and leaves less brain-space available to consider the question.
A good alternative is to use a separate scale for each topic, with question-appropriate options such as Strongly-agree – Strongly disagree, Very satisfied – Very unsatisfied or Always – Never.
Take a different approach to questions about sensitive subjects
If you want to gather data about topics like religion, politics, personal hygiene or sexual health, a couple of potential problems appear.
The first is that respondents may just reject questions that make them uncomfortable. Avoid putting this type of question near the start of a survey, where it can seem more intrusive and inappropriate. Instead, place it mid-survey and warm up to it with questions that cover similar, less sensitive topics. Remember that taboos can vary widely across demographics and cultures, so you will need to bear in mind the composition of your panel when wording this type of question.
The second is social acceptability bias – the tendency to give answers that are less accurate in order to avoid identifying with behaviours seen as negative. See tips on how to work around social acceptability bias.
Think twice before including ‘don’t know’ response options
Adding ‘don’t know’ responses can be problematic, because they give respondents a path of least resistance which can skew your data. Checking the ‘don’t know’ or ‘no opinion’ option is easier than thinking through whether a yes or no answer applies, allowing some respondents to simply disengage with the survey.
Even if a respondent genuinely doesn’t know the answer, their best guess is likely to be more beneficial to your data than a ‘don’t know’, according to research. Read more about the trouble with don’t know questions.
Use scale questions
Rather than asking respondents a basic yes or no question, use scales that measure both the direction and the intensity of opinions. This is critical for research. Someone who “Strongly Supports” a decision is very different from someone who “Slightly Supports” it. Scales extend the power of analysis from basic percentages to high-level analyses based on means and variance estimates (like t-test, ANOVA, regression, and the like).
Scales make it easier for respondents to answer and for you to conduct your analysis. If scales have the same scale of points, you can quickly compare responses to different questions.
Survey respondents are usually adults, and adults like to know they’re not wasting their time. While your reason for asking a particular question might be perfectly clear to you, it may not be apparent to your respondents. When a respondent doesn’t see the point of answering certain questions, they may respond without enough thought.
Here are four ways you can make sure respondents understand your questions:
- Explain Unexpected Questions. For instance, if it’s important for you to ask toy store customers what time of the day they usually visit the store, briefly explain why that’s relevant.
- Justify Requests for Sensitive Information. This is particularly true with any information about ethnicity, family status, income, education, etc. You can diffuse respondents’ concerns by telling them how it is going to be used. For instance, you can explain that purchasing habits will only be analysed in aggregate for benchmarking purposes or that results will not be shared outside your organisation.
- Speak Your Respondents’ Language. Asking about caloric content, bits, bytes, and other industry-specific jargon and acronyms is confusing. Make sure your audience understands your language level and terminology and above all, that they understand what you are asking. The best move is to write to your least-informed respondent. If a respondent won’t understand an acronym, either define it or don’t use it.
- Use the Funnel Approach. Make your survey easier for respondents by keeping questions in their logical order and avoiding changing topics unnecessarily.
The funnel approach is a survey flow technique that makes the respondent’s job easier. This approach suggests that you:
- Follow a logical order
- Start with broad and general questions that qualify the respondent and introduce the topic
- Move into more specific questions
- Finish with general, easy-to-answer questions (like demographics)
This approach allows respondents to warm up with broad and general questions, work into more specific and in-depth questions, and cool down at the end.
This turns the survey into a smooth road for respondents, which decreases drop-out rates and may even increase the quality of answers you receive. Even the best researchers have the occasional typo, misdirected question, or unfamiliar buzzword in their surveys.
Finding these last little issues is a difficult process. Fortunately, there is an easy solution:
Ask 5 people who match your sample to take your survey
- Then ask them:
- How long did it take?
- Which questions were confusing?
- Were there any other problems while taking the survey?
This allows you to quickly correct lingering problems before distribution.
Write surveys you would answer yourself
Survey building is as much an art as it is a science. It involves attention to detail in the design and flow of your survey. Keep it simple, keep your scales consistent, and communicate well. Review your question flow and then pre-test.
These simple tips will go a long way toward building your “perfect” survey.