Avoid matrices and grids
A grid or matrix question format asks respondents to give a rating along a scale for a number of different variables. While it may look like a neat and effective format, it asks the respondent to do quite a bit of work and could actually negatively impact your data.
When confronted with a grid of checkboxes or radio buttons, it’s not surprising that some respondents disengage from so many possibilities. They also need to match up the correct row with the correct column for each answer, which adds to the cognitive load of the task and leaves less brain-space available to consider the question.
A good alternative is to use a separate scale for each topic, with question-appropriate options such as Strongly-agree – Strongly disagree, Very satisfied – Very unsatisfied or Always – Never.
Take a different approach to questions about sensitive subjects
If you want to gather data about topics like religion, politics, personal hygiene or sexual health, a couple of potential problems appear.
The first is that respondents may just reject questions that make them uncomfortable. Avoid putting this type of question near the start of a survey, where it can seem more intrusive and inappropriate. Instead, place it mid-survey and warm up to it with questions that cover similar, less sensitive topics. Remember that taboos can vary widely across demographics and cultures, so you will need to bear in mind the composition of your panel when wording this type of question.
The second is social acceptability bias – the tendency to give answers that are less accurate in order to avoid identifying with behaviours seen as negative. See tips on how to work around social acceptability bias.
Think twice before including ‘don’t know’ response options
Adding ‘don’t know’ responses can be problematic, because they give respondents a path of least resistance which can skew your data. Checking the ‘don’t know’ or ‘no opinion’ option is easier than thinking through whether a yes or no answer applies, allowing some respondents to simply disengage with the survey.
Even if a respondent genuinely doesn’t know the answer, their best guess is likely to be more beneficial to your data than a ‘don’t know’, according to research. Read more about the trouble with don’t know questions.