Academic Experience

Top 10 Survey Pitfalls

In a perfect world, we’d all be survey experts.

But since nobody’s perfect, sometimes we still make common mistakes when designing surveys.

There are best practices for creating surveys, and worst practices.

Start creating surveys today with a free Qualtrics survey account.

Here are some survey pitfalls that you need to avoid in order to get better data and make better decisions.

1. Loaded Questions or Leading Words

  • Avoid wording differences (could, would, should, might)
  • Watch for strong words (prohibit)
  • Don’t use biased phrases (wouldn’t you…)

2. Misplaced Questions

  • Consider question order and context
  • Use the Funnel Approach
  • Avoid “ringer” or “throw away” questions

3. Mutually Non-Exclusive Response Categories

  • Use mutually exclusive multiple choice questions unless otherwise specified

4. Non-Exhaustive Listings

  • Are all of the options covered?
  • Do a pretest or use an “Other (please specify)” option
  • Cover at least 90% of the respondent answers with your survey

5. Nonspecific Questions

  • Build questions that are clearly understood
  • Be specific about what you want to know

6. Confusing or Unfamiliar Words

  • Avoid specific jargon or confusing acronyms
  • Make sure your audience understands your language level, terminology, and intent
  • Keep it simple (9-11th grade reading level)

7. Non-directed Questions that Give Excessive Latitude

  • Provide clear intent and direction for the question

8. Forcing Respondents to Answer

  • Avoid being too intrusive
  • Try offering incentives and assurances of confidentiality

9. Unbalanced Listings

  • Avoid unbalanced scales
  • Clearly identify end points

10. Double Barreled Questions

  • Don’t ask two questions simultaneously

eBook: The Qualtrics handbook of survey question design