Topic: surveys


Survey Speeding, Part 2: Designing Surveys to Avoid Harm

Previously we highlighted some of the concerns surrounding people that appear to respond very quickly to surveys. While concerns about survey speeding are valid, there are indications that the solution is not to simply throw out these respondents from your dataset.   Despite the evidence that speeders don’t really appear to be doing that much […]

Read More
Survey Speeding, Part 1: Is It Harmful or Harmless?

If you’ve ever looked at completion times for individual questions or an entire questionnaire, you’ve probably noticed that there can be a lot of variation. Some respondents are slower than you expect, some are in your expected range, and some are faster than you expect. If the fast completion times trouble you in particular, you’re […]

Read More
Why Using Grid Questions Is Probably Hurting Your Data

Even for experienced researchers, intuition about survey design often runs counter to best practices outlined in survey methodology literature. One of the question types that we see used in web surveys most frequently is a particularly problematic type known as the grid or matrix question type.     In some cases, researchers use grid questions […]

Read More

Over the past few weeks, we’ve talked about getting in your respondent’s heads, the threats posed by acquiescence bias, ‘don’t know’ responses, straightlining, and selecting the first acceptable response option. These threats are predicted and described by the theory of survey satisficing.

Read More
Reducing Bias In Survey Responses: The Role of Randomization

A common way survey designers risk biasing their data is by including questions with lists of possible responses. Respondents who are not motivated to fully engage with your survey may avoid reading all of the response options and instead choose the first reasonable option before moving on.

Read More