Previously we highlighted some of the concerns surrounding people that appear to respond very quickly to surveys. While concerns about survey speeding are valid, there are indications that the solution is not to simply throw out these respondents from your dataset. Despite the evidence that speeders don’t really appear to be doing that much […]
If you’ve ever looked at completion times for individual questions or an entire questionnaire, you’ve probably noticed that there can be a lot of variation. Some respondents are slower than you expect, some are in your expected range, and some are faster than you expect. If the fast completion times trouble you in particular, you’re […]
Even for experienced researchers, intuition about survey design often runs counter to best practices outlined in survey methodology literature. One of the question types that we see used in web surveys most frequently is a particularly problematic type known as the grid or matrix question type. In some cases, researchers use grid questions […]
Even if you’re good at sending survey invitations without being irritating, some of your panelists will want to opt out and some emails will bounce. Knowing how to manage and limit bounces and opt-outs is an important part of panel management.
Over the past few weeks, we’ve talked about getting in your respondent’s heads, the threats posed by acquiescence bias, ‘don’t know’ responses, straightlining, and selecting the first acceptable response option. These threats are predicted and described by the theory of survey satisficing.
A common way survey designers risk biasing their data is by including questions with lists of possible responses. Respondents who are not motivated to fully engage with your survey may avoid reading all of the response options and instead choose the first reasonable option before moving on.