Thinking of checking up on respondent attention mid-survey to make sure that you’re getting good data? Think again. In this article, we highlight how new findings from our Qualtrics Methodology Lab are helping us to revisit and refine advice that is commonly given to survey researchers, namely the use of attention check questions to ensure […]
The polling industry is reeling after the 2016 election. While the absolute level of errors was not unreasonable for most of the polls (between 3-4 percentage points), those errors were not randomly distributed, and in fact nearly all of the polls missed in the same direction—favoring Hillary Clinton over Donald Trump. When errors point in […]
The survey is the heart of any survey research project. Whether it’s an intercept survey, a survey of the general population, or a survey of customers or employees, the way you design your survey plays a critical role in shaping the quality of the data and insights that you’ll get. Because people ask each […]
Otto von Bismarck one remarked that “Laws are like sausages, it is better not to see them being made.” He obviously wasn’t a huge proponent of process transparency. There are researchers that might advance a similar philosophy about the results or insights of their work. In some cases, this is because they’ve massaged the […]
Previously we highlighted some of the concerns surrounding people that appear to respond very quickly to surveys. While concerns about survey speeding are valid, there are indications that the solution is not to simply throw out these respondents from your dataset. Despite the evidence that speeders don’t really appear to be doing that much […]
If you’ve ever looked at completion times for individual questions or an entire questionnaire, you’ve probably noticed that there can be a lot of variation. Some respondents are slower than you expect, some are in your expected range, and some are faster than you expect. If the fast completion times trouble you in particular, you’re […]
Even for experienced researchers, intuition about survey design often runs counter to best practices outlined in survey methodology literature. One of the question types that we see used in web surveys most frequently is a particularly problematic type known as the grid or matrix question type. In some cases, researchers use grid questions […]
Have you ever been following navigation instructions while driving and wished that an instruction or street sign would’ve been more clear or specific? Maybe you couldn’t tell in time which lane led to that “slight left” you were supposed to take, and that miscue put you on the wrong route. Similarly, improperly labeled response […]
How you ask survey questions can make or break your data, and if you’re asking questions about sensitive topics, how you ask becomes even more critical. When answering questions about sensitive topics – such as sexual activity or drug use – people tend to give what they perceive to be a socially acceptable answer, instead […]
