Mine Metadata to Strike Survey Gold
Utah, where Qualtrics is headquartered, is home to one of the largest copper mines in the world. The open-pit mine spans a massive 2.5 miles wide and nearly 1 mile deep. Every year as the mine expands and the pit gets deeper, more copper and minerals can be found and extracted that eventually go into medicine, food, shampoo, cell phones, computers, CAT scans, hybrid electric cars and more.
So what does this have to do with surveys? Learning to dig deeper than the surface results of your well-designed survey can produce more gems than expected.
If you’ve designed a good survey that fits your research needs and you’ve tested it on an appropriate sample of respondents, you’ll end up with plenty of solid data to sift through. But underneath your respondents’ answers, there’s a deeper level of data that can be just as valuable.
Metadata, or data about the data, includes things like the type of browser that each respondent used when completing the survey, the amount of time it took from start to completion for each respondent, the response time for each question or page, the time of day that the survey was completed, the IP address of the respondent, and many similar data points about each survey started or completed.
At first glance, these may not seem like extraordinarily useful nuggets of information, but they might have a big effect on how you manage the dataset and interpret your results.
When you look at your metadata, consider analyzing:
- Device information. Look at what devices your respondents were using to complete the survey. In some cases, particularly if your survey is not mobile-friendly, you may want to analyze the respondents that answered from PCs separately from those that answered from a mobile device to make sure that there weren’t glaring differences in respondents’ answers.
- Individual question response times. If certain questions took unexpectedly long for respondents to answer then there may have been problems with those questions. Make sure that there were not any issues with typos, ambiguous words, double-barreled questions, complex language or jargon that may have confused your respondents.
- Overall response times. If the survey as a whole took longer than expected, then you might consider whether or not there are ways that you could make it shorter to reduce respondent burden and improve response quality.
There are many additional ways that you can make use of survey metadata, but these are likely to vary depending on your goals. Exploring your survey metadata is a worthwhile investment because you might discover insights you didn’t expect to find. The insights that you gather from metadata can shine new light on your research and help you design better surveys in the future.