Brand Experience

Social distancing in market research: Pivoting in-person methodologies

As public health authorities urge citizens to follow social distancing guidelines - or implement even more stringent measures - many market researchers are pausing or delaying in-person data collection.

Whereas once a researcher may have surveyed respondents exiting a grocery store, or gone door-to-door, today these actions might be hazardous to both respondents and to the in-person interviewers. Even the U.S. Census Bureau has temporarily suspended their in-person follow-up visits to households that have not completed the form by mail or online.

Fortunately, not all in-person data collection needs to be put on hold; in many cases, market researchers can effectively transition their in-person data collection into the online realm. Read on for some things to consider before you make the switch.

Sampling and Recruitment

Is your target population online? If a large portion of your target population is not online, then an online survey won’t be a good fit. Instead, computer-assisted telephone interviewing (CATI) is a good alternative for reaching offline populations without the need for face-to-face interaction. If you have phone numbers available, you can also use SMS distributions to send your survey to audiences which otherwise may not take surveys online.

If most of your target population is online, then the main consideration is how to reach them.

Ideally, your online survey respondents would reflect the sample population you were interviewing with your in-person survey. If you have mailing addresses, you can use a “push-to-web” strategy to move responses to the web. Mail your sample members a letter with a link to complete the survey online. Offering a small prepaid or postpaid incentive can increase the likelihood that people respond.

If you don’t have mailing addresses, you can also purchase a random sample of mailing addresses. Note that sampling mailing addresses is best when you want to survey a geographically-based population, rather than to specific market segments.

There are also a number of online recruitment options. The goal is to find a method that will generate an equivalent set of respondents to your in-person data collection methods. A few options to consider:

  • Internal sources. If you have an email list of your target population (e.g., customers), then you can easily send them an email with a link to an online survey. Other options include engaging people as they visit your website or after they make a point of sale.
  • Third-party marketing lists. If you do not have a list that includes the entire population you want to reach, consider whether it is possible to purchase a third-party marketing list.
  • Targeted online ads. To reach specialized populations or niche markets, you can place targeted ads online (e.g., Facebook, Instagram, LinkedIn, Craigslist). Most of these platforms allow you to limit the ad to specific subsets based on factors such as age, gender, purchasing behavior, and interest/hobbies.
  • Online panels. Online panels are a sample of people who have agreed to complete surveys online. Online panels typically allow you to target respondents based on demographic information such as race, location, age, gender, or ethnicity. With online panels, you can usually capture a large amount of data within only a few days or a week.

Response Quality

Not surprisingly, research shows that respondents are more likely to respond truthfully in online surveys than in interviewer-administered surveys. Online surveys are perceived as more private and confidential, which makes respondents answer more accurately.

However, online surveys can suffer from respondent satisficing, which occurs when respondents provide a “good enough” answer rather than an optimal answer. To reduce satisficing, online surveys need to be shorter than in-person surveys - ideally 5-20 minutes.

Cost Implications

Online surveys tend to be cheaper than in-person surveys, so there may be a potential for cost savings. When it comes to online data collection, the major cost drivers are:

  • Costs to purchase sample or lists
  • Web survey software license
  • Labor hours to program and test the online survey
  • Incentives (if offered)
  • Mailings and postage (if using push-to-web approaches)

Replicating Complex Samples and Studies Online

Studies that involve complex sampling or data collection strategies will be harder to replicate online. Researchers using probability-based random samples may have more difficulty replicating their survey online with the same level of precision. If this is the case, try to use advanced weighting or modeling techniques to lessen the potential for bias. Researchers can also take advantage of probability-based online panels. These tend to be more expensive than non-probability samples, but are associated with more accurate and reliable results.

Also, surveys that require the interviewers to collect data other than self-reports (e.g., scanning barcodes of recent purchases) will be harder to replicate online. Researchers will need to consider other methods for collecting this data such as using an app or having respondents mail the materials.

Before trying to perfectly replicate your in-person survey, consider the general information you’re trying to obtain, and identify aspects of your research that can be answered online. It may not be as robust, but collecting some data will be better than no data.

Replicating Qualitative Research Online

Online research can absolutely go beyond online surveys. Whatever video-conferencing software (e.g., Zoom, WebEx) your organization is using for online meetings, can also be used for one-on-one qualitative interviews as well as focus groups. Participants can even use their phones or tablets to participate!

To make this remote experience better, consider the following:

  • Prescreen participants to determine if they have the needed equipment.
  • Send participants instructions for logging in from the device they will be using in advance.
  • Assume that the first 5-10 minutes of your session will be spent waiting for people to set up.
  • Encourage people to use their phone for audio rather than their computer. That way if their wifi is spotty and the video goes in and out, they can still participate.
  • Consider recording to your computer instead of the cloud for enhanced security.
  • Check with your organization’s IRB or legal teams to ensure you are properly protecting participants’ privacy and confidentiality.

And if you want a stress-free approach, there are a number of companies that handle all of the online logistics for you, including participant recruitment and providing technical support to participants.

Alternatively, you can integrate video interviews directly into your quantitative survey through a video insights solution like VoxPopMe; this is another avenue to gain qualitative insights in a timely and cost-effective manner.

Transitioning Studies That Are Already In-Flight

Switching modes in the middle of data collection could potentially negatively affect data quality. Different people are more willing to respond in some modes compared to others. This could lead to bias due simply to the different mix of people answering the survey. Quotas and weighting by demographics can curb this to some extent, as long as specific differences in who responds in-person versus online are known and measured in both modes.

Furthermore, questions may be understood and answered slightly differently when administered via an interviewer compared with self-administration. It may not be possible to accurately combine the online data with the in-person data. If you have already collected a lot of responses in person, you will need to consider whether you can afford to delay or whether you should start over with an online survey.

It’s impossible to say exactly how long social distancing is expected to last - but it will not last forever. And certainly, in-person study methodologies will remain an important piece of the market research puzzle. Even in developed countries, significant swaths of the population cannot or do not regularly access the internet. And, in-person data collection will continue to be needed to reach hard-to-reach or niche markets, to achieve high response rates, or to enable real-time data collection when timing is critical (e.g., exit polls).

In the meantime, we’re here to help. Our Research Services team is on-hand to answer questions as you explore online alternatives to your current study design, so that you can keep your research moving forward in a time of so many new unknowns.

You can find more information on how Qualtrics is helping organizations navigate and respond to the unfolding COVID-19 situation at qualtrics.com/here-to-help.

Emily Geisen // Sr. Experience Management Scientist

Emily Geisen in a Senior Experience Management Scientist at Qualtrics, focused on improving data quality and reducing respondent burden.

Related articles