Market research is a $44.5 B market and growing. Online research is among the fastest growing parts of the market thanks to the pervasiveness of the web and the ease with which we can now collect data.

However, as the world conducts more and more survey research, the issues that we see elsewhere with big data are now affecting the survey research industry as well, specifically the issue of data quality.

Thanks to the growth in online survey research, billions of survey responses are collected every year. But 1/4th of those responses are of poor quality[1]. In fact Quality of Data Insights generated is the most important criteria (59% voted as most important decision) for deciding the Market Research software or vendor[2]. A survey of market researchers found Data Quality related challenges to be the most important[3]!

This is due to several reasons. Almost all of them have their origins in a lack of understanding on what impacts data quality. For example, a recent report by GRIT showed that quality of the responses is getting worse every year: 39% of researchers surveyed felt the quality of the respondent sample is going to deteriorate. Fraud via bots and human cheaters is on the rise, making many responses invalid. And surveys that take longer than 20 mins to complete are already seeing huge drop off rates. The research industry estimates $3-$4Billion in losses due to poor data quality every year[4]!

If the quality of the collected data is poor, the decisions made off those research findings will in the best case be unusable and in the worst case lead to harmful business decisions.

This is where AI and Big Data can help.

AI and Big Data allow you to automate ways to check for data quality. Based on a recent study by Greenbook, 10% of market researchers have already adopted methods to automate survey and project design and 20% are actively exploring ways to automate their survey design process.

And there is a reason for this. AI is most effective when (a) there is enough data and (b) the data is structured. Enter survey research data. It is, by definition, highly structured. And with companies like Qualtrics conducting billions of surveys, it is a match made in heaven.

Qualtrics and AI

Below are 4 key areas where Qualtrics has been effectively using AI to predict and improve poor data quality,

    1. How You Ask:  Asking the right questions matter. Too many questions and it can lead to fatigue and disengagement from the respondent. The words and answer choices used in a question can lead to various biases in how users answer those question. The type of question – a choice based one or an open-end text question or speaking out the answer in a voice-based device – all of these impact the quality of the response. For example, Qualtrics ExpertReview – an AI based digital research assistant – can scan through millions of anonymized responses to understand the impact various questions and their content can have on the quality of the response. It can then suggest how users can improve the questions they ask to maximize the response rates and quality of these responses.
    2. Whom You Ask: The audience matters. For example, if you are polling opinions of voters for an upcoming election, you need to be careful that you are asking the representative sample. If you were to only ask one group of people that does not represent the entire voting population, you will make wrong predictions on the outcomes of the election. This has been the biggest reason why pre-election polls and pundits often get it wrong.
      AI can match respondents answering the questions against their demographic information to make inferences on the representative sample that is being surveyed. AI can then predict how representative the current set of respondents are of the actual population and recommend any changes that are needed. Qualtrics ExpertReview is expected to include this feature in 2019.
    3. Where You Ask: People are willing to provide insightful answers, if you ask in the right channel. If you ask questions over mail the response rates and even the answers will be very different than say asking a set of questions via the Alexa integration in many new cars. The medium matters.
      AI can predict the quality of the response based on the response channel and then use that to recommend the best medium that researchers can ask questions on. This is another one of the planned extensions for ExpertReview coming in 2019.
    4. How They Respond: The research industry is seeing a rise in ‘human abusers’ – folks who, for a few quick bucks, will answer surveys posing as someone else. Bots taking surveys is also on the rise. The answers you will get from these sources will clearly be invalid.
      Secondly, with increasing regulations, if respondents provide information that is sensitive, it could get the researchers who are collecting this data into trouble. Recent high-profile data breaches cost billions in shareholder value because customers personal data was compromised. And GDPR regulations is a reminder with teeth that when researchers collect data, it needs to be compliant. AI can help detect these fraud patterns or Personal Identifiable Information (PII) in the responses that come through and recommend changes to the survey or flag responses that contain sensitive PII information.

At Qualtrics iQ Research Labs, improving research data quality is our mission. We are actively applying AI in all the above four ways to help researchers all over the world collect data of the highest quality.

Our vision is to prevent bad economic and political decisions for the world, one high-quality answer at a time.

 

ExpertReview – a AI based Digital Research Assistant analyzes billions of responses to predict the expected completion rate and recommend optimizations to the design of our research

Qualtrics expert review

Sources:

[1] Resonate Report on Fraud Detection

[2] GRIT Report from Greenbook Research Industry Trends Report 2017 Pg. 56 

[3] ‘Challenges to the Insight Industry’ GRIT Report from Greenbook Research Industry Trends Report 2007 Pg 76

[4] Resonate Report on Fraud Detection