Skip to main content
Loading...
Skip to article
  • Qualtrics Platform
    Qualtrics Platform
  • Customer Journey Optimizer
    Customer Journey Optimizer
  • XM Discover
    XM Discover
  • Qualtrics Social Connect
    Qualtrics Social Connect

Fraud Detection


Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Fraud Detection

Some of the biggest threats to data quality are bots and cheaters. Often bots will complete surveys en masse, or a person will take surveys on behalf of someone else multiple times. Thankfully, Expert Review tracks these patterns, so you can ensure your surveys are collecting only the highest quality of data.

With Expert Review Fraud Detection, you can do the following with responses identified as fraudulent:

  • Discard them, preventing them from being counted against auditable responses or quotas.
  • Redirect these responses for analysis separately.
  • Flag these responses so they can be filtered, reported on, etc.
  • Analyze the number of fraudulent responses you’ve received, breaking them down by duplicates and bots.

You can also prevent email scanning software from inadvertently submitting a survey response to begin with.

You can enable Prevent multiple submissions, Bot detection, Security scan monitor, and RelevantID all in the same survey.

Enabling Fraud Detection for a User

Attention: Fraud Detection is not available for all licenses. Please have a Brand Administrator contact your Account Executive or XM Success Manager if you are interested in this feature.

Before you can enable any of the individual fraud detection settings in your project, you need to have access to the features.

Once Fraud Detection is added to your license, a Brand Administrator can adjust the following permissions for a user, user type, or group:

  • RelevantID
  • Bot Detection (security scan monitor)
  • Enhanced Ballot Box Stuffing (prevent multiple submissions)
Qtip: Fraud Detection is only available in surveys, conjoint, MaxDiff, and in most Employee Experience projects. Fraud Detection is not available in 360.

Prevent Multiple Submissions

Attention: In order to accurately capture fraudulent responses, you must have Prevent multiple submissions enabled before you collect data! (And if you are done customizing your survey, don’t forget to publish.)

When enabled in your survey options, Prevent multiple submissions allows you to detect duplicates and keep them from making their way through the survey. During the first survey session, Prevent multiple submissions places a cookie on the browser. If the same respondent comes back on the same browser and device, without having cleared their cookies, they are flagged as a duplicate.

Qtip: This feature used to be called “prevent ballot box stuffing” in the old survey options.

Enabling Prevent Multiple Submissions

  1. Go to Survey options.
    Survey options is opened. Prevent multiple submissions has a dropdown with the options described in the steps below
  2. Go to Security.
  3. Enable Prevent multiple submissions.
  4. Choose one of the following actions:
    • End survey with a message: Once a respondent is determined to have already taken the survey, the respondent is sent out of the survey and we display an end of survey message. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
    • Redirect to URL: Once a respondent is determined to have already taken the survey, immediately redirect them out of the survey to a URL. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
      Qtip: This feature is not available for free accounts.
    • Flag responses: Duplicate respondents will not be sent out of the survey, and will be allowed to finish their response. However, their responses will be assigned a value under the field Q_BallotBoxStuffing.
  5. If you selected End survey with a message, you’ll be asked whether you want to use the default end of survey message, or select a custom one. You can write a new message or use one saved in your library in the past.
    Setting a custom message
  6. If you selected Redirect to URL, provide the full URL of the website you’d like to redirect respondents to.
    Adding a custom URL. In the upper-right, you can see the publish button
  7. After your survey is completed and before you are ready to distribute to respondents, click Publish.
    Qtip: Save this step for when you are done setting up your survey!

Q_BallotBoxStuffing

When you select Continue survey and set embedded data field, respondents will not be sent out of the survey, but their response will be recorded and be assigned a value under the field Q_BallotBoxStuffing. Here’s how you interpret this field:

Embedded Data Field Name Min Value Max Value Interpretation
Q_BallotBoxStuffing NULL (0) True (1) If true (1), it means the response is likely a duplicate.

You can report on or analyze Q_BallotBoxStuffing without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.

Bot Detection

Bot detection allows you to track which responses are likely bots by adding a field to each response called Q_RecaptchaScore. Every response is rated on the probability that the respondent was a bot, which you can then use to filter out data or build reports.

Enabling Bot Detection

Attention: In order to accurately determine the likelihood a response came from a bot, you must have your survey options configured before you collect data!

Bot detection in the survey options

  1. Go to Survey options.
  2. Go to Security.
  3. Enable Bot Detection.
  4. After your survey is completed and before you are ready to distribute to respondents, click Publish.
    Qtip: Save this step for when you are done setting up your survey!

Q_RecaptchaScore

When you enable bot detection, this activates the field Q_RecaptchaScore, which can be used in reporting to indicate whether a response is more likely a bot or a human. This field uses Google’s invisible reCaptcha technology.

Embedded Data Field Name Source Technology Survey Option Enabled Min Value Max Value Interpretation
Q_RecaptchaScore Google’s invisible reCaptcha Bot Detection 0 1 A score of greater than or equal to 0.5 means the respondent is likely a human. A score of Less than 0.5 means the respondent is likely a bot.
Attention: The survey needs to be open in your respondent’s web browser for a few seconds for data to be collected for this field. If your survey immediately terminates, then this field will have no recorded data.

You can report on or analyze Q_RecaptchaScore without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.

Qtip: Although the Captcha question and bot detection both use Google reCAPTCHA technology, how they use it is very different. The Captcha question requires the respondent to successfully interact with it and complete the proposed challenge in order to proceed. On the other hand, bot detection uses reCAPTCHA V3 to flag the response with the likelihood of it being taken by a bot. This requires no interaction from the respondent, but will not block them from proceeding if they are indeed a bot. (Although a user can add a survey flow branch to kick the respondent out of the survey based on the likelihood of them being a bot.)

 

Security Scan Monitor

When enabled in your survey options, security scan monitor allows you to prevent email scanning software from inadvertently starting a survey session when a survey link is included in the email. While this feature is designed to stop email scanning software from opening emailed links, it will apply to links of any type, regardless if they were distributed via Qualtrics or a third-party system.

Qtip: You must have Fraud Detection included in your license to use security scan monitor.
Qtip: This feature used to be called “email scan roadblock” in the old survey options.

How Security Scan Monitor Works

Bot detection is used to determine whether or not a roadblock should be placed at the start of a survey, preventing the email scanning software from starting a survey session. If a bot is detected, a start page will be shown instead of the first question in the survey. If a bot was incorrectly detected, the customer can click the “next” button and be able to continue taking the survey. The look and feel of the survey start page will be consistent with what has been selected from the rest of the survey.

the message when a bot is detected, which reads "tap the button to continue the survey" with a next button present

Enabling Security Scan Monitor

Security scan monitor option

  1. Go to Survey options.
  2. Go to Security.
  3. Enable Security Scan Monitor.
  4. After your survey is completed and before you are ready to distribute to respondents, click Publish.
Qtip: Although the Captcha question and security scan monitor both use Google reCAPTCHA technology, how they use it is very different. The Captcha question requires the respondent to successfully interact with it and complete the proposed challenge in order to proceed. On the other hand, security scan monitor uses reCAPTCHA V3 to identify when the survey is being tested by email security scanners. When using the security scan monitor feature, “Protected by reCAPTCHA” will appear at the bottom of your survey.

 

RelevantID

RelevantID improves fraud detection by assessing respondent metadata to determine the likelihood that the same respondent is answering over and over; RelevantID does not necessarily check the content of the responses for duplicates, since respondent can answer multiple times while giving different answers. This feature has four fields associated with it that you can report on: Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, and Q_RelevantIDLastStartDate. These fields are calculated using RelevantID technology. This technology checks if the respondent is cheating by taking the survey multiple times or whether a survey taker is fraudulent by analyzing a user’s browser, operating system, and location to provide a fraud score. See RelevantID‘s site for more details.

Enabling RelevantID

Attention: In order to accurately determine the likelihood a response is fraudulent, you must have your Survey Options saved before you collect data!

Relavant ID in the survey options

  1. Go to Survey options.
  2. Go to Security.
  3. Enable RelevantID.
  4. Set the timeframe during which RelevantID resets and will not flag a response that seems to be a duplicate from a previous session. Your options are 1 week, 30 days, 3 months, 6 months, or 1 year.
    Qtip: This setting is helpful if you need to let the same respondents retake a survey after a period of time. This way, if you are flagging and possibly screening out duplicate respondents with RelevantID, you do not have to change your Survey Flow setup when it comes time for the respondents to retake the survey.
  5. After your survey is completed and before you are ready to distribute to respondents, click Publish.
    Qtip: Save this step for when you are done setting up your survey!

Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, and Q_RelevantIDLastStartDate

Embedded Data Field Name Source Technology Survey Option Enabled Min Value Max Value Interpretation
Q_RelevantIDDuplicate RelevantID RelevantID NULL (0) True (1) If true (1), it means the response is likely a duplicate.
Q_RelevantIDDuplicateScore RelevantID RelevantID 0 100 A score of greater than or equal to 75 means the response is likely a duplicate.
Q_RelevantIDFraudScore RelevantID RelevantID 0 130 A score greater than or equal to 30 means the response is likely fraudulent and a bot.
Q_RelevantIDLastStartDate RelevantID RelevantID N/A N/A If Q_RelevantIDDuplicate is true (1), this field will report the last date the survey was started.
Attention: The survey needs to be open in your respondent’s web browser for a few seconds for data to be collected for these fields. If your survey immediately terminates, then these fields will have no recorded data.

You can report on or analyze these fields without adding them to the survey flow. However, in order to set up logic based on these fields (such as screening out fraudulent responses), you must add this embedded data field to your survey flow.

Adding Fraudulent Detection Fields to the Survey Flow

You can report on or analyze Fraud Detection fields like Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, Q_RecaptchaScore, and Q_BallotBoxStuffing simply by having the corresponding survey option enabled, without adding them to the survey flow.

However, in order to set up logic based on these fields (such as screening out possible bots or fraudulent responses), you must add these embedded data fields to your survey flow.

  1. In the Survey options, turn on Prevent multiple submissions (Flag responses), Bot detection, and / or RelevantID.
    Survey options, security section, wiht the settings described enabled
  2. Go to the Survey flow.
    Adding an embedded data element
  3. Click Add a New Element Here.
  4. Select Embedded Data.
  5. Select the name of the embedded data, click the blue dropdown, highlight Survey Metadata, and select the desired fraudulent response metrics.
    Selecting fields

    Qtip: Q_BallotBoxStuffing is under “Ballot Box Stuffing.”
  6. Move the embedded data to the top of the survey flow.
    Finished
  7. Click Apply.

You can now build your desired branch logic.

Example: Use these embedded data fields and the values below to redirect fraudulent respondents using branch logic. Note how the block appears before the branch logic.
image of using the fraud detection fields to branch out fraudulent responses