Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM

Fraud Detection

What's on This Page:


Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


Attention: Fraud Detection is not available for all licenses. Please contact your Account Executive or Customer Success Representative if you are interested in this feature.

About Fraud Detection

Some of the biggest threats to data quality are bots and cheaters. Often bots will complete surveys en masse, or a person will take surveys on behalf of someone else multiple times. Thankfully, ExpertReview tracks these patterns, so you can ensure your surveys are collecting only the highest quality of data.

With ExpertReview Fraud Detection, you can do the following with responses identified as fraudulent:

  1. Discard them, preventing them from being counted against auditable responses or quotas.
  2. Redirect these responses for analysis separately.
  3. Flag these responses so they can be filtered, reported on, etc.
  4. Analyze the number of fraudulent responses you’ve received, breaking them down by duplicates and bots.

You can also prevent email scanning software from inadvertently submitting a survey response to begin with.

You can enable Prevent Ballot Box Stuffing, Bot Detection, Email Scan Roadblock, and RelevantID all in the same survey.

Prevent Ballot Box Stuffing

Attention: In order to accurately capture fraudulent responses, you must have Prevent Ballot Box Stuffing enabled before you collect data! (And if you are done customizing your survey, don’t forget to publish.)

When enabled in your Survey Options, Prevent Ballot Box Stuffing allows you to detect duplicates and keep them from making their way through the survey. During the first survey session, Prevent Ballot Box Stuffing places a cookie on the browser. If the same respondent comes back on the same browser and device, without having cleared their cookies, they are flagged as a duplicate.

Prevent ballot box stuffing sub options

Once enabled, you can choose one of Prevent Ballot Box Stuffing’s four modes:

  • End survey and display default message: Once a respondent is determined to have already taken the survey, the respondent is sent out of the survey and we display the standard end of survey message. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
  • End survey and display custom message: Once a respondent is determined to have already taken the survey, send the respondent out of the survey and display a special end of survey message. You can either write this message by clicking Load a Saved Message, your Library, and then New Message, or you can use an End of Survey message you saved in your library in the past. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
  • Redirect to a full URL: Once a respondent is determined to have already taken the survey, immediately redirect them out of the survey to a URL. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
    Qtip: This feature is not available for free accounts.
  • Continue Survey and set embedded data field: Duplicate respondents will not be sent out of the survey, but their responses will be recorded and be assigned a value under the field Q_BallotBoxStuffing.

Q_BallotBoxStuffing

When you select Continue survey and set embedded data field, respondents will not be sent out of the survey, but their response will be recorded and be assigned a value under the field Q_BallotBoxStuffing. Here’s how you interpret this field:

Embedded Data Field Name Min Value Max Value Interpretation
Q_BallotBoxStuffing NULL (0) True (1) If true (1), it means the response is likely a duplicate.

You can report on or analyze Q_BallotBoxStuffing without adding it to the Survey Flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this Embedded Data field to your Survey Flow.

Bot Detection

Bot Detection allows you to track which responses are likely bots by adding a field to each response called Q_RecaptchaScore. Every response is rated on the probability that the respondent was a bot, which you can then use to filter out data or build reports.

Enabling Bot Detection

Attention: In order to accurately determine the likelihood a response came from a bot, you must have your Survey Options saved before you collect data!
  1. Go to Survey Options.
    Survey options from toolbar along top of survey project
  2. Select Bot Detection.
    Bot detection option in the survey protection section
  3. Click Save.
  4. After your survey is completed and before you are ready to distribute to respondents, click Publish.
    Green publish button upper-right back on the survey editor

    Qtip: Save this step for when you are done setting up your survey!

Q_RecaptchaScore

When you enable Bot Detection, this activates the field Q_RecaptchaScore, which can be used in reporting to indicate whether a response is more likely a bot or a human. This field uses Google’s invisible reCaptcha technology.

Embedded Data Field Name Source Technology Survey Option Enabled Min Value Max Value Interpretation
Q_RecaptchaScore Google’s invisible reCaptcha Bot Detection 0 1 A score of greater than or equal to 0.5 means the respondent is likely a human. A score of Less than 0.5 means the respondent is likely a bot.
Attention: The survey needs to be open in your respondent’s web browser for a few seconds for data to be collected for this field. If your survey immediately terminates, then this field will have no recorded data.

You can report on or analyze Q_RecaptchaScore without adding it to the Survey Flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this Embedded Data field to your Survey Flow.

Qtip: Although the Captcha question and Bot Detection both use Google reCAPTCHA technology, how they use it is very different. The Captcha question requires the respondent to successfully interact with it and complete the proposed challenge in order to proceed. On the other hand, Bot Detection uses reCAPTCHA V3 to flag the response with the likelihood of it being taken by a bot. This requires no interaction from the respondent, but will not block them from proceeding if they are indeed a bot. (Although a user can add a Survey Flow branch to kick the respondent out of the survey based on the likelihood of them being a bot.)

 

Email Scan Roadblock

When enabled in your Survey Options, Email Scan Roadblock allows you to prevent email scanning software from inadvertently starting a survey session when a survey link is included in the email. While this feature is designed to stop email scanning software from opening emailed links, it will apply to links of any type, regardless if they were distributed via Qualtrics or a third-party system.

Qtip: You must have Fraud Detection included in your license to use Email Scan Roadblock.

How Email Scan Roadblock Works

Bot Detection is used to determine whether or not a roadblock should be placed at the start of a survey, preventing the email scanning software from starting a survey session. If a bot is detected, a start page will be shown instead of the first question in the survey. If a bot was incorrectly detected, the customer can click the Next button and be able to continue taking the survey. The Look and Feel of the survey start page will be consistent with what has been selected from the rest of the survey.

the message when a bot is detected, which reads "tap the button to continue the survey" with a next button present

Enabling Email Scan Roadblock

  1. Go to Survey Options.
  2. Select Email Scan Roadblock.
    Email scan roadblock option in survey options
  3. Click Save.
  4. After your survey is completed and before you are ready to distribute to respondents, click Publish.
Qtip: Although the Captcha question and Email Scan Roadblock both use Google reCAPTCHA technology, how they use it is very different. The Captcha question requires the respondent to successfully interact with it and complete the proposed challenge in order to proceed. On the other hand, Email Scan Roadblock uses reCAPTCHA V3 to identify when the survey is being tested by email security scanners. When using the Email Scan Roadblock feature, “Protected by reCAPTCHA” will appear at the bottom of your survey.

 

RelevantID

Relevant ID improves fraud detection by assessing respondent metadata to determine the likelihood that the same respondent is answering over and over; Relevant ID does not necessarily check the content of the responses for duplicates, since respondent can answer multiple times while giving different answers. This feature has three fields associated with it that you can report on: Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, and Q_RelevantIDFraudScore. These fields are calculated using RelevantID technology. This technology checks if the respondent is cheating by taking the survey multiple times or whether a survey taker is fraudulent by analyzing a user’s browser, operating system, and location to provide a fraud score.

Enabling RelevantID

Attention: In order to accurately determine the likelihood a response is fraudulent, you must have your Survey Options saved before you collect data!
  1. Go to Survey Options.
    Survey options from toolbar along top of survey project
  2. Select RelevantID.
    Relevant ID option in the survey protection section of Survey Options window. When selected, another dropdown appears letting you select the timeframe
  3. Set the timeframe during which Relevant ID resets and will not flag a response that seems to be a duplicate from a previous session. Your options are 1 week, 30 days, 3 months, 6 months, or 1 year.
    Qtip: This setting is helpful if you need to let the same respondents retake a survey after a period of time. This way, if you are flagging and possibly screening out duplicate respondents with Relevant ID, you do not have to change your Survey Flow setup when it comes time for the respondents to retake the survey.
  4. Click Save.
  5. After your survey is completed and before you are ready to distribute to respondents, click Publish.
    Green publish button upper-right back on the survey editor

    Qtip: Save this step for when you are done setting up your survey!

Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, and Q_RelevantIDFraudScore

Embedded Data Field Name Source Technology Survey Option Enabled Min Value Max Value Interpretation
Q_RelevantIDDuplicate RelevantID RelevantID NULL (0) True (1) If true (1), it means the response is likely a duplicate.
Q_RelevantIDDuplicateScore RelevantID RelevantID 0 100 A score of greater than or equal to 75 means the response is likely a duplicate.
Q_RelevantIDFraudScore RelevantID RelevantID 0 130 A score greater than or equal to 30 means the response is likely fraudulent and a bot.
Attention: The survey needs to be open in your respondent’s web browser for a few seconds for data to be collected for these fields. If your survey immediately terminates, then these fields will have no recorded data.

You can report on or analyze these fields without adding them to the Survey Flow. However, in order to set up logic based on these fields (such as screening out fraudulent responses), you must add this Embedded Data field to your Survey Flow.

Adding Fraudulent Detection Fields to the Survey Flow

You can report on or analyze Fraud Detection fields like Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, Q_RecaptchaScore, and Q_BallotBoxStuffing simply by having the correspondingSurvey Option enabled, without adding them to the Survey Flow.

However, in order to set up logic based on these fields (such as screening out possible bots or fraudulent responses), you must add these Embedded Data fields to your Survey Flow.

  1. In the Survey Options, turn on Prevent Ballot Box Stuffing (Continue Survey and set embedded data field), Bot Detection, and / or RelevantID.
    In toolbar, survey flow then survey options button
  2. Once your Survey Options are saved, go to the Survey Flow.
  3. Click Add a New Element Here.
    Adding an embedded data element
  4. Select Embedded Data.
  5. Select the name of the Embedded Data, click the blue dropdown, highlight Survey Metadata, and select the desired Fraudulent Response Metrics.
    Selecting fields
  6. Move the Embedded Data to the top of the Survey Flow.
    Finished
  7. Click Save Flow.
Qtip: Use these Embedded Data fields and the values below to redirect fraudulent respondents using Branch Logic.
image of using the fraud detection fields to branch out fraudulent responses