Fraud Detection
About Fraud Detection
Some of the biggest threats to data quality are bots and cheaters. Often bots will complete surveys en masse, or a person will take surveys on behalf of someone else multiple times. Thankfully, Expert Review tracks these patterns, so you can ensure your surveys are collecting only the highest quality of data.
With Expert Review Fraud Detection, you can do the following with responses identified as fraudulent:
- Discard them, preventing them from being counted against auditable responses or quotas.
- Redirect these responses for analysis separately.
- Flag these responses so they can be filtered, reported on, etc.
- Analyze the number of fraudulent responses you’ve received, breaking them down by duplicates and bots.
You can also prevent email scanning software from inadvertently submitting a survey response to begin with.
You can enable “prevent multiple submissions,” bot detection, security scan monitor, and duplicate detection all in the same survey.
Enabling Fraud Detection
Once Fraud Detection is added to your license, users can decide whether to enable all or some fraud detection settings for each of their surveys.
The only way to limit user access to fraud detection is by using division permissions. Those permissions are in the General section, and they’re called:
- Bot Detection
- Enhanced Ballot Box Stuffing (prevent multiple submissions fraud detection features)
See this linked page on division permissions for more details. Duplicate detection and security scan monitor cannot be controlled by division permissions.
Prevent Multiple Submissions
When enabled in your survey options, Prevent multiple submissions allows you to detect duplicates and keep them from making their way through the survey. During the first survey session, Prevent multiple submissions places a cookie on the browser. If the same respondent comes back on the same browser and device, without having cleared their cookies, they are flagged as a duplicate.
Enabling Prevent Multiple Submissions
- Go to Survey options.
- Go to Security.
- Enable In Survey: Prevent multiple submissions.
- Choose one of the following actions:
- End survey with a message: Once a respondent is determined to have already taken the survey, the respondent is sent out of the survey and we display an end of survey message. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
- Redirect to URL: Once a respondent is determined to have already taken the survey, immediately redirect them out of the survey to a URL. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
Qtip: This feature is not available for free accounts.
- Flag responses: Duplicate respondents will not be sent out of the survey, and will be allowed to finish their response. However, their responses will be assigned a value under the field Q_BallotBoxStuffing.
- If you selected End survey with a message, you’ll be asked whether you want to use the default end of survey message, or select a custom one. You can write a new message or use one saved in your library in the past.
- If you selected Redirect to URL, provide the full URL of the website you’d like to redirect respondents to.
- After your survey is completed and before you are ready to distribute to respondents, click Publish.
Qtip: Save this step for when you are done setting up your survey!
Q_BallotBoxStuffing
When you select Continue survey and set embedded data field, respondents will not be sent out of the survey, but their response will be recorded and be assigned a value under the field Q_BallotBoxStuffing. Here’s how you interpret this field:
Embedded Data Field Name | Min Value | Max Value | Interpretation |
Q_BallotBoxStuffing | NULL (0) | True (1) | If true (1), it means the response is likely a duplicate. |
You can report on or analyze Q_BallotBoxStuffing without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.
Bot Detection
Bot detection allows you to track which responses are likely bots by adding a field to each response called Q_RecaptchaScore. Every response is rated on the probability that the respondent was a bot, which you can then use to filter out data or build reports.
Enabling Bot Detection
- Go to Survey options.
- Go to Security.
- Enable Bot Detection.
- After your survey is completed and before you are ready to distribute to respondents, click Publish.
Qtip: Save this step for when you are done setting up your survey!
Q_RecaptchaScore
When you enable bot detection, this activates the field Q_RecaptchaScore, which can be used in reporting to indicate whether a response is more likely a bot or a human. This field uses Google’s invisible reCaptcha technology.
Embedded Data Field Name | Source Technology | Survey Option Enabled | Min Value | Max Value | Interpretation |
Q_RecaptchaScore | Google’s invisible reCaptcha | Bot Detection | 0 | 1 | A score of greater than or equal to 0.5 means the respondent is likely a human. A score of Less than 0.5 means the respondent is likely a bot. |
You can report on or analyze Q_RecaptchaScore without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.
Security Scan Monitor
When enabled in your survey options, security scan monitor allows you to prevent email scanning software from inadvertently starting a survey session when a survey link is included in the email. While this feature is designed to stop email scanning software from opening emailed links, it will apply to links of any type, regardless if they were distributed via Qualtrics or a third-party system.
How Security Scan Monitor Works
Bot detection is used to determine whether or not a roadblock should be placed at the start of a survey, preventing the email scanning software from starting a survey session. If a bot is detected, a start page will be shown instead of the first question in the survey. If a bot was incorrectly detected, the customer can click the “next” button and be able to continue taking the survey. The look and feel of the survey start page will be consistent with what has been selected from the rest of the survey.
Enabling Security Scan Monitor
- Go to Survey options.
- Go to Security.
- Enable Security Scan Monitor.
- After your survey is completed and before you are ready to distribute to respondents, click Publish.
Duplicate Detection
The Duplicate Detection feature analyzes a response after it has been submitted to determine if it’s a duplicate. This option detects duplicates by analyzing device and browser metadata. When enabled, Duplicate Detection will collect the survey respondent’s IP address and certain metadata to identify likely duplicate responses. IP addresses will not be collected or used for Duplicate Detection if you have otherwise disabled their collection.
Enabling duplicate detection
- Go to Survey options.
- Go to Security.
- Enable Post-survey: Prevent multiple submissions with Duplicate Detection.
- Choose one of the following post-survey behaviors:
- Flag response: Record the response, but add the field Q_DuplicateRespondent = true to each duplicate.
- Do NOT record survey response (not recommended): This option deletes the response entirely, preventing it from being recorded against your response limits. Keep in mind that no duplicate detection system is perfect, so if you choose this option, you consent to the small chance that real responses are occasionally deleted. Once a response is screened out this way, it is irretrievable.
- Decide on the duration to track duplicate responses. This determines how long the system should wait before it considers responses coming in from familiar devices and browsers to be new responses, as opposed to duplicates.
Qtip: Options include 1 week, 30 days, 3 months, 6 months, and 1 year.
- Remember to publish your survey when you’re ready to distribute.
Qtip: Save this step for when you are done setting up your survey!
Q_DuplicateRespondent
If you record duplicates instead of deleting them, an embedded data field named Q_DuplicateRespondent will be added to your responses.
- When a response’s Q_DuplicateRespondent field is true, that means the response was detected as a likely duplicate.
- When the field is empty, it means the response was not flagged as a duplicate.
Replacing RelevantID with Duplicate Detection
In the past, you may have used RelevantID to set up different ways to detect fraudulent responses. For example, you may have used:
- Embedded data
- Workflows
- Branches
- Piped text
- Filters in reports and data
- Dashboards
Because RelevantID embedded data will no longer work the same way, this section covers how to replace this embedded data anywhere in Qualtrics you might be using it.
Fields that No Longer Record Data
All existing RelevantID embedded data fields will stop working, and instead only record a value of 0 (zero), “false,” or “undefined.” This includes:
- Q_RelevantIDDuplicate
- Q_RelevantIDDuplicateScore
- Q_RelevantIDFraudScore
- Q_RelevantIDLastStartDate
Keep in mind that any existing responses that have data for RelevantID will not be affected. Only new responses going forward will have empty data.
Replacing Embedded Data
The new embedded data field is Q_DuplicateRespondent. Its values are:
- true: This response is a duplicate response.
- Blank / empty value: This is not a duplicate response.
Example: Here is a branch dependent on Q_RelevantIDDuplicateScore. This embedded data is also saved at the top of the survey flow.
Because Q_DuplicateRespondent doesn’t work until the response is submitted, you cannot use it in branch logic. Instead, you need to set your survey options to delete duplicate responses, as shown below. This will ensure duplicate responses do not count towards interaction limits.
Example: Here is a workflow dependent on Q_RelevantIDDuplicate.
Here, we can replace the condition with Q_DuplicateRespondent. This works because workflows don’t trigger until after a response is submitted.
Example: Imagine the workflow in the previous example has piped text for different Relevant ID fields.
Below is a corrected email with Q_DuplicateRespondent piped in. Notice how the new embedded data only has true and false, so we have to remove the text where we show a numeric score.
Keep in mind that this change only affects new responses. Responses recorded before RelevantID was deprecated will be tagged with the same information (Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, and Q_RelevantIDLastStartDate).
Example: We have the old embedded data mapped in our dashboard. We don’t delete the old field, but we do add a new field for Q_DuplicateRespondent.
Adding Fraudulent Detection Fields to the Survey Flow
You can report on or analyze Fraud Detection fields like Q_RecaptchaScore and Q_BallotBoxStuffing simply by having the corresponding survey option enabled, without adding them to the survey flow.
However, in order to set up logic based on these fields (such as screening out possible bots or fraudulent responses), you must add these embedded data fields to your survey flow.
- In the Survey options, turn on Prevent multiple submissions (Flag responses) and/or Bot detection.
- Go to the Survey flow.
- Click Add a New Element Here.
- Select Embedded Data.
- Select the name of the embedded data, click the blue dropdown, highlight Survey Metadata, and select the desired fraudulent response metrics.
Qtip: Q_BallotBoxStuffing is under “Ballot Box Stuffing.” - Move the embedded data to the top of the survey flow.
- Click Apply.
You can now build your desired branch logic.
