Screen-Out Management

About Screen-Out Management

What happens when a respondent doesn’t meet your qualifications for completing the survey? By default, all responses that are submitted, regardless of which questions are answered, are saved and considered “Complete”. All collected responses will count toward any response limit on your license.

In addition to the default behavior, there are three alternative methods to handle screened out respondents:

• Save all responses, but flag those that are screened out with Embedded Data for easier identification and filtering in the results.
• Discard screened out responses so they are not saved and do not count toward any response limit on your license.
• Discard screened out responses, but keep a tally of how many respondents screen out at different points in the survey (also known as “Incidence Reporting”, this method is an add-on feature not included in the standard Qualtrics license).

This page further details the default behavior for screened out responses and outlines these methods of managing screen outs in the Survey Flow.

Recording Screened Out Responses

By default, Qualtrics records any respondent who makes it to an ending point in the survey as a complete response. This includes respondents who answer all questions, as well as those who answer a few questions and then are sent to the end of the survey with Skip Logic, Display Logic, or Branch Logic. All these responses are included in the results and in the downloaded dataset, and are considered “complete” responses. See more on what would not be considered a “complete” response on the Partial Completion page.

The number of responses indicated above (58) refers to any respondent who made it to an ending point in the survey.

In the Finished column of the downloaded survey data, “1” refers to any respondent who made it to an ending point in the survey. Columns F, G, and H are hidden to make the screenshot more readable.

Screening Out Respondents in the Survey Flow

Methods for managing screened out responses—such as flagging, deleting, and tallying screen outs—are all handled in the Survey Flow. To use any of these methods, you’ll first need to screen respondents out using the Survey Flow. This process consists of two steps:

1. Skipping respondents ahead to the end of the question block they are currently answering
2. Using a Branch in the Survey Flow to end the survey for those respondents.

To skip respondents to the end of a question block

1. Navigate to the Survey tab and select the question that will screen out respondents.
   
2. Open the Question Options menu and choose Add Skip Logic.
   
3. Set up logic in the first two dropdown menus to reflect those who should be screened out from the survey.
   
4. Skip those who meet the criteria to End of Block and click Done.

Branching Respondents Out of the Survey

Now that screened out respondents have been skipped past the questions they no longer qualify for, we can use the Survey Flow to screen them out of the survey.

1. Navigate to your Survey Flow in the Survey tab.
   
2. Click Add Below on the question block that contains the screening questions.
   
3. Choose Branch and set up logic to capture those who should be screened out of the survey.
   
4. Click Add a New Element Here underneath the Branch.
   
5. Choose End of Survey.
   

With the steps above, respondents are now screened out in the Survey Flow rather than with just Skip Logic. This method opens up the options to flag, delete, or tally these responses. See the following sections for more information on how to implement these options.

Flagging Screened Out Responses

One common scenario for handling screened out responses is to save them in the dataset and flag them with Embedded Data for easy identification and quick data filtering.

Embedded Data is any information you choose to save with survey responses in addition to the answers to the survey questions. In this case, you might save an Embedded Data field with the name “Response Type” and set it equal to “Partial”, “Screened Out”, or “Complete”, depending on where the respondent exits the survey.

To flag screened out responses

1. Screen out respondents using the Survey Flow, as described above.
   
2. Click Add a New Element Here below the End of Survey element.
   
3. Choose Embedded Data.
   
4. Click anywhere on the Embedded Data element and press the ↑ key to move it directly above the End of Survey element.
   
5. Type a name for the Embedded Data field and click Set a Value Now.
   
6. Type a value for the Embedded Data field.
   

In the results, Embedded Data is reported along with the survey questions. In the example above, a field called “Response Type” will be included in the data, with entries of “Screened Out” for respondents who completed the survey this way.

Embedded Data can also be placed at other points in the Survey Flow to track other places where respondents leave the survey.

Example of flagging responses

Let’s say we want to assign values of Partial, Complete, and Screened Out to our respondents. Our Survey Flow might look something like this:

All respondents at the start of the survey are assigned a “Response Type” of “Partial”. Those who are screened out after the screener questions have “Response Type” changed from “Partial” to “Screened Out”. Those who either finish the entire survey have “Response Type” changed from “Partial” to “Complete”. Those who do not finish the survey will remain with a “Response Type” of “Partial”.

The last step is to make sure you change your Embedded Data type to Multi-Value Text Set.

With the above Survey Flow, a table such as the following visualization could be included in your Reports, outlining how many respondents fall into each of the three categories.

Discarding Screened Out Responses

In some situations, you may not want to save the responses of those who are screened out of the survey. Discarding these responses ensures they do not count toward any response limits on your license.

To discard screened out responses

1. Screen out respondents using the Survey Flow, as described above.
   
2. Click Customize on the End of Survey element.
   
3. Enable Override Survey Options.
   
4. Select Do NOT record survey response.
   
5. Click OK and save the flow.
   

Tallying Screened Out Responses

Tallying represents the best of both worlds—screened out responses are not saved in the results, helping to limit the total response count needed for each project, and a tally is provided that identifies how many respondents screened out at each point.

This option, known as Incidence Reporting, is not included in the standard Qualtrics license. You can check if it is available for your account by following the steps below. If you have access to the “Screen-out Response” option in step 3, your account has Incidence Reporting enabled.

To tally screened out responses

1. Screen out respondents using the Survey Flow, as described above.
   
2. Click Customize on the End of Survey element.
   
3. Enable Override Survey Options.
   
4. Select Screen-out Response and type a name for the screen-out. This name will help you analyze the screened out responses.
   
5. Click OK and save the flow.
   

Answers for respondents finishing the survey at this point will not be saved, but a tally of these respondents will be kept.

The steps above show the creation of one general screen-out point to capture all respondents who exit the survey. To tally why each respondent screened out, follow the steps in Screening Out Respondents in the Survey Flow and the steps above, but create a separate Branch for each screen out condition, each with a different screen out name.

To view your screen-out counts

1. Open your Survey Flow.
   
2. Click Customize on any End of Survey element.
   
3. Click Show Screen-out Counts.
   
4. View your tallies for each screen-out category.
   
5. Click Close when finished.
   

Using Screen Outs with the Qualtrics Mailer

If you have Screen-out Response enabled and you send your survey through the Qualtrics Mailer, you will be able to identify which contacts were screened out.

1. Navigate to the Emails section in the Distributions tab.
   
2. Click the dropdown arrow to the right of your email distribution.
3. Choose Download History.
4. Check the Status column in the CSV file for “Screened-out” or any of your custom screen-out names.
   

The Distribution History provides a list of all contacts you emailed the survey to, as well as a number of other details. The Status column indicates who was screened out and gives the Screen-out point name.