Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM
  • Design XM
    Design XM

Common Survey Errors

What's on This Page:


Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Expert Review

Expert Review is a feature in the Qualtrics platform that checks the overall quality of your survey before you distribute it. It will make suggestions as to how best to improve your survey to ensure that you are collecting the highest quality data possible. Below are possible error messages that you may receive via Expert Review and ways that you can resolve each one.

Invalid Display Logic

If Expert Review has indicated the use of “invalid display logic,” your display logic may be referencing a question later in the survey, a deleted question, or another deleted item.

Below are some common ways people set up display logic incorrectly, and how to correct this. For a guide on creating display logic from scratch, visit the Display Logic support page.

Order of Logic

The order of your questions is very important when setting up display logic. Display logic can only hide or reveal questions based on previously answered questions. Another way to think of it is this: display logic cannot look into the future and tell what respondents will say to a question they haven’t answered yet. Even if the question is on the same page, the order is important.

Example: In Q1, the survey asks if the respondent has kids. If they say yes, then Q2, asking how many kids they have, will appear if in page display logic is enabled. If in page display logic is not enabled, then Q2 will not appear until the respondent goes to the next page of the survey.
Example display logic
Example: Q1 asks how many kids the respondent has. But because of the display logic on the question and the order the questions appear in, respondents will only see Q1 if in page display logic is enabled. Using in page display logic this way should be avoided since display logic should be used to display follow-up questions, not preceding questions.
Order of questions for display logic

Deleted Questions

Invalid Logic appears on your question when you’ve deleted the question being used in the display logic.

Example: Q1, asking if the respondent has kids, has been deleted. Q2, asking how many kids the respondent has, now has an Invalid Logic message on it.
Invalid Logic error message

If you try to delete a question being used in display logic, Qualtrics will warn you before letting you proceed.
Warning before deleting question used in logic

If you do delete an important question, you may still be able to retrieve it from your trash. Scroll to the very bottom of the survey, expand the trash, click the question, and click Restore in the upper-left. If this is the correct question, it will automatically fix the display logic.
Restoring a question from the trash

If the question is not in the Trash, or if the Invalid Logic error remains, you will have to rewrite the display logic. Once you’ve created your new question or decided on a different question you want to base the logic off, open the dropdown and select Edit.
Editing invalid logic

Deleted Quotas

Qualtrics will not tell you if you’ve deleted a quota being used in logic. So first, check to see if there’s a match between what the display logic says and what’s on your quotas page. (Quotas can be found in the survey options.)
Quota being used in logic

If you are still having troubles with the display logic, click the dropdown arrow on the blue banner and select Edit. The field that should display the quota’s name will be empty if the original quota has been deleted. You will have to recreate the quota and rewrite the display logic.

Example: The Provo Citizens quota has been deleted, so when you go to edit the quota, it says to Select a quota instead of saying Provo Citizens.
What logic looks like when a quota has been deleted
What logic looks like when a quota has been deleted

Deleted Embedded Data

If you delete embedded data from your survey flow, the system won’t warn you about connected display logic. Make sure that the embedded data used in your display logic has a match in the survey flow. Once this is done, you do not have to edit the display logic, so long as the spelling, punctuation, spacing, and capitalization is the same. For more on adding embedded data to the survey flow, see the linked support page.
What logic looks like if Embedded Data has been deleted

Qtip: So long as there are values set in the contact list, you can add the embedded data back to the survey flow at any time to preserve your data and your display logic. However, if you set embedded data values in branches after collecting data, this will only record data for future responses, not retroactively.

Display Logic in Branches

While you can use branching and display logic together, you will need to be careful that branching does not unintentionally invalidate your display logic. It is possible to create display logic based on a question that is only shown in one survey path. Any respondents who do not go down this survey path will not see the subsequent questions with the display logic. Expert Review will flag these cases for you to review before publishing your survey. Additionally, if you have multiple branches with the same condition, Expert Review will flag the latter branches. This is due to the fact that the logic parameter can change in between branches (for example, if an embedded data value is set and then overwritten in between branches and the Embedded Data field is used in the logic).

Example: Q1 asks respondents their home region and then branches them accordingly.
Image of blocks branched by region with follow up question block
Q2 asks respondents if they like to ski, but this question is only in one region’s block. Later on, Q3 has display logic set to display if respondents answer “Yes” to Q2. Since Q2 is located in just one region’s block, only respondents from that block will see Q3 even though this question was meant to be asked of all respondents.
Image of two region blocks and follow up question block in survey editor

Invalid Piped Text

If Expert Review has indicated the use of “invalid piped text,” then your piped text may be referencing a question on the same page, a question later in the survey, a deleted question, or another deleted item.

Here are some common ways people set up piped text incorrectly, and how to correct this. For a guide on adding piped text from scratch, see the linked support page.

Order of Piped Text

If you are piping one answer or value into another part of the survey, you must make sure that the following conditions are met:

  1. The answer being piped in and the place where it’s being piped are on separate pages of the survey.
  2. The answer being piped in is asked for before it’s being displayed.

What this means is that Qualtrics can’t pipe in an answer until it knows what it is; this includes submitting the answer so the survey has officially saved it and can use it on future pages. So when piping text in, you want to do the following:

  1. Create the question whose answer will be displayed elsewhere in the survey.
  2. Add a page break. This can be done with page breaks, blocks, the look and feel settings, or skip logic that happens in between the questions.
  3. Create the question or descriptive text where the answer will be piped.
Example: In the image below, the piped text in Q6 will display the answer from Q5 because there’s a page break between the two questions.
Page break between piped text
Example: In the example below, the piped text in Q6 will display the answer from Q5 if In Page was not selected when setting up the display logic. If In Page was selected, however, then the piped text will be blank since Q5 and Q6 will display on the same page.
Piped text with logic
Qtip: Display logic automatically creates page breaks, but in page display logic doesn’t. If you are piping answers into questions with display logic added to them, then In Page should not be selected.
Qtip: Since page breaks are incompatible with question randomization, you should not use piped text on questions within the same randomized block. However, if the questions are separated into different blocks, even if one is randomized, the page break between the blocks will ensure the piped text displays correctly.

Deleted Items

Is your piped text not displaying a value? It’s possible you’ve deleted the item it’s trying to display data from. Here are a few ways to see if this has happened, in addition to solutions for these issues:

  • Every question has an internal QID, which is used in the piped text. Copy this QID and search for it in the search bar to the upper-right. The question that appears either has this QID (or has this piped text pasted into the question/choice). If the search results are blank, you’ve deleted the question.
    Internal ID indicated in Piped Text
    Searching by an internal ID
  • Quotas get an ID, too, but this is harder to search. Go to the quotas page and see if a quota of the same name exists. If it does, generate new piped text for this quota. If the ID that starts with QO_ and is followed by a string of letters and numbers matches between the new and old piped text, the quota has not been removed.
    Quota ID in piped text
  • So long as there are values set in the contact list, you can add the embedded data back to the survey flow anytime to ensure future respondents have their embedded data displayed to them. However, if you are using branch logic to set embedded data values, you may want to make sure your branches are set up correctly, as this may be the reason your piped text isn’t displaying.

End of Survey Scoring Not Set Up

This message means you are trying to display the score at the end of the survey, but there’s no scoring on your survey.

Scoring categories allow you to score your respondents on a group of related questions. For example, let’s say there are 18 questions on biology and 10 questions on chemistry in your survey. Your 18 biology questions can be added together in the Biology scoring category for a total of 18 points max, separate from the Chemistry category. In addition, you can add the same questions to different categories, accounting for overlap.

You can navigate to Scoring by going to Survey options and selecting Scoring.
Where scoring can be accessed

On this page, you will be able to assign scored values to scoring categories. See the linked page for more details on how to add scores to questions.

Removing Score Reports

Of course, there’s a chance you don’t want to add scoring to your survey at all, or if you do, you don’t want respondents to know their score. Follow the directions to remove the score report.

  1. Navigate to Scoring by going to Survey options and selecting Scoring.
    Where scoring can be accessed
  2. Go to Scoring Options.
    Scoring Options in the scoring menu
  3. Make sure that next to Show Scoring Summary for Category, nothing is selected.
    Options for displaying a scoring summary
  4. Click Save.

Invalid HTML

Expert Review may identify an issue with the HTML in your survey. This check will validate the HTML to ensure that it will render. Expert Review should highlight the specific line of HTML that is causing issues.

This section will advise on two things: how to access the HTML editor, and how to remove HTML so you can start from scratch, without losing your plain text.

Qtip: Unfortunately, Qualtrics Support cannot provide assistance with custom coding. However, you can always consult the Developer Corner on the Qualtrics Community if you have any questions about your HTML.

Viewing HTML

Click on a question as if to edit it. Click HTML View on the right.
HTML view option

Also, if you are in the rich content editor (in questions or answer choices), you can select the Source button to see the HTML.
Source code option in the Rich Content Editor

Removing HTML From One Question

Click into the text of a question like you’re going to edit it. Select Remove Formatting.
Remove Formatting option in the question

In this example, the font and any other HTML that is not visible to us will be removed, but the text, “Do you have kids?” will remain.

Removing HTML From Multiple Questions At Once

Click the checkbox next to each question you want to select. Then go to Tools, highlight Review, and select Strip Formatting.
Strip Formatting option for multiple questions

Qtip: You can also select multiple questions by doing the following: Click the first question you want to select. Scroll to the last question, but don’t click it yet. Press Shift on your keyboard, and while holding it down, click the last question. Everything in between will be highlighted.

Incomplete Translation

If you are still editing a survey or working from a copy of an older survey, there may be times that any translations you’ve added are incomplete. Expert Review should identify the questions missing translations.

This section will discuss how to add missing translations and how to remove unwanted translations. Navigate to translations by going to Survey options and Translations.
In the survey options, lower-left, "Translations" is the last thing listed

Adding a Translation to a Question

Use the arrows to navigate between questions. The circle containing a percentage of completion will be red when a question isn’t translated, or green when it is completely translated.
Circular red icon that says 0 percent above an untranslated question on the right

  • Adding your own translations: Click into a question or answer choice text on the right side to enter the text for the translation.
  • Automatic translations with Google: Click the red circle. Qualtrics will ask if you’re sure before applying Google Translate to the text.
  • Selecting the language to translate: Click the dropdown next to the question number on the right. In the example above, it’s set to Spanish (Latin America). You can add or remove languages from this dropdown using the checkboxes.

Translating Multiple Fields Using a Spreadsheet

You can also use a CSV or TSV to translate multiple fields at once. See the page on Importing a Translation for more information.

Removing Unwanted Translations

Especially when working with a copy of an older survey, you may not want to use the translations in the survey. See the page on Removing a Translation for instructions.

Timing/Metadata Questions Are Alone on the Page

Sometimes, the way your timing or meta info question has been set up may negatively impact your survey.

Common Mistakes with Timing Questions

If you are trying to time a question or set of questions, or if you are trying to limit the time respondents may spend on the page of a survey, you need to make sure that the timing question is on the same page as the questions you are trying to time and that each group of questions you’re trying to time has its own timing question paired with it. Check the Timing Question page for information on adding page breaks.

If you have questions per page customized in your look and feel and you are using timing questions, do not ever set the default number of pages to 1. Each timing question counts as one question, and will not properly time the question or questions if it appears on a page alone.

Common Mistakes with Meta Info Questions

The meta info question will not record data until the respondent reaches it and submits the page of the survey it’s on. That means if you add a meta info question to the last page of the survey, respondents who completed only part of the survey won’t have any meta info data.

In order to ensure that data for the question is recorded, be sure to place the meta info question in a block with other questions that are displayed! We typically recommend placing it as the first question in your first block of questions. This can help you avoid any logic that may cause the question to be skipped. Since the meta info question is not displayed to the respondents, they will not know the difference.
Options for moving a question

You can move a question to the top in two ways:

  • Clicking, dragging, and dropping the question to the desired location.
  • Open the question editing menu and click Move question. You will then be able to choose the block where you want the question to be moved and specify that it should go to the top of the block.
Example: Here, we put it at the top of the Default Question Block because that’s the first block in the survey.
Window that opens after you use the move question option; asks for a block and a position

More Than One Timing / Meta Info Per Page

There should never be more than one timing question per page. There should also never be more than one meta info question per page. (You can add one of each to the same page, however.)

For guidance on the many ways you can add page breaks, see the Add Page Break support page.

The Last Question in the Survey is Descriptive Text

Often times people will add a text / graphic question to the end of their survey because they think it’s the best way to thank the respondents and let them know they’re done with the survey. However, this is not the case! Because this creates one more page of the survey, the respondent who reaches this page but doesn’t click the next button technically hasn’t submitted their response. This can cause issues with your data and with your completion rates.

Adding an End of Survey Message

Instead of using text / graphic questions, Qualtrics already has a message that appears after the respondent completes the survey. Completion is indicated by clicking the very last next button.

If you would like to add your own message, you can do one of the following.

  • If every user should get the same message when they finish, you can simply scroll to the bottom of the survey builder, select the End of Survey block, and add a custom message to it. See the Changing the End of Survey Message page for detailed steps.
  • If you are using branch logic in your survey flow to send respondents down different paths, you may want to use end of survey elements in the survey flow to add different messages to different paths. On each branch that leads to a special message, add an end of survey element and click Customize to add a new message. Select Override Survey Options and Custom End of Survey Message to create or select a new message.
    Options for customizing the end of survey message

Adding a Submit Button to the Last Page of the Survey

Sometimes you may wish to have a separate message right before the end of your survey and after your survey has concluded. Other times you may just want a button that clearly indicates the respondent is about to submit their survey, without adding additional instruction.

You can add a Submit button to the end of your survey by doing the following:

  1. The first step to adding a submit button is to put the last page of your survey on its own block. First, select the question or question you wish to have on the last page.Options for moving a question
  2. Click Move to a New Block. This will move the content to a new block at the end of the survey.
  3. Scroll down to your new block and click on it.Next / Previous Button Text option in Block Options
  4. Select Next/Previous button text.
  5. Under Next Button Text, type Submit, or the desired text.Customizing the Next Button Text
  6. Click Save.

Large Fieldset that Creates Reporting Issues

Expert Review may warn you that your survey will generate a very large fieldset that will make it difficult to use on the reporting end. What this means is that the structure of the survey adds additional fieldsets that make your exported data files gigantic and your standard reporting visualizations unable to display meaningful data.

Sometimes, your study’s design may necessitate a complex build that expands the data set. Often, this means you will have to clean the data yourself. However, this section will go over common sources of expanded fieldsets so that you can make reductions where able.

Loop and Merge

Loop and merge is a feature that repeats blocks either based on the answer to a question, or based on the number of fields the survey creator has added.

Example: Because each question in the block will be repeated for each loop field, this can exponentially expand the number of data columns. For example, in this screenshot, the loop and merge has five rows of fields. If this is set on a block of three questions, this results in 15 columns devoted to recorded data from each loop of these questions.Loop and merge fields

Here are a few ways to reduce fieldsets created by loop and merge:

  • Ask yourself if you need loop and merge, or if a unique block for each set of follow-up questions would reduce the number of total fields. For example, let’s say you’re looping based off a question, and only 3 of the 5 choices should lead to follow-up questions. Or, let’s say there are pretty big differences in which questions are asked for each choice. Then you would want to create different blocks for each possible answer and then use branch logic to determine when respondents go to each block.
  • If you are looping based off of a number, make sure you set a Max number of loops and ensure that this number isn’t too high.
  • If every respondent is looping the same number of times, make sure the total number of loops isn’t too high.
  • Make sure the looped block only contains questions you absolutely need to repeat.
  • Remember that you can filter data, so that if respondents are only supposed to answer questions about one choice they made in a previous question, you don’t need to loop at all. Instead, try using one block of questions with piped text. Piped text can indicate the answer provided in previous questions, which tells the respondent what they are being asked follow-up questions about. Loop and merge should only be used if respondents are selecting multiple items they must then answer the same set of questions about.

Survey Flow

Sometimes, the sheer number of branches and blocks you’ve added can expand your fieldset. When building elaborate branches, consider:

  • If the difference between choices is just a few questions, consider using display logic on each question.
  • Is your embedded data set up correctly? Each field should contain unique information. See the section below for more.

Embedded Data

Having thousands of embedded data can significantly expand your dataset. Here are some ways to cut down on embedded data:

  • Make sure you aren’t using multiple terms for the same thing. Figure out if you need “Date of Birth” or “D.O.B.,” but don’t use both. The same can be said between questions and embedded data. If you already asked for gender, do you really need to set embedded data for it, too?
  • If you’re setting values in the survey flow, make sure you have no spelling, capitalization, punctuation, or spacing errors. The minute you misspell an embedded data field’s name, you create a new column of data.
Example: In this screenshot, Water Quality (space between the words) and WaterQuality (no space between the words) become two separate embedded data.
Water Quality Embedded Data field in the survey flow
  • It can be easy to pull in the names of embedded data from your contact list. Especially if you have tons of fields to add, this can save you time! But sometimes you’ll pull in fields you know you don’t need for your study. You can cut down on data columns (but still be able to add data from the contact list back later) by deleting fields. In the Survey Flow, just click the name of the embedded data and press Backspace on your keyboard to remove it.
    Removing an embedded data field

Drill Down Questions

The drill down question is used to help respondents choose from a long list of options. Respondents first pick from a general dropdown list and from their answers are presented with specific follow-up lists to “drill down” to their answer. Because of the multi-tiered nature of this question type, it can result in the creation of many columns of data.

The best way to reduce the size of drill down questions in data exports is to make sure you export is to export without Split multi-value fields into columns selected. There is not an equivalent for reports.
Splitting multi-value fields into columns option

Warning: Do not ever edit a drill down after data has been collected! Even if your drill down is extremely long, you must leave it as it is or risk invalidating your data!

Before you collect any data, you can also consider whether multiple questions would be a better format than one drill down, especially if there is a large difference in the size of each drill down.

Example: Respondents are specifying what their first pet was. When a Species is selected, a dropdown for Breed appears. “Dog” is the only species with options, because the researchers are not interested in cat breeds or any other specific breakdown for the species listed. To cut down on the number of data cells, you can always create a separate question that only appears if you answer “Dog,” which asks for the breed.

Number of Choices & Carry Forward

Choices do not affect the reports as much as they affect exports. The number of choices can expand the export when you have Split multi-value fields into columns selected.

One of the ways you may be able to cut down on answer choices is if you are using carry forward. Make sure you are using this option because you need it. If you don’t actually need to carry every choice forward and are only interested in a few, you should probably add the choices manually.

Remember that before your survey has been activated (so long as your survey isn’t copied from a previously active survey), you can click Edit Multiple in the question editing pane to add or delete multiple choices at a time.
Editing multiple options in a question