Qualtrics Platform
Customer Journey Optimizer
XM Discover
Qualtrics Social Connect

Question Randomization

About Question Randomization

Sometimes you need to randomize the order of questions in a block. With question randomization, you can do just that—and so much more. Some common options include:

Displaying all the questions in a random order
Displaying a set number of questions from a larger group
Locking certain questions in a specific position
Hiding specific questions

Setting Up Question Randomization

Qtip: Questions are randomized on a block basis, so be sure all of the questions you would like to randomize appear in the same block of questions.

Select the block you want to randomize. Once selected, the block should be outlined in blue.
In the Edit block menu, click Question randomization.
Select the randomization option you would like to use.
Click Save.

Qtip: Question randomization will override page breaks, questions per page, and skip logic. Disabling question randomization will restore these features.

Randomization Options

There are 4 main randomization options:

No Randomization: Display your questions in the order they appear in your survey editor.
Randomize the order of all questions: Present all of the questions in the block in random order.
Present only __ of total questions: Specify the number of choices (out of the total) to have randomly displayed in the survey (2 of 4, 3 of 7, etc.).
Advanced Randomization: Access even more randomization options.

Advanced Randomization

Advanced randomization has similar options to the randomize order and present only selections, with additional and more sophisticated settings.

Fixed Display Order: Shows the fixed questions in the block and where the random subset falls in the order of questions. You can rearrange the choice order with the up/down arrow buttons and move choices in or out of the randomization options using the left/right arrow buttons.
Qtip: Questions with no text (such as graphics) will still appear in the Fixed Display Order box, but they will appear as empty lines. You can address this by adding question labels.
Randomize Questions: Randomizes the order of all questions moved into the Randomize Questions box.
Random Subset: Randomly displays a specified number of questions moved into the Random Subset box (2 of 4, 3 of 7, etc.).
Excluded Items: Hides questions you don’t want displayed at all.
Questions per Page: Determines the number of questions that will be displayed on each page (this will override any page breaks on the block).

Qtip: Are you looking to randomize groups of questions, not just the questions themselves? See the Randomizer page for instructions on how to do this.

Random Subsets

Random subset allows you to ensure that each randomized question is seen the same amount of times by your respondent pool. To make sure your randomized questions are shown evenly, enable Evenly Present Elements.

You can also view and adjust the counts for how many times a given question has been presented. To do so, click Edit Count next to the Evenly Present Elements option. When viewing your counts, these numbers should be roughly the same, but you can override the counts if you need to temporarily give preference to a specific question. To change a question’s count, type the desired count into the box next to the question text.

You also have the option to reset all counts back to 0 or to restore all counts to what they were before you changed them.

Restore Original Counts: This option will restore the counts back to their original number before you edited them.
Warning: This option only restores counts for the timeframe in which you have the Edit Counts window open. If you edit your counts, save the changes, then return to edit the counts, the Restore Original Counts option will not revert your counts to before your edits.
Reset All Counts: This option will set the count for all elements to 0.

Randomization Data

You can view the order in which your choices were displayed to respondents by enabling Export Randomized Viewing Order before exporting your data. The display order of questions will appear as a column in your data file with multiple questions separated by a bar “|” if multiple questions were seen.

If you export with Split multi-value fields into columns selected, too, this will separate the viewing order into separate columns, instead of listing all displayed questions underneath one column.

To download randomization data

Navigate to the Data & Analysis tab.
Click Export & Import.
Click Export Data.
Choose the file type you would like to download.
Click the blue colored More Options text.
Select the Export viewing order data for randomized surveys checkbox.
Click the green Download button.

Qtip: To learn more about exporting data, view our Download Options page.