Tree Testing Question

About Tree Testing Questions

Tree testing is a research method that is used to measure the efficacy of your website’s structure (also called a “tree”) by asking users to find items based on the website’s information architecture and terminology. The tree testing question enables you to analyze your website’s navigation and update your website to make it easier for users to find what they’re looking for.

Customizing a Tree Testing Question

This section covers how to customize a tree testing question so it matches your website’s structure. For general instruction on adding and editing questions in a survey, see Creating Questions.

1. Click on the question text to begin writing your question. We recommend adding instructions for survey takers on how to respond to the question.
   
2. Click Add Node to add the top-level categories of your website. You can add as many nodes as needed.
   Qtip: You can also press the Enter key on your keyboard to add another node at the same level.
3. Type your website categories in the Enter a value boxes.
   
4. You can add child nodes nested under your top-level nodes by clicking the plus sign ( + ) to the right of a node.
   Qtip: If needed, you can add additional layers of nesting by clicking the plus sign to the right of a nested node.
5. To remove a node, click the minus sign ( – ) to the right of it.
   
   Attention: If you delete a node with nested nodes, all nested nodes will also be deleted.
6. You can rearrange nodes by clicking the icon to the left of it and then dragging your mouse to where you want to move it.
7. To designate the correct path, click the star icon to the right of the path. You can designate multiple correct paths if needed.
   Qtip: You can only select nodes that do not have any child nodes.

Additional Customization Options

The following customization options are available for a tree testing question:

• Full tree view: When enabled, the entire tree will be visible to survey respondents. When disabled, the respondent will only see the area of the tree they select. Choose the format that best matches how menus are shown on your website.
  Example: The images below show how this option affects the survey taking experience.
  In the first image, this option is disabled, so only the current level and its nested nodes are displayed.
  
  In the second image, this option is enabled, so all levels are displayed even when they are not selected.
• Allow participant to skip task by answering “Not sure”: When enabled, this option adds a “Not sure” option to the question for respondents to select if they can’t find the right answer.
  
  Qtip: You can use this option with the force response option to require that users either select a node or the “not sure” option before continuing the survey.

Calculating Time Spent Responding

You can calculate the respondent’s time spent answering this question by adding a timing question to the same page as each individual tree testing question. See Timing a Question for step-by-step instructions.

Data Analysis

Once your responses have been collected, Qualtrics offers various methods to create reports both in and out of the platform. This section covers how to view your collected data and display the results using Qualtrics dashboards.

Data & Analysis

You can view the raw data for your survey by navigating to the Data & Analysis tab. The following data fields are available for each response with a tree testing question.

• Status: Indicates if the respondent reached the correct option. Possible values include:
  • Success: The final option selected matches the correct option provided by the researcher, regardless of the path taken to get there.
  • Failure: The final option selected does not match the correct option provided by the researcher
  • Abandoned: The survey respondent selected the “Not sure” option. This value will also display for the “Path Taken,” “First Click,” and “Final answer” fields.
  • Blank: If you have no data for this field, then the survey taker skipped the question without answering it, or the question was not displayed to the respondent.
• Path Directness: Indicates if the respondent navigated directly to their final choice or not. Possible values include:
  • Direct: ​​The survey taker took a direct path (no back-tracking) to their final choice (even if it was wrong). For example: Node 2 > Node 2b > Node 2b1.
  • Indirect: The survey taker back-tracked or took multiple paths before selecting an option. For example: Node 2 > Node 2b > Node 2b1 >Node 2c > Node 2c3.
• Path Taken: The actual path that respondent took (e.g., Men’s > Shirts > Men’s > Bottoms > Shorts).
• First Click: Indicate the first top-level node the respondent clicked. This will always be the first node listed in the “Path Taken” data field.
• Final answer: The final node the respondent selected. This will be the last child node listed in the “Path Taken” data field.
  Qtip: If a child node was never selected, then this field will be blank.

CX Dashboards

To analyze tree testing data in a CX dashboard, all associated data fields need to be changed to “text set” fields. Once your fields are remapped, you can display your data in different dashboard widgets.

If you want to combine different tree testing questions into the same widget, create a measure group for each metric.

Results Dashboards

You can also view tree testing data in a results dashboard, but you will first need to remap your tree testing fields by creating custom fields in the Data & Analysis tab. When mapping a field, create a new manual field, and then choose “text set” for your field type.

Widget Examples

After remapping your fields, you can analyze your data in a CX or results dashboard.

Below are a few widget ideas to get you started:

• Use a table widget to display the overall percentage of respondents who reached the correct destination for each path. This widget can only be created in CX dashboards since it uses measure groups.
  
• Use a table widget to display the percentage of respondents who reached each path directly and indirectly. This widget can only be created in CX dashboards since it uses measure groups.
  
• Use a bar chart widget to display distribution of first clicks for a given path. This widget can be created in both CX dashboards and results dashboards.
  
• Use a bar chart widget to display the percentage of respondents who took each possible path. This widget can be created in both CX dashboards and results dashboards.