Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM
  • Design XM
    Design XM

Significance Testing in Simple Charts & Simple Tables (CX)

What's on This Page:


Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Significance Testing in Simple Charts & Simple Tables

Dashboards can help you understand whether the differences you see over time or between groups are statistically significant, and therefore worthy of driving important business decisions. For example, you may have found yourself asking the following:

  • Did the NPS really go up this month, or is it a small change that’s just noise in the data?
  • Does the Midwest group actually have higher satisfaction scores than the West group?
  • Which of my five segments had higher or lower than typical scores on this metric?

With significance testing in simple line charts and simple tables, you can discover what data changes matter most.

Qtip: A similar feature is available for pivot tables, but significance is displayed differently.

Available Widgets and Metrics

Significance testing is currently available in the following widgets, with the following parameters. We’ll go over these in more detail in the following sections.

Widget Metrics X-Axis Dimension Additional Parameters
Simple charts Date fields only Line chart format only
Simple tables Any field N/A
Qtip: Only proportional custom metrics with a single field as the divisor can be used for significance testing. Proportional custom metrics follow the general format of (A + B) / C where A, B, and C are different data fields. A / B also works, since there is just a single field as the divisor. When creating custom metrics for this purpose, you can only use counts, as shown below. There cannot be static number in the equation; e.g., (A + 5) / B will not work.
Custom metrics as described

Significance Testing in Simple Charts

Simple line charts can flag the significantly high or low values of a change over time for NPS scores and top / bottom boxes. Each time period is compared against each time period previous when determining significance of a change.

  1. Add a simple chart widget.
  2. Make it a line graph.
    image of editing a simple chart widget to have a top box / bottom box metric
  3. Click Add Metric.
  4. Choose one of the eligible metrics for simple chart.
  5. Click Set X Axis Dimension.
    image of adding an x axis date dimension to a line chart
  6. Add a date field of your choice.
    Qtip: You can only have one x-axis dimension for this to work, and it must be a date field.
  7. Click on your metric.
    image of enabling significance testing in the options menu of a simple chart widget's metric
  8. Go to Options.
  9. Select Enable Significance Testing.
  10. Select your Confidence Interval.

Qtip: In the graph shown below, we’ve grouped the data by six month intervals.

image of a simple chart with the x axis dates grouped by Quarter

Significance Testing in Simple Tables

Significance testing can be performed on a simple table when using NPS or top / bottom box metrics broken out over an x-axis dimension of your choice.

  1. Add a simple table widget.
  2. Choose one of the eligible metrics for simple table.
    image of a top box / bottom box metric in a simple table
  3. Click Set Row Dimension and add a field of your choice.
  4. Click on your metric.
    image of enabling significance testing in the options menu of a simple table widget's metric
  5. Go to Options.
  6. Select Enable Significance Testing.
  7. Determine what you want your significance testing should be based off of. You have the following options:
    • Identify particularly high or low values: Most common selection for data that doesn’t involve time or dates.
      Example: Is Brazil’s score higher than other South American countries?
    • Compare each time period to previous: Most common selection for data that involves time or dates. Because of this, the option only appears if your x axis dimension is a date field.
      Example: Is this month’s overall score higher than last month’s?
    • Compare each value to each other value: This option is less commonly used, but is useful when comparing one baseline group to all other groups.
      Example: Is Brazil’s score higher than Venezuela’s score? Is Brazil’s score higher than Colombia’s? Is Venezuela’s higher than Colombia’s?
      Qtip: This option only appears if you have 20 or fewer data groups to compare. For example, let’s say you’ve collected data for 3 quarters. A simple table broken out by day cannot compare each day to each other day because there are 90 days of data. In contrast, if the Simple Table were broken out by quarter, there are only 3 quarters to compare to each other.
  8. Select your Confidence Interval.

Understanding Significance Testing

The Confidence Interval indicates how confident you would like to be that the results generated through the analysis match the general population. Higher confidence levels raise the threshold for a difference to be considered statistically significant, meaning only the clearest differences will be marked as such.

Once you have enabled significance testing, you might notice upwards and downwards arrows in your widget. These arrows indicate statistically significant values.

Turquoise line chart labelled "Averagw NPS." There's an upwards arrow at the highest point and a downwards arrow at the lowest point, showing these are both statistically significant in opposite directions

You can hover over an arrow to determine why the value is considered significant, and what the confidence interval of that test was.

Example: Here we hover over the blue arrow next to Q1 2019’s CSAT score. The tooltip tells us this value is higher than typical, and the confidence interval for this is 95%.

image of a simple table with the hover-over tooltip explaining a statistically significant value

Example: Here we hover over the arrow over the January 2020 – June 2020 NPS score. The tooltip tells us this value for this six month period is lower than the previous six month period (July 2019 – December 2019), and the confidence interval for this is 80%.

Turquoise line chart labelled "Average NPS." There's an upwards arrow at the highest point and a downwards arrow at the lowest point, showing these are both statistically significant in opposite directions

Technical Notes on Significance Testing

When comparing one NPS score to another, regardless of chart type or type of comparison (e.g., over time), the following process is used:

  1. Create a new column of data that recodes NPS scores in the following fashion:
    • Promoters = 100
    • Neutrals = 0
    • Detractors = -100
  2. Run a two-tailed Welch’s independent samples t-test.

When comparing one top box score to another, regardless of chart type or type of comparison (e.g., over time), the following process is used:

  1. Create a new column of data that recodes raw scores into TRUE or FALSE, depending on whether they meet the top box criteria.
  2. Run a two-tailed z-test for difference in two proportions.