Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM

Significance Testing in Simple Charts & Simple Tables (CX)

What's on This Page:


Was this helpful?


The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Significance Testing in Simple Charts & Simple Tables

Dashboards can help you understand whether the differences you see over time or between groups are statistically significant, and therefore worthy of driving important business decisions. For example, you may have found yourself asking the following:

  • Did the NPS really go up this month, or is it a small change that’s just noise in the data?
  • Does the Midwest group actually have higher satisfaction scores than the West group?
  • Which of my five segments had higher or lower than typical scores on this metric?

With significance testing in simple line charts and simple tables, you can discover what data changes matter most.

Available Widgets and Metrics

Significance testing is currently available in the following widgets, with the following parameters. We’ll go over these in more detail in the following sections.

Widget Metrics X-Axis Dimension Additional Parameters
Simple Charts
  • NPS
  • Top / Bottom Boxes
  • Average
  • Custom Metrics
    Qtip: Only proportional custom metrics with a single field as the divisor can be used for significance testing. Proportional custom metrics follow the general format of (A + B) / C where A, B, and C are different data fields. Custom metrics in any other form cannot be used in significance testing
Date fields only Line Chart format only
Simple Tables
  • NPS
  • Top / Bottom Boxes
Any field N/A

Significance Testing in Simple Charts

Simple Line Charts can flag the significantly high or low values of a change over time for NPS scores and Top / Bottom Boxes. Each time period is compared against each time period previous when determining significance of a change.

  1. Add a Simple Chart widget.
  2. Make it a Line Graph.
    image of editing a simple chart widget to have a top box / bottom box metric
  3. Click Add Metric.
  4. For your metric, choose Net Promoter Score or Top / Bottom Box.
  5. Click Set X Axis Dimension.
    image of adding an x axis date dimension to a line chart
  6. Add a date field of your choice.
    Qtip: You can only have one X Axis Dimension for this to work, and it must be a date field.
  7. Click on your Metric.
    image of enabling significance testing in the options menu of a simple chart widget's metric
  8. Go to Options.
  9. Select Enable Significance Testing.
  10. Select your Confidence Interval.

Qtip: Our graph is split into Q1, Q2, and Q3 because we group our date dimension by Quarter.

image of a simple chart with the x axis dates grouped by Quarter

Significance Testing in Simple Tables

Significance testing can be performed on a Simple Table when using NPS or Top / Bottom Box metrics broken out over an x-axis dimension of your choice.

  1. Add a Simple Table widget.
  2. Set your metric to either Net Promoter Score or Top / Bottom Box.
    image of a top box / bottom box metric in a simple table
  3. Click Set Row Dimension and add a field of your choice.
  4. Click on your Metric.
    image of enabling significance testing in the options menu of a simple table widget's metric
  5. Go to Options.
  6. Select Enable Significance Testing.
  7. Determine what you want your significance testing should be based off of. You have the following options:
    • Identify particularly high or low values: Most common selection for data that doesn’t involve time or dates.
      Example: Is Brazil’s score higher than other South American countries?
    • Compare each time period to previous: Most common selection for data that involves time or dates. Because of this, the option only appears if your x axis dimension is a date field.
      Example: Is this month’s overall score higher than last month’s?
    • Compare each value to each other value: This option is less commonly used, but is useful when comparing one baseline group to all other groups.
      Example: Is Brazil’s score higher than Venezuela’s score? Is Brazil’s score higher than Colombia’s? Is Venezuela’s higher than Colombia’s?
      Qtip: This option only appears if you have 20 or fewer data groups to compare. For example, let’s say you’ve collected data for 3 quarters. A Simple Table broken out by day cannot compare each day to each other day because there are 90 days of data. In contrast, if the Simple Table were broken out by quarter, there are only 3 quarters to compare to each other.
  8. Select your Confidence Interval.

Qtip: Our graph is split by Q1 2019, Q2 2019, and Q3 2019 because we group our date dimension by Quarter.

image of a simple table with the response date field grouped by quarter

Understanding Significance Testing

The Confidence Interval indicates how confident you would like to be that the results generated through the analysis match the general population. Higher confidence levels raise the threshold for a difference to be considered statistically significant, meaning only the clearest differences will be marked as such.

Once you have enabled significance testing, you might notice red and blue arrows in your widget. These arrows indicate statistically significant values.

image of a simple chart with a data point indicating a statistically significant value for Q2

You can hover over an arrow to determine why the value is considered significant, and what the confidence interval of that test was.

Example: Here we hover over the blue arrow next to Q1 2019’s CSAT score. The tooltip tells us this value is higher than typical, and the confidence interval for this is 95%.

image of a simple table with the hover-over tooltip explaining a statistically significant value

Qtip: A similar feature is available for Pivot Tables, but significance is displayed differently.

Technical Notes on Significance Testing

When comparing one NPS score to another, regardless of chart type or type of comparison (e.g., over time), the following process is used:

  1. Create a new column of data that recodes NPS scores in the following fashion:
    • Promoters = 100
    • Neutrals = 0
    • Detractors = -100
  2. Run a two-tailed Welch’s independent samples t-test.

When comparing one top box score to another, regardless of chart type or type of comparison (e.g., over time), the following process is used:

  1. Create a new column of data that recodes raw scores into TRUE or FALSE, depending on whether they meet the top box criteria.
  2. Run a two-tailed z-test for difference in two proportions.