Calculate Metric Task

About the Calculate Metric Task

The Calculate Metric task lets you trigger a response to metric changes calculated in a dataset over a period of time. In other words, you can set up automated workflows based on trends or events you observe in experience data. The results of your calculate metric task can be used to trigger other tasks within your workflow. For example, you can notify your marketing team’s Slack channel when your brand NPS dips by 10% within a period of a week, or you can create a follow-up ticket for your team if more than 100 opt-outs occur in XM Directory within a week.

Setting Up a Calculate Metric Task

1. Follow the linked directions to set up a scheduled workflow.
2. Click the plus sign ( + ) and then Task to create a task.
3. Select Calculate metric.
   
4. Here, you’ll specify the data source that contains your data, the metric you’d like to calculate, and the field to perform the calculation on.
   • Source: The survey or dashboard that hosts the data you will be calculating your metric from. All of your available surveys and dashboards will appear in the dropdown.
     Attention: If you are using a dashboard as your source, make sure that the dashboard has been visited at least once in the last 30 days. If it’s been close to 30 days since the dashboard was last visited, you will receive a warning message via email to the email address associated with your Qualtrics account.
     
   • Metric: The type of metric you’ll be calculating based on. Choose from the following metric types:
     • Count of records: Display response counts from the selected fields (i.e., number of respondents choosing a value in a five-point scale).
     • Average: Exhibit the average value for a selected field.
     • NPS: Net Promoter Score. Identify the Detractors, Passives, and Promoters from NPS questions.
   • Field: The specific field or question you will be calculating your metric on. This field does not appear if you select “count of records” as your metric type.
5. If you selected Top Box / Bottom Box, define what falls into this range using the Metric range minimum and Metric range maximum. When the task runs, you will get the percentage of responses that are within this range of values.
6. If desired, add Filters. This can be used if you’d like to have the metric calculate on a subset of your data. You can filter based on one of the data source’s survey questions or one of your survey’s embedded data fields. You can also filter based on Text iQ sentiments and topics. For more information on setting conditions in filters, visit Creating Conditions.
   
   Attention: Some of the resources for condition building use examples in data filtering instead of Workflows. However, the logic behind these conditions are the same.
   Qtip: When setting up the choice field in your filter you have the option to use piped text from your selected survey. Piped text is only available for multiple choice questions and text entry questions.
   Qtip: If you’re using a dashboard data source, all fields will be listed under Question.
7. Define the time periods you want to compare.
   
   • Single time period: The metric will be calculated using data from the chosen time period.
   • Comparison time period: This period is optional. If you select a comparison time period, the metric will be calculated using a comparison between your chosen time periods. The comparison time period is immediately prior to the single time period specified above.
     Example: For example, if I want to calculate the average score over the past 2 weeks, I would select “Over the past day” for the single time period and “Previous 14 days” for the comparison time period.
     Qtip: Check out the graph inside the task for an example comparing these two time periods and the resulting calculation for each.
   Attention: If you select Count of records as your metric type, you can only choose a single time period. The comparison time period option will be grayed-out.
   
8. Click Save.

Adding Tasks and Conditions

After setting up your calculate metric task, add conditions and tasks to decide what happens next. Conditions act as thresholds; for example, specifying how much a metric should rise or drop before notifying a team. Tasks determine what specifically happens next; for example, sending an email or a Slack message.

1. If you want to add thresholds, click the plus sign ( + ) and then Conditions.
2. Click Condition.
   .
3. Click Select Condition Type and select Tasks.
4. Select Calculate metric.
   
   Qtip: You’ll see the tasks’s ID in this step. This can be helpful if you’re adding multiple calculate metric tasks to a workflow.
5. Click Select Option to select the calculation type. See linked section for more details.
6. Define your threshold.
   Example: In the screenshot above, we want to trigger a workflow whenever our metric changes by 3 or more points. This change is measured over an initial and comparison time period defined during setup.
7. Click the plus sign ( + ) and select Task to what should happen once metrics are calculated and conditions are met. For example, you can send an email, a Slack message, a Microsoft Teams message, or even an in-product notification.
   
   Qtip: Each tasks are labeled with a unique ID (e.g., “T-ID 1,” “T-ID 2”) for easier identification and organization. If you add a calculate metric task after an email task, but you’re piping the results of the calculate metric task into your email task, the tasks will be automatically reordered so that they are in the correct order.

Qtip: You can also use piped text to report the results from the calculate metric task in subsequent tasks. For example, if you wanted to include information about your calculate metric task in a subsequent email task, the results from the calculate metric task will be available in the piped text portion of the rich content editor.

When Calculate Metric Tasks Are Completed

All types of survey responses can cause changes to the metrics calculated in a calculate metric task. This will cause subsequent tasks to trigger under certain conditions.

Options differ based on whether you chose only a single time period, or also added a comparison time period.

When only a single time period is defined

• Average: Calculate a threshold based on an average / mean metric.
• Count of records: Calculate the number of responses collected for this metric, regardless of the metric’s actual score.
• Standard deviation: Calculate the standard deviation for the metric.

• Absolute change: The aggregate data from the first time period subtracted from the aggregate from the second time period.
• Comparison time period: If you configure the calculate metric task with the Change over a period of time option, this is the group of output values from the second time period you set in your task.
• Relative change (%): The aggregate data from the first time period subtracted from the aggregate from the second time period, divided by your aggregate from the second time period.
• Single time period: If you configure your calculate metric task with the Change over a period of time option, this will be the output values from the initial time period you set in your task.

Troubleshooting Calculate Metric Tasks

This section has links to additional resources that can help you troubleshoot issues related to email tasks.

• Identifying Whether Calculate Metric Tasks Failed: Workflows reporting allows you to check on the status of tasks in your survey, and identify whether they succeeded. This troubleshooting tool is best for checking on issues on the task setup side, and will not help you dig deeper into email deliverability issues, such as invalid recipients.
• Troubleshooting Conditions: The way a condition on an task is worded can change when the task is executed. First, remember to check the general rules for who receives email tasks, and when. If you need help troubleshooting the way conditions are written, check out these resources:
  • Creating Conditions (With step-by-step example)
  • New Conditions and Condition Sets
  • All vs. Any
  • Nesting Logic

Filtering by Dashboard Data

When you set up your task, you can base a metric off of data in a survey or off data in a dashboard.

If you choose to base your metric off dashboard data, there are a few things you should know before you add a filter to your task.

Make sure all of the data you want to use in this task is mapped to your dashboard. Also make sure to choose the best type for each field. For example, questions scored on a scale should be number sets, whereas options in a list should be text set.

All dashboard data is listed under Question, not Embedded Data. It doesn’t matter what type of data each field originally was in the survey; they’re all listed under “Question” for dashboards.

The values you want to filter with might not immediately appear. First, make sure you’ve collected data for the missing values. If you have, it’s time to recode the field.

For the field you want to break out by, make sure you define each of the values you want to use in the recode editor.

Once your fields are recoded, you should be able to add filters to your task.