Orienting your XM program on impact: Insight tracking

Last year we launched the value chain framework – a guide to help Experience Management professionals connect their work to business outcomes. As highlighted in the guide, we see a number of areas where organizations get stuck. This article focuses on one of these areas: the black box trap.

Can you point to 3 meaningful business outcomes your team’s work has driven over the past few months? If not, then you might be falling into the black box trap.

Many Experience Management (XM) teams operate as a centralized function, relying on other departments to generate and act on insights. While this is normal – and we support a Center of Excellence organizational structure – it tends to create a blindspot: A lack of visibility into the actions taken as a result of their work.

Limited visibility into actions is problematic because actions are the path through which XM teams create value for the organization. Great insights are only potential value – actions are what turn them into value. No action = no value.

Fortunately, “driving change across the organization” and “helping other people use insights” are the most enjoyed activities for XM professionals. Further, XM professionals cite “leading multi-year transformation” and “responding to insights with action” as the most important skills for them to develop over the next few years of their careers.

In this article, we’ll provide a framework for assessing and improving how effectively you are driving action with your efforts.

What is your relationship with actions?

Demonstrating the impact of Experience Management is a continual effort. Further, it isn’t enough to quantify program activities (# of survey responses, role based dashboards, interviews, etc.) or changes in XM scores. Success hinges on the ability to track actions, not only to assess and communicate their impact, but also to remove obstacles preventing stakeholders from taking action.

To start addressing this, let’s look at the types of relationships XM teams have with insights and actions:

In practice, many teams have a combination of these relationships, but understanding them helps illuminate the gaps. Further, this is not a judgment on any of those scenarios – every organization is different and there is a case to be made for each structure. However, scenarios 1 and 2 introduce blind spots that need to be addressed in order to demonstrate the impact of the XM program.

1. Unknown insights, unknown actions

The “unknown insights, unknown actions” scenario is common among teams that operate a self service model. This is a great approach and an important part of scaling Experience Management. But, these teams may need to take extra steps to ensure they can measure and communicate their impact.

Imagine a simple survey that goes to the active users of a given report, asking “Thanks for using XYZ report last quarter. Did you learn anything new? What did you learn? Did you do anything with that information? What?” While this approach may not be comprehensive, it gives a good idea of the types of things people are learning from the data and what they are doing with it.

While you’re at it, you might also ask the low usage individuals why they haven’t been using the report. Maybe they weren’t aware it existed, they didn’t think it was relevant for their work, or something else. This is useful information for improving the effectiveness of the self service model.

2. Known insights, unknown actions

The “Known insights, unknown actions” scenario is common among XM teams that develop specific insights and share them with other teams.

In this scenario, document every insight you generate and who you shared it with. Then, on a quarterly cadence, follow up with these individuals to ask what they have done with the insights and what the results were.

If the number of stakeholders you serve is small enough, you might be able to do this on a call or over chat. If you have dozens or more stakeholders, you might consider a survey along the lines of, “We shared XYZ insight with you, have you done anything with it? What? If not, what kept you from doing something with it?” All useful information for measuring your impact and driving more change.

Insight tracking

Once you have a good idea of your insights and actions, regardless of how you got there, document them. Imagine a two-column spreadsheet:

• Column A: All insights. This is the same as the first column in the value chain framework.
• Column B: All actions taken on those insights. This is the same as the second column in the value chain framework.

It requires discipline but goes a long way in helping you demonstrate the impact of your work. Here are 3 ways to get the most out of your insight tracker:

1. Evaluate program effectiveness. Use the tracker to gauge how well your program is generating insights and driving actions. Are you surprised by the number of insights generated or actions taken over the previous period? Are the insights sufficiently actionable and compelling? What is your insight-to-action rate? Why aren’t certain insights being acted on? What can you do to improve the action rate of your insights? Do the actions represent real change, or just program activities?
2. Build your history of action. A timeline of actions you have driven in the business is a great asset when requesting resources. Even if they don’t understand all details of your work, senior leaders understand actions. Show that you are driving change that moves the organization toward its goals. This is a fantastic way of demonstrating that your team has been a good steward of the resources you were given.
3. Inspect the impact of specific actions. Evaluate whether you can measure the impact of large actions. This corresponds to the third column in the value chain framework. Look through your history of action to identify actions that were particularly significant and where there might be a natural experiment that allows you to measure the impact. You may not always find what you are looking for, but it’s usually the right place to start.

Insight tracking examples

When an XM team asks me for help measuring their impact, my first question is always about what actions they have driven. Far too often, they don’t know… But sometimes I am pleasantly impressed.

Below are examples of great insights and actions I have seen over the course of my work:

In practice, there are typically more details behind the insights and actions. However, the insight tracker can serve as a simple library, summarizing and pointing people to more detail should they need it.

Resources and call to action

Insight tracking is not a technically difficult exercise but it does require discipline. Consider putting in place an insight tracker for your program and review it at least quarterly.

To help, we have published several new tools. Modify them to suit your needs:

• Insight tracking spreadsheet: A simple template to track insights and their associated actions. Modify as needed for your purposes. For example, some organizations add the subsequent columns from the value chain framework (KPI Improvements, and Business Impact), and others add an upfront column to categorize by priority.
• Survey for unknown insights: A survey template to help you learn from those who consume your dashboards and reports. The survey asks whether they have found insights from the data, and what they have done with it.
• Survey for known insights: A survey template to help you identify the actions resulting from the insights you have provided to specific stakeholders.

The bottom line: Insight tracking helps you assess and increase the impact of your program.