10 Days with Jay: Q&A with Jay Choi, General Manager of Research Core



  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @mattyb513 said:
    Excited to see you here, Jay. Coming at you with a question for a large trade organization.

    We have thousands of member organizations that we support. Traditionally all our interactions with them have been via email or phone calls, very personalized interactions, the DC way. We would like to move to a system that allows us to better capture their experience in a measurable way so we can course correct and let them guide our decision making. Essentially, we want to do better at experience management but have no guidelines for how to get there.

    1) How would you think about evolving from a very person/relationship-centric model to a more scalable software solution without losing the personal touch?

    2) What kind of questions do you ask to help people feel like their responses are not just going to be aggregated into a dashboard?

    3) Who are some companies that have successfully done this who might be open to a short guidance call?

    Looking forward to picking that big brain of yours!

    Matt Bloomfield

    Great questions, Matt. I will DM you for more info.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @Cenzi said:

    How can I get the most out of my dual subscription to CX and EX?

    Want to show how my staff metrics impact my customers experience but not sure where to start.


    This is a great question, Adam. I’m glad you recognize the importance of marrying EX and CX, because how we treat our employees very often directly impacts customer experience. There is a lot of theoretical analysis on this, but I thought I would give one of the more pragmatic examples I had seen.

    One of our customers is an automotive company who has both CX and EX. At first, these were separate programs run independently at the company. When they first got their CX program, they correlated it with sales data and found a strong correlation between CX and dealership sales. The dealers with the highest sales also had the highest CX scores. They then took this analysis further and found that the dealerships with the highest CX had the highest employee engagement scores. While that was a powerful insight, I was really impressed by how they operationalized that insight.

    Here are a few things they’re doing:
    1. Aligned their CX reporting to their EX reporting - particularly around customer facing teams. Now managers can see their CX dashboards filtered in the same manner as their EX dashboard.
    2. Increased the frequency of the EX pulses to quarterly, because annual measurement didn’t allow them to track changes and intervene with their teams.
    3. Held weekly standups on their CX score. They literally sit around the table with every department in the dealership represented and solve problems in the moment
    And took immediate action on key EX issues:

    • Regular Trainings
    • Sales programs
    • Holding managers accountable for engagement on their teams

    Taking these simple actions completely changed the culture of the dealership. Customer-facing Employees could see that the organization was taking real action to improve their engagement and addressed many of the issues they faced. They could also see immediately that the organization was taking CX seriously through their weekly standups. By seeing that the organization was measuring and managing each of these elements, it drove a cultural shift and, impressively, even created a sales lift.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @mujtanibul said:
    Hi Jay,
    How can we measure customer journey success thru Q research core tool ? and is it possible to measure ( both X and O data) journey wise user experience ?

    How can you map product experience with revenue and a method to quality on routine basis?

    One example for UX research is the combination of clickstream (O-data) and Customer Effort Score (X-data). Oftentimes we will A/B test a new feature or the location of a button in our UX research. We utilize the drop off rates to understand how many people interacted with a new features followed by an in-product Effort Score study. Customer effort measures how difficult was it to complete your task, whether you accomplished your task, etc. Through this combination we can understand what is happening and why it is happening.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @jpardicusick said:
    Hi @JayC,

    Thanks for taking the time to do this with us.

    I have a question about industry XM benchmarking. We're a B2B communications/technology business and I'm struggling with finding meaningful benchmarks and standards to use for goal setting, especially as it relates to response rates and XM-specific metrics. What advice can you give about what KPIs to focus on (besides NPS) and where to find benchmarking information?



    NPS is great for measuring relationship strength over a long period of time, since you’re asking people to recommend a product or service to a relative or friend… aka, to the people who you trust the most, and who matter the most to you. In contrast, CSAT is typically a better measure for transactional engagements based on a single point in time. When the experience is still fresh in your mind, it’s great to have the chance to provide genuine feedback and let brands know how you feel. Customer Effort Score (CES) is another measure that focuses on how easy or difficult a task or engagement is. Obviously, we want to provide customers the easiest, quickest, and most enjoyable experiences when engaging with our brand.

    As for benchmarks, we have some very exciting news! At X4, Qualtrics’ incredible customer event, we are announcing several benchmark options to help you better understand how your scores measure up. We’ll have benchmarks from both JD Power and Bain & Company running on Qualtrics very soon. We also have the XM Institute, run by Bruce Temkin, and will soon be making XM Institute Benchmarks (formerly known as Temkin Benchmarks) available to all of our CX customers. Lots of exciting things are happening to help you better understand your customers and your business, and to help you make the best decisions.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @Shashi said:
    Hi @JayC ,

    With the regular updates in the product and new feature request coming in, how do you manage your work along with your personal life ( i.e. your work - life balance ) ?

    I think the most important thing you can do for yourself with regards to work-life balance is draw up some boundaries. Some projects are important enough that they are going to take up your time even when the work day has ended; that’s just life, and the results you get from putting in the extra hours can be incredibly rewarding. But it’s important to recognize when there’s nothing substantial that you can contribute anymore. Maybe the teams you need to work on a solution with went home, or it’s time to admit that the project demanding your attention just isn’t that dire. On particularly crazy days, I always make a point to shut off my phone at dinner so I can take some time to decompress with my family.

    It also helps that Qualtrics deeply encourages us to take at least one big, exciting vacation every year to just experience something new. I take my vacation time very seriously. I even did a week-long technology cleanse last year, and it was incredible.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @JasonHill said:
    Hi @JayC,

    We at UVU have built an integration with Canvas LMS. It's working beautifully and provides grade pass back to the Canvas grade book. We accomplish this, in part, using the Scoring option. However, we currently don't have a method to grab the Total Score possible as default embedded data. Instead, we add a custom embedded data field and manually set the value to the Total Score possible. Could I entice you to add the Total Score possible as a default embedded data field?

    Also, our integration is currently built using the Qualtrics and Canvas APIs, but we are in the final stages of the LTI development. I can arrange a sneak peak of this awesome sauce for you if you have something equally cool to share with me...could be anything from something awesome coming to the Research Core platform, or some cool skill you got.

    Wow, that’s incredible! I’m always impressed with the integrations our power users manage to make on their own. It makes me proud for what our API team has developed and people like you can leverage it!

    I would love to see you share this in the Community when you’re done - I’m sure we can all benefit from it! In terms of “Total Score,” I have to confess I’m not familiar enough with Canvas to make any promises.

    As for our roadmap… DM me, and we can trade secrets. :)

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @Cenzi said:
    Hi Jay

    Any advice best way to share insights with non-technical users without having to train them on how to be a survey guru?

    Is there an easy way in the research core which enables you to see who is looking at the dashboards we share and create?


    Hello again! For your first question, I’d say the support site and Basecamp are great places for users to stop by if they need help getting started. We have lots of resources for newbies that don’t dive too deep into the product, meaning they can get the basics without necessarily being a guru.

    As for your second question, that’s a great feature idea! I’m not sure if we have anything that currently fits that niche, since Research Core doesn’t cover dashboards, but we should have a feature like that if we don’t.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @bansalpeeyush29 said:
    Hi Jay,

    Qualtrics is said to be "one platform" for all kind of research. How you and your team managed to have all functionalities included within single platform?


    Hi, Peeyush! We keep our eyes and ears wide open for the challenges our customers want solutions for, as well as innovations happening in the marketplace around us. But it’s not enough to just do the research and have an enormous flux of information to sift through; I’d say our true key to success is our organization into the pillars of experience (Brand Experience, Customer Experience, Product Experience, Employee Experience, and then the Research Core at the center). This helps us stay broad in scope, while narrowing our focus team-by-team.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    @Alexandra_OPecko said:
    Hi @JayC!

    I am excited to be attending the summit again this year! I was so inspired last year that I made significant changes in my personal life and my career. Was it Dr. Lovatt? Alex Honnold? Arianna Huffington? Perhaps it was all of them combined...

    I started an online JavaScript course (to become a Qualtrics super user one day), and much to the shock of my family...I now take a beginners hip hop class. (Yes, I did break out with some dance moves when I read what was happening on Fight Night! :D )

    So, I was wondering: From where do you draw your inspiration? And how has this resource influenced your life?

    All my best,

    I’m so glad to hear that! I loved our keynotes last year, and I’m so pumped for the incredible guests we managed to get this year. And congrats on the classes!

    For inspiration, I look at consumer products and consumer software. While B2B software like Qualtrics has historically been held to a different standard. I believe B2C solutions have made the user experience beautiful and delightful and I think that should translate into what we use for our business tools

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭

    Hi @JayC,

    Thank you for taking the time to answer our questions. I have two questions for you.

    1) Are their any improvements in the works for Results Reporting? Our office still prefers Results Reporting over New Reporting and it would be great if Results Reporting could receive the same features that have rolled out to New Reporting.

    2) Are there any plans to better communicate new features for the Research Core as they roll out? Currently some features are communicated through the Community, others are communicated through the Product Update page (https://www.qualtrics.com/product-updates/?solution=rc) and others I just find on my own while using the tool. For example, the new Text IQ was announced in the Community and never showed up on the Product Update page. It is my responsibility to keep my team up to date on new RC features and this has become increasingly more difficult in the last few months.

    Thanks again!

    Thanks for the feedback! New Reporting mostly got to be the star for a while because it’s the newest reporting feature we offer for RC. We’ll take what you and our other customers have said into account as we proceed with Q2 planning.

    I’m sorry to hear it’s been difficult. We as a company are always working on new ways to get product updates out, and while the Product Updates page and the Community are currently the best forums for product improvements, we’ll be working on more channels in the future.

  • JayCJayC Provo, UTQualtrics / SAP Employee Qubie ✭
    edited March 2019

    @VirginiaM said:
    @JayC thanks for your thoughtful answers! I thought of another question, if you don’t mind.
    What made you want to work at Qualtrics and can you tell us about your journey to get where you are today?
    It seems like an incredible company to work for (and if I ever work there I’ll be a member of your Taco Bell fan club)!

    Haha, the Taco Bell fan club here is certainly very welcoming. :)

    Qualtrics’ culture of innovation and positivity are really what drew me in. I love seeing a company that treats employees and customers really, really well that can also maintain a breakneck growth.

This discussion has been closed.