Featured Researchers Kevin O’Neill and Christopher Korody from AUDIENCE METRIX Talk with Qualtrics
Names: Kevin O’Neill, Christopher Korody
Industry: Market Research at Corporate Events
Organization: Audience Metrix
Research Experience: 25 years, 12 years
Audience Metrix is a new company with a long history in corporate customer event research. Currently they are launching an Event Measurement Best Practices Survey, making innovative use of Qualtrics’ features. If you are involved with customer events at your company, we hope that you will support them by taking the survey.
Tell us about yourself. We would love to hear about your hobbies, education, and family.
Kevin O’Neill – I have a BS in Chemistry from the University of Wisconsin. After college, I was a Navy search and rescue helicopter pilot in Vietnam and later in the Mediterranean. When I finished my tours, I moved to Boston and joined Booz, Allen. I transitioned to several management roles with Prime; then into research as Vice President at the Business Research Group and later VP at Market Perspectives. In 2001, I founded Audience Insights and conducted market research for commercial tradeshows like MacWorld and LinuxWorld, as well as for conferences like the Game Developer Conference, Design Automation and the IEEE Microwave Symposium.
I live in Framingham, MA and have served as the Chairman of our local Conservation Commission for the past three years. In my free time, I enjoy kayaking, sailing, shooting skeet and playing with my granddaughter.
Christopher Korody – I have a Masters in Instructional Design from the University of Southern California. The underlying focus on understanding audiences, defining objectives and measuring results has always stood me in good stead. Over the years I have founded three event agencies, and held positions as a senior creative director with Dick Clark Corporate Productions and Vice President, Senior Creative Director at Jack Morton Worldwide.
I live in Taos, NM. I enjoy photography and fly-fishing, as well as cooking and gardening – you will find me at the Farmers Market most Saturdays in the summer.
What company/organization do you work for, and how long have you been in the research community?
Kevin has been working in research for 25 years, Christopher for 12. We met at the 2000 CES Show and have been working together doing event research on a project basis ever since. This year we formalized the partnership, and formed Audience Metrix.
In the words of one of our clients, “customer events are a goldmine of information about customer perceptions, preferences and behavior”. You could say that we use Qualtrics to create the picks and shovels that people need to get at the gold.
What has been the focus of your research recently? What is your favorite/most interesting project you have done with Qualtrics?
We are new to Qualtrics and are kicking things off with two very different projects. The first one is the first annual Event Measurement Best Practices Survey, which we have designed and are launching here.
Event ROI is becoming a very high profile issue, but there is very little information available that companies can use for benchmarking their processes and improving their own measurement performance. This survey will provide an in-depth view of how corporate event practitioners are addressing a wide range of measurement issues. We expect it to stimulate a lot of discussion.
To encourage participation, everyone who completes the survey will receive an individualized report comparing his or her responses to the aggregate average. We’ll do a Whitepaper summarizing the findings and identifying the most interesting trends.
Can you talk about some of the surveys you’ve been developing? Do you have some specific examples of interesting data you’ve collected from them?
The second project we are doing right now is for the Global Events Department at SAP. We are using a classic pre-post model to isolate what we call “the event effect”. After the event, we integrate multiple data sources to develop a very complete picture of how attendees use the event, and the impact that it has on their purchasing intentions.
For another client, we’ve been building a “global lens” that will enable managers to compare the effectiveness of customer events at a national, regional and global level. It’s a self-service model in which the Qualtrics translation engine makes it possible for local event managers to translate standardized surveys into the local language. We’re not sure if we are going to use all 53 of the languages that Qualtrics supports, but we expect it to be close!
By developing and applying a standardized question set, local teams will be able to compare their performance on the factors that they can control – the event itself instead of revenue. Our clients are excited by the idea developing these benchmarks will lead to a company wide exchange of best practices.
How has Qualtrics software assisted you in this research?
Here’s one that we really like. We are incenting people to participate in the Event Management Best Practices Survey by offering them a report comparing their individual scores with the average of all respondents.
The fact that the Qualtrics software can collect each respondents email address, calculate the individual reports and e-mail them to each participant with minimal involvement from us is fantastic. This is exactly the kind of thing that Qualtrics CEO Dr. Scott Smith was talking about, and a great example of how much thought has gone in to the product over the years.
What conclusions have you been able to draw from your research, and what impact has that had?
Our clients want to know how effective their events are at influencing their customers’ perceptions about the company. And they want actionable data to help them identify where increased investment will have a tangible impact on core business goals.
Probably the best “success story” is that we quantified a very significant revenue opportunity that had never been measured before. Since then both sales and marketing have focused on how to capture that business. You don’t get a win like that every time out, but when you do; it makes a convincing case for research.
What mistakes have you made that you wish you hadn’t?
For a long time we worked with the people who wanted to count things to prove that they had done their jobs. Ultimately, we realized that counting things is not very valuable – and worse, not very interesting.
What we are passionate about, is doing work that influences strategy; that increases the value of the event experience for attendees and that enables sponsors to demonstrate a return.
So maybe it wasn’t so much a mistake, as much as it was a matter of focusing on the kind of research that we wanted to do. And finding clients who value that. We work hard, and its satisfying when that hard work makes a difference.
What are the most valuable research related ‘epiphanies’ you have had in the past year that you wish you would have been aware of before?
One of the things that we are very interested in is understanding if an attendee’s event experience is a predictor of the kinds of online comments they are likely to make during and after an event.
Our hypothesis is that people who answer “Dissatisfied” and “Very Dissatisfied” are the same people who are the most likely to post negative comments. This is as much common sense as science, but it’s a connection that we haven’t seen described elsewhere.
The “aha” came when we realized that traditional event reporting – which uses techniques like Top Box Scoring and Averaging – was doing a fine job of hiding these kinds of attendees from view.
So working with our client at SAP, we came up with a concept called the AIR Score, for Audience Impact Rating, which takes the same 5 point Likert Scale that everybody uses, and looks at the ratio of the low scoring respondents against the highest scoring respondents. The AIR Score is now part of the standard reporting we do for SAP.
We describe the AIR Score in a whitepaper entitled Measuring The Modern Event. We are making it available for free to everyone in the Qualtrics community. In fact, we are hoping that Qualtrics will make the AIR Score a standard calculation. We look forward to hearing back from readers after they have had a chance to review the paper, and apply the AIR Score to their own data.
Do you have a favorite experience using Qualtrics tools?
We recently heard Qualtrics CEO Dr. Scott Smith talk about how the whole idea behind Qualtrics is to facilitate routine tasks – freeing up researchers to focus on the high value work. As a specialty research shop, that really resonates!
We love the ease of use, the depth of resources and the commitment to support. The rich array of advanced question types is enabling us to get to a deeper level of understanding faster. For instance, by combining the ‘drag, group and prioritize’ question with the sophisticated follow-up logic, our surveys are more self-adaptive than was previously possible.
The cloud architecture is a great thing since the upgrades happen seamlessly; and of course we’re thrilled that we don’t have to worry about the security of our clients’ data. And given that the partners are two time zones apart, being able to collaborate and work on different parts of the same instrument is incredibly productive.
Finally, what advice would you give to fellow Market Researchers?
It is increasingly clear that the Holy Grail for both B2C and B2B marketers is building one-to-one relationships based on a detailed understanding of customer preferences. Clearly, market research is one place those insights can come from… We’re excited about the future and the role that we as an industry will play. And we’re glad that we have Qualtrics in our bag of tricks.