Qualtrics Blog

For most companies and businesses, obtaining feedback from customers and clients is essential.  It provides them with a glimpse into what they are doing right, what they are doing wrong, as well as ways in which operations and relationships could be improved.  But obtaining and organizing this feedback in a way it can easily be sorted and delivered to the right people within the company can often be challenging.

Qualtrics, an online-survey company has the ability to tackle this challenge.  They have easy to create surveys and a lot of functionality.  You can make a survey that begins with asking the customer what topic best relates their subject of feedback.  And then you can provide them with space to jot down their feedback.  The neat thing about Qualtrics is that you can have the feedback sent to specific people based upon what topic was selected.  They call it the "trigger" option, because a selection triggers the email.

Let's say you make cellphones and you want feedback about the "cellphones breaking" sent to one group of your employees, and "troubleshooting" issues sent to another group.  When they select the "troubleshooting" topic, then that feedback will be automatically emailed to eveyone who has been pre-selected to be on that list.  It is really easy to set this all up and is great for organizing customer satisfaction feedback so that important feedback doesn't get lost.

This week Qualtrics Labs, Inc. published a study regarding an ongoing study dubbed the "worry index." The worry index stems from survey research conducted by Qualtrics that measures America's feelings on key social, economic and political issues. The report entitled "American Optimism for Future Trumps Worry Over Economy," was picked up by a number of traditional and online news sources that were surprised by the relative optimism of Americans in this time of economic crisis. Among other findings, the latest worry index indicated that "34 percent of Americans who believe their overall quality of life will improve in the next year, compared with 17 percent in February 2008."

For the sake of our national and world economies, let's hope that their hope is more than a hunch.

Survey Software doesn't do everything for you when building great online surveys. Survey building is an art, and like all arts requires a great amount of work, practice, and help from others. The following points provide an overview of the common pitfalls we often see in "bad questionnaires". Avoid these pitfalls and you are well on your way to understanding how to create a great survey questionnaire.

1. Avoid loaded or leading words or questions. Subtle wording differences can produce great differences in results. "Could", "should", and "might" all sound about the same, but may produce a 20% difference in agreement to a question (The supreme court could … should … might … change the limits on free speech in light of terrorist activities).
   Strong words, such as "prohibit" may represent control or action and influence your results (Do you believe that congress should prohibit insurance companies from raising rates?)
   Sometimes wording is just biased: You wouldn't want to go to Rudolpho's Restaurant for the company's annual party would you?
2. Misplaced questions. Questions placed out of order or out of context should be avoided. In general, a funnel approach is advised. Broad and general questions at the beginning of the questionnaire as a warm-up (What kind of restaurants do you most often go to?). Then more specific questions, followed by more general easy to answer questions (like demographics) at the end of the questionnaire.
3. Mutually non-exclusive response categories. Multiple choice response categories should be mutually exclusive so that clear choices can be made. Non-exclusive answers frustrate the respondent and make interpretation difficult at best.
4. Nonspecific questions. Build questions that are clearly understood.
   Do you like orange juice? This is very unclear…do I like what about orange juice? Taste, texture, nutritional content, Vitamin C, cancer prevention properties, the current price, concentrate, or fresh squeezed? Be specific in what you want to know. Issues of meaning and frequency are particularly difficult to specify: Do you watch TV regularly? (What is regularly? Does it matter what I watch — is a DVD the same as TV?).
5. Confusing or unfamiliar words. Asking about caloric content, bits, bytes and other industry specific jargon and acronyms is confusing. Make sure your audience understands your language level, terminology and above all — Do they understand what you are asking?
6. Non-directed questions that give respondents excessive latitude.
   What suggestions do you have for improving tomato juice? The question may be intended to obtain suggestions about improving taste, but the respondent may offer suggestions about texture, the type of can or bottle, about mixing juices, or even suggestions relating to use of tomato juice as a mixer or in recipes.
7. Forcing respondents to answer. Respondents may not want, or may not be able to provide the information requested. Privacy is an important issue to most people. Questions about income, occupation, finances, family life, personal hygiene and personal, political or religious beliefs can be too intrusive and rejected by the respondent. Incentives and assurances of confidentiality do make it easier to obtain private information.
8. Non-exhaustive listings. Do you have all of the options covered? If you are unsure, conduct a pretest using "Other (please specify)" as an option. When building multiple choice survey questions, make sure to cover at least 90% of the respondent answers.
9. Unbalanced listings. Unbalanced scales may be appropriate for some situations and biased in others. When measuring alcohol consumption patterns, one study used a quantity scale that made the heavy drinker appear in the middle of the scale with the polar ends reflecting no consumption and an impossible amount to consume. Similarly, we expect all hotels to offer good care and may use a scale of excellent, very good, good, and fair. We do not expect poor care.
10. Double barreled questions. What is the fastest and most economical Internet service for you? The fastest is certainly not the most economical. Two questions should be not asked simultaneously.
11. Dichotomous questions. When building a survey, answers should always be independent. The question "Do you think basketball players as being independent agents or as employees of their team?" is not dichotomous — Many believe that basketball players are both.
12. Long questions. Multiple choice questions are the longest and most complex. Free text answers are the shortest and easiest to answer. When you Increase the length of questions and surveys, you decrease the chance of receiving a completed response.
13. Questions on future intentions. Yogi Berra (Famous New York Yankees Baseball Player) once said that making predictions is difficult, especially when they are about the future. Predictions are rarely accurate more than a few weeks or in some case months ahead. The predictive accuracy of future intentions varies with the behavior, the frequency with which it is performed, the cost, behavior situation or occasion, and type product or service.

Sliding scale question types are useful if you want to keep your respondent excited about your survey.  Instead of throwing a bunch of multiple choice questions at the respondent, switch it up a little.  An example of a good question to use on a sliding scale would be "how happy are you with your Professor?"  Qualtrics provides different sliding scale icons (smiley face, letter grade, etc.) — the respondent can then slide a tab up or down a scale to make the smiley face smile or frown, based on how happy they with their professor.

Diversified question types are proven to improve attention spans in survey takers, so make the most of them!

Having a library of surveys available can be a very useful tool when building surveys. It is important to consider previous questions used in an effort to see which questions and wording have been useful, and which have not. The idea being over time you will be able to improve your survey efficacy and receive better, more useful data to use when making critical decisions.

Qualtrics enables you to create a specialized library of surveys and questions within your own account or a group account that you can pull from, or you can just use questions or groups of question in existing surveys you have used in the past. You can even pull from live surveys that are currently collecting data. This means every survey you create is essentially part of your survey "library", though not formally in the library. This is an important tool because it helps you to avoid having to recreate the wheel each time you need to create a survey. You can build new surveys faster, as well as make changes and improvements as you go, keeping questions up to date.

Try out this useful tool on the next survey you create by either clicking the "Copy Questions From…" button at the bottom of each question block, or by clicking on the green "Change Question Type" button to the right of a selected question and choosing the purple "Use Question from the Library". Both options allow you to navigate through your own survey or question library, a group's survey or question library, or the questions within your existing surveys on your My Surveys page.

One of the most captivating features in Qualtrics is the Loop and Merge function. This hidden helper makes a lazy survey maker into a survey master. The Loop and Merge function provides two shortcuts to a lengthy problem in normal online survey creation. The first solution takes a question and plugs different values into it. For example, let’s imagine that P. Diddy is trying to find a new type of fur for his clothing line. He has narrowed it down to chipmunk, mongoose, and ferret. P. Diddy wants to ask his respondents the same question, what is your opinion of _____?, about each fur type. The Loop and Merge function does this automatically. It will place each fur type; chipmunk, mongoose, or ferret, one at a time into the blank. This type of Loop and Merge is especially helpful for evaluating lengthy lists of items.

Another way the Loop and Merge can solve survey snags is by repeating survey question answers. This works when the same scale is used throughout an entire survey. For example, let’s imagine Meryl Streep wants to conduct a study about male stereotypes in the Chinese communities of North America. In order to keep uniformity, she’s decided to use a seven point satisfaction likert scale for her study. Loop and Merge can repeat that scale for all of her question responses. In other words, Meryl won’t have to spend seven hours typing in agree, disagree, etc. into the scale points.  Loop and Merge definitely speeds up the process of survey creation.