Introduction to Survey Research

About Surveys

Sometimes it’s good to review the basics. Surveys have been used throughout the years in a variety of forms. Our clients use Qualtrics for customer feedback, employee satisfaction, product registration, market analysis, new product development, academic research, voice of the customer, enterprise feedback management, and more. People are pushing the boundaries of traditional surveys and being creative with Qualtrics. The tool can be used for thousands of needs.

Although surveys are rapidly changing, the main parts of a traditional survey still remain:

  1. Surveys take a comprehensive view of something through a formal or official examination of the particulars.
  2. Visualize all of the relevant information items you need.
  3. Surveys are a sampling, partial collection of facts, figures or opinions gathered.
  4. Survey responses are used to approximate or indicate what a complete collection and analysis might reveal.

In summary, a survey is a systematic snapshot used to infer for a larger whole.

Benefits of Online Surveys

Online surveys provide five key benefits, as outlined below.

Efficiently develop and send multiple surveys or versions of your surveys

One of the major benefits of online surveys is the ability to edit your survey quickly and cleanly. A survey editor often has question types or a library of questions to help you get started. With a point-and-click editor, going back and making changes to your survey is simple. Also, with an online solution, you can send multiple versions of your surveys, depending on what you want to discover. Online surveys also let you collaborate with others if you are working on designing a survey together.

Distribute surveys/reports quickly via a browser to everyone who needs them

With a click of a button your survey can go all across the globe. Whether you are sending a survey or transferring reports, online is the way to go.

Send reminders and track respondents with a single click of a button

You have the luxury to track who has taken your survey. Results come in real-time, so you instantly know how many people you have reached. Instead of tedious phone surveys, paper surveys, mail options, online gives you more time-efficient results. Plus, if you are low on numbers, simply send out a reminder to increase response rates.

Produce results that reflect the right questions asked in the right way

Avoid the bias of personal interviews or face-to-face surveys. Control how your questions are asked and get results that reflect the right information.

Advanced security and confidentiality for your results through password protection, secure connections, and firewalls

Online data collection gives you control over your results. No more misplaced papers or respondents hesitant to hand-in results. You can password protect surveys, create anonymous links, and host data on secure servers.

Being a Quality Researcher

By Scott Smith, Ph.D.

Organizations that have taken the initial lead in using online survey software as a convenient way of gathering data have opened the door to more uses and more users who want more timely and accurate information. Online survey software and data collection, once seen as a supplement, is now poised to supplant much of traditional research methodologies. As those less experienced are empowered to conduct their own research, survey research quality is more important than ever. Qualtrics strives to be on the forefront of online survey innovation both in features and quality. Increasing research quality by increasing question quality is perhaps the lowest hanging of the fruit that can significantly reduce measurement error and increase the validity of research results.

Qualtrics has found that five simple “B’s” help refine the art of using survey software and avoid the common pitfalls we often see in “bad questionnaires.” Quality in survey research is a state of “BE-ing” that helps you create great questionnaires that greatly reduces “measurement error”.

1. Be Well Thought-Out

Avoid placing survey questions out of order or out of context. In general, a funnel approach is advised. Place broad and general questions at the beginning of the questionnaire as a warm-up, followed by more specific questions, followed by more general easy to answer questions (like demographics) at the end of the questionnaire.

2. Be Unbiased

Slight wording changes can produce great differences in survey results. “Could”, “Should”, and “Might” all sound almost the same, but may produce a 20% difference in agreement to a question: DNA modification of human food supplies could.. should.. might.. be allowed in the United States.

Strong words that represent control or action, (such as “prohibit”) produce similar survey results:

“Do you believe that congress should prohibit gasoline refiners from raising prices?”

Biased wording or statements should also be avoided:

“You wouldn’t want to go to Rudolpho’s Restaurant for the company’s annual party would you?”

3. Be Specific

Unclear survey questions produce answers that lack meaning. The question “Do you like orange juice?” produces a less than meaningful answer, but is the respondent referring to taste, texture, nutritional content, vitamin C, the current price, concentrated, or fresh squeezed? Again, “Do you watch TV regularly?” begs for a definition of “regularly”. Specific questions produce specific understanding.

Questions must be specific. Though intended as a question about taste, the question “What suggestions do you have for improving energy drinks?” may produce suggestions about texture, the type of can or bottle, mixing juices, additives, or something related to use as a mixer or in recipes.

4. Be Exact

Avoid Double Barreled Questions.The question “What is the fastest and most economical Internet service for you?” is problematic because the fastest is certainly not the most economical. Two questions should be asked in one question.

Avoid Overly Compressed Answers. Make sure answers choices are independent and cover the landscape of possible answers. For example the question “Do you think of basketball players as being independent agents or as employees of their team?” Some believe that they are both.

Response categories for multiple choice survey questions should be mutually exclusive so that clear choices can be made. Likewise, questions that do not provide all acceptable or meaningful answers frustrate the respondent and make interpretation of results difficult at best. If you are unsure, conduct a pretest using the “Other (please specify)” option. Then revise the question making sure that you cover at least 90% of the respondent answers.

5. Be Considerate

Avoid intrusive questions. Respondents may not have access to, remember, or want to provide the information requested. Likewise, privacy is an increasingly important issue. Questions about finances, income, occupation, family life, personal hygiene and beliefs (personal, political, religious) can be too intrusive and rejected by the respondent.

Avoid techno-babble. Caloric content, bits, bytes, MBS, and other industry specific jargon and acronyms produce confusion. Your audience must understand your language level, terminology and above all, the question being asked.

Avoid long questions. Multiple choice questions are the longest and most complex to write. Free text answers are the shortest and easiest to answer. Increasing the length of questions and surveys decreases the chance of receiving a completed response.

Avoid impossible questions on future intentions. Yogi Berra (famous New York Yankees baseball player) once said that making predictions is difficult, especially when they are about the future. Predictions are rarely accurate more than a few weeks or in some case months ahead.

These very important rules open the door to quality interactions with customers, employees, and valued partners. The recent and pressing need for enterprise-wide survey solutions has fueled an explosive growth in the industry, and Qualtrics strives to be at the forefront of online survey software innovation. Over the next few years we anticipate that survey software will be an even more popular corporate solution for gathering quality information. Qualtrics supports quality research through advanced survey software that is “easy enough for an intern, but sophisticated enough for a Ph.D.”

Scott Smith is the founder of Qualtrics.com. He is the James Passey Professor of Marketing and Director of the Institute of Marketing at Brigham Young University. He received his Ph.D. in Marketing and Quantitative Methods from Pennsylvania State University.

Google+