The Best Approach to Survey Building
Building or creating a great survey is a struggle for most researchers. Understanding how to build a survey starts with understanding the research process. The most important step is to properly formulate the problem that you are trying to understand. Simply stated, if you don’t formulate the problem correctly, you can never build the optimal (or even good) survey or questions.
- Review the basic research objectives of the study.
- What is at the heart of what you are trying to discover?
- What actions do you want to take as a result of the survey?
- Visualize all of the relevant information items you need.
- What will the output report look like?
- What charts and graphs will be prepared?
- What information do you need to be assured that action is warranted?
- Prepare a written list of the topics in items 1 and 2 and order them according to their value in solving the research problem. List the most important topics first. Revisit items 1 and 2 again to make sure the objectives, topics, and information you need are appropriate. Remember, you can’t solve the problem if you don’t ask the right questions.
- Next, ask yourself “How easy or difficult is it for the respondent to provide information on each topic?” If it is difficult (they probably don’t know, can’t remember, can’t access the information, or won’t take time), then ask yourself if there is another way to obtain the information. Perhaps asking another question or using another data collection technique.
The fifth step in building a survey is to review the sequence of topics to make sure they are unbiased. Do the questions asked first influence or bias the results of the next questions? Sometimes providing too much information, or disclosing the purpose of the study can create bias.
Asking the Right Questions
Determine the type of question that is best suited to answer the question. We must also think about the type of data produced by a given question type.
One easy way to do this is to create a table in MS Word or Excel that has three columns:
- Column 1 – Enter the questions.
- Column 2 – Indicate the possible answers and their answer format: open-ended text questions, dichotomous, multiple choice, rank order, scaled, or constant sum (ratio scale).
- Column 3 – Determine the type of analysis needed (percentages, means and standard deviations, cross tabulations, and statistical tests). Do the question and answer formats provide enough robustness to meet analysis requirements?
Types of Questions:
Open-ended text questions, dichotomous, multiple choice, rank order, multiple choice matrix, Likert or Semantic Differential scales, constant sum, conjoint, side by side, percentages, frequency counts, means and standard deviations, cross tabulations, and statistical tests (chi-square, t-test, ANOVA, regression, multivariate analysis).
- Write the questions. You will need to write several versions of each question when you are building your survey. Building a great survey and building great questions often require six or more drafts. Select the best one.
- Review the question sequence, looking for any biases and if there is a logical flow.
- Repeat all of the steps above to find any major holes. Are the questions really answered? Are the answers what you really need to know? Have someone review it for you.
- Time the length of the survey. A survey should take less than five to ten minutes. At six questions per minute, and depending on the question difficulty, you are limited to about 30-40 questions. When building a survey, remember that one open-ended text question counts for three multiple choice questions.
- Pretest the survey to 20 or more people. Obtain detailed feedback, and critically look at their responses.
- Do they make sense, or do they have a different frame of reference than you had imagined?
- What were they unsure about?
- Did they have questions?
- Did they have trouble understanding what you wanted?
- Did they take a point of view not covered in your answers or question?
- Revise your questionnaire and either pre-test again or begin data collection.