What Is A Survey?
Welcome to part one of the Qualtrics Survey Methodology Series. In our first post, we’re asking a super simple question: what is a survey? It seems like a simple question, but as with many things the answer is more complex than many people appreciate. Fundamentally, a survey is a method of gathering information from a sample of people, traditionally with the intention of generalizing the results to a larger population. Surveys provide a critical source of data and insights for nearly everyone engaged in the information economy, from businesses and the media to government and academics.
There are four modes of survey data collection that are commonly used.
- Face-to-face surveys
- Telephone surveys
- Self-administered paper and pencil surveys
- Self-administered computer surveys (typically online)
While surveys vary widely in how they are conducted and used, there are a number of components that are common across nearly all surveys. Many of these common features have been studied in extensive detail by survey methodologists, psychologists, statisticians, and many other fields of research. The general process of survey research is outlined in the figure below.
Tip: Always think about sources of error that can occur at each stage of the survey process.
Let’s dig into each step noted in the figure above.
- Define the research question: This is critically important to the success of a survey research project. Without a clearly defined question, it is difficult to determine the best approach for conducting the survey. For example, based on the research question, are the needed data exploratory, descriptive, or causal? The answer to this basic question has huge implications for the entire research process, yet it is often not directly addressed.
- Specifying the population of interest: This simply refers to determining who you want your data to represent. If you want generalizable information about your customers then your population of interest is your existing customer base. If you want generalizable information about the U.S. then the population of interest is the entire U.S. population.
- Identify a sample frame: Identifying a sampling frame is the process of determining how you will reach the population of interest. If you are surveying your existing customer base then you could use frames such as mailing addresses, telephone numbers, email addresses, or other existing points of contact that you have with them.
- Choose a data collection mode: The choice of data collection mode is largely driven by the sampling frame that was selected. If the sampling frame is a database of customer email addresses then the mode of data collection will typically be an online self-administered survey.
- a) Design and pre-test questionnaires: Designing the questionnaire carefully and then pre-testing it before fielding it to your entire sample is crucial to getting data that are valid and reliable. For example, careful questionnaire design and pre-testing can help reduce the chance that respondents may interpret the meaning of questions differently. Future posts in this series will tackle these important steps in much greater detail.
- Recruit and measure the sample respondents: This simply refers to the process of sending the survey out for data collection. Here it is key to put substantial effort into getting responses from everyone in the sample, this will determine the response rate of the survey.
- Code and edit the unadjusted data and the conduct post hoc data adjustments: Coding and adjusting the data is often necessary, particularly if open-ended questions were asked or if certain branched variables need to be combined. This is also when the data are often weighted to match known population parameters.
b) Select a representative sample: Selecting a representative sample from your sampling frame is also important for collecting valid and reliable data about the population of interest. For example, if you are sampling from a large database of customer email addresses and only wanted one response per household, you might want to cross-check each email address against mailing addresses and remove duplicates to avoid some households having a greater probability of selection. Then you would likely draw a random sample from the remaining list of email addresses.
In future weeks we’ll cover the remaining items, including data analysis and interpretation. Of course these steps are also far from trivial and in future weeks this series will also address some common issues that people run into when analyzing and interpreting survey data.
The Qualtrics Survey Methodology Series appears every Wednesday on the Qualtrics blog. If you have topics you’d like to see covered, please take this one question survey to let us know what content you would find valuable.