Market Research

11 Tips for building effective surveys

You don’t have to be an expert to build and distribute an effective online survey, but by following a few survey best practices you can make sure you’re collecting the best data possible.

When you don’t have hours to devote to becoming a survey-creation guru, a quick guide to the essentials is a great way to get started.

Here’s how to make a survey that’s easy for respondents to complete, hits the research questions you’re interested in, and produces data that’s easy to work with at the analysis stage.

Start creating surveys today with a free Qualtrics Surveys account

1. Make every question count

You’re building your survey questionnaire to obtain important insights, so every question should play a direct role in hitting that target.

Make sure each question adds value and drives responses that relate directly to your research goals. For example, if your participant’s precise age or home state is relevant to your results, go ahead and ask. If not, save yourself and your respondents some time and skip it.

It’s best to plan your survey by first identifying the data you need to collect and then writing your questions. For a deeper dive into the art and science of question-writing and survey best practices, check out Survey questions 101

2. Keep it short and simple

Although you may be deeply committed to your survey, the chances are that your respondents... aren’t. As a survey designer, a big part of your job is keeping their attention and making sure they stay focused until the end of the survey.

Respondents are less likely to complete long surveys, or surveys that bounce around haphazardly from topic to topic. Make sure your survey follows a logical order and takes a reasonable amount of time to complete.

Although they don’t need to know everything about your research project, it can help to let respondents know why you’re asking about a certain topic. Knowing the basics about who you are and what you’re researching means they’re more likely to keep their responses focused and in scope.

3. Ask direct questions

Vaguely worded survey questions confuse respondents and make your resulting data less useful. Be as specific as possible, and strive for clear and precise language that will make your questions easy to answer.

It can be helpful to mention a specific situation or behavior rather than a general tendency. That way you focus the respondent on the facts of their life rather than asking them to consider abstract beliefs or ideas.

See an example:

4. Ask one question at a time

Although it’s important to keep your survey as short and sweet as possible, that doesn’t mean doubling up on questions. Trying to pack too much into a single question can lead to confusion and inaccuracies in the responses.

Take a closer look at questions in your survey that contain the word “and” – it can be a red flag that your question has two parts. For example: “Which of these cell phone service providers has the best customer support and reliability?” This is problematic because a respondent may feel that one service is more reliable, but another has better customer support.

See an example:

 

 

5. Avoid leading and biased questions

Although you don’t intend them to, certain words and phrases can introduce bias into your questions or point the respondent in the direction of a particular answer.

As a rule of thumb, when you conduct a survey it’s best to provide only as much wording as a respondent needs to give an informed answer. Keep your question wording focused on the respondent and their opinions, rather than introducing anything that could be construed as a point of view of your own.

In particular, scrutinize adjectives and adverbs in your questions. If they’re not needed, take them out.

See an example:

 

6. Speak your respondent's language

This tip goes hand in hand with many others in this guide – it’s about making language only as complex or as detailed as it needs to be when you conduct a survey.

Use language and terminology that your respondents will understand, keeping language as plain as possible, avoiding technical jargon and keeping sentences short. However, beware of oversimplifying a question to the point that its meaning changes.

See an example:

 

 

7. Use response scales whenever possible

Response scales capture the direction and intensity of attitudes, providing rich data. In contrast, categorical or binary response options, such as true/false or yes/no response options, generally produce less informative data.

If you’re in the position of choosing between the two, the response scale is likely to be the better option.

Avoid using scales that ask respondents to agree or disagree with statements, however. Some people are biased toward agreeing with statements, and this can result in invalid and unreliable data.

See an example:

 

 

8. Avoid using grids or matrices for responses

Grids or matrices of answers demand a lot more thinking from your respondent than a scale or multiple choice construction. They need to understand and weigh up multiple items at once, and oftentimes they don’t fill in grids accurately or according to their true feelings.

Another pitfall to be aware of is that grids aren’t mobile-friendly. It’s better to separate questions with grid responses into multiple questions in your survey with a different structure such as a response scale.

See an example:

 

9. Rephrase yes/no questions if possible

As we’ve described, yes/no questions provide less detailed data than a response scale or multiple-choice, since they only yield one of two possible answers.

Many yes/no questions can be reworked by including phrases such as “How much,” “How often,” or “How likely.” Make this change whenever possible and include a response scale for richer data.

See an example:

 

10. Start with the straightforward stuff

Ease your respondent into the survey by asking easy questions at the start of your questionnaire, then moving on to more complex or thought-provoking elements once they’re engaged in the process.

This is especially valuable if you need to cover any potentially sensitive topics in your survey. Never put sensitive questions at the start of the questionnaire where they’re more likely to feel off-putting.

Your respondent will probably become more prone to fatigue and distraction towards the end of the survey, so keep your most complex or contentious questions in the middle of the survey flow rather than saving them until last.

11. Use unbalanced scales with care

An unbalanced response scale can lead a respondent in the same way a poorly worded question might.
For example, if you’ve asked them to rate a product or service and you provide a scale that includes “poor”, “satisfactory”, “good” and “excellent”, they could be swayed towards the “excellent” end of the scale because there are more positive options available.

Make sure your response scales have a definitive, neutral midpoint (aim for odd numbers of possible responses) and that they cover the whole range of possible reactions to the question.

12. Take your survey for a test drive

Want to know how to make a survey a potential disaster? Send it out before you pre-test.

However short or straightforward your questionnaire is, It’s always a good idea to pre-test your survey before you roll it out fully, so that you can catch any possible errors before they have a chance to mess up your results.

Ask at least five people to test your survey to help you catch and correct problems before you distribute.


Start creating surveys today with a free Qualtrics account