Qualtrics Blog

Qualtrics provides users with a Cross Tabulation feature that can be used to find relationships between two variables. When making a Cross Tabulation, at the bottom of the table, there is a Chi-Squared statistic and a p-value. This statistic helps you measure the probability of seeing the distributions we do in the data, assuming the two variables of interest are independent. A low p-value (less that 0.05) suggests there is a relationship between the two variables. The Chi-Squared is best used to find a relationship between two categorical variables like any Likert scale question and many demographic questions. If you want to find a relationship between two continuous variables, consider using R² in Excel, SPSS, or another statistics package.

Many Qualtrics users want to evaluate how several variables affect a single response variable. The Cross Tabulation would be an exhausting process since each variable would have to be evaluated separately. The better way to do this would be to run a linear regression or logistic regression analysis in a statistics software package. This would allow you to see which variables influence your response and which ones do not. This will also help you make forecasts for your response variable. The data may have to be transformed to account for categorical variables, but this is not hard in most software packages.

A conjoint study is a great technique in analyzing a market’s desire for a product or service and their preferences for the features that it consists of. Surveying allows you to decipher these preferences. To do this, you need respondents to rank different combinations or ‘packages’ of a product. For an easy example, let’s consider a car as our product.

Type     Color     Transmission     Seats
Sport    Black     Manual               Leather
SUV      White    Automatic           Regular
Truck    Red
Van       Blue

For this example there are 64 different combinations. This, in almost any case, is far too many for a respondent to rank all of them. We will run into respondent fatigue. So instead of showing them every package, we show them a fraction of them. We obtain the packages by running a fractional factorial algorithm that will give us a proper representation of all the combinations and will allow us to have sound and reliable results to examine.

A good rule of thumb in how many packages should be shown in a survey is:

Number of Packages = (Number of Levels) – (Number of Features) +1

So in our car example, we have:
Number of Levels=12
Number of Features=4
Number of Packages = 12 – 4 + 1
Number of Packages = 9

A common algorithm used in obtaining the packages shown in the survey is the Federov algorithm. This can be found in the AlgDesign Package in R (a statistical-computing environment). Simply put, you insert your matrix consisting of all the possible combinations as well as how many packages you want it to spit out and it will respond with which rows in your matrix of combinations you will use to make up your fractional representation.

Translating a survey can be a pain. Making copies of surveys or questions, translating text, and setting up the logic (branching) or mailing lists so that each respondent gets the correct language all can be frustrating and time consuming.

So we decided to make survey creation in multiple languages easier.

In the past, users of online survey software had to switch back and forth between the native language and the translated language, scrolling up and down or changing windows.

Now, Qualtrics users can see the native language question and the foreign language version of the same question side by side. NO scrolling up and down or changing in between windows.

And if you don't want to go through and retype the entire survey in a different language three times, use the automatic translator to get a Google translation and automatically put it into your survey. Then you can skip straight to your proofreading phase.

Determining how data will appear before results are gathered is critical to creating a good survey, but it can be difficult to do. Clicking through the survey numerous times to simulate responses is very tedious, and data sets of five or ten often aren’t useful in determining how a survey should be structured.

The Test Survey feature has resolved this issue. It allows the user in the Qualtrics tool to collect ‘fake’ responses in order to see how data will appear. This component will allow a full data set to appear to the user before they ever distribute the link.

In the Edit Survey tab, click on ‘Test Survey’ in the Advanced Options drop-down menu (just to the right of the Launch Survey icon). Decide on the number of iterations you would like, then click ‘Start Test.’ When completed, the user can access the randomized data in the View Results tab.