Qualtrics has provided a cross tabulation tool for you to perform multivariate analysis on two or more variables at a time. This tool includes numerous options to customize your crosstabs, including the ability to calculate Chi-squared statistics and ANOVAs.
This page covers the basics of setting up a crosstab, in addition to all the various ways you can configure your variables or make calculations. See Crosstab Options for additional functions.
Qtip: Crosstabs are statistical tests. See more about cross tab theory.
If you are looking for a table that displays the number of times each choice was selected, you should look at Simple Tables (in Results-Reports) or Data Tables (in Advanced-Reports), not crosstabs. If you want basic mean, minimum, maximum, and other stats information for only one question, look into Statistics Tables (Results) or Statistics Tables (Advanced), not crosstabs.
Creating New Crosstabs
- Navigate to the Data & Analysis tab.
- Click Crosstabs.
- Click Create your Crosstab if this is your first, or select Create New Crosstab from the dropdown menu if you’d like to make another one.
- To the left are your Variables. This includes survey data such as questions, embedded data, metadata, and text analysis results.
- Click Next.
- Choose a variable from the list and drag it into the Columns (Banner) box to create columns. Usually these are demographic or “input” variables, such as age, income, or gender.
Qtip: Numeric text entry questions cannot act as a column or row.Qtip: You can select multiple variables by holding down the command key on Mac or the control key on PC and clicking the variables you want to select. You can also select many variables in a row by holding down the shift key on your keyboard and then clicking the first and last variable in your desired selection.
- Choose a variable from the list and drag it into the Rows (Stub) box to create rows. Usually these are ratings or “output” variables, such as satisfaction.
- Now you can add cells of information you want added to your crosstab. Learn more about these options in Available Calculations section.
Qtip: Are some of your cells grayed out? Chances are you need to either recode some of your data or select other options first. See the Available Calculations section for what you need to do.
- You now have a crosstab! You can make edits to this crosstab’s cells, columns, or rows at any time.
Adding New Responses to Crosstabs
As you collect more responses, your crosstabs will need to be recalculated. Click the Settings icon in the top right hand corner and select Import Latest Data to add the new responses to your dataset. Your crosstab will be unavailable while it is recalculating.
Weighting in Crosstabs
You may also want to apply Weighting to your crosstabs data. This can be done based on response weighting or based on one of the existing numeric variables in your data.
- After you’ve applied the desired weighting to your data, navigate to the Crosstabs section.
- Underneath the Weighting section, click the dropdown menu. This will allow you to view the available numeric variables and weights.
Qtip: “Qualtrics Weighting” is the weighting created in the Weighting tab. If you haven’t applied weighting to your data, this variable won’t appear in the crosstabs menu.
- Select the numeric variable you’d like to apply to your data.
Navigating Columns and Rows
Columns (banners) are “input” variables. This includes demographics, such as gender, income, or age. Columns should be variables you are treating as unchanging or independent.
Rows (stubs) are “output” variables. This includes variables that are ratings, like satisfaction, CSAT, CES, NPS, etc. Rows should be variables that you think may change based on conditions in your research.
Above, you see a crosstab composed of the following elements:
- The column “What is your gender?” which splits into male and female.
- The column, “What is the highest level of education you have completed?” which splits into each level of education.
- The row, “How often do you contact our support team?” which splits into each level of frequency.
You can add multiple fields to your row, but you cannot view them all at once; these calculations are separated from each other because each field in a row is run in a separate calculation against the chosen columns. Click on a row to view the calculations for that row.
Nesting columns allows you to have one set of variables splitting another. So instead of having High-Income and Low-Income, and separately USA and Canada, you get High-Income-USA, High-Income-Canada, Low-Income-USA, Low-Income-Canada.
- Add your first variable under Columns (Banner).
- Drag your second column variable over the first one.
- If you did this successfully, the second variable will be nested under the first.
- Add your rows (Stubs).
- Select your cells.
- The columns on the crosstab will split into each possible combination of answers.
Before you can generate certain statistics, such as finding averages, or conducting an ANOVA, you must recode your variables so crosstabs knows they are numeric.
- Click the gear next to a column or banner variable you want to recode.
- Click and drag the dots to the left to reorder choices, if necessary. This helps to make sure your choices are in order of escalation, from least to most.
- Enter the value you want each choice to have.
Qtip: Generally, you want these values to escalate from least to most. Thus in this screenshot, “Extremely often” is 5, and “Never” is 1.
- Click the eye icon to show or hide options from your crosstab. This will remove this option from the corresponding row or column.
- Select Exclude to exclude this choice from analysis. This is most common for “Not Applicable” and “I don’t know” options.
- Click Save to finish.
There are many different kinds of data you can display in crosstabs. Each calculation can be selected in the Cell field after you have configured your columns and rows. In this section, we discuss what every option entails, and what requirements you must meet to use it.
Your column will be treated as categorical, but your row should be numeric or have recode values before you select one of these options. For example, with average selected, marital status added to your column, and CSAT added to your row, you will see the average CSAT broken out by marital status.
Below are the available statistics you can display.
- Standard Deviation
- Standard Error
When selected, the following columns will display the count, which is the number of respondents.
- Total Count: Adds a column that lists the total number of people who responded to both the column and row questions.
- Missing Count: Shows the number of people who answered other parts of the survey, but didn’t answer this question, whether because it was not displayed to them or they skipped it. Missing count and total count will sum together to total the respondents who answered the column question(s). If there are no column questions in the crosstab (meaning just row questions), missing count plus total count will equal the total responses in the crosstab dataset.
- Counts: Shows how many people from each category of the column gave each available answer for the question selected in the row.
- Bucketed Counts: If you have bucketed your selected row, this will show how many people from each category of the column fit into each bucket.
Values are rounded to the nearest one decimal point. Columns add up to roughly 100%.
- Column Percentages (All): Gives the percentage of those in each column category that gave each answer in the selected row. Calculated using the total number of respondents to the survey.
- Bucketed Percentages (All): If you have bucketed your selected row, this gives the percentage that those in each column category fit into each bucket. Calculated using the total number of respondents to the survey.
- Column Percentages (Answering): This is specifically for questions with display logic applied, meaning there are respondents who might not answer the question because they don’t see it, and for multiple-answer questions, where multiple answer choices can be selected per each respondent. Gives the percentage of those in each column category that gave each answer in the selected row. Calculated using the total number of answers provided to the question, instead of total respondents.
Qtip: There is something to keep in mind for Column Percentages (Answering) if you’ve used a multiple-answer question or group as the stub. If any of the choices had choice display logic applied to them, you will not see a Total (Answering) column like usual; instead you will see a discrete value for each of the choices, since the denominator used to calculate them will vary based on the number of respondents who could see the choice.Qtip: There is something to keep in mind for Column Percentages (Answering) if you’ve used a matrix table as the stub. Regardless of whether any matrix questions were hidden via display logic or skipped by the respondent, the Total (Answering) field will display the number of respondents who answered every question. That said, all the calculations are correct – if respondents skipped a question, that stub will not use the Total (Answering) value as its denominator; instead it’ll use the actual number of respondents who answered the question.
- Bucketed Percentages (Answering): This is specifically for questions with display logic applied, meaning there are respondents who might not answer the question because they don’t see it, and for multiple-answer questions, where multiple answer choices can be selected per each respondent. If you have bucketed your selected row, this gives the percentage that those in each column category fit into each bucket. Calculated using the total number of answers provided to the question, instead of total respondents.
Qtip: If Column Percentages (Answering) doesn’t appear as an option (meaning it is not grayed out, but excluded from the list altogether), you can reach out to support to see whether this feature can be enabled for your account.
Overall Stat Test of Percentages
The Overall Stats Test of Percentages acts as a Chi-squared test. A chi-squared statistic tests the relationship between two categorical variables. This test produces a p-value to determine whether the relationship is significant or not. Hover over the p-value in your crosstab to learn whether the test was significant or not.
Example: In the screenshot below, the relationship between gender and satisfaction rating was found to be insignificant.
The Overall Stat Test of Percentages is most useful when your banner is a numeric variable and your stub is a categorical variable. You can configure when a p-value is significant by adjusting the Confidence Level.
Qtip: There are two ways to ensure your stub is categorical:
If you have bucketed a variable and would like to conduct a Chi Square on the bucketed version, select Bucketed Overall Stats Test of Percentages.
Overall Stat Test of Averages
The Overall Stat Test of Averages acts as an Analysis of Variance (ANOVA). An ANOVA tests the relationship between a categorical and a numeric variable by testing the differences between two or more means. This test produces a p-value to determine whether the relationship is significant or not. Hover over the p-value in your crosstab to learn whether the test was significant or not.
You can configure when a p-value is significant by adjusting the confidence level.
Column Stat Tests
Column Stat Tests (All) is a pairwise z-test. Z-tests use the standard deviation to determine if two data samples are different from each other. Z-tests are similar to t-tests, but z-tests are more common where the sample size is larger (generally over 30).
Qtip: Before selecting Column Stat Tests (All), please select Column Percentages (All).
Column Stat Tests can be performed on bucketed variables by selecting Bucketed Column Stats Test (All).
Column Stat Tests (Answering) is also a pairwise z-test. The major difference between (All) and (Answering) is that instead of being based on number of responses, (Answering) is based on number of answers to a question. This is helpful in situations involving display logic, since there are respondents who might not answer the question because they don’t see it, and for multiple-answer questions, where multiple answer choices can be selected per each respondent.
Stat Test of Column Averages
The Stat Test of Column Averages is a pairwise z-test. Z-tests use the standard deviation to determine if two data samples are different from each other. Z-tests are similar to t-tests, but z-tests are more common where the sample size is larger (generally over 30).
In this case, the column averages are being compared.
Interpreting Significance of Pairwise z-tests
This section explains how to interpret the results of the following:
- Stat Test of Column Averages
- Column Stat Tests (All)
- Bucketed Column Stats Test (All)
When values are compared, the set confidence level is used to determine how sure we are that this difference is statistically significant. Every column is compared with one another to determine which column has the higher statistically significant higher value.
In the example above, we look at how respondents of different marital statuses rated how easy it was to apply for vacation days at their shared workplace. We can draw several conclusions from these results.
- In Column A, Divorced respondents, we see the letters B and D on the “Moderately difficult” row. That means Married respondents (B) and Separated respondents (D) were significantly less likely to describe the process as “Moderately difficult” than Divorced respondents.
- In Column B, Married respondents, we see a blank on the “Moderately difficult” row. This does not mean there were no significant results regarding B, but it does mean Column B does not have a significantly higher instance of the “Moderately difficult” rating.
- In the “Extremely difficult” row, there are no letters. This means that no one marital status was more likely than another to rate the process as “Extremely difficult.”
Bucketing allows you to combine choices from previously existing questions into new groups. For example, let’s say you internationally distribute a survey asking what country each respondent lives in. After data collection, you realize you don’t want to do analyses on the countries, but the whole continents. Bucketing would allow you to group each country by continent, so you could analyze your data that way instead.
None of the cells with “Bucketed” in the name will be available until you set up bucketing.
- Click the gear next to a column or row variable you want to recode.
- Select Bucketing in the upper-right.
- Name your desired groups.
- Drag values from the left into the appropriate groups on the right.
- To remove a group click the X next to the name.
- To add a group, click New Group.
- Click Save to finish.
Using Imported Data with Crosstabs
Imported data and embedded data are compatible with crosstabs, but must be added to survey data before you create your first crosstab. Crosstabs are compatible with the Text, Text Set, Number Set, Number, and Filter Only formats of embedded data.