Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM
  • Design XM
    Design XM

Sentiment Analysis

What's on This Page:


Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Sentiment Analysis

Qualtrics will assign a Very Negative, Negative, Neutral, Positive, Very positive or Mixed sentiment to a text response as soon as it is loaded in Text iQ. This sentiment is based off of the language in the response, the question text itself, and edits you’ve made to your sentiment analysis. The Text iQ model is trained on a large and diverse set of real experience data to substantially increase classification quality and minimize uncertainties when the sentiment is not expressed as strongly. Once assigned, each sentiment comes with a numeric score, the sentiment score.

Qualtrics’ sentiment analysis is constantly improving. Our analysis tool periodically runs a batch analysis on all manual corrections made by users to improve the accuracy of the model, assess improvements, and correct mistakes that Text iQ is making in sentiment identification. The Very Negative and Very Positive sentiment labels have been implemented to indicate the strongest sentiment, helping you to focus on the most critical feedback in each comment.

These sentiment measures can be used to filter your data or be displayed in visualizations in the Reports tab.

Qtip: Sentiment Analysis is only available to Advanced Text clients.
Qtip: If you would like to branch based on a sentiment or topic in your Text iQ analysis, see Text iQ-Powered Survey Flows.

Sentiment Functionality

For most Text iQ analysis, the question text is not relevant to the sentiment of the answer (e.g. “Why did you give us that score?”). However, occasionally the framing of the question will imply a sentiment for the answer (e.g. “What’s one thing we could improve about our product?”). In these cases, Text iQ needs both the question and the answer to make an accurate prediction of sentiment. For example, if the question was, “What do you love about our company?” then unless the answer is explicitly negative (e.g. “Nothing, we hate your company”) then the sentiment of the response is probably positive, even if the answer text on its own appears neutral (e.g. “pricing”). This Sentiment Analysis update incorporates the question text, if available, in the sentiment analysis so that the predicted sentiment is as accurate as possible.

This model also now takes sentiment edits into account in order to learn how to better assign sentiments in the future. However, please note that these edits do not immediately affect the sentiment assignment of projects in your account. Instead, these edits are periodically analyzed across all Qualtrics users and used to improve the sentiment analysis model as a whole.

Qtip: This section describes sentiment analysis performed on English responses. Multi-Language Sentiment functions differently because it uses a Third-Party software.

Overall Sentiment vs. Topic Sentiment

When Qualtrics analyzes your text responses, there are two different sentiment scores that can be assigned. These scores are Overall Sentiment and Topic Sentiment. Understanding the difference between these two sentiment scores is important in understanding your text analysis data.

Overall Sentiment is the sentiment score for a given response. Every response analyzed in Text iQ will have only one overall sentiment score.

Topic Sentiment is the sentiment score of a particular topic in your text response. Responses can have multiple Topic Sentiment scores as each topic is assigned its own score.

Qtip: You can edit both the Overall Sentiment and the Topic Sentiments for your responses. Read the Viewing and Changing Sentiment section for more information.
Example: Let’s say you receive a survey response that reads, “The service and the food were excellent, but everything was very overpriced.” If this response is tagged with the topics Service, Food, and Price, then the Topic Sentiments of each topic are Service=Positive, Food=Positive, Price=Negative. The Overall Sentiment is Positive as the response was mostly Positive.

Overall sentiment, overall sentiment score, topic-level sentiment, and topic-level sentiment score are included in response exports. Overall sentiment is labelled just as “Sentiment” and “Sentiment-Score.” Topic-level is labelled “Topic Sentiment Label” and “Topic Sentiment Score.” Topic-level fields can have multiple values (to account for multiple topics) and are listed as Topic: Sentiment. E.g., Dessert: Negative. Topics are also included in response exports.

Screenshot of data where it has the columns Q1 - Sentiment Score; Q1 - Sentiment; Q1 . - Topics, and then comma-separated topics, such as All Food, Burgers, Dessert; Q1 - Topic Sentiment Label, then the aforeentioned values in that column such as Dessert: Negative, All Food: Neutral, etc; and finally, a column for Q1 - Topic Sentiment Score, where values are comma-separated such as Dessert: -1, All Food: 0, etc.

Viewing and Changing Sentiment

  1. Make sure comment mode is opened. You will see a blue checkmark in the upper-right, but you will not see the Add Widget button.Navigating to the sentiment analysis in Text iQ by opening comment mode and navigating to specific comments.
  2. Navigate to your responses. There will be a diamond to the left of each indicating sentiment. This represents the sentiment score. Sentiment score key, with diamonds showing Very Negative, Negative, Mixed, Positive, Very Positive, and Neutral
    • Solid Red: Very Negative.
    • Half Red: Negative.
    • Purple with a line in the middle: Mixed.
    • Half Blue: Positive.
    • Solid Blue: Very Positive.
    • White with Gray Outline: Neutral.
  3. Click the edit comment button.
  4. Navigate to the Comment Overall Sentiment section. The comment overall section, which shows the comment overall scores
    Qtip: Note that you are editing the Comment Overall Sentiment of the entire response.
  5. Select a sentiment score.

Topic Sentiment

When you select Topic Sentiment, you can specify the sentiment for each topic tagged on the response. This is especially helpful if your overall sentiment is mixed, or if the intensity of the sentiment varies for each topic the respondent covered.

Topic sentiment score in the Edit Comment window

You will be able to select sentiment and sentiment score under each topic name.

In the above screenshot, the respondent loves the food at the restaurant but believes that the staff is overworked, so we want to make sure the topic “All Food” has a positive sentiment, while the topic “Workforce” does not. The overall sentiment is mixed due to these two topic sentiments interacting with each other. Suggested topic tags for the comment

You also have the option to add additional topic tags and sentiments directly on the response by selecting suggested topics in the top right corner or by creating your own topic with the blue plus (+) sign.

Filtering by Sentiment

  1. Inside the Data & Analysis tab or the Reports tab, click Add Filter. Adding a sentiment filter in the Data & Analysis
  2. Select Embedded Data.
  3. Select the sentiment you are interested in. They will be labeled by question number.
  4. Choose which type of sentiment you are interested in filtering by (Negative, Neutral, Positive, Very Negative, or Very Positive).Choosing which sentiment to filter by: Negative, Neutral, Positive, Very Negative, Very Positive.

Sentiment Score

In addition to labelling a comment positive, negative, mixed, or neutral, there will be a numeric value assigned to each sentiment.

The Sentiment Score is a -2 to +2 score for the sentiment of a comment. For example, a very intense, positive response such as “I love Love LOVE YOUR COMPANY!” would score a +2. An intensely negative response, such as, “I UTTERLY DESPISE your company!” would get a -2. 0 is a neutral score.

The Sentiment Polarity is a 0 to 10 score for the polarity of the comment. This is a calculation of how mixed the comment is. For extremely positive and negative sentiment scores (like the previous examples), there’s usually a corresponding 0 polarity. But for a comment such as “I love your food, but your restaurant is so dirty,” you will get a more nuanced score.

Attention: Once you have opted-in to the 5-sentiment model of Text iQ (very negative to very positive), you will no longer see the sentiment polarity functionality.

The sentiment score and polarity can be found with the sentiment itself under the Data & Analysis and in the Reports tabs. Like most text analysis variables, you will find them under the embedded data category.

Sentiment score column in the Data & Analysis

Qtip: Use Choose Columns to bring these variables into the Data & Analysis tab if you haven’t already!
Sentiment data in the Results-Reports tab Sentiment data shown in the Reports-Reports tab

Multi-Language Sentiment Analysis

Sentiment analysis is currently available in languages other than English, including: Chinese (Simplified), Chinese (Traditional), French, German, Italian, Japanese, Korean, Portuguese, Dutch, Thai, Indonesian, and Spanish.

Analyzing sentiment in other languages is available to all users who have Sentiment Analysis, and thus all Advanced Text clients.

Allow third-party services in Text iQ permission is highlighted in the Admin section, under the Organization Settings tab

Qtip: Although this feature is available to all sentiment users, it is turned off by default. That is because we use Google software to perform sentiment analysis on these responses in their native languages. If you would like this feature enabled for you and are comfortable sharing your response data with a third party software, please contact your Brand Administrator and ask them to enable the Allow Third-Party iQ Analysis permission for the brand. Note that this is a brand-wide setting, so your administrator will likely decide based on what’s best for all users in the brand.

When Multi-Language Sentiment Analysis is enabled, Qualtrics will detect the language of the response. So long as it is one of the languages listed above, this response will have sentiment analysis performed on it in its native language, capturing the nuances of the response in ways a translation may not. If this feature is not enabled, the sentiment analysis will not detect the language of the response and will assign sentiments as if it were written in English. Thus, when Multi-Language Sentiment Analysis is disabled, the sentiments assigned to responses not written in English are not as accurate.

FAQ