Sentiment Analysis

About Sentiment Analysis

Qualtrics will assign a Positive, Negative, Neutral, or Mixed sentiment to a text response as soon as it is loaded in Text iQ. This sentiment is based off of the language in the response, the question text itself, and edits you’ve made to your sentiment analysis. Once assigned, each sentiment comes with a polarity and a numeric score.

Qualtrics’ sentiment analysis is constantly improving. Our analysis tool periodically runs a batch analysis on all manual corrections made by users to improve the accuracy of the model, assess improvements, and correct mistakes that Text iQ is making in sentiment identification.

These sentiment measures can be used to filter your data or be displayed in visualizations in the Reports tab.

Sentiment Functionality

For most Text iQ analysis, the question text is not relevant to the sentiment of the answer (e.g. “Why did you give us that score?”). However, occasionally the framing of the question will imply a sentiment for the answer (e.g. “What’s one thing we could improve about our product?”). In these cases, Text iQ needs both the question and the answer to make an accurate prediction of sentiment. For example, if the question was, “What do you love about our company?” then unless the answer is explicitly negative (e.g. “Nothing, you suck in every way”) then the sentiment of the response is probably positive, even if the answer text on its own appears neutral (e.g. “pricing”). This Sentiment Analysis update incorporates the question text, if available, in the sentiment analysis so that the predicted sentiment is as accurate as possible.

This model also now takes sentiment edits into account in order to learn how to better assign sentiments in the future. However, please note that these edits do not immediately affect the sentiment assignment of projects in your account. Instead, these edits are periodically analyzed across all Qualtrics users and used to improve the sentiment analysis model as a whole.

Overall Sentiment vs. Topic Sentiment

When Qualtrics analyzes your text responses, there are two different sentiment scores that can be assigned. These scores are Overall Sentiment and Topic Sentiment. Understanding the difference between these two sentiment scores is important in understanding your text analysis data.

Overall Sentiment is the sentiment score for a given response. Every response analyzed in Text iQ will have only one Overall Sentiment score.

Topic Sentiment is the sentiment score of a particular topic in your text response. Responses can have multiple Topic Sentiment scores as each topic is assigned its own score.

Overall sentiment, overall sentiment score, topic-level sentiment, and topic-level sentiment score are included in response exports. Overall sentiment is labelled just as “Sentiment” and “Sentiment-Score.” Topic-level is labelled “Topic Sentiment Label” and “Topic Sentiment Score.” Topic-level fields can have multiple values (to account for multiple topics) and are listed as Topic: Sentiment. E.g., Dessert: Negative. Topics are also included in response exports.

Viewing and Changing Sentiment

1. Make sure comment mode is opened. You will see a green checkmark in the upper-right, but you will not see the Add Widget button.
   
2. Navigate to your responses. There will be a bubble to the left of each indicating sentiment. The number is the Sentiment Score, and the color is the sentiment  itself.
   • Pink: Negative.
   • Yellow: Mixed.
   • Gray: Neutral.
   • Turquoise / Teal: Positive.
3. Click edit button.
4. Note that you are editing the Overall Sentiment of the entire response.
   
5. Select a sentiment.
6. Select a sentiment score.
   Qtip: If you select a score and sentiment that do not match, the system will inform you so you can adjust as needed.

Topic Sentiment

When you select Topic Sentiment, you can specify the sentiment for each topic tagged on the response. This is especially helpful if your overall sentiment is mixed, or if the intensity of the sentiment varies for each topic the respondent covered.

Click a topic’s name to expand the options and select sentiment and sentiment score.

In this screenshot, the respondent likes the burgers at the restaurant but doesn’t care for the ice cream, so we want to make sure the topic “burgers” has a positive sentiment, while the topic “desserts” does not. “Food” would be mixed.

Filtering by Sentiment

1. Inside the Data & Analysis tab or the Reports tab, click Add Filter.
   
2. Select Embedded Data.
3. Select the sentiment you are interested in. They will be labeled by question number.
4. Choose which type of sentiment you are interested in filtering by (Mixed, Negative, Neutral, or Positive).
   

Sentiment Score and Polarity

In addition to labelling a comment positive, negative, mixed, or neutral, there are two numeric values assigned to each sentiment.

The Sentiment Score is a -10 to +10 score for the sentiment of a comment. For example, a very intense, positive response such as “I love Love LOVE YOUR COMPANY!” would score a +10. An intensely negative response, such as, “I UTTERLY DESPISE your company!” would get a -10. A 0 is a neutral score.

The Sentiment Polarity is a 0 to 10 score for the polarity of the comment. This is a calculation of how mixed the comment is. For extremely positive and negative Sentiment Scores (like the previous examples), there’s usually a corresponding 0 polarity. But for a comment such as “I love your food, but your restaurant is so dirty,” you will get a more nuanced score.

The Sentiment Score and Polarity can be found with the Sentiment itself under Data & Analysis and in the Reports tabs. Like most text analysis variables, are under the Embedded Data category.

Multi-Language Sentiment Analysis

Sentiment analysis is currently available in languages other than English, including: Chinese (Simplified), Chinese (Traditional), French, German, Italian, Japanese, Korean, Portuguese, Dutch, Thai, Indonesian, and Spanish.

Analyzing sentiment in other languages is available to all users who have Sentiment Analysis, and thus all Advanced Text clients.

When Multi-Language Sentiment Analysis is enabled, Qualtrics will detect the language of the response. So long as it is one of the languages listed above, this response will have sentiment analysis performed on it in its native language, capturing the nuances of the response in ways a translation may not. If this feature is not enabled, the sentiment analysis will not detect the language of the response and will assign sentiments as if it were written in English. Thus, when Multi-Language Sentiment Analysis is disabled, the sentiments assigned to responses not written in English are not as accurate.

FAQ