Loading...
  • Customer Experience
    Customer Experience
  • Employee Experience
    Employee Experience
  • Brand Experience
    Brand Experience
  • Product Experience
    Product Experience
  • Core XM
    Core XM

Sentiment Analysis

What's on This Page:


Was this helpful?


The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!


About Sentiment Analysis

Qualtrics will assign a Positive, Negative, Neutral, or Mixed sentiment to a text response as soon as it is loaded in Text iQ. This sentiment is based off of the language in the response, the question text itself, and edits you’ve made to your sentiment analysis. Once assigned, each sentiment comes with a polarity and a numeric score.

Qualtrics’ sentiment analysis is constantly improving. Our analysis tool periodically runs a batch analysis on all manual corrections made by users to improve the accuracy of the model, assess improvements, and correct mistakes that Text iQ is making in sentiment identification.

These sentiment measures can be used to filter your data or be displayed in visualizations in the Reports tab.

Qtip: Sentiment Analysis is only available to Advanced Text clients.

 

Sentiment Functionality

For most Text iQ analysis, the question text is not relevant to the sentiment of the answer (e.g. “Why did you give us that score?”). However, occasionally the framing of the question will imply a sentiment for the answer (e.g. “What’s one thing we could improve about our product?”). In these cases, Text iQ needs both the question and the answer to make an accurate prediction of sentiment. For example, if the question was, “What do you love about our company?” then unless the answer is explicitly negative (e.g. “Nothing, you suck in every way”) then the sentiment of the response is probably positive, even if the answer text on its own appears neutral (e.g. “pricing”). This Sentiment Analysis update incorporates the question text, if available, in the sentiment analysis so that the predicted sentiment is as accurate as possible.

This model also now takes sentiment edits into account in order to learn how to better assign sentiments in the future. However, please note that these edits do not immediately affect the sentiment assignment of projects in your account. Instead, these edits are periodically analyzed across all Qualtrics users and used to improve the sentiment analysis model as a whole.

Qtip: This section describes sentiment analysis performed on English responses. Multi-Language Sentiment functions differently because it uses a Third-Party software.

Overall Sentiment vs. Topic Sentiment

When Qualtrics analyzes your text responses, there are two different sentiment scores that can be assigned. These scores are Overall Sentiment and Topic Sentiment. Understanding the difference between these two sentiment scores is important in understanding your text analysis data.

Overall Sentiment is the sentiment score for a given response. Every response analyzed in Text iQ will have only one Overall Sentiment score.

Topic Sentiment is the sentiment score of a particular topic in your text response. Responses can have multiple Topic Sentiment scores as each topic is assigned its own score.

Qtip: You can edit both the Overall Sentiment and the Topic Sentiments for your responses. Read the Viewing and Changing Sentiment section for more information.
Example: Let’s say you receive a survey response that reads, “The service and the food were excellent, but everything was very overpriced.” If this response is tagged with the topics Service, Food, and Price, then the Topic Sentiments of each topic are Service=Positive, Food=Positive, Price=Negative. The Overall Sentiment is Positive as the response was mostly Positive.

Overall sentiment, overall sentiment score, topic-level sentiment, and topic-level sentiment score are included in response exports. Overall sentiment is labelled just as “Sentiment” and “Sentiment-Score.” Topic-level is labelled “Topic Sentiment Label” and “Topic Sentiment Score.” Topic-level fields can have multiple values (to account for multiple topics) and are listed as Topic: Sentiment. E.g., Dessert: Negative. Topics are also included in response exports.

Screenshot of data where it has the columns Q1 - Sentiment Score; Q1 - Sentiment; Q1 . - Topics, and then comma-separated topics, such as All Food, Burgers, Dessert; Q1 - Topic Sentiment Label, then the aforeentioned values in that column such as Dessert: Negative, All Food: Neutral, etc; and finally, a column for Q1 - Topic Sentiment Score, where values are comma-separated such as Dessert: -1, All Food: 0, etc.

Viewing and Changing Sentiment

  1. Make sure comment mode is opened. You will see a green checkmark in the upper-right, but you will not see the Add Widget button.
    Text iQ interface
  2. Navigate to your responses. There will be a bubble to the left of each indicating sentiment. The number is the Sentiment Score, and the color is the sentiment  itself.
    • Pink: Negative.
    • Yellow: Mixed.
    • Gray: Neutral.
    • Turquoise / Teal: Positive.
  3. Click edit button.
  4. Note that you are editing the Overall Sentiment of the entire response.
    New window opened. LEft is response and list of topics. Right is sentiment
  5. Select a sentiment.
  6. Select a sentiment score.
    Qtip: If you select a score and sentiment that do not match, the system will inform you so you can adjust as needed.

Topic Sentiment

When you select Topic Sentiment, you can specify the sentiment for each topic tagged on the response. This is especially helpful if your overall sentiment is mixed, or if the intensity of the sentiment varies for each topic the respondent covered.

Right of new window has sentiment. Topic sentiment select top of this right side, then a list of topics with sentiments that can be edited underneath

Click a topic’s name to expand the options and select sentiment and sentiment score.

In this screenshot, the respondent likes the burgers at the restaurant but doesn’t care for the ice cream, so we want to make sure the topic “burgers” has a positive sentiment, while the topic “desserts” does not. “Food” would be mixed.

Filtering by Sentiment

  1. Inside the Data & Analysis tab or the Reports tab, click Add Filter.
    Option to filter by Embedded Data called Sentiment in the Add Filter dropdown
  2. Select Embedded Data.
  3. Select the sentiment you are interested in. They will be labeled by question number.
  4. Choose which type of sentiment you are interested in filtering by (Mixed, Negative, Neutral, or Positive).
    Select Operand dropdown for filtering by sentiment

Sentiment Score and Polarity

In addition to labelling a comment positive, negative, mixed, or neutral, there are two numeric values assigned to each sentiment.

The Sentiment Score is a -10 to +10 score for the sentiment of a comment. For example, a very intense, positive response such as “I love Love LOVE YOUR COMPANY!” would score a +10. An intensely negative response, such as, “I UTTERLY DESPISE your company!” would get a -10. A 0 is a neutral score.

The Sentiment Polarity is a 0 to 10 score for the polarity of the comment. This is a calculation of how mixed the comment is. For extremely positive and negative Sentiment Scores (like the previous examples), there’s usually a corresponding 0 polarity. But for a comment such as “I love your food, but your restaurant is so dirty,” you will get a more nuanced score.

The Sentiment Score and Polarity can be found with the Sentiment itself under Data & Analysis and in the Reports tabs. Like most text analysis variables, are under the Embedded Data category.

Sentiment, sentiment score, and sentiment polarity columns on the Data section

Qtip: Use Choose Columns to bring these variables onto the Data & Analysis tab if you haven’t already!

Sentiment, sentiment score, and sentiment polarity in the sidebar of the Results section

Sentiment, sentiment score, and sentiment polarity in the sidebar of the Reports section

Multi-Language Sentiment Analysis

Sentiment analysis is currently available in languages other than English, including: Chinese (Simplified), Chinese (Traditional), French, German, Italian, Japanese, Korean, Portuguese, Dutch, Thai, Indonesian, and Spanish.

Analyzing sentiment in other languages is available to all users who have Sentiment Analysis, and thus all Advanced Text clients.

Qtip: Although this feature is available to all sentiment users, it is turned off by default. That is because we use Google software to perform sentiment analysis on these responses in their native languages. If you would like this feature enabled for you and are comfortable sharing your response data with a third party software, please contact your Brand Administrator and ask them to enable the Allow Third-Party iQ Analysis permission for the brand. Note that this is a brand-wide setting, so your administrator will likely decide based on what’s best for all users in the brand.

Organization settings tab of Admin page. Checkbox down the page that says Allow third party services in text iQ

When Multi-Language Sentiment Analysis is enabled, Qualtrics will detect the language of the response. So long as it is one of the languages listed above, this response will have sentiment analysis performed on it in its native language, capturing the nuances of the response in ways a translation may not. If this feature is not enabled, the sentiment analysis will not detect the language of the response and will assign sentiments as if it were written in English. Thus, when Multi-Language Sentiment Analysis is disabled, the sentiments assigned to responses not written in English are not as accurate.

FAQ