Conversational Feedback

About Conversational Feedback

Qtip: While we have guardrails in place and are continually refining our products, artificial intelligence may at times generate output that is inaccurate, incomplete, or outdated. Prior to using any output from Qualtrics’ AI features, you must review the output for accuracy and ensure that it is suitable for your use case. Output from Qualtrics’ AI features is not a substitute for human review or professional guidance.

Feature Availability

Attention: Conversational feedback is only available to users on the new simplified pricing and packaging plans or customers who actively participated in the Preview Program for this feature. For more information, see Qualtrics Public Preview. If you have questions about your Qualtrics pricing and packaging plan, reach out to Account Services.

Adaptive follow-up questions can be used in all projects that support text entry questions, except for 360 and adhoc survey projects.

Qtip: Only 1 follow-up question is supported for Engagement and Lifecycle projects. 

Response clarity questions can be used in all projects that support text entry questions, including  360 and adhoc survey projects.

Adaptive Follow-Up Questions

Adaptive follow-up questions are created instantly using artificial intelligence, tailored to the feedback your respondents write in the open text.

Qtip: Survey takers can only receive up to 2 advanced follow-ups per survey. If a survey taker has already been asked a question with 2 follow-ups enabled, then when they get to the next question with an advanced follow-up, the follow-up will not trigger. However, basic follow-ups (first follow-up, in English-only, with no topics to avoid) are unlimited.

Enabling Adaptive Follow-Up

Qtip: For help enabling this feature, reach out to your Brand Administrator.

To use this feature, you must have the AI Adaptive Followup – Advanced third party organization setting enabled. Once the organization setting is enabled, you must have either the AI Adaptive followup – Basic or the AI Adaptive followup – Advanced user permission.

With advanced adaptive follow-up enabled, you can set up 2 follow-up questions instead of just 1. You can also add topics that you want the AI to avoid when generating follow-up questions.

Qtip: Currently, adaptive follow-up can be used for survey respondents taking a survey in the following languages: English (EN and EN-GB), French (FR), German (DE), Japanese (JA), Brazilian Portuguese (PT-BR), Spanish (ES and ES-ES), Dutch (NL), and Chinese (Traditional [ZH-T] and Simplified [ZH-S]). For non-English languages, the AI Adaptive followup – Advanced permission must be enabled, and only 1 follow-up question is supported.

Setting Up Adaptive Follow-Up

Example: Adaptive follow-up questions work best when asking a specific, detailed text entry question. For example, the question “How was your experience at Qualtrics.com” will provide a better opportunity for follow-up questions than “Tell us more”.

Select your text entry question.

---

×

close

Look under Conversational feedback.

Switch on Adaptive follow-up questions.

Enter the name of your company. 

---

×

close

If you’d like to ensure your company name is not displayed to respondents during the survey, select Omit company name from survey questions.

Qtip: If the company name is already being displayed elsewhere in the survey, it may still be included in generated follow-up questions.

Select the maximum number of follow-up questions a respondent can receive. You can select up to 2.

Qtip: This setting is only available for advanced follow-up in English.

Enter any topics you would like to avoid.

Attention: This setting is only available for advanced follow-up. 

Qtip: Do not add more than one topic to each topic line. Topics can be phrases describing what you would like to avoid, and are not limited to key words.

Click Add topic to add more topics. 

Qtip: If you’d like to delete a topic, click the minus sign ( – ) next to that topic.

Click Confirm.

To apply display logic to the follow-up question, click Display logic.

---

×

close

To edit your follow-up question settings, click Configure.

Attention: You can only show 1 follow-up question per page. If you add a second follow-up question, the system will automatically add a page break, even if it doesn't look like it in the editor.

Qtip: All Javascript tags and images are ignored by the AI when providing a follow up question.

Video Walkthrough

Need help getting started with adaptive follow-up? Check out this 2-minute video walkthrough:

For questions and discussions about adaptive follow-up, visit the Qualtrics Experience Community AI Center, where you can connect with other Qualtrics users about all things AI.

Response Clarity Validation

When enabled on a text entry question, response clarity validation analyzes the respondent’s answer using AI, and prompts them to give a fuller response if their answer is vague or only partially addresses the question. This prompt will appear when the respondent clicks the Next button to continue to the next page in your survey.

At this time, response clarity validation can only be used for surveys with English (EN, EN-GB), Spanish (ES, ES-ES), French (FR), German (DE), Japanese (JA), Portuguese (PT, PT-BR), Chinese (ZH-T, ZH-S) or Dutch (NL) as the base language.

---

×

close

Qtip: This prompt only appears a maximum of 1 time per question it is enabled on. If the respondent’s answer is still not clear after seeing this prompt, they will not see it again and will be allowed to continue the survey.

Enabling Response Clarity Validation

Qtip: For help enabling this feature, reach out to your Brand Administrator.

To use this feature, you must have the Response Clarity first party organization setting enabled. Once the organization setting is enabled, you must have  the Response Clarity user permission.

Setting Up Response Clarity Validation

Qtip: You cannot use this option with the password question variation.

Select a text entry question.

---

×

close

Look under Conversational feedback.

Choose Response clarity validation.

To customize the prompts survey takers see, click Configure.

There are 5 different criteria that are used to evaluate the response’s clarity:
 

---

×

close

• Partial answer: The respondent’s answer did not fully answer the question or only answered parts of it.
• Too general answer: The respondent’s answer was too vague.
• Not insightful answer: The respondent’s answer does not provide any insights when it is reasonable to have an opinion on the question asked.
• Lacks details: The respondent’s answer did not provide any meaningful details.
• Unintelligible answer: The respondent’s answer does not make sense or does not contain real words.

Select Default prompt if you’d like to use the default prompt messages for a given criteria. The default messages are:
 

---

×

close

• Partial answer:
  • We appreciate your response. Can you answer all parts of the question.
  • That’s helpful. Can you review the question and answer all parts?
  • Thanks for the feedback. Can you add additional information to fully answer the entire question?
• Too generalized answer:
  • We’d like to hear more about that. Can you add additional detail to your response?
  • Can you please explain that in a bit more detail?
  • Can you tell me more about that? What makes you say X?
• Non-insightful answer:
  • We value your opinion. Is there anything else you can share?
  • Your input is valuable. Anything else to add?
  • We’d like to get more specifics. Is there anything else you’d like to add?
• Lacks details:
  • This is helpful. Can you tell us a little more about that?
  • Can you provide more detail about your answer or give an example?
  • We are curious to hear more, can you expand on that?
• Unintelligible answer:
  • Your response was hard to understand. Can you try answering the question again?
  • Your response is unclear. Could you clarify what you mean?
  • Your response is important to us. Can you try answering the question again?

Qtip: When a response fits into one of these criteria, only 1 of the possible prompt messages will be displayed to the respondent.

To use custom prompts for a criteria, enable the Custom prompt option.
 

---

×

close

Use the Select prompt menu to choose a message from any messages library you have access to. You can use any general message as the prompt. Select New Message to create a new message instead.

You can add up to 3 possible messages for a criteria by clicking Add custom prompt.

Qtip: When a response fits into one of these criteria, only 1 of the possible prompt messages will be displayed to the respondent.

To remove a custom prompt, click the minus sign ( – ) next to it.

To disable prompt messages for a certain criteria entirely, select Deactivate for that criteria.
 

---

×

close

When finished, click Confirm.

Data Output for Conversational Feedback 

Adaptive Follow-Up Data

The following columns will be added to your dataset for each adaptive follow-up question:
 

---

×

close

• Q# – {Question Text} – Follow-Up Transcript: The source question, source response, follow-up questions, and follow-up question responses.

  

  Qtip: Display this data in a response ticker widget to follow the conversation, allowing you to read each response at a time.

• Q# – {Question Text} – Follow-Up Concatenated Answers: The source question response and follow-up response, combined together into a single text response.

  

  Qtip: Analyze this field for Text iQ topics and text enrichments.

• Q# = {Question Text} – Follow-Up Question: The follow-up question that was created in response to the answer of text entry data.
• Q# = {Question Text} – Follow-Up Answer: The response to the follow-up question that was generated.

Termination Reasons

A column with the name Termination_Reason is added to Data & Analysis for each question that has adaptive follow-up enabled. If a respondent receives a follow-up question, this column will be empty. If they do not receive a follow-up question, this column will contain one of the following termination reasons:

• NotEligibleForFollowUp: The response was too vague or low quality to receive a follow-up question.
• NoFollowUpNeeded: The response was sufficient and does not need a follow-up.
• FollowUpNotAvailable: The network timed out or the model could not provide a follow-up.

Attention: Adaptive follow-up only triggers if a respondent provides a sufficiently long and relevant answer. No follow-up questions are asked if there is not enough information in a respondent's answer.

Response Clarity Embedded Data

When you enable response quality validation on a question, there are additional embedded data fields available in your dataset to help track response clarity. These fields include:

• <QID>_AICategory: The category the original response was classified as. Possible values include:
  • Passed – Good Answer: The respondent’s answer was sufficient and did not trigger response clarity validation.
  • Passed – Not Applicable: The respondent understood the question and put in an effort to answer it, but the question did not apply to them
  • AIError: There was an error analyzing the response.
  • AITimeout: The AI analysis request timed-out.
  • UnsupportedLanguage: The response was in an unsupported language.
  • Gibberish: The response matched the “unintelligible answer” criteria.
  • Insufficient Detail: The response matched the “lacks detail” criteria.
  • Non-Insightful: The response matched the “non-insightful answer” criteria.
  • Overly Generalized Answer: The response matched the “too generalized answer” criteria.
  • Partial Answer: The response matched the “partial answer” criteria.
• <QID>_OriginalResponse: The user’s original response.
• <QID>_FollowUpPrompt: The exact prompt that was displayed to the user. This will be blank if the response passed the response clarity validation without triggering it.

Qtip: In the above embedded data field names, <QID> will be replaced with the question’s internal question ID.