Pairwise Comparisons texts: How to? | XM Community
Solved

Pairwise Comparisons texts: How to?

  • 28 August 2018
  • 3 replies
  • 361 views
T
For research into the readability of texts, I want participants to compare multiple sets of two texts. They have to state which text is easier/harder to read. I have a database of 25 texts, all these text have to be compared to each other. Since it will be a lot of work to create all the sets by hand, I was wondering if such a thing is possible in Qualtrics.

I was thinking of putting all the texts separately in Qualtrics, choose: present 2 of the elements, followed by a randomizer which presents all of the elements evenly. However, I was wondering if there is an option to present all the possible sets evenly. Because if text 1 ... text 25 are all presented i.e. 10 times, it doesn't mean that the combination 1 - 2 and 1 - 25 are presented the same amount of times. Or does it?

It would be great if anyone could help me out!
icon

Best answer by TomG 28 August 2018, 15:41

View original

3 replies

PeeyushBansal
Rank
Userlevel 7
Badge +33
You can do so by conjoint by making possible combinations and then presenting 2 out of these at a time.

https://www.qualtrics.com/support/conjoint-project/getting-started-conjoints/getting-started-with-conjoint-projects/
Best Regards, Peeyush
T
The texts consist of 300 words each. Is there any possibility to dothis in 'Research Core'?
TomG
Rank
Userlevel 7
Badge +27
I think you can get close enough (i.e. you'll end up with a fairly even distribution of pairs) by putting your 25 texts as choices in an MC question and using advanced randomization to randomly display 2 of the 25 and evenly present them. If you want to ask each respondent about multiple pairs, put your MC question in a loop & merge block and loop for as many times as you want.

This is easy enough to set up, so you could quickly generate some test data then download it with the "viewing order data for randomized surveys" and look at the distribution of pairs.

A Plan B could be to use the survey flow randomizer to evenly present all 300 pairs (24+25)/2, but it would be much more tedious to set up and the resulting data would be more difficult to work with because you would be piping the choices.

Leave a Reply


Terms & Conditions