Product Concept Testing with more than 3 concepts? | XM Community
Solved

Product Concept Testing with more than 3 concepts?

  • 13 September 2018
  • 9 replies
  • 21 views
M
Badge
Let's say I have a flat concept that I want to test, and there are 6 of them.
If I want to create a serial monadic design (meaning testing only a subset of these together), is it possible to only create two groups, such as:
ABC vs DEF?
Or what's the most affordable way to test this many concepts? Ideas?
icon

Best answer by PeeyushBansal 13 September 2018, 20:36

View original

9 replies

PeeyushBansal
Rank
Userlevel 7
Badge +33
You have to use randomizer for selecting 2 out of these and than comparing these with each other.
Best Regards, Peeyush
M
Badge
@bansalpeeyush29 so that part makes sense, I was more asking about whether or not it makes sense to only do ABC vs. DEF so that it is statistically valid.
PeeyushBansal
Rank
Userlevel 7
Badge +33
Statisticaly it make sense for mondaic design.
Best Regards, Peeyush
Priyanka_jain
Rank
Userlevel 1
Badge +1
We have recently done a mondaic study this is the best approch to make two groups ABC and DEF.
M
Badge
One issue I have found with this is that do you get any statistical effect where because the group ABC has not seen designs DEF, can you directly compare soft metrics for all?

For example, let's say you have a metric such as:
"This concept is appealing" and there are different responses for all 6 concepts.
Can you truly say that "A" was rated statistically more appealing than "F"?
Priyanka_jain
Rank
Userlevel 1
Badge +1
Yes these responses can be significantly compared with each other for all your 6 concepts.
M
Badge
OK so I guess what I could do is:
Set up study to randomize questions, and then assign 3.
What is a likely sample size for such an effort, where I can still segment by demographic information? I'm trying to figure out what is the best way to identify the sample size.
Priyanka_jain
Rank
Userlevel 1
Badge +1
We had 100 for each distributed equaly actoss gender and age.
M
Badge
@Priyanka_jain OK so let's say you have n=500 total. You want to show each person 3 concepts, which ends up being about 250 per concept.
At the end of the survey, you want to ask each person "of the 3 concepts that you saw, which one did you prefer the most?"

How would you go about analyzing that data? In this case, there would be 20 total combinations of concepts. Is there a simple way to analyze the preference data so that you can say "X concept was preferred the most?"

Leave a Reply


Terms & Conditions