Randomization and Quotas of Scenarios | XM Community
Solved

Randomization and Quotas of Scenarios

  • 19 December 2018
  • 7 replies
  • 282 views
J
Hi together,
I have a technical question about scenario randomization.
I have 4 Scenarios which are randomized and equally shown to participants. After that the participants need to fill in the "Manipulation Check".
If they answer one of these 3 questions wrongly, they are screened out.
So I applied the Branch Logic when participants are screened out after the respective scenario and set "End of Survey".
Within the "End of Survey" I customized the settings and set "do not increment the quota counts".

Here is the problem:
I need 40 participant per scenario without counting the screened out ones.
Currently the quota still increases and counts the screened out participants, e.g.
Scenario 1 was shown 24 times but only 10 are not screened out
Scenario 2 was shown 25 times but only 21 are not screened out
Scenario 3 was shown 27 times but only 14 are not screened out
Scenario 4 was shown 29 times but only 17 are not screened out

How can I assure that each scenario is shown that many times that I have 40 not screened out answers per scenario? Can this be done by quotas?

Thanks for your help!

!
!
icon

Best answer by bstrahin 20 December 2018, 16:24

View original

7 replies

bstrahin
Rank
Userlevel 7
Badge +38
@JaJu I assume your quotas are set to count if a question in the block is shown. I would make sure the condition on tallying your quotas is if a question in the scenario block is shown AND Q_TerminateFlag is not True. That should delay your quota tallying until they make it past the screener.
J
@bstrahin that makes sense! Please a further question on this.
I set the counts to the current non-screened numbers and adjusted the quotas as you suggested.
If I now want to say, as soon as the quota is full, show the other scenarios.
Is ist correct then to set the setting "When the quota has been met, then do not display the block e.g. Scenario 1"? Or should I set it to "When the quota has been met, then do None (For Skip logic or Survey Flow)"/ "This quota can be evaluated with Display Logic, or the Survey Flow"?
I assume "End current survey" would produce the wrong result

!

Thanks so much!
SaurabhPujare_Ugam
Rank
Userlevel 6
Badge +18
@JaJu

Answer for your second question,

you can use the later option "When the quota has been met, then do None (For Skip logic or Survey Flow)" and set the branch condition on block stating if Quota Scenario 1 has not been met.
Best, Saurabh
J
but is it then still randomized? i only want to not show the scenario for which the quota is full.

@SaurabhPujare_Ugam said:
"set the branch condition on block stating if Quota Scenario 1 has not been met."
bstrahin
Rank
Userlevel 7
Badge +38
@JaJu if you want to be really strict on preventing over quotas and keeping randomization I usually make a survey flow like the attached. There are branches to randomize what blocks are eligible to be shown based on what quotas are full and which ones are still open. You can also look under tools to see the settings I used for Quotas (e.g. None setting for counting the blocks since I put all the 'rules' in the survey flow). You may still get some over quotas from multiple completing at once and the bit of lag with the system catching up but it should be less than other approaches.

If you've never loaded a .qsf file before here's instructions: https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/import-and-export-surveys/
J
@bstrahin that is an amazing solution! thanks so much 🙂
L
Userlevel 1
Hello, I have a similar issue maybe you can help me out too.

I want to randomize 200 subjects 4 treatment groups with 50 subjects each. They all have set their treatment as embedded data right away (see T4 in picture). Similar to JaJu some get screened out in an attention check in their treatment groups before the actual survey starts.

I have 3 response types:
- screened out = exclude from quota
- reached pf limit = include in quota
- last question= include in quota

Do you think that a quota setup like this works? I know that there can be a person still in completing when the quota is met, but that would not be so relevant to me.

!

!

Leave a Reply


Terms & Conditions