Setting Quotas that use Embedded Data for Preventing New Surveys and Ending Current Survey | XM Community
Solved

Setting Quotas that use Embedded Data for Preventing New Surveys and Ending Current Survey

  • 7 April 2020
  • 7 replies
  • 657 views

Userlevel 4
Badge +4

I like to think I'm pretty facile with Qualtrics, but I've never been able to set a quota group and feel comfortable about it. I'm programming more and more studies that award incentives to completes and I get nervous each and every time that I'm going to mess up quotas resulting in over-run of incentives. Or I'm unsure if I'm unnecessarily running respondents through a survey in which they will eventually be flagged as overquota.
I'd appreciate some guidance if there is a better way to set my quotas so that when I, say, want 400 total responses with 200 residing in one state and the other 200 residing in any other state. My state variable is based on an embedded variable from the original contact list.
So I generally set a General Quota at 400. Quota is set to increment when the text description on very final question is displayed. Once quota is hit, I've set rule so that it Prevents All New Survey Sessions.
Then I set two additional quotas for in-state and out-of-state.
For in-state quota, I have it set to increment quota when response is submitted where embedded data variable is equal to code that indicates in-state. Once quota is met, it is set to End Current Survey.
The same is set for the out-of-state quota to increment when response is submitted where embedded data variable is equal to code that indicated out-of-state.
However, I've noticed in one study with this that my in- and out-of-state quotas are outpacing my General Quota (the sum of in- and out-of-state are higher than General Quota). It seems that respondents that are terminated out early in survey are counting as a submitted response within my in- and out-of-state quotas. They, however, are not counted in the General Quota because they don't reach final page of survey.
If I add a condition to my in-state quota that, in addition to the embedded data variable indicating in-state, the respondent must also display the last text description, will this make my quotas increment at the same rate? With this design, if the in-state quota is already full and an in-state respondent tries to enter the survey, will they be immediately notified that the quota has been filled? Or will they have to get to the last text description before being notified that the quota is full?
I seriously appreciate any help on this.

icon

Best answer by TomG 7 April 2020, 14:08

View original

7 replies

Userlevel 7
Badge +23

With regards to getting more responses then Quota allows. I have noticed that the responses are "cancelled" when the last response that is meeting quota conditions is submitted. These responses are saved as "Finished", however, they all have different number of questions answered, so they may be hard to track. We have formulae based on the very last question. So I use filtering "is not empty" for finding only the complete responses.
Otherwise until the last response is submitted, Qualtrics doesn't "see" that quota is full"

Userlevel 7
Badge +27

I always set an embedded variable that contains the respondent's final status (e.g., complete, terminate, overquota). The variable is always set before an end of survey block. All quotas contain a condition that makes sure only completes are counted.

Userlevel 4
Badge +4

TomG Thanks for the suggestion. Would you be able to screenshot an example with how you typically do this for each status?
I assume Complete occurs after your last survey block.
Then the Terminate status would update after each Terminate block, right? So say, terminating on geography. Show Geography block. Immediately follow with branch saying if, say, non-US, change status to Terminate, End of Survey?
For the overquota, are you setting a branch that says, if particular quota is met, set status to overquota, and then End of Survey? Do you put that at beginning or end of survey flow? or both?

Userlevel 7
Badge +27

I typically have a lot of other stuff going on so screen shots might add confusion.
I initialize the status (don't use 'status' because it is reserved) variable to 'abort'.
For complete and terminate, I do pretty much what you describe.
For overquota, it depends on the quota. For a 'total' I would put it near the beginning of the survey flow. Otherwise, I treat it like a terminate: ask the question, if choice selected and quota for choice has been met, set status to overquota and end survey.

Userlevel 4
Badge +4

https://www.qualtrics.com/community/discussion/comment/23847#Comment_23847I've been taking this approach in some recent studies and found it to be much smoother. Thanks for help on this.

Badge +1

@TomG can you please explain how you setup the Overquota embedded variable again? Are you creating the new variables for each quota within branch logic (I don’t see a way to use existing quotas from the Quota Groups menu)? How do you go about creating overquota embedded variables for cross-logic quotas? 

Thanks.

Userlevel 7
Badge +27

@TomG can you please explain how you setup the Overquota embedded variable again? Are you creating the new variables for each quota within branch logic (I don’t see a way to use existing quotas from the Quota Groups menu)? How do you go about creating overquota embedded variables for cross-logic quotas? 

Thanks.

‘Overquota’ can have different meanings depending on context. I’m going to assume you mean that when a response matches criteria for a quota and that quota has already been met, the response is ‘overquota’.  Generally, you would have a survey flow branch something like ‘if criteria AND quota has been met’ then set the status to overquota and end survey.

I generally don’t use cross-logic quotas. I use simple quotas with survey flow logic and embedded data fields where needed.

Leave a Reply