Setting Quotas that use Embedded Data for Preventing New Surveys and Ending Current Survey

TerryHTerryH Chapel Hill, NCCommunity Member Qubie ✭
edited April 2020 in Best Practices

Setting Quotas that use Embedded Data for Preventing New Surveys and Ending Current Survey

I like to think I'm pretty facile with Qualtrics, but I've never been able to set a quota group and feel comfortable about it. I'm programming more and more studies that award incentives to completes and I get nervous each and every time that I'm going to mess up quotas resulting in over-run of incentives. Or I'm unsure if I'm unnecessarily running respondents through a survey in which they will eventually be flagged as overquota.

I'd appreciate some guidance if there is a better way to set my quotas so that when I, say, want 400 total responses with 200 residing in one state and the other 200 residing in any other state. My state variable is based on an embedded variable from the original contact list.

So I generally set a General Quota at 400. Quota is set to increment when the text description on very final question is displayed. Once quota is hit, I've set rule so that it Prevents All New Survey Sessions.

Then I set two additional quotas for in-state and out-of-state.

For in-state quota, I have it set to increment quota when response is submitted where embedded data variable is equal to code that indicates in-state. Once quota is met, it is set to End Current Survey.

The same is set for the out-of-state quota to increment when response is submitted where embedded data variable is equal to code that indicated out-of-state.

However, I've noticed in one study with this that my in- and out-of-state quotas are outpacing my General Quota (the sum of in- and out-of-state are higher than General Quota). It seems that respondents that are terminated out early in survey are counting as a submitted response within my in- and out-of-state quotas. They, however, are not counted in the General Quota because they don't reach final page of survey.

If I add a condition to my in-state quota that, in addition to the embedded data variable indicating in-state, the respondent must also display the last text description, will this make my quotas increment at the same rate? With this design, if the in-state quota is already full and an in-state respondent tries to enter the survey, will they be immediately notified that the quota has been filled? Or will they have to get to the last text description before being notified that the quota is full?

I seriously appreciate any help on this.


Best Answer

  • TomGTomG Raleigh, NCCommunity Member Wizard ✭✭✭✭✭
    Accepted Answer

    I always set an embedded variable that contains the respondent's final status (e.g., complete, terminate, overquota). The variable is always set before an end of survey block. All quotas contain a condition that makes sure only completes are counted.


  • MsIreenMsIreen FinlandCommunity Member, XMPN Member Superuser ✭✭✭✭

    With regards to getting more responses then Quota allows. I have noticed that the responses are "cancelled" when the last response that is meeting quota conditions is submitted. These responses are saved as "Finished", however, they all have different number of questions answered, so they may be hard to track. We have formulae based on the very last question. So I use filtering "is not empty" for finding only the complete responses.

    Otherwise until the last response is submitted, Qualtrics doesn't "see" that quota is full"

  • TerryHTerryH Chapel Hill, NCCommunity Member Qubie ✭

    @TomG Thanks for the suggestion. Would you be able to screenshot an example with how you typically do this for each status?

    I assume Complete occurs after your last survey block.

    Then the Terminate status would update after each Terminate block, right? So say, terminating on geography. Show Geography block. Immediately follow with branch saying if, say, non-US, change status to Terminate, End of Survey?

    For the overquota, are you setting a branch that says, if particular quota is met, set status to overquota, and then End of Survey? Do you put that at beginning or end of survey flow? or both?

  • TomGTomG Raleigh, NCCommunity Member Wizard ✭✭✭✭✭
    edited April 2020

    I typically have a lot of other stuff going on so screen shots might add confusion.

    I initialize the status (don't use 'status' because it is reserved) variable to 'abort'.

    For complete and terminate, I do pretty much what you describe.

    For overquota, it depends on the quota. For a 'total' I would put it near the beginning of the survey flow. Otherwise, I treat it like a terminate: ask the question, if choice selected and quota for choice has been met, set status to overquota and end survey.

  • TerryHTerryH Chapel Hill, NCCommunity Member Qubie ✭

    I've been taking this approach in some recent studies and found it to be much smoother. Thanks for help on this.

Sign In to Comment