Intro
We have selected Kallen Tsikalas as the Qualtrics researcher of the month. Check out this interview with Kallen where she discusses how Qualtrics has been critical with what she is currently doing with the Girl Scout Research Institute to make strategic decisions and discover important information.
Name: Kallen Tsikalas
Industry: Non-Profit (Youth Development)
Organization: Girl Scout Research Institute, Girl Scouts of the USA
Position: Senior Researcher
Research Experience: 17 Years
Tell us about yourself. We would love to hear about your hobbies, education, and family.
2011 was a momentous year for me, because I both got married and finished my dissertation for a PhD in educational psychology! Outside of work and school, my husband and I are both avid hikers and fans of live roots music. I also really enjoy puppet theater and taking photos of fungi in the wild. ☺
What company/organization do you work for, and how long have you been in the research community?
I work at the Girl Scout Research Institute (GSRI), which is part of Girl Scouts of the USA. I have been at GSRI for 3 years, and we have worked with the Qualtrics research community for about a year and a half.
What has been the focus of your research in 2011? What is your favorite/most interesting project you have done with Qualtrics?
In Spring 2011, I led a study for GSRI on the relationship between girls’ experiences in Girl Scouting (e.g., participation in community service and camp, extent to which they felt they had opportunities to lead and to work cooperatively with other girls) and their academic achievement and engagement. The project was a collaboration between Headquarters and 10 partner Girl Scout councils (affiliates) around the country. We worked with these councils to design and implement the study, and we surveyed ~3,000 girls between the ages of 10 and 14 years.

While the academic study was certainly interesting and rewarding, my most interesting project in Qualtrics was actually a smaller, more experimental initiative. GSRI has been exploring the use of “scenario-based” assessments to evaluate girls’ leadership skills and attitudes and also to provide real-time, individualized feedback to them about their strengths and opportunities for improvement.
To this end, we designed and pilot-tested three scenario-based assessments of specific Girl Scout Leadership Experience outcomes in Qualtrics. These assessments include a very elaborate scoring system that uses some of the scoring tags in Qualtrics but also customized javascript. Particular patterns of scores are linked with feedback statements that users receive after taking the surveys.
What conclusions have you been able to draw from your research, and what impact has that had?
While we are still finalizing conclusions from the 2011 academic success study, we pursued two parallel types of analyses that have both yielded positive and provocative results. One set of analyses sought to predict (through regression and mediational analyses) which Girl Scout experience factors had the greatest influence on girls’ academic success as well as the amount of variance these factors explained. The second set of analyses, segmentation analyses, explored whether there were clusters of qualitatively identifiable types of Girl Scout (GS) experiences.
In the case of predicting academic success, we have found that GS experiences which lead girls to seek positive challenges and to be more resourceful problem solvers, are highly predictive of academic success—especially academic efficacy, mastery goal orientation and valuing school. The actual experiences that lead to these outcomes differ somewhat for older and younger girls. However, in general, when girls have more opportunities to work cooperatively with other girls, to lead activities and to complete longer-term community service/Take Action projects, they tend to gain these outcomes and achieve greater academic success.

In the case of segmentation, we have identified five very distinct clusters of Girl Scout experiences. Three are highly engaged and impacted experiences; two are less engaged and impacted. These analyses will help us target messaging and programming more effectively and ensure the needs of all girls are being met through Girl Scouting.
What mistakes have you made that you wish you hadn’t?
Well… this was a mistake from which we learned A LOT, so I don’t really regret it, but it did cause quite a stir. Our initial eMail messaging that invited several thousand girls to register for our panel (a process that also includes parent consent procedures) was targeted to GIRLS: It was phrased in child-friendly language with cartoony graphics. The message was clearly affiliated with Girl Scouting, but we took liberties in the language and imagery to make this invitation more playful and fun.
However, the eMail addresses we had on file for these girls largely belonged to their parents. So, adults received messaging that was designed for girls. Needless to say, the reaction of many parents was instant DISTRUST. They thought that the initiative was a scam, and that their daughter’s private information had been compromised. Our partner councils got scores of calls from angry parents.
We had to back-pedal immediately! We created official-sounding messaging with all the proper verbiage about privacy and safety, links to GSUSA’s policies about these issues, and standard corporate branding. Also, while we did have phone and eMail support lines in place for parents to contact if they had questions, we made this information more prominent in the invitation, so that parents would be sure to see it immediately. Finally, each council included messaging on its web pages that assured parents the campaign was legitimate. Disaster was averted!
How has Qualtrics software assisted you in this research?
With the academic success study, obviously the various data collection features of Qualtrics (e.g., panel management, survey design and distribution) assisted our research. We were also able to share data with our partner councils relatively easily and to show them how to do basic analyses on the data using the Qualtrics reporting tools.
For our experimental initiative in building scenario-based assessments, the versatile scoring and display options of Qualtrics were particularly valuable.
What are the most valuable research related ‘epiphanies’ you have had in the past year that you wish you would have been aware of before?
We have a process for reviewing and testing our surveys to make sure there are no errors in grammar, spelling, survey logic, survey flow, etc. However, we don’t really have a process for examining the instruments holistically to make sure they are measuring everything that we need them to measure. Our academic success study was well-designed with a model of impact and measures associated with all the factors we thought would be important. However, once we were waist-deep in data analysis, we realized there were a few more questions that would have shed much more light on the data, had we asked them. This year, we’re trying to design a process that would enable us to anticipate and address potential “holes.”
Do you have a favorite experience using Qualtrics tools?
I especially appreciate the ease and versatility of the survey creation tools. Some of the functions of the beta reporting tool are also great, for instance, being able to pull data from different surveys into the same graph.
**Are you using Qualtrics to do interesting, fun, ground-breaking research? Apply to become the Qualtrics researcher of the month by emailing the editor**











