Investigating Impacts of Educational Experiences: FAQ
Originally published on the UWC Website in 2018
Below are some questions commonly asked of us by study participants and other individuals interested in the impact study. If your question is not answered below or in the two methodology documents, please don’t hesitate to contact us at sarah_magagna@harvard.edu.
1. What is Project Zero?
Project Zero (PZ) is an educational research organization based at the Harvard Graduate School of Education (HGSE). Established in 1967 with an initial focus on the intersection of education and the arts, PZ has since expanded its areas of inquiry to include broad topics such as creativity, ethics, cognition, and intelligence. We partner with a variety of institutions and organizations and have many research projects underway, from Educating with Digital Dilemmas to Higher Education in the 21st Century. You can find out more about PZ and our research here: http://www.pz.harvard.edu/
2. Who are the researchers for this study?
We are a team of four PZ researchers with diverse backgrounds, bringing different types of experience and expertise to this project. Howard Gardner, Principal Investigator, is the Hobbs Professor of Cognition and Education at HGSE. Well-known for his Theory of Multiple Intelligences, Professor Gardner has carried out extensive research over the course of five decades. His research includes the various strands of The Good Project (thegoodproject.org) and a current project on Higher Education in the 21st Century. Shelby Clark, Senior Research Manager, has a Ph.D. in Applied Human Development from Boston University, and her many research interests include adolescent character development and mission-based school cultures. Danny Mucinskas, Project Manager, has worked at HGSE for six years and also serves as Project Manager for The Good Project, which explores the concepts of good work, citizenship, and collaboration. Sarah Magagna, Research Assistant, has an Ed.M. from HGSE and a background in teaching in international schools, including an international boarding school.
3. How did this research study come about?
United World Colleges (UWC), a movement of eighteen schools around the world, had previously conducted an exploratory 1 study involving a subset of their schools. Following the completion of this preliminary study, the leaders of UWC approached Project Zero to discuss conducting a more comprehensive study. The goal of this more comprehensive study is to learn about the overall impact of educational experiences on students and alumni and whether and how the involved schools are fulfilling their mission.
As there was significant overlap in areas of interest (we at PZ and The Good Project have been investigating the impacts of educational experiences and programs on students and graduates in terms of outcomes like civic dispositions and ethical reasoning), the project moved forward. After discussing goals for the research with UWC and gathering background information, the research team then began developing the study instruments and worked to identify schools outside the UWC movement who would also be able to contribute to and benefit from the research. Although UWC is the sponsor of the research and the study’s largest partner, the research is being conducted independently by our team at Project Zero and, as mentioned, involves a number of other schools outside the UWC movement.
[This includes International School Moshi in Tanzania, which is joining the UWC movement in August 2019. Students currently in IBDP1 are being integrated into the Study by taking the online survey and participating in student interviews before transitioning into UWC as IBDP2 students.]
4. How are you choosing to define impact?
Broadly speaking, we are thinking about two kinds of impact: the effects of certain educational programs, practices, or approaches on students and alumni, and the impact of those students and alumni on their communities and the world. The first form of impact includes student and alumni attitudes, beliefs, and behaviors, as well as which specific elements of their educational experiences are or were particularly salient. We will furthermore be analyzing the specific elements of educational experiences as they relate to these attitudes and beliefs.
To gather ideas about the meaning of impact, we spoke with a number of figures involved in the UWC movement and in the wider field of education; we also considered the mission statements and core values of the participating schools. Although our ideas about impact are informed by our preliminary interviews and the relevant scholarship, and although we are drawing on some frameworks (such as the UN Sustainable Development Goals), we are in general taking a “bottom up” approach to this study and allowing the definition of impact to come from the participants themselves. As such, we envision that one of the key findings of this study will be how participants at the involved schools define impact and whether there is any convergence around those ideas or definitions. For now, we are both keeping our definition of impact fairly broad in scope and focusing in on particular dimensions that seem to be important to stakeholders.
5. What kinds of student and alumni outcomes are you looking for?
As mentioned previously, this is more of a “bottom up” study. We are not trying to look for specific outcome behaviors and are trying to avoid making prior assumptions about the outcomes we might find. Based on our research interests and the areas about which our participating schools are hoping to learn, we are investigating the following: certain dispositions related to international-mindedness (e.g. open-mindedness); attitudes, beliefs, and ethical reasoning skills; participation in social cause-related activities (e.g. volunteering); and other ways in which participants may feel they are having an impact (e.g. promoting certain values, civic/political engagement, etc.) and at different levels (e.g. on family and friends, organizations, society). For alumni, we are also interested in participants’ post-secondary education, professional paths, and interests.
6. Why are all your study documents and activities in English?
Although English is the sole or primary language of instruction at nearly all the participating schools, questions around language and accessibility were discussed extensively by the research team. However, participants in the study come from over 100 countries and represent nearly as many languages, making equitable translation of materials difficult. Furthermore, because our surveys use scale measures previously validated in other studies in English, we felt that using those scales in translation might jeopardize the validity of our results. We were assured by all participating schools that students and alumni could participate in the study even if the instruments (e.g. surveys) were offered only in English, although some schools informed us that participation for certain students might be delayed until the second survey, by which time their English language skills should have improved. We did, however, try to make sure that the language used was simplified as much as possible and that our study instruments are culturally sensitive, having them reviewed by a diverse group of individuals and conducting several rounds of piloting. After Harvard’s Institutional Review Board relaxed its previously strict regulations concerning the translation of consent materials, we also decided to make parental consent forms available in the six official UN languages.
7. How many people are participating in the study?
Over 2,000 students from UWC and the other schools have participated in the study to date. This number will increase as students, who were not able to take part in the first surveys, or will be included into the study as part of the 2019 IBDP1 intake, continue to join. Although we have not yet started collecting data from alumni, as many as 85,000 are eligible to participate from all UWCs and the other schools. Since not all schools have up-to-date alumni records, if you are an alum of a participating
school, please make sure to spread the word about the alumni survey to your classmates! Additionally, roughly 250 teachers, staff, and administrators from UWC schools are expected to participate in the study.
8. When will you be able to share the results of the study? Do you plan to publish anything?
We cannot share any results of the study until all data collection has been completed, as we don’t want any preliminary results to affect or bias the later data collection. Some preliminary data analysis is concurrent with our data collection, but the last year of the study (2020-2021) will be devoted exclusively to data analysis. We expect to be able to share the results of the study with all participating schools at the end of 2021.
At this point, we envision sharing the results more widely through multiple methods, such as academic articles, a conference, or possibly a book, but we will be able to better identify the most effective means of disseminating our findings to the public when the study is nearing completion. If you would like to receive any reports on the study when they become publicly available, please click here to share your contact information with us.
9. What were the selection criteria for schools outside the UWC movement? How did you identify good candidates for the study?
In order to explore a wide range of educational programs and practices, and to contribute to knowledge about “best practices” in international education (as well as education more generally), we sought a diverse set of schools from outside the UWC movement to participate in the study. In addition to seeking schools that covered a broad geographic area, we sought out internationally-oriented independent schools that displayed some combination of the following:
A diverse student body (in terms of national/cultural, socioeconomic, and religious backgrounds)
The International Baccalaureate (IB) Diploma Programme or a similar academic program
A residential/boarding element
A strong mission with a set of core values
Potential schools were identified by the research team and colleagues at Project Zero as well as through our various contacts in the field of international education. Some suggestions were also put forth by individuals from within the UWC movement, as many of them have extensive experience in and knowledge of international education. All potential schools were carefully researched by the team at PZ to ensure that they would be a good fit for the study and that they would benefit from participating. Schools from outside the UWC movement have the option of remaining anonymous in any reports or publications.
10. If I am a student or alum eligible to participate in the study, will my participation (or lack thereof) have any influence on my application to Harvard, if I decide to apply?
The Harvard Graduate School of Education is a separate entity from Harvard College, and participation (or decision not to participate) in our study will have no bearing whatsoever on applications to Harvard College or any graduate schools at Harvard. The survey results will be completely anonymous and individual responses will not be shared with anyone outside the research team. Furthermore, all data (for example, information shared in interviews) will be de-identified, meaning we will remove any identifiable details such as name, country of origin, etc. Participation in this study is not required, and a decision not to participate will not be held against anyone.