Originally published March 28, 2020
At the outset of this study, we were asked to measure the impact of the United World Colleges movement. We had to start by asking what was meant by impact—impact on students? Impact on the world? Economic impact? Social impact? There are a myriad of ways that one can measure impact, and one study cannot capture all of them. We therefore began to think about some of the ways to capture ideas of “impact” in this study.
As educational researchers, we quickly ruled out some ideas of impact; indeed, calculating returns on investment, as an economist might, is not our forte. Instead, our goal throughout this project has been to allow our vision and understanding of what impact and “making a difference” mean to arise, a priori, from the words and responses of our participants themselves.
On the other hand, this does not mean that we, as the research team, do not hold on our own views on terms like “impact”, or that we do not want to have a firm understanding of how other studies and scholars are framing these ideas. So, we have tried to proceed in the study by moving back and forth between asking open-ended questions to garner broad responses regarding the meaning of impact, while, at the same time, using academic literature to help us think about how to potentially make sense of, or probe deeper into, the concept of impact.
In the course of our literature review we’ve found making a difference in the world to mean:
Having good character, acting responsibly, and being generally law abiding
Taking leadership positions in community organizations and structures
Questioning and changing the systems and structures that lead to inequity and injustice
Earning money in order to donate to the impactful causes—causes that are large in scale, highly neglected, and highly solvable
Breaking away from “Market World” (those who desire to do good while profiting from doing good) in order to effect systemic change
Using social norms and moral nudges to bring about small forms of change
Finding a way to form “normative cascades”, or ripple effects, where one small change in a small groups of connected networks can ultimately bring about a social movement
Understanding the complex causality involved in making system-wide change, which requires constantly adapting in order to achieve long-term goals
Addressing underlying systems (e.g. pressures, policies, power dynamics, perceptions, and underlying purposes) in order to achieve a desired result
Becoming a social entrepreneur by using direct actions, through creating a product, service, or methodology, in order to address societal inequities.
While this list is in no way exhaustive, each of these models of impact have helped us build a vision of what impact can look like and how it might come about.
As noted above, however, we wanted to enter this study with an open-ended understanding of what impact could mean for our participants. So, we began our research with 47 “informant” interviews in which we spoke with people throughout the UWC movement—alumni, former teachers, former heads, national committee members, etc.—and asked them about what they thought students learned from UWC, what they thought impact meant, and so on. It was through these interviews that we began to form our views of the different concepts and questions that we ultimately included on our surveys and interview questionnaires.
Ultimately, in our final semi-structured interview protocols created for UWC students, staff, and alumni, we ask several questions about how participants think about impact and making a difference in the world. Some of these question prompts include:
“What do you think it means to make a difference in the world?”
“What do you think success for a student from this school looks/looked like?”
“What types of activities are/were you engaged in in the community?”
“What are your future goals?” or “How do you feel your life would be different if you hadn’t attended UWC?”
All of these types of questions give us an opportunity to hear about the types of activities that UWC stakeholders are engaged in and how they are thinking about interacting with, and potentially impacting, the world.
Our student and alumni surveys also ask several questions that probe how UWC and non-UWC participants are thinking about and making an impact in the world. Some of these questions include:
Questions about how participants might act if particular dilemmas arose in their communities (e.g. an oil spill)
Questions about which of the UN Sustainable Development Goals participants feel they are impacting and examples of how and via which means (e.g. job, volunteering, research, etc.)
Asking participants to rank the level of impact of four profiles of individuals making a difference and to explain their reasoning for their ranking (e.g. a parent vs. an investment banker)
Descriptions of college and career choices
Questions regarding decisions to return or not return to one’s home country
To be sure, our vision of impact throughout this study is ever evolving; each visit to a school we take, each student or alum we speak with, and each book or article we read continues to inform the ways in which we think about impact. As we move forward into the final stages of data collection and begin our data analysis in earnest, we look forward to understanding how our various sources of impact data fit together; moving forward, such work might help us all gain a broader and more nuanced understanding of how we might make a meaningful difference in the world.