Efforts to solve sustainability challenges require us individually and collectively as researchers and practitioners to move beyond our disciplinary silos (Kates et al. 2001, Lang et al. 2012, Cvitanovic et al. 2018). In turn, opportunities for more reflexive and transdisciplinary approaches in our science and practice are needed to highlight the norms, values, and worldviews that shape how we interpret the world around us and the actions we take to advance sustainability (Hirsch Hadorn et al. 2006, Enquist et al. 2017). Graduate research and training in sustainability is an important dimension in this effort (Schwartz et al. 2017), and further development of specific pedagogical tools to support graduate studies are needed (Ban et al. 2015).
To address the wicked challenges (i.e., problems with no clear initial framing or solution; Rittel and Weber 1973, Commonwealth of Australia 2007) of sustainability, current and future generations of students will need to think about their research and professional practice in ways that are socially robust and relevant to context (Enquist et al. 2017, Lebel and McLean 2018). Opportunities are needed to coproduce knowledge and practice with a wide range of stakeholders (see Klein 2013, Adler et al. 2018) and to think about how knowledge should be developed in ways that foster social learning or changes in understanding that go beyond the individual and that are embedded or institutionalized in social groups or communities of practice (Reed et al. 2010). This is a difficult and often uncomfortable proposition in many disciplinary graduate programs, as well as in those explicitly aiming to foster transdisciplinary training.
Here, we reflect critically on the experiences and perceptions of a cohort Master of Environmental Studies graduate students and the course instructor (all coauthors of this paper) involved in a class-based assignment about sustainability research and practice. Specifically, we reflect on the use of a “theory of change” process as a pedagogical tool in which early career researchers can situate their specific research projects in a broader sustainability context. In doing so, we examine this pedagogical activity in the realm of transdisciplinary sustainability education and consider the strengths, weaknesses, and opportunities of the theory of change process for early career sustainability researchers. Given the diversity of student backgrounds and interests, our approach to sustainability is open but is framed here as the ability to meet the needs of present generations without compromising the ability of future generations to meet their needs (WCED 1987).
We define the theory of change as a process for individual and organizational learning that includes analysis of actions, outcomes, and consideration of the explicit and implicit assumptions about how actions and outcomes are interconnected (see Taplin and Clark 2012, Valters 2015). Key principles of a theory of change include an emphasis on learning through the complexity of problem situations, a commitment to have a process led by those most affected by a particular problem, and a recognition that the framework is a “compass” for navigating change as opposed to a defined map or blueprint (see James 2011, Valters 2015).
There is a productive body of literature on inter- and transdisciplinary research and practice and the implications for sustainability (Hirsch Hadorn et al. 2006, Pohl and Hirsch Hadorn 2008, Brandt et al. 2013). Interdisciplinarity refers primarily to different academic disciplines working together in the context of shared problems and research goals. This approach is relevant and appropriate for many early career researchers engaged in environment and sustainability challenges. However, transdisciplinarity involves different academic disciplines engaging with each other and nonacademic collaborators to coproduce, from different types and sources of knowledge, insights and actions for shared research and applied problems (for a review, see Kelly et al. 2019). Such concepts and interpretations of transdisciplinarity are also consistent with strategies to improve how we measure research impact more generally (see Lebel and McLean 2018).
The barriers and opportunities associated with inter- or transdisciplinary research and practice are increasingly well articulated (see Campbell 2005, Lélé and Norgaard 2005, Miller et al. 2008, Alexander et al. 2018). For example, Kelly et al. (2019) highlight the culture and institutional structure of academic settings as a continuing challenge, although there are examples of innovative practice (Schwartz et al. 2017). Kelly et al. (2019) also draw attention to a series of “tips” and strategies to foster interdisciplinary research and practice, including a range of knowledge-based, attitude-oriented, and practice-focused initiatives. Implicit in many of these tips is a need to build capacity within and outside academia.
According to Pohl and Hirsch Hadorn (2008), a transdisciplinary approach to research includes the development of descriptive, normative, and practice-oriented knowledge to help solve, mitigate, or prevent real-world problems. To do so requires a willingness to “...(a) grasp the complexity of problems, (b) take into account the diversity of scientific and life-world perceptions of problems, (c) link abstract and case-specific knowledge, and (d) develop knowledge and practices that promote what is perceived to be the common good” (Pohl and Hirsch Hadorn 2008:111). Mitchell et al. (2015) highlight three key realms within which to consider transdisciplinary approaches, emphasizing a need for: (1) improved understanding of the situation or field of inquiry (i.e., the problem context); (2) generation of scholarly knowledge and other societal knowledge forms, with those insights made accessible and meaningful to researchers, participants, and beneficiaries; and (3) mutual and transformational learning by researchers and research participants to increase the likelihood of persistent, and presumably positive, change. Similarly, Enquist et al. (2017:541) define their interpretation of a transdisciplinary approach as “...the nexus where knowledge meets action, and is situated at the intersection of a broad spectrum of institutions and information pathways where scientists, practitioners, and stakeholders work together to build trust and to develop ideas, products, and outcomes that are accessible, actionable, shaped by all participating parties, and can be readily used in decision making.”
To achieve these ideals, however, early career researchers and others engaged in these efforts need sustained opportunities or access to capacity-building efforts. Designing and testing these opportunities is an area in need of additional analysis, although key insights are emerging about the relationships among transdisciplinary principles and their integration into graduate pedagogical learning opportunities (see Eigenbrode et al. 2007, Ban et al. 2015, Schwartz et al. 2017). Miller et al. (2008) note, for example, the importance of epistemological pluralism as necessary for collaboration that is at the root of strategies to deal with complex problems. This pluralism includes the need to value different ways of knowing and thinking, but it also needs to be tested through experience and practice, including in the classroom. Indeed, Hackett and Rhoten (2009) analyzed the outcomes of the Integrated Graduate Education and Research Training (IGERT) program in the United States. They found substantial and consistent differences among those with interdisciplinary training, but only in the early stages of careers, and they also highlighted the value of interactive pedagogy (a “design charrette” in their case) to create collaborative spaces for discussion and analysis among student participants.
Baldauf McBride et al. (2011) also analyzed a larger scale program for graduate education (the GK–12 program) aimed at encouraging more integration across scientific disciplines, policy makers, land managers, and the general public. Specifically, they track perceptions of program participants at one institution (the University of Montana) and document a heightened awareness of the need for more interdisciplinary experiences. Welch-Devine at el. (2014) further show the need for integrative training in conservation and sustainability contexts. They highlight the importance of training researchers and practitioners for “agility” and point to the translational skills needed to converse across disciplines and knowledge domains, to move between academia and practice, and to translate research into action. This skill set is consistent with the “toolbox for philosophical dialogue” (see Eigenbrode et al. 2007), which aims to support the collaborative science needed to address complex problems. The tool box described by Eigenbrode et al. (2007) includes a series of issues and key questions relating to motivations, values, methodology, and the tensions between reductionist and constructivist approaches. More recently, Schwartz et al. (2017) outlined a number of examples of graduate training initiatives and institutional contexts where such goals are being pursued.
Across these assessments of programs, however, there is less of an emphasis on how specific pedagogical tools are being used to pursue transdisciplinary training outcomes. In this regard, two main insights emerge. First, there are a range of contexts variously described as collaborative, interdisciplinary, and/or transdisciplinary in which capacity building is emphasized, but for which the specific pedagogical tools or in-class strategies are less commonly discussed. Second, where tools or methods are highlighted, few of them specifically document the experiences of early career researchers in their own words as they grapple with new expectations and challenges of transdisciplinarity. We next highlight one such experience.
Theory of change is a process-oriented approach and pedagogical tool to encourage transdisciplinary education and action. We outline in more detail the definitions and steps involved in undertaking a theory of change and, in particular, focus on how the process was used in the context of a major class assignment. We critically reflect below on the strengths, weaknesses, and opportunities of the theory of change process to foster greater engagement and interest in applying transdisciplinary thinking in graduate research and future professional practice. We offer specific examples and illustrations of insights based on the coauthors’ experiences.
Theory of change is defined by James (2011) as an ongoing process of reflection to explore change, how it happens, and what it means for the part that people play in a particular context, sector, or group of people. There are, of course, similar strategies to reflect on change and its outcomes, including the application of logical framework analysis (Gasper 2000), strategic planning, and outcome mapping (Earl et al. 2001, Mitchell et al. 2015). The latter has been used, in particular, by the International Development Research Centre as a program evaluation tool to track initiatives with a particular emphasis on relationships, behaviour change, and the roles of different actor groups (Earl et al. 2001).
The theory of change process has gained increased traction among a wide range of research and development agencies seeking to better influence program directions and outcomes given a recognition of the complexity (social, political, ecological) in which those organizations operate (CARE 2012, Conservation International 2013; WorldFish Center [Douthwaite et al. 2013, van Tulder and Keen 2018]). There are two general categories associated with theory of change: (1) a focus on how a particular program will bring about change, and (2) an exploratory orientation of how change happens more broadly and how a specific initiative (research, program activity) can contribute to desirable change. Central to both of these general categories, and what sets the theory of change framework apart from other strategies, is a particular emphasis on a systems perspective in which opportunities for positive outcomes are linked to a broad range of drivers, assumptions, and connections across scales.
There is likely more than one theory of change for any wicked problem; it is thus worth remembering that different theories of change that are developed around a common problem might sometimes be in conflict. However, these limitations with theory of change are in large measure why using the approach in a graduate sustainability course can be valuable (see also van Tulder and Keen 2018). What may appear on the surface to be tractable problems amenable to technical solutions rarely are, and the process can be a valuable tool that initiates a process to evaluate critically the wide range of assumptions associated with any intervention.
Graduate students in the School of Environment, Resources and Sustainability (SERS; https://uwaterloo.ca/environment-resources-and-sustainability/), University of Waterloo, Ontario, Canada, come from various backgrounds and have diverse sustainability expectations and goals. The core mandate in SERS is to offer an academic context to protect, restore, reform, and transform, where necessary, the social and ecological systems upon which we all depend. These are not disciplinary aspirations. As is the likely experience in other integrative programs, offering core course content for a diverse group can be a challenge. Some students are explicit about their interest in a more transdisciplinary approach as a foundation for future career goals, whereas others are seeking more technical and discipline-specific material.
“Sustainability Applications” is a required course for all Master of Environmental Studies students in SERS. The course seeks to emphasize actionable concepts and transdisciplinary approaches to foster applied research and policy for linked systems of people and nature. An important focus of the class is thus to understand how individuals and societies can complement disciplinary thinking to build individual and collective capacity to foster sustainability in the context of change and uncertainty. Learning objectives of the course, and of the SERS graduate program more generally, are: (1) to encourage a systemic perspective by thinking in terms of social and ecological connections and considering feedbacks across scales; (2) to adopt a critical lens through which the relationships of power that influence sustainable outcomes are at least considered, even if they are not the focus of research (as is the case for the majority of students); and (3) to maintain an applied orientation, that is, developing and using concepts, tools, and skills in the context of real-world challenges, with an emphasis on each student’s own areas of application. Specific learning objectives of the theory of change assignment were to have class participants consider and reflect on the importance of theory to good practice, to think through the broader sustainability implications of their own work and career aspirations, and to develop the skills to communicate and present their ideas in narrative and visual forms.
Our strategy to make these program features actionable was to use the theory of change process. Approaches to the development of a theory of change vary depending on the context (see Vogel 2012) and include a wide range of potential resources that provide guidelines for those interested in their development (see https://www.actknowledge.org/). However, the process adopted for our class activity involved five main steps (Fig. 1):
We use the class context and experience here as an exploratory case of the development and application of theory of change to support transdisciplinary sustainability education. Our approach to distil lessons from experiences with the process was qualitative, given the class size (18 students) and direct nature of the experience for each individual. There are valuable insights to be gained from this grounded approach (see Creswell 2005) in which course participants engage in critical thinking about their own learning experiences. We are not seeking to make broad claims or generalizations about the use of a theory of change process with reference to specific learning outcomes. However, these insights can form the basis for propositions to be further tested.
To develop these insights, we first undertook a two-hour workshop and adopted a modified “strengths, weaknesses, opportunities, and threats” (SWOT) analysis. A SWOT analysis is a widely used approach to solicit and analyze the dimensions of a particular intervention or program (see USDA 2008). The SWOT workshop occurred approximately six weeks after the class was completed and all assignments and marks had been submitted. We did not explicitly engage with the “threats” category because it was not deemed particularly relevant, and any perceived threats would be captured as weaknesses.
To structure the SWOT workshop further, we used a World Café process (see Brown and Isaacs 2005). A World Café is a group-based approach to foster structured dialogue and to encourage cumulative insights on key themes through successive rounds of deliberation and reflection (Elliot et al. 2005). Three small groups were initially tasked with brainstorming key ideas and insights in relation to strengths, weaknesses, and opportunities, respectively. After approximately 25–30 min, the three groups rotated to the next theme and, using the points from the previous group, further debated and added to the analysis. In total, three World Café rotations were undertaken, and a final plenary discussion was completed to summarize our collective insights. The results were subsequently shared with the individuals who were unable to participate in the workshop because of field commitments. These individuals commented upon and added to the analysis.
A final synthesis and verification process was coordinated among the group once all inputs had been received. The verification process included a collective review and commenting process by all group members on the core themes and their implications. No disagreements on the core themes were identified, although some additional examples were included. In the next section, we elaborate on the potential utility of theory of change as experienced by the authors of this paper.
As indicated previously, all of the coauthors were participants in the graduate class, giving them a unique perspective to share their insights and experiences, both positive and negative. Indeed, the course cohort was typically diverse and included, for example, individuals working on wildlife ecology and management, ecological restoration projects, human dimensions of coastal fisheries, First Nations reconciliation and education, forest comanagement, riparian soil nutrient and carbon sequestration, energy sustainability, urban transitions, and food sovereignty.
Some students involved in the class were familiar with ideas that are similar to a theory of change process (e.g., systems mapping), whereas for others, the task was quite novel. Correspondingly, the process and the final paper produced by each individual varied with regard to individual interests and emphasis. The class exercise thus aimed to provide appropriate structure while leaving adequate room for creativity and a focus on the issues of most relevance to each participant. Notably, an important outcome for many was the development of corresponding visuals as a way to capture the narrative components of each theory of change. For example, the core components of a theory of change for a transition to a low carbon energy future centers on the discourses around social acceptance of clean energy technology (Fig. 2); however, theory of change is used to situate social acceptance in the context of economic feasibility, institutional and policy drivers, infrastructure, and technology development. Another project focused on landscape fragmentation, but this interest is situated in the broader context of addressing habitat loss (Fig. 3). In this case, the responses or strategies to address key drivers of change (e.g., urban sprawl) include theoretical innovations, policy changes, and education or awareness opportunities. As both of these figures highlight, the theory of change required students to situate their more disciplinary research in a transdisciplinary context and to reflect on the corresponding assumptions and indicators of progress.
We next synthesize some of the key strengths, weaknesses, and opportunities identified in the context of the completed process. Notably, the strengths for some (e.g., flexibility) can emerge as weaknesses for others (e.g., lack of structure). However, we have aimed to distinguish our collective experiences as clearly as possible and have incorporated examples and direct quotes as appropriate.
The theory of change process had a wide range of identified strengths with regard to fostering a more transdisciplinary perspective on graduate student research initiatives. Four main strengths were identified in our World Café process: (1) revealing the diversity of actors associated with transdisciplinary approaches to sustainability, (2) unpacking the assumptions that frame how individuals understand and interpret complex problems, (3) helping to highlight important causal linkages, and (4) supporting the development of a systems perspective. These strengths reflect specific experiences with the process, but other benefits were also noted, including new ideas for related projects and thinking about new collaborative opportunities.
A commonly perceived strength of the theory of change was how it encouraged course participants to think about the disconnect between ourselves as researchers and policy makers, and other research partners with which we hope to engage. Specifically, the process for each of the class participants proved very helpful in thinking through the diversity of actors involved in each of the research areas in which we are involved, and, as one class participant noted, in asking, “Who is the research for, who is invested in the project, and what data do these people actually need to make decisions?” In this regard, the process proved particularly helpful in mapping the various actors and key stakeholders involved or potential to be involved in large and complex systems, their relationships and responsibilities, and their impact on the system of interest. For some students, this primarily meant capturing the full range of potential actors that their research may influence or be influenced by, whereas for others, the process helped in understanding to whom knowledge about the research might be disseminated. The process of developing pathways to achieve the overarching research goal helps to highlight the many different people and stakeholders who may be interested in or engaged in the research process. However, as one of us noted, the theory of change also helped “to blur the traditionally distinct roles” of scientists, landowners, and land managers, and to recognize that individuals can, and organizations do and probably should, be seen as contributing in multiple ways, rather than as silos of knowledge and action.
One class participant noted that the theory of change process was about more than understanding the diversity of actors and importance of collaboration, and that it, in fact, helped reveal the “distribution of knowledge” and its implications for her project. This individual emphasized, “…working through a theory of change approach has made me much more aware of the importance of carefully crafting a presentation or report to partners that caters to their understanding of the topic so the research is digestible.” More fundamentally, she used her theory of change project to reflect on the disconnect between researchers, the general public, and policy makers, and to consider how data are made easily accessible and understandable. Another of us noted in this regard, “…visualizing and then narrating these tiers [of actors] helped me better assimilate the needed role of partnership between multiple organizations as well as the collaborations of the disciplines within each.”
One important strength of the process identified by the group was the manner in which it helps to shed light on the underlying assumptions of our research, and how it encourages each of us to consider those assumptions and to identify (and address) the associated uncertainties. We were able to draw out a number of examples of this from the individual theory of change experiences and during the World Café process. For example, a major assumption for one of us working on invasive species issues involved notions of successful intergovernmental collaboration, i.e., the governments of Canada and the United States working together to prevent, detect, and manage Great Lakes invasive species. However, that assumption was an entry point to unpack further the importance of collaboration, not only at the federal level, but also in terms of the combined efforts from federal, provincial or state, and municipal levels. Unpacking assumptions about the different roles of the actors involved was “verified” with personal experience and observations. Ultimately, these insights proved important in articulating and achieving the desired outcomes. One class participant noted, “…this is because the theory of change helped me identify the different ‘layers’ of actors that can and will need to be involved in the prevention, detection or eradication, and management of invasive species…,” and subsequently, the positions and assumptions they might hold.
Also emerging from our collective assessment was how the theory of change challenged held assumptions about the central importance of our own research interests in relation to fundamental social and ecological challenges and that what we learn in our own study sites is generally relevant and easily comparable to other contexts. Similarly, the process helped the group to rethink held assumptions that the data we gather and present to partners will be easily accessible and understandable. The process of trying to make sense of a complex problem in our own theory of change highlighted how others must perceive and be challenged by the same ideas and information. As articulated by one class member, “…this theory of change approach to reconciling the wicked problems of my research forced me to approach these issues through an unfamiliar lens. In doing so, the elucidation of various links, assumptions, and phenomena allowed me to shift away from the deeply rooted, preconceived notions of where my research lies in the greater theme of sustainability.”
A key feature of the theory of change process is the development of causal pathways that link actions and outcomes in relation to clearly articulated objectives. One of us noted, “…developing a theory of change enabled me to embrace complexity and transdisciplinary thinking in order to unpack my research problem and identify tangible opportunities for energy system change.” In this regard, the emphasis on spending time to develop causal links also contributed to the identification of various leverage points for action, as well as those areas in any sustainability problem where there are, as one individual put it, “confounding interactions”. There is a recognition that the process encourages the development of multiple angles from which to approach the final research goal and requires us to develop a diversity of perspectives. One participant noted that the use of the process to determine “…what the desirable future will look like and then back-casting to the present allows steps to be generated that will guide a trajectory toward a sustainable future. I think this is one of the most important benefits offered by using a theory of change approach to understanding sustainability.”
These insights on the value of the causal linkages developed through the theory of change illustrate how it can enable a more rigorous systems analysis and, in doing so, supports long-term sustainability planning (Capra 2004). For many class participants, the theory of change helped to reveal potential leverage points or points for intervention and provided insights on opportunities to measure and track progress about long-term goals (e.g., conservation, energy system change). One class participant noted, for example, how the process helped her identify public perception of urban green spaces as a missing leverage point in her project on tools for planning urban restoration. Another of us noted that the process also served to highlight, in practical ways, how sustainability challenges and goals are constantly shifting, and this translated into individual theory of change projects in some different ways. For example, class participants noted that as new knowledge and information comes forth, the way we think of sustainability and the way we attempt to achieve it will shift. The process thus served as a practical reminder that our analyses must account for the effects or consequences of new ideas and interventions.
The theory of change was an important tool in helping participants recognize the underlying issues that are outside the boundaries of their immediate analysis, but that need to change to achieve desirable outcomes. In the context of a project on community forestry and indigenous people, one class member used the theory of change to reflect on the broader issues confronting indigenous communities that have more significant impacts than the focus on forestry. These include basic needs concerns related to food, water, housing security, and reconciliation more broadly. Similarly, another participant undertaking program evaluation in a cross-cultural education context noted how the process helped identify a variety of external elements that could affect program success outside of the scope of the educational initiative itself. These issues included the influence of community politics, national conversations on reconciliation, and the effects of organizational mandates and priorities. Considering the program and its activities from a broader systems perspective made these “external factors” and their linkages more apparent. One participant noted, for instance, how the “process forced me to… picture how my specific research results would not just answer my research objectives, but also contribute to a broader sense of change [in the Great Lakes]. This in itself helped me visualize a desirable, sustainable future that could be achieved through my research. Now, with a start point [the research] and an end point [desirable future], I had to think critically… on how to fill in the gap” (i.e., what needs to occur to connect the start and end point, how feasible are the inputs and assumptions, can this be measured and if so how, etc.). Importantly, this individual noted that the strategy and skills of developing starting and end points first, then thinking critically through the messy middle, are likely transferable to many other professional situations.
The theory of change process offered a number of positive outcomes. However, applying a theory of change in the context of a graduate class on “Sustainability Applications” had some weaknesses and limitations as well, and we reflected on these as a group. Four main weaknesses or limitations were identified: (1) meaningfully grappling with complexity, (2) the challenge of articulating aspirational goals, (3) the difficulty in establishing clear boundaries for the theory of change, and (4) some practical and logistical challenges. Some of the insights on weaknesses and challenges can be addressed in iterations of the course, whereas others are more reflective of the application of a theory of change more generally.
Despite the systems emphasis, a theory of change may oversimplify real-world challenges, especially when the focus is related to wicked problem contexts that involve climate change, poverty, issues of power, and so forth. Any process or tool will be challenged to capture longer term effects and relationships; it is easier to identify and document existing relationships and interactions among different variables of interest. Similarly, there will always be unintended consequences that are not effectively or fully considered in a theory of change, and even if we aim to emphasize a systems perspective, many of us have a natural tendency to approach change in a unidirectional or linear fashion. Developing the tools and cognitive capacities to forge a truly systems perspective requires time.
A core discussion point was the extent to which the process truly helps us as researchers to grapple with the complexity of transdisciplinary problems. One participant noted, “…it can be difficult, especially as an early career researcher, to fully understand the complexity of these wicked problems. Trying to build up my baseline understanding of many of these issues was my largest barrier in tackling this project.” As noted by several course participants, the process does make it hard to make decisions about how to focus. For example, one individual noted that ecological systems have a multitude of factors driving changes within them, and it is impossible for a researcher to acknowledge every single driver of ecological change within their own study. We noted that important factors of change could be missed in the absence of more in-depth consideration that is not always possible when undertaking the process. For some participants, this challenge of envisioning diverse inputs and assumptions was difficult and, at times, overwhelming. As a result of this challenge, some theory of change efforts risk getting too complicated, making them hard to read and follow. However, trying to narrow the theory of change down to the most critical issues and then identifying causality among these inputs and assumptions is also difficult and may lead to “stretching” the key linkages (i.e., weak linkages in relation to the larger problem context). This can lead to a process that is too ambiguous and is characterized by vague interventions, rather than grounded in critical reflection and action. In this regard, one participant in this process noted that the theory of change is a “good tool, not a great tool”.
A core feature of the process is to articulate overarching or aspirational goals. However, articulating such goals as individual early career researchers can be rather challenging, especially because many are still in the process of developing more specific proposals. As some individuals noted, it was easy to overbroaden a desirable future, and significant effort was needed to refine aspirational statements to keep them realistic. Often, master’s projects are limited in impact and scope based on time requirements; therefore, it may not be easy to directly identify the sustainability contributions or the aspirational goals of the system. For example, one student undertaking evaluation of a program in reconciliation studies described being “uncomfortable” making linkages between the program itself and the broader goal of reconciliation in Canada. While not discrediting the value of the research being undertaken, it can still feel uncomfortable overassuming the value and impact of master’s research in the larger context of sustainability.
Moreover, there is inevitably some bias in how any theory of change is framed in this context, as several participants noted. In the context of this assignment, the theories of change were created individually, and so the process and aspirational goals did not easily emerge as a shared vision of the future. This aspect makes it more challenging to justify choices. However, while this is an issue that students faced in the context of the assignment, the challenge of creating a tangible, shared vision for a better future (by all stakeholders involved) for any particular social-ecological system is an unavoidable challenge faced by even the most experienced sustainability professionals, researchers, and decision makers. Thus, the process reflected, at least in part, how difficult and complex fostering sustainable change can be (e.g., planning and implementing appropriate policy, changing public behavior, etc.).
Similarly, for most of us, the insights we used for the theory of change were predominately sourced from Western science perspectives, yet many of the problems being considered must include indigenous perspectives and experiences. Despite the fact that a process requires us as researchers to think about multiple perspectives, finding meaningful ways to bring these perspectives into the process itself remains a significant barrier. Of note, some individuals argued that this same weakness or challenge also applied to thinking through the indicators we might use to track progress. Specifically, determining which indicators are most effective depends on the topic or issue being addressed and the extent to which there are adequate perspectives to identify specific, measurable, realistic, and timely indicators.
A further weakness or challenge of the approach for many in the class involved the process of setting boundaries for their theory of change. For example, one of us noted, “…the theory of change exercise enabled me to focus on the importance of collaboration with many partners, which is crucial in reaching long-term project goals.” She added, however, “… I believe that continued thought into how to approach a theory of change will lead to them being useful tools, if specific boundaries are developed into a theory of change framework to minimize the broadness of a project’s scope.” This insight highlighted a common challenge for all of us as we have engaged in theory of change activities in different contexts. Boundary setting is a crucial task and one for which significant time is needed unless the project risks getting too big. Indeed, another of us noted that the process, in some ways, does the opposite of bounding the project, especially for early career researchers focusing on a finer scale project; e.g., species-specific requirements for habitat, food, etc., cannot be considered as easily through a theory of change process. Overall, many found that it was difficult to answer questions and advance through a project when boundaries can be limitless and nonspecific because it leads to a lack of ability to use well-justified methods of analysis. For many, as the projects were developing, everything “kept scaling up,” making it difficult to put boundaries around the issues being considered. One individual noted, “…it is difficult to evaluate the needs of one system component without evaluating the relationship between the roles of all components and their contributions to the whole system functioning and to the functioning of other, related, and embedded systems.”
The process also involved a few practical or situational challenges for the students. The most central of these challenges was the time constraint on getting the project done. The course runs over four months, but most participants are simultaneously taking other courses, working on their specific research proposal, and serving as teaching assistants or working other part-time jobs. Engaging in a complex thought process such as a theory of change places an additional time burden over and above what might be required for a more conventional assignment. Also identified during our collective assessment was the challenge of completing a theory of change as an individual assignment, and so despite “looking good on paper,” the final product can still be somewhat unrealistic and reflect only one possible vision for the future. This makes it difficult to validate the various inputs and steps, especially when considering that many of these steps rely on uncertain policies and politics that may change over time. One participant noted, “…there’s really no good way to evaluate the success of using a theory of change to shape a project… how do we know that by actually doing that exercise (e.g., outlining the actors, assumptions, indicators, etc.) actually helped us to better achieve our [aspirational] goal? We don’t concretely know if it actually made the research process more efficient, nor do we know if it made the projects themselves more well-rounded.” This is a legitimate concern in undertaking these forms of learning approaches, but as the other comments and insights above tend to reveal, the collective benefits of more critical analysis and thinking about complexity are likely to be advantageous in the longer term. Ultimately, creating a vision of sustainability is often in a best-case scenario framing, as opposed to what might be the most likely, plausible, or ideal situation in the future.
A number of opportunities to extend the theory of change in productive ways were also identified in our synthesis process. Many of these opportunties reflect ways in which the process could be used that were not initially emphasized or designed into the course activity. These opportunities are related to: (1) better use of the theory of change to communicate ideas, (2) opportunities to use the process to develop common assessment criteria, and (3) the adaptability of the theory of change to diverse problem contexts.
As noted, the theory of change in this context was largely designed as an individual project. However, the value of collaborating more explicitly in the development of the activity was highlighted as a potentially important opportunity, and one that is consistent with the principles of transdisciplinarity. Specifically, the opportunity exists for many of us to reach out early in the process to potential research partners (e.g., community organizations, government representatives, city planners, etc.) and seek opportunities to give more voice to those with whom we intend to work. As early career researchers still working to develop research ideas, this is no small task, but there are opportunities for many (not everyone) to work through supervisor contacts to initiate this process. The outcomes of early engagement would enhance the thinking and help to ground the analysis. However, when completing the theory of change, there are also opportunities for better collaboration within the class, which would also enhance our collective understanding of transdisciplinary efforts. For example, students working with the same supervisor or project type could develop shared visuals but generate individual narratives. Regardless of the context (within class or with potential research partners), it is likely that some conflicts might arise about how to frame the theory of change and what interventions make the most sense. Such conflicts, however, should be seen as opportunities to colearn and develop agreements on how to achieve sustainability outcomes.
As reflected in the two visual examples of a theory of change from course participants (Figs. 2 and 3), there is an opportunity expressed by the group to use the process and outcome more effectively as a communication and presentation tool. For example, in seeking to communicate research ideas and the broader context in which individuals are seeking to make change, the clearly visualized pathways to achieve aspirational outcomes can be more easily understood by people who may not be directly involved in that process or issue. Moreover, some of the associated diagrams (e.g., results chains) and mind maps can be used in presentations and other media. This capacity to communicate ideas using the process can also be helpful in identifying research gaps and, therefore, in opening avenues for new research questions. One participant noted that her project on ecological restoration highlighted a divide between how land managers perceive and use the restoration tools being developed and how they were intended to be used. However, her theory of change provides a way to communicate how these tools should fit together to support the broader aspiration of sustainable land management. Finally, a participant recognized the benefit of adopting a theory of change process outside of the classroom. She noted, in particular, that in applying for jobs (e.g., in the environmental consulting field), companies seem interested in individuals with the ability to consider the contributions and insights from multiple sectors. She found her theory of change fostered this mindset and would give her some additional tools to communicate these skills to potential employers.
The process has a common structure, and there is a focus on developing specific and reliable indictors to measure progress. However, as was pointed out by the group, there are no common criteria with which to assess the validity, efficacy, or success of a theory of change. Thus, it was suggested that there exists an opportunity to integrate more comprehensively well-established sustainability assessment criteria (Gibson et al. 2005) to provide a common foundation for analysis. Sustainability assessment criteria may include equity (inter- and intragenerational), precaution, integration, integrity, efficiency, and good governance. Sustainability criteria can also be used as an analytical tool to provide additional justification for different choices and elements of the theory of change. Finally, sustainability criteria can also help to translate the theory of change into more practical actions that will lead to positive outcomes on the ground (Gibson et al. 2005).
Ultimately, the projects developed as part of the graduate class reflect individual visions for how to situate more specific research in a larger problem context and in relation to the communities and organizations most affected by that problem. As noted, more could be made of the adaptability and transferability of these theory of change outcomes because they may be applicable to other projects or similar situations elsewhere. As an example, a theory of change for improving food security in one area of Canada could indicate the first necessary steps that should be taken for another area in the country also seeking to improve food security. Regardless, based on experience with the assignment, more emphasis on recognizing theory of change initiatives as being fluid and adaptable was encouraged. Emphasizing the ability of a theory of change to be an effective problem-solving tool in multiple contexts can highlight the importance of disseminating and sharing these discussions on sustainability.
There is a well-established idiom that states “good theory makes good practice.” Our aim here was to reflect critically on the theory of change as one pedagogical tool to help early career researchers in their development of transdisciplinary thinking and action. A growing number of programs are emphasizing the protection and recovery of threatened or damaged communities and ecologies, and simultaneously, are reflecting on and engaging with a range of stakeholders to reform or transform the institutions and practices that contribute to unsustainability (Schwartz et al. 2017). However, there is significant scope to further develop and test tools and processes to support these efforts, and to engage with early career researchers in reflections about how these tools support their own learning outcomes.
As our insights show, and regardless of our intellectual footing (e.g., ecology, policy, restoration, governance, etc.) or applied aspirations (e.g., researcher, resource manager, outreach and communication specialist), a theory of change can help encourage transdisciplinary thinking for sustainability. Specifically, the process provides helpful architecture for systems thinking while creating the space for creativity as students focus on the issues that align with their own more focused research. As one participant summarized, “…the theory of change exercise provided me with a toolkit for stepping outside of my own silos and predispositions in order to situate my research within a broader sustainability context. It gave new meaning to my work and aspirations as an early scholar and change-maker.”
There are a number of strengths and opportunities associated with the theory of change process, but some weaknesses and challenges as well. Encouraging early career researchers and future practitioners to develop a systems-informed theory of change supports the transdisciplinary thinking and skills needed to address the complex or wicked problems we will increasingly confront, regardless of our disciplinary starting points. As applied in the context of one course, the theory of change did encourage reflection on the broader implications for each course participant’s more narrow research. For those considering the use of a similar process in the context of their own classes (or even as individual graduate students), the insights from this class experience highlight a few key issues.
First, the process of developing a theory of change is challenging, and adequate time is needed for missteps and reflection. Establishing an aspirational outcome that is framed at the appropriate level inevitably involves numerous attempts. Similarly, setting objectives that are too broad makes the process too abstract, whereas setting objectives that are too narrow constrains efforts to reflect deeply on the core assumptions and connections that influence our research efforts.
Second, as an academic or class-based exercise, the direct implications or benefits of the process are not immediately clear, although the thinking behind a theory of change emerges in the eventual development of proposals and theses. However, as one participant noted, “The goal of the theory of change is not to solve all of the world’s wicked problems in one go. These problems exist because no one has the answers or has figured out how to solve them. The goal of the theory of change mirrors that of any graduate research project: to think critically about these problems and find potential pathways for contributions and change.”
Third, like similar program evaluation tools and logic models, the process may encourage early career researchers (and others) to be overly mechanistic, such as when seeking causal relationships among inputs, activities, and outcomes. However, the combined emphasis on unpacking core assumptions in the theory of change process along with exposure to transdisciplinary literature does provide an opportunity for individuals to think more systemically. Ultimately, no single tool is adequate to foster transdisciplinary thinking and action, but experiential processes and repeated exposure to the ideas is needed to foster desired learning outcomes.
Finally, many students have not yet established clear relationships with research partners, and this does place some limits on the transdisciplinary intent of the process. Nevertheless, using the process to engage sooner with potential community, government, and civil society partners opens the door to the types of novel institutional arrangements needed to blend science, policy, and action for sustainability challenges (Cvitinovic et al. 2018). As summarized by one participant, “…by taking the time to uncover the ways in which [my] research could help create change… the more driven I will be.”
We gratefully acknowledge the constructive feedback on this manuscript from the anonymous reviews and subject editor.
Adler, C., G. Hirsch Hadorn, T. Breu, U. Wiesmann, and C. Pohl. 2018. Conceptualizing the transfer of knowledge across cases in transdisciplinary research. Sustainability Science 13(1):179-190.
Alexander, K. A., A. J. Hobday, C. Cvitanovic, E. Ogier, K. L. Nash, R. S. Cottrell, A. Fleming, M. Fudge, E. A. Fulton, S. Frusher, R. Kelly, C. K. Macleod, G. T. Pecl, I. van Putten, J. Vince, and R. A. Watson. 2018. Progress in integrating natural and social science in marine ecosystem-based management research. Marine and Freshwater Research 70(1):71-83. https://doi.org/10.1071/MF17248
Baldauf McBride, B., C. A. Brewer, M. Bricker, and M. Machura. 2011. Training the next generation of Renaissance scientists: the GK–12 ecologists, educators, and schools program at the University of Montana. BioScience 61(6):466-476. https://doi.org/10.1525/bio.2011.61.6.9
Ban, N. C., E. Boyd, M. Cox, C. L. Meek, M. Schoon, and S. Villamayor-Tomas. 2015. Linking classroom learning and research to advance ideas about social-ecological resilience. Ecology and Society 20(3):35. http://dx.doi.org/10.5751/ES-07517-200335
Brandt, P., A. Ernst, F. Gralla, C. Luederitz, D. J. Lang, J. Newig, F. Reinert, D. J. Abson, and H. von Wehrden. 2013. A review of transdisciplinary research in sustainability science. Ecological Economics 92:1-15. https://doi.org/10.1016/j.ecolecon.2013.04.008
Brown, J., and D. Isaacs 2005. The world café: shaping our futures through conversations that matter. Berrett-Koehler, San Francisco, California, USA.
Campbell, L. M. 2005. Overcoming obstacles to interdisicplinary research. Conservation Biology 19(2):574-577. https://doi.org/10.1111/j.1523-1739.2005.00058.x
Capra, F. 2004. The hidden connections: a science for sustainable living. Random House, New York, New York, USA.
CARE. 2012. Peacebuilding with impact: defining theories of change. CARE International, London, UK. [online] URL: https://insights.careinternational.org.uk/media/k2/attachments/CARE-Defining-Theories-of-Change-document.pdf
Commonwealth of Australia. 2007. Tackling wicked problems: a public policy perspective. Government of Australia, Canberra, Australia. [online] URL: https://www.apsc.gov.au/tackling-wicked-problems-public-policy-perspective
Conservation International. 2013. Constructing theories of change models for ecosystem-based adaptation projects: a guidance document. Conservation International, Arlington, Virginia, USA.
Creswell, J. W. 2005. Educational research: planning, conducting, and evaluating quantitative and qualitative research. Pearson Education, Upper Saddle River, New Jersey, USA.
Cvitanovic, C., M. F. Löf, A. V. Norström, and M. S. Reed. 2018. Building university-based boundary organisations that facilitate impacts on environmental policy and practice. Plos One 13(9):e0203752. https://doi.org/10.1371/journal.pone.0203752
Douthwaite, B., K. Kamp, C. Longley, F. Kruijssen, R. Puskur, T. Chiuta, M. Apgar, and P. Dugan. 2013. Using theory of change to achieve impact in AAS. AAS Working Paper. CGIAR, Montpellier, France. [online] URL: http://pubs.iclarm.net/resource_centre/AAS-theory-of-change-impact.pdf
Earl, S., F. Carden, and T. Smutylo. 2001. Outcome mapping: building learning and reflection into development programs. International Development Research Centre, Ottawa, Canada. [online] URL: https://www.idrc.ca/en/book/outcome-mapping-building-learning-and-reflection-development-programs
Eigenbrode, S. D., M. O’Rourke, J. D. Wulfhorst, D. M. Althoff, C. S. Goldberg, K. Merrill, W. Morse, M. Nielsen-Pincus, J. Stephens, L. Winowiecki, and N. A. Bosque-Pérez. 2007. Employing philosophical dialogue in collaborative science. Bioscience 57(1):55-64. https://doi.org/10.1641/B570109
Elliot, J., S. Heesterbeek, C. J. Lukensmeyer, and N. Slocum. 2005. Participatory methods toolkit: a practitioner’s manual. King Baudouin Foundation and Flemish Institute for Science and Technology Assessment, Brussels and Antwerp, Belgium. [online] URL: https://www.livingknowledge.org/fileadmin/Dateien-Living-Knowledge/Dokumente_Dateien/Toolbox/LK_A_Particpatory_Methods.pdf
Enquist, C. A. F., S. T. Jackson, G. M. Garfin, F. W. Davis, L. R. Gerber, J. A. Littell, J. L. Tank, A. J. Terando, T. U. Wall, B. Halpern, J. K. Hiers, T. L. Morelli, E. McNie, N. L. Stephenson, M. A. Williamson, C. A. Woodhouse, L. Yung, M. W. Brunson, K. R. Hall, L. M. Hallett, D. M. Lawson, M. A. Moritz, K. Nydick, A. Pairis, A. J. Ray, C. Regan, H. D. Safford, M. W. Schwartz, and M. R. Shaw. 2017. Foundations of translational ecology. Frontiers in Ecology and the Environment 15(10):541-550. https://doi.org/10.1002/fee.1733
Gasper, D. 2000. Evaluating the ‘logical framework approach’ towards learning-oriented development evaluation. Public Administration and Development 20(1):17-28. https://doi.org/10.1002/1099-162X(200002)20:1<17::AID-PAD89>3.0.CO;2-5
Gibson, R. B., S. Hassan, S. Holtz, J. Tansey, and G. Whitelaw. 2005. Sustainability assessment: criteria and processes. Earthscan, London, UK.
Hackett, E. J., and D. R. Rhoten. 2009. The snowbird charrette: integrative interdisciplinary collaboration in environmental research design. Minerva 47(4):407-440. https://doi.org/10.1007/s11024-009-9136-0
Hirsch Hadorn, G., D. Bradley, C. Pohl, S. Rist, and U. Wiesmann. 2006. Implications of transdisciplinarity for sustainability research. Ecological Economics 60(1):119-128. https://doi.org/10.1016/j.ecolecon.2005.12.002
James, C. 2011. Theory of change review. Report commissioned for Comic Relief. Comic Relief, London, UK. [online] URL: http://www.theoryofchange.nl/sites/default/files/resource/2012comicrelieftocreviewfinal.pdf
Kates, R. W., W. C. Clark, R. Corell, J. M. Hall, C. C. Jaeger, I. Lowe, J. J. McCarthy, H. J. Schellnhuber, B. Bolin, N. M. Dickson, S. Faucheux, G. C. Gallopin, A. Grübler, B. Huntley, J. Jäger, N. S. Jodha, R. E. Kasparson, A. Mabogunje, P. Matson, H. Mooney, B. Moore III, T. O’Riordan, and U. Svedin. 2001. Sustainability science. Science 292(5517):641-642. http://dx.doi.org/10.1126/science.1059386
Kelly, R., M. Mackay, K. L. Nash, C. Cvitanovic, E. H. Allison, D. Armitage, A. Bonn, S. J. Cooke, S. Frusher, E. A. Fulton, B. S. Halpern, P. F. M. Lopes, E. J. Milner-Gulland, M. A. Peck, G. T. Pecl, R. L. Stephenson, and F. Werner. 2019. Ten tips for interdisciplinary socio-ecological researchers. Social-Ecological Practice Research 1(2):149-161. https://doi.org/10.1007/s42532-019-00018-2
Klein, J. T. 2013. The transdisciplinary moment(um). Integral Review 9(2):189-199. [online] URL: http://www.integral-review.org/issues/vol_9_no_2_klein_the_transdiciplinary_moment(um).pdf
Lang, D. J., A. Wiek, M. Bergmann, M. Stauffacher, P. Martens, P. Moll, M. Swilling, and C. J. Thomas. 2012. Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustainability Science 7(S1):25-43. https://doi.org/10.1007/s11625-011-0149-x
Lebel, J., and R. McLean. 2018. A better measure of research from the global south. Nature 559:23-26. https://doi.org/10.1038/d41586-018-05581-4
Lélé, S., and R. B. Norgaard. 2005. Practicing interdisciplinarity. Bioscience 55(11):967-975. https://doi.org/10.1641/0006-3568(2005)055[0967:PI]2.0.CO;2
Miller, T. R., T. D. Baird, C. M. Littlefield, G. Kofinas, F. S. Chapin III, and C. L. Redman. 2008. Epistemological pluralism: reorganizing interdisciplinary research. Ecology and Society 13(2):46. [online] URL: http://www.ecologyandsociety.org/vol13/iss2/art46/
Mitchell, C., D. Cordell, and D. Fam. 2015. Beginning at the end: the outcome spaces framework to guide purposive transdisciplinary research. Futures 65:86-96. https://doi.org/10.1016/j.futures.2014.10.007
Pohl, C., and G. Hirsch Hadorn. 2008. Methodological challenges of transdisciplinary research. Natures Sciences Sociétés 16(2):111-121. https://doi.org/10.1051/nss:2008035
Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and L. C. Stringer. 2010. What is social learning? Ecology and Society 15(4):r1. [online] URL: http://www.ecologyandsociety.org/vol15/iss4/resp1/
Rittel, H. W. J., and M. M. Weber. 1973. Dilemmas in a general theory of planning. Policy Science 4(2):155-169. https://doi.org/10.1007/BF01405730
Schwartz, M. W., J. K. Hiers, F. W. Davis, G. M. Garfin, S. T. Jackson, A. J. Terando, C. A. Woodhouse, T. L. Morelli, M. A. Williamson, and M. W. Brunson. 2017. Developing a translational ecology workforce. Frontiers in Ecology and the Environment 15(10):587-596. https://doi.org/10.1002/fee.1732
Taplin, D., and H. Clark. 2012. Theory of change basics: a primer on theory of change. ActKnowledge, New York, New York, USA.
USDA (United States Department of Agriculture). 2008. SWOT analysis: a tool for making better business decisions. USDA Risk Management Agency, Washington, D.C., USA. [online] URL: https://legacy.rma.usda.gov/pubs/2011/swot_brochure.pdf
Valters, C. 2015. Theories of change: time for a radical approach to learning in development. Overseas Development Institute, London, UK. [online] URL: https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9835.pdf
van Tulder, R., and N. Keen. 2018. Capturing collaborative challenges: designing complexity-sensitive theories of change for cross-sector partnerships. Journal of Business Ethics 150(2):315-332. https://doi.org/10.1007/s10551-018-3857-7
Vogel, I. 2012. Review of the use of ‘Theory of Change’ in international development: review report. Department of International Development, London, UK. [online] URL: http://www.theoryofchange.org/pdf/DFID_ToC_Review_VogelV7.pdf
Welch-Devine, M., D., Hardy, J. P. Brosius, and N. Heynen. 2014. A pedagogical model for integrative training in conservation and sustainability. Ecology and Society 19(2):10. https://doi.org/10.5751/ES-06197-190210
World Commission on Environment and Development (WCED). 1987. Our common future. Oxford University Press, Oxford, UK.