Addressing sustainability challenges presented by current social and environmental change requires governance of environmental commons built on a clear understanding of social-ecological system (SES) relationships. A large body of literature on SES governance has identified a number of influential social and ecological attributes and the relationships among them (Partelow 2018). This literature has largely consisted of case studies using inconsistent measures for SES attributes, which has hindered comparative analyses and thus, generalizations about SES relationships.
Developing comparable case descriptions requires the use of a common framing to ensure that standard research protocols are used. In support of this common framing, Elinor Ostrom and colleagues at the Workshop in Political Theory and Policy Analysis at Indiana University developed several analytical frameworks. These “workshop frameworks” are the Institutional Analysis and Development (IAD) Framework (Ostrom 2005), the Social-Ecological Systems (SES) Framework (Ostrom 2007), and the Robustness Framework (Anderies et al. 2004). These frameworks are designed to enable analyses of systems in which human actors, frequently resource users, interact with each other and their social and biophysical environments. The IAD framework centers around the idea of an action situation, or a situation in which actors make decisions that jointly affect each other’s welfare. How decision makers interact depends in turn on biophysical, social, and institutional conditions. The SES framework directly expands on the IAD, although it incorporates the language of variable tiers and more extensively unpacks the social and ecological variables that affect an action situation. The Robustness Framework similarly considers social-ecological systems and the resource users within them that interact to use natural resources; but it focuses less on focal action situations and more on adaptability as an outcome and the public infrastructure that affects resource user interactions.
Although these frameworks provide a common perspective for scholars, the literature on commons governance that employs them has mostly remained idiosyncratic in terms of the choice, definitions, and measures of variables (Thiel et al. 2015, Partelow 2018, Schlager and Cox 2018). This, together with a tendency to employ a case study approach (Poteete and Ostrom 2008), hinders synthetic and comparative research. Nevertheless, there have been several collaborative, data-driven projects based on the workshop frameworks that have stood out in utilizing them to produce consistent descriptions of a set of commons-oriented cases. In this paper, we summarize these projects (Table 1) and the lessons learned from them. Strengths and weaknesses are discussed in order to derive guidance for future broadly comparative social-ecological research.
In this paper we focus on broadly comparative research projects that use the workshop frameworks to make comparisons among a relatively large number of commons-oriented cases. What we mean by “broadly” here is domain specific. Much of the truly large-n analysis found in social science research occurs at the individual level or the national level (Poteete et al. 2010). Within our context, the unit of analysis ranges from small, e.g., local communities of natural resource users, to large governance systems, e.g., regional marine protected area systems, conservation agreements on the open seas, or international outcomes of conservation interventions. With these larger units of analysis, “broadly” does not mean that thousands of observations are analyzed; rather we are referring to databases of several dozen to several hundred observations, and often comprising different social and ecological contexts (see Table 1).
Arguably, this shortage of comparative data at critical scales reflects a failure of collaboration and coordination within the commons community. The commons community itself has developed the analytical tools to understand this as well. As Poteete et al. (2010:262) describe, collaborative research partnerships, like the ones we are considering, face collective-action problems, or a divergence between the interests of the group as a whole, and its individual members:
Most scholars are individually better off if they invest in specialized skills and publish primarily single-method research in disciplinary journals; they are more likely to get a good job, get their research published quickly, receive tenure, and win grants and awards. The scholarly and policy communities, however, would be collectively better off if more research made use of multiple methods and integrated insights from a variety of disciplines.
This challenge is ubiquitous and derives from the public good nature of science and collaborative enterprises, and is compounded for data-intensive projects because of the infrastructure involved, itself a public good. Moreover, the number of participants involved in many of these projects raises coordination costs and makes it more difficult to prevent free-riding (Casari and Tagliapietra 2018). The diversity of perspectives that are involved can also make coordination more difficult. Representational diversity, for example, can help groups to a point, but, after that it becomes increasingly difficult for people to begin to understand each other, especially if the diversity in question is related to core values beliefs, and objectives of individuals (Cronin and Weingart 2019). As Poteete et al. (2010) also describe, the level of fragmentation within academia stymies efforts to address these challenges. Within academia there exists a high level of individualism at the level of the researcher, the research group, and the discipline. For each unit of social organization, collaboration is threatened by competition.
These challenges have arguably been more effectively met within the life sciences than in the social sciences, or within the social-ecological space between them (Parker et al. 2010). The most notable examples of research networks within ecology include the Long-Term Ecological Research (LTER) and National Ecological Observatory Network (NEON) programs. Also of note is the National Center for Ecological Analysis and Synthesis (NCEAS), which has supported the synthesis of ecological data and research outputs for some time (Baron et al. 2017). Based on the NCEAS model, the National Socio-Environmental Synthesis Center (SESYNC) has stood since its inception as the pre-eminent center for the synthesis of social-ecological information, based on a rigorous approach to team science (Palmer et al. 2016). Indeed, this paper was conceived at a SESYNC working group meeting.
Although the findings associated with the research on working groups at these synthesis centers are important (Hampton and Parker 2011), these groups operate in a specific context, during a relatively short period of time and are supported by the center’s infrastructure with the goal of aggregating existing data. Less is known about projects in “the wild,” particularly interdisciplinary and transdisciplinary data-driven social-ecological projects of the kind we focus on here.
In addition to the challenge of collaboration, Poteete and Ostrom (2008) find that a critical challenge facing broadly comparative research is the tension between complexity and comparability (see also Cox 2008). With a limited amount of resources, each project must incur some opportunity cost by focusing more on individual cases, or more on representing them in a way that enables a comparison across them. Poteete and Ostrom (2008:189) describe a way to meet this challenge:
Guidelines and checklists facilitate comparative research while leaving researchers with considerable flexibility to adapt their data-collection strategies to local conditions, or to supplement the basic set of issues to pursue local concerns.
We believe that these challenges are intertwined, and that a guiding checklist can help with each of them. The use of the word “checklist” here might surprise some readers. In a popular book on the topic, Gawande (2014) describes how checklists can be used to increase the consistency of a set of practices among practitioners. From this perspective, checklists are the enemy of inconsistency (also known as flexibility), by design. This lens has been used to criticize supposedly “blueprint” approaches to management as being overly formulaic (Cox et al. 2010). Here we reinterpret this concept to be at the center of the just discussed tension between flexibility and uniformity. Checklists can help researchers maintain a balance between flexibility (complexity) and uniformity (comparability) if they are presented not as simple boxes to be checked based on highly legible criteria, but more as guiding questions to be used as the basis for discussion among participants. In this way, our view of checklists is closer to the idea of a diagnosis rather than that of a blueprint. In what follows we reflect on each of the projects, their benefits, and the two fundamental issues addressed herein. We conclude by offering a version of the checklist that Poteete and Ostrom (2008) had called for. In so doing we hope to contribute to the ability of future scholars to address the key challenges discussed.
See Appendix 1 for full descriptions of each of the seven projects. Each project is summarized by the following categories: (1) origins and purpose, (2) description and process, and (3) innovations, outcomes, and lessons learned. The Common-Pool Resource (CPR), Nepal Irrigation Institutions and Systems (NIIS), and the International Forestry Resources and Institutions (IFRI) projects were all developed at the Ostrom Workshop in the 1990s, while the others discussed here were developed in the following decades. These first three projects were designed to analyze the institutional dynamics of community-based natural resource management. The databases for these projects have several similarities owing to the overlapping network of participants and a common underlying framework (the IAD), variables, and definitions, which were established through the development of the first CPR database. The NIIS database further developed and refined the CPR database to specifically address the irrigation sector, and later to incorporate primary data. According to Elinor Ostrom (verbal communication), NIIS and IFRI are each a further development of the CPR adapted to the respective resource. Of the 593 variables that comprise the CPR database, 319 are also found in NIIS (54%), accounting for approximately 58% of its 554 variables. For IFRI, it has been estimated that the overlap is roughly 20%, although direct comparison is not feasible because of the use of an alternative system of naming conventions. The final four projects all drew on the original three and their use of the frameworks in different ways. Interestingly, none of them used the IAD framework on its own, although this still played a central conceptual role in each project.
The primary benefit of datasets containing a large number of cases is that they help address the “many variables, few cases” problem that characterizes smaller-n research, and facilitate comparative analyses that allow us to assess the generalizability of important relationships. Each project examined here has been used to develop and test hypotheses concerning the social, institutional, and biophysical drivers of environmental sustainability. The databases have thus enabled the study of SES governance to evolve from single-case based research to multiple-case comparisons enabling higher generalization to theory. Perhaps counterintuitively, the supporting infrastructure developed by several projects enables users to unpack the complexity of specific cases and objects, partly through visualization (SES Library, SESMAD, MACMON). In the SESMAD database, for example, the complexity of case representation facilitated a novel technique for case study analysis.
Some benefits are based on the type of data collected, and are not limited to broadly comparative research. For the projects that involve the coding of secondary data, this process serves as a type of structured literature review, and participants can gather a synthetic understanding of a literature. For projects that involve primary data there is the opportunity to develop collaborative relationships with local partners. This creates the potential for the knowledge produced by these projects to fill knowledge gaps of local partners in particular times and places.
For the broader research community, comparative databases provide scholars with a broad menu of potentially relevant factors, and an approach for consistently measuring them. Although lists of variables may seem a rather limited contribution, they encapsulate the evolution of the sustainability sciences from a field emphasizing simple panacea solutions to one involving complex interactions among a wide range of social, biophysical, and institutional variables (Agrawal 2003, Ostrom 2007, Liu et al. 2007). Students and researchers can draw from these lists to develop comparable research protocols and findings, and avoid reinventing the wheel when it comes to operationalizing and measuring key concepts and variables. The SESMAD website and the practitioner’s manual for the MACMON project (Gurney and Darling 2017), for example, provide guidance for how future scholars and practitioners can implement the SES framework. Through such learning we may be able to produce more consistent projects and analyses in the future, even if these are not fully integrated into a common project.
Producing the benefits just described requires collaboration among a set of academics and often nonacademic partners. As described above, the challenge of scientific collaboration can be straightforwardly interpreted as a public goods dilemma: it can be difficult to exclude members of a collaboration from the benefits it produces, leading to the free-rider problem. As seen in the previous section, there are multiple benefit streams that the projects produce. For the research group, a direct benefit comes from the publications through which the contributions of the project become legible to the broader community, enabling participants to benefit reputationally (see Fig. 1).
In order to produce these publications, several inputs are needed. Underlying all other aspects of a project are the resources needed to carry out activities. These include social capital, by which we mean the mix of formal rules and informal norms that are required in order to provide incentives for researchers to contribute and a predictable social structure within which to operate: ideally a well-ordered group has positions that confer well-understood rights and responsibilities, and these are at least minimally codified. In each of the projects discussed here, a subset of actors acted as leaders, lowering the cost of participation for other participants, in part by establishing what the rights and responsibilities for the group members are.
A project’s social structure should guide the process of developing the needed technical infrastructure, which includes guidance materials that provide project partners with a common definition of cases, components, and variables, as well as the procedures to use for collecting and entering data into the database. The construction of the databases involves the creation of at least two public goods. First, there is the data storage and management infrastructure. This includes the development of at least a coding manual and data storage platform. Providing all this infrastructure requires reaching a minimum of collaboration (threshold); however, once the infrastructure is provided, its maintenance is comparatively lower. Then, there is the data itself, which is a good that grows with the number of contributors. Thus, while infrastructure can be provided by a small group of collaborators or even a single leader, the population of the database requires the collaboration of larger numbers of contributors. The quality of the data is dependent upon the skills and fidelity of those who collect and enter them, and this issue becomes more challenging over time as different researchers (and other contributors) enter and exit the team.
A significant barrier to infrastructure development is that researchers often lack the knowledge required to design, develop, and maintain databases. In multiple projects this gap has been filled by IT professionals, but longevity of these contractual relationships can be difficult to maintain. Open-source software products, i.e., Kobo Toolbox, are a potential solution to sustainability and customization. Furthermore, technical difficulties, such as formulating queries in relational databases, can persist beyond the data collection stage, requiring an ongoing emphasis on strengthening data science skills with project teams, or collaborations with data scientists. In the past, this led to the replacement of complex analyses by simple ones, limited to a few key variables, with which simpler analyses such as linear regressions are then performed. This means that the division of the data into many tables (users, resources, rules) because of good technical practice de facto leads to simpler scientific analyses, since the lack of IT knowledge means that information can no longer be brought together in analyses. Here it can help to use graphical user interfaces to increase accessibility even for users who are not so technically proficient and for whom a collaborative project is one among many.
As discussed in Poteete et al. (2010), there are aspects of the research ecosystem that can make these dilemmas particularly challenging. Academic organizations tend to be individualistic, with individual-level metrics that lead to rewards and prestige. Collaborations such as those discussed here most often involve partners from different institutions who voluntarily join temporary collaborative groups, rather than belonging to a relatively fixed team within an organization, unlike groups in national science labs in the United States or others within the CGIAR system. A further problem is that grants are usually short-term, i.e., 2–3 years. Developing and maintaining a database, e.g., updating the technical infrastructure or entering new cases, however, is a long-term project.
As a result of these collective action challenges, progress in maintaining and expanding the databases has been intermittent for most of these projects as scholars struggle to maintain incentives for detailed data collection and coding, and to collect longitudinal data, with the notable exception of the IFRI database, where for some locations up to five measurements are available at intervals of a few years, and the NIIS database, where an extremely high coverage and completeness of the data was achieved through targeted visits to irrigation systems where the data situation was inadequate.
Further, those broadly comparative projects that employ an interdisciplinary approach or involve collaborations with nonacademic actors face additional collaboration challenges. Although most of the projects focus primarily on social and institutional information, in others, substantial biophysical (e.g., MACMON, IFRI, NIIS) data are collected and represented, thus involving engagement with natural scientists. Further, some of the projects including primary data collection also involved engagement with resource management practitioners, from consultative to transdisciplinary approaches that entail knowledge coproduction to meet real-world information needs (Mauser et al. 2013). In the case of MACMON, the need for the project was identified by conservation practitioners and required subsequent collaboration with academics to develop and ground the project in theory. The subsequent diversity of perspectives involved in these cross-boundary approaches results in heterogeneous interests, priorities, project histories, epistemologies, and potentially tension over what constitutes salient, credible, and legitimate knowledge (Cash et al. 2003), particularly with respect to which SES attributes to include and how to operationalize them. Overcoming this tension and integrating diverse perspectives has been aided by employing the SES and IAD frameworks as basis for the projects because they effectively act as “boundary objects,” being both adaptable to different viewpoints and robust enough to maintain identity across them (Star and Griesemer 1989). Indeed, the frameworks provide a shared language, a point of reference, and a means to facilitate discussion and learning across boundaries (Gurney et al. 2019).
Collaborations with large conservation and development government and nongovernment organizations may provide a way to generate large-n longitudinal primary datasets. For example, given that the MACMON project was designed to meet the information and SES monitoring needs of the Wildlife Conservation Society’s global coral reef program, it now forms the basis of much of their monitoring and its future sustainability is being built on integration across multiple projects and country programs. Indeed, many similar organizations have ongoing commons management projects in multiple sites for which they are required to undertake social and ecological monitoring, thereby circumnavigating the challenges associated with largely academic initiatives.
In the face of these challenges, we believe that researchers must ask themselves if it is worthwhile to engage in a broadly comparative research project. If a group decides to engage in such a project, it will be helpful to tightly integrate the ongoing maintenance of the database with teaching practice, and/or funding and collaborative agreements. In general, the more synergies that can be built between the larger project and related side-projects and activities, the more sustainable it will be.
A central challenge inherent to developing broadly comparable data projects is navigating the trade-off between case-based relevancy and generalizability. This challenge stems from tensions between the goal of accumulating datasets of comparable cases and the underlying theoretical framework that suggests that SES outcomes depend upon interactions among a wide range of factors. The former encourages scholars to adopt a reductionist approach by applying rigid definitions of the types of cases analyzed, the number of components included, and rely heavily upon a small number of variables and associated questions with fixed-item responses. It can privilege SES attributes that are more easily quantified than those that are not, e.g., power, values, and cultural context. A standardized approach increases the possibility of oversimplification and omission of SES attributes that are important in a certain place, including local expressions of those attributes (Gurney et al. 2019). In contrast, the latter encourages scholars to adopt a flexible approach to provide a detailed representation of the components and the relationships among them as they occur in the world, and code them using large numbers of fixed and open-ended questions. The approach taken to reducing the trade-off between case-based relevancy and generalizability in the MACMON project was to design the standardized SES “core” framework with the intention that it be supplemented with context-specific indicators. A similar division between a “core” set of variables and more peripheral, context-specific variables has been discussed for both the IFRI and SESMAD projects as well.
Selection of variables is a core challenge that involves important questions about their theoretical relevance, the availability of indicators, and the relative costs and benefits of including them. In theory, we would expect the number of variables to increase with the number of research questions and diversity of cases that a project aims to address. In practice, however, database projects tend to err on the side of inclusion, reflecting the generally recognized importance of many factors (e.g., see Agrawal 2003). Large numbers of variables have the benefit of facilitating numerous publications from a single project, as most notably seen in the IFRI program. It can also help avoid the trap of “tunnel vision” where a very narrow portion of a system is examined to the exclusion of others, and analytical pluralism in general. But this also results in a laundry list of variables, most of which never feature in an empirical analysis. The IFRI data protocols, for example, contain a variable about improved bee-keeping techniques, which to our knowledge has never been used. Additionally, this situation can encourage theory-free data mining and related analytical issues, e.g., p-hacking, which can undermine the replicability of comparative work. Although these behaviors are exhibited at the analysis stage, we believe that they are facilitated by the presence of many measured variables without an immediate analytical “home.” It is easier to go on a “fishing expedition” if you have many fishing grounds. At the same time, large datasets can be explored to generate new hypotheses, but in this case, scholars need to be clear that such hypotheses are being generated and not tested.
There is similarly a tendency to develop complex database structures, e.g., with a relational database, that reflect the interactions of actors, resources, and governance systems that collectively comprise SESs. The SESMAD database, for instance, is highly flexible in its ability to capture interactions among system components, but this creates challenges for comparing these heterogeneously characterized cases. Such data structures can pose challenges for aggregating measures across multiple components, i.e., 2 or more actor groups, requiring analysts to make nontrivial decisions about whether to use sums, means, medians, or other measures. For instance, empirical studies of forest outcomes using the IFRI database have tended to aggregate multiple user groups by taking the sum of features such as group size, the mean of others such as fuelwood dependence, and the maximum value for institutional arrangements such as local monitoring.
This example reflects a broader challenge that commons research has faced for some time, this being the consistent and valid measurement of (often biophysical) outcomes (Young et al. 2006). Investigators often have to decide how to pool nonstandardized indicators from different methodologies, or change their existing methods to a standardized and more comparable approach. For example, biodiversity is often a desired outcome for SES systems, which can include measurements of species richness, ecosystem integrity and functioning, or other ecological proxies for productive and sustainable ecosystems. Ecological monitoring can identify key indicators for comparative studies, however measuring biophysical variables using standardized and comparable methods can remain challenging and requires insight and forward-thinking strategies within an SES research team.
An underlying driver of some of these challenges is the lack of a core set of well-defined theories to drive data collection and analysis. Although there has been some work done to encourage midrange theorization (Young 2002, Cox et al. 2016), projects tend to be designed to test a multiplicity of theories. The projects included here share a broad framing (social dilemmas in environmental management and governance), but this is inclusive of a great many theoretically important factors. NIIS is a mild exception here, in that it was designed to test the effects of decentralization in Nepal and compare top-down vs. more bottom-up irrigation systems, and yet still includes over 500 variables. Likewise, the MACMON project was designed to understand the social and ecological outcomes of comanagement and the institutional and contextual characteristics associated with different sets of outcomes.
We have suggested that scholars ask themselves whether engaging in broadly comparative work is worth it, given the challenges that we have just described. These are significant challenges, but we believe that they must be met in order to advance a critical line of social-ecological research. Simply put, we believe there is too much reinventing of the wheel in terms of frameworks and empirical work. There is deep value in having a diversity of methods, but much of this comes from the ability to learn from and consolidate the practice of successful approaches. We do not mean to diminish the importance of more case-oriented research in this endeavor. In the end, good research is good research, whatever its framing and perspective, and our goal here is to support good comparative research.
With all of this in mind, we have developed a checklist of questions for future groups to consider as they develop their own broadly comparative data projects (Table 2). “Checking” an item here does not mean that a particular task is finished, but rather that a question has been asked and various answers considered. It can be helpful for any group launching a collaboratively comparative project to brainstorm their responses to these questions as they proceed. Some of the social content in this checklist represents the application of collective-action theory with which we are expertly familiar, such as Ostrom’s (1990) design principles, and the importance of boundary objects and actors in facilitating cooperation within diverse groups (Gurney et al. 2019). We believe we ought to view research groups as communities, and that members, and particularly leaders of such groups, should maintain a “reflexive” frame of mind, by turning their analytical lenses inward to understand the factors that affect their own behavior and the behavior of other group members.
Beyond these considerations for individual projects and groups, it would help if the broader research community valued less traditional outputs more. We need to move away from a productivist culture in social-ecological science (Paasche and Österblom 2019), and toward one that creates space and time to build understandings across intellectual and professional divides. We also need to be cautious that the drive for large-n research does not ignore relevant theory and squeeze out the detailed small-n case studies (e.g., Morrison 2017) that are a critical means for generating new hypotheses (Yin 2017, Cairney et al. 2019). We also need to consider how large datasets and single case studies can more usefully intersect.
Ultimately, we need to do a better job at sharing the public goods that over time make everything else possible. Making research protocols publicly available in a single platform could help. Open data platforms are also a promising option. Other scientific fields have already demonstrated that this is feasible and offers enormous benefits. For renewable energies, there are various open data platforms, e.g., https://www.renewables.ninja/ or https://open-power-system-data.org/. Such a platform could also contribute to setting standards in data quality, variables measured, and so greatly enhance comparability. Finally, a more wiki-type database with broad community development could help support the needed social capital for future collaborative projects. These platforms could also be promoted via linkages with open access journals like Ecology and Society and the International Journal of the Commons. As a decentralized research community, we should leverage our diversity by experimenting with these approaches, and over time, select and maintain those that support methodological and theoretical synthesis.
Data/code sharing is not applicable to this article because no new data/code were created or analyzed in this study.
Agrawal, A. 2003. Sustainable governance of common-pool resources: context, methods, and politics. Annual Review of Anthropology 32:243-262. https://doi.org/10.1146/annurev.anthro.32.061002.093112
Anderies, J. M., M. A. Janssen, and E. Ostrom. 2004. A framework to analyze the robustness of social-ecological systems from an institutional perspective. Ecology and Society 9(1):18. https://doi.org/10.5751/ES-00610-090118
Baron, J. S., A. Specht, E. Garnier, P. Bishop, C. A. Campbell, F. W. Davis, B. Fady, D. Field, L. J. Gross, S. M. Guru, B. S. Halpern, S. E. Hampton, P. R. Leavitt, T. R. Meagher, J. Ometto, J. N. Parker, R. Price, C. H. Rawson, A. Rodrigo, L. A. Sheble, and M. Winter. 2017. Synthesis centers as critical research infrastructure. Bioscience 67(8):750–759. https://doi.org/10.1093/biosci/bix053
Cairney, P., T. Heikkila, and M. Wood. 2019. Making policy in a complex world. Cambridge University Press, Cambridge, UK. https://doi.org/10.1017/9781108679053
Casari, M., and C. Tagliapietra. 2018. Group size in social-ecological systems. Proceedings of the National Academy of Sciences of the United States of America 115(11):2728–2733. https://doi.org/10.1073/pnas.1713496115
Cash, D. W., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, D. H. Guston, J. Jäger, and R. B. Mitchell. 2003. Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences of the United States of America 100(14):8086-8091. https://doi.org/10.1073/pnas.1231332100
Cox, M. 2008. Balancing accuracy and meaning in common-pool resource theory. Ecology and Society 13(2):44. https://doi.org/10.5751/ES-02683-130244
Cox, M., G. Arnold, and S. Villamayor-Tomas. 2010. A review of design principles for community-based natural resource management. Ecology and Society 15(4):38. https://doi.org/10.5751/es-03704-150438
Cox, M., S. Villamayor-Tomas, G. Epstein, L. Evans, N. C. Ban, F. Fleischman, M. Nenadovic, and G. Garcia-Lopez. 2016. Synthesizing theories of natural resource management and governance. Global Environmental Change 39:45–56. https://doi.org/10.1016/j.gloenvcha.2016.04.011
Cronin, M. A., and L. R. Weingart. 2019. Conflict across representational gaps: threats to and opportunities for improved communication. Proceedings of the National Academy of Sciences of the United States of America 116(16):7642–7649. https://doi.org/10.1073/pnas.1805866116
Gawande, A. 2014. The checklist manifesto: how to get things right. Penguin, New York, New York, USA.
Gurney, G. G., and E. S. Darling. 2017. A global social-ecological systems framework for coral reef fisheries: a practical monitoring handbook. Wildlife Conservation Society, New York, New York, USA.
Gurney, G. G., E. S. Darling, S. D. Jupiter, S. Mangubhai, T. R. McClanahan, P. Lestari, S. Pardede, S. J. Campbell, M. Fox, W. Naisilisili, N. A. Muthiga, S. D'agata, K. E. Holmes, and N. A. Rossi. 2019. Implementing a social-ecological systems framework for conservation monitoring: lessons from a multi-country coral reef program. Biological Conservation 240:108298. https://doi.org/10.1016/j.biocon.2019.108298
Hampton, S. E., and J. N. Parker. 2011. Collaboration and productivity in scientific synthesis. Bioscience 61(11):900–910. https://doi.org/10.1525/bio.2011.61.11.9
Liu, J., T. Dietz, S. R. Carpenter, M. Alberti, C. Folke, E. Moran, A. N. Pell, P. Deadman, T. Kratz, J. Lubchenco, E. Ostrom, Z. Ouyang, W. Provencher, C. L. Redman, S. H. Schneider, and W. W. Taylor. 2007. Complexity of coupled human and natural systems. Science 317(5844):1513-1516. https://doi.org/10.1126/science.1144004
Mauser, W., G. Klepper, M. Rice, B. S. Schmalzbauer, H. Hackmann, R. Leemans, and H. Moore. 2013. Transdisciplinary global change research: the co-creation of knowledge for sustainability. Open Issue 5(3-4):420-431. https://doi.org/10.1016/j.cosust.2013.07.001
Morrison, T. H. 2017. Evolving polycentric governance of the Great Barrier Reef. Proceedings of the National Academy of Sciences of the United States of America 114(15):E3013-E3021. https://doi.org/10.1073/pnas.1620830114
Ostrom, E. 1990. Governing the commons: the evolution of institutions for collective action. Cambridge University Press, Cambridge, UK.
Ostrom, E. 2005. Understanding institutional diversity. Princeton University Press, Princeton, New Jersey, USA.
Ostrom, E. 2007. A diagnostic approach for going beyond panaceas. Proceedings of the National Academy of Sciences of the United States of America 104(39):15181-15187. https://doi.org/10.1073/pnas.0702288104
Paasche, Ø., and H. Österblom. 2019. Unsustainable science. One Earth 1(1):39–42. https://doi.org/10.1016/j.oneear.2019.08.011
Palmer, M. A., J. G. Kramer, J. Boyd, and D. Hawthorne. 2016. Practices for facilitating interdisciplinary synthetic research: the National Socio-Environmental Synthesis Center (SESYNC). Current Opinion in Environmental Sustainability 19:111–122. https://doi.org/10.1016/j.cosust.2016.01.002
Parker, J. N., N. Vermeulen, and B. Penders. 2010. Collaboration in the new life sciences. Routledge, London, UK.
Partelow, S. 2018. A review of the social-ecological systems framework: applications, methods, modifications, and challenges. Ecology and Society 23(4):36. https://doi.org/10.5751/ES-10594-230436
Poteete, A. R., M. A. Janssen, and E. Ostrom. 2010. Working together: collective action, the commons, and multiple methods in practice. Princeton University Press, Princeton, New Jersey, USA.
Poteete, A. R., and E. Ostrom. 2008. Fifteen years of empirical research on collective action in natural resource management: struggling to build large-N databases based on qualitative research. World Development 36(1):176–195. https://doi.org/10.1016/j.worlddev.2007.02.012
Schlager, E., and M. Cox. 2018. The IAD framework and the SES framework: an introduction and assessment of the Ostrom Workshop frameworks. Chapter 6 in C. Weible and P. Sabatier, editors. Theories of the policy process. Fourth edition. Westview Press, Boulder, Colorado, USA. https://doi.org/10.4324/9780429494284-7
Star, S. L., and J. R. Griesemer. 1989. Institutional ecology, ‘translations’ and boundary objects: amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science 19(3):387-420. https://doi.org/10.1177/030631289019003001
Thiel, A., M. E. Adamseged, and C. Baake. 2015. Evaluating an instrument for institutional crafting: how Ostrom’s social-ecological systems framework is applied. Environmental Science & Policy 53(Part B):152-164. https://doi.org/10.1016/j.envsci.2015.04.020
Yin, R. 2017. Case study research and applications. SAGE, Los Angeles, California, USA.
Young, O. R. 2002. The institutional dimensions of environmental change: fit, interplay, and scale. MIT Press, Cambridge, Massachusetts, USA. https://doi.org/10.7551/mitpress/3807.001.0001
Young, O. R., E. F. Lambin, F. Alcock, H. Haberl, S. I. Karlsson, W. J. McConnell, T. Myint, C. Pahl-Wostl, C. Polsky, P. S. Ramakrishnan, H. Shroeder, M. Scouvart, and P. H. Verburg. 2006. A portfolio approach to analyzing complex human-environment interactions: institutions and land change. Ecology and Society 11(2):31. https://doi.org/10.5751/es-01799-110231