INTRODUCTION

Almost two decades ago the edited book Panarchy: Understanding Transformations in Human and Natural Systems (Gunderson and Holling 2002) presented a synthetic perspective on how a group of social-ecological researchers associated with the Resilience Alliance understood change in social-ecological systems. The concept of panarchy was a key focus of this influential book. In the late 1990s and early 2000s the Resilience Alliance was a productive, innovative, and highly collaborative group of interdisciplinary scientists who focused on addressing social-ecological problems. They did this by combining insights from the social and natural sciences, arts, and humanities, as well as by bridging theory and practice (Parker and Hackett 2012). Panarchy remains a boundary object that has inspired research topics, enabled collaborations, and nurtured new scientific frameworks (Parker and Hackett 2012). The ideas put forward have been applied in field studies, archaeology, mathematical models, participatory work, and scenario development (Gunderson et al. 2022). Panarchy has inspired resilience assessments and guided decision making. In this article we pay tribute to the book by studying how the concepts and metaphors proposed have been further developed in the academic literature. We also document criticism of the concepts and identify key research frontiers.

The panarchy concept builds on Holling’s adaptive cycle (Holling 1986) by extending the idea across spatial and temporal scales. Panarchy proposes that it is useful to conceptualize systems in terms of interacting adaptive cycles. The adaptive cycle was an idea first proposed by Holling based upon his experience working and studying managed ecosystems (Holling 1986). It was meant to be a conceptual tool that focused attention on processes of destruction and reorganization that had been neglected in comparison with those of growth and conservation.

An adaptive cycle alternates between long periods of system aggregation, connection, and accumulation, and shorter periods of disruption and reorganization. The adaptive cycle exhibits four phases arranged into two major loops (Fig. 1). The phases are exploitation (r), conservation (K), release (Ω), and reorganization (α), the latter characterized by events of creative destruction (Fig. 1). The first loop, often referred to as the front-loop, from r to K, is the slow, incremental phase of growth and accumulation. The second loop, referred to as the back-loop, from Ω to α, is the rapid phase of reorganization leading to renewal. A system going into the back-loop can either remain in a similar form, or may transition to a new type of system with new boundaries and key components (Holling et al. 2002). Resilience researchers proposed that tensions between demands for organization and efficiency, versus demands for novelty and diversity, drive adaptive cycle dynamics in many different types of complex systems (Holling et al. 2002).

“Panarchy is not a theory of what it is, but a metaphor of what might be” (Gunderson and Holling 2002:32). It is not a predictive tool, but aims to understand adaptive change. The adaptive cycle is proposed to exist within a three-dimensional space defined by three properties: potential, connectedness, and resilience (Gunderson and Holling 2002). Potential refers to the capital available to the system, e.g., nutrients and carbon captured by a forest, or to human capital, i.e., skills and knowledge accumulated to run the economy. Connectedness is a proxy of the structure of the system; it is the network of interactions and strength between its elements. Resilience is the capacity of any system to absorb disturbance and reorganize while undergoing change so as to retain essentially the same identity, as determined by its function, structure, and feedbacks (Folke 2016).

Panarchy posits that systems are organized in nested hierarchies across space and time, where each level of the hierarchy is a subsystem that can be in a different phase of the adaptive cycle. These phases are inspired by mathematical models used in economics and ecology, but broadly describe patterns of growth, collapse, and recovery that are common to populations, ecological communities, markets, or political organizations. Whereas each subsystem in the hierarchy can be at a different phase of the adaptive cycle, these subsystems can particularly influence each other through cross-scale interactions, either called revolt, where release in a smaller, faster sub-system can trigger release in a larger, slower sub-system, or called remember, in which the structure of a larger-slower sub-system shapes the dynamics of reorganization of a smaller, faster sub-system (Gunderson and Holling 2002).

The three-dimensional space defined by potential, connectedness, and resilience has corners with attractors that can derail the adaptive cycle: poverty and rigidity traps (Gunderson and Holling 2002). Poverty traps are described in Panarchy as maladaptive states where potential, connectedness, and resilience are low. Poverty traps are a series of feedback mechanisms that reinforce impoverished states (Allison and Hobbs 2004, Bowles et al. 2006, Maru et al. 2012), limiting the system’s capacity to innovate and increase potential. The opposite corner, where potential, connectivity, and resilience are high, is another maladaptive space called a rigidity trap. In that corner there is little space for experimentation and innovation. Examples include systems where ecological resilience has been extensively replaced by artificial processes to maintain the system such as levees, flood barriers, or chemical control of pests (Holling and Meffe 1996).

Panarchy offers a rich conceptual framework for interpreting environmental problems. Although inspired by several mathematical constructs (e.g., cycles, traps, scaling laws), it is general enough to invite scholars from multiple disciplinary backgrounds, ontologies, and epistemologies to collaborate around research questions and applied problems. As such, it is useful as a boundary object and can be used empirically or metaphorically. The book presents a series of case studies where geographers, economists, political scientists, and ecologists have demonstrated the utility of the framework to their area of research.

Scientific theories are bodies of knowledge that answer the question, Why? The answer implies causal reasoning. Regardless of discipline, a theory should therefore provide a series of testable expectations that help distinguish situations when it is a good explanation of reality, versus situations when it falls short. The practice of science implies avoiding confirmation bias and testing when explanations are valid representations of reality. For this paper, we explored how the conceptual framework of panarchy has been applied, developed, and tested over the past twenty years. For panarchy to become a useful theory in sustainability science, it needs to be able to explain phenomena in social-ecological systems across different contexts, in a way that is testable, and withstands the rigour of those tests. Here, we have reviewed the academic literature of the last two decades to trace how the ideas proposed in Panarchy have evolved. We conclude by suggesting research frontiers for future work.

METHODS

To answer these questions we combined an automated literature review based on topic modeling (Griffiths and Steyvers 2004, Blei 2012) with human-coded document analysis (Bryman 2008). We first describe the data curation and how it was used in the topic models. Then we introduce the qualitative approach.

Data

We used the Web of Science, Scopus, and GoogleScholar to survey academic literature that has used or referenced works that trace back to panarchy (Gunderson and Holling 2002). We extracted complete records from the Scopus database that matched the search for “panarchy” or “adaptive cycle” (N = 595), or the search “panarchy” or “adaptive cycle” and “resilience” (N = 278). The data was combined with all papers (N = 1923) that cited the inaugural paper that introduced the book to the academic community (Holling 2001). Records with missing abstracts were dropped (N = 191), and records with missing years were set to 2020 given that they are accepted manuscripts with digital object identifiers (DOIs) scheduled to be published later in 2020 or 2021. Our final sample is 2177 documents.

To prepare the data for topic modeling, we constructed a document term matrix with documents in rows (N = 2177) and words in columns (N = 12,744). Here, our unit of analysis for the document is the abstracts retrieved, and the matrix contains the count of the number of occurrences of each word per abstract. We removed stop words (e.g., “the,” “a”) and digits from the matrix, as well as a list of words that were over-represented in our data and are common in the scientific literature but are unrelated to the papers’ topics (e.g., “paper,” “study,” “aim”). The lexicon of stop words has 1149 entries, curated by various research groups, and accessible in the R language under the packages tm and tidytext (Feinerer et al. 2008, Feinerer and Hornik 2020, Hornik 2020, Grün and Hornik 2021, Robinson and Silge 2021), e.g., by tidytext::stop_words. The data were also lemmatized using the hash lemmas dataset available at https://www.lexiconista.com. Lemmatization is the process of simplifying words to their root lemmas, e.g., “teaching” and “taught” are lemmatized to “teach.”

Topic models

Topic models are an unsupervised statistical technique to reduce the dimensionality of a corpus of data, typically but not necessarily text, into topics (Blei 2012). Here, a topic is a latent variable that ranks words with high probability of appearing together within the same document. Documents in turn can be described by the probability distribution of a particular set of topics. Because they are (posterior) probabilities, the sum of the probability of all words for any given topic should be one, and the sum of the probability of all topics for any given document should also be one. An iterative process or algorithm is what allows the model to learn the ranking of words that best explain topics, and the ranking of topics that best explain documents.

The underlying statistical technique for this machine-learning approach is called Latent Dirichlet Allocation (LDA; Blei et al. 2003). It allocates probabilities to latent variables, i.e., topics, based on the distribution of words in text data, assuming a multivariate continuous, i.e., Dirichlet, distribution. We compared three LDA algorithms: correlated topic models (CTM), variational expectation maximization (VEM), and Gibbs sampling (Gibbs), by assessing their performance against their log-likelihood estimation, entropy, and perplexity (Grün and Hornik 2011). Entropy is a measure of order or disorder of a system. In the context of topic models it measures how evenly the topic distribution is spread. Perplexity measures the uncertainty of predicting a single word, so if the model performance were the same as random, perplexity would approximate the vocabulary size (N = 12744 words). These performance metrics enable us to choose which algorithm best fits our data, what is the optimal number of topics to fit, and how to avoid overfitting.

Document analysis

We complemented our unsupervised approach with document analysis (Bryman 2008) by coding an additional set of categorical variables for a sample of papers (N = 42). The papers read were part of the corpus used for topic modeling. We read papers, avoiding biasing our selection to highly cited papers, specific journals, older papers, or particular disciplines or methods.

Based on the reading of Panarchy, we first developed a series of categorical variables that captured the main components of the scientific framework. The variables were presented and received feedback at the science meeting of the Resilience Alliance in 2019. We used the variables to annotate qualitative aspects such as use of the adaptive cycle, identification of its phases, and whether the paper is conceptual, modeling, or an empirical study. When empirical, we recorded the temporal and spatial scales of the case study. We also distinguished empirical cases from empirical constructs. An empirical case is when a study is trying to test an idea or proposition put forward by panarchy, e.g., a cross-scale interaction, or that the back-loop is faster than the front-loop. An empirical construct is when panarchy is used as inspiration to interpret a case study, typically with a historical component to it, e.g., identify the phases of adaptive cycle. But it does not find support for or against a panarchy preposition. That is, there was no mechanism in place to avoid confirmation bias.

We also identified what aspects of panarchy were most used in the papers, e.g., whether there was an emphasis on cross-scale interactions, or poverty and rigidity traps. We used text annotations to capture potential criticisms as well as the methods used. We did a preliminary scoping and coding of papers, and through an iterative process we kept track of potential fields under review by our corpus, or simply absent from it. We coded papers until we reached conceptual saturation, i.e., additional papers were not proving additional categories. The code book is available at https://doi.org/10.6084/m9.figshare.13490919.

RESULTS

Panarchy, the book published in 2002, has been cited over 8600 times according to Google Scholar in March 2022. The scientific paper that introduced the book to the scientific community (Holling 2001) has received, at the time of writing, 1853 citations in the Web of Science, and 2197 in Scopus. Roughly half of the citations have come from environmental (28%) and social sciences (22%). Computer science (2.1%) and arts and humanities (2.6%) have been the least represented.

Topic modeling was done using Gibbs sampling. In our comparison of alternative LDA algorithms, Gibbs sampling provided the best fit for the data (Fig. 2). As a rule of thumb, an ideal method should maximize entropy and the log-likelihood estimation while minimizing perplexity (Grün and Hornik 2011). Gibbs sampling maximized both entropy and likelihood with our data when compared to other alternatives. The second best was the variational-expectation maximization (VEM) algorithm when α was not set constant. α is a hyper parameter that weights the evenness of topic distribution. A lower α than default values indicates that the documents can be described by rather fewer topics, or that its distribution is very uneven. In fact, we observed that increasing the number of topics from five to 100 topics did increase entropy at the expense of reducing α, meaning that despite the larger number of topics, the main content of a document is still captured by a few of them. The log-likelihood maximization stopped at around 25 to 50 topics. Thus, we restricted the rest of our analysis to 25 topics.

A topic is a set of words that are ranked according to a probability that they represent an underlying content of the document (Fig. 3). For example, the words “resilience,” “adaptive,” or “capacity” have a high probability to capture the content of topic 14. Papers early in our time series (2001–2003) have high content largely dominated by topics 14 on resilience, 9 on resource management, and 23 on cross-scale interactions. Toward the end of the time series, topic 7 on urban systems and 25 on local communities and knowledge have spikes up to 9% of the content of each year (2019–2020). For comparison, if all topics were equally represented in the content, they would have 4% in the corpus (grey line in B, Fig. 3). The topics presented lack names for reproducibility purposes. Whereas here we present, for example, topic 7 as the urban topic, if a researcher replicates the analysis, that person will find the same list of words and probability distributions listed under a topic with a different number. That is because the labeling of topics is random and we urge readers not to over-interpret the word patterns found.

Despite fluctuations, most topics showed a relatively constant level of interest over time (Figs. 3, 4). We did not observe strong trends, but some topics have gained a small amount of attention. For example, topic 11 on sustainability and topic 15 on ecosystem services appeared consistently across time. In contrast, research on innovation (topic 18), urban infrastructure (topic 7), and local communities (topic 13) have gained attention in recent years. Topic 6 was an outlier, with a selection of papers ranking high on content related to network infrastructure and performance, possibly from engineering disciplines. It was the only topic with a set of papers that was clearly distinct from the rest of the collection, and showed a decreasing trend over time.

The human-coded document analysis revealed that the most common feature of panarchy in the literature was the adaptive cycle (81%, 34 out of N = 42) followed by cross-scale interactions (Fig. 5). Poverty and rigidity traps were less studied features in our sample (14% and 19%, respectively), even when accounting for slightly different terminology such as “lock-in.” About half (52.4% or 22/42) of the papers analyzed were conceptual papers, many without defined methods or a clear research question. Roughly half (20/42) of the papers used panarchy as a metaphor. About 40% of papers in our sample (17/42) went a step further and used panarchy as an empirical construct, for example by attempting to identify the phases of the adaptive cycle (26/42). Of the 22 empirical cases, six were at the time scale of centuries, seven of decades, six of years, and one in weeks. Spatially, four were at a city scale, 17 regional, and one national. Empirical papers predominantly used qualitative methods (17) and were primarily retrospective historical reviews.

Most of the papers reviewed (81%, 34 of N = 42) identified adaptive cycles either through scientific methods, or as analogy or framework to make sense of a narrative. Identification of adaptive cycles has found applications in a wide range of disciplines and research problems, from delimitation of periods in archaeology and anthropology (Redman and Kinzig 2003), systemic interactions between forest-related sectors in Sweden (Moen and Keskitalo 2010), financial crises in Europe (Castell and Schrenk 2020), to traffic jams in China (Zeng et al. 2020). The adaptive cycle has been used as a model to structure rich empirical information about participation in adaptive management and, based on that analysis, to develop recommendations for more flexible stakeholder engagement (Stringer et al. 2006). The adaptive cycle has also been used to provide a more mechanistic description of the history of a variety of specific social-ecological systems, including the Galapagos Islands (González et al. 2008), the Western Australian agricultural region (Allison and Hobbs 2004), the Erhai Lake watershed in southwest China (Dearing 2008), and the cities of Cambridge and Swansea in the UK (Simmie and Martin 2010). The adaptive cycle has also been used to describe more general social and ecological processes, such as lake eutrophication (Carpenter et al. 2001) and the development of regional industrial clusters (Martin and Sunley 2011).

Recent empirical tests of the adaptive cycle used innovative types of data and methods. For example, information-transfer methods based on entropy have been suggested to approximate relevant components of a system and the empirical proxies of the axes within which the adaptive cycle unfolds: potential, connectedness, and resilience (Castell and Schrenk 2020). Percolation methods combined with big data have been proven useful for testing hypotheses of regime shifts in urban systems, an idea originally proposed in Panarchy (Gunderson and Holling 2002) and deriving the temporal and spatial scales at which the adaptive cycle emerges (Zeng et al. 2020).

DISCUSSION

We examined how the conceptual framework of panarchy has been developed and tested over the past twenty years. We did this using a topic modeling to analyze a large number of papers and a focused review of a selection of panarchy papers. We found that panarchy has been used as an interpretative tool across a wide variety of topics, but there has been little effort to substantially operationalize or test panarchy.

Sustainability science is defined by the problems it addresses rather than the disciplines it employs (Folke 2016). The goal of this paper was to explore how the ideas proposed in panarchy have evolved, to what type of problems they have been applied and found useful, and, finally, what key research frontiers remain. Our topic-modeling results helped in identifying some of the problems that the community is addressing and how they have changed over time. Some of these problems include climate change (topic 24), urban development and disaster response (7), conservation and landscape management trade-offs (5), natural resource management (9), food production (3), policy tools and plans (16), community based management (13), or knowledge and local social practices (25). We also observed topics closely related to methods or approaches to study social-ecological systems such as modeling (19, 21), networks (6), or ecosystem services (15). Topic modeling can be used to further investigate particular academic communities, identify key landmark papers in certain topics, or identify the network of key collaborators and experts.

This paper aimed to study the use of panarchy related concepts and their evolution since the publication of the book (Gunderson and Holling 2002). Systematic literature reviews are often restricted to a limited subset of the literature of dozens to maybe hundreds of papers, based on what is readable on the time frame of a project. Qualitative analysis offers rich insights into the papers reviewed but is limited by sample size in its ability to draw general conclusions. In our analysis, the qualitative insights could be biased toward a non-random selection of papers that aligned with our research interests, or highly cited review papers. Topic modeling enabled us to complement the analysis and include all papers indexed by major scientific databases. It has the advantage of reproducibility and reduces sample bias, but offers limited insights into the dimensions of panarchy used, methods, or criticisms. By combining methods, as we did in this paper, we believe literature reviews can become more powerful.

Our choice of methods does, however, have some limitations. We could not directly compare the two methods because they do not share a common categorical axis that we could use to translate between methods. Topic modeling contributes understanding of what topics, i.e., sets of words, panarchy has been associated with; our document analysis contributes in-depth understanding on how people have interpreted and use these concepts. The topics gave us a broad but shallow overview, while the document analysis provides deep insights on particular work but limited generalizations. Our method also ignored rich areas of social science research (e.g., anthropology), often published in books and book chapters, which are not archived by scientific search engines. Finally, an iterative inductive approach to the qualitative coding could have given us different insights on the literature outside the pre-defined categories we initially shared with the Resilience Alliance.

Future studies of panarchy, or other concepts, can benefit from our experience. Panarchy is a collection of concepts, some of them more used and developed than others, like the adaptive cycle. Future reviews would benefit from including grey literature such as theses, books, non-governmental organization reports, government agencies reports, or non-English literature. This would let us see the usage of panarchy concepts in more applied settings such as management and decision making. The use of full text instead of abstracts can also offer additional insights on the automated analyses, e.g., in attributing impacts of ecosystem services from regime shifts in social-ecological systems (Rocha and Wikström 2015). Although databases are improving, full-text analyses are only possible for open-access journals, and text analysis is difficult across multiple languages. Databases are increasing their coverage of books and reports, which suggest our approach may be able to access a broader set of literature in the future.

Our review of panarchy papers (N = 42) revealed a high level of conceptualization but lack of theory development or testing. Conceptual frameworks, such as panarchy, have an important role as boundary objects that enable interdisciplinary dialogues. The usefulness of panarchy in this role is revealed by how it has been used to address a variety of issues in fields ranging from anthropology to engineering. Many of the papers we reviewed used the adaptive cycle to interpret past events, but did not engage in measurement or observation to test adaptive cycles in the present or potential future trajectories, nor other features of panarchy such as poverty or rigidity traps, or revolt and remember cross-scale interactions. There remains a lack of testable mechanistic theories to explain the dynamics of panarchy and, consequently, there are a lack of methods to measure or observe it in real social-ecological systems.

The high level of conceptualization but lack of theory testing came across as one of the major limitations, given our sample of papers reviewed (N = 42). What makes a theory a scientific theory is that the plurality of scientific approaches (e.g., positivism, constructivism) should have mechanisms in place to avoid confirmation bias, allowing the emergence of alternative explanations, and identify the circumstances under which a theory may fail (Hansson 2017). Many of the papers we read confirmed but did not challenge the expectations that panarchy put forward, though a few of the most recent works did falsify some of the ideas proposed, e.g., the back-loop is faster than the front-loop (Castell and Schrenk 2020). We believe more work is needed to better understand the circumstances when panarchy works, and when it falls short: in other words, to refine the scientific framework into a scientific theory.

The criticism of over-conceptualization is not unique to the literature that engages with panarchy ideas. By over-conceptualization we mean the problem of scientific jargon, where different communities use slightly different wordings to refer to the same phenomenon, reducing the chances that lessons from one discipline percolate to another. An example from panarchy is the endorsement of ideas such as creative destruction, from economics, or the theory of intermediate disturbances, from ecology. The latter has been debunked in the ecological literature (Fox 2013). A recent review of sustainability science mapped the different schools of thought that the discipline has developed over the last decades (Clark and Harley 2020) and reached a similar conclusion: too many conceptual frameworks have been developed, but there have been too few empirical attempts to test the frameworks against data, and falsify hypotheses. The review also emphasizes the problem of measurement and observation (Clark and Harley 2020). In the context of panarchy, recent work has developed an information theory–based approach that enables the identification of adaptive cycles (Castell and Schrenk 2020). The authors identify phases of the adaptive cycle in the European financial crises and in grassland ecosystems, but fail to find support for the difference in speed between the forward and backward loops originally proposed in panarchy. Lack of support for the same ideas was also found in studies of urban traffic (Zeng et al. 2020).

The panarchy dimensions that received less attention in our qualitative analysis included creative destruction, rigidity, and poverty traps. This may be at least partly because of the scope of our data: papers that have cited panarchy or Holling’s paper (Holling 2001, Gunderson and Holling 2002), and a relatively small sample size (N = 42). Concepts such as creative destruction and poverty traps precede panarchy and therefore have been theorized and empirically grounded outside the panarchy stream of thinking. For example, the theory of poverty traps dates back to the 1950s in economics, and has received both theoretical development (Bowles et al. 2006) as well as empirical grounding (Banerjee and Duflo 2012, Banerjee et al. 2015) that has enabled researchers and governments to distinguish what kind of interventions are likely to reduce poverty. Considering what researchers have done with poverty traps, we believe it would be fruitful to propose and evaluate potential mechanisms that could produce adaptive cycles, their cross-scale interactions (i.e., remember and revolt), or how the hierarchical, nested nature of complex systems enhances or erodes resilience in social-ecological systems.

CONCLUSION

This paper explores how the conceptual framework of panarchy has been used, developed, and tested over the past 20 years. Despite a growing body of literature, no topic appears to dominate papers using panarchy concepts. The adaptive cycle is the most widely used concept, and it has been most used to interpret histories that range from the dynamics of natural resource management and urban development, to archeological and anthropological studies that span millennia. Cross-scale dynamics, structure, and traps have received less attention. Operationalzing panarchy theory remains a challenge. How to measure and forecast system behaviour in ways that are useful for panarchy concepts remains an open question. Addressing these issues usefully would include developing tools to answer critical questions about the dynamics of resilience across linked social-ecological systems. For example, how do current changes in the global food system alter its resilience, connectedness, and potential? How do we identify whether a social-ecological system is entering a different phase of the adaptive cycle? What type of remember or revolt dynamics shape the current food systems of Stockholm or Bogotà? Although our survey of the literature identified a few promising pathways, developing ways to operationalize theories that explain the dynamics of resilience is vitally needed to confront the challenges of creating a sustainable, just world.

LITERATURE CITED

Allison, H. E., and R. Hobbs. 2004. Resilience, adaptive capacity, and the “lock-in trap” of the Western Australian agricultural region. Ecology and Society 9(1):3. https://doi.org/10.5751/ES-00641-090103

Banerjee, A. V., and E. Duflo. 2012. Poor economics: a radical rethinking of the way to fight global poverty. Public Affairs, New York, New York, USA.

Banerjee, A., E. Duflo, N. Goldberg, D. Karlan, R. Osei, W. Parienté, J. Shapiro, B. Thuysbaert, and C. Udry. 2015. A multifaceted program causes lasting progress for the very poor: evidence from six countries. Science 348(6236):1260799-1260799. https://doi.org/10.1126/science.1260799

Blei, D. M. 2012. Probabilistic topic models. Communications of the Association for Computing Machinery 55(4):77-84. https://doi.org/10.1145/2107736.2107741

Blei, D. M., A. Y. Ng, and M. I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3:993-1022.

Bowles, S., S. N. Durlauf, and K. Hoff. 2006. Poverty Traps. Princeton University Press, Princeton, New Jersey, USA. https://doi.org/10.1515/9781400841295

Bryman, A. 2008. Social research methods. Third edition. Oxford University Press, Oxford, UK.

Carpenter, S., B. Walker, J. M. Anderies, and N. Abel. 2001. From metaphor to measurement: resilience of what to what? Ecosystems 4:765-781. https://doi.org/10.1007/s10021-001-0045-9

Castell, W. Z., and H. Schrenk. 2020. Computing the adaptive cycle. Scientific Reports 10:18175. https://doi.org/10.1038/s41598-020-74888-y

Clark, W. C., and A. G. Harley. 2020. Sustainability science: toward a synthesis. Annual Review of Environment and Resources 45(1):331-386. https://doi.org/10.1146/annurev-environ-012420-043621

Dearing, J. A. 2008. Landscape change and resilience theory: a palaeoenvironmental assessment from Yunnan, SW China. Holocene 18(1):117-127. https://doi.org/10.1177/0959683607085601

Feinerer, I., and K. Hornik. 2020. tm: Text Mining Package, R Package Version 0.7-8. http://tm.r-forge.r-project.org

Feinerer, I., K. Hornik, and D. Meyer. 2008. Text mining infrastructure in R. Journal of Statistical Software 25(5):1-54. https://doi.org/10.18637/jss.v025.i05

Folke, C. 2016. Resilience (republished). Ecology and Society 21(4):44. https://doi.org/10.5751/ES-09088-210444

Fox, J. W. 2013. The intermediate disturbance hypothesis should be abandoned. Trends in Ecology & Evolution 28(2):86-92. https://doi.org/10.1016/j.tree.2012.08.014

González, J. A., C. Montes, J. Rodrı́guez, and W. Tapia. 2008. Rethinking the galapagos islands as a complex social-ecological system: implications for conservation and management. Ecology and Society 13(2):13. https://doi.org/10.5751/ES-02557-130213

Griffiths, T. L., and M. Steyvers. 2004. Finding scientific topics. Proceedings of the National Academy of Sciences 101:5228-5235. https://doi.org/10.1073/pnas.0307752101

Grün, B., and K. Hornik. 2011. topicmodels: an R Package for fitting topic models. Journal of Statistical Software 40(13):1-30. https://doi.org/10.18637/jss.v040.i13

Grün, B., and K. Hornik. 2021. Topicmodels: Topic models, R Package Version 0.2-12. https://cran.r-project.org/web/packages/topicmodels/index.html

Gunderson, L. H., C. R. Allen, and A. Garmestani. 2022. Applied panarchy: applications and diffusion across disciplines. Island Press.

Gunderson, L. H., and C. S. Holling. 2002. Panarchy. Island, Washington, D.C., USA.

Hansson, S. O. 2017. Science and pseudo-science. In E. Zalta, editor. The Stanford encyclopedia of philosophy. Metaphysics Research Lab, Stanford University, Stanford, California, USA. https://plato.stanford.edu/archives/sum2017/entries/pseudo-science/

Holling, C. S. 1986. The resilience of terrestrial ecosystems: local surprise and global change. Pages 292-317 in W. C. Clark and R. E. Munn, editors. Sustainable development of the biosphere. Cambridge University Press, Cambridge, UK.

Holling, C. S. 2001. Understanding the complexity of economic, ecological, and social systems. Ecosystems 4:390-405. https://doi.org/10.1007/s10021-001-0101-5

Holling, C. S., L. H. Gunderson, and G. Peterson. 2002. Sustainability and panarchies. Pages 63-102 in L. H. Gunderson and C. S. Holling, editors. Panarchy: understanding transformations in human and natural systems. Island, Washington, D.C., USA.

Holling, C. S., and G. K. Meffe. 1996. Command and control and the pathology of natural resource management. Conservation Biology 10(2):328-337. https://doi.org/10.1046/j.1523-1739.1996.10020328.x

Hornik, K. 2020. NLP: Natural Language Processing Infrastructure, R Package Version 0.2-1. https://cran.r-project.org/web/packages/NLP/index.html

Martin, R., and P. Sunley. 2011. Conceptualizing cluster evolution: Beyond the life cycle model? Regional Studies 45(10):1299-1318. https://doi.org/10.1080/00343404.2011.622263

Maru, Y. T., C. S. Fletcher, and V. Chewings. 2012. A synthesis of current approaches to traps Is useful but needs rethinking for indigenous disadvantage and poverty research. Ecology and Society 17(2):7. https://doi.org/10.5751/ES-04793-170207

Moen, J., and E. C. H. Keskitalo. 2010. Interlocking panarchies in multi-use boreal forests in Sweden. Ecology and Society 15(3):17. https://doi.org/10.5751/ES-03444-150317

Parker, J. N., and E. J. Hackett. 2012. Hot spots and hot moments in scientific collaborations and social movements. American Sociological Review 77(1):21-44. https://doi.org/10.1177/0003122411433763

Redman, C. L., and A. P. Kinzig. 2003. Resilience of past landscapes: resilience theory, society, and the longue durée. Conservation Ecology 7(1):14. https://doi.org/10.5751/ES-00510-070114

Robinson, D., and J. Silge. 2021. tidytext: Text Mining Using dplyr, ggplot2, and Other Tidy Tools, R Package Version 0.3.3. https://github.com/juliasilge/tidytext

Rocha, J. C., and R. Wikström. 2015. Detecting potential impacts on ecosystem services related to ecological regime shifts – a matter of wording. Pages 94-113 in J. C. Rocha, Regime Shifts in the Anthropocene. Doctoral thesis, Stockholm Resilience Centre, Stockholm University, Stockholm, Sweden.

Simmie, J., and R. Martin. 2010. The economic resilience of regions: towards an evolutionary approach. Cambridge Journal of Regions, Economy, and Society 3(1):27-43. https://doi.org/10.1093/cjres/rsp029

Stringer, L. C., A. J. Dougill, E. Fraser, K. Hubacek, C. Prell, and M. S. Reed. 2006. Unpacking “participation” in the adaptive management of social-ecological systems: A critical review. Ecology and Society 11(2):39. https://doi.org/10.5751/ES-01896-110239

Zeng, G., J. Gao, L. Shekhtman, S. Guo, W. Lv, J. Wu, H. Liu, O. Levy, D. Li, Z. Gao, et al. 2020. Multiple metastable network states in urban traffic. Proceedings of the National Academy of Sciences 117(30):17528-17534. https://doi.org/10.1073/pnas.1907493117