Home | Archives | About | Login | Submissions | Notify | Contact | Search
 E&S Home > Vol. 12, No. 2 > Art. 6

Copyright © 2007 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf version of this article

The following is the established format for referencing this article:
Muñoz-Erickson, T. A., B. Aguilar-González, and T. D. Sisk. 2007. Linking ecosystem health indicators and collaborative management: a systematic framework to evaluate ecological and social outcomes. Ecology and Society 12(2): 6. [online] URL: http://www.ecologyandsociety.org/vol12/iss2/art6/


Linking Ecosystem Health Indicators and Collaborative Management: a Systematic Framework to Evaluate Ecological and Social Outcomes

Tischa A. Muñoz-Erickson 1, Bernardo Aguilar-González 2 and Thomas D. Sisk 3

1Arizona State University, 2Prescott College, 3Northern Arizona University


Collaborative management has gained popularity across the United States as a means of addressing the sustainability of mixed-ownership landscapes and resolving persistent conflicts in public lands management. At the same time, it has generated skepticism because its ecological and social outcomes are seldom measured. Evaluating the success of collaborative efforts is difficult because frameworks to assess on-the-ground outcomes are poorly developed or altogether lacking. Ecosystem health indicators are valuable tools for evaluating site-specific outcomes of collaboration based on the effects of collaboration on ecological and socioeconomic conditions. We present the holistic ecosystem health indicator, a promising framework for evaluating the outcomes of collaborative processes, which uses ecological, social, and interactive indicators to monitor conditions through time. Finally, we draw upon our experience working with the Diablo Trust, a community-based collaborative group in northern Arizona, USA, to illustrate the development of an indicator selection model generated through a stakeholder-driven process.

Key words: adaptive management; collaborative management; holistic ecosystem health indicator; northern Arizona rangeland; outcome evaluation; participatory approach; socio-ecological systems; sustainability


Around the world, citizens are coming together in partnerships, watershed councils, and community-based groups to participate in the management of public lands and natural resources. In the United States, this trend has contributed to a growing interest in collaboration as a promising decision-making approach for resolving conflicts over the management of public lands and natural resources (Snow 2001). As collaboration gains popularity, it is often greeted with skepticism because its benefits for the environment and human communities remain hypothetical. This has engendered a call for more formal evaluation of the effectiveness of collaborative processes in decision making for land management (Moote et al. 2000).

Here we show that the growing literature on evaluating the effectiveness of collaboration is focused primarily on the internal dynamics of the process itself, whereas efforts to measure specific outcomes related to land and resource management actions are few. Although recent work on process-oriented evaluation has achieved great strides in establishing the value of collaborative management, we argue that the understanding of the effectiveness of collaboration would be enhanced by the use of assessment approaches that address both the processes and outcomes of this management approach. To address this challenge, we explore the use of ecosystem health indicators to evaluate the outcomes of collaborative processes. Specifically, we focus on an approach that combines quantitative and qualitative information regarding ecological and socioeconomic factors that influence the success or failure of collaborative approaches. We illustrate the potential synergy between ecosystem health and the collaborative process by developing an indicator selection model that incorporates stakeholder input. In the future, we will explore both ongoing efforts to implement this collaboratively developed monitoring program and the program’s influence on decision making.

We have organized this paper into four sections. The first section provides the theoretical background underlying collaborative management to explain the complex nature of these social-ecological systems. In the second section, we review the literature that addresses the evaluation of collaborative processes and demonstrate that the measurement of ecological and social outcomes is necessary for a comprehensive evaluation of this management approach. The third section explores our central argument that ecosystem health indicators provide a suitable framework with which to characterize the complexity of the collaborative process and link it to environmental and social outcomes. Specifically, we present the holistic ecosystem health indicator (Aguilar 1999) as an appropriate framework with which to evaluate outcomes of collaborative land management because it uses ecological, social, and interactive indicators that can be used to monitor conditions through time. Finally, because the selection of indicators involves negotiation among diverse stakeholder values and perspectives of the system, we describe the indicator selection model that we used to identify core indicators with stakeholders of the Diablo Trust, a community-based collaborative group developing sustainable rangeland management approaches for more than 170,000 ha of mixed-ownership lands in northern Arizona, USA.


Collaborative management, or co-management, is commonly defined as the process by which citizens and managers “come together across disciplinary, management, and ownership boundaries to collectively determine management goals, develop management plans, implement those plans, and monitor and adjust [them] as necessary” (Cortner and Moote 1999:91). Increasingly, community-based groups, partnerships, watershed councils, and other forms of stakeholder organizations are using collaborative decision making because it provides an organizational structure that facilitates dialog and equalizes the power distribution among a broad range of stakeholders (Gray 1989, Cortner and Moote 1999, Wondolleck and Yaffee 2000).

Collaborative processes for natural resource management are based on a number of theoretical models. The most influential to this study are participatory and discursive democracy (Dryzek 1990), organizational theory (Gray 1989), collaborative learning (Daniels and Walker 1996), and adaptive governance (Brunner et al. 2005). More recently, collaborative management systems have been viewed as social-ecological units that behave as complex adaptive systems. As such, their organizational structure and function continuously change to link the ecological and social systems that underlie ecosystem management (Olsson et al. 2004, Carlsson and Berkes 2005). Complex adaptive systems are characterized as self-organizing, with the ability to acquire information about the environment and adapt to changing and uncertain conditions (Gell-Mann 1994).

Because of this adaptive capacity, some authors argue that collaborative forms of management are suitable for maintaining and increasing ecosystem resilience, a key component of sustainability (Walker et al. 2002, Olsson et al. 2004, Armitage 2005). Resilience is defined as the ability of a system to absorb and recover from disturbance (Holling 1978). The concept of resilience acknowledges that the inherent variability in social-ecological systems can result in more than one possible equilibrium state. However, when an ecosystem moves from one stable state to another as a result of some disturbance, it may lose resilience, which in turn can result in a decrease in important ecosystem services produced in the desired state (Elmqvist et al. 2003). Managing for ecosystem resilience recognizes that managers should focus on maintaining those elements associated with renewal and reorganization through experimentation and mutual learning, rather than relying solely on forecasting methods (Berkes and Folke 1998, Walker et al. 2002).

To achieve success in managing ecosystem resilience, stakeholders participating in collaborative management must commit themselves to adaptive strategies, especially monitoring and responding to environmental feedback. At the core of adaptive management, monitoring is what allows institutions to learn as individuals do (Berkes et al. 2003). Groups and management institutions that engage in the collection and sharing of various sources of information among stakeholders through monitoring are able to anticipate systemic change and build adaptive capacity (Gunderson 2003, Olsson et al. 2004). Monitoring also helps stakeholders to negotiate complicated resource issues by providing information that helps them arrive at collective decisions. However, feedback loops linking monitoring information to the revision of management goals are critical in an adaptive management framework (Moir and Block 2001). Bliss et al. (2001) suggest that multiparty monitoring approaches, in which both stakeholders and experts collect and interpret ecological and social data, are more likely to have multiple feedbacks linking information to decision making than efforts carried out by scientists alone.

However, despite a strong theoretical basis and broad participation in collaborative management efforts, critics often describe them as “unproven experiments” (Kenney 2000). Environmental groups, in particular, have questioned the effectiveness of collaboration, given its susceptibility to co-option by economic interests (Kenney 1999) and the imbalance of power among stakeholders (McCloskey 1996). The debates over the practical implications of these concepts have led to widespread interest in evaluating the effectiveness of collaborative processes (Kenney 2000, Moote et al. 2000).


To determine the potential and limitations of collaborative management, both proponents and critics of the approach require an unambiguous means to evaluate its effectiveness (Conley and Moote 2003). However, identifying the appropriate criteria, methods, and measurable variables for assessing success is a difficult task. Most evaluation research has focused on examining case studies that describe lessons and keys to successful collaborative processes (Kenney 1999, Sabatier et al. 2005). The assessment of participant perceptions is the most common sampling method used in this type of evaluation (Conley and Moote 2003). Studies regularly cited in the literature (i.e., Williams and Ellefson 1996, Cestero 1999, Wondolleck and Yaffee 2000) have used similar definitions for collaboration and have developed rigorous criteria for success based on internal processes and organizational dynamics, including an effective consensus process, trust building, open communication and information sharing, increased learning, and the development of mutually agreeable management plans.

A few studies have attempted to evaluate the ecological and social impacts of collaborative efforts. Most notably, Sabatier et al. (2005) recently compiled the most comprehensive empirical assessment of watershed collaborations that uses qualitative and quantitative data to understand the nature and consequences of collaboration. This effort provided a second generation of scholarship by going beyond descriptive analyses of collaborative management to look at substantive outcomes as indicators of effectiveness. However, the evaluation primarily relied on measuring people’s perceptions about the implementation and results, rather than on-the-ground outcomes. Direct measurement of change was difficult because of the lack of scientific monitoring data linking partnership activities to changes in ecological and socioeconomic conditions and because of the time required to monitor and test changes on the ground (Sabatier et al. 2005).

Given the challenges that measuring on-the-ground impacts pose, process evaluation remains an important methodological approach to determine whether collaborative management is leading to the desired results (Conley and Moote 2003). Carlsson and Berkes (2005) argue that, because collaborative management is a continuous problem-solving process and not a fixed state, research should continue to give greater emphasis to the function of the process as a result. Therefore, as Fernández-Giménez et al. (2006) propose, the evaluation of the collaborative group’s capacity to conduct and assess its own learning process may serve as a surrogate by which to measure environmental and social outcomes. The assumption is that increased adaptive capacity will lead to improved management and ultimately, improved environmental and socioeconomic conditions. However, given that collaborative efforts happen in social-ecological systems that are affected by factors beyond management, this assumption requires further empirical exploration.

Our intention here is not to undermine the importance of evaluating the social processes of collaboration, but rather to enhance it by linking it to changes in ecological and social conditions. Our motivation is also practical in that we recognize that an effective and lasting process also depends on the ability of stakeholders to evaluate the impacts of their management decisions through site-specific monitoring. Although we acknowledge the difficulty in measuring collaborative management outcomes, we think that there are tools available to initiate this process. In this process, interdisciplinary indicators are suitable tools that allow for the biophysical and social outcomes of the collaborative process to be measured in tandem.


Ecosystem health indicators provide a powerful framework with which to address these methodological challenges in evaluating collaborative management outcomes. Ecosystem health is a transdisciplinary concept that bridges the natural, social, and health sciences. It can incorporate the human values and perceptions that are inseparable parts of management. A healthy ecosystem is defined as a social-ecological unit that is stable and sustainable, maintaining its characteristic composition, organization, and function over time while remaining economically viable and sustaining human communities (Costanza 1992, Rapport 1998). The breadth of this definition indicates that ecosystem health is an integrative notion that acknowledges societal values in defining future desired conditions while relying on scientific criteria (Steedman 1994).

Because the concept of ecosystem health can involve inherent value judgments about what actions are socially desirable, its definition and use require public involvement (Costanza et al. 1998a, Lackey 2001). As an integrative notion, ecosystem health is not meant to rely solely on its scientific basis, in the hypothetico-deductive sense, nor does it pretend to give predictive descriptions of causal mechanisms of the complex behavior of social-ecological systems; instead, it provides for case-by-case evaluations in real-world settings with stakeholder input (Wilcox 2001). Viewed in this light, ecosystem health furnishes a theoretical framework and practical methods for monitoring and assessing the condition and quality of ecosystems (Bertollo 1998). Because most people have an intuitive idea of what constitutes health, the term facilitates understanding among citizens, managers, and scientists when formulating management decisions (Rapport 1998).

We recognize that the concept of ecosystem health indicators, and indicators in general for that matter, is not without its critiques (see, for example, Lackey 2001). Nonetheless, managers and decision makers find indicators useful in detecting stressors and changes in ecosystem and social conditions by consolidating large amounts of information whenever the primary information of an object is too extensive or complex to be handled without aggregations (Müller et al. 2000). Indicators are widely used to assess ecosystem health in both protected and managed ecosystems and for environmental planning purposes, as demonstrated by the extensive compilation of case studies by Rapport et al. (2003). Ecosystem health indicators are particularly applicable for monitoring managed ecosystems such as collaboratively governed ecosystems in which the environment is highly influenced by human activity and thus conditions need to be assessed with respect to both ecological integrity and societal goals (Rapport et al. 1998). Beyond attempting to perform a reductionistic exercise, the use of indicators allows the comparison of different stages of the collaborative conservation process and diverse collaborative organizations.

Practical approaches to assessing ecosystem health involve the identification of ecological and social goals and the development of important indicators that measure progress toward these end points (Costanza 1992). Assessments of ecosystem health should encompass indicators that reflect properties of resilience, organization, and vigor (Rapport et al. 1998). Vigor is measured in terms of the metabolism or primary productivity of the existing organic base; organization is assessed by examining the diversity and number of interactions between system components. These attributes are derived from ecological perspectives, but they have analogous measures in other fields and may be applied to the biophysical, socioeconomic, and human health dimensions of the ecosystem (Costanza et al. 1998b).

The holistic ecosystem health indicator as an outcome evaluation framework

We propose that the integrative holistic ecosystem health indicator (HEHI) is a powerful tool for measuring ecosystem health (Aguilar 1999). We think that it is particularly useful for assessing the performance of collaborative processes. Developed in 1999 by the Center for Sustainable Development Studies to assess and evaluate ecosystem health in managed ecosystems in Costa Rica, the HEHI is an interdisciplinary methodology based on the ecosystem health criteria proposed by Costanza et al. (1998b) and Rapport (1998).

The HEHI provides an organizational framework with which to integrate diverse ecosystem health indicators, including ecological and social variables. It follows a hierarchical structure based on three main branches or dimensions of sustainability: ecological, social, and interactive (Fig. 1). The ecological branch focuses on measures of the condition and trends of the ecosystem under study. Socioeconomic measures of the human communities dependent on the ecosystem or affected by management decisions are organized within the social branch. The interactive branch considers measures related to land use and management decisions that define the interactions between the human communities and the ecosystem. Each branch is subdivided into appropriate categories reflecting attributes of the management objectives for the particular system under study. Indicators that reflect the conditions and trends of important system variables are selected and assigned a benchmark or threshold value based on the current understanding of system dynamics and the specific management objectives.

To prioritize the importance of each category and indicator, weighting values are assigned to each in a manner that reflects its relative importance to the health of the system and to management goals (Table 1). In this case, we structure the HEHI to include a total of 1000 points for each branch. Other applications might require a different point scale. Weights are used to adjust indicator importance to account for differences in data availability, quality, and sampling considerations. Individual indicators are then assigned a portion of the total points available in each category (Fig. 1). The information content increases as one moves upward in this hierarchical structure from primary data collection to the evaluation of each indicator to categories and branches by aggregation (Fig. 1). Once data have been collected for all indicators, they are compared to the benchmark for each indicator and assigned the points according to how well the data meet the desired target. If the value of the data measured for a particular indicator falls within the target or benchmark range, all possible points are assigned for that indicator; if the value falls below the benchmark range, it receives an appropriate fraction of the points. Once the scores have been assigned to each indicator, the points for each of the indicators are summed to obtain a total for the category, and these are totaled for the three HEHI branches. Scores can be viewed separately for all indicators, categories, and branches, or as a combined, final HEHI score. The final HEHI score provides an integrated assessment of the ecological and social outcomes of management, as well as the management processes affecting the outcomes, which are key elements to evaluating collaborative management.

The selection of the appropriate number and type of indicators and their benchmarks, as well as the appropriate time frame for tracking management progress, is critical to the success of the HEHI approach (Muñoz-Erickson et al. 2004). Too often, system complexity and uncertainty leads managers to use a uniform set of indicators across broad regions, even though the context of these variables may vary substantially (Noon 2003). Our past experience with the HEHI approach (Aguilar 1999) led to similar limitations. To address these concerns and assure that measures are identified based on the context-specific conditions and success criteria of the socio-ecological system under study, we used a participatory approach to assist in the process of developing appropriate indicators for the HEHI framework for the case study documented here. In the next section, we describe the methodological steps involved in the framework used by the Diablo Trust, a rangeland collaborative management group based in Flagstaff, Arizona, USA.



In 1993, a group of ranchers, representatives of federal and state management agencies, research scientists, and involved citizens, all of whom were concerned about ongoing land-management conflicts affecting 170,000 ha of mixed private and public lands southeast of Flagstaff, Arizona, USA, came together to form the Diablo Trust, a non-profit land-management organization. Although the entire area falls within the boundaries of two large ranches, legal management responsibilities fall to a number of public and private entities whose jurisdictions are arrayed in a complex, overlapping patchwork pattern across the landscape. The Diablo Trust indirectly influences management decision making by serving as the convener of multiparty discussions, integrator of management perspectives, and developer of management plans that reflect the values of a large and diverse group of stakeholders. Diablo Trust stakeholders have developed a distinctive collaborative culture that has allowed the group to overcome differences in interests over the years and to agree on specific objectives and desired future conditions that would satisfy the widely held goals of sustainable use and conservation of public resources. Over the initial two years of collaboration, the group developed a three-part holistic goal composed of objectives for quality of life, goals for production, and goals for a future landscape and resource base (Table 2; Diablo Trust 1999). Whereas the specific management actions associated with each goal are routinely negotiated among the stakeholders, they all, including the federal and state agencies, agreed that the three-part holistic goal should be used to define their measures of progress (Muñoz-Erickson 2004). Therefore, this statement of goals provided the essential foundation from which we developed an assessment tool based on the holistic ecosystem health indicator (HEHI).

After the failure of earlier attempts to develop a monitoring plan using a method upon which all stakeholders could agree, we began working with the group to develop the HEHI as a monitoring framework, and most importantly, to design a process by which to select indicators collaboratively. This project began in 2001 and built upon a 5-yr collaborative research relationship with the group, which allowed us to design a process for indicator selection that incorporated multiple sources of knowledge and values while maintaining the researchers’ reflective independence. It is in this spirit that we present the steps that we developed to generate ecological, social, and interactive indicators for the Diablo Trust.

Selection of core indicators for the Diablo Trust

We relied on three seminal papers on indicator selection for guidance in developing an approach that was both rigorous in incorporating multiple sources of knowledge and practical for implementation in a collaborative management context. Noon (2003) proposed a series of steps to select indicators that are relevant to the management unit under consideration. Although these guidelines use habitat-based variables as indicators of conditions for sensitive species, they are based on ecological integrity and were easily adapted to broader management goals. We found the methodology described by Salafsky and Margolis (2001) useful for integrating community-level economic and social concerns with ecological indicators. Finally, we used the criteria to weight indicator points identified by Tegler et al. (2001) to select appropriate monitoring variables.

This integrated quantitative and qualitative methodology combines the ecological and socioeconomic factors involved in the management of Diablo Trust lands. It also defines an approach to identify the key variables for monitoring the site-specific ecological, social, and interactive processes that define the health of any managed ecosystem. In defining ecosystem health indicators for this system, we incorporated the best available science while drawing on the collaborative process itself to incorporate stakeholder views and knowledge. Therefore, this approach to indicator selection reflects the multiple values held by stakeholders and integrates both normative and factual information in the evaluation process. The steps that we followed in developing the ecosystem health indicators (Fig. 2) are described below.

Step 1: Setting the social-ecological context

The first step was to describe the current ecological and social conditions of the study area and define the management objectives of the collaborative group. To this end, we referred to the Diablo Trust’s official vision and goal statements to clarify numerous management objectives that would guide actions and influence the health of the inclusive social-ecological system. We also reviewed the scientific literature to identify characteristics that define the health of the ecosystem and the community, especially if disagreements existed regarding the desired conditions defined by the Diablo Trust. The observations and input of Diablo Trust members were also essential at this preliminary stage of understanding the social-ecological context. The synthesis of this information became the basis for developing a definition of ecosystem health that is specific to the Diablo Trust land base.

Step 2: Characterizing complexity: a conceptual model based on local site conditions

We next developed a conceptual model to describe the explicit relationships between key variables of the socio-ecological system. A conceptual model specifies the relationship between the natural and human driving forces or stressors affecting ecosystem components, the structure and composition of the intertwined ecological and social systems, and the management objectives and target conditions for the study area (Salafsky and Margolis 2001, Noon 2003). Conceptual models are also used to identify: the ecosystem services that are most important to the stakeholders; the components of the system for which there is adequate information, as well as areas of uncertainty; and the variables that are essential in maintaining the resilience of the system (Walker et al. 2002). The model was intended to depict how the Diablo Trust system functions and provide clues as to the outcomes or target conditions that management actions are trying to achieve. Our model for the 170,000-ha Diablo Trust land base specifies natural and human-induced drivers and suggests how management objectives should influence the selection of ecosystem health indicators (Fig. 3). We consulted Diablo Trust stakeholders to provide practical context and validate how well the model represented their vision of the system. Although this model does not formally depict causal relationships, which is a daunting task given the system’s inherent complexity, we think that it captures the influential variables affecting the management unit as a whole.

Step 3: Stakeholder input

Local knowledge is invaluable in efforts to characterize the management context and identify appropriate desired future conditions for integrated ecological and social systems (Fischer 2000, Olsson et al. 2004). We used a variety of social research methods, including focus groups, survey questionnaires, and interviews, to gather local knowledge about natural resources and local communities. In this way, we were able to identify the ecological, social, and interactive variables that were most relevant to Diablo Trust lands and objectives. In addition, we assessed the well-being of the community, sources of conflict and uncertainty, and information gaps. For the Diablo Trust case study, we conducted interviews with key participants in the collaborative organization, including ranchers, scientists, environmentalists, community members, and federal, state, and local government agencies. These interviews inquired about the effectiveness of the collaborative process and the factors that stakeholders believed facilitated or limited success. We also asked respondents to prioritize a preliminary list of ecological, socioeconomic, and management or institutional factors, identified in the literature, that were believed to influence outcomes. Using information gained from these efforts, we revised the conceptual model and proposed a candidate set of indicators with which to monitor the effectiveness of collaborative management based on stakeholder input.

Step 4: Assigning priorities to core indicators

Once the key aspects of the system and the candidate indicators had been identified, we narrowed the list to a smaller number of core indicators. We used a selection matrix to aid in assigning priorities to indicators based on the relevance of the indicator to the ecosystem health criteria and management objectives, along with other practical criteria such as the availability and quality of baseline data, sensitivity at multiple scales, clarity and relevance to stakeholders, and cost-effectiveness (Table 3). The value of the indicators as components of an overarching evaluation tool depends on the ability of the collaborative group to monitor them through time; thus, they should be as practical as possible without compromising the information requirements. We consulted with stakeholders and outside experts to help identify appropriate sampling protocols for the indicators and to evaluate their cost-effectiveness.

Step 5: Selection of a final list of indicators and score ranking

A final list of indicators for the Diablo Trust was completed and incorporated into the HEHI framework. The indicators’ relative weights represent a negotiated agreement with stakeholders based on multiple scientific, practical, and social criteria discussed in the collaborative process. As part of this process, each indicator was re-evaluated using both the selection matrix (Table 3) and the abovementioned guidelines by Aguilar (1999; Table 1). This process resulted in the selection of 19 indicators for the ecological branch (Table 4), 18 indicators for the social branch (Table 5), and 14 indicators for the interactive branch (Table 6).

The ecological branch focuses on the biophysical parameters over which the Diablo Trust collaborative has the most influence through management; thus, they are outcome indicators (Table 4). The indicators are grouped into five categories, with vegetation and watershed health receiving the highest point totals because they are directly linked to the management objectives of the collaborative and management agencies. Soil quality and primary productivity follow in the middle category because they are also directly related to management and are critical to ecosystem health, but they are costly parameters to collect in the field. Finally, a wildlife category was included because wildlife population sizes and habitat quality play an important role in determining management actions on public lands.

The social branch includes indicators relating to the internal process of the collaborative, i.e., process indicators, and outcome indicators for those social aspects that the Diablo Trust directly influences through management (Table 5). This branch also includes measurements relating to the context of the larger social system that influences stakeholder decisions and the goals of the collaborative organization, as well as variables that might also affect environmental conditions but are out of the control of the Diablo Trust. The indicators are grouped into five categories. Collaborative social outcomes received high point totals because of the importance of evaluating the functional aspects of the collaborative organization. Economic stability and community strength are direct measures of the adaptive capacity of local communities and received intermediate weights. Finally, demographics and income, which are not direct measures of social outcomes, provide a picture of the socioeconomic context of the community.

The interactive branch includes indicators that measure the direct interaction of management and the environment through land uses, regulations, and environmental awareness (Table 6). This branch also includes context and process indicators that further establish the social-ecological context of these interactions. The indicators are grouped into four categories, with land-use practices, which includes indicators measuring the direct interaction between humans and nature receiving the highest point totals. Awareness and public attitudes and collaborative-management outcomes measure the indirect use of resources through stakeholders’ perceptions. As the final category, implementation of agency management actions provides a context of the effectiveness of the current institutional framework in which the collaborative organization functions.

More detail on the original structure of the HEHI (Aguilar 1999) and the selected indicators can be found in Muñoz-Erickson et al. (2004, 2006). However, we caution that the specifics of indicator selection are place based, and the results presented here should not be applied uncritically to other regions (Noon 2003). Efforts are now underway to measure each indicator and examine the outcomes of the Diablo Trust collaborative based on the performance of the indicators of the HEHI. The steps outlined in this section are not meant to be conclusive; rather, they are likely the first stage in an iterative process. As learning and communication increases among stakeholders in the implementation stage of this project, and as stakeholders pick new meanings and find new values within their social-ecological system, the content of the HEHI may change to better reflect the system’s feedback. After several iterations of evaluating the HEHI, stakeholders may find that some of the indicators are no longer relevant or that new indicators are necessary to better reflect changing ecological and social conditions. Similarly, indicators may be re-evaluated based on changes in agency policy or if the Diablo Trust modifies its long-term management goals. Therefore, the HEHI and the steps to develop it are intended to be flexible enough to accommodate changes in ecosystem conditions, management context, and stakeholder values, as the collaborative group learns and evolves. In this way, the HEHI serves as a reflective tool to encourage the adaptive capacity of collaborative groups, allowing them to incorporate new information and adjust to changing levels of uncertainty.


In this paper, we proposed the use of ecosystem health indicators as tools for evaluating the ecological and social outcomes of collaborative management. Using the case study of the Diablo Trust, the holistic ecosystem health indicator (HEHI) was identified as a promising method to measure the effectiveness of collaborative management because it allows for the evaluation of outcomes based on an integration of ecological and social criteria. The interactive branch of the HEHI is particularly useful for collaborative management because it can incorporate stakeholder perceptions, awareness, and involvement in management decisions, permitting the evaluation of the process of collaboration, as well as its outcomes.

This case study from northern Arizona presents a series of steps for the selection of appropriate variables to serve as indicators within the HEHI. These steps are part of the participatory process itself, incorporating stakeholder knowledge and organizing it in a form that facilitates integration with expert information, when appropriate. From our experience in this process, we conclude that the indicators selected reflect what the stakeholders value and how they view their social-ecological system. Thus, as Norton (2005) argues, the process of choosing indicators proved to be a way to elucidate community values, which are at the core of any definition of sustainability. The primary result of these efforts was an HEHI-based monitoring program for the Diablo Trust that will guide the collaborative group’s decisions as they pursue an informed, adaptive approach to management. The Diablo Trust is currently implementing and monitoring indicators across its 170,000-ha landscape; we expect that over the next 2 yr the group will be negotiating the specific short- and long-term management actions needed in response to the HEHI results. These management responses and the utility of the HEHI as an evaluation framework for collaborative outcomes will be the subject of a forthcoming paper. Although the indicators for the HEHI resulting from this process are place based and thus are tailored to meet the site-specific ecological and social information needs of the Diablo Trust, we think that the process used to identify indicators and structure the HEHI will be applicable to any situation in which adaptive approaches to ecosystem management involve multiple stakeholders and a desire for collaborative decision making.


Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link


We are grateful to the Diablo Trust, Flying M Ranch, and Bar T Bar Ranch for their cooperation and support throughout this project. We thank Matthew R. Loeser, David Schlosberg, and David Ostergren for their valuable input toward this research and comments on drafts of this manuscript. This research was supported by funding from the Ecological Restoration Institute of Northern Arizona University and T. and E., Inc., and by in-kind contributions from the Cultural and Regional Studies Program of Prescott College.


Aguilar, B. J. 1999. Applications of ecosystem health for the sustainability of managed systems in Costa Rica. Ecosystem Health 5:36-48.

Armitage, D. 2005. Adaptive capacity and community-based natural resource management. Environmental Management 35:703-715.

Berkes, F., J. Colding, and C. Folke, editors. 2003. Navigating social–ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK.

Berkes, F., and C. Folke. 1998. Linking social and ecological systems for resilience and sustainability. Pages 1-25 in F. Berkes and C. Folke, editors. Linking social and ecological systems: management practices and social mechanisms for building resilience. Cambridge University Press, Cambridge, UK.

Bertollo, P. 1998. Assessing ecosystem health in governed landscapes: a framework for developing core indicators. Ecosystem Health 4(1):33-51.

Bliss, J., G. Aplet, C. Hartzell, P. Harwood, P. Jahnige, D. Kittredge, S. Lewandowski, and M. L. Soscia. 2001. Community-based ecosystem monitoring. Pages 143-167 in G. J. Gray, M. J. Enzer, and J. Kusel, editors. Understanding community-based forest ecosystem management. Food Products Press, Binghamton, New York, USA.

Brunner, R. D., T. A. Steelman, L. Coe-Juell, C. M. Cromley, C. M. Edwards, and D. W. Tucker. 2005. Adaptive governance: integrating science, policy, and decision making. Columbia University Press, New York, New York, USA.

Carlsson, L., and F. Berkes. 2005. Co-management: concepts and methodological implications. Journal of Environmental Management 75:65-76.

Cestero, B. 1999. Beyond the hundredth meeting: a field guide to collaborative conservation on the West’s public lands. Sonoran Institute, Tucson, Arizona, USA.

Conley, A., and M. A. Moote. 2003. Evaluating collaborative natural resource management. Society and Natural Resources 16(5):371-386.

Cortner, H. J., and M. A. Moote. 1999. The politics of ecosystem management. Island Press, Washington, D.C., USA.

Costanza, R. 1992. Toward an operational definition of ecosystem health. Pages 239-256 in R. Costanza, B. G. Norton, and B. D. Haskell, editors. Ecosystem health: new goals for environmental management. Island Press, Washington, D.C., USA.

Costanza, R., M. Mageau, B. Norton, and B. C. Patten. 1998a. Social decision making. Pages 251-260 in D. Rapport, R. Costanza, P. Epstein, C. Gaudet, and R. Levins, editors. Ecosystem health. Blackwell, Malden, Massachusetts, USA.

Costanza, R., M. Mageau, B. Norton, and B. C. Patten. 1998b. Predictors of ecosystem health. Pages 240-250 in D. Rapport, R. Costanza, P. Epstein, C. Gaudet, and R. Levins, editors. Ecosystem health Blackwell, Malden, Massachusetts, USA.

Daniels, S. E., and G. B. Walker. 1996. Collaborative learning: improving public deliberation in ecosystem-based management. Environmental Impact Assessment Review 16:71-102.

Diablo Trust. 1999. Diablo Trust area range management plan and proposed action. Diablo Trust, Flagstaff, Arizona, USA.

Dryzek, J. S. 1990. Discursive democracy: politics, policy, and political science. Cambridge University Press, Cambridge, UK.

Elmqvist, T., C. Folke, M. Nyström, G. Peterson, J. Bengtsson, B. Walker, and J. Norberg. 2003. Response diversity, ecosystem change, and resilience. Frontiers in Ecology 1(9):488-494.

Fernández-Giménez, M. E., B. Aguilar-González, T. Muñoz-Erickson, and C. G. Curtin. 2006. Assessing the adaptive capacity of collaboratively managed rangelands: a test of the concept and comparison of 3 rangeland CBCs. Journal of Community Based Collaboratives Research. Winter 2006. Available online at: http://www.cbcrc.org/php-bin/news/showArticle.php?id=75.

Fischer, F. 2000. Citizens, experts, and the environment: the politics of local knowledge. Duke University Press, Durham, North Carolina, USA.

Gell-Mann, M. 1994. The quark and the jaguar: adventures in the simple and the complex. Freeman, New York, New York, USA.

Gray, B. 1989. Collaborating: finding common ground for multiparty problem. Jossey–Bass, San Francisco, California, USA.

Gunderson, L. H. 2003. Foreward: learning to monitor or monitoring to learn? Pages xi-xv in D. E. Busch and J. C. Trexler, editors. Monitoring ecosystems: interdisciplinary approaches for evaluating ecoregional initiatives. Island Press, Washington, D.C., USA.

Holling, C. S., editor. 1978. Adaptive environmental assessment and management. Wiley, London, UK.

Kenney, D. S. 1999. Are community-based watershed groups really effective? Confronting the thorny issue of measuring success. Chronicle of Community 3(2):33-38.

Kenney, D. S. 2000. Arguing about consensus: examining the case against western watershed initiatives and other collaborative groups active in natural resources management. University of Colorado School of Law, Natural Resources Law Center, Boulder, Colorado, USA.

Lackey, R. T. 2001. Values, policy, and ecosystem health. BioScience 51(6):437-443.

McCloskey, M. 1996. The skeptic: collaboration has its limits. High Country News May 13. Available online at: http://www.hcn.org/servlets/hcn.Article?article_id=1839.

Moir, W. H., and W. M. Block. 2001. Adaptive management on public lands in the United States: commitment or rhetoric? Environmental Management 28(2):141-148.

Moote, A., A. Conley, K. Firehock, and F. Dukes. 2000. Assessing research needs: a summary of a workshop on community-based collaboratives. Udall Center for Studies in Public Policy, University of Arizona, Tucson, Arizona, USA.

Müller, F., R. Hoffmann-Kroll, and H. Wiggering. 2000. Indicating ecosystem integrity—theoretical concepts and environmental requirements. Ecological Modeling 130:13-23.

Muñoz-Erickson, T. A. 2004. Evaluating the ecological and social outcomes of collaborative management: ecosystem health indicators for monitoring effectiveness. Thesis. Center for Environmental Sciences and Education, Northern Arizona University, Flagstaff, Arizona, USA.

Muñoz-Erickson, T., B. Aguilar-González, T. D. Sisk, and M. R. Loeser. 2006. Assessing the effectiveness of the holistic ecosystem health indicator (HEHI) as a monitoring tool to assess the adaptive capacity of community-based collaboratives. Journal of Community Based Collaboratives Research. Winter 2006. Available online at: http://www.cbcrc.org/php-bin/news/showArticle.php?id=77.

Muñoz-Erickson, T. A., M. R. Loeser, and B. J. Aguilar-González. 2004. Identifying indicators of ecosystem health for a semiarid ecosystem: a conceptual approach. Pages 139-152 in C. van Riper III and K. L. Cole, editors. The Colorado Plateau: cultural, biological, and physical research. University of Arizona Press, Tucson, Arizona, USA.

Noon, B. R. 2003. Conceptual issues in monitoring ecological resources. Pages 27-71 in D. E. Busch and J. C. Trexler, editors. Monitoring ecosystems: interdisciplinary approaches for evaluating ecoregional initiatives. Island Press, Washington, D.C., USA.

Norton, B. G. 2005. Sustainability: a philosophy of adaptive ecosystem management. University of Chicago Press, Chicago, Illinois, USA.

Olsson, P., C. Folke, and F. Berkes. 2004. Adaptive comanagement for building resilience in social–ecological systems. Environmental Management 34(1):75-90.

Rapport, D. J. 1998. Defining ecosystem health. Pages 18-33 in D. Rapport, R. Costanza, P. Epstein, C. Gaudet, and R. Levins, editors. Ecosystem health. Blackwell, Malden, Massachusetts, USA.

Rapport, D. J., R. Costanza, and A. J. McMichael. 1998. Assessing ecosystem health. Trends in Ecology and Evolution 13(10):397-402.

Rapport, D. J., W. L. Lasley, D. E. Rolston, N. O. Nielsen, C. O. Qualset, and A. B. Damania, editors. 2003. Managing for healthy ecosystems. CRC Press, Boca Raton, Florida, USA.

Sabatier, P. A., W. Focht, M. Lubell, Z. Trachtenberg, A. Vedlitz, and M. Matlock. 2005. Collaborative approaches to watershed management. Pages 3-21 in P. A. Sabatier, W. Focht, M. Lubell, Z. Trachtenberg, A. Vedlitz, and M. Matlock, editors. Swimming upstream: collaborative approaches to watershed management. MIT Press, Cambridge, Massachusetts, USA.

Salafsky, N., and R. Margoluis. 2001. Overview of a systematic approach to designing, managing, and monitoring conservation and development projects. Pages 313-328 in L. E. Buck, C. C. Geisler, J. Schelhas, and E. Wollenberg, editors. Biological diversity: balancing interests through adaptive collaborative management. CRC Press, Boca Raton, Florida, USA.

Snow, D. 2001. Coming home: an introduction to collaborative conservation. Pages 1-11 in P. Brick, D. Snow, and S. Van de Wetering. Across the Great Divide: explorations in collaborative conservation and the American West. Island Press, Washington, D.C., USA.

Steedman, R. J. 1994. Ecosystem health as a management goal. Journal of the North American Benthological Society 13(4):605-610.

Tegler, B., M. Sharp, and M. A. Johnson. 2001. Ecological monitoring and assessment network’s proposed core monitoring variables: an early warning of environmental change. Environmental Monitoring and Assessment 67:29-55.

Walker, B., S. Carpenter, J. Anderies, N. Abel, G. S. Cumming, M. Janssen, L. Lebel, J. Norberg, G. D. Peterson, and R. Pritchard. 2002. Resilience management in social–ecological systems: a working hypothesis for a participatory approach. Conservation Ecology 6(1):14. [online] URL: http:www.consecol.org/vol6/iss/art14.

Wilcox, B. A. 2001. Ecosystem health in practice: emerging areas of application in environment and human health. Ecosystem Health 7(4): 317-325.

Williams, E. M., and P. V. Ellefson. 1996. Natural resource partnerships: Factors leading to cooperative success in the management of landscape level ecosystems involving mixed ownership. Staff Paper Series Number 113. Department of Forest Resources, University of Minnesota, St. Paul, Minnesota, USA. Available online at: http://fr.cfans.umn.edu/publications/staffpapers/Staffpaper113.pdf.

Wondolleck, J. M., and S. L. Yaffee. 2000. Making collaboration work: lessons from innovation in natural resource management. Island Press, Washington, D.C., USA.

Address of Correspondent:
Tischa A. Muñoz-Erickson
Arizona State University
School of Sustainability
P.O. Box 872511
Tempe, Arizona, USA 85287-2511


Home | Archives | About | Login | Submissions | Notify | Contact | Search