An appraisal of adaptive management planning and implementation in ecological restoration: case studies from the San Francisco Bay Delta, USA

• Abstract
• Introduction
• Materials and Methods
  • Site description
  • Case study selection
  • Case study analysis
• Results
  • Analysis of planning
  • Analysis of practice
• Discussion
• Conclusion
• Responses to this Article
• Acknowledgments
• Literature Cited

INTRODUCTION

A common conceptual basis of ecological restoration is restoring an ecosystem to its historical trajectory, a process that is inextricable from the human context in which a restoration project is situated. Each restoration project operates within a social-ecological system, and the growing vulnerability of these systems, as well as failures in management, have led to calls for more adaptive governance regimes that can deal with uncertainty and change (Folke et al. 2005, Lebel et al. 2006, Olsson et al. 2006, Butler et al. 2015).

Active or passive adaptive management approaches (Box 1) are now widely recommended to govern ecological restoration because implementation often becomes a continuing series of management decisions or actions that have effects on the entire restoration site, as well as beyond its borders (Murray and Marmorek 2003, Clewell et al. 2005). Each restoration decision is strengthened if it is based on a growing pool of research information, updated measurements of ecosystem responses, and routine progress evaluations toward achieving goals and objectives (Davis and Ogden 1994, Buchsbaum and Wigand 2012).

The San Francisco Bay Delta region in northern California, USA, comprises the San Francisco Bay and the Sacramento-San Joaquin Delta. It is an area with sizable restoration goals, and there have been several significant attempts to implement adaptive management since 2000, when a strategy was developed for the CALFED Ecosystem Restoration Project (Delta Stewardship Council 2013a). In 2009, through the Sacramento-San Joaquin Delta Reform Act, adaptive management was incorporated as a formal legislative requirement for the Sacramento-San Joaquin Delta (henceforth referred to as the Delta). The strategy was adopted with the aim of increasing the likelihood of meeting goals successfully, effectively, and economically in the face of substantial ecological uncertainty (Delta Stewardship Council 2013a). To support this legislation, a flexible, iterative adaptive management framework was created as a tool for decision making at the program, plan, or project level (Fig. 1)

Here, we analyze eight tidal marsh restoration case studies in and around the Delta to understand the relationship between adaptive management theory, planning, and practice. Through an analysis of adaptive management documents and in-depth interviews, we tested the hypothesis that adaptive management planning and implementation are not yet well standardized in ecological restoration projects. In doing so, our approach attempted to go beyond identifying challenges commonly faced by practitioners to determine how improved project planning might streamline and increase the ease of adaptive management implementation, without sacrificing scientific rigor.

MATERIALS AND METHODS

A case study approach was used to develop a complete picture of both the planning and practice aspects of ecological restoration projects. The planning documents from eight restoration projects were analyzed to reveal how adaptive management theory was incorporated into each project. Interviews with associated restoration practitioners were analyzed to reveal the practical realities of planning and implementation in the Delta region.

Site description

The Delta was historically a vast dendritic tidal-freshwater marsh characterized by geomorphic and hydrological heterogeneity (Whipple et al. 2012). Reclamation and development since the 1800s, primarily diking for agriculture, has removed 95% of those wetlands from tidal and floodwater inundation (Reed 2002, Delta Stewardship Council 2013a).

This extreme transformation of the landscape has resulted in the collapse of the Delta food web and affected the resilience and size of wildlife populations. Six drivers of change have been implicated in the loss of ecological functions: (1) reduction in the amount of habitat; (2) loss of heterogeneity within habitats; (3) loss of connectivity within and among habitat types; (4) degradation of habitat quality; (5) disconnection of habitats from the physical processes that form, sustain, and confer resilience to them; and (6) the invasion by ecosystem engineers such as Brazilian waterweed, invasive clams, and predatory fish (Whipple et al. 2012).

However, the Delta is still a place of extreme ecological, economic, and social importance. Its waters support > 240 special-status species, an agricultural industry valued at hundreds of millions of dollars (USD) annually, and California’s complex water conveyance system. Therefore, the current ecological collapse, extenuated by drought conditions, is inextricable from social and political controversy (Delta Stewardship Council 2013a). The current approach to address these issues includes a massive push toward restoring extensive areas of diked former marsh (Orr et al. 2003). Tidal marsh restoration design and planning has evolved in the last decade, with projects increasing in size and complexity. However, few undisturbed reference sites remain, and there is still considerable scientific uncertainty about how to restore marsh habitat (Callaway et al. 2011).

Of the eight tidal marsh restoration case studies examined herein (Box 2), five are located within the bounds of the Delta, where adaptive management is legally required. Three projects are located outside the Delta, either in the vicinity of Suisun Marsh or San Francisco Bay. Four projects are still in the planning phase, two are currently under construction, and two have completed construction.

Case study selection

To settle on these eight case studies, a variety of tidal marsh restoration projects were identified that use an adaptive management approach and have publicly available planning documents (Table 1). As long as the projects had these attributes, the phase of project completion (construction planned, ongoing, or completed) was considered irrelevant. The projects were identified using information from the Sacramento-San Joaquin Delta Conservancy and the project mapping tool EcoAtlas (California Wetlands Monitoring Workgroup 2015). Prime case study candidates were then identified using a maximum variation sampling approach (Patton 2002). Current research does not agree on the ideal number of case studies to use in a multiple case study approach, but Eisenhardt (1989) suggests a maximum of seven cases, whereas Yin (1994) suggests that data should be collected until saturation.

Representatives of 12 projects were contacted by email with an interview request, and subsequently, 12 in-depth, semistructured interviews were conducted during winter 2015. Of these 12 projects, 2 fell outside the scope of this examination because the participants did not discuss a singular project, and 2 were not usable because the project did not yet have any publicly available adaptive management planning documents.

Case study analysis

Eight case studies, each with an associated interview, were usable. The interviews explored how the representative of each restoration project defined adaptive management, their attitudes toward it, experiences with implementation, and suggestions to improve its ease of use. In addition, interviews delved into project setbacks and best practices. However, the opinions and personal views shared by interviewees are not necessarily reflective of the organizations for which they work. An interview guide ensured comparability across interviews (Appendix 1). After gaining consent, all interviews were recorded using a Sony ICD-UX533F digital voice recorder and transcribed by hand in Microsoft Word. Initially, the transcripts, ranging from 11–20 pages, were uploaded in MAXQDA, a qualitative data analysis software program, to conduct a preliminary within-case analysis. Each transcript was coded according to key research themes based on both the interview question topics and a background literature review. Themes included adaptive management definitions, challenges, and recommendations. Subsequently, the results of the coding were exported to Microsoft Excel, and a cross-case analysis was conducted. Through categorization and pattern matching, subthemes such as the different types of challenges were determined.

A similar approach was taken to analyze project planning documents. First, the number of key elements in each project’s planning documents was identified. Then, the specificity and detail allotted to each key element was assessed and compared across projects.

RESULTS

Analysis of planning

It is clear from the evaluation of restoration project documents that the general format for adaptive management planning documents is relatively undefined, and that the inclusion of key planning elements varies by project (Table 1). A majority of the eight projects’ plans did not include all of the key elements identified in adaptive management planning guides for the Delta, and only a few of the projects included additional key elements recommended by a wider array of guides. However, several plans are still in draft format, and all aspects are not yet finalized. Therefore, planning related to these projects may still evolve. As they currently stand, almost every project gave a detailed description of the key elements related to early project design: (1) a description of goals and objectives, (2) use of quantitative models to identify trade-offs of different management actions, (3) consideration of restoration design alternatives, and (4) defined performance measures. Six of the eight plans included a monitoring procedure specific to the project objectives. Two projects failed to include a detailed description of monitoring, but these plans are still in draft format. Five of the eight plans stated key uncertainties, specified reporting requirements, and included conceptual models showing cause and effect pathways. The specified reporting requirements, however, related primarily to the permit process rather than to how results would be shared with the public.

Several of the key elements potentially more relevant to adaptive management and the project postplanning were often more ambiguous. Only four projects explicitly translated the key uncertainties into hypotheses of system functions or defined who would be responsible for coordinating and carrying out specific tasks. Only three included a framework for decision making or considered potential future management actions in terms of adaptive changes. Just two projects planned out data management and documentation procedures such as data or information storage; and just two very clearly defined the role of science and technical experts in the project. Further, although all projects considered applied studies or experimentation in some way, a majority (six of eight) stated that additional resources or support were necessary to pursue them. Only two projects explicitly planned experimentation as an integral part of the project.

Analysis of practice

Practitioners’ experiences revealed that in reality, adaptive management is an ambiguous concept, and project planning and implementation is complex and slow. Selected quotations regarding practitioners’ interpretations of adaptive management can be found in Appendix 2. Practitioners often described the adaptive management process as fluid and organic. Beyond this, two differing views became clear. Some practitioners described adaptive management as a common-sense approach without a set form and driven purely by project objectives. As one practitioner stated, “It’s just going to be completely fluid… totally driven by whatever your hypothesis is and if you’re meeting your goals and objectives.” In contrast, some practitioners described it as a very structured and defined strategy. For example, one practitioner stated, “It doesn’t do any good to just react, you have to have a framework, a process by which you respond sensibly and productively.” When considering their projects, practitioners also showed a tendency to break adaptive management into different components, mentioning adaptive management of monitoring elements, of the physical restoration configuration, and of communication.

Projects were exposed to a variety of interrelated challenges, which were similar between projects, although they varied in extent and difficulty (Table 2). Our analysis focuses less on scientific and technical challenges because a consensus was clear that these aspects could be addressed fairly easily, whereas the problems related to financial, bureaucratic, regulatory, and political aspects of the projects were harder to resolve. Supporting quotations defining these challenges can be found in Appendix 3.

The challenges that came up most frequently during interviews dealt with regulatory aspects of the projects (Table 2). Five practitioners felt that the cost and time needed to reacquire permits to make physical adaptive changes after opening the site to tidal inundation were prohibitive. Two projects did not encounter this challenge, perhaps because of the way the projects were set up, with a mind to constant scientific review, cooperation with the regulatory agencies issuing permits, and a long project timeline (e.g., 50 years). Five practitioners felt that the environmental review process was complex, costly, and slow. The main frustration was the fact that restoration projects meant to benefit the environment were faced with the same challenging permit-acquisition process as development projects. However, several practitioners felt that the existing process was necessary and straightforward if approached correctly. Finally, three practitioners detailed the challenges of conflicting regulatory priorities. Tidal marsh restoration is complex from a regulatory standpoint because it generally involves converting one wetland type into another (e.g., managed wetland to tidal marsh). The act of restoration, therefore, can have a harmful effect on preexisting species, a fact that leaves regulatory bodies, as well as the project managers, unsure of how to move forward.

Other frequent challenges that projects faced were bureaucratic, such as acquiring property or funding. Four practitioners encountered difficulty in acquiring the necessary land for restoration. Public agencies are not allowed to offer more than market value for property, and therefore, generally focus on restoring public lands. However, all four projects in question had mandated restoration targets, and agencies encountered difficulty in acquiring enough land to fulfill their obligations while still ensuring the land was conducive to tidal marsh restoration. Four practitioners found it challenging to acquire funding for specific aspects of their projects. Whereas the construction phase of projects was generally well funded, practitioners found that support was limited for aspects of the project related to evaluation and learning such as monitoring, publishing results, attending seminars, and conducting experiments.

A majority of challenges were linked to aspects of project management itself. Three practitioners described challenges in coordinating science and management, and four practitioners described challenges in coordinating the multiple agencies involved in their restoration project. The former was a challenge in trying to balance scientific rigor with the many constraints of managing a project. The later was seen as difficult because of the large number of agencies, each with their own rules, concerns, and stakes in the project. Three practitioners faced shifting internal priorities, including waxing and waning internal support for the restoration project, as well as a lack of priority given to fixing monitoring problems or writing management documents. Four practitioners mentioned the challenge of risk aversion. Practitioners felt that restoration projects were contentious and the people involved faced considerable pressure not to take unnecessary risks. Practitioners therefore placed extraordinary time and effort on project design and planning to avoid failure after implementation. Finally, three practitioners stated that staff turnover was a challenge. Changes in staffing were common because of the number of years, or even decades, needed to bring projects to completion. Effects included project setbacks, changes to standing agreements, or starting projects on a different track.

The final two challenges dealt with project-public interactions. Four practitioners stated that negotiations with adjacent landowners were intricate and caused project delays. Landowners were interested in how water moved on and off the site and in ensuring that there would be no adverse effects from the restoration operations. Two practitioners found it challenging to gain public support for their projects. Miscommunications and misunderstandings between restoration staff and the public were common and at times created derisive and contentious atmospheres around these projects.

In addition to challenges, practitioners were asked to describe best practices and ideas to improve the facilitation of adaptive management. Selected quotes can be found in Appendix 4. Some of these recommendations are achievable at the project level, whereas others require structural changes at a higher level. For example, several practitioners called for a more formalized ecological restoration community and a regional monitoring framework to increase opportunities for collaboration and coordination across projects. In addition, several practitioners felt that flexible permitting, increased funding, and an improved system for acquiring land would facilitate both restoration and adaptive management.

At the project level, a majority of practitioners felt that there needed to be “more doing” so that learning would be based on experience. They spoke of the importance of proper site preparation, as well as frequent site visits to notice surprises that might affect the trajectory of the restoration. One practitioner felt that adaptive management could be facilitated better if projects incorporated a standing independent scientific review team that worked as an intermediary between the project and the regulatory agencies. Another practitioner felt that a multibenefit approach was beneficial, for example, combining flood protection and restoration goals. The most common themes were the need for increased communication between restoration participants, and the need to create a culture of patience, listening, and openness. In line with this, practitioners recommended trying to make plans as specific as possible, with defined hypotheses, organizational structures, and requirements for documentation and continuity.

DISCUSSION

The investigated case studies are essentially some of the first restoration projects in the Delta that incorporated the Delta Reform Act concept of adaptive management, which makes them a valuable source of information. It is too early to evaluate whether adaptive management is a successful management technique for each individual restoration project. However, one can start to evaluate the realities of the adaptive management planning and implementation process.

There are some limitations to the assessments presented herein. All relevant plans used in the analysis of project documents may not have been finalized because of the varying stages of project completion. In addition, not all planning documents had the same format, making comparison sometimes difficult. Planning elements were clearly defined to make the evaluations as objective as possible. In addition, the interview approach reveals perspectives specific to the practitioners’ duties and experiences, which may vary somewhat between projects. Because the questions asked were often open ended, the most important, but perhaps not all, recommendations and challenges were captured. Restoration in the Delta operates in an extremely complex social, political, and economic setting, and the nuances of each project’s challenges may be lost in this overview.

However, the challenges described by restoration practitioners in and around the Delta have all been identified previously as reasons for adaptive management failure, revealing that they are not specific to these case studies or even ecological restoration (Gregory et al. 2006, Walters 2007, Allen and Gunderson 2011). For this reason, the lessons learned from this analysis may be valuable beyond the field of tidal marsh restoration and relevant beyond the geographical context of the San Francisco Bay Delta Estuary.

While a majority of the challenges indirectly affect adaptive management by delaying the rate at which decisions and changes can be made, several directly influence the implementation of adaptive management. These challenges include: making physical adaptive changes, acquiring funding, coordinating science and management, and risk aversion. These four challenges essentially interrupt the iterative adaptive management cycle between the evaluation and planning stages or between the planning and implementation stages (Fig. 1). In doing so, they eliminate opportunities to use learning to inform key management decisions.

The feedback between learning and decision making is a defining feature of adaptive management (Williams and Brown 2012). Theoretically, it allows for action, even in the face of uncertainty, because of the understanding that new learning will fill the gaps in knowledge over time (Buchsbaum and Wigand 2012). Perhaps because the iterative process is being interrupted repeatedly, this feedback process does not seem to be occurring yet in the Delta. Addressing the four aforementioned challenges may help streamline the feedback process.

However, compounding this problem, planning documents do not necessarily define key uncertainties or, more importantly, specify how to reduce them. The plans generally tried to account for all of the identified key elements, but often did not include enough details to provide clear guidance. Ambiguity was common, particularly in planning related to decision making, potential management actions, data management, documentation procedures, organizational structure, and scientific review.

Research suggests that project success is positively correlated with investment in defining requirements and in developing technical specifications (Dvir et al. 2003). Therefore, addressing ambiguity and the lack of detailed planning for integral adaptive management elements may help address the aforementioned challenges and increase chances of successful adaptive management.

In fact, practitioners called for more detailed planning, including increased specificity, improved documentation, defined hypotheses, defined organizational structures, and the inclusion of independent science review. One practitioner stated that if plans were more specific, “Then it’s not me going to my boss saying, ‘Oh, I think we need to add a breach here.’ It’s me going to my boss saying, ‘This action on site is not occurring, this is what is laid out to fix and remedy that issue, and this is what the suggestion is in the restoration plan to do that.’” Projects that included detailed planning for the key elements identified in Table 1 were more able to ensure an iterative adaptive management cycle of learning. For example, two projects that defined a very specific science review process were able to work more closely with regulatory agencies to make physical adaptive changes and address risk aversion (Appendix 4).

Changes at the project level may therefore begin to address some of the internal, external, and regulatory challenges (Table 2) and help to deconstruct obstacles to the iterative cycle of learning. However, these changes may add additional expenses or may be difficult to plan early on because of a lack of information (Cabin 2007). The remaining regulatory and bureaucratic challenges cannot simply be addressed at the project level but may require changes at a governance level (Table 2). Large-scale modifications are currently being considered for ecological restoration in the Delta, and efforts are ongoing to increase structural flexibility to support better an adaptive management approach (Delta Stewardship Council 2013b, 2014, Lund and Moyle 2013). The analyses presented herein support efforts to create a formalized ecological restoration community for the San Francisco Bay Delta Estuary, which could potentially act as a centralized forum to address the challenges or knowledge-sharing issues addressed herein.

Restoration projects are immensely complex, and creating shared interpretations of adaptive management may also be essential to adaptive management success. From a theoretical standpoint, the cycle is a structured, iterative process marked by set decision-making points, at which time management actions or adjustments should be made (Williams 2011). Under a passive approach, the best management action is simply implemented, and monitoring and evaluation are used to introduce the appropriate adjustments. Under an active adaptive management approach, monitoring and evaluation of each alternative helps determine which one is more effective in meeting objectives, and adjustments are made in the next round of management decisions (Murray and Marmorek 2003). However, in reality, evaluation and decision making occur regularly and frequently throughout a project’s timeline (Nilsson et al. 2016), suggesting that learning must be reapplied to a project at all times, not solely at set decision points. As one restoration practitioner acknowledges, decisions occur “…more by osmosis in practice, than through deliberate actions.”

This reality of learning and decision making may explain the interpretation of adaptive management as a common-sense approach without a set form. Managers must constantly be able to make decisions on a variety of topics, including monitoring, communication, and physical design. However, this must not take away from the particular importance of a well-structured decision-making process. Although Bart (1993) and others claim that too much planning can curtail creativity, our case study analysis suggests that specific planning, when applied as a strong framework for projects, to be updated as learning occurs, may to some degree address both the challenges and ambiguity that restoration practitioners in the Delta face when carrying out adaptive management.

CONCLUSION

Adaptive management is not yet a well-defined or easily implemented strategy for restoration managers in the Delta. Implementation seems to be directly and indirectly affected by challenges and convoluted by ambiguity. Our case study analysis suggests that planning should be very specific about potential management actions, decision making, scientific review, documentation, and responsibilities, to support the scientific underpinnings of an adaptive management approach. In addition, plans that are updated as projects progress and focus on how key uncertainties will be reduced may allow for more action on the ground. Increased attention to detail in planning can be seen clearly in projects in which adaptive management became more than just a legislative requirement, and was actually integral to the restoration trajectory. With a combination of changes at the project level and the governance level, it may be possible to make adaptive management more accessible to managers, increasing knowledge of restoration processes as well as chances for restoration success.

As restoration initiatives move toward large, landscape-scale projects, as in the Delta, adaptive management may provide a scientific framework that can support more complex projects and result in better ecological restoration, as also described by Simenstad et al. (2006) and Cabin (2007). Innovative solutions to the challenges encountered by restoration practitioners will help transition adaptive management from a promising theoretical concept to a workable management strategy.

LITERATURE CITED

Allen, C. R., and L. H. Gunderson. 2011. Pathology and failure in the design and implementation of adaptive management. Journal of Environmental Management 92(5):1379-1384. http://dx.doi.org/10.1016/j.jenvman.2010.10.063

Bart, C. K. 1993. Controlling new product R&D projects. R&D Management 23(3):187-197. http://dx.doi.org/10.1111/j.1467-9310.1993.tb00822.x

Buchsbaum, R. N., and C. Wigand. 2012. Adaptive management and monitoring as fundamental tools to effective salt marsh restoration. Pages 213-231 in C. T. Roman and D. M. Burdick, editors. Tidal marsh restoration: a synthesis of science and management. Island Press, Washington, D.C., USA.

Butler, W. H., A. Monroe, and S. McCaffrey. 2015. Collaborative implementation for ecological restoration on US public lands: implications for legal context, accountability, and adaptive management. Environmental Management 55(3):564-577. http://dx.doi.org/10.1007/s00267-014-0430-8

Cabin, R. J. 2007. Science-driven restoration: a square grid on a round earth? Restoration Ecology 15(1):1-7. http://dx.doi.org/10.1111/j.1526-100X.2006.00183.x

California Wetlands Monitoring Workgroup. 2015. EcoAtlas. San Francisco Estuary Institute, San Francisco, California, USA. [online] URL: http://www.ecoatlas.org

Callaway, J. C., V. T. Parker, M. C. Vasey, L. M. Schile, and E. R. Herbert. 2011. Tidal wetland restoration in San Francisco Bay: history and current issues. San Francisco Estuary and Watershed Science 9(3):1-12. [online] URL: http://escholarship.org/uc/item/5dd3n9x3

Clewell, A., J. Rieger, and J. Munro. 2005. Guidelines for developing and managing ecological restoration projects. Second edition. Society for Ecological Restoration International, Tuscon, Arizona, USA. [online] URL: http://www.ser.org/docs/default-document-library/ser_international_guidelines.pdf

Davis, S. M., and J. C. Ogden. 1994. Toward ecosystem restoration. Pages 769-796 in S. M. Davis and J. C. Ogden, editors. Everglades: the ecosystem and its restoration. St. Lucie Press, Boca Raton, Florida, USA.

Delta Stewardship Council. 2013a. The Delta Plan Appendix 3. Delta Stewardship Council, Sacramento, California, USA. [online] URL: http://www.deltacouncil.ca.gov/delta-plan-0

Delta Stewardship Council. 2013b. The Delta Science Plan. Delta Stewardship Council, Sacramento, California, USA. [online] URL: http://deltacouncil.ca.gov/sites/default/files/documents/files/Delta-Science-Plan-12-30-2013.pdf

Delta Stewardship Council. 2014. Delta restoration hub - an integrative approach to Delta restoration. Delta Stewardship Council, Sacramento, California, USA. [online] URL: http://deltacouncil.ca.gov/events/science-program-seminar/delta-restoration-hub-integrative-approach-delta-restoration

Dvir, D., T. Raz, and A. J. Shenhar. 2003. An empirical analysis of the relationship between project planning and project success. International Journal of Project Management 21(2):89-95. http://dx.doi.org/10.1016/s0263-7863(02)00012-1

Eisenhardt, K. M. 1989. Building theories from case study research. Academy of Management Review 14(4):532-550. http://dx.doi.org/10.5465/AMR.1989.4308385

Fischenich, J. C., C. Vogt, and others. 2012. The application of adaptive management to ecosystem restoration projects. ERDC TN-EMRRP-EBA-10. U.S. Army Engineer Research and Development Center, Vicksburg, Mississippi, USA. [online] URL: http://el.erdc.usace.army.mil/elpubs/pdf/eba10.pdf

Folke, C., T. Hahn, P. Olsson, and J. Norberg. 2005. Adaptive governance of social-ecological systems. Annual Review of Environment and Resources 30:441-473. http://dx.doi.org/10.1146/annurev.energy.30.050504.144511

Gregory, R., D. Ohlson, and J. Arvai. 2006. Deconstructing adaptive management: criteria for applications to environmental management. Ecological Applications 16(6):2411-2425. http://dx.doi.org/10.1890/1051-0761(2006)016[2411:damcfa]2.0.co;2

Healey, M., M. Dettinger, and R. Norgaard. 2008. The state of Bay-Delta science, 2008. CALFED Science Program, Sacramento, California, USA. [online] URL: http://www.science.calwater.ca.gov/pdf/publications/sbds/sbds_final_update_122408.pdf

Lebel, L., J. M. Anderies, B. Campbell, C. Folke, S. Hatfield-Dodds, T. P. Hughes, and J. Wilson. 2006. Governance and the capacity to manage resilience in regional social-ecological systems. Ecology and Society 11(1):19. [online] URL: http://www.ecologyandsociety.org/vol11/iss1/art19/

Lee, K. N. 1999. Appraising adaptive management. Conservation Ecology 3(2):3. [online] URL: http://www.consecol.org/vol3/iss2/art3/

Lund, J. R., and P. Moyle. 2013. Adaptive management and science for the Delta ecosystem. San Francisco Estuary and Watershed Science 11(3):1-6. [online] URL: http://escholarship.org/uc/item/1h57p2nb

McLain, R. J., and R. G. Lee. 1996. Adaptive management: promises and pitfalls. Environmental Management 20(4):437-448. http://dx.doi.org/10.1007/bf01474647

Murray, C., and D. Marmorek. 2003. Adaptive management and ecological restoration. Pages 417-428 in P. Freiderici, editor. Ecological restoration of southwestern ponderosa pine forests. Island Press, Washington, D.C., USA.

Nilsson, C., A. L. Aradottir, D. Hagen, G. Halldórsson, K. Høegh, R. J. Mitchell, K. Raulund-Rasmussen, K. Svavarsdóttir, A. Tolvanen, and S. D. Wilson. 2016. Evaluating the process of ecological restoration. Ecology and Society 21(1):41. http://dx.doi.org/10.5751/es-08289-210141

Nyberg, B. 1999. An introductory guide to adaptive management for project leaders and participants. BC Forest Service, Victoria, Canada. [online] URL: http://planet.uwc.ac.za/NISL/ESS/ESS121/Introductory-Guide-AM.pdf

Olsson, P., L. H. Gunderson, S. R. Carpenter, P. Ryan, L. Lebel, C. Folke, and C. S. Holling. 2006. Shooting the rapids: navigating transitions to adaptive governance of social-ecological systems. Ecology and Society 11(1):18. [online] URL: http://www.ecologyandsociety.org/vol11/iss1/art18/

Orr, M., S. Crooks, and P. B. Williams. 2003. Will restored tidal marshes be sustainable? San Francisco Estuary and Watershed Science 1(1):1-33. [online] URL: http://escholarship.org/uc/item/8hj3d20t

Patton, M. Q. 2002. Qualitative research and evaluation methods. Third edition. Sage, Thousand Oaks, California, USA.

Reed, D. J. 2002. Understanding tidal marsh sedimentation in the Sacramento-San Joaquin delta, California. Journal of Coastal Research Special Issue 36:605-611.

Simenstad, C., D. Reed, and M. Ford. 2006. When is restoration not?: Incorporating landscape-scale processes to restore self-sustaining ecosystems in coastal wetland restoration. Ecological Engineering 26(1):27-39. http://dx.doi.org/10.1016/j.ecoleng.2005.09.007

Walters, C. J. 2007. Is adaptive management helping to solve fisheries problems? Ambio 36(4):304-307. http://dx.doi.org/10.1579/0044-7447(2007)36[304:IAMHTS]2.0.CO;2

Whipple, A. A., R. M. Grossinger, D. Rankin, B. Stanford, and R. A. Askevold. 2012. Sacramento-San Joaquin Delta historical ecology investigation: exploring pattern and process. Prepared for the California Department of Fish and Game and Ecosystem Restoration Program. SFEI-ASC’s Historical Ecology Program Publication 672. San Francisco Estuary Institute-Aquatic Science Center, Richmond, California, USA. [online] URL: http://www.sfei.org/documents/sacramento-san-joaquin-delta-historical-ecology-investigation-exploring-pattern-and-proces

Williams, B. K. 2011. Adaptive management of natural resources—framework and issues. Journal of Environmental Management 92(5):1346-1353. http://dx.doi.org/10.1016/j.jenvman.2010.10.041

Williams, B. K., and E. D. Brown. 2012. Adaptive management: the U.S. Department of the Interior applications guide. Adaptive Management Working Group, U.S. Department of the Interior, Washington, D.C., USA. [online] URL: https://www2.usgs.gov/sdc/doc/DOI-Adaptive-Management-Applications-Guide-27.pdf

Yin, R. K. 1994. Case study research: design and methods. Sage, Thousand Oaks, California, USA.