Participatory identification and selection of ecosystem services: building on field experiences

• Abstract
• Introduction
• Methods
  • Description of case studies
  • Framing our self-evaluation
  • Self-evaluation procedure
• Results
  • Get a mandate from a locally trusted organization and organize the participatory exercise at the case study location
  • Include outsiders among the facilitator team and carefully discuss and agree on shared expectations and objectives
  • Anticipate the time load and ensure sufficient time for preparation and implementation of the participatory exercise
  • Increase participants’ engagement by gathering their input at the outset of the project and involving them in goal setting and in ecosystem service identification
  • Find a good balance in the group’s heterogeneity and provide informal time to increase exchanges
  • Have high-quality facilitators and work in small groups to help manage group discussions
  • Encourage stakeholders to explain the reasons behind their choices and discuss ecosystem services in terms of desired future
  • Seek consent not consensus
  • Opt for easily accessible methods and activities
  • Leave room for information that falls beyond ecosystem service scores and ecosystem service lists, being aware this may require new expertise
  • Use the ecosystem service concept as a boundary object, keep its limitations in mind, and carefully introduce it to participants
• Discussion
  • The support of participatory literature to participatory ecosystem service science
  • Further insights from our reflexive work
  • Opportunities and challenges for the ecosystem service concept
• Conclusion
• Responses to This Article
• Acknowledgments
• Literature Cited

INTRODUCTION

The ecosystem service (ES) concept has been increasingly advocated for inclusion in decision support tools related to natural resource management (e.g., Bryan et al. 2010, Ernstson 2013, Schaefer et al. 2015). Defined as the benefits humans obtain from nature, the ES concept clarifies how ecosystems contribute to human well-being (Reyers et al. 2013, Abson et al. 2014, Spangenberg et al. 2014). Notwithstanding this assumed potential, the ES concept is scarcely documented as being implemented in decisions (Cowling et al. 2008, Laurans et al. 2013, Förster et al. 2015, Guerry et al. 2015, Polasky et al. 2015). Only a minority of ES assessments specifically report outcomes in decision-making processes (e.g., MacDonald et al. 2014, Arkema et al. 2015, Ouyang et al. 2016). Based on the analysis of several case studies (CSs), some attempts have been made to provide a framework for conducting decision-relevant ES assessments (Nahlik et al. 2012, Rosenthal et al. 2015), share lessons learned (Ruckelshaus et al. 2015), or identify factors in ES assessments that impact decision making (Carpenter et al. 2009, Posner et al. 2016, Grêt-Regamey et al. 2017).

From this emerging and growing body of literature, some conclusions arise. All agree on the importance of including stakeholders at the outset of the ES assessment to define what kind of ES information is needed. Recent work suggests the use of “integrated ES valuation” as a conceptual framework for sustainable natural resource management. Integrated valuations combine ecological, socio-cultural, and economic valuation as tools used in a participatory way to elicit the plurality of values related to ESs, including the intrinsic and relational values that go beyond strict “benefits for humans” (Díaz et al. 2015, Kelemen et al. 2015, Pascual et al. 2017). This integrated approach explicitly aims to include multiple values and worldviews in a coherent and operational framework, aiming at societal rather than only academic impact. It requires collaboration with stakeholders in on-the-ground realities to perform quantitative or qualitative assessment of these values, to increase the effectiveness and legitimacy of decision making (Dendoncker et al. 2013, Raymond et al. 2014, Spangenberg et al. 2014). In doing this, integrated valuation inevitably deals with postnormal science issues such as power relations, science-society interfaces, and the contextual and normative framing of each valuation exercise (Jacobs et al. 2016).

Within this integrated approach, the identification and selection of ESs are critical steps that directly influence the relevance to decision making. The identification and selection of ESs occur in the first (“scoping”) phase of the valuation. They interact in an iterative process, where stakeholders (re)define the problem and information needs relevant to the context (Chan et al. 2012, Spangenberg et al. 2015). Identifying context-relevant ESs guides ES assessments toward specific natural resource management issues. As ecological processes only become ESs when someone values them or benefits from them, identifying ESs involves subjective judgments (Förster et al. 2015). To capture these judgments, it is thus critical to involve multiple knowledge sources by including stakeholders in the process of identifying and prioritizing ESs.

However, most of the time, researchers perform ES identification based on data/model availability or literature reviews, which ignores the socio-cultural context in which the project takes place (Chan et al. 2012, Malinga et al. 2013, Mascarenhas et al. 2016). This leads to blind spots of potentially important ESs and associated values, as well as bias toward other ESs or values, ignoring the diversity in ES benefits and information needs for stakeholders (Opdam et al. 2013, Kenter et al. 2015).

Participatory ES selections have been implemented within ES valuations (e.g., Bryan et al. 2010, Fontaine et al. 2014, Martínez-Sastre et al. 2017) but are rarely explicitly detailed and discussed (Malinga et al. 2013, Mascarenhas et al. 2016). Hence, scientists lack guidelines on how to carry out ES identification and selection (Burkhard et al. 2010). As the impact of selection on the relevance of valuation and decision outcomes is clear (Förster et al. 2015), there is a need for more reflexive research presenting organizational and personal learned lessons (Jacobs et al. 2016).

To address this, we evaluate the process of five participatory ES identification and selection processes that all fit within on-the-ground ES-based natural resource management projects in Belgium. We use existing literature on the evaluation of participatory research in general, not specifically embedded in ES assessments, to guide our evaluation. The bulk of the literature that addresses the evaluation of participatory research in the context of decision making is considerable as it includes several research fields. Among others, it includes research about transdisciplinary research in decision making (Klein 2008, Jahn and Keil 2015, Vilsmaier et al. 2015), participatory research in sustainability science or natural resource management (Blackstock et al. 2007, van der Wal et al. 2014, Wiek et al. 2014), public participation (Rowe and Frewer 2000, Grant and Curtis 2004), participatory planning processes (Hassenforder et al. 2016), collaborative management (Conley and Moote 2003), and participatory action research (Mackenzie et al. 2012). This literature provides a good basis to identify potentially relevant approaches to the evaluation of participatory ES identification and selection.

More specifically, we use the frameworks of Hassenforder et al. (2016) and Blackstock et al. (2007) to structure our work. These frameworks are designed to evaluate participatory planning projects and participatory research, respectively. The first is based on a comprehensive literature review and has been endorsed by other research (Triste et al. 2014, Jahn and Keil 2015), and the latter offers a detailed approach to frame the evaluation and a list of evaluation criteria based on a review of the literature.

We examine the CSs in a reflexive way, i.e., an explicit and structured self-evaluation. Reflexivity goes beyond the rigidity of checklists and evaluation criteria of normal science and acknowledges scientific uncertainties by allowing researchers to situate themselves in the research process and make them aware of the implicit assumptions and normative orientations that shape their decisions (Finlay 2002, Jacobs et al. 2016). Reflexive approaches are increasingly endorsed by the transdisciplinary and postnormal research communities (Stige et al. 2009, Jahn and Keil 2015, Popa et al. 2015). Following Funtowicz and Ravetz (1994), several authors suggest such postnormal posture is well adapted to the highly dynamic, complex, and unpredictable nature of social-ecological systems in which the management deals with uncertain facts, values in dispute, and high stakes (Wondolleck and Yaffee 2000, Regan et al. 2005, Barnaud and Antona 2014, Fontaine et al. 2014).

Our aim is thus twofold. First, in the Results, we share our experience of implementing participatory ES identification and selection. Adopting a reflexive posture, we draw recommendations from identified issues of success and barriers that facilitated or hampered effective implementation. Second, we discuss to what extent our findings corroborate existing guidelines from participatory literature. Such reflection aims to provide insights on the use of existing knowledge in participatory science in the specific case of participatory ES identification and selection. In doing so, we hope to contribute to answering the need to collect feedbacks on participatory ES identification and selection processes in a structured and reflexive way (Malinga et al. 2013, Mascarenhas et al. 2016).

METHODS

To evaluate the process of participatory ES identification and selection in our five CSs, we adopt a reflexive position structured by the frameworks of Hassenforder et al. (2016) and of Blackstock et al. (2007). These are designed for the evaluation of participatory planning projects and participatory research, respectively. As Hassenforder et al. (2016) suggest, we have structured the Methods around the following phases:

1. Description of the CSs using the descriptive variables of context, process, and outcomes.
2. Framing of the evaluation, following Blackstock et al. (2007), by delineating the objective, timing, purpose, and focus of the evaluation.
3. Description of the evaluation procedure.

To avoid confusion between terms, Box 1 presents some definitions of terms we have used.

Description of case studies

The five CSs were identified through the Belgium Ecosystems and Society community (Belgian Biodiversity Platform 2017). A more detailed presentation of the CSs is available in Appendix 1 and is summarized in Table 1. The selection criteria were (1) to be an ES-related project or research, (2) to have taken place in Belgium, and (3) to have implemented a participatory ES identification and selection that (4) followed a similar procedure (Table 2) and was (5) facilitated by researchers (“CS researchers”). The procedure followed by the five CSs detailed in Table 2 is a rather common methodology relied on for participatory ES selection. It includes an individual then a collective scoring process (Table 2) and has the advantage of being low resource demanding and easily interpretable thanks to the scoring approach. The five CSs were run independently with no or few interactions between the CS researchers. Their selection for this self-evaluation took place after they implemented the participatory ES identification and selection.

Framing our self-evaluation

A reflexive analysis is an explicit, self-aware meta-analysis (Finlay 2002) focusing on the process (Jahn and Keil 2015). As reflexive evaluation is subjective by definition (Finlay 2002), it needs to be clearly framed to be reliable, explicit, and transparent (Triste et al. 2014, Hassenforder et al. 2016). To frame our self-evaluation, we rely on the framing approach of Blackstock et al. (2007), which depicts the objective, timing, purpose, and focus of the self-evaluation:

1. The general objective of our self-evaluation is to provide a reflexive analysis of five CSs, which include participatory ES identification and selection.
2. Our self-evaluation timing fits within the Blackstock et al. (2007) category of “process evaluation” as it occurs while projects are still ongoing and focuses on the operation of the participatory exercise in order to build on strengths. Thus, we focus on how the outcome is produced rather than on the outcome itself, i.e., the selected ES for each CS.
3. Blackstock et al. (2007) identify four types of purpose for self-evaluation. We locate our self-evaluation purposes in the categories of “controlling” and “improving” as we suggest a reflection on the quality process of participatory exercises to provide guidance for future work to improve and reach their objectives.
4. The focus of a self-evaluation can either be strategic, i.e., investigates the achievement of the intended results, or operational, i.e., focuses on quality of the planned activities. The focus of our self-evaluation is operational as our aim is to provide a reflection on the process of the organization and implementation rather than on the outcomes.

Self-evaluation procedure

Our self-evaluation follows a qualitative approach based on a reflexive analysis. We are thus the evaluators and the researchers who took part in the organization and implementation of the participatory ES identification and selection (hereafter “CS researchers”). Each of the CS researchers was responsible for one of the five CSs. To guide the self-evaluation work, we organized a reflexive workshop among the CS researchers that took place after the implementation of the participatory exercises. We distinguish the “participatory exercises,” which are the participatory ES identification and selection that took place within the CSs, and the “self-evaluation workshop,” which is the evaluation workshop for the CS researchers that took place a posteriori (Box 1).

During the first step of the self-evaluation workshop, CS researchers gathered and wrote down personal experiences of success or barriers encountered during the preparation and implementation of their participatory exercise. In plenary, CS researchers explained and discussed their issues. We then mapped these onto the evaluation criteria for participatory research from the literature review of Blackstock et al. (2007) to structure the outcomes into larger clusters. In a second step, the CS researchers went through all the identified issues and assigned scores to indicate whether the issue also applied to their personal experience in their CSs: score 1 (true) or 0 (false). This last scoring provided an overview of the most frequently mentioned successes and barriers, which were then reformulated into recommendations.

RESULTS

CS researchers brought up 68 different issues (of success “+” or barriers “-”, Table 3) during the self-evaluation workshop. The issues were then mapped onto the criteria of Blackstock et al. (2007). Out of the 22 Blackstock criteria, 4 were considered redundant or nonapplicable to our CSs. The criteria framework suggested by Blackstock et al. (2007) proved to be well suited because only a minority of their criteria did not fit any of our issues. It helped us to structure our views by merging or grouping some converging issues.

The two-step procedure followed during the self-evaluation workshop distinguished between issues mentioned spontaneously and independently (Table 3, column 3) and issues acknowledged to be applicable to other cases (Table 3, column 4). Overall, a majority of positive experiences were reported (60% in step 1 and 70% in step 2). Only 30% of the issues raised are CS specific, whereas the other 70% are general issues relevant to several or all studies. This majority of experiences shared through 5 independent CSs highlight the importance of sharing lessons learned.

By reflexively identifying issues of success and barriers, we gathered 11 recommendations. The recommendations are listed and detailed subsequently. In brackets, we indicate how many of the 5 CSs are concerned in the issue discussed (also in Table 3, column 4).

Get a mandate from a locally trusted organization and organize the participatory exercise at the case study location

In our studied cases, official mandates from locally trusted organizations, e.g., farmers association (5/5); political support (4/5); or a legal context (1/5) created a trustworthy environment. “Keeping it local,” by organizing the participatory exercise at the physical context/location under discussion seemed like a significant advantage to reach and engage stakeholders (5/5).

Include outsiders among the facilitator team and carefully discuss and agree on shared expectations and objectives

CS researchers, who were also facilitators of the participatory exercise, were accompanied by outsiders to avoid facilitators guiding discussions toward the project objectives (5/5). Additionally, this brought together different areas of expertise, which improved the success rate of participatory exercises (4/5) and offered the required skills for participatory process guidance (4/5). However, in two cases, this sharing of leadership between facilitators and outsiders led to diverging initial objectives between the two parties and miscommunication (2/5).

Anticipate the time load and ensure sufficient time for preparation and implementation of the participatory exercise

“Available time” was experienced as a major limiting resource (3/5), which was either determined by the project itself, because of deadlines, financial constraints, and so forth, or by the type of participants involved, e.g., farmers are typically little available because of their work constraints. This time limit hampered the setting of commonly agreed on objectives (5/5) and sometimes a proper preparation of the participatory exercise (3/5). It can also impact the process; for instance, having to rush during the participatory exercise led to mistakes and thus decreased the credibility of CS researchers (2/5). Overall, CS researchers judged participatory exercises to bear low implementation costs (4/5).

Increase participants’ engagement by gathering their input at the outset of the project and involving them in goal setting and in ecosystem service identification

The timing of the participatory exercise with regard to the context was seen to be crucial (4/5). For instance, for CS 2 the participatory exercise took place within a broader project that had started a few years previously, which created resistance and a priori expectations regarding the participatory exercise. For this reason, gathering stakeholders’ input at the very beginning of the project seems to be a recurrent positive experience.

To avoid “stakeholder fatigue” and ensure participants’ engagement, researchers perceived it to be important that participants felt their involvement can have an impact (5/5). To do so, the goal of the participatory exercise should be relevant for the participants and society, and not only for research purposes (4/5). Involving participants at an early stage, such as in goal setting (2/5) or in identifying ESs to be selected before the prioritization and selection (5/5), was also identified to be a crucial step. In all CSs, the process of ES identification implied a combination of participants’ input and ESs proposed by CS researchers based on scientific ES classifications. Despite being acknowledged to be time consuming (5/5), it helped to make topics more recognizable to participants, and they started with a shared background and understanding.

Find a good balance in the group’s heterogeneity and provide informal time to increase exchanges

All CS researchers were satisfied by the attendance of participants, but not always by their representativeness. Some faced over- or underrepresentation of some sectors and had to adapt their methodologies accordingly (2/5). Some also found it difficult to know when this representativeness was reached (3/5). The heterogeneity of the group contributed to increased exchanges and mutual learning (5/5), yet too much heterogeneity within the group can generate polarization among participants (2/5). Adding informal time, such as a break for food and drinks, increased networking exchanges and contributed to a trusting environment (5/5).

Have high-quality facilitators and work in small groups to help manage group discussions

Including everyone and making them express their opinion can be difficult (2/5), and some “powerful” participants can potentially dominate the discussions (2/5). Having high-quality facilitators (5/5) or dividing participants into small groups (3/5) can help reduce the effect of dominant participants. If the project includes political issues, there is a risk that less room is left for trust and sympathy among participants (3/5).

Encourage stakeholders to explain the reasons behind their choices and discuss ecosystem services in terms of desired future

Instead of asking participants whether they agree or disagree, the emphasis was on asking participants to explain the reasons behind their choices to encourage understanding within the group (5/5). Two of the five CS researchers reported highly positive outcomes from suggesting that participants only formulate suggestions that benefit at least one other participant and do not affect any of the others negatively. In two CSs, it was also decided to discuss ESs in terms of a desired future. This resulted in more positive dialogue, as it is less threatening to discuss the future than present issues. On the other hand, in one CS, it was thought that focusing on desired futures bears the risk of not being translated into present actions.

Seek consent not consensus

In two CSs, “consent” was distinguished from “consensus” in the sense that the former does not seek common agreements on every detail but seeks an option for which nobody has fundamental objections. In a third CS, this was not done, but it was thought that it would have helped the debate.

Opt for easily accessible methods and activities

Overall, CS researchers declared positive outcomes from easily accessible methods and activities for participants (4/5). For example, one of the cases organized a field trip to bring participants with variable understanding of the area and the relevant issue to a more common level. Being transparent about the aims and the methods was also seen to be a major advantage (5/5). Similarly, the combination of individual votes and group discussion was judged to have added value (5/5).

Leave room for information that falls beyond ecosystem service scores and ecosystem service lists, being aware this may require new expertise

The use of numbers through ranking and scoring bears a small risk of restricting debates to numbers (1/5) but was mostly found to foster information-rich but sometimes difficult to grasp discussions (3/5). Participants suggested some values and services absent in scientific ES classifications, providing complementary and important information for the relevance of the project (5/5). This information was sometimes difficult to include further in the ES valuation because it fell beyond the expertise covered by the CS researchers. Involving new expertise was not always possible as the researchers were also dependent on external constraints, e.g., the funder’s deadline in CS 1.

Use the ecosystem service concept as a boundary object, keep its limitations in mind, and carefully introduce it to participants

Overall, the ES concept appeared to have contributed to building bridges between stakeholders, playing the role of “boundary object” to build a common language (3/5). The knowledge generated during the participatory exercise often formed a relevant basis for the project (4/5), although it was not always directly implementable (4/5; values expressed sometimes fell beyond the researchers’ expertise). Most CS researchers agreed that the participatory exercise helped to build a common ground for their ES valuation project (5/5). There was no open conflict nor strong divergences of opinion, overall consent was reached on the diversity of ES values raised during the exercise (5/5), and participants were willing to discuss and negotiate, in a constructive atmosphere of trust (3/5). This was noticed, for example, through indications of learning processes (5/5), enthusiastic engagement of some typically less engaged stakeholders (3/5), and feedbacks on the process from participants, who considered it to be a new, original way of working (4/5). Only one CS noted some disagreements, specifically with stakeholders who were not present at the participatory exercise.

Participants showed various levels of understanding of the concept and of ecosystem functioning (5/5). Working with too many ESs was sometimes confusing for participants (2/5), and some ESs appeared to be redundant to them (3/5). Additionally, the way the ES concept was introduced was found to influence participants (5/5).

DISCUSSION

We examine the 11 recommendations emanating from our self-evaluation in the light of participatory literature. Such reflection aims to provide insights on the use of the existing knowledge in participatory science in the specific case of participatory ES identification and selection.

The support of participatory literature to participatory ecosystem service science

Some of the recommendations we propose are well-known “good practices” for participatory science. Including stakeholders from the outset of the project is a recommendation repeatedly mentioned in participatory science literature (Wondolleck and Yaffee 2000, Grant and Curtis 2004, Reed 2008, de Vente et al. 2016), and well implemented by ES researchers (Baker et al. 2013, Förster et al. 2015, Rosenthal et al. 2015). Doing this guides the research project toward objectives relevant to stakeholders and society, and not only to scientific research (Grant and Curtis 2004, Mackenzie et al. 2012). This increases participants’ feeling that their engagement can have an impact (Klein 2008, Stige et al. 2009, de Vente et al. 2016). Ultimately, it improves the implementation of the research outcomes as participants in a project take ownership of its questions and results and are thus more likely to take actions and engage with the situation later on (Biggs et al. 2011, Cuéllar-Padilla and Calle-Collado 2011, Vilsmaier et al. 2015).

Our findings also concur with previous experiences that show how reliance on accessible tools enables stakeholders to actively engage in the deliberation process (Vilsmaier et al. 2015). The process should be accessible in terms of understandability and in terms of transparency (Klein 2008). In transparent processes, the way decisions are made is explicitly explained to participants, enabling a trustworthy relationship with the researchers to be built (Rowe and Frewer 2000). This recommendation is also well acknowledged by the ES scientific community (McKenzie et al. 2014, Rosenthal et al. 2015, Ruckelshaus et al. 2015, Posner et al. 2016).

Recent studies concur with our reflections that there is a need to be familiar with the context, to gain insights on what works where (Byrne 2013), producing grounded knowledge, rather than generalizable knowledge (Ashwood et al. 2014, Popa et al. 2015). Being familiar with the context helps the project to fit within a “policy window,” i.e., an opportunity for decision making, to interpret, apply, and champion the outcomes of the participatory process (Triste et al. 2014, Polasky et al. 2015, Grêt-Regamey et al. 2017). This may require mandates, facilitation, or initiation by governmental bodies. Such co-lead with an external facilitator has been suggested in previous ES work (Chan et al. 2012, Mackenzie et al. 2012, Jacobs et al. 2016). However, as shown by this previous research, and also experienced outside ES work (Mackenzie et al. 2012, de Vente et al. 2016), this bears the risk of miscommunication, diverging objectives, and a potential loss of information.

Another concern emerging from our CSs, which is also frequently expressed in the participatory literature, is the representativeness of the stakeholders involved (Rowe and Frewer 2000, Grant and Curtis 2004, de Vente et al. 2016). To fairly represent stakeholders, a large sample is required, but large groups do not function efficiently (Grant and Curtis 2004). Stakeholder analysis is believed to guide stakeholder selection toward higher representativeness (Reed et al. 2009), although generally the aim is not to reach statistical representation.

To avoid conflicting situations, two of the CS researchers suggested talking in terms of desired future, which has been reported positively in earlier work (Malinga et al. 2013, Martínez-Sastre et al. 2017). Discussions were also smoothed by asking participants to explain the reasons behind their choices, rather than just agreeing or disagreeing, a recommendation that was also formulated by Vilsmaier et al. (2015). With the same aim to facilitate group deliberation, some participatory literature has suggested the distinction between “consent” and “consensus” in the sense that the former does not seek common agreement on every detail but seeks an option for which nobody has fundamental objections (Endenburg 1998, Christian 2014). This distinction is not found in existing ES participatory recommendations, to our knowledge, although being very effective.

Finally, to apply all these recommendations, to design accessible and transparent methods, adequately select stakeholders, define commonly agreed on goals, and appropriately fit the exercise within its context, requires time, a major limiting resource as experienced in our CSs and in previous participatory work (Klein 2008, Mackenzie et al. 2012, Jahn and Keil 2015).

Further insights from our reflexive work

Our self-evaluation also led to recommendations not present in the ES participatory literature. For instance, we suggest to “keep it local,” i.e., to organize the participatory exercise in the geographic context in which the project takes place to increase participants’ feelings of legitimacy and engagement.

To decrease the chances of opposition within the group, two of the five CS researchers reported highly positive outcomes from suggesting that participants only formulate suggestions that benefit at least one other participant and do not affect any of the others negatively. In so doing, participants are encouraged to think beyond their own needs and to think about solutions beneficial to several stakeholders. This strategy has been applied outside the present work and has so far proved to be a powerful approach (Ulenaers et al. 2014). We believe this is a way to have participants aim for consent by linking self-interest with public interest. We also noticed that adding informal time, e.g., free time or a coffee break, within the exercise increases exchanges between participants and creates a trusting environment.

Most of our CSs reported relevant information emerging from the participatory exercise, but which could not always be directly implementable. Indeed, participants sometimes expressed values falling beyond the expertise covered by the researchers involved. Although similar experiences are shared in the literature (Grant and Curtis 2004, Baker et al. 2013, Chan et al. 2016, De Vreese et al. 2016), this is rarely translated into a recommendation to researchers to prepare for flexibility and adaptive postures. This is a crucial challenge, which may be hampered by institutional and academic standards (Cowling et al. 2008, Jahn and Keil 2015).

Opportunities and challenges for the ecosystem service concept

In our CSs, as in many others (Lewan and Söderqvist 2002, Baker et al. 2013, MacDonald et al. 2014, Mascarenhas et al. 2016), various levels of understanding of the concept and of ecosystem functioning were reported. In fact, the understanding of the concept depends on how it is introduced (Klein et al. 2016). It is well known that methods can influence outcomes of participatory exercises (Kenter et al. 2011, Malinga et al. 2013, Raymond et al. 2014). Hence, it is essential to bear in mind that the ES concept used as a tool to elicit values also shapes them (Martín-López et al. 2014). The mere choices of which stakeholder to include and which valuation method to use (Jacobs et al. 2018) are value laden, or “value articulating institutions” (Vatn 2005). What is more, although the concept definition is outwardly simple, people attribute various meanings to it (Nahlik et al. 2012, Flint et al. 2013, Barnaud and Antona 2014, Polasky et al. 2015), expanding the framing possibilities (Steger et al. 2018). The concept thus needs a stronger engagement with its normative foundations (Abson et al. 2014), and researchers using it must acknowledge that there is no single service-value relation, because multiple values can be held for one service and vice versa. Hence, no valuation method covers the whole range of values, and researchers need to consciously select complementary valuation methods (Jacobs et al. 2018).

On the other hand, in our cases the ES concept has proved to be an effective entry point for discussions between stakeholders, playing its role of “boundary object” (Abson et al. 2014, Steger et al. 2018). There was neither open conflict nor strong divergences, and issues were discussed constructively. This may have been because of multiple causes, i.e., contexts mainly offering opportunities for all, talking in terms of the future making discussions less threatening, and so forth, but was arguably favored by the positive discourse of ESs. The ES concept helps the understanding of dependencies on ecosystems, social relations, and conflicts of interest (Barnaud and Antona 2014, Steger et al. 2018). As illustrated by a participant in CS 5 who attested to “gain[ing] new insights about the functions of the valley by discussing them with other participants,” the ES approach increases people’s awareness of their social-ecological interdependencies and encourages collective benefits, leaving aside individual preferences.

CONCLUSION

We analyzes five CSs that included stakeholders in the identification and selection of ESs as a first step within a broader project. This reflexive analysis provided valuable insights on the common barriers or success factors, which allowed us to formulate several recommendations. We notice that many of the recommendations we have drawn concur with the wide body of existing knowledge on participatory research. We also highlight additional specific pieces of advice that are, to our knowledge, insufficiently addressed in the current literature despite having a high potential influence on the participatory process. As most of these issues raised were shared by several CS researchers, we believe these recommendations can be of interest for future work on participatory ES identification and selection as part of integrated ES valuations.

Although we recognize that there is no “one-size-fits-all solution” and that methods should be “fit-for-purposes,” we believe that feeding back experiences of participatory exercise implementation may be of great support to help future work. Our results show that reflexive analyses are valuable tools for both researchers reflecting on their own cases and for researchers willing to follow similar approaches. We hope we have opened the way to future self-evaluations of participative work to increase lessons learned and ensure future work to build on strengths. As Cowling et al. (2008:9483) state, “being mission-oriented, ES research should be stakeholder-inspired and stakeholder-useful, which will require that researchers respond to stakeholders’ needs and collaborate with them.”

LITERATURE CITED

Abson, D. J., H. von Wehrden, S. Baumgärtner, J. Fischer, J. Hanspach, W. Härdtle, H. Heinrichs, A. M. Klein, D. J. Lang, P. Martens, and D. Walmsley. 2014. Ecosystem services as a boundary object for sustainability. Ecological Economics 103:29-37. http://dx.doi.org/10.1016/j.ecolecon.2014.04.012

Arkema, K. K., G. M. Verutes, S. A. Wood, C. Clarke-Samuels, S. Rosado, M. Canto, A. Rosenthal, M. Ruckelshaus, G. Guannel, J. Toft, J. Faries, J. M. Silver, R. Griffin, and A. D. Guerry. 2015. Embedding ecosystem services in coastal planning leads to better outcomes for people and nature. Proceedings of the National Academy of Sciences of the United States of America 112(24):7390-7395. http://dx.doi.org/10.1073/pnas.1406483112

Ashwood, L., N. Harden, M. M. Bell, and W. Bland. 2014. Linked and situated: grounded knowledge. Rural Sociology 79(4):427-452. http://dx.doi.org/10.1111/ruso.12042

Baker, J., W. R. Sheate, P. Phillips, and R. Eales. 2013. Ecosystem services in environmental assessment—help or hindrance? Environmental Impact Assessment Review 40:3-13. http://dx.doi.org/10.1016/j.eiar.2012.11.004

Barnaud, C., and M. Antona. 2014. Deconstructing ecosystem services: uncertainties and controversies around a socially constructed concept. Geoforum 56:113-123. http://dx.doi.org/10.1016/j.geoforum.2014.07.003

Belgian Biodiversity Platform. 2017. Belgian Ecosystems and Society community (BEES). Belgian Biodiversity Platform, Brussels, Belgium. [online] URL: http://www.beescommunity.be/en/

Biggs, D., N. Abel, A. T. Knight, A. Leitch, A. Langston, and N. C. Ban. 2011. The implementation crises in conservation planning: could “mental models” help? Conservation Letters 4:169-183. http://dx.doi.org/10.1111/j.1755-263X.2011.00170.x

Blackstock, K. L., G. J. Kelly, and B. L. Horsey. 2007. Developing and applying a framework to evaluate participatory research for sustainability. Ecological Economics 60(4):726-742. http://dx.doi.org/10.1016/j.ecolecon.2006.05.014

Bryan, B. A., A. Grandgirard, and J. R. Ward. 2010. Quantifying and exploring strategic regional priorities for managing natural capital and ecosystem services given multiple stakeholder perspectives. Ecosystems 13(4):539-555. http://dx.doi.org/10.1007/s10021-010-9339-0

Burkhard, B., I. Petrosillo, and R. Costanza. 2010. Ecosystem services – bridging ecology, economy and social sciences. Ecological Complexity 7(3):257-259. http://dx.doi.org/10.1016/j.ecocom.2010.07.001

Byrne, D. 2013. Evaluating complex social interventions in a complex world. Evaluation 19(3):217-228. http://dx.doi.org/10.1177/1356389013495617

Carpenter, S. R., H. A. Mooney, J. Agard, D. Capistrano, R. S. DeFries, S. Díaz, T. Dietz, A. K. Duraiappah, A. Oteng-Yeboah, H. M. Pereira, C. Perrings, W. V. Reid, J. Sarukhan, R. J. Scholes, and A. White. 2009. Science for managing ecosystem services: beyond the Millennium Ecosystem Assessment. Proceedings of the National Academy of Sciences of the United States of America 106(5):1305-1312. http://dx.doi.org/10.1073/pnas.0808772106

Chan, K. M. A., P. Balvanera, K. Benessaiah, M. Chapman, S. Díaz, E. Gómez-Baggethun, R. Gould, N. Hannahs, K. Jax, S. Klain, G. W. Luck, B. Martín-López, B. Muraca, B. Norton, K. Ott, U. Pascual, T. Satterfield, M. Tadaki, J. Taggart, and N. Turner. 2016. Why protect nature? Rethinking values and the environment. Proceedings of the National Academy of Sciences of the United States of America 113(6):1462-1465. http://dx.doi.org/10.1073/pnas.1525002113

Chan, K. M. A., A. D. Guerry, P. Balvanera, S. Klain, T. Satterfield, X. Basurto, A. Bostrom, R. Chuenpagdee, R. Gould, and B. S. Halpern. 2012. Where are cultural and social in ecosystem services? A framework for constructive engagement. BioScience 62(8):744-756. http://dx.doi.org/10.1525/bio.2012.62.8.7

Christian, D. L. 2014. Consent decision-making and community vision, mission and aim: how sociocracy can help communities, part III. Communities 163:57-63.

Conley, A., and M. A. Moote. 2003. Evaluating collaborative natural resource management. Society & Natural Resources 16(5):371-386. http://dx.doi.org/10.1080/08941920309181

Cowling, R. M., B. Egoh, A. T. Knight, P. J. O’Farrell, B. Reyers, M. Rouget, D. J. Roux, A. Welz, and A. Wilhelm-Rechman. 2008. An operational model for mainstreaming ecosystem services for implementation. Proceedings of the National Academy of Sciences of the United States of America 105(28):9483-9488. http://dx.doi.org/10.1073/pnas.0706559105

Cuéllar-Padilla, M., and Á. Calle-Collado. 2011. Can we find solutions with people? Participatory action research with small organic producers in Andalusia. Journal of Rural Studies 27(4):372-383. http://dx.doi.org/10.1016/j.jrurstud.2011.08.004

Dendoncker, N., H. Keune, S. Jacobs, and E. Gómez-Baggethun. 2013. Inclusive ecosystem services valuation. Pages 3-12 in S. Jacobs, N. Dendoncker, and H. Keune, editors. Ecosystem services: global issues, local practices. Elsevier, San Diego, California, USA. http://dx.doi.org/10.1016/B978-0-12-419964-4.00001-9

de Vente, J., M. S. Reed, L. C. Stringer, S. Valente, and J. Newig. 2016. How does the context and design of participatory decision making processes affect their outcomes? Evidence from sustainable land management in global drylands. Ecology and Society 21(2):24. http://dx.doi.org/10.5751/ES-08053-210224

De Vreese, R., M. Leys, N. Dendoncker, A. Van Herzele, and C. M. Fontaine. 2016. Images of nature as a boundary object in social and integrated ecosystem services assessments. Reflections from a Belgian case study. Ecosystem Services 22:269-279. http://dx.doi.org/10.1016/j.ecoser.2016.06.008

Díaz, S., S. Demissew, C. Joly, W. M. Lonsdale, and A. Larigauderie. 2015. A Rosetta stone for nature’s benefits to people. PLoS Biology 13(1):e1002040. http://dx.doi.org/10.1371/journal.pbio.1002040

Endenburg, G. 1998. Books on Sociocracy by Gerard Endenburg. Eburon Academic, Delft, The Netherlands.

Ernstson, H. 2013. The social production of ecosystem services: a framework for studying environmental justice and ecological complexity in urbanized landscapes. Landscape and Urban Planning 109(1):7-17. http://dx.doi.org/10.1016/j.landurbplan.2012.10.005

Finlay, L. 2002. “Outing” the researcher: the provenance, process, and practice of reflexivity. Qualitative Health Research 12(4):531-545. http://dx.doi.org/10.1177/104973202129120052

Flint, C. G., I. Kunze, A. Muhar, Y. Yoshida, and M. Penker. 2013. Exploring empirical typologies of human–nature relationships and linkages to the ecosystem services concept. Landscape and Urban Planning 120:208-217. http://dx.doi.org/10.1016/j.landurbplan.2013.09.002

Fontaine, C. M., N. Dendoncker, R. De Vreese, I. Jacquemin, A. Marek, A. Van Herzele, G. Devillet, D. Mortelmans, and L. François. 2014. Towards participatory integrated valuation and modelling of ecosystem services under land-use change. Journal of Land Use Science 9(3):278-303. http://dx.doi.org/10.1080/1747423X.2013.786150

Förster, J., J. Barkmann, R. Fricke, S. Hotes, M. Kleyer, S. Kobbe, D. Kübler, C. Rumbaur, M. Siegmund-Schultze, R. Seppelt, J. Settele, J. H. Spangenberg, V. Tekken, T. Václavík, and H. Wittmer. 2015. Assessing ecosystem services for informing land-use decisions: a problem-oriented approach. Ecology and Society 20(3):31. http://dx.doi.org/10.5751/ES-07804-200331

Funtowicz, S. O., and J. R. Ravetz. 1994. The worth of a songbird: ecological economics as a post-normal science. Ecological Economics 10(3):197-207. http://dx.doi.org/10.1016/0921-8009(94)90108-2

Grant, A., and A. Curtis. 2004. Refining evaluation criteria for public participation using stakeholder perspectives of process and outcomes. Rural Society 14(2):142-162. http://dx.doi.org/10.5172/rsj.351.14.2.142

Grêt-Regamey, A., E. Sirén, S. H. Brunner, and B. Weibel. 2017. Review of decision support tools to operationalize the ecosystem services concept. Ecosystem Services 26:306-315. http://dx.doi.org/10.1016/j.ecoser.2016.10.012

Guerry, A. D., S. Polasky, J. Lubchenco, R. Chaplin-Kramer, G. C. Daily, R. Griffin, M. Ruckelshaus, I. J. Bateman, A. Duraiappah, T. Elmqvist, M. W. Feldman, C. Folke, J. Hoekstra, P. M. Kareiva, B. L. Keeler, S. Li, E. McKenzie, Z. Ouyang, B. Reyers, T. H. Ricketts, J. Rockström, H. Tallis, and B. Vira. 2015. Natural capital and ecosystem services informing decisions: from promise to practice. Proceedings of the National Academy of Sciences of the United States of America 112(24):7348-7355. http://dx.doi.org/10.1073/pnas.1503751112

Hassenforder, E., J. Pittock, O. Barreteau, K. A. Daniell, and N. Ferrand. 2016. The MEPPP framework: a framework for monitoring and evaluating participatory planning processes. Environmental Management 57(1):79-96. http://dx.doi.org/10.1007/s00267-015-0599-5

Jacobs, S., N. Dendoncker, B. Martín-López, D. N. Barton, E. Gomez-Baggethun, F. Boeraeve, F. L. McGrath, K. Vierikko, D. Geneletti, K. J. Sevecke, N. Pipart, E. Primmer, P. Mederly, S. Schmidt, A. Aragão, H. Baral, R. Bark, T. Briceno, D. Brogna, P. Cabral, R. De Vreese, C. Liquete, H. Mueller, K. S.-H. Peh, A. Phelan, A. Rincón, S. H. Rogers, F. Turkelboom, W. Van Reeth, B. T. van Zanten, H. K. Wam, and C.-L. Washbourne. 2016. A new valuation school: integrating diverse values of nature in resource and land use decisions. Ecosystem Services 22:213-220. http://dx.doi.org/10.1016/j.ecoser.2016.11.007

Jacobs, S., B. Martín-López, D. N. Barton, R. Dunford, P. A. Harrison, E. Kelemen, H. Saarikoski, M. Termansen, M. García-Llorente, E. Gómez-Baggethun, L. Kopperoinen, S. Luque, I. Palomo, J. A. Priess, G. M. Rusch, P. Tenerelli, F. Turkelboom, R. Demeyer, J. Hauck, H. Keune, and R. Smith. 2018. The means determine the end – pursuing integrated valuation in practice. Ecosystem Services 29:515-528. http://dx.doi.org/10.1016/j.ecoser.2017.07.011

Jahn, T., and F. Keil. 2015. An actor-specific guideline for quality assurance in transdisciplinary research. Futures 65:195-208. http://dx.doi.org/10.1016/j.futures.2014.10.015

Kelemen, E., D. N. Barton, S. Jacobs, and F. Turkelboom. 2015. Preliminary guidelines for integrated assessment and valuation of ecosystem services in specific policy contexts. Technical report, European Commission, Brussels, Belgium.

Kenter, J. O., T. Hyde, M. Christie, and I. Fazey. 2011. The importance of deliberation in valuing ecosystem services in developing countries – evidence from the Solomon Islands. Global Environmental Change 21(2):505-521. http://dx.doi.org/10.1016/j.gloenvcha.2011.01.001

Kenter, J. O., L. O’Brien, N. Hockley, N. Ravenscroft, I. Fazey, K. N. Irvine, M. S. Reed, M. Christie, E. Brady, R. Bryce, A. Church, N. Cooper, A. Davies, A. Evely, M. Everard, R. Fish, J. A. Fisher, N. Jobstvogt, C. Molloy, J. Orchard-Webb, S. Ranger, M. Ryan, V. Watson, and S. Williams. 2015. What are shared and social values of ecosystems? Ecological Economics 111:86-99. http://dx.doi.org/10.1016/j.ecolecon.2015.01.006

Klein, J. T. 2008. Evaluation of interdisciplinary and transdisciplinary research. American Journal of Preventive Medicine 35(2):S116-S123. http://dx.doi.org/10.1016/j.amepre.2008.05.010

Klein, T. M., T. Drobnik, and A. Grêt-Regamey. 2016. Shedding light on the usability of ecosystem services–based decision support systems: an eye-tracking study linked to the cognitive probing approach. Ecosystem Services 19:65-86. http://dx.doi.org/10.1016/j.ecoser.2016.04.002

Laurans, Y., A. Rankovic, R. Billé, R. Pirard, and L. Mermet. 2013. Use of ecosystem services economic valuation for decision making: questioning a literature blindspot. Journal of Environmental Management 119:208-219. http://dx.doi.org/10.1016/j.jenvman.2013.01.008

Lewan, L., and T. Söderqvist. 2002. Knowledge and recognition of ecosystem services among the general public in a drainage basin in Scania, Southern Sweden. Ecological Economics 42(3):459-467. http://dx.doi.org/10.1016/S0921-8009(02)00127-1

MacDonald, D. H., R. H. Bark, and A. Coggan. 2014. Is ecosystem service research used by decision-makers? A case study of the Murray-Darling Basin, Australia. Landscape Ecology 29(8):1447-1460. http://dx.doi.org/10.1007/s10980-014-0021-3

Mackenzie, J., P.-L. Tan, S. Hoverman, and C. Baldwin. 2012. The value and limitations of Participatory Action Research methodology. Journal of Hydrology 474:11-21. http://dx.doi.org/10.1016/j.jhydrol.2012.09.008

Malinga, R., L. J. Gordon, R. Lindborg, and G. Jewitt. 2013. Using participatory scenario planning to identify ecosystem services in changing landscapes. Ecology and Society 18(4):10. http://dx.doi.org/10.5751/ES-05494-180410

Martínez-Sastre, R., F. Ravera, J. A. González, C. López Santiago, I. Bidegain, and G. Munda. 2017. Mediterranean landscapes under change: combining social multicriteria evaluation and the ecosystem services framework for land use planning. Land Use Policy 67:472-486. http://dx.doi.org/10.1016/j.landusepol.2017.06.001

Martín-López, B., E. Gómez-Baggethun, M. García-Llorente, and C. Montes. 2014. Trade-offs across value-domains in ecosystem services assessment. Ecological Indicators 37:220-228. http://dx.doi.org/10.1016/j.ecolind.2013.03.003

Mascarenhas, A., T. B. Ramos, D. Haase, and R. Santos. 2016. Participatory selection of ecosystem services for spatial planning: insights from the Lisbon Metropolitan Area, Portugal. Ecosystem Services 18:87-99. http://dx.doi.org/10.1016/j.ecoser.2016.02.011

McKenzie, E., S. Posner, P. Tillmann, J. R. Bernhardt, K. Howard, and A. Rosenthal. 2014. Understanding the use of ecosystem service knowledge in decision making: lessons from international experiences of spatial planning. Environment and Planning C: Government and Policy 32(2):320-340. http://dx.doi.org/10.1068/c12292j

Nahlik, A. M., M. E. Kentula, M. S. Fennessy, and D. H. Landers. 2012. Where is the consensus? A proposed foundation for moving ecosystem service concepts into practice. Ecological Economics 77:27-35. http://dx.doi.org/10.1016/j.ecolecon.2012.01.001

Opdam, P., J. I. Nassauer, Z. Wang, C. Albert, G. Bentrup, J.-C. Castella, C. McAlpine, J. Liu, S. Sheppard, and S. Swaffield. 2013. Science for action at the local landscape scale. Landscape Ecology 28(8):1439-1445. http://dx.doi.org/10.1007/s10980-013-9925-6

Ouyang, Z., H. Zheng, Y. Xiao, S. Polasky, J. Liu, W. Xu, Q. Wang, L. Zhang, Y. Xiao, E. Rao, L. Jiang, F. Lu, X. Wang, G. Yang, S. Gong, B. Wu, Y. Zeng, W. Yang, and G. C. Daily. 2016. Improvements in ecosystem services from investments in natural capital. Science 352(6292):1455-1459. http://dx.doi.org/10.1126/science.aaf2295

Pascual, U., P. Balvanera, S. Díaz, G. Pataki, E. Roth, M. Stenseke, R. T. Watson, E. B. Dessane, M. Islar, E. Kelemen, V. Maris, M. Quaas, S. M. Subramanian, H. Wittmer, A. Adlan, S. Ahn, Y. S. Al-Hafedh, E. Amankwah, S. T. Asah, P. Berry, A. Bilgin, S. J. Breslow, C. Bullock, D. Cáceres, H. Daly-Hassen, E. Figueroa, C. D. Golden, E. Gómez-Baggethun, D. González-Jiménez, J. Houdet, H. Keune, R. Kumar, K. Ma, P. H. May, A. Mead, P. O’Farrell, R. Pandit, W. Pengue, R. Pichis-Madruga, F. Popa, S. Preston, D. Pacheco-Balanza, H. Saarikoski, B. B. Strassburg, M. van den Belt, M. Verma, F. Wickson, and N. Yagi. 2017. Valuing nature’s contributions to people: the IPBES approach. Current Opinion in Environmental Sustainability 26-27:7-16. http://dx.doi.org/10.1016/j.cosust.2016.12.006

Polasky, S., H. Tallis, and B. Reyers. 2015. Setting the bar: standards for ecosystem services. Proceedings of the National Academy of Sciences of the United States of America 112(24):7356-7361. http://dx.doi.org/10.1073/pnas.1406490112

Popa, F., M. Guillermin, and T. Dedeurwaerdere. 2015. A pragmatist approach to transdisciplinarity in sustainability research: from complex systems theory to reflexive science. Futures 65:45-56. http://dx.doi.org/10.1016/j.futures.2014.02.002

Posner, S. M., E. McKenzie, and T. H. Ricketts. 2016. Policy impacts of ecosystem services knowledge. Proceedings of the National Academy of Sciences of the United States of America 113(7):1760-1765. http://dx.doi.org/10.1073/pnas.1502452113

Raymond, C. M., J. O. Kenter, T. Plieninger, N. J. Turner, and K. A. Alexander. 2014. Comparing instrumental and deliberative paradigms underpinning the assessment of social values for cultural ecosystem services. Ecological Economics 107:145-156. http://dx.doi.org/10.1016/j.ecolecon.2014.07.033

Reed, M. S. 2008. Stakeholder participation for environmental management: a literature review. Biological Conservation 141(10):2417-2431. http://dx.doi.org/10.1016/j.biocon.2008.07.014

Reed, M. S., A. Graves, N. Dandy, H. Posthumus, K. Hubacek, J. Morris, C. Prell, C. H. Quinn, and L. C. Stringer. 2009. Who’s in and why? A typology of stakeholder analysis methods for natural resource management. Journal of Environmental Management 90(5):1933-1949. http://dx.doi.org/10.1016/j.jenvman.2009.01.001

Regan, H. M., Y. Ben-Haim, B. Langford, W. G. Wilson, P. Lundberg, S. J. Andelman, and M. A. Burgman. 2005. Robust decision-making under severe uncertainty for conservation management. Ecological Applications 15(4):1471-1477. http://dx.doi.org/10.1890/03-5419

Reyers, B., R. Biggs, G. S. Cumming, T. Elmqvist, A. P. Hejnowicz, and S. Polasky. 2013. Getting the measure of ecosystem services: a social-ecological approach. Frontiers in Ecology and the Environment 11(5):268-273. http://dx.doi.org/10.1890/120144

Rosenthal, A., G. Verutes, E. McKenzie, K. K. Arkema, N. Bhagabati, L. L. Bremer, N. Olwero, and A. L. Vogl. 2015. Process matters: a framework for conducting decision-relevant assessments of ecosystem services. International Journal of Biodiversity Science, Ecosystem Services & Management 11(3):190-204. http://dx.doi.org/10.1080/21513732.2014.966149

Rowe, G., and L. J. Frewer. 2000. Public participation methods: a framework for evaluation. Science, Technology & Human Values 25(1):3-29. http://dx.doi.org/10.1177/016224390002500101

Ruckelshaus, M., E. McKenzie, H. Tallis, A. Guerry, G. Daily, P. Kareiva, S. Polasky, T. Ricketts, N. Bhagabati, S. A. Wood, and J. Bernhardt. 2015. Notes from the field: lessons learned from using ecosystem service approaches to inform real-world decisions. Ecological Economics 115:11-21. http://dx.doi.org/10.1016/j.ecolecon.2013.07.009

Schaefer, M., E. Goldman, A. M. Bartuska, A. Sutton-Grier, and J. Lubchenco. 2015. Nature as capital: advancing and incorporating ecosystem services in United States federal policies and programs. Proceedings of the National Academy of Sciences of the United States of America 112(24):7383-7389. http://dx.doi.org/10.1073/pnas.1420500112

Spangenberg, J. H., C. Görg, and J. Settele. 2015. Stakeholder involvement in ESS research and governance: between conceptual ambition and practical experiences – risks, challenges and tested tools. Ecosystem Services 16:201-211. http://dx.doi.org/10.1016/j.ecoser.2015.10.006

Spangenberg, J. H., C. von Haaren, and J. Settele. 2014. The ecosystem service cascade: further developing the metaphor. Integrating societal processes to accommodate social processes and planning, and the case of bioenergy. Ecological Economics 104:22-32. http://dx.doi.org/10.1016/j.ecolecon.2014.04.025

Steger, C., S. Hirsch, C. Evers, B. Branoff, M. Petrova, M. Nielsen-Pincus, C. Wardropper, and C. J. van Riper. 2018. Ecosystem services as boundary objects for transdisciplinary collaboration. Ecological Economics 143:153-160. http://dx.doi.org/10.1016/j.ecolecon.2017.07.016

Stige, B., K. Malterud, and T. Midtgarden. 2009. Toward an agenda for evaluation of qualitative research. Qualitative Health Research 19(10):1504-1516. http://dx.doi.org/10.1177/1049732309348501

Triste, L., F. Marchand, L. Debruyne, M. Meul, and L. Lauwers. 2014. Reflection on the development process of a sustainability assessment tool: learning from a Flemish case. Ecology and Society 19(3):47. http://dx.doi.org/10.5751/ES-06789-190347

Ulenaers P., F. Turkelboom, I. Simoens, H. Keune, H. Deneef, and K. Stevens. 2014. Participatieve gebiedsvisieontwikkeling voor De Wijers via het ecosysteemdienstenconcept - Procesarchitectuur en procesevaluatie. Rapporten van het Instituut voor Natuur- en Bosonderzoek 2014 (INBO.R.2014.2853501). Instituut voor Natuur- en Bosonderzoek, Brussels, Belgium.

van der Wal, M., J. De Kraker, A. Offermans, C. Kroeze, P. A. Kirschner, and M. van Ittersum. 2014. Measuring social learning in participatory approaches to natural resource management. Environmental Policy and Governance 24(1):1-15. http://dx.doi.org/10.1002/eet.1627

Vatn, A. 2005. Institutions and the environment. Edward Elgar, Cheltenham, United Kingdom.

Vilsmaier, U., M. Engbers, P. Luthardt, R. M. Maas-Deipenbrock, S. Wunderlich, and R. W. Scholz. 2015. Case-based mutual learning sessions: knowledge integration and transfer in transdisciplinary processes. Sustainability Science 10(4):563-580. http://dx.doi.org/10.1007/s11625-015-0335-3

Wiek, A., S. Talwar, M. O’Shea, and J. Robinson. 2014. Toward a methodological scheme for capturing societal effects of participatory sustainability research. Research Evaluation 23(2):117-132. http://dx.doi.org/10.1093/reseval/rvt031

Wondolleck, J. M., and S. L. Yaffee. 2000. Making collaboration work: lessons from innovation in natural resource management. Island, Washington, D.C., USA.