Copyright © 2014 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf
version of this article
The following is the established format for referencing this article:
Priess, J. A., and J. Hauck. 2014. Integrative scenario development. Ecology and Society 19(1): 12.
Integrative Scenario Development
1Helmholtz Centre for Environmental Research - UFZ, Department Computational Landscape Ecology, 2Helmholtz Centre for Environmental Research - UFZ, Department Environmental Policy
Scenarios are employed to address a large number of future environmental and socioeconomic challenges. We present a conceptual framework for the development of scenarios to integrate the objectives of different stakeholder groups. Based on the framework, land-use scenarios were developed to provide a common base for further research. At the same time, these scenarios assisted regional stakeholders to bring forward their concerns and arrive at a shared understanding of challenges between scientific and regional stakeholders, which allowed them to eventually support regional decision making. The focus on the integration of views and knowledge domains of different stakeholder groups, such as scientists and practitioners, required rigorous and repeated measures of quality control. The application of the integrative concept provided products for both stakeholder groups, and the process of scenario development facilitated cooperation and learning within both the scientist and practitioner groups as well as between the two groups.
Key words: interdisciplinary research; participatory process; quality control; regional scale; transdisciplinary research
Scientific literature about the uses and benefits of scenarios reveals that scenarios are applied to address a large number of current and future challenges. In science, they are often used to capture complexity, to understand uncertainties, to assess the impact and interactions of drivers of change, or to test alternative development trajectories. Other scientific uses are the integration of current thinking regarding future changes, and the establishment of scientifically based consensus (Acreman 2005, Kok et al. 2006, 2011, Biggs et al. 2007, Walz et al. 2007, Zurek and Henrichs 2007, Liu et al. 2008, Mahmoud et al. 2009).
Scenarios can be used to scientifically inform decisions in situations in which the problem is already defined beforehand (Kaljonen et al. 2012). To this end, scenarios can be used to assess ex ante the consequences of different policies or strategies (Giljum et al. 2008, Waldhardt et al. 2010, Palomo et al. 2011) and to make decisions that are robust under a wide range of possible futures (Masini 2000, Harries 2003, EEA 2009). In these examples, scenarios are used to inform either scientists or decision makers.
Scenarios can serve other purposes in which the development process is important, rather than the scenario as a product. Scenarios can be used as a method to stimulate creative debate and causal thinking, for example, to analyze interrelated problems and solutions in a creative environment and to facilitate the exchange of knowledge (Harries 2003, Toderi et al. 2007, Henrichs et al. 2010, Haasnoot and Middelkoop 2012, Johnson et al. 2012). Further, Kahane (2007) and Johnson et al. (2012) emphasized the motivation of organizational learning, the achievement of consensus of goals, and a shared understanding of challenges.
We introduce a conceptual framework that uses a participatory scenario development process to integrate both the benefits derived from the scenario development process as well as from the products. The framework was developed as a contribution to the research program “Land Use Options – Strategies and Adaptation to Global Change” (Seppelt et al. 2009). The application of the integrative scenario framework is seen as a contribution to the final objective of the research program, which was to develop strategies for sustainable land use on a regional level.
Thus one major task, in applying the framework, was the interdisciplinary integration, i.e., the integration of a wide range of scientific disciplines, perspectives, methods, and land-use-related thematic foci, including the analyses of the impact of land use on biodiversity, the impact of different policies and regulations on land use, and the impact bioenergy production has on land conversion (stakeholder group 1: scientists). Because the envisaged strategies for sustainable land use should not only be scientifically sound, but also be useful and applicable for practitioners, the second major task was to enable regional stakeholders (stakeholder group 2: practitioners) to contribute their knowledge and bring forward their concerns to arrive at a shared understanding of challenges between the two different stakeholder groups. This kind of knowledge integration is seen as essential to ensure salience and legitimacy of the scenarios (Alcamo and Henrichs 2008) and thus achieve the benefits listed above.
A number of conceptual scenario frameworks were already available, for example, by van Notten et al. (2003), Biggs et al. (2007), and Bishop et al. (2007). Among the most prominent frameworks are those presented by Alcamo (2001) and Henrichs et al. (2010). Although these two are the starting points for our conceptual framework, we found several augmentations necessary to enable the interdisciplinary integration as well as the salience and legitimacy of scenarios for all stakeholder groups. Biggs et al. (2007), for example, discussed four similar key aspects, i.e., quantification, level of detail, stakeholder involvement, and communication with users/stakeholders, mainly in the context of the roles they played at different spatial scales. These extensions were deemed necessary because existing frameworks usually focused on decision support, and they paid less attention to the interdisciplinary integration, or the other way around.
Conceptual framework for integrative scenario development
The three components of the scenario framework are: (1) stakeholder participation, (2) knowledge integration, and (3) quality control, which we consider prerequisites for the development of integrative scenarios that assist stakeholders in bringing forward their concerns, ensure a shared understanding of challenges between different stakeholder groups, and provide a common knowledge base to support regional decision making.
Component 1: stakeholder participation
To achieve the benefits associated with the scenario development process, such as joint learning and the development of consensus about goals, or to improve the ability to deal with uncertainty in decision making, it is necessary to involve the relevant stakeholder groups in the development process (Kahane 2007, Henrichs et al. 2010, Johnson et al. 2012). Furthermore, the participation of stakeholders ensures that their issues are represented in the scenario exercise, thus increasing the salience and the legitimacy of the scenarios (Alcamo and Henrichs 2008). A lack of legitimacy of scenarios, e.g., when they only promote a particular set of beliefs or values, likely causes the scenario outcomes to be ignored by decision makers. Thus, it is not only necessary to include stakeholders but to include a wide enough range of stakeholders. In the literature, several methods are described regarding whom to involve in scenario exercises and how to proceed (Patel et al. 2007, Cuppen et al. 2010). If, because of the spatial and/or thematic coverage of scenarios, an unmanageable number of individual stakeholders are to be involved, Mostert (2003) pragmatically suggested involving mainly representatives of ‘organized’ groups, such as companies, NGOs, state or communal agencies, etc., ensuring balanced representation between different regions or states or between governmental and nongovernmental organizations.
For the design of integrative scenario development processes, a good number of methods are available, such as ex ante surveys, workshops, expert discussions/interviews, and ex post evaluations.
Ex ante quantitative surveys involving all stakeholder groups can serve to first assess the potential variety and interests of stakeholder groups. Surveys can serve to identify the research questions, relevant drivers/uncertainties, and/or their assumed rates of change. The surveys can be considered a simpler and faster form of Delphi processes, which have repeatedly been used during scenario development (Bernarie 1988, Kosow and Gaßner 2008).
To realize the potentials associated with scenario processes, such as fostering interdisciplinary and social learning, scenario approaches need to facilitate inter- and transdisciplinary communication (Winowiecki et al. 2011). Workshops are considered a necessary component to enable the exchange of opinions, knowledge sharing, and consensus building. This is achieved by inviting stakeholders to jointly develop assumptions for scenario storylines, which build the base for narrative scenarios. Key axes of uncertainties may be suggested by the developer team or be developed during workshops.
When time or resources limit the number and/or the duration of workshops, expert discussions/interviews serve as an opportunity to address the outcomes of workshop evaluations. Experts may also contribute by addressing knowledge gaps or quantifying drivers.
Ex post evaluation surveys offer the possibility of identifying positive elements, as well as organizational or thematic deficits from workshops or other components of a scenario process.
Component 2: knowledge integration
The inclusion of multiple stakeholder groups in the scenario development process alone does neither ensure the integration of knowledge from different disciplines or sectors nor the integration of scientific and other forms of knowledge. In the following section, several approaches on how to integrate knowledge are discussed.
A multiscale scenario approach is one more way to integrate knowledge from stakeholders, researchers, and decision makers from different geographical settings (Biggs et al. 2007, Zurek and Henrichs et al. 2007). A wide range of global scenarios have been developed either by scientists or by scientists together with stakeholders and decision makers for environmental assessments in different application domains. They are often used as boundary conditions for regional environmental change assessments, in which regional narratives are interpreted from global storylines (Rounsevell et al. 2006, Rounsevell and Metzger 2010, Kaljonen et al. 2012). According to Messner (2007), this top-down approach ensures that all important external change processes or drivers, which are relevant at the regional level, are included. Several procedures of downscaling or contextualizing global scenarios, e.g., to a subnational level, have been described (Zurek and Henrichs 2007, Alcamo and Henrichs 2008, Metzger et al. 2010, Kaljonen et al. 2012). A frequently used method is to include regional stakeholders in the contextualization of global scenarios. This, however, may negatively affect internal consistency (Rounsevell and Metzger 2010). The problem occurs because participating stakeholders may not always have a sufficiently complete mental model of the system that is being described in a storyline. Therefore, their input, although creative, might be perceived as inconsistent and consequently be rejected by scientists. However, it is well known that actual regional processes might deviate from the ones downscaled from national or global levels, thus perhaps requiring the consideration of bottom-up approaches for scenario development. Zurek and Henrichs (2007) discuss a number of possibilities to link scenarios across scales and still ensure internal consistency.
A more methodological way of integrating knowledge from different disciplines, including scientific and other forms of knowledge, is the development of scenario storylines in which the assumptions of multiple stakeholders are consolidated as narratives.
Like Alcamo (2001), Biggs et al. (2007) emphasized the strong focus on quantification and numerical modeling mostly in large scale/global environmental assessments, e.g., the the fourth Global Environmental Outlook (GEO-4), IPCC, or Millennium Ecosystem Assessment (MEA) scenarios, but also at smaller scales (de Nijs et al. 2004, Walz et al. 2007). We consider these aspects to be optional (Bohunovsky et al. 2011) depending on the objectives of scenario developers and users who may prefer qualitative or semiquantitative scenario approaches (Kok 2009, Henrichs et al. 2010). However, quantification might be a possibility for knowledge integration, for instance as part of down- or up-scaling processes.
Alternatively, Kok (2009) suggested the use of fuzzy cognitive maps, e.g., to avoid mismatches between scenario-narratives and models or to support quantification. The application of (semi-) quantitative methods may occur repeatedly in an iterative review process. At least one round of revision is considered necessary to ensure quality control, e.g., with respect to the consistency of numerical assumptions of different drivers. Another alternative for integration was proposed by Walz et al. (2007), who handled the identification of regional uncertainties by establishing ‘satellite groups’ and applying methods of systems analysis to organize the integration of different stakeholder groups.
Component 3: quality control
Building on quality criteria for scenario exercises (Alcamo and Henrichs 2008, Alcamo et al. 2008), this component comprises various means of review and feedback to ensure rigorous quality control in terms of the consistency of assumptions, scientific credibility, and transparency throughout the scenario exercise.
One approach is to appoint a review panel of scientists from multiple disciplines during the initial phase of the scenario exercise. We adopted this idea of a ‘scenario-panel’ (Alcamo 2001), but in regional studies it is feasible to extend its tasks to function as a review panel and provide guidance and repeated revisions of the scenario storylines, for example. Contrasting with global scale studies, regional studies facilitate repeated panel meetings. We propose three to five meetings depending on progress, the amount of conflicting issues, and open questions. The iterative process with the panel is intended to last throughout the entire process and is considered a key element of quality control ensuring the consistency of assumptions, drivers, as well as staying in line with targeted objectives of the scenario exercise.
Additional elements of quality control to increase transparency and salience for stakeholders can be included:
- evaluation of stakeholder workshops via ex post questionnaires to identify organizational or thematic problems, lack of clarity with respect to methods, term definitions, and missing elements, etc.;
- editing of the storylines by stakeholders to ensure that stakeholder views and inputs are adequately addressed;
- review of the final storylines by external laypeople and editors to ensure clarity and readability of the storylines.
Implementation of the conceptual framework in Central Germany
The conceptual framework was developed to facilitate collaboration and knowledge integration of 20 different research groups involved in the research program “Land Use Options – Strategies and Adaptation to Global Change” (Seppelt et al. 2009) in their effort to jointly develop strategies for sustainable land use for Central Germany. However, research on the uptake of scientific findings for decision making shows that the uptake increases when decision makers and other stakeholders are involved in research (Alcamo and Henrichs 2008). Thus, it was decided to go beyond interdisciplinary collaboration and integrate regional stakeholders and decision makers in an implementing process of the conceptual framework. The implementation process is summarized in Figure 1.
Although the implementation process was mainly driven by the authors (hereafter called the scenario developers/team), the team was supported and reviewed by a review panel, as outlined in component 3. The interdisciplinary panel, i.e., political and social science, law and economics, ecology and biology, climatology, and geography, was established in the beginning of the scenario exercise and mainly consisted of scientists involved in the research program. As shown in the right column, Figure 1, the panel was consulted repeatedly and reviewed the activities of the scenario team.
As a second step during scenario development, a survey was conducted inquiring which drivers and indicators were considered relevant by the scientific stakeholders, namely the 20 research groups of the research program “Land Use Options – Strategies and Adaptation to Global Change.” Drivers considered relevant by the scientists were mainly related to large or global scales, whereas drivers mentioned with decreasing frequency were climate, land use, economy, and trade. In consensus with the review panel it was thus decided to embed the regional scenarios in large-scale scenarios.
The global GEO-4 scenarios of the United Nations Environment Program (UNEP 2007) were selected to represent the boundary conditions of the regional scenarios for several reasons. At the time of selection, they were the most recent set of scientifically accepted global scenarios with a set of key uncertainties and drivers strongly overlapping with the drivers addressed by the scientists, as the first stakeholder group. Furthermore, in the GEO-4 scenarios, many quantified assumptions are provided at country scale, facilitating the intended downscaling. Scenario development aimed at the 2 x 2 design along two axes of key uncertainties, successfully implemented in the IPCC, GEO-4, and other scenario approaches. The 2 x 2 design was also chosen to avoid the effect that scientific or regional stakeholders focused on a ‘business-as-usual’ scenario implicitly assigning a higher probability of occurrence because of the similarity to present conditions (Tress and Tress 2003). Based on the survey of drivers of the first stakeholder group, it was envisaged to use one axis for the large scale/global drivers of this group and reserve the second axis for the expected focus on regional drivers of the regional practitioners. The scenario team suggested the scenarios “Markets First” and “Sustainability First,” later named “Radical Market Forces” (RaMa) and “Sustainable & Citizen Friendly” (NaBü), respectively, in the regional scenario process. Boundary conditions were derived in three different forms. First, in qualitative form, from the GEO-4 Report, chapter 9 (p. 400-454); second, in semiquantitative form, from the figures provided in the same chapter. Third, in quantitative form derived from the detailed output tables provided by one of the GEO-4 models, the International Futures (IFs) model (Hughes 2009). The top-down approach was supplemented by bottom-up approaches during the stakeholder workshops described in Step 3 and 4, to account for stakeholder views and region-specific developments not present in the global scenarios (Palomo et al. 2011).
Based on the surveys, the boundary conditions and axes were suggested by the scenario team and the review panel. Decisions about relevant drivers, i.e., factors and uncertainties that cause changes, were part of the participatory process. This process further included the development of the retrospective storylines describing society, economy, and land use in the final year 2050, as well as a stepwise description of changes, i.e., events and developments leading to the status in the end year 2050.
The process started with two half-day workshops with scientific stakeholders from the 20 research groups. After introducing the scenario concept and the envisaged approach, the participants were asked to formulate assumptions on how land use in Central Germany could look like in 2050, and which processes would lead to this state, assuming the boundary conditions of the two GEO-4 scenarios “Markets First” and “Sustainability First.” The assumptions and processes were discussed among the scientists to reach a consensus.
After the workshops, the scenario team converted the assumptions into storylines, which were then revised by the review panel. The revision was repeated twice with refined assumptions and storylines.
Taking the most frequently mentioned driver, climate change, into account, a minimum temporal coverage until 2050 was uniformly assumed necessary to capture climate change effects. Land-use change was mentioned as the second most important driver, with some research groups specifying the agricultural sector, namely food and bioenergy production as the sectors in which the largest changes or uncertainties were to be expected. Scientists anticipated impacts on environmental and socioeconomic factors, most frequently mentioning ‘ecosystem services,’ followed by different ‘socioeconomic indicators’ as the means to assess climate and land-use change related impacts. The spatial extent of the scenarios was more controversial, reflecting the variety of locations and scales of research. Scientists and the developer team agreed on a region including most key research sites, covering the catchments of the Saale and Mulde rivers in Central Germany (approx. 30,000 km²). During the discussions it turned out that scientific stakeholders were often hesitant to phrase assumptions in thematic fields beyond their expertise, e.g., during discussions of changes in infrastructure, transportation, or urban and rural lifestyles. It was repeatedly argued that additional more ‘practical’ expertise from public administration or regional planning departments would be needed to make the scenario assumptions more concrete and to check their plausibility.
In a fourth step, we started to involve practitioners from Central Germany. During the stakeholder analysis we identified three different land-use-related groups:
- influencing federal, state, or regional land-use decisions, e.g., via land use and landscape planning from the local to the federal state level;
- depending on the resource land, e.g., farmers’ associations, bioenergy associations;
- involved in research about (sustainable) land use or environmental protection, e.g., research institutions, environmental conservationists.
About 160 representatives from these groups were invited from the 3 federal states, Saxony, Saxony-Anhalt, and Thuringia (see results section). From the invited representatives, 25 stakeholders participated in the workshop. Participants were evenly distributed between the 3 groups mentioned above, and the federal states.
Before the workshop, all registered participants received questionnaires to assess: (1) their area of work, e.g., regional planning, environmental conservation, agriculture; (2) the goals of their organizations concerning land use; and (3) their views of regional uncertainties and problems related to future land-use change. The answers reflected the perceived most important regional drivers of change. In addition the answers also revealed the perceived large uncertainties related to these drivers, expressed as widely varying expected rates of change. Strongly contrasting the driver selection of the first stakeholder group, the ‘decrease of the human population’ was considered the most important driver by the second stakeholder group. The second driver, ‘increase in renewable energy production,’ could be considered a more specific version of the driver ‘land-use change’ addressed by the scientists. Interestingly, many of the respondents pointed out that they assumed the expansion of renewable energies would have profound consequences on land use. However, their assumptions on how these changes would evolve differed considerably, reflecting the uncertainties, e.g., of how much bioenergy would be produced, and how much land would be needed for this purpose. Similar differences occurred in the assumptions about the expected regional population changes, which on one hand were uniformly assumed to be negative, but on the other hand covered a wide range from -10% to -50% until 2050.
Based on the widely differing assumptions about the rates of change to be expected, the scenario team suggested using the second key uncertainty axis reserved for this stakeholder group to split the “Radical Market Forces” and the ”Sustainable & Citizen Friendly” scenarios into moderate and extreme pathways of change. Thus using slow versus fast rates of change for the second key uncertainty axis. The procedure was approved by the participants, who then volunteered for the four different scenario groups to discuss and develop the assumptions for the four storylines, focusing on the three prioritized regional drivers, i.e., regional population changes, renewable energy, and scarcity of natural resources, and the three land-use types, i.e., settlements, organic agriculture, and protected areas.
The assumptions about climate change as an important, large-scale driver made by the scientific stakeholders was not questioned or challenged by the practitioners. When addressed by the moderators in the four discussion groups, the participants either argued that the issue of climate change was beyond their expertise or that it was just not considered an urgent problem in the region. This was also reflected in the fact that climate change was not among the 10 most important regional drivers. With respect to the temporal extent of the scenarios, most practitioners argued that the next one or two decades would be much more relevant for them than the period beyond 2030 until 2050, whereas some participants and the developer team argued that a period until 2050 would be needed to address climate change effects. Integrating both views was achieved by covering the period until 2050, while providing sufficient detail for the period up to 2020/30, which was more relevant to practitioners. Contrastingly, later discussions with forestry experts of the Saxonian State Forestry Service revealed their need to consider much longer periods, preferably ~100 years or more. With respect to the spatial extent of the scenarios, stakeholders of state agencies and NGOs strongly argued for the use of state boundaries instead of watersheds, as suggested by the scientists in step 3, to ensure the applicability of results at the state level. Taking these preferences into account, stakeholders and developers agreed that the scenarios should be developed for the entire Central Germany, comprising the states of Saxony, Saxony-Anhalt, and Thuringia (~55,000 km²), including the watersheds (~30,000 km²) preferred by the scientific stakeholders.
Following Patel et al. (2007), two weeks after the workshop, an ex post questionnaire was sent to the participants to gather feedback concerning the organization of the workshop, the composition of the workshop participants, as well as their expectations concerning the results and the follow-up process.
The input of the stakeholder groups was used to develop four storylines, ”Radical Market Forces” (RaMa) with moderate and extreme rate of change and ”Sustainable & Citizen Friendly” (NaBü) with moderate and extreme rates of change. Based on the gap analysis from the ex post questionnaires, the storylines were complemented with literature reviews and expert discussions/interviews. Next, the storylines were revised by the review panel and a number of workshop participants involved in step 4.
Some participants of both stakeholder groups voiced their preference for quantifications of scenario assumptions to make the scenarios more useful and applicable in planning processes and research, e.g., for simulation studies using numerical models. Thus, the scenario team motivated all contributing scientific as well as nonscientific stakeholders to quantify drivers, uncertainties, and rates of change during discussions, workshops, and reviews. The intention was to generate as much of the stakeholder-demanded information as possible within the stakeholder process. To facilitate quantification, the scenario team provided references and orientation from historical rates of change, political targets, and examples from other scenario exercises. The plausibility and consistency of potentially conflicting rates of change were repeatedly checked and discussed, e.g., land demands for different purposes. Among the most controversial topics were (1) developments in bioenergy production, (2) size and role of protected areas, (3) the role of organic agriculture, and (4), to a lesser degree, expected changes in the population until 2050. The discussions about the quantification of the driver ‘bioenergy production,’ in the extreme arguments of some participants, reflected partly the strong emotions observed, but also the problem of confounding plausibility and probability of potential future developments. It was argued by supporters that bioenergy production could cover up to 50% of the agricultural area in the NaBü scenarios. Scientific opponents argued that this form of renewable energy would not make sense at all, because only 1% of radiation can be captured by plants, whereas practitioners argued that people in the region, namely in one of the federal states, are not willing to accept bioenergy. However, when rediscussing the topic in the second workshop with scientists, and on the second day of the practitioner workshop, independently both groups agreed on an upper limit of 30% of the agricultural land for bioenergy crop use in the NaBü extreme scenario.
To support the identification with and the use of the scenarios, dissemination strategies needed to be adapted so that the different stakeholder groups could avoid focusing on forms or media not accessible or not usually used by them (Jacobs et al. 2005). Thus, the outcomes of the scenario exercises were distributed in different formats familiar to the respective users. One dissemination pathway ran along scientific channels, such as scientific papers, conferences, and a download area on the Centre’s homepage (http://www.ufz.de/index.php?en=21894
). As a second dissemination pathway, the scenario team produced a brochure in a language accessible to practitioners and laypeople, including a summary of the research process, the storylines, visualizations, and exemplary quantifications of the scenarios. The brochure was distributed via the postal service and email and at various events and workshops with regional foci organized by regional stakeholders.
Short versions of the four storylines, i.e., “Sustainable & Citizen Friendly” moderate/extreme and “Radical Market Forces” moderate/extreme, are provided in Appendix 1. The full versions are accessible and downloadable via http://www.ufz.de/index.php?en=21894
To provide multiple means of dissemination, the visualization of the endpoints of the scenarios in the form of virtual maps were used. Based on the storylines, characteristic elements of land use were identified, and their spatial extents and configurations in the year 2050 were estimated. Supported by professional designers, we developed a simple symbology and color-coding to visualize the four different pathways into the future. The maps underwent three rounds of reviews by scientists and practitioners to ensure that the intended messages, i.e., key elements in the landscape, spatial extent, and spatial configuration, were evident and plausible for the intended users. In Figure 2, the virtual map for NaBü extreme scenario is presented. All four virtual maps are downloadable via http://www.ufz.de/index.php?en=21894
Quantification of storylines
Expected changes and rates of change during the period until 2050 were repeatedly debated within and between the groups of stakeholders, experts, and the scenario team. Whenever available, the quantification was guided by the quantified output of the corresponding GEO-4 scenarios at the highest spatial resolution (country level). In the following paragraphs, we focus on the quantification of changes in organic agriculture and human population, because both of them are considered to be specific for the region, and neither of them can directly be deduced or downscaled from the GEO-4 scenarios.
Spatial dynamics of organic agriculture
All scenarios included assumptions about changes in lifestyle and food consumption, as well as assumptions about the extent of population change. Scientists as well as practitioners assumed increasing preferences and consequently a much higher demand for regionally produced organic food under NaBü conditions. Independently, both groups assumed a maximum coverage of 30% of agricultural land in 2050, surpassing the (outdated) target of the national government in 2001, which was 25%. The rates of change assumed in NaBü moderate, resulting in 20% organic agriculture in 2050, reflected the rates of change observed in the region during the last decade (Fig. 3).
Under RaMa moderate conditions, both stakeholder groups assumed that the trend of the recent past would not continue, but demands and areas for organic production would remain at the current level until 2050. Under extreme conditions, stakeholders assumed that organically produced food would be a niche product for wealthy people and thus, demands and spatial coverage would decrease from the current 5% to 3% until 2050 (Fig. 3). Some stakeholders, mostly practitioners, suggested more diverging developments of up to 50% of the agricultural area in NaBü extreme and down to 1% in RaMa extreme. However, extreme values were considered implausible by scientists and most practitioners, and consensus was built based on the values stakeholders found most plausible.
Based on the recent historical dynamics, i.e., ~ linear increase, no additional assumptions were made with respect to variations in the rates of change, which is why linear changes were assumed for the entire period. Note that changes in total agricultural areas may cause nonlinear changes in the increase or decrease of areas for organic agriculture.
Quantification of changes in human population
The quantification of population changes had a high priority because they were ranked number one by the practitioners on the list of regional uncertainties. Both groups of stakeholders assumed that the human population would continue to decrease in all four scenarios, but practitioners provided much richer details concerning expected changes in urban-rural population shifts. For the RaMa scenarios, they assumed ± strong decreases in rural population up to completely abandoned villages, the merging of communities, and, simultaneously, strong suburbanization and partial segregation processes in urban areas; both processes facilitated by strongly reduced regional and urban planning. For the NaBü scenarios, population decreases and rural depopulation were assumed to be less dramatic. On the other hand, during the regional practitioners’ workshop, discussions about changes in urban and rural areas were influenced by planners, mostly arguing against extreme assumptions phrased by other participants, such as giving up villages completely, downgrading roads to dirt roads, or removing roads to connect biotopes. The assumptions influenced by the professional views of planners, based on historical or current changes, were partly challenged by other participants as ‘too narrow.’
The quantification of changes in population shown in Figure 4 used assumptions of the twelfth population forecast for Germany, which provides immigration and outmigration patterns across national boundaries, harmonized interstate migration, as well as assumptions for life expectancy, +7 years compared to currently, and a fertility of 1.4 children per woman (Statistisches Bundesamt 2010). For the RaMa scenarios, we adapted a population forecast of the Federal Statistical Office (variant V1W1: national immigration 100,000 from 2020 onward; Statistisches Bundesamt 2010). Comparably, for the NaBü scenarios, forecast variant V1W2 (national immigration 200,000 from 2020 onward) was adapted, because higher immigration rates matched the assumed attractive social and environmental situation in the NaBü scenarios. Internal outmigration until 2030 was kept for RaMa moderate, cut by 50% for NaBü moderate and doubled for both of the extreme scenarios. The resulting population losses until 2050 vary considerably between federal states and scenarios ranging between 17% in Saxony (NaBü moderate) and 41% in Saxony-Anhalt (RaMa extreme). Note that the larger populations of the NaBü scenarios and the lower populations of the RaMa scenarios are consistent with the assumptions made for Germany in the corresponding GEO-4 scenarios.
We demonstrated the suitability of scenario exercises for integrating diverse interests and forms of knowledge, not only between, but also within the two different groups of stakeholders. However, even when focusing on the integration of the concerns and knowledge of different stakeholder groups, there are limits to what can be included and represented in a single regional approach, without losing scientific credibility and salience for the majority of intended users. The advantages and limitations of the concept are presented below using the experiences made during the case study.
The learning effects associated with stakeholders participating in scenario exercises could be found in our case study. Surveys and personal communication confirmed that scientists, acting as stakeholders as well as members of the review panel, enjoyed the exchange with colleagues outside their subject areas and confirmed an improved understanding of each other’s work. Further, the exercises had a positive impact on collaboration, even resulting in the development of further research proposals.
However, scientific stakeholders repeatedly stated their lack of practical knowledge, such as land use planning or agricultural practices, confirming the necessity of involving regional stakeholders (Biggs et al. 2007). Although most of the scientists positively commented about learning effects during the scenario exercise, expectations of and comments from regional practitioners were more varied, covering the entire range from ‘curiosity’ through ‘opportunities for professional discussions’ to ‘intended use of results’.
In contrast with the Biggs’ et al. (2007) assumption that the process is of importance mainly for the decision making of regional stakeholders, we found that both scientists and regional stakeholders benefited from the process of scenario development, i.e., learned from each other and broadened their perspectives. Furthermore, scientific and regional stakeholders expressed their demand for the products of the exercise, which we interpreted as a sign of salience and legitimacy of the scenarios developed in this exercise. Thus we considered both aspects, the process and the product, to be essential for successful integrative scenario development (O’Neill et al. 2008).
The developers are aware that, at best, they can claim that they included the perspectives of the involved stakeholders although they used different forms of participation (see component 1) to bridge potential gaps between diverging perspectives, and between varying cultures of discourse and collaboration. It was not evaluated to what extent the scenarios are legitimate to anyone other than the participating stakeholders, and it is acknowledged that there might be limits to representativeness.
As one means to facilitate interdisciplinary/intersectorial knowledge integration, experts were invited to present short statements at the beginning of the workshops covering key aspects relevant for land-use scenarios, e.g., different drivers of change, historical land-use change, observed and expected impacts, etc., to reduce information deficits of participating stakeholders and contextualize disciplinary/sectorial views and arguments.
Because of the diversity of scientific and regional stakeholders involved, drivers, assumptions, and, subsequently, the storylines in this study already covered a large range of land-use related issues based on extensive scientific knowledge. The scenario exercise proved to be a useful step in integrating different research strands. Thereby, the sequential participation of the two stakeholder groups turned out to be highly beneficial because it enabled developers and scientific stakeholders to focus on interdisciplinary issues, which could to a large degree be discussed and settled during the first two workshops. Thus, in the subsequent workshop involving practitioners as the second stakeholder group, it was possible to focus on the mostly regional scale uncertainties addressed by this group and on discussing different sectorial perspectives, e.g., of planners and environmentalists. In this context, it was advantageous that one axis to represent key uncertainties was foreseen for each stakeholder group. We had anticipated to use the axis for different drivers of change, instead the survey of the second stakeholder group suggested we employ one axis for drivers and the second one to address slow versus fast rates of change; a strategy successfully applied in other scenario approaches. We admit there was a risk that the second group could have rejected the assumptions of the first group, and this would have led to a rediscussion of all uncertainties during the workshop.
As a general pattern, scientific stakeholders tended to provide input on more general or large-scale aspects, e.g., climate change, demographic trends, whereas regional stakeholders tended to provide more (creative) regional details on how the future may unfold, e.g., the controversial issue of urban sprawl, abandoned villages, deconstruction of roads, etc., and many assumptions were complementary. However, because of a number of contradictions, it turned out that the integration of knowledge between the multiple disciplines, i.e., scientific stakeholders, and sectors, i.e., regional planners and other practitioners, was even more challenging and time consuming than expected, and further review steps were necessary (see discussion about quality control). A second pattern affecting knowledge integration within and between stakeholder groups was related to the views and assumptions presented from a personal or from a professional perspective. Assumptions based on professional perspectives were less frequently challenged by other participants than personal views and tended to be closer to current developments/problems and less creative.
We consider quantification an optional step, but did quantify assumptions based on the requirements of some stakeholders. However, we also found that quantification of scenario assumptions provided some means for knowledge integration because a lack of detail or contradictions in assumptions may become more apparent than in the case of qualitative assumptions. Thus, contradictions can be used either as an entry point for further discussions or also for the integration of knowledge in different groups of participants. However, quantification efforts also may uncover different world views or assumptions about how different drivers are linked. This occurred during discussions about the expected increase or decrease of protected areas in the NaBü scenarios. Some participants assumed that because of the increase of organic agriculture, up to 30% of agricultural land, the need for protected areas would decrease because this type of agriculture was perceived as less intense, more diverse, and closer to nature, whereas another fraction argued that in a nature respecting scenario, people would tend to protect larger fractions of land than today. This thematic field was also one of the few examples in which assumptions based on professional perspectives were strongly challenged because the major opponents in this discussion were all professionally involved in nature protection, either in research, planning, or working with NGOs.
The objective, to integrate the demands and perspectives of different stakeholders, required a strong focus on quality control. Criteria to assess the quality of scenarios and the process of developing them have been suggested by various authors (Alcamo and Henrichs 2008, Alcamo et al. 2008, Hulme and Dessai 2008). We combined different methods of quality control in component 3, with a review panel as the key element for repeated rounds of revision to ensure consistency, credibility, and transparency of the process. The expected critical view of scientific stakeholders toward the credibility of the scenarios was accounted for by the early establishment of the scientific review panel and the involvement of additional expertise covering knowledge gaps of the scenario team and the panel, e.g., in the fields of forestry and bioenergy.
During the process it appeared that scientists as well as practitioners often viewed the future under assumed probability instead of plausibility perspectives, especially in their core fields of expertise, an effect well known in psychology (Kahneman 2012). Additionally, they frequently were very hesitant to make assumptions in topics beyond their core expertise. In contrast, some of the participants of both stakeholder groups, who were more familiar with the method of scenario development, formulated much more creative and provocative assumptions.
More rounds of review than foreseen were needed to accommodate open or conflicting questions. After the scenario developers processed the assumptions, ex post surveys and revisions by different stakeholders beyond the review panel were considered necessary to ensure adequate representation of the views and perceptions of all stakeholder groups. These were also further steps toward knowledge integration. They could be used either to cross-check that knowledge provided by stakeholders is sufficiently integrated or to flesh out coarse assumptions with more detailed data. Apart from the additional time needed for consolidation, the conflicting assumptions caused a narrowing of the extremes in scenario assumptions. This was also reflected in the quantitative assumptions, e.g., the changes in protected areas in both “Radical Market Forces” scenarios, which, in the first version, were assumed to decrease from the current 21% to 1.5% and 3% of the surface area, whereas in the revised version, the assumptions were narrowed to decreases of 9% and 18% of the surface area.
Global and regional drivers and uncertainties were assessed in quantitative surveys and in subsequent workshops with different stakeholder groups. However, the review panel was only composed of scientists, and only toward the end were other nonscientific stakeholders involved in the review process. In contrast to our case study regarding the integration of scientific and nonscientific stakeholders, Walz et al. (2007) established an advisory board of well-respected local stakeholders instead of a review panel, reflecting the stronger focus on integrating different local stakeholder groups rather than scientists and practitioners. We did not consider this alternative feasible for our exercise because the overall review process was very time consuming, and we did not expect that practitioners could afford to spend the necessary time, especially because our research program did not foresee financial resources for such long-term involvement of nonscientists. However, if these resources are available, the integration of scientific and nonscientific stakeholders, via a common review panel is considered an adequate alternative.
CONCLUSIONS AND OUTLOOK
In this integrative scenario exercise, a conceptual framework was presented, which may serve, as a complete package or in separate components, as a guide or template for other integrative scenario exercises at regional or local scales. The implementation in Central Germany showed that the concept is able to capture widely varying assumptions, demands, and perspectives in one set of regional scenarios, by applying rigorous measures of quality control to ensure salience for the intended users. Knowledge integration was facilitated by the general willingness of both stakeholder groups to accept arguments and views based on complementary professional perspectives. Furthermore, although not all stakeholders could be involved simultaneously, both groups seemed to be satisfied with the representation of views and perceptions in the scenarios. This success was facilitated by our flexibility in representing the drivers of change identified by both groups, i.e., by providing one axis per stakeholder group for the key uncertainties. However, the success was not only based on suitable concepts, the stakeholders invested considerable time and effort in the exercise, as did the review panel and the scenario team.
Stakeholders mainly criticized the complexity of the task, i.e., scenario development, and the limited time available during the workshops, whereas during the final presentation, some scientific stakeholders and members of the review panel commented that the scenario assumptions could be more extreme. Between the broadness of plausible scenarios and the objective to integrate different groups of stakeholders, the scenario team had a clear preference for the integration goal, based on broadly accepted plausible scenarios. The latter was considered more relevant for a regional assessment than (slightly) broader scenario assumptions.
Results of this scenario exercise have been disseminated in various forms and data have been made available for follow-up studies planned for the research topic. Currently, simulation studies are being prepared using the scenarios to drive numerical models, e.g., addressing land-use change, environmental impacts, and other topics. Modeling results will in turn be presented to regional stakeholders in future workshops, which will also serve to jointly develop recommendations for more sustainable land use in Central Germany. Although we could ensure legitimacy, salience, and relevance for the participating scientific and regional stakeholders, we agree, for example, with Chilvers (2009) or Lövbrand et al. (2011) that the evaluation of the legitimacy of outcomes beyond the stakeholders involved and the usefulness, i.e., the application of the scenarios and related research results (Hulme and Dessai 2008) is of utmost importance and will also be part of future research. A clear indication that integrative scenarios are considered relevant and useful by groups beyond the stakeholder groups involved in their development, is their application by other users. Since the storylines and visualizations presented were made publicly available, they are being applied by other users in scientific research projects focusing on green infrastructure, on the regional forestry sector, and in schoolbooks addressing environmental change.
We greatly appreciate the contributions of all members of the review panel to the development of the regional scenarios, the numerous additional reviewers of the storylines, and all our stakeholders who contributed to scenario development. We thank the Helmholtz Centre for Environmental Research for providing additional funds to host the stakeholder workshop and to produce a printed report. We are also grateful for the very constructive comments from the editor and two anonymous reviewers, whose contributions have considerably improved this paper.
Acreman, M. 2005. Linking science and decision-making: features and experience from environmental river flow setting. Environmental Modelling and Software
Alcamo, J. 2001. Scenarios as tools for international environmental assessments.
Environmental Issue Report no. 24. European Environment Agency, Copenhagen, Denmark. [online] URL: http://www.eea.europa.eu/publications/environmental_issue_report_2001_24
Alcamo, J., and T. Henrichs. 2008. Towards guidelines for environmental scenario analysis. Pages 13-35 in
J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis.
Elsevier, Amsterdam, the Netherlands. http://dx.doi.org/10.1016/S1574-101X(08)00402-X
Alcamo, J., K. Kok, G. Busch, and J. Priess. 2008. Searching for the future of land: scenarios from the local to global scale. Pages 67-103 in
J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis.
Elsevier, Amsterdam, The Netherlands.
Bernarie, M. 1988. Delphi- and Delphi like approaches with special regard to environmental standard setting. Technological Forecasting and Social Change
Biggs, R., C. Raudsepp-Hearne, C. Atkinson-Palombo, E. Bohensky, E. Boyd, G. Cundill, H. Fox, S. Ingram, K. Kok, S. Spehar, M. Tengö, D. Timmer, and M. Zurek. 2007. Linking futures across scales: a dialog on multiscale scenarios. Ecology and Society
12(1): 17. [online] URL: http://www.ecologyandsociety.org/vol12/iss1/art17/
Bishop, P., A. Hines, and T. Collins. 2007. The current state of scenario development: an overview of techniques. Foresight
Bohunovsky, L., J. Jäger, and I. Omann. 2011. Participatory scenario development for integrated sustainability assessment. Regional Environmental Change
Chilvers, J. 2009. Deliberative and participatory approaches in environmental geography. Pages 400-417 in
N. Castree, D. Demeritt, D. Liverman, and B. Rhoads, editors. A companion to environmental geography
. Wiley-Blackwell, Chichester, West Sussex, UK. http://dx.doi.org/10.1002/9781444305722.ch24
Cuppen, E., S. Breukers, M. Hisschemöller, and E. Bergsma. 2010. Q methodology to select participants for a stakeholder dialogue on energy options from biomass in the Netherlands. Ecological Economics
de Nijs, T. C. M., R. de Niet, and L. Crommentuijn. 2004. Constructing land-use maps of the Netherlands in 2030. Journal of Environmental Management
European Environment Agency (EEA). 2009. Looking back on looking forward: a review of evaluative scenario literature.
EEA Technical Report No 3/2009, European Environment Agency, Copenhagen, Denmark. [online] URL: http://www.eea.europa.eu/publications/looking-back-on-looking-forward-a-review-of-evaluative-scenario-literature
Giljum, S., A. Behrens, F. Hinterberger, C. Lutz, and B. Meyer. 2008. Modelling scenarios towards a sustainable use of natural resources in Europe. Environmental Science and Policy
Haasnoot, M., and H. Middelkoop. 2012. A history of futures: a review of scenario use in water policy studies in the Netherlands. Environmental Science and Policy
Harries, C. 2003. Correspondence to what? Coherence to what? What is good scenario-based decision making? Technological Forecasting and Social Change
Henrichs, T., M. Zurek, B. Eickhout, K. Kok, C. Raudsepp-Hearne, T. Ribeiro, D. van Vuuren, and A.Volkery. 2010. Scenario development and analysis for forward-looking ecosystem assessments. Pages 151-220 in
N. Ash, H. Blanco, C. Brown, K. Garcia, T. Henrichs, N. Lucas, C. Raudsepp-Hearne, R. D. Simpson, R. Scholes, T. P. Tomich, B. Vira, and M. Zurek, editors. Ecosystems and human well-being - a manual for assessment practitioners
, Island, Washington, D.C., USA. [online] URL: http://www.unep-wcmc.org/medialibrary/2010/10/31/90af3045/EcosystemsHumanWellbeing.pdf
Hughes, B. B. 2009. Forecasting long-term global change: introduction to international futures (IFs).
Frederick S. Pardee Center for International Futures, Josef Korbel School of International Studies, University of Denver, Denver, Colorado, USA. [online] URL: http://www.ifs.du.edu/assets/documents/IntroductiontoIFsv5_04.pdf
Hulme, M., and S. Dessai. 2008. Predicting, deciding, learning: can one evaluate the "success" of national climate scenarios? Environmental Research Letters
Jacobs, K., G. Garfin, and M. Lenart. 2005. More than just talk: connecting science and decisionmaking. Environment
Johnson. K. A., G. Dana, N. R. Jordan, K. J. Draeger, A. Kapuscinski, L. K. Schmitt Olabisi, and P. B. Reich. 2012. Using participatory scenarios to stimulate social learning for collaborative sustainable development. Ecology and Society
17(2): 9. http://dx.doi.org/10.5751/ES-04780-170209
Kahane, A. 2007. Solving tough problems: an open way of talking, listening, and creating new realities.
Berrett-Koehler, San Francisco, California, USA.
Kahneman, D. 2012. Thinking, fast and slow.
Penguin, London, UK.
Kaljonen, M., R. Varjopuro, M. Gieczewski, and A. Iital. 2012. Seeking policy-relevant knowledge: a comparative study of the contextualisation of participatory scenarios for Narew River and Lake Peipsi. Environmental Science and Policy
Kok, K. 2009. The potential of fuzzy cognitive maps for semi-quantitative scenario development, with an example from Brazil. Global Environmental Change
Kok, K., M. Patel, D. S. Rothman, and G. Quaranta. 2006. Multi-scale narratives from an IA perspective: part II. Participatory local scenario development. Futures
Kok, K., M. van Vliet, I. Bärlund, A. Dubel, and J. Sendzimir. 2011. Combining participative backcasting and exploratory scenario development: experiences from the SCENES project. Technological Forecasting and Social Change
Kosow, H., and R. Gaßner. 2008. Methods of future and scenario analysis: overview, assessment, and selection criteria.
Studies no 39. Deutsches Institut für Entwicklungspolitik, Bonn, Germany. [online] URL: http://dspace.cigilibrary.org/jspui/bitstream/123456789/26008/1/Methods%20of%20future%20and%20scenario%20analysis%20-%20overview,%20assessment,%20and%20selection%20criteria.pdf?1
Liu, Y., H. Gupta, E. Springer, and T. Wagener. 2008. Linking science with environmental decision making: experiences from an integrated modeling approach to supporting sustainable water resources management. Environmental Modelling and Software
Lövbrand, E., R. Pielke, Jr., and S. Beck. 2011. A democracy paradox in studies of science and technology. Science, Technology, and Human Values
Mahmoud, M., Y. Liu, H. Hartmann, S. Stewart, T. Wagener, D. Semmens, R. Stewart, H. Gupta, D. Dominguez, F. Dominguez, D. Hulse, R. Letcher, B. Rashleigh, C. Smith, R. Street, J. Ticehurst, M. Twery, H. van Delden, R. Waldick, D. White, and L. Winter. 2009. A formal framework for scenario development in support of environmental decision-making. Environmental Modelling and Software
Masini, E. B., and J. M. Vasquez. 2000. Scenarios as seen from a human and social perspective. Technological Forecasting and Social Change
Metzger, M. J., M. D. A. Rounsevell, H. Van den Heiligenberg, M. Pérez-Soba, and P. Soto Hardiman. 2010. How personal judgment influences scenario development: an example for future rural development in Europe. Ecology and Society
15(2): 5. [online] URL: http://www.ecologyandsociety.org/vol15/iss2/art5/
Messner, F. 2007. Integrated assessment of water policy strategies in the context of global change: the integrative methodological approach and its application in the Spree and Schwarze Elster river basins. Pages 265-303 in
J. D. Erickson, F. Messner, and I. Ring, editors. Ecological economics of sustainable watershed management.
(Advances in the economics of environmental resources
, Volume 7). Elsevier, Amsterdam, Netherlands. http://dx.doi.org/10.1016/S1569-3740(07)07012-5
Mostert, E. 2003. The challenge of public participation. Water Policy
O’Neill, B., S. Pulver, S. VanDeveer, and Y. Garb. 2008. Where next with global environmental scenarios? Environmental Research Letters
Palomo, I., B. Martín-López, C. López-Santiago, and C. Montes. 2011. Participatory scenario planning for protected areas management under the ecosystem services framework: the Doñana social-ecological system in southwestern Spain. Ecology and Society
16(1): 23. [online] URL: http://www.ecologyandsociety.org/vol16/iss1/art23/
Patel, M., K. Kok, and D. S. Rothman. 2007. Participatory planning in land use analysis: an insight into the experiences created by stakeholder involvement in the northern Mediterranean. Land Use Policy
Rounsevell, M. D. A., and M. J. Metzger. 2010. Developing qualitative scenario storylines for environmental change assessment. Wiley Interdisciplinary Reviews: Climate Change 1(4):606-619. http://dx.doi.org/10.1002/wcc.63
Rounsevell, M. D. A., I. Reginster, M. B. Araújo, T. R. Carter, N. Dendoncker, F. Ewert, J. I. House, S. Kankaanpää, R. Leemans, M. J. Metzger, C. Schmit, P. Smith, and G. Tuck. 2006. A coherent set of future land use change scenarios for Europe. Agriculture, Ecosystems and Environment
Seppelt, R., I. Kühn, S. Klotz, K. Frank, M. Schloter, H. Auge, S. Kabisch, C. Görg, and K. Jax. 2009. Land use options – strategies and adaptation to global change. Terrestrial environmental research. Gaia
Statistische Ämter des Bundes und der Länder. 2010. Regionalstatistik
. Statistische Ämter des Bundes und der Länder, Wiesbaden, Germany. [online] URL: https://www.regionalstatistik.de
Statistisches Bundesamt. 2010. Bevölkerungsvorausberechnungen
. Statistisches Bundesamt, Wiesbaden, Germany. [online] URL: http://www-genesis.destatis.de
Toderi, M., N. Powell, G. Seddaiu, P. P. Roggero, and D. Gibbon. 2007. Combining social learning with agro-ecological research practice for more effective management of nitrate pollution. Environmental Science and Policy
Tress, B., and G. Tress. 2003. Scenario visualisation for participatory landscape planning – a study from Denmark. Landscape and Urban Planning
United Nations Environmental Programme (UNEP). 2007. Global environment outlook 4. Environment for development.
UNEP, Nairobi, Kenya. [online] URL: http://www.unep.org/geo/geo4.asp
van Notten, P. W. F., J. Rotmans, M. B. A. van Asselt, and D. S. Rothman. 2003. An updated scenario typology. Futures
Waldhardt, R., M. Bach, R. Borresch, L. Breuer, T. Diekötter, H. Frede, S. Gäth, O. Ginzler, T. Gottschalk, S. Julich, M. Krumpholz, F. Kuhlmann, A. Otte, B. Reger, W. Reiher, K. Schmitz, P. Schmitz, P. Sheridan, D. Simmering, C. Weist, V. Wolters, and D. Zörner. 2010. Evaluating today’s landscape multifunctionality and providing an alternative future: a normative scenario approach. Ecology and Society
15(3): 30. [online] URL: http://www.ecologyandsociety.org/vol15/iss3/art30
Walz, A., C. Lardelli, H. Behrendt, A. Grêt-Regamey, C. Lundstöm, S. Kytzia, and P. Bebi. 2007. Participatory scenario analysis for integrated regional modelling. Landscape and Urban Planning
Winowiecki, L., S. Smukler, K. Shirley, R. Remans, G. Peltier, E. Lothes, E. King, L. Comita, S. Baptista, and L. Alkema. 2011. Tools for enhancing interdisciplinary communication. Sustainability: Science, Practice, and Policy
Zurek, M. B., and T. Henrichs. 2007. Linking scenarios across geographical scales in international environmental assessments. Technological Forecasting and Social Change