INTRODUCTION

In this article, we discuss a project that demonstrates how philosophical dialogue can be structured to build collaborative capacity within a diverse community around a contentious issue, viz., climate change. This project was the focus of a partnership between the Toolbox Dialogue Initiative (TDI)—a philosophical research and facilitation effort based at Michigan State University (MSU) that focuses on building collaborative capacity in cross-disciplinary teams (Hubbs et al. 2020)—and the West Michigan Sustainable Business Forum (WMSBF), a “regional collaboration of business, government, nonprofits and academia dedicated to promoting business practices that demonstrate environmental stewardship, economic vitality, and social responsibility” (WMSBF 2020). This project is a case study of how philosophy can be synthesized with deliberative processes to address consequential issues outside academia (Brister 2020).

Between 2013 and 2015, TDI and WMSBF together with collaborators from Penn State University (PSU) and the RAND Corporation developed a framework for organizing and motivating region-scale efforts to adapt to climate change. Philosophy, along with methods drawn from mental modeling (Mayer et al. 2017), was used to structure dialogue among stakeholders pursuing climate resiliency at a regional scale. Specifically, the Toolbox dialogue method—a philosophically grounded approach to enhancing communication and collaboration in complex research and practice (Eigenbrode et al. 2007, O’Rourke and Crowley 2013, Hubbs et al. 2020)—was combined with a values-informed mental modeling approach rooted in risk analysis, assessment, and communication to structure conversations among non-academic stakeholders interested in planning for a sustainable future.

We begin by describing the background of this project, including details about climate resiliency efforts in West Michigan and the Toolbox dialogue method, which was extended in this project from academic research into community organization involving the West Michigan Climate Resiliency Framework Initiative. This extension involved application of several methods, discussed below. We then present preliminary results that suggest the potential for philosophical dialogue to enhance mutual understanding in community initiatives that focus on sustainable responses to climate change. Overall, the article supplies a detailed example of how philosophy can support policy-relevant, community-level decision-making processes.

PROJECT BACKGROUND

The project detailed here emerged from a research effort that addressed sustainable climate risk management. The “Transdisciplinary Research Network for Sustainable Climate Risk Management,” or SCRiM (http://scrimhub.org/), based at PSU, aims to answer the question “What are sustainable, scientifically sound, technologically feasible, economically efficient, and ethically defensible climate risk management strategies?” Research extending the methodology of mental modeling to accommodate human values relevant to climate decision making was sponsored by SCRiM (Mayer et al. 2017). They also worked with TDI to enhance collaborative and communicative capacity within their community.

During a SCRiM meeting, the idea arose to use the new values-informed mental models method to guide development of a climate-focused Toolbox application. Prior collaboration between TDI and WMSBF created an opportunity to apply this approach in a community context, focusing on efforts in West Michigan to remain resilient in the face of a changing climate. In this section, we describe two key background considerations for this project: (a) concern about climate resiliency in West Michigan, and (b) the Toolbox dialogue method, emphasizing its extension into a non-academic, community-based context.

Climate Resiliency in West Michigan

West Michigan experienced a series of events between 2012 and 2014 generally associated with climate change. These included a historic flood (Torregrossa 2013), extreme temperatures in summer and winter (Jackson 2014), loss of crops due to inclement weather events (Parker 2012), and unpredictable growing seasons (Whetstone 2014). Local institutional leaders took notice of these events in the context of the national dialogue among planning and sustainability professionals about the need for climate-change adaptation and resilience in the wake of Hurricane Sandy and other disasters.

A collective recognition emerged in West Michigan that environmental change associated with climate change will have economic and social impacts on the quality of life in the region, potentially altering its globally important natural resources. In response, a resiliency dialogue began in two distinct modes: a community-scale effort anchored to municipal and stakeholder concerns in the City of Grand Rapids, and a cross-sector dialogue among environmental and sustainable business professionals employed by local businesses and institutions. The former yielded a report to the community containing recommendations aligned with the sustainability goals of the City of Grand Rapids (West Michigan Environmental Action Council (WMEAC) 2013). The latter featured a series of lectures and facilitated discussions hosted by WMSBF that positioned climate change as a “wicked problem” requiring shared capacity solutions. Both efforts identified diverse and interrelated issues and concerns, large numbers of stakeholders, and a general lack of understanding of the potential impacts of climate change.

Working with its City of Grand Rapids counterparts, WMSBF concluded that preparing for climate change should be an important part of the operations and regional planning of institutions, including small and large businesses, academic institutions, non-profit organizations, and governmental units. Moreover, these diverse parties should coordinate their planning. This established a need for a framework that would allow West Michigan to respond to climate change at the regional, community, and organizational levels accessibly, inclusively, resiliently, and measurably.

With this in mind, WMSBF launched the West Michigan Climate Resiliency Framework Initiative to help local institutions better respond to extreme weather events and other impacts commonly associated with climate change (WMSBF 2014). Its initial efforts consisted of sector-specific informational meetings, interviews with representatives of various business sectors and those of important stakeholder groups, and a culminating conference. In addition to gathering data on perceived risk and vulnerabilities, these efforts educated businesses, institutions, and their stakeholders on potential impacts from climate change and how they might think about responding to those impacts, and demonstrated a need for continued, collaborative adaptation planning.

The Toolbox Dialogue Method

Grounded in philosophical concepts and methods, the Toolbox dialogue method employs structured, reflexive dialogue about implicit research commitments to enhance communicative and collaborative capacity (Gonnerman et al. 2015, Hubbs et al. 2020, O’Rourke and Crowley 2020). Toolbox dialogue participants are typically research collaborators who gather in a workshop setting to discuss prompts developed for them by TDI. These prompts are designed to initiate reflection on research practice, and the dialogue-based workshop provides a setting for collaborators to articulate and share what they discover through that reflection. The typical result is an experience that abstracts away from concrete differences that exist among them as disciplinarians toward mutual understanding of their research commitments.

The Toolbox dialogue method is grounded in the commonly made observation that unacknowledged differences among the implicit research commitments of collaborators can be the source of confusion, disagreement, and dysfunction (e.g., Ramadier 2004, Lélé and Norgaard 2005, Choi and Richards 2017, Robinson et al. 2019). These commitments include constitutive and often foundational elements of one’s research worldview that become implicit by being invariably part of one’s engagement with research projects; because of their foundational nature, implicit research commitments help shape how one thinks about research problems (O’Rourke and Crowley 2013).

When collaborators work together, collective deliberation will reflect the possibly incommensurable or inconsistent influence of these implicit commitments, leading at times to impasse in ways that remain a mystery to the collaborators. The key to avoiding impasse in these situations is surfacing and acknowledging the differences so that their influence on collective deliberation can be recognized and, if need be, negotiated; although the result might be recognition of real disagreements (MacLeod 2018), it is better that these be discovered and, if possible, managed so that they don’t emerge as “deal-breakers” later in the collaborative process.

In a standard deployment of the Toolbox dialogue method, TDI works with a team of interdisciplinary research collaborators who are interested in enhancing mutual understanding of each other’s implicit research commitments. One underlying assumption is that philosophically structured dialogue has value for collaborators engaged in complex decision-making processes so long as it is designed with their specific context in mind. The goal is to enable them to see their common research problem from each other’s perspective (Looney et al. 2014). This goal is pursued in three steps.

First, TDI gathers input from team leaders and project collaborators via conversations and surveys about the conceptual landscape of the research project, which can reveal information about collaborator worldviews and potential sources of difference and disagreement.

Second, TDI uses the gathered input to identify project-relevant conceptual or normative issues about which collaborators appear to have different opinions, e.g., the role of hypotheses (Donovan et al. 2015) or whether advocacy is legitimate (Nelson and Vucetich 2009). The Toolbox Dialogue Initiative collaborates with the partner in formulating these issues as statements, or “dialogue prompts,” that express a particular point of view, and then these prompts are associated with a rating-response scale that helps tease out differences of opinion about the issues in question. The prompts are organized into thematically unified sets, termed “modules,” that highlight higher-level themes that should strike project collaborators as relevant to their common research effort. Together, the resulting modules, each containing six to eight dialogue prompts, constitute the “Toolbox instrument” (Hubbs et al. 2020; see Appendix 1 for the instrument used in this project).

Third, the project collaborators gather for a 2- to 3-h workshop, which begins with a preamble delivered by a TDI facilitator describing the theoretical background of the Toolbox dialogue method, the development of their workshop, and details about participation (e.g., institutional review board (IRB) protection). Participants then fill out the instrument and discuss the prompts for up to 2 h. This discussion is lightly facilitated so that the dialogue is owned by the participants. After filling out a copy of the instrument a second time, the workshop ends with a debrief conversation.

This project extended the Toolbox dialogue method in two ways. First, it was one of the first applications of the method outside of academia. The 86 participants in the Toolbox workshop at the West Michigan Climate Resiliency Conference were predominantly non-academic stakeholders from the Grand Rapids area. Many were affiliated with businesses at risk of losing income from extreme weather events due to climate change. (See Table 1 for participants’ occupations.) Although the previous subsection described the specific context of this extension, the wider context represented by climate resiliency efforts in West Michigan includes community-based, policy-relevant decision-making processes where structured dialogue can help clarify the perspectives involved and the issues at stake.

Academics can be a tough crowd for a process-focused workshop because professional incentives are typically tied to products, not process. The philosophical background of the Toolbox approach, however, is familiar to many academics, and typically there is professional courtesy shown to fellow academics who utilize an evidence-based approach in an effort to help. Outside the academy, however, these conditions are often not satisfied. Public perception of academics as elitist or out-of-touch, “ivory tower” types can create mistrust at the outset (Klein et al. 2011), and lack of familiarity by academics with the values and priorities of non-academic stakeholders can lead to engagements that don’t address stakeholder needs (Elliott 2017). In an effort to address these two concerns, this workshop was designed with stakeholders, making sure that their concerns were the focal point.

Relevance of Toolbox workshops for stakeholders depends on work done in the first step described above, where information about collaborator worldviews and potential sources of difference and disagreement are discovered. Because this information yields topics articulated as dialogue prompts, the dialogue will be relevant to the extent that this step reveals issues that matter to project success, from the perspectives of the participants. The standard ways of gathering this information, mentioned above, often confront two problems. First, conversations with leaders typically reveal only a subset of the issues that matter to the larger team. Because TDI representatives are usually unfamiliar with the partner, they are not in a position to recognize this shortcoming in all cases. The gap can be bridged by surveying project participants, but then another problem arises. The desire to respect the participants’ time and not exhaust them with the process of making the workshop happen militates against the follow-up that is sometimes necessary to fully comprehend the survey data. Consequently, these surveys can be somewhat superficial.

This leads to the second way this project extended the Toolbox dialogue method, viz., by modifying how information used to design the Toolbox instrument was collected from the participant community. Rather than talk in an unstructured way with project leaders or conduct a community survey, information was gathered via a mental modeling approach that has been employed successfully in contexts of risk analysis, assessment, and communication. This approach was modified, augmenting the standard way of developing mental models with value-focused coding that highlights the ethical dimensions of these models (Bessette et al. 2017, Mayer et al. 2017).

The mental modeling work, described below, supplied key assumptions and commitments about which there were differences of stakeholder opinion. These assumptions and commitments were then integrated as prompts into a Toolbox instrument used to structure a dialogue among relevant stakeholders. An important determinant of success in this venture was identifying key informants to sit for mental modeling interviews. These informants were selected on the basis of participation in informational meetings that focused on climate change in West Michigan, a central part of the process that we address in the next section.

METHODS

In this section, we describe the process of developing philosophical dialogue for community members in West Michigan around the topic of climate resiliency. This process involved four methods: informational meetings, values-informed mental modeling, Toolbox instrument development, and Toolbox workshops (see Fig. 1). As described in what follows, each method produced outputs that were inputs into the next: the meetings helped identify key informants to sit for the mental modeling interviews, the interviews provided themes that figured into instrument development, and the instrument structured the dialogue in the Toolbox workshops.

Informational Meetings

Although industry and community stakeholders possessed some understanding of climate change and its potential impacts, WMSBF had little confidence that representatives were prepared to have a thoughtful dialogue about the roles of their organizations in a regional resiliency effort. With few exceptions, organizations were only discussing climate change in terms of sustainability programs or requirements to reduce greenhouse gas emissions. Although awareness of a need for planning was emerging, there was little understanding of local impacts and no guidance on how organizations might begin internal discussions around those impacts. For constructive regional planning to occur, stakeholders would require some initial education and training.

A series of informational meetings was scheduled with key leadership groups. The West Michigan Sustainable Business Forum chose to limit its scope to local businesses and institutions, with the hope that the framework developed could be used later to facilitate collaborative regional planning for other stakeholder groups, such as neighborhoods, families, and disadvantaged communities. Meetings were held with representatives from each of nine industry sectors: Manufacturing; Built Environment: Commercial, Infrastructure, Design and Management; Tourism, Recreation and Hospitality; Food and Farming; Supply Chain and Logistics; Facilities Management; Residential Building; Healthcare and Assisted Living; and Neighborhood Business.

One goal of the informational meetings was to cultivate a network of invested and informed leaders capable of representing their organizations and stakeholder groups in a regional climate resiliency planning initiative. The meetings would also serve to promote the existence of the initiative, gather stakeholder opinions relevant to the process, and identify key informants interested in participating in the development of the Toolbox instrument.

Hosted during a 2-mo period in spring 2014, each meeting was approximately 2 h, divided into three parts: a presentation on actual and predicted local climate change events, a presentation on and facilitated discussion of how these events might impact the operations of each stakeholder group, and an introduction to and facilitated discussion of climate adaptation planning. Participants were asked to complete worksheets to guide the facilitated discussions, which were collected and aggregated after each session. Collected data were used to develop programming and panel discussions at the conference. Participants were also given fact sheets at the end of each meeting with further information on potential impacts to their sectors. Meeting facilitators included the authors of the City of Grand Rapids Climate Resiliency Report and a student co-author of a similar report defining climate change vulnerability in Detroit. A local guest speaker at each meeting highlighted some specific ways climate change was impacting their organization.

This format delivered mixed results in the first three meetings. Some participants failed to see how the content was relevant to their organizations or positions, or they failed to find common interests with others at the meeting. To address this, the remaining meetings were reorganized to focus on specific business functions, as opposed to industry sectors. The change simplified the content of each meeting and made it easier for stakeholders to determine whether a meeting was relevant to their interests. Over the course of the nine meetings, WMSBF identified a diverse group of 10 key informants who would represent stakeholders in the mental model development, and a larger pool of participants for the facilitated dialogue groups at the conference.

Mental Modeling

To inform the design of the Toolbox workshop, we used a variation on the mental models method, “values-informed mental models” (or ViMM), to gain an understanding of how residents of Grand Rapids thought about risks associated with climate change and how these affected what they valued about their community. The mental models method is a qualitative social sciences method used in fields such as risk communication to represent how individuals understand decision-making situations and what factors influence their decisions (Morgan et al. 2001). There is a large literature on the mental models method and its use in environmental policy (Jones et al. 2011). The central idea behind the method is that individuals make decisions by manipulating internal mental representations of decision or reasoning environments.

Cognitive scientists have long theorized that individuals reason and make decisions by manipulating models of situations (Johnson-Laird 1983). The idea goes back to the work of the American pragmatist C. S. Peirce and has proved useful in explanations of everything from abduction to how individuals reason about epidemics (Thagard 2012). Mental models are not models in the formal sense, like the Bohr model of the atom, the double helix model of DNA, or global climate models. From an ontological perspective, the two types of models are distinguishable. Mental models comprise mental representations, making them mental (hence private) entities. Models in science, on the other hand, are public. Moreover, semantically speaking, the two types of models may be distinct. It is at least arguable that models in science aim for their elements to be isomorphic to what they represent (e.g., van Fraassen 1980; cf. Giere 2004), whereas in forming mental models, individuals are hypothesized to get by with significantly less (Johnson-Laird 2010). For example, the height of a tree that is taller than an adjacent tree may be represented in the mental model by this relational property (“taller than”), and not in any more fine-grained manner.¹

In the mental models method, analysts choose a representative sample from a population and conduct structured interviews in which individuals talk through hypothetical decisions, discuss how they understand or feel about elements of the decision environment, and explain how things appear to them. These interviews are then coded, and the coded data are analyzed to find interesting relationships between codes, such as frequency of co-mentions (Morgan et al. 2001). Often there is a significant disconnect between how individuals and policy experts understand a situation; effective risk communication requires policy experts to appreciate these differences, especially if the situation is extremely complex (e.g., climate change).

The ViMM variation on the mental models approach used in the Grand Rapids project aims to augment the traditional mental models method by capturing how values impact individuals’ mental models (Bessette et al. 2017, Mayer et al. 2017). Values can have a variety of influences on how individuals represent decision situations, such as highlighting some features of situations as more salient, treating some outcomes as more likely, and generally treating some elements of a situation as more relevant to decision making than others. On the ViMM approach, values are understood as part of the cognitive and affective machinery that affects how individuals represent a decision situation in their mental model. Values are not “represented” in models; rather, they affect how individuals represent parts of the world. For example, different levels of risk tolerance can cause parents to treat unlikely adverse effects from vaccination as more salient than they are, and so overrate the risk from vaccination (Downs et al. 2008). In these situations, parents’ mental models are affected by their values. With climate change, especially in the USA, numerous values impact how the decision situation is understood, including values drawn from political and religious beliefs, local differences in culture, and background knowledge.

Values are complicated psychological items. In ViMM, they are understood as reactive attitudes (Strawson 2008); or roughly, they manifest as attitudes toward different outcomes, and expression of these attitudes is taken as a proxy for (not constitutive of) values. The presence of reactive attitudes revealed in mental model interviews are coded by value terms (e.g., an attitude in favor of maintaining an existing swath of forest in its current state will be coded as “forest conservation”), and these codes are treated like other codes in the data set.

For this project, ViMM was used to build an initial representation of how individuals in Grand Rapids understood climate change, including its impacts on the future of their city and the outcomes of various actions aimed at managing climate risk in their region. Of particular importance here was a sense of how values impacted individual understanding of trade-offs involved in different climate risk management strategies. One of the most difficult things about climate change policy is balancing different goals, interests, and outcomes that do not admit of mutual maximization but must be prioritized. Robust climate change risk management policy involves getting these trade-offs correct (e.g., between future economic growth and emissions abatement), and this requires an understanding of how individuals value different outcomes and what risks and costs they are willing to accept (McInerney et al. 2012).

The Grand Rapids ViMM was specifically designed to reveal how individuals represented these trade-offs, and what their respective values were concerning different outcomes. As noted above, individual participants serving as key informants were drawn from those who attended a set of informational meetings. An interview protocol was designed over the course of Spring 2014, and interviews were conducted in June and July 2014 in Grand Rapids. The interviews were coded, and coded data were analyzed over the course of July and August 2014. The resulting ViMM was used to inform the development of the instrument used in the Toolbox workshops conducted in Grand Rapids in October 2014.

Toolbox Development

A common strategy for designing a survey instrument that uncovers fundamental attitudes is to include many survey prompts per targeted item. Instrument designers craft prompts to minimize distinct interpretations across respondents from the relevant population(s). One reason behind this approach is that, as the psychological opacity of the targeted item increases, it becomes less susceptible to direct introspection and straightforward report. In such cases, the best one can do is measure associated indicators while minimizing measurement noise due to ambiguous, vague, and unclear survey language (Tourangeau 1992).

Instrument design for Toolbox workshops, however, pursues a different strategy. The primary goal in these workshops is to get participants talking about assumptions that are unacknowledged, unrecognized, and unclear, and this goal is better served by prompts that provoke response rather than minimize measurement noise (Hubbs et al. 2020). Philosophical dialogue is the primary vehicle for enabling workshop participants to discover these implicit assumptions and thereby enhance mutual understanding. At its best, philosophical dialogue features (1) the freedom to discuss topics most salient to conversational participants; (2) the expectation that participants not only articulate their position on the topics but also their reasons, in ways that others will understand; and (3) the recognition that when participant contributions are not understood or embraced, others may query, expand on, or even challenge the participant (Gonnerman et al. 2015).

In light of the approach taken by TDI, the development of a Toolbox instrument for the West Michigan Climate Resiliency Conference was largely shaped by a single question: what will get people talking? More fully, what prompts could trigger dialogue that reveals fundamental assumptions through sharing, elaboration, querying, and challenging of participant responses and reasons? In searching for prompts of this sort, potential differences in climate change views among representatives from key stakeholder communities and business sectors in West Michigan were examined. It was a procedure driven by materials associated with the mental models (e.g., influence diagrams, coded interviews, and codebook), discussions with WMSBF leadership, and a review of the philosophical literature on climate change.

The design of the instrument began with the mental models interviews. There are, of course, many approaches to interview data in the qualitative research literatures. These approaches include long-established methodologies such as hermeneutics, phenomenology, and grounded theory (Rouston 2013). As the point of our engagement with the mental models interviews was rather unlike extant approaches to interviews—namely, the development of an instrument that could reveal unacknowledged assumptions through philosophical dialogue—the approach we took to the coded interviews was closer to strategies used when reading philosophical texts (Concepción 2004). More fully, the process of building candidate prompts from the coded interviews proceeded through three phases. First, given the importance of background knowledge when constructing textual interpretations (e.g., Bruer 1993: chapter 6), the instrument developer began by trying to get a general sense of the contents of the interviews. Time was spent examining the interview questions (e.g., mapping out their connections), the codebook, and the code cloud. Second, the interviews were given a fast read to develop a basic understanding of their content. Small notes in the margins were made along the way. Most of these aimed at identifying interview chunks relevant to themes that seemed pressing to the region based on conversations with WMSBF leadership (e.g., energy, sustainability, and emergency preparedness). The third phase focused on understanding what the interviewee meant by giving the interviews a very close read. Following Creswell (2007:153), the instrument developer frequently stepped back and asked, “What is happening here?” A second guiding question was, “What are the philosophically interesting differences found in, or presupposed by, the answers given by the respondents to the interview questions?” When the developer ran across a point of potential difference with another interview, this interview was tracked down. The question then became, what might a Toolbox prompt look like that centered on this apparent difference?

Two types of characteristics in interview responses were especially likely to be treated as indicative of potentially interesting philosophical differences. The first was language suggestive of a normative position on issues related to climate adaptation and resilience. An example is the following, which the interviewee offered in response to a question about community-level impacts of climate change:

So I get a little bit frustrated with questions that I—when I bring it back down to a local level there’re certainly things as predicted that, you know, should scare everyone, water scarcity issues, food is a big one for me. So I think that we’re dependent on a lot of outside places and distribution channels and not being able to do enough things ourselves which is a potential problem.

Here, the interviewee expresses a normative view about the limitations of local-level responses to climate change. As such, it served as a signpost for a range of philosophical assumptions that may merit inclusion in the Toolbox instrument, such as the relative advantages and disadvantages of mitigation and adaptation strategies pitched at different organizational scales.

The second type of characteristic likely to be treated as philosophically interesting was language suggestive of a position on conceptual (or “What is X?”) issues related to climate adaptation and resilience. For instance, in response to a question about experience in environmental issues, another interviewee pointed to their history of working on sustainable building and renewable energy, which was then immediately followed by this comment:

... both sustainable building and renewable energy are I think directly related to the concept of climate change. Using less of the resources that are available on this planet, conserving those resources.

Notice the quick transition from the notions of sustainability and renewables to that of conservation. The response thus served as a guidepost for philosophical issues pertaining to the nature of sustainability. For example, exactly what does “being sustainable” entail? Is it compatible with some degree or forms of non-conservation?

The development of Toolbox prompts was also guided by more specific design considerations, such as:

• Avoid logical complexity because responses to logically complex prompts can be hard to interpret, as when someone disagrees with a conjunction,
• Use vague, ambiguous, and extreme language to encourage fine tuning in the degree and content of participant commitments, and
• Avoid language that is too easy to agree or disagree with, as this often fails to generate conversation (Looney et al. 2014, Rinkus and Vasko 2020).

In addition, because the West Michigan Climate Resiliency Conference would extend the Toolbox approach to a non-academic context, it was important to avoid designing overly academic prompts that might discourage dialogue and undermine the ability of the workshop to enhance mutual understanding and collaborative capacity.

The original Toolbox dialogue instrument, intended primarily for scientific research teams, consists of 34 prompts organized into six modules (Looney et al. 2014). The design of a Toolbox instrument for the resiliency conference was guided by the structure of the original instrument. Toolbox personnel vetted the first draft of the instrument, organizing the prompts into modules, selecting prompts that seem most promising for sparking dialogue among conference participants, and rewording prompts to increase chances of dialogical engagement with prompt content. This process was then repeated with central organizers of the conference. In the end, four modules were produced, each containing six or seven prompts: Emergency Preparedness, Sustainability, Energy, and Government. These tracked issues of common concern that emerged from the ViMM interviews. (See Appendix 1 for the full instrument.)

Toolbox Workshop

The ViMM-based Toolbox instrument was used to structure dialogue in a workshop conducted at the West Michigan Climate Resiliency Conference. This conference was designed for “Executives, Sustainability Directors, Facilities Managers, EH&S [environment, health, and safety] Managers, Architects, Engineers, Contractors, Community Planners, Logistics Managers, Health Professionals and Other Public and Private Interests” and was intended to “introduce participants to national and local thought leaders on climate resiliency in the public and private sectors” (WMSBF 2014). One goal of the meeting was to enable participants to think collectively about a climate resiliency framework for West Michigan. The Toolbox workshop was a vehicle for achieving this goal by enabling participants to meet one another and exchange perspectives on climate change impacts in West Michigan.

A plenary part of the conference, the Toolbox workshop began with a preamble by the contact author that framed the experience for the participants. This brief presentation introduced the plan for the workshop, articulated goals for the participants and the research team, described the nature and history of the Toolbox dialogue method, discussed the IRB approval for the project, and issued instructions. Workshop goals for the participants were (a) to focus on climate resiliency in the public and private sectors, and (b) to enhance cross-sector capacity for communication and collaboration as the participants move toward a more resilient West Michigan. The research team announced their goals as (a) understand and improve communication about climate resiliency in West Michigan by fostering value-focused dialogue among industry and community stakeholders, (b) examine the effectiveness of dialogue that flows out of interviews with stakeholders in the region, and (c) collect pilot data for use in designing a more extensive study of value-focused communication.

After the preamble, participants divided into different dialogue groups. Eighteen influential individuals were invited to participate in two facilitated breakout dialogue sessions; these individuals were selected for breakout sessions by WMSBF, which had a special interest in their views on the prompts and in their reaction to the experience. The remaining 68 participants divided into 11 table groups for dialogue in the main meeting room. The table groups were self-facilitated, following the dialogue instructions issued at the end of the preamble. Three Toolbox team members circulated in the meeting room to address questions during the dialogue session and distribute post-dialogue Toolbox instruments to capture attitudes toward the prompts after the discussion. Lack of a sufficient number of trained Toolbox facilitators made it impossible to facilitate all the groups while keeping them small enough—no more than 10–12—to encourage meaningful participation.

Two types of data were collected during the workshop. First, the Toolbox instrument was administered before and after the dialogue for all 13 dialogue groups, yielding a set of pre- and post-dialogue rating-response scores for each participant. Additionally, audio recordings were made of the two facilitated dialogue sessions, which were then transcribed.

RESULTS FROM THE TOOLBOX WORKSHOP

This Toolbox workshop was a pilot designed to test both the ability of ViMM to inform cross-sector communication about climate resiliency in West Michigan and the extensibility of the Toolbox dialogue approach outside of academia. In light of these objectives, it was important to collect data that could help us understand the impact of the experience on the participants. In what follows, we focus on the transcripts from the two facilitated sessions.² These qualitative data are critical for assessing our goals, as the recordings indicate whether the participants were engaged by the issues and used the dialogue to identify differences and enhance their mutual understanding. Two co-authors reviewed the transcripts, determined what parts of the instrument they addressed, and analyzed them for impact; specifically, the transcripts were examined for differences among the participants about issues raised by the instrument. In this section, we describe aspects of the facilitated conversations that correspond to productive differences among the participants.

The first recorded dialogue session involved ten participants (five women, five men) from NGOs, government agencies, and businesses in the region. They were primarily mid-career professionals (i.e., 8–20 years into their career) and moderate to liberal politically. The dialogue in this session lasted for 40 min, and the participants focused primarily on the Emergency Preparedness and Energy modules, with time also spent discussing prompts in the Sustainability module.

This session’s dialogue began with the observation that people are more likely to help in an emergency if it is close to home—that it isn’t “real” unless it happens to them. This attitude is reflected in resistance to infrastructure investment and in a general inclination to be more reactive than proactive. Consideration of the economics of emergencies led to the suggestion that the market cannot be trusted to be a guide during climate change, given that risk calculations based on past events are not likely to be accurate. The participants were inclined to agree that there are two ways to deal with emergencies: be proactive and build infrastructure or have a response in place that minimizes risk. For several participants, the latter involved educating the public about emergency preparedness. Education then became a point of emphasis in the dialogue about energy. Because, as one participant put it, “investing in (energy) generation vs. efficiency, efficiency almost always wins,” it is important to emphasize education as a way of ensuring that people know about energy conservation strategies. Efficiency also depends on holding people accountable for upgrading their property to enhance efficiency. The dialogue concluded with attention to sustainability. This part of the dialogue began with the difficulty of planning for a long-term sustainable future when businesses must remain competitive in the short term. Education figured into this discussion as well, with one participant stressing the importance of motivating people to care about climate change by appealing to their value systems, and another emphasizing the need to make sure that consumer choices are well-informed, given their impact on sustainability.

In this session, the comments were not often focused on specific prompts. Separation from the prompts is not an uncommon aspect of Toolbox dialogues when participants are confident about the range of issues covered by the instrument, and participants in this session had substantial expertise about risk, emergency preparedness, energy efficiency, and other topics related to climate resiliency. Even so, the dialogue was clearly grounded in the instrument because concepts introduced by instrument prompts figured importantly in the dialogue, e.g., uncertainty, incentives, sustainability. Although education is not mentioned in the instrument, it was introduced in discussion of planning, infrastructure, and lost structures, all of which do explicitly figure into the instrument. This reflects the thinking in TDI that Toolbox modules serve as conceptual and normative “constellations,” explicitly highlighting salient aspects of themes relevant to the participants’ joint project, while also positioning participants so that they can foreground other aspects of project-relevant themes and make relevant connections among salient ones. This illustrates one way in which the prompts, when conjoined with a light facilitation approach, can structure dialogue without unduly constraining it.

Among the differences that emerged in the first dialogue session, two stand out. The first involved the value of education as a mechanism for social change. Education was broached in connection with each of the three modules receiving attention in this dialogue. It was noted that emergency preparedness includes an educational component, as people need to know “the infrastructure that they have in place,” and this may not be obvious without instruction. Likewise, because “[a] lot of people don’t understand that they have options,” education can be used to ensure that people are aware of energy conservation strategies. Sustainable living also turns on “consumer education” about the “entire life-cycle of a product” they buy.

Not everyone was equally excited about education as a means of effecting social change and achieving climate resiliency, though. For some, the value of education depends on one’s other values and beliefs, and it is important to recognize that education is not a panacea—“My God,” one participant exclaimed, “We have so many educational approaches that are already out there around this issue, and so education obviously isn’t, isn’t a change here.” This participant wanted “short-term proof,” and education offers little in the near term. Education is also problematic because it takes time away from other things that matter, such as raising a family and developing a career, and it fades and becomes outdated as time goes by. There is a need to “strike a balance,” as it won’t work to ask everyone to “become energy experts.”

The second topic where prominent differences were revealed was the role government should play in securing climate resiliency for the region. Government was seen as a vehicle for promoting new, environmentally sensitive technologies. One participant argued that government has a role to play in educating people about what is in their best interests when it comes to emergency preparedness, even if they don’t recognize that. But this paternalistic role was not the consensus view—several participants believed that it was better to let economics dictate that “you can’t build or you can’t live in a certain area” rather than the government. But this too was viewed as problematic because, in an emergency, people will expect the government to help even if economics have dictated lower taxes and fewer resources for government agencies.

The second recorded dialogue session involved eight participants (three women, five men) from NGOs, local government, regional healthcare providers, and local businesses. They ranged from early to late in their career phases. Three self-identified as “liberal,” two as “moderate,” (with two more in between), and one as “conservative.” The dialogue lasted 40 min. Participants primarily focused on eight prompts, with their discussion concentrating on the role of government, sustainability, and energy.

The dialogue in this session focused on, as one participant put it, the “intersection between the private sector and the public sector.” It began with a criticism of the instrument by a few participants who thought it focused too much on the role of government and not enough on private business in preparing for and responding to climate change. This observation spurred over 20 min of discussion on this topic. Initially there was clear disagreement over the role of business. One participant stated, “I think it’s important to emphasize the role of government. However, I also believe the role of businesses is primarily and probably more, uh, what we need to be investing in.” Another participant concurred, “Business really, if it gets it right, can harness the market and make some things happen that otherwise wouldn’t happen.” The faith in business was not shared by the entire group, however. As one participant put it, “I don’t trust business, I guess, as it comes down to it on that topic.”

As the conversation progressed, some consensus did emerge around long-term return on investment. One participant, who worked for a large private business that is a major employer in the area, noted that he could convince his company to invest in sustainable energy, but only to a point. “I can find ways, for instance, to buy more wind power for my company and I do that. But, I can’t invest, for instance, in a solar park and I’ve done numbers in the past where, you know, I cannot bring up a large capital project that has a 20-year payback.” Participants then noted and discussed various ways in which private and public sectors have similar responsibilities and limitations. A local government representative reported that they too could not propose a highly expensive city initiative to enhance sustainability and lower fossil-fuel use. “There would be no way that I can justify that in front of a city commissioner or public. And you may have stakeholders or your board, but we have to show constituents that we really ... don’t just waste money.”

The dialogue then briefly turned to energy costs and comparisons to China, noting the dearth of American-made solar panels and lack of government support for solar panel installation. This led to the final topic: ways in which private and public sectors can cooperate. It was noted that the demand for technicians to install solar panels was not being met by local community colleges. An educator in the room explained that the community college program to train solar-panel technicians was cut as their graduates could not get jobs right away back when demand for these technicians was still very low. As one participant nicely summarized, “The government should be there to offset that immediate situation ... you don’t want to be too far ahead of the supply curve or too far behind the demand curve.”

Overall, the dialogue examined how to plan for energy sustainability and respond to environmental disasters, such as recent flooding in the city. The conversation was based on the instrument and regularly came back to various prompts, which then led to discussion of further themes relevant to resiliency planning. In the end, one participant noted that there were several undiscussed prompts and they wished the conversation could continue. Given that the aim of a Toolbox dialogue is not to be the final word but rather to start conversations that extend beyond the workshop, we take this desire to continue their dialogue as a sign of success.

By exhibiting two effects sought in Toolbox workshops, the breakout sessions highlight the value and distinctiveness of philosophically structured dialogue for promoting mutual understanding, particularly when compared with more informal, unstructured conversation. First, they facilitated deep, quick engagement with an issue of common concern to the workshop participants, viz., climate resiliency in West Michigan, which is helpful when trying to promote mutual understanding among strangers in a limited time span. Again, the first dialogue immediately launched into consideration of emergency responsiveness, whereas the second used criticism of the instrument to initiate a lengthy examination of the comparative roles of government and business in preparing for and responding to climate change. Deep, quick engagement with issues related to climate resiliency was likely facilitated by a form of psychological safety produced by two general features of Toolbox workshops: first, they provide an environment in which participants are removed from concrete professional contexts where more specific concerns can get in the way of dialogue that promotes mutual understanding, and second, they leverage interests held in common by participants to coordinate their different perspectives on matters elicited by the prompts (O’Rourke and Crowley 2013). Both of these features were present in the workshops described in this paper, and it is reasonable to propose that they facilitated psychological safety, understood as “a shared belief” among workshop participants that the workshop “is safe for interpersonal risk taking” (Edmondson 1999: 350).

A second desired effect was close consideration in the breakout sessions of important matters related to climate resiliency in West Michigan (e.g., emergency preparedness, the role of government). The differences of opinion that emerged, and the respectful exploration of those differences, indicate that the prompts were successful at focusing the attention of participants on central aspects of climate resiliency, maximizing the relatively brief time these strangers would spend together in dialogue. We can contrast the breakout sessions with the more unstructured discussions that occurred at the informational meetings. Participants in the Toolbox dialogues immediately found topics of common interest and stayed focused on these topics. This was largely not the case in the informational meetings, especially the initial ones. It is worth stressing that as participants in the facilitated Toolbox dialogues were not together in prior informational meetings, this increase in conversational efficiency cannot be attributed to past interactions in those meetings.

Quick identification of common interests was due in part to the ViMM interviews revealing topics about which citizens in West Michigan have different views and values. As these are differences about core beliefs (e.g., whether government or business is more reliable in their responses to environmental disasters) and core values (e.g., whether short-term investment always outweighs long-term investment), they are differences that are likely to influence future collaborations related to climate resiliency. By bringing them to the surface for collective consideration, the workshop enhanced mutual understanding and thereby made it easier to communicate in a more transparent way about related matters going forward.

Both effects indicate that the Climate Resiliency Conference Toolbox workshop functioned like successful Toolbox workshops conducted in academic contexts. Although the data set from the workshop is small, its affinity with more traditional, academic Toolbox workshops coincides with existing evidence of effectiveness in academic contexts (e.g., Western Michigan University Evaluation Center (WMUEC) 2017, Rinkus and O’Rourke 2020, Robinson and Gonnerman 2020). As such, the conversations we facilitated in the Climate Resiliency Conference Toolbox workshop line up with the kinds of conversations we have reason to believe work well.

DISCUSSION

In this section, we consider the implications this project has for TDI and for WMSBF. For TDI, the project represented its first significant foray into a non-academic context. The Toolbox Dialogue Initiative has delivered dialogue-based workshops to academic researchers for 15 years, collecting data and publishing research articles that detail the approach and describe its findings, including its impact on research teams. This paper adds to this body of work by demonstrating that a philosophically based approach to collaborative capacity building can be extended into a non-academic, community context. It provides reason to think that philosophers can join forces with social scientists and community organizers to initiate and structure philosophical dialogue among community members that enhances mutual understanding, and thus helps to develop the capacity of a community to address complex, policy-related challenges.

More specifically, this project reinforces some of the distinctive contributions that philosophy can make in non-academic, community contexts. Although the ways in which philosophy informs Toolbox instruments and workshops are many (O’Rourke et al. 2020:51-54), we highlight just one contribution of philosophy to contexts like the one described in this paper. Climate resiliency efforts at community and regional scales require bringing together a wide range of individuals, groups, and institutions. The core beliefs, interests, and values of these players rarely align. When left unrecognized, unacknowledged, or unclear, core differences lead to confusion, divisiveness, and dysfunction. Philosophically structured dialogue can help surface such differences, and thus support mutual understanding. This stems in part from the nature of philosophical dialogue, which tends to be argumentative. In general, simply asserting a philosophical claim in a dialectical exchange is not enough. It needs to be elaborated, illustrated, and defended—contributions that in turn may be challenged by others, leading to further iterations of elaboration, illustration, and defense. In a Toolbox workshop, the argumentative nature of philosophical dialogue encourages examination of the reasons for how participants respond to Toolbox prompts, oneself included. (For more on the role of philosophical argumentation in Toolbox dialogues, see Gonnerman et al. 2015:678-679, Laursen et al. 2021:62.) When the dialogue is structured by abstract themes relevant to problems common to workshop participants, the result is an increased understanding of project-relevant beliefs, interests, and values that individual members bring to the group. And although this epistemic achievement doesn’t guarantee collaborative functionality, it clarifies existing differences and positions a group to collaborate with greater awareness of potential challenges.

From the WMSBF perspective, the Toolbox workshop worked as intended, mostly. Three limitations emerged over the course of the project. First, although the interview participants were selected to provide ideal diversity and relevance for instrument development, these individuals may have had more extreme views on climate adaptation than is representative of the region. This bias could have skewed the instrument and the resulting discussion toward certain issues that might be more germane to a discussion of politics or environmental regulation rather than climate adaptation planning. Likewise, there was consistent confusion among participants in the informational meetings about the scope of the represented issues: climate adaptation was often misunderstood as greenhouse gas mitigation and environmental conservation.

Second, the initial scale and scope of the initiative—viz., mobilize stakeholders in West Michigan under the banner of the West Michigan Climate Resiliency Framework Initiative to engage collaboratively in climate adaptation planning—was overly ambitious. This project identified a lack of opportunity for many of the participating individuals and organizations to work together in a tangible and practical way. Relative to the limited opportunities available in West Michigan for climate adaptation planning, the Toolbox workshop was tangible, practical, and grounded in the realities of West Michigan, and it provided participants a means to communicate and empathize with stakeholders who they might someday collaborate with; unfortunately, there was little opportunity to build on these connections because, at a community scale, there is limited opportunity for collaboration. Despite the lack of opportunity for further collaboration beyond the project, the Toolbox dialogues provided distinct value in understanding other interests in the community and how they may be influenced by the actions of one’s organization or institution.

Third, interviews and facilitated sessions showed that participants believed there was a need for organizations to examine climate change internally, but that organizations were not prepared to perform vulnerability assessments on their operations and assets, or to integrate climate science into their decision making, with the exception of those with the most conspicuous impacts (e.g., agribusiness and enterprises experiencing catastrophic impacts abroad from hurricanes).

The project was the first attempt to organize western Michigan stakeholders for collaborative climate adaptation planning. Although its overarching objectives proved unrealistic, the effort has been leveraged for multiple projects and initiatives in the years since. It educated businesses, institutions, and their stakeholders on potential impacts from climate change and how they might think about responding to those impacts, and it demonstrated to the sustainable business community a need for resiliency and adaptation planning. The West Michigan Sustainable Business Forum has also continued its work in climate adaptation planning, most recently through a partnership with Great Lakes Integrated Science Assessment to develop sector-relevant regional climate information and tools for organizations to internally educate decision makers. Building on the work from the Toolbox workshop, it is now seeking to define the context for climate science in decision making at businesses and institutions, demonstrating how, when, and why it should be used by these organizations. This more focused approach is a direct outcome of the lessons from the conference, limiting the scope and scale to specific industries and functions, and allowing for a more actionable planning process.

CONCLUSION

The work reported in this article demonstrates the instrumental value of philosophy as a way of framing consequential conversations about climate change, and more generally, about matters of grave public consequence. Philosophy figured into this effort in several ways. First, it informed the mental model coding that helped us understand what values animate concern among stakeholders in West Michigan about extreme weather events due to climate change. The ViMM approach is grounded in philosophical appreciation for different ways people rank their preferences about potential risks, including those that arise from a changing climate. Second, philosophy supported the discovery of fundamental assumptions and commitments that constitute the climate-relevant worldviews of stakeholder communities. Specifically, epistemology and value theory guided the search for assumptions and commitments that represent values and core beliefs in the mental model interviews. Finally, philosophy informed the development of the Toolbox instrument that structured conversational exchanges among dialogue participants motivated to participate by a desire to learn more about one another so that they could collaborate more effectively on issues related to climate adaptation. As such, the approach we have described represents a systematic and structural way of bringing philosophy to bear on policy-relevant decision-making processes.

__________
¹ There are, to be sure, significant philosophical questions about the theory behind mental models, up to and including questions about the utility of mental models in understanding decision making. Answering these questions would require an extensive engagement with the literature in psychology, cognitive science, and philosophy of mind and is beyond the scope of this paper. (For a philosophical discussion of mental models, see Thagard 2012.) Similar questions can be asked about a range of accepted social sciences methods that are used in policy analysis; for instance, there are significant philosophical questions about econometric analysis, but giving a defense of the method and using it to understand the behavior of financial markets are different projects. (On methodological defense of models in social sciences, see Morgan 2012, especially Chapter 1; Morgan is specifically concerned with economics, but much of what she says can generalize.) Thanks to an anonymous referee for pushing us to clarify this point.
² We also subjected the participants’ pre- and post-dialogue rating response scores to a series of dependent sample t-tests. We don’t report the results of these analyses in this paper because, as we now realize, they are uninformative for the purposes of assessing the ability of ViMM to inform cross-sector communication or for evaluating the extensibility of the Toolbox dialogue approach to contexts outside of academia; however, the results of these analyses are available upon request.

LITERATURE CITED

Bessette, D. L., L. A. Mayer, B. Cwik, M. Vezér, K. Keller, R. J. Lempert, and N. Tuana. 2017. Building a values-informed mental model for New Orleans climate risk management. Risk Analysis 37(10):1993-2004. https://doi.org/10.1111/risa.12743

Brister, E. 2020. Field philosophy and social justice. Social Epistemology 35(4):393-404. https://doi.org/10.1080/02691728.2020.1757176

Bruer, J. T. 1993. Schools of thought: a science of learning in the classroom. MIT Press, Cambridge, Massachusetts, USA.

Choi, S., and K. Richards. 2017. Interdisciplinary discourse: communicating across disciplines. Palgrave Macmillan, London, UK. https://doi.org/10.1057/978-1-137-47040-9

Concepción, D. W. 2004. Reading philosophy with background knowledge and metacognition. Teaching Philosophy 27:351-368. https://doi.org/10.5840/teachphil200427443

Creswell, J. W. 2007. Qualitative inquiry and research design: choosing among five approaches. Sage Publications, Thousand Oaks, California, USA.

Donovan, S. M., M. O’Rourke, and C. Looney. 2015. Your hypothesis or mine? Terminological and conceptual variation across disciplines. SAGE Open 5(2):1-13. https://doi.org/10.1177/2158244015586237

Downs, J. S., W. Bruine de Bruin, and B. Fischoff. 2008. Parents’ vaccination comprehension and decisions. Vaccine 26(12):1595-1607. https://doi.org/10.1016/j.vaccine.2008.01.011

Edmondson, A. 1999. Psychological safety and learning behavior in work teams. Administrative Science Quarterly 44:350-383. https://doi.org/10.2307/2666999

Eigenbrode, S., M. O’Rourke, J. D. Wulfhorst, D. M. Althoff, C. S. Goldberg, K. Merrill, W. Morse, M. Nielsen-Pincus, J. Stephens, L. Winowiecki, and N. A. Bosque-Pérez. 2007. Employing philosophical dialogue in collaborative science. BioScience 57:55-64. https://doi.org/10.1641/B570109

Elliott, K. C. 2017. A tapestry of values: an introduction to values in science. Oxford University Press, New York, New York, USA. https://doi.org/10.1093/acprof:oso/9780190260804.001.0001

Giere, R. N. 2004. How models are used to represent reality. Philosophy of Science 71:742-752. https://doi.org/10.1086/425063

Gonnerman, C., M. O’Rourke, S. J. Crowley, and T. E. Hall. 2015. Discovering philosophical assumptions that guide action research: the reflexive Toolbox approach. Pages 673-680 in H. Bradbury Huang and P. Reason, editors. The SAGE handbook of action research. Sage Publications, Thousand Oaks, California, USA. https://doi.org/10.4135/9781473921290.n70

Hubbs, G., M. O’Rourke, and S. H. Orzack, editors. 2020. The Toolbox dialogue initiative: the power of cross-disciplinary practice. CRC Press, Boca Raton, Florida, USA. https://doi.org/10.1201/9780429440014

Jackson, A. 2014. Top 10 West Michigan weather events of 2013: flooding, ice storm, tornadoes make the list. MLive Media Group. URL: https://www.mlive.com/news/grand-rapids/2014/01/top_10_top_west_michigan_weath.html

Johnson-Laird, P. N. 1983. Mental Models. Cambridge University Press, Cambridge, UK.

Johnson-Laird, P. N. 2010. Mental models and human reasoning. Proceedings of the National Academy of Sciences 107:18243-18250. https://doi.org/10.1073/pnas.1012933107

Jones, N. A., H. Ross, T. Lynam, P. Perez, and A. Leitch. 2011. Mental models: an interdisciplinary synthesis of theory and methods. Ecology and Society 16(1): 46. https://doi.org/10.5751/ES-03802-160146

Klein, P., M. Fatima, L. McEwen, S. C. Moser, D. Schmidt, and S. Zupan. 2011. Dismantling the ivory tower: engaging geographers in university—community partnerships. Journal of Geography in Higher Education 35(3): 425-444. https://doi.org/10.1080/03098265.2011.576337

Laursen, B. K., C. Gonnerman, and S. J. Crowley. 2021. Improving philosophical dialogue interventions to better resolve problematic value pluralism in collaborative environmental science. Studies in History and Philosophy of Science 87: 54-71. https://doi.org/10.1016/j.shpsa.2021.02.004

Lélé, S., and R. B. Norgaard. 2005. Practicing interdisciplinarity. BioScience 55(11):967-975. https://doi.org/10.1641/0006-3568(2005)055[0967:PI]2.0.CO;2

Looney, C., S. Donovan, M. O’Rourke, S. Crowley, S. D. Eigenbrode, L. Rotschy, N. A. Bosque-Pérez, and J. D. Wulfhorst. 2014. Seeing through the eyes of collaborators: using Toolbox workshops to enhance cross-disciplinary communication. Pages 220-243 in M. O’Rourke, S. Crowley, S. D. Eigenbrode, and J. D. Wulfhorst, editors. Enhancing communication and collaboration in interdisciplinary research. Sage Publications, Thousand Oaks, California, USA. https://doi.org/10.4135/9781483352947.n11

MacLeod, M. 2018. What makes interdisciplinarity so difficult? Some consequences of domain specificity in interdisciplinary practice. Synthese 195(2):697-720. https://doi.org/10.1007/s11229-016-1236-4

Mayer, L. A., K. Loa, B. Cwik, N. Tuana, K. Keller, C. Gonnerman, A. M. Parker, and R. J. Lempert. 2017. Understanding scientists’ computational modeling decisions about climate risk management strategies using values-informed mental models. Global Environmental Change 42:107-116. https://doi.org/10.1016/j.gloenvcha.2016.12.007

McInerney, D., R. Lempert, and K. Keller. 2012. What are robust strategies in the face of uncertain climate threshold responses? Climatic Change 112:547-568. https://doi.org/10.1007/s10584-011-0377-1

Morgan, M. S. 2012. The world in the model: how economists work and think. Cambridge University Press, Cambridge, UK. https://doi.org/10.1017/CBO9781139026185

Morgan, M. G., B. Fischhoff, A. Bostrom, and C. J. Altman. 2001. Risk communication: a mental models approach. Cambridge University Press, Cambridge, UK. https://doi.org/10.1017/CBO9780511814679

Nelson, M. P., and J. A. Vucetich. 2009. On advocacy by environmental scientists: what, whether, why, and how. Conservation Biology 23(5):1090-1101. https://doi.org/10.1111/j.1523-1739.2009.01250.x

O’Rourke, M., and S. Crowley. 2013. Philosophical intervention and cross-disciplinary science: The story of the Toolbox Project. Synthese 190: 1937-1954 https://doi.org/10.1007/s11229-012-0175-y

O’Rourke, M., and S. Crowley. 2020. How it works: the Toolbox dialogue method in practice. Pages 17-36 in G. Hubbs, M. O’Rourke, and S. H. Orzack, editors. The Toolbox dialogue initiative: the power of cross-disciplinary practice. CRC Press, Boca Raton, Florida, USA. https://doi.org/10.1201/9780429440014-2

O'Rourke, M., S. Vasko, C. McLeskey, and M. Rinkus. 2020. Philosophical dialogue as field philosophy. Pages 48-65 in E. Brister and R. Frodeman, editors. A guide to field philosophy: case studies and practical strategies. Routledge, New York, New York, USA. https://doi.org/10.4324/9781351169080-4

Parker, T. 2012. Loss of Michigan apples, cherries, peaches to freeze has far-reaching consequences. MLive Media Group. URL: https://www.mlive.com/news/kalamazoo/2012/05/no_michigan_apples_cherries_pe.html

Ramadier, T. 2004. Transdisciplinarity and its challenges: the case of urban studies. Futures 36:423-439. https://doi.org/10.1016/j.futures.2003.10.009

Rinkus, M. A., and M. O’Rourke. 2020. Qualitative analyses of the effectiveness of Toolbox dialogues. Pages 142-161 in G. Hubbs, M. O’Rourke, and S. H. Orzack, editors. The Toolbox dialogue initiative: the power of cross-disciplinary practice. CRC Press, Boca Raton, Florida, USA. https://doi.org/10.1201/9780429440014-10

Rinkus, M. A., and S. E. Vasko. 2020. Best practices for planning and running a Toolbox workshop. Pages 116-126 in G. Hubbs, M. O’Rourke, and S. H. Orzack, editors. The Toolbox dialogue initiative: the power of cross-disciplinary practice. CRC Press, Boca Raton, Florida, USA. https://doi.org/10.4324/9780429440014-8

Robinson, B., and C. Gonnerman. 2020. Enhancing cross-disciplinary science through philosophical dialogue: evidence of improved group metacognition for effective collaboration. Pages 127-141 in G. Hubbs, M. O’Rourke, and S. H. Orzack, editors. The Toolbox dialogue initiative: the power of cross-disciplinary practice. CRC Press, Boca Raton, Florida, USA. https://doi.org/10.1201/9780429440014-9

Robinson, B., C. Gonnerman, and M. O’Rourke. 2019. Experimental philosophy of science and philosophical differences across the sciences. Philosophy of Science 86(3):551-576. https://doi.org/10.1086/703553

Rouston, K. 2013. Analysing interviews. Pages 297-312 in U. Flick, editor. The SAGE handbook of qualitative data analysis. SAGE, London, UK. https://doi.org/10.4135/9781446282243.n20

Strawson, P. F. 2008. Freedom and resentment. Pages 1-28 in P. F. Strawson. Freedom and resentment and other essays. Routledge, London, UK. https://doi.org/10.4324/9780203882566-7

Thagard, P. 2012. The cognitive science of science: explanation, discovery, and conceptual change. MIT Press, Cambridge, Massachusetts, USA. https://doi.org/10.7551/mitpress/9218.001.0001

Torregrossa, M. 2013. Michigan weather update: how much rain did we get in total? MLive Media Group. URL: http://www.mlive.com/weather/index.ssf/2013/04/michigan_weather_update_how_mu_2.html

Tourangeau, R. 1992. Context effects on responses to attitude questions: attitudes as memory structures. Pages 35-47 in N. Schwarz and S. Sudman, editors. Context effects in social and psychological research. Springer, Berlin, Germany. https://doi.org/10.1007/978-1-4612-2848-6_4

van Fraassen, B. C. 1980. The scientific image. Oxford University Press, Oxford, UK. https://doi.org/10.1093/0198244274.001.0001

West Michigan Environmental Action Council (WMEAC). 2013. Grand Rapids climate resiliency report. West Michigan Environmental Action Council. URL: https://wmeac.org/climate-resiliency-report/

West Michigan Sustainable Business Forum (WMSBF). 2014. West Michigan climate resiliency framework initiative. West Michigan Sustainable Business Forum. URL: https://wmsbf.org/climate/

West Michigan Sustainable Business Forum (WMSBF). 2020. West Michigan sustainable business forum: about. West Michigan Sustainable Business Forum. URL: https://wmsbf.org/about/#forum

Western Michigan University Evaluation Center (WMUEC). 2017. MSU Toolbox Dialogue Initiative evaluation report. Zenodo, CERN European Organization for Nuclear Research, Geneva, Switzerland. https://doi.org/10.5281/zenodo.6555532

Whetstone, H. 2014. Weathering the climate: Michigan agriculture braces for Mother Nature with help of MSU research. Michigan State University AgBioResearch. URL: https://www.canr.msu.edu/news/weathering_the_climate_michigan_agriculture_braces_for_mother_nature_with_h