Calls for evidence-based decision and policy making in the public sector have grown louder over recent decades, even though buzzwords such as “evidence” are easily worn out and replaced by similar terms. Evidence-basing is sometimes characterized as the use of scientific evidence to support decisions (House of Commons, Parliamentary Office of Science and Technology 2011). The idea of making better use of evidence and research in policy making first became prominent in the “modernizing government” agenda of New Labour in the UK in the late 1990s. Since then it has become prominent in conservation management and more recently in sustainability studies more generally (e.g., Pullin and Knight 2001, Sutherland et al. 2004, Bilotta et al. 2014; see also Caniglia et al. 2017), particularly in connection with demands for an improved understanding of complex causal relations and a focus on actionable and transformational knowledge (Wiek et al. 2012). One of its salient features is that it seeks to replace ideologically driven politics with more rational decision making (Munro 2014). The aim is to minimize the risk of bias and inaccuracy in policy makers’ reasoning by providing an antidote to “crowd pleasing, political pandering, window dressing, and god-acting” (Pawson 2002:160). An example of such risk in natural resource management can be extracted from Fleischman and Briske’s (2016) work on professional ecological knowledge, a kind of knowledge that is claimed to be relatively insensitive to new evidence and experience. The evidence-based approach is ostensibly antidotal because it builds on a narrower, more explicit, and rigorous understanding of evidence than we normally take for granted. The approach therefore usefully contrasts with other recent and influential perspectives, such as values-led management, which might instead be founded on values that underpin steward-like relationships between people and place (Artelle et al. 2018).
Sustainability studies are often, and for good reason, informed by practical experience. In sub-Saharan Africa, it is practical experience, not scientific knowledge, that is dominant in almost all practical aspects of agriculture, even though the former is politically and ideologically marginalized (Hountondji 2002). Small-scale farming in particular—one of the most common forms of employment in the world—is still based largely on knowledge acquired through practical experience (Altieri 2004, Akullo et al. 2007), and some of the most sustainable farming systems in the world are entirely based on knowledge and practices acquired through the practical experience of generations of farmers, the so-called Globally Important Agricultural Heritage Systems (GIAHS; Koohafkan and Altieri 2011, Koohafkan and Cruz 2011). Sustainability scientists often try to uncover local concerns and (sustainability) challenges, and to understand complex and site-specific causal relationships. It is not only people’s effect on the environment that is studied. The environment’s effect on people is investigated as well, as are social relations, e.g., between company employees and village inhabitants, and environment-environment relations, e.g., farmland expansion and its effects on the migration paths of animals. If we want to understand the challenges presented by sustainability, and if our goal is to create sustainable solutions, we need to draw on both scientific and local knowledge and understand local concerns.
Here, however, a real tension emerges with the rigorous and defining view of evidence often assumed by the evidence-based movement. For instance, the Evidence-Based Policymaking Collaborative (https://www.evidencecollaborative.org/) describes four principles that policy makers, agency heads, and other public leaders can use to improve results in the public sector: (1) build and compile rigorous evidence about what works, including costs and benefits; (2) monitor program delivery and use impact evaluation to measure program effectiveness; (3) use rigorous evidence to improve programs, scale what works, and redirect funds away from consistently ineffective programs; (4) encourage innovation and test new approaches (Evidence-Based Policymaking Collaborative 2016). A similar approach, more strongly focused on evidence-based decision making in conservation, is formulated in Pullin and Knight (2003):
Within these approaches, the integration of evidence, interpreted in this more rigorous way, with practical experience becomes a challenge, and the risk is that practical experience will have no real place as evidence in evidence-based sustainability studies. This would create a number of difficulties for the position, difficulties the seriousness of which need to be scrutinized. It is clear that if this exclusion is indeed inevitable evidence-based sustainability might end up reinforcing the dangerous division between two knowledge systems that has distorted and hampered sustainability issues in agriculture and elsewhere. Many GIAHS are nowadays threatened by agricultural modernization policies, underpinned by modern science and technology, and hence in need of protection. The FAO (http://www.fao.org/giahs/en) and UNESCO have taken steps to protect such knowledge systems with the rationale that they can provide sources of knowledge for reinventing agriculture elsewhere (Koohafkan and Cruz 2011). Equally problematic would be the response that in view of their apparent incompatibility with practical experience, evidence-based sustainability studies, in their different varieties, have to be abandoned. Our intention, then, is not to argue against evidence-based sustainability studies. It is to argue for a way of adapting them so that they can relate in a meaningful way to local knowledge.
We start with the challenge. The tension within evidence-based sustainability studies is best understood by reviewing some of the challenges and pitfalls of its forerunner, evidence-based medicine.
The expression “evidence-based medicine” was coined first by Gordon Guyatt, in 1990 in the course of his argument for an alternative, supposedly better way of making clinical decisions in health care (Daly 2005). The group of clinical epidemiologists to which he belonged wanted to position themselves against the received practice of medical education, which they thought bred authority-dependent, uncritical physicians who trusted clinical experience too much. Similar complaints were later voiced in conservation practice. For instance, Sutherland et al. (2004:305) claim that “much of current conservation practice is based upon anecdote and myth rather than upon the systematic appraisal of the evidence, including experience of others who have tackled the same problem.” Noting the development of evidence-based practice in health care brought about by “the application of scientific method in determining the optimal management of the individual patient” (Guyatt 1991:A16), Sutherland et al. (2004:305) contend that conservation needs “a similar radical revolution.”
Clinical epidemiologists and conservation biologists both sought to construct hierarchies of evidence so that evidence could be summarized. In these hierarchies, the evidence obtained from well-designed clinical trials (randomized controlled trials, or RCTs) was typically rated high and (unsystematic) practical experience, intuition, and theoretical reasoning were not (Guyatt et al. 1992).
The hierarchies were codified in instruments such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). GRADE is now a familiar system for synthesizing and rating the quality of evidence of different sorts, and it has been used to an extent in sustainability studies (Bilotta et al. 2014). It does more than codify a hierarchy of evidence; it also provides acceptance rules for evidence.
Historically, the sort of integrative tension that we want to focus on here was especially prominent in the shift in evidence-based medicine to a position whereby policy architects and decision makers became primarily evidence users rather than evidence-based practitioners. This shift divorced the movement from the original, but often unrealistic, idea that each physician should critically and systematically evaluate the evidence in a scientific way and integrate it with personal experiences and patients’ preferences; it moved it toward a position where each doctor should be an “evidence user” with “a readiness to identify evidence-based sources which summarize the evidence for them” (Guyatt, as quoted in Daly 2005:91). Likewise, in evidence-based conservation it may never have been believed that the conservation manager should be more than an evidence user (Pullin and Knight 2001).
The shift to evidence users effectively drives a qualitative wedge between (research) evidence and practical experience. Following it, the difference between evidence and practical experience is no longer a difference of degree but one of kind. Research evidence becomes “the” evidence. Practical experience becomes something else.
Pace Pullin and Knight (2001), and bearing in mind that practical experience is no longer evidentially relevant once we become wedded to the idea of rigorous evidence, the evolution of the decision maker into an evidence user clearly reveals potential shortcomings of the evidence-based perspective as a decision strategy in many sustainability cases.
Rigorous evidence might be lacking. Where it does exist, it may be relevant to the drawing of well-grounded conclusions at population level but incapable of being used in order to answer specific questions at a local level, i.e., there may a problem of external validity, and indeed other problems of implementation.
Nevertheless, from the point of view of a policy maker, the focus of assessing the quality of a study from environmental science is to establish (i) how near the truth its findings are likely to be, and (ii) how relevant and transferable the findings are to the particular setting or population of policy interest. (Bilotta et al. 2014)
Moreover, some sustainability questions are simply not about what kinds of intervention work, and where this is the case the evidential hierarchies need to be revised (the evidence is partly of the wrong kind). Trials “can run you ragged on the trivial” (Fraser Mustard, as quoted in Daly 2005:108).
That relevant research evidence for decision making is sometimes scarce is not surprising when we consider why we wanted evidence in the first place. A lot of research evidence is of poor quality or simply irrelevant for a particular decision or policy purpose, even if it remains of high quality for other purposes. Some questions might be important for practical matters but scientifically peripheral, and vice versa.
In medicine, the pharmaceutical industry effectively, in part, at least, compensates for this asymmetry. In other fields, notably involving sustainability and policy issues, relevant and sufficient rigorous evidence is often lacking. The implication is that the rigorous evidence we have acquired might answer the wrong question.
Thus, in the majority of cases not even a well-designed RCT (assuming such a study is possible) provides all the answers that we need for policy design and decision making. A well-designed RCT will test, quite efficiently, whether a particular intervention is effective. It will reveal a causal connection. But it does not provide information about the other kinds of environment in which this causal connection will be exhibited. Nor does it necessarily help us to identify the items of causal connection. Typically, then, the RCT shows that the intervention worked in the “experimental” setting. It gives no indication as to (a) whether it would work in a new context, and (b) whether a certain action is an instance of the intervention. For that, extra assumptions are needed, assumptions about the causal principles in the experimental setting and the new context (see also Cartwright and Hardie 2012). Some way of determining whether a certain action is the kind of intervention the causal principle latches on to is also needed. Some causal principles are the same throughout space-time; but most are not.
For instance, Varroa destructor is known to be the most devastating pest of western honey bees (Apis mellifera). Varroa mites are ectoparasites that feed on the hemolymph of immature and adult honey bees. They reproduce on developing bee larvae. However, researchers have found genetic holes in the pest’s armor, and recently, double-stranded RNA that causes mortality in varroa has been identified (Huang et al. 2017). This finding, including high quality mechanistic evidence, leaves us with an important question: Would feeding bee colonies in the farms with single gene solutions reduce the problem? Potentially, to answer this question more evidence and evidence of different kinds are needed. This is because high internal validity, i.e., proof that the experimental stimulus made a significant difference in this specific instance, does not guarantee external validity, i.e., that this effect can be generalized to other populations, settings, and variables, as Donald Campbell (1957) discussed long ago. There is no guarantee that feeding is the right intervention, and, of course, we have no assurance in advance that the bees are of the right kind, i.e., two problems of implementation.
Even if this asymmetry between the internal and external validity of conclusions is amplified in laboratory studies, similar problems, versions of the inductive problem, haunt all science, RCTs conducted in the clinics or in the field included. The evidence-base is thus often too narrow for the policy maker.
Finally, the question may not be what works? It may be a question of some other type. Evidence in evidence-based policy has been tailored to yield optimal evidence on the effectiveness of potential interventions. However, policy is not, and should not always be, about what works, not in a narrow interventionist sense, at least. This kind of critique of the evidence-based movement can be found in several domains, one of which is education. The critique there often begins by drawing attention to the differences between contexts where interventions and their effectiveness are relevant to focus on, i.e., questions about what works, and the school where interventions and their effects are only part of the totality one is managing (e.g., Biesta 2005). It may even be that the causal processes, even when they are conceived of as more complex than interventions and their effects, are relatively uninteresting compared with symbolically mediated cognitive processes and processes of improved ethical awareness. However, whether these processes are sufficiently similar to the kind of causal relations RCTs are tailored to identify is an open question. Sometimes, the upshot of such observations will be the insight that the evidential hierarchies have promoted the wrong methodologies (e.g., Petticrew and Roberts 2003).
For example, many conditional cash transfer (CCT) programs (where people on low incomes receive cash to cover school fees) have been assessed using RCT, and the assessments often generate positive results, for example, that school attendance rates increase. However, because the assessments do not discuss the alternative of making education free of charge, in essence the RCTs are being used to promote a particular ideological approach by testing only one kind of intervention. Most probably it would be better to provide free education, but that would be at odds with the current ideology.
Importantly, a narrow focus on what works is alien to sustainability issues. Ideally, the evidence will be about what works sustainably, at least. In order to accomplish that, attention has to be given to how a proposed solution works. Knowing how is crucial if we are to develop understanding, as Norgaard (2001:58) emphasizes when he speaks of “the importance of understanding to achieving sustainability.” Sometimes the key question might even be: What is going on in the particular environment? But there is no obvious connection between answers to questions of these three types. For instance, complex or detailed questions about how something works are far from easy to answer with evidence about what works. How-questions typically need more structural, mechanistic answers, i.e., less straightforwardly causal answers. Similarly, questions about what is going on in a particular environment might require more descriptive answers than those provided in the form of causal inference tickets from cause to effect. Such questions typically depend on a more flexible research strategy than that involved RCTs, or in large-N research in general.
The problems we have identified clearly spotlight the need for policy makers managing sustainability issues to be evidence-based practitioners rather than evidence users. And for the evidence-based practitioner it is obvious that the process has to include elements that might lead to the tailoring, and sometimes the discovery, of new interventions rather than implementation and evaluation of existing policy. But evidence-based policy is a vehicle for neither tailoring nor discovery. It is rather telling that innovation comes as step four in The Evidence-Based Policymaking Collaborative’s list.
Hence, when requests for answers that differ, qualitatively, from the answers to narrow what-works questions are made, typically there is not enough research evidence of the right kind, i.e., the kind promoted by the evidence-based movement, to tackle them. The implication is that some of the most relevant questions raised in calls for evidence-based public decision and policy making cannot be answered by relying exclusively on the assessments of evidence governed by GRADE and similar instruments. There simply is not enough relevant evidence of the right kind to make it work. And in some cases, there never will be.
In other words, the risk is obvious that, instead of managing and answering the important sustainability questions with evidence-based solutions, we will provide the answers for which there is rigorous evidence (note the imperative “ask an answerable question” in Pullin and Knight 2003). As a consequence, evidence-based sustainability studies run the risk of answering the wrong question.
Note that the risk of answering the wrong question generated by the demand for but absence of rigorous evidence does not necessarily entail that relevant research is nonexistent. Nor does it necessarily entail that relevant (local) knowledge is lacking:
Everyone in society has some knowledge special to the varied and particular contexts in which they work and inevitably learn, and sometimes this particular knowledge can be extremely important. (Norgaard 2001:60)
This is not to say that such common-sense naturalistic observation is objective, dependable, or unbiased. But it is all that we have. It is the only route to knowledge-noisy, fallible, and biased though it be. (Campbell 1975:178)
Instead, the problem for the evidence-based perspective is that adequate mechanisms for integrating practical experience seem to be lacking. Some critics argue that the required mechanisms are more or less necessarily excluded (Adams and Sandbrook 2013), and that proponents of evidence-based conservation view practical experience as no more than myth. In reply, the proponents have insisted that this does not reflect their ambition (Haddaway and Pullin 2013). Pullin and Salafsky (2010), for example, have called for more information to be recorded by practitioners, and the case for including local and expert knowledge has certainly been made in the evidence-based conservation literature. Haddaway and Pullin (2013) state that the integration is normally tackled at the consultation stage, during question formulation, where the Collaboration for Environmental Evidence (CEE) Guidelines state that all relevant stakeholders should be consulted (CEE 2013).
It must, therefore, be crucial for an adaptive version of evidence-based sustainability studies to ensure that the varieties of local knowledge that evidence-based sustainability or policy cannot traditionally capture, the kind we call practical experience, are harnessed in a more integrated way for scientific knowledge production. How to integrate the two sources of evidence is one of the most fundamental epistemological problems society is facing. What kind of practical experience and scientific evidence it is that we need to integrate is an almost equally challenging question? Fortunately, harnessing can be secured, and in more ways than one. The best way to proceed depends on whether we are evidence-based practitioners or evidence users.
In both cases we might agree, if necessary, that although what is added by practical experience is not science proper, it is, arguably, evidence nevertheless. For sustainability scientists (1) is clearly the more appropriate option. This entails that practical experience, and local knowledge more generally, should be integrated early on in the research process. It is not a matter of integrating readymade research and practical experience. However, this implies that several earlier takes on the problem become less timely. To quote an early formulation:
Indigenous knowledge (IK) is local knowledge - knowledge that is unique to a given culture or society. IK contrasts with the international knowledge system generated by universities, research institutions and private firms. It is the basis for local-level decision-making in agriculture, health care, food preparation, education, natural resource management, and a host of other activities in rural communities. Such knowledge is passed down from generation to generation, in many societies by word of mouth. Indigenous knowledge has value not only for the culture in which it evolves, but also for scientists and planners striving to improve conditions in rural localities. (Warren, as quoted in Agrawal 1995:416)
According to (1), the separation of science and local knowledge implied in these remarks is misguided. Similarly, the divisions and discontinuities between western scientists and indigenous people, and, more generally, between any two particular groups, described by (Norgaard 2001) must be partly overstated if (1) is to be a viable alternative. Finally, the “other knowledge systems” in Nakashima et al. (2000) must not be entirely isolated from scientific knowledge production. In saying this, we do not downplay the importance of observing two knowledge systems. In mainstream agricultural policies in Africa, for instance, local knowledge is often seen as a (or even the) problem (Norgaard 1984), and the solution is to modernize agriculture by harnessing modern science and technology to boost crop yields (Juma et al. 2001, Tomlin 2006, Toenniessen et al. 2008, Juma 2015). According to one of the most influential scholars in development economics, John Mellor, “farmers may well innovate but innovation is generated locally from local practices and is innately slower paced than that from modern research institutions. It is not embodied in purchased inputs” (Mellor 2017:75). But progress in the quest for modernization has been slow. Often it has not been made at all, as in the case of Malawi (Messina et al. 2017), or has been downright deleterious, some would argue (Bryceson 2002). Efforts in Uganda to aggressively modernize the agricultural sector are a telling example of the clash between different views on knowledge (World Bank 2007).
The trick here, if (1) is to be successful, is to integrate science and practical experience without fully integrating scientific theories and local knowledge systems, a task, as we should acknowledge, that often leads to conflict and inconsistency (Aikenhead and Jegede 1999). That inconsistency is to be expected, and the position that this risk should be avoided, at least from the point of view of an otherwise well-functioning science, is argued in another paper in this special feature (Persson et al. 2018).
A substantial part of the solution to the above problem is, surprisingly perhaps, almost trivial. Rather than trying, and failing, to integrate entire belief systems, we should settle for less. In sustainability studies this means we should try to establish reliable mechanisms for problem-feeding (Thorén and Persson 2013) between the two domains instead of attempting to integrate them. Problem-feeding is a form of, typically interdisciplinary, exchange in which the central object of exchange is a problem, or set of problems, rather than, say, evidence, a concept, model, or theory. The original idea is that problems occasionally arise within a discipline1 in which they cannot be solved easily (Thorén and Persson 2013; see also Sherif and Sherif 1969, Darden and Maull 1977) and are therefore exported to another discipline2. In bilateral problem-feeding, the solutions to the problems are then fed back (to discipline1). The simplest type of bilateral problem-feeding would thus consist of four steps (Wahlberg and Persson 2017): (1) the formulation of the problem in context1; (2) the uptake (and reformulation) of the problem in context2; (3) the solution of the problem in context2; (4) the (reformulation and) acceptance of the solution in context1.
We can illustrate how local knowledge can be harnessed for scientific knowledge production in this way by examining the depiction of narratives of socio-environmental change using participatory art (Johansson and Isgren 2017). The participatory art method used was developed by a LUCID (Lund University Centre of Excellence for Integration of Social and Natural Dimensions of Sustainability) PhD scholar in order to uncover local concerns, coproduce knowledge, and disseminate local experiences of land use and land cover change (LULCC) to a wide audience, i.e., academics, policy makers, and the public. This section offers a brief overview of the method and explains how it could be usefully applied in sustainability science.
The aim of the study we take as our case probe is to understand sustainability challenges connected with LULCC and how such change affects people and the environment in areas experiencing land grabbing (de Schutter 2011, Borras et al. 2011). The method was developed as an alternative to remote sensing commonly used to quantify and explore LULCC. Where remote sensing is used the researcher often determines the most important challenge that he or she wishes to understand. This can be problematic because the scale of analysis or the land cover categories selected may not be the most relevant for people in the local context. They may not therefore be crucial for sustainable development in the local area. Causal effects and complex feedbacks between people and the environment might remain undiscovered if the researcher focuses too heavily on answering a predetermined set of research questions. In Johansson and Isgren (2017), participatory research methods building on focus group discussions and subsequent painting workshops were used in order to identify the key challenges experienced by people in the area.
Participatory art allows people who are dependent on local natural resources and vulnerable to environmental change to describe and depict the challenges and benefits of LULCC as they see them. The research focus and questions are heavily influenced by participants, and this helps the researcher to identify the most pressing sustainability challenges in the local context as well as to identify complex feedbacks and interactions between the local society and the environment.
The fieldwork was conducted in 2015 over a two-month period in the Kilombero Valley, Tanzania, in villages that lease land to foreign agribusinesses. Village land has been leased with long-term contracts by two different companies: Kilombero Plantations Limited (rice production) since 2008 and Kilombero Valley Teak Company (teak production) since 1992.
Interviews, focus group discussions, narrative walks, and painting workshops were the main methods used to investigate the past, present, and future socio-environmental situation of the area. These methods do not quantify change. Instead they indicate how people experience their situation, because participants are allowed to steer the discussion to what they judge to be important to emphasize. Perceptions of change, independently of their truth, can tell us about the socio-political situation of people who experience land grabbing, and why certain processes take place.
Focus group discussions were arranged in five villages where land has been acquired by one of the two companies. The groups were of approximately 12 people who had been selected to represent the dominant livelihoods in the area (farmers, fishermen, pastoralists), men and women, young and old (for past and present engagement experiences). During interviews and discussions, the participants were asked to explain what resources they obtain from nature, and then to discuss whether, and if so how, these resources have changed. As a result, there was a discussion about what the participants considered to be the main reasons for change. Finally, the researchers and focus group participants talked about future aspirations and how to reach that future.
After this the idea of painting narratives of change was introduced to the focus group participants, and the farmers, fishermen, and pastoralists painted stories of the past, present, and future with the guidance of a Tanzanian artist who instructed them on how to visualize the stories as Tanzanian Tinga-Tinga paintings. In total, six paintings representing the past, i.e., the period immediately before the arrival of the agribusinesses, the present, and the future were created for the two case study areas.
The paintings enabled communication in general and problem-feeding in two particular ways. First, the process of painting created a platform for the researcher to stay in the village for a fortnight and become more familiar with the local people and the environment than she would have done had she stayed for a day. The slow process of making paintings made it possible to peer-review the stories in the field, allowing passersby to confirm, reject, or adjust the visualized pattern. Second, the paintings were used successfully in knowledge dissemination in the form of presentations and art exhibitions staged at international scientific conferences and the National Museum and House of Culture in Dar es Salaam, Tanzania. The exhibitions and presentations spread local knowledge and disseminated experiences of land use and land cover change.
Several local perceptions and instances of practical experience in this case pointed to sustainability challenges that the scientist would perhaps struggle to identify as an outsider. For example, there were local perceptions that the insects had changed since natural forests were cleared for teak plantations. Farmers believed that their crop yields had decreased because the pollinators had changed. However, because the yields of the crops being cultivated did not rely on pollination, the reason for change must have been something else, although the observation that the insects had changed may have been true. This raised new questions about why the farmers made that inference, and what the real reason for lower crop yields was.
Another local issue that is difficult to observe and fully understand as an outsider is rapid deforestation. Local narratives and field observations point to accelerated deforestation over the past 10 years due to population growth, farmland expansion, and illegal logging. It has been difficult to find scientific evidence for these changes, however, and an initial remote sensing analysis of land cover change has shown that forest cover has not changed in the area. The changes in forest cover in this particular area are difficult to observe with remote sensing as a result of the poor resolution of existing satellite images, and cloudiness of available satellite images.
In another strange side-story that emerged during the painting process, it was found that some participants painted people digging a hole at the large-scale plantation site. The people and the hole were shown as surrounded by a fence. The participants said that the company asks them to dig a hole to look for groundwater, but that after digging for a while they are forced to dig a new hole in another place. They were not persuaded that the company was interested in finding groundwater in this way: the plantation is located in a wet-land area, so water is everywhere. They believed that the company was searching for minerals or even uranium.
Two examples of problem-feeding in this example are the following:
Problem A (community concern): Crop yields have decreased because insect population composition has changed following teak plantation.
Problem A1 (sustainability scientists): Crop yields have decreased (but the crop does not need to be pollinated).
Problem A2 (sustainability scientists): Has insect population composition changed?
Problem A3 (sustainability scientist): Are there other potential problems that could have been caused by changing insect populations, e.g., agricultural pests?
Problem A4 (community concern): Are there other important changes to your crops?
Problem B (community concern): There is rapid deforestation due to population growth, farmland expansion, and illegal logging.
Problem B1 (sustainability scientists): There is perceived deforestation (but forest cover has increased).
Problem B2 (sustainability scientists): Have the forests become patchier?
Problem B3 (sustainability scientist): Are there particular (useful) species that have disappeared even if the total forest cover has increased?
Problem B4 (community concern): What are the most useful tree species, and have they disappeared?
Thus, the problem, once extracted, needs to be fed, without distorting it too much (Wahlberg and Persson 2017), into the scientific domain. Furthermore, often, and ideally, problem-feeding is bilateral. Often, something more than the problem needs to be fed into the scientific domain. Observations, even evidence, that might be difficult to produce using standard scientific methods may be needed as well. Thus problems A2 and B2 could be fed to the community for further evidence-acquisition.
The main difference between the participatory art technique and typical evidence-based approaches is that, in the former, the problems focused upon are extracted from local concerns; with the exception of the focus on land use change and its socio-environmental consequences, they are not determined by the scientist. The first two steps in the process of problem-feeding are thus clearly involved. And if we really want to solve sustainability challenges, we also need to understand what those challenges are in the local context. Using participatory art to discuss and develop future scenarios might be an important contribution to sustainable policy making because the solutions would be based on local experiences and concerns.
Agriculture is an interesting example of the way in which the shift from knowledge acquired through practical experience to scientific knowledge can transform practices. Farming was once an economic activity that relied heavily on knowledge acquired through practical experience. As such there was reluctance when attempts were made to integrate it into capitalist modes of production (Kloppenburg 2005). Before the introduction of modern inputs such as synthetic fertilizers and fossil fuel-based machinery, the sustainability of agriculture was secured through the photosynthetic conversion of solar energy, and this provided food for people and feed for domestic animals. It was simply a thermodynamic necessity for more food calories to be produced on farms than the farmers had invested in growing the food (Crews et al. 2018). Key to successful agriculture, under such conditions, was the maintenance of soil fertility through various practices whose effect was to circulate nutrients from harvested crops, via people and livestock, back into the soil. Most of these practices relied entirely on knowledge acquired through practical experience by generations of farmers rather than scientific knowledge: the early scientific inputs to agriculture, such as stump-pullers, improved ploughs, and threshing machines were almost exclusively mechanical, rather than biological or chemical (Huffman and Evenson 2008). Today the priorities have changed, and we are in a situation where seeds and agrochemicals dominate the scientific inputs to agriculture, most of which are detrimental to the long-term integrity of agro-ecosystems (Foley et al. 2011, Crews et al. 2018)
The potential pitfalls of taking practical experience into account are, of course, to some extent, the very flaws that prompted the evidence-based movement to begin with (Munro 2014):
When we integrate practical experience and scientific evidence through the mechanism of problem-feeding rather than in a more direct way, we avoid these flaws, or at least significantly mitigate the risk that they will arise. We also circumvent what is perhaps the most serious problem with a practice-based approach, a problem the approach shares, to some extent, with every other kind of evidence-based thinking that focuses narrowly on experimental techniques such as RCT as a source of evidence. The problem is that the evidence from practical experience may be too shallow to generate explanations. This difficulty makes it possible to see clearly the advantages of harnessing local knowledge for scientific knowledge production in a structurally more complicated way than that achieved by simply adding practical experience as evidence. By scaling up and scaling out, science can add explanation to any regularities discovered at the local level through practical experience. In a Senegalese study of people’s vulnerability to climate change and their coping mechanisms, Tschakert (2007) applied the technique of conceptual mapping among villagers (“nonexperts”) and extension agents (“experts”). By comparing their respective conceptual understandings of the causes and impacts of climate change, she derived a more complete and informed understanding of the vulnerabilities and potential remedies.
The evidential hierarchies defining the evidence-based movement, especially in health care, normally render practical experience unacceptable as evidence. Because sustainability studies often aim to uncover local concerns and challenges, and also because those conducting the studies need to understand complex and site-specific causal relationships, a tension then emerges within evidence-based studies of sustainability. Yet, the evidence-based approaches have many advantages. Adaptive approaches to the evidence-based study of sustainability in which local knowledge is harnessed need to be developed. There are many reasons why practical experience should not be simply added on to scientific evidence. As against the integration of entire belief systems, scientific evidence and indigenous knowledge, for example, we have argued that problem-feeding based on practical experience is preferable. Participatory art is a promising method of problem-feeding. In some cases it will also provide crucial evidence.
Lennart Olsson and Johannes Persson conceived the idea. Johannes Persson wrote the first draft and coauthored the final manuscript with Emma Johansson and Lennart Olsson. Emma Johansson contributed ideas to the first draft, contributed a case probe, and coauthored the final manuscript. Lennart Olsson contributed ideas to the first draft, contributed a case probe, and coauthored the final manuscript. The authors declare that they have no competing interests.
The authors want to thank Niklas Vareman and Paul Robinson in particular for a number of valuable comments. The research has been funded by the following research projects: Swedish Research Council Formas Linnaeus grant LUCID, Lund University Centre of Excellence for integration of Social and Natural Dimensions of Sustainability (259-2008-1718) [to J.P., E.J., and L.O.] and the Swedish Foundation for Humanities and Social Sciences, grant number M14-0138:1 [to J.P.].
Adams, W. M., and C. Sandbrook. 2013. Conservation, evidence and policy. Oryx 47:329-335. http://dx.doi.org/10.1017/S0030605312001470
Agrawal, A. 1995. Dismantling the divide between indigenous and scientific knowledge. Development and Change 26(3):413-439. http://dx.doi.org/10.1111/j.1467-7660.1995.tb00560.x
Aikenhead, G. S., and O. J. Jegede. 1999. Cross-cultural science education: a cognitive explanation of a cultural phenomenon. Journal of Research in Teaching 36(3):269-287. http://dx.doi.org/10.1002/(SICI)1098-2736(199903)36:3<269::AID-TEA3>3.0.CO;2-T
Akullo, D., R. Kanzikwera, P. Birungi, W. Alum, L. Aliguma, and M. Barwogeza. 2007. Indigenous knowledge in agriculture: a case study of the challenges in sharing knowledge of past generations in a globalized context in Uganda. World Library and Information Congress: 73rd IFLA General Conference and Council, Durban, South Africa.
Altieri, M. A. 2004. Linking ecologists and traditional farmers in the search for sustainable agriculture. Frontiers in Ecology and the Environment 2:35-42. http://dx.doi.org/10.1890/1540-9295(2004)002[0035:LEATFI]2.0.CO;2
Artelle, K. A., J. Stephenson, C. Bragg, J. A. Housty, W. G. Housty, M. Kawharu, and N. J. Turner. 2018. Values-led management: the guidance of place-based values in environmental relationships of the past, present, and future. Ecology and Society 23(3):35. http://dx.doi.org/10.5751/ES-10357-230335
Biesta, G. 2005. Knowledge production and democracy in educational research: the case of evidence-based education. South African Journal of Higher Education: Special Edition 19:1334-1349.
Bilotta, G. S., A. M. Milner, and I. L. Boyd. 2014. Quality assessment tools for evidence from environmental science. Environmental Evidence 3(14):1-14. http://dx.doi.org/10.1186/2047-2382-3-14
Borras Jr, S. M., R. Hall, I. Scoones, B. White, and W. Wolford. 2011. Towards a better understanding of global land grabbing: an editorial introduction. Journal of Peasant Studies 38(2):209-216. http://dx.doi.org/10.1080/03066150.2011.559005
Bryceson, D. F. 2002. The scramble in Africa: reorienting rural livelihoods. World Development 30:725-739. http://dx.doi.org/10.1016/S0305-750X(02)00006-2
Campbell, D. T. 1957. Factors relevant to the validity of experiments in social setting. Psychological Bulletin 54(4):297-312. http://dx.doi.org/10.1037/h0040950
Campbell, D. T. 1975. “Degrees of freedom” and the case study. Comparative Political Studies 8(2):178-191. http://dx.doi.org/10.1177/001041407500800204
Caniglia, G., N. Schäpke, D. J. Lang, D. J. Abson, C. Luederitz, A. Wiek, M. D. Laubichler, F. Gralla, and H. von Wehrden. 2017. Experiments and evidence in sustainability science: a typology. Journal of Cleaner Production 169:39-47. http://dx.doi.org/10.1016/j.jclepro.2017.05.164
Cartwright, N., and J. Hardie. 2012. Evidence-based policy: a practical guide to doing it better. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/acprof:osobl/9780199841608.001.0001
Cleveland, C. J., D. I. Stern, and R. Costanza, editors. 2000. The economics of nature and the nature of economics: advances in ecological economics. Edward Elgar, Cheltenham, UK.
Collaboration for Environmental Evidence (CEE). 2013. Guidelines for systematic review and evidence synthesis in environmental management. Version 4.2. CEE, Bangor University, Gwynedd, UK. [online] URL: http://www.environmentalevidence.org/Documents/Guidelines.pdf
Crews, T. E., W. Carton, and L. Olsson. 2018. Is the future of agriculture perennial? Imperatives and opportunities to reinvent agriculture by shifting from annual monocultures to perennial polycultures. Global Sustainability 1:e11. http://dx.doi.org/10.1017/sus.2018.11
de Schutter, O. 2011. How not to think of land-grabbing: three critiques of large-scale investments in farmland. Journal of Peasant Studies 38:249-279. http://dx.doi.org/10.1080/03066150.2011.559008
Daly, J. 2005. Evidence-based medicine and the search for a science of clinical care. University of California Press, Berkeley, California, USA.
Darden, L., and N. Maull. 1977. Interfield theories. Philosophy of Science 44(1):43-64. http://dx.doi.org/10.1086/288723
Evidence-Based Policymaking Collaborative. 2016. Principles of evidence-based policymaking. [online] URL: https://www.evidencecollaborative.org/
Fleischman, F., and D. D. Briske. 2016. Professional ecological knowledge: an unrecognized knowledge domain within natural resource management. Ecology and Society 21(1):32. http://dx.doi.org/10.5751/ES-08274-210132
Foley, J. A., N. Ramankutty, K. A. Brauman, E. S. Cassidy, J. S. Gerber, M. Johnston, N. D. Mueller, C. O'Connell, D. K. Ray, P. C. West, et al. 2011. Solutions for a cultivated planet. Nature 478:337-342. http://dx.doi.org/10.1038/nature10452
Guyatt, G. 1991. Evidence-based medicine. ACP Journal Club (supplement 2 to Annals of Internal Medicine) 114:A16.
Guyatt, G., J. Cairns, D. Churchill, D. Cook, B. Haynes, J. Hirsh, J. Irvine, M. Levine, J. Nishikawa, D. Sackett, et al. 1992. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 268(17):2420-2425. http://dx.doi.org/10.1001/jama.1992.03490170092032
Haddaway, N., and A. S. Pullin. 2013. Evidence-based conservation and evidence-informed policy: a response to Adams & Sandbrook. Oryx 47(3):336-338. http://dx.doi.org/10.1017/S0030605313000811
Hountondji, P. J. 2002. Knowledge appropriation in a post-colonial context. Pages 23-38 in C. A. Odora Hoppers, editor. Indigenous knowledge and the integration of knowledge systems: towards a philosophy of articulation. New Africa Books, Claremont, South Africa.
House of Commons, Parliamentary Office of Science and Technology. 2011. Evidence-based conservation. PostNote 379. Parliamentary Office of Science and Technology, Houses of Parliament, London, UK.
Huang, Z. Y., G. Bian, Z. Xi, and X. Xie. 2017. Genes important for survival or reproduction in Varroa destructor identified by RNAi. Insect Science. http://dx.doi.org/10.1111/1744-7917.12513
Huffman, W. E., and R. E. Evenson. 2008. Science for agriculture: a long-term perspective. John Wiley & Sons, Hoboken, New Jersey, USA. http://dx.doi.org/10.1002/9780470752555
Johansson, E. L., and E. Isgren. 2017. Local perceptions of land-use change: using participatory art to reveal direct and indirect socioenvironmental effects of land acquisitions in Kilombero Valley, Tanzania. Ecology and Society 22(1)3. http://dx.doi.org/10.5751/ES-08986-220103
Juma, C. 2015. The new harvest: agricultural innovation in Africa. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/acprof:oso/9780190237233.001.0001
Juma, C., K. Fang, D. Honca, J. Huete-Perez, V. Konde, S. H. Lee, J. Arenas, A. Ivinson, H. Robinson, and S. Singh. 2001. Global governance of technology: meeting the needs of developing countries. International Journal of Technology Management 22:629-655. http://dx.doi.org/10.1504/IJTM.2001.002982
Kloppenburg, J. R. 2005. First the seed: the political economy of plant biotechnology. University of Wisconsin Press, Madison, Wisconsin, USA.
Koohafkan, P., and M. A. Altieri. 2011. Globally important agricultural heritage systems: a legacy for the future. Food and Agriculture Organization of the United Nations, Rome, Italy.
Koohafkan, P., and M. J. D. Cruz. 2011. Conservation and adaptive management of globally important agricultural heritage systems (GIAHS). Journal of Resources and Ecology 2:22-28.
Mellor, J. W. 2017. Agricultural development and economic transformation: promoting growth with poverty reduction. Palgrave Macmillan, London, UK. http://dx.doi.org/10.1007/978-3-319-65259-7
Messina, J. P., B. G. Peter, and S. S. Snapp. 2017. Re-evaluating the Malawian farm input subsidy programme. Nature Plants 3:17013. http://dx.doi.org/10.1038/nplants.2017.13
Munro, E. 2014. Evidence-based policy. Pages 48-67 in N. Cartwright and E. Montuschi, editors. Philosophy of social science: a new introduction. Oxford University Press, Oxford, UK.
Nakashima, D., L. Prott, and P. Bridgewater. 2000. Tapping into the world’s wisdom. UNESCO Sources 125:11-12.
Norgaard, R. B. 1984. Traditional agricultural knowledge: past performance, future prospects, and institutional implications. American Journal of Agricultural Economics 66:874-878. http://dx.doi.org/10.2307/1241018
Norgaard, R. B. 2001. The improvisation of discordant knowledges. Pages 57-73 in C. J. Cleveland, D. I. Stern, and R. Costanza, editors. The economics of nature and the nature of economics. Edward Elgar, Cheltenham, UK. http://dx.doi.org/10.4337/9781843761419.00011
Pawson, R. 2002. Evidence-based policy: in search of a method. Evaluation 8(2):157-181. http://dx.doi.org/10.1177/1358902002008002512
Persson, J., H. Thorén, and L. Olsson. 2018. The interdisciplinary decision problem: Popperian optimism and Kuhnian pessimism in forestry. Ecology and Society 23(3):40. http://dx.doi.org/10.5751/ES-10401-230340
Petticrew M., and H. Roberts. 2003. Evidence, hierarchies, and typologies: horses for courses. Journal of Epidemiology & Community Health 57(7):527-529. http://dx.doi.org/10.1136/jech.57.7.527
Pullin, A. S., and T. M. Knight. 2001. Effectiveness in conservation practice: pointers from medicine and public health. Conservation Biology 15(1):50-54. http://dx.doi.org/10.1111/j.1523-1739.2001.99499.x
Pullin, A. S., and T. M. Knight. 2003. Support for decision making in conservation practice: an evidence-based approach. Journal for Nature Conservation 11:83-90. http://dx.doi.org/10.1078/1617-1381-00040
Pullin, A. S. and N. Salafsky. 2010. Save the whales? Save the rainforest? Save the data! Conservation Biology 24:915-917. http://dx.doi.org/10.1111/j.1523-1739.2010.01537.x
Sherif, M., and C. W. Sherif. 1969. Interdisciplinary coordination as validity check: retrospect and prospects. Chapter 1 in M. Sherif and C. W. Sherif, editors. Interdisciplinary relationships in the social sciences. Aldine Transaction, New Brunswick, New Jersey, USA.
Sutherland, W. J., A. S. Pullin, P. M. Dolman, and T. M. Knight. 2004. The need for evidence-based conservation. Trends in Ecology and Evolution 19(6):305-308. http://dx.doi.org/10.1016/j.tree.2004.03.018
Thorén, H., and J. Persson. 2013. The philosophy of interdisciplinarity: sustainability science and problem-feeding. Journal for General Philosophy of Science 44(2):337-355. http://dx.doi.org/10.1007/s10838-013-9233-5
Toenniessen, G., A. Adesina, and J. DeVries. 2008. Building an alliance for a green revolution in Africa. Annals of the New York Academy of Sciences 1136:233-242. http://dx.doi.org/10.1196/annals.1425.028
Tomlin, S. 2006. Development: harvest of hope. Nature 442:22-25. http://dx.doi.org/10.1038/442022a
Tschakert, P. 2007. Views from the vulnerable: understanding climatic and other stressors in the Sahel. Global Environmental Change 17:381-396. http://dx.doi.org/10.1016/j.gloenvcha.2006.11.008
Wahlberg, L., and J. Persson. 2017. Importing notions in health law: science and proven experience. European Journal of Health Law 24(5):565-590. http://dx.doi.org/10.1163/15718093-12453308
Wiek, A., B. Ness, P. Schweizer-Ries, F. S. Brand, and F. Farrioli. 2012. From complex systems analysis to transformational change: a comparative appraisal of sustainability science projects. Sustainability Science 7(Supplement 1):5-24. http://dx.doi.org/10.1007/s11625-011-0148-y
World Bank. 2007. World Development Report 2008: agriculture for development. World Bank, Washington, D.C., USA.