Local knowledge in ecological modeling

• Abstract
• Introduction
• Published Studies
• Issue 1: Consistency between the Degree of LEK Involvement and Modeling Objectives
• Issue 2: Combination of Concepts and Methods from Natural and Social Sciences
• Issue 3: Reliability of the Data Collection Process
• Issue 4: Model Accuracy
• Conclusion
• Responses to this article
• Acknowledgments
• Literature Cited

INTRODUCTION

Modeling plays a substantial role in ecology. Models allow researchers to better understand ecosystem functioning, forecast the future according to various scenarios, and provide resource managers with relevant information for decision making (Jørgensen and Bendoricchio 2001). Models address a range of questions, from very simple with few variables to “big” and complex ones involving nonlinear and multiscale processes (Sutherland et al. 2013). Because models are simplified representations of a complex reality (Box 1976), modelers rely on their judgement to decide what component of reality is to be represented, what parameters are most relevant and what level of complexity is necessary (Krueger et al. 2012). Inherent subjectivity makes modeling especially appropriate to combine different forms of expertise arising from scientific and local knowledge (Barber and Jackson 2015).

Research involving local ecological knowledge (LEK) has surged over the last decades (Fig. 1), with the increased recognition among ecologists of the many ways LEK can complement scientific knowledge (Asselin 2015). Scientific ecological knowledge (SEK) generally arises from hypothetico-deductive approaches, while LEK stems from direct contact of people with the environment (Box 1). The trend for increased involvement of LEK into ecological research is fueled by international conventions and declarations where SEK and LEK, including indigenous and traditional ecological knowledge, are found side by side, as United Nations’ Convention on Biological Diversity, UNESCO’s Declaration on Science and the Use of Scientific Knowledge, and more recently the Paris Agreement on climate change.

LEK not only finds a place in theoretical and applied ecology, but also in ecological modeling (Fig. 1). Indeed, LEK can provide ecological models with information hardly accessible using classical research designs. Reliability, scope, and predictive power of a model depend on data quality and quantity (Rykiel 1996) but data collection can be time- and resource-consuming from a researcher’s viewpoint. Alternatively, local people interact with the environment on a daily basis, yearlong, and over the long term. Their knowledge of ecological processes can reach a precision level that is virtually impossible to match with fieldwork conducted over a few weeks and based on a limited sample size. In addition to data provision, LEK may be used to build the conceptual framework behind a model, set the scope, limits, and assumptions, estimate model parameters, and validate model outputs (Krueger et al. 2012).

The expected benefits of involving LEK in ecological modeling extend beyond concerns of model performance. The legitimacy of an ecological model increases if it takes into account the knowledge, needs, concerns, and perceptions of those primarily concerned (Ericksen and Woodley 2005), especially when tackling sensitive issues. Moreover, involving local communities in the research process contributes to local development (Sillitoe 1998, Blaikie 2006), providing local experts with opportunities to be active players in both research and natural resource management. Modeling may thus promote community empowerment by providing a platform for communication, knowledge sharing, appropriation of scientific tools, and joint knowledge creation and learning (Voinov and Bousquet 2010).

Although the combination of LEK and SEK is increasingly encouraged, it is not free of critics. At one end of the spectrum, research involving LEK can be politically charged (Davis and Ruddle 2010) and seen as another instance of appropriation of marginalized cultures to the benefit of the dominant western one (Oguamanam 2008). At the other end, because LEK have their own epistemologies and meanings (Agrawal 1995), some might question their reliability as part of a systematic and rigorous research process (Gilchrist and Mallory 2007). Moreover, accessing and understanding LEK calls upon concepts and methods from both ecological and social sciences. Interdisciplinarity is thus an important and challenging component of research projects involving LEK, and researchers need to adapt modeling methodologies to live up to both ethical and scientific standards (Davis and Ruddle 2010).

We reviewed the scientific literature to summarize general issues regarding LEK inclusion in ecological modeling. We considered the following four issues as the most important: (1) consistency between the degree of LEK involvement and modeling objectives, (2) combination of concepts and methods from natural and social sciences, (3) reliability of the data collection process, and (4) model accuracy. We designed an analysis grid to evaluate ecological modeling exercises. We used this tool to assess how 23 published studies dealt with each of the four issues.

LEK is a heterogeneous bloc stemming from culturally specific epistemologies, assumed unknown to the researcher unless specifically investigated (Agrawal 1995, Sillitoe 1998). We refer to “scientific knowledge” as the one generated by methods and epistemologies accepted in ecology as a field of biological sciences (Begon 1996).

PUBLISHED STUDIES

We compiled published studies including both local knowledge and an ecological (or environmental) model. We searched Google Scholar and Scopus for different combinations of the following keywords: “local,” “traditional,” “indigenous,” “ecological knowledge” (Box 1), and “model(l)ing.” We then selected all scientific papers that presented an ecological model involving LEK. We extended the search to articles cited in synthesis papers. We ended up with 23 studies published between 2000 and 2017 in peer-reviewed journals. Models span all continents, cover a range of organizational (from species to ecosystems) and spatial (from local to nationwide) scales, environments (land, water, or both), and purposes (fundamental or applied research; Fig. 2). The analysis grid and references for all published studies are available in online material (Appendices 1 and 2).

ISSUE 1: CONSISTENCY BETWEEN THE DEGREE OF LEK INVOLVEMENT AND MODELING OBJECTIVES

There are many reasons for involving LEK in modeling, from wider and easier access to data (e.g., Anadón et al. 2010) to a will to foster social learning and development (e.g., Mendoza and Prabhu 2006, Rajaram and Das 2008). The level of LEK involvement is also quite variable, from basic empirical data collection to full involvement of local people and organizations as coresearchers. In this section, we address the problem of consistency between modeling objectives and degree of LEK involvement. We propose a framework to analyze the rationale behind LEK involvement in ecological modeling and review the methods used to do so, with a focus on participatory research.

Blackstock et al. (2007), inspired by the principles of deliberative democracy (see Dryzek 2002), summarized reasons to involve stakeholders in sustainability research into three functions. We adapted their framework to the specificities of LEK involvement into ecological modeling:

1. The substantive function relies on complementarity between LEK and SEK. LEK is especially useful when experimental data is incomplete, for example, when studying rare species, long time-series, or remote areas (Anadón et al. 2010, Ehrich et al. 2016, Bastari et al. 2017).
2. The normative function is about the legitimacy of scientific assessments for local populations (Ericksen and Woodley 2005). Ecological research is sometimes criticized for providing only partial knowledge with little external validity, i.e., becoming meaningless when taken out of its local context (Menzies and Butler 2006). Legitimacy of SEK is especially challenged when used to justify resource management policies with consequences for local populations (Booth and Skelton 2011).
3. The instrumental function refers to LEK as a social development tool empowering local communities and institutions for resource management (Fraser et al. 2006). It relies on colearning and coproduction of knowledge between researchers and local people and organizations (Mendoza and Prabhu 2006, Lane et al. 2011).

The degree of involvement of local experts and stakeholders in modeling should be in line with the objectives. To fulfill the substantive function, local understanding of an ecosystem can help build the conceptual framework and observations can be included as first-hand data. However, normative and instrumental functions require deeper involvement (Briggs 2013). Although the importance of opening science to community is generally acknowledged, mere sprinkling of LEK onto an otherwise classical experimental research design may lead to adverse outcomes such as knowledge instrumentalization or cultural appropriation (Oguamanam 2008).

Enforcing one or the other of the substantive, normative, and instrumental functions can be fostered by a participatory modeling process (Lynam et al. 2007, Voinov and Gaddis 2008) where local experts and organizations can contribute to the following:

• Define the objectives;
• Design the conceptual model;
• Collect or communicate data;
• Analyze, validate, and revise the results.

Most published studies (20) claimed substantive function for including LEK, with statements such as “TEK can potentially inform scientific approaches to management, [...] as a source of baseline data to fill information gaps that cannot otherwise be addressed” (Espinoza-Tenorio et al. 2013). Eight published studies sought to increase legitimacy, arguing LEK and SEK are valuable and need to be considered side by side: “local experts were frustrated when Western scientific studies conducted in the region neglected TEK and produced conclusions that were easily invalidated by local observations” (Olsen et al. 2015:11866). Eight published studies endeavored to foster local development. Mantyka-Pringle et al. (2017:126) claimed that “Co-production of TK [traditional knowledge] and SK [scientific knowledge] can also enhance capacity in rural or vulnerable communities observing resource declines, allow new ideas and tools to improve both local and scientific practices, and provide checks and balances to ensure new ideas are acceptable in terms of customary institutions and values.”

In the 23 published studies, the most common pattern (18) was to involve LEK in data collection thus fulfilling the substantive function (Table 1). LEK was also involved to formulate hypotheses and to design the underlying conceptual model (15). A few studies involved LEK in setting the research objectives (5), and analyzing and validating research results (6). We observed contradictions in two published studies claiming normative functions but without consequent LEK involvement beyond data provision. One published study (Mantyka-Pringle et al. 2017) should be commended for having involved LEK at all steps of the modeling process.

In the light of our analysis of published studies, we argue that there is a potential to involve LEK from the beginning to the end of a research process. We recommend that scientists and local people design and perform research together in order to reach the full potential of the LEK-SEK combination.

ISSUE 2: COMBINATION OF CONCEPTS AND METHODS FROM NATURAL AND SOCIAL SCIENCES

In ecology, LEK does not constitute a research object in itself but is rather used to extend the understanding of ecological phenomena. Thus, ecologists interested in integrating LEK and SEK have to build upon concepts developed within the social sciences (Davis and Ruddle 2010). For example, knowledge systems are studied in ethnology, cultural geography is interested in the relation to the land, whereas knowledge acquisition and expert judgement are concepts relevant to cognitive psychology. Consequently, most of the published studies were in interdisciplinary journals (8) such as Ecology and Society or Human Ecology, or in thematic journals (6) with no disciplinary specificity such as Arctic or Frontiers in Marine Science.

Bridging disciplines goes along with challenges. First, concepts often bear different meanings according to disciplines so that their integration requires communication and adaptation efforts (Miller et al. 2008). For example, the concept of “landscape” refers to a spatial scale in ecology and to a combination of physical features, perceptions, and mental constructions in cultural geography (Tress et al. 2001). Moreover, natural and social science epistemologies are different and refer to different standards to evaluate research quality and validity (Moon and Blackman 2014). Published studies entrenched in a single discipline had difficulty reaching the standards from another discipline. For example, McGregor et al. (2010) addressed traditional fire management in wetlands of Australia with an anthropological lens, but omitted to describe natural disturbance regimes and ecological processes occurring in the study area, which are basic information from an ecologist’s perspective. Conversely, Luizza et al. (2016) studied an invasive plant in Ethiopian agrosystems using farmers’ and villagers’ knowledge. However, neither culture (ethnology), nor social organization (sociology) or relationship with the land (human geography) were discussed in an elaborated fashion.

Social sciences also play an important role in the assessment and validation of ecological models. The information provided by an ecological model should always be considered in the light of the model’s assumptions, parameters, scope, limits, and uncertainties (Jørgensen and Bendoricchio 2001). Yet, ecological methods are rarely accurate for this kind of examination. For example, they are not suited to appraise limits and uncertainty of LEK that may take the form of myths, legends, or rituals (e.g., Colding and Folke 2001). Moreover, validation of LEK according to experimental ecology standards raises ethical questions, especially in intercultural and indigenous contexts (Brook and McLachlan 2005). Although indigenous people still struggle with the aftermath of a colonial history, attempts to validate a knowledge system through the lens of another will contribute to maintain power inequity (Asselin 2015). Alternatively, model assessment can be facilitated by methods of the social sciences suited to analyze the meaning and scope of LEK as part of a knowledge system. According to Davis and Ruddle (2010) and Usher (2000), such an assessment could allow for the following:

• Discern observations from inferences;
• Analyze how knowledge is created from observations;
• Determine whether information is widely shared within a community or held by a happy few;
• Describe knowledge transmission processes from one generation to the next;
• Describe how individual experiences and interactions with other cultures change a knowledge system.

Two published studies directly addressed the question of discipline integration. Liedloff et al. (2013) provide an interesting example of interdisciplinarity, where methods and espitemologies of anthropology, ecology, and hydro-geosciences were brought together in a single model (Miller et al. 2008). Authors built an integrative framework based on two independent studies of the Fitzroy River (Western Australia), respectively about hydrogeology and socioeconomy of the local indigenous population. The resulting model is consistent with local conceptions of the environment, e.g., indigenous seasonal calendar, and validated with both LEK and SEK.

Espinoza-Tenorio et al. (2013) address fisheries’ sustainability in Mexico using a transdisciplinary design. Compared with interdisciplinarity, transdisciplinarity relies on a common epistemology developed ad hoc (Miller et al. 2008). Authors thus built their own conceptual framework by combining the theoretical bases and methods of impact assessment, landscape ecology, and TEK.

Although few published studies directly addressed discipline integration, efforts dedicated to interdisciplinarity or transdisciplinarity contribute to reach quality, validity, and reliability standards from both natural and social sciences.

ISSUE 3: RELIABILITY OF THE DATA COLLECTION PROCESS

Elicitation is the process used to access expert knowledge and measure its uncertainty (O’Hagan et al. 2006). LEK holders can be considered as experts: their knowledge is based on empirical observations, is grounded in local context, and it can be used to make inferences and judgements (Usher 2000, O’Hagan et al. 2006). Importance of rigor in elicitation designs was underlined in research involving LEK (Davis and Wagner 2003) or more generally expert ecological knowledge (Martin et al. 2012). Expert knowledge elicitation is a research area in and of itself. It addresses issues relative to the selection of local experts, balance between representativeness and knowledgeability, dosage of sampling effort, bias control, and quantification of uncertainty (Ayyub 2001). It can be performed by semistructured interviews, workshops, questionnaires, or collaborative fieldwork (Huntingdon 2000).

A good LEK elicitation design for modeling purposes should provide details on at least five basic elements (adapted from Martin et al. 2012): (1) methods used to select participants; (2) number of participants; (3) methods used to pool information; (4) discussion on uncertainty; and (5) discussion on bias.

Most published studies selected respondents according to explicit criteria, e.g., occupation, age, or experience. However, only 11 clearly explained their selection procedure, such as random or snow-ball sampling. Sixteen published studies mentioned the number of respondents, 14 explained how they pooled data from many experts, five discussed uncertainties, and four discussed bias. We calculated an elicitation score from zero (when none of the five elements were presented) to five (when information was provided for all elements) for each of the 23 published studies (Fig. 3). Most published studies (15) scored below three, meaning critical information is generally lacking. Only two published studies obtained a perfect score (Bridger et al. 2016, Mantyka-Pringle et al. 2017).

We noted a nearly systematic lack of critical information in elicitation designs throughout the published studies. Elicitation designs should be systematic, rigorous, and reproducible, just as any other form of data/knowledge collection (Davis and Wagner 2003). We recommend peer-reviewers and editorial board members to be more critical of research designs before accepting manuscripts for publication.

ISSUE 4: MODEL ACCURACY

Statistical and empirical models that are commonly used in ecology are designed to deal with data from experimental designs and are poorly adapted to deal with LEK and their specificities (Krueger et al. 2012). LEK may take a quantitative or qualitative form (Berkes 2012). It can be explicit (enunciated), implicit (could be enunciated but is not), or tacit (cannot be enunciated; Fazey et al. 2006). Scientists can only access LEK through their holders, involving inherent uncertainties and biases that need to be quantified, which might prove easier said than done. Modelers could turn to alternative model families better suited to welcome LEK as expert judgement rather than experimental data. Those so-called “expert models” rely on artificial intelligence to introduce judgement by emulating human reasoning with mathematical language (Krueger et al. 2012). They are increasingly used to combine data from experimental design and expert knowledge. Eleven published studies used such models with a platform specifically adapted to work with LEK, while 12 used classical ecological models (e.g., multivariate analyses, linear regressions, habitat suitability indices) or other model families.

Two families of expert models are recurrent in the published studies and bear a great potential for LEK-SEK integration: fuzzy rule-based models (FRBM; 4 published studies) and Bayesian networks (5 published studies; Fig. 4). They can deal with qualitative and quantitative data and they consider uncertainty intrinsically (Adriaenssens et al. 2004, Kuhnert et al. 2010). Moreover, both can be represented with a simple graphic structure, easy to understand and to modify, making them well suited for participatory modeling (MacKinson 2000, Aguilera et al. 2011).

FRBM address complex systems dealing with the interrelations between qualitative, uncertain, and imprecise variables (Yager and Filev 1994). They rely on the mathematical theory of fuzzy sets, an extension of the set theory (Zadeh 1965). An object, instead of being described by its belonging to one set or another, is described by its “degree of belonging” to these sets. For instance, MacKinson (2000), studying herring shoals through fishers’ knowledge, described the shore size with “degrees of belonging” to the small, medium, and large sets (for example, small: 0%, medium: 20%, large: 80%). Links between variables are formulated as “IF/THEN” rules and variables are described by belonging functions (Yager and Filev 1994). LEK provide observational data to feed the model and to calibrate belonging functions. Local experts may also share their understanding of the links between parameters to formulate the rules (MacKinson 2000).

Bayesian networks combine probabilistic and graph theories (Aguilera et al. 2011). They are represented as multivariate, acyclic, and directional causality networks. Probabilistic statistics differ from frequentist statistics, of general use in ecology, by their probabilistic and inferential approach (Ellison 1996). In probabilistic statistics, parameters are not considered to have a fix value with a confidence interval. Instead, parameters are considered random and are described by a probability distribution. Bayes’ theorem infers a posterior probability distribution for a parameter using prior knowledge and likelihood. For example, Girondot and Rizzo (2015) used LEK of turtle nesting phenology as prior probability distributions in combination with experimental data as likelihood distributions. As in FRBM, LEK can also contribute to build the conceptual model (e.g., Mantyka-Pringle et al. 2017).

The review of published studies indicates that model families adapted to include expert judgement are also well suited for LEK inclusion. However, efforts should be made to better consider the uncertainties and biases in both elicitation and modeling.

CONCLUSION

Modeling has great potential for LEK-SEK integration and its popularity will likely keep growing in the near future. Despite methodological issues, modeling offers a great opportunity to involve local populations at all steps of a research project, thus fostering knowledge sharing and empowerment. From the analysis of 23 published studies, we conclude that methodological guidelines are not completely settled yet, especially regarding participatory methods and elicitation designs. The most pressing challenge relies in the integration of methods and concepts from social and natural sciences.

We make four recommendations to favor best practices of LEK-SEK integration in ecological modeling:

1. Participatory research is a helpful tool to reach the full potential of the LEK-SEK combination. Researchers and local managers should work together to design research projects able to share, enhance, and legitimate knowledge of ecosystems.
2. Research teams aiming at LEK-SEK integration in ecological modeling should include scientists from different disciplines to make sure the process meets the quality, validity, and reliability standards of both natural and social sciences.
3. Efforts should be made to design rigorous, appropriate, and reproducible methodologies for LEK elicitation.
4. Great potential lies in expert models designed to bring together expert knowledge and experimental data. Bayesian networks and FRBM are well suited for this task and widely accepted in ecology (Krueger et al. 2012).

LITERATURE CITED

Adriaenssens, V., B. De Baets, P. L. M. Goethals, and N. De Pauw. 2004. Fuzzy rule-based models for decision support in ecosystem management. Science of the Total Environment 319(1-3):1-12. http://dx.doi.org/10.1016/S0048-9697(03)00433-9

Agrawal, A. 1995. Dismantling the divide between indigenous and scientific knowledge. Development and Change 26(3):413-439. http://dx.doi.org/10.1111/j.1467-7660.1995.tb00560.x

Aguilera, P. A., A. Fernández, R. Fernández, R. Rumí, and A. Salmerón. 2011. Bayesian networks in environmental modelling. Environmental Modelling and Software 26(12):1376-1388. http://dx.doi.org/10.1016/j.envsoft.2011.06.004

Anadón, J. D., A. Giménez, and R. Ballestar. 2010. Linking local ecological knowledge and habitat modelling to predict absolute species abundance on large scales. Biodiversity and Conservation 19(5):1443-1454. http://dx.doi.org/10.1007/s10531-009-9774-4

Asselin, H. 2015. Indigenous forest knowledge. Pages 586-596 in K. Peh, R. Corlett, and Y. Bergeron, editors. Routledge handbook of forest ecology. Routledge, New York, New York, USA.

Ayyub, B. M. 2001. Elicitation of expert opinions for uncertainty and risks. CRC, Boca Raton, Florida, USA. http://dx.doi.org/10.1201/9781420040906

Barber, M., and S. Jackson. 2015. ‘Knowledge making’: issues in modelling local and indigenous ecological knowledge. Human Ecology 43(1):119-130. http://dx.doi.org/10.1007/s10745-015-9726-4

Bastari, A., J. Beccacece, F. Ferretti, F. Micheli, and C. Cerrano. 2017. Local ecological knowledge indicates temporal trends of benthic invertebrates species of the Adriatic Sea. Frontiers in Marine Science 4(May):157. http://dx.doi.org/10.3389/fmars.2017.00157

Begon, M. 1996. Ecology : individuals, populations and communities. Third edition. Blackwell Science, Oxford, UK.

Berkes, F. 2012. Sacred ecology. Third edition. Routledge, New York, New York, USA.

Blackstock, K. L., G. J. Kelly, and B. L. Horsey. 2007. Developing and applying a framework to evaluate participatory research for sustainability. Ecological Economics 60(4):726-742. http://dx.doi.org/10.1016/j.ecolecon.2006.05.014

Blaikie, P. 2006. Is small really beautiful? Community-based natural resource management in Malawi and Botswana. World Development 34(11):1942-1957. http://dx.doi.org/10.1016/j.worlddev.2005.11.023

Booth, A. L., and N. W. Skelton. 2011. “You spoil everything!” Indigenous peoples and the consequences of industrial development in British Columbia. Environment, Development and Sustainability 13(4):685-702. http://dx.doi.org/10.1007/s10668-011-9284-x

Box, G. E. P. 1976. Science and statistics. Journal of the American Statistical Association 71(356):791-799. http://dx.doi.org/10.1080/01621459.1976.10480949

Bridger, M. C., C. J. Johnson, and M. P. Gillingham. 2016. Assessing cumulative impacts of forest development on the distribution of furbearers using expert-based habitat modeling. Ecological Applications 26(2):499-514. http://dx.doi.org/10.1890/15-0555

Briggs, J. 2013. Indigenous knowledge: a false dawn for development theory and practice? Progress in Development Studies 13(3):231-243. http://dx.doi.org/10.1177/1464993413486549

Brook, R. K., and S. M. McLachlan. 2005. On using expert-based science to “test” local ecological knowledge. Ecology and Society 10(2):r3. http://dx.doi.org/10.5751/ES-01478-1002r03

Colding, J., and C. Folke. 2001. Social taboos: “invisible” systems of local resource management and biological conservation. Ecological Applications 11(2):584-600. http://dx.doi.org/10.2307/3060911

Davis, A., and K. Ruddle. 2010. Constructing confidence: rational scepticism and systematic enquiry in local ecological knowledge research. Ecological Applications 20(3):880-894. http://dx.doi.org/10.1890/09-0422.1

Davis, A., and J. R. Wagner. 2003. Who knows? On the importance of identifying “experts” when researching local ecological knowledge. Human Ecology 31(3):463-489. http://dx.doi.org/10.1023/A:1025075923297

Dryzek, J. S. 2002. Deliberative democracy and beyond: liberals, critics, contestations. Oxford University Press, New York, New York, USA. http://dx.doi.org/10.1093/019925043X.001.0001

Ehrich, D., M. A. Strømeng, and S. T. Killengreen. 2016. Interference in the tundra predator guild studied using local ecological knowledge. Oecologia 180(4):1195-1203. http://dx.doi.org/10.1007/s00442-015-3521-1

Ellison, A. M. 1996. An introduction to Bayesian inference for ecological research and environmental decision-making. Ecological Applications 6(4):1036-1046. http://dx.doi.org/10.2307/2269588

Ericksen, P., and E. Woodley. 2005. Using multiple knowledge systems: benefits and challenges. Pages 85-117 in Millennium Ecosystem Assessment. Ecosystems and human well-being: multiscale assessments Island, Washington, D.C., USA.

Espinoza-Tenorio, A., M. Wolff, I. Espejel, and G. Montaño-Moctezuma. 2013. Using traditional ecological knowledge to improve holistic fisheries management: transdisciplinary modeling of a lagoon ecosystem of southern Mexico. Ecology and Society 18(2):6. http://dx.doi.org/10.5751/ES-05369-180206

Fazey, I., J. A. Fazey, J. G. Salisbury, D. B. Lindenmayer, and S. Dovers. 2006. The nature and role of experiential knowledge for environmental conservation. Environmental Conservation 33(1):1-10. http://dx.doi.org/10.1017/S037689290600275X

Fraser, E. D. G., A. J. Dougill, W. E. Mabee, M. Reed, and P. McAlpine. 2006. Bottom up and top down: analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. Journal of Environmental Management 78(2):114-127. http://dx.doi.org/10.1016/j.jenvman.2005.04.009

Gilchrist, G., and M. L. Mallory. 2007. Comparing expert-based science with local ecological knowledge: What are we afraid of? Ecology and Society 12(1):02. http://dx.doi.org/10.5751/ES-01972-1201r01

Girondot, M., and A. Rizzo. 2015. Bayesian framework to integrate traditional ecological knowledge into ecological modeling: a case study. Journal of Ethnobiology 35(2):337-353. http://dx.doi.org/10.2993/etbi-35-02-337-353.1

Huntingdon, H. P. 2000. Using traditional ecological knowledge in science: methods and applications. Ecological Applications 10(5):1270-1274. http://dx.doi.org/10.1890/1051-0761(2000)010[1270:UTEKIS]2.0.CO;2

Jørgensen, S. E., and G. Bendoricchio. 2001. Fundamentals of ecological modelling. Elsevier, Amsterdam, The Netherlands.

Krueger, T., T. Page, K. Hubacek, L. Smith, and K. Hiscock. 2012. The role of expert opinion in environmental modelling. Environmental Modelling and Software 36:4-18. http://dx.doi.org/10.1016/j.envsoft.2012.01.011

Kuhnert, P. M., T. G. Martin, and S. P. Griffiths. 2010. A guide to eliciting and using expert knowledge in Bayesian ecological models. Ecology Letters 13(7):900-914. http://dx.doi.org/10.1111/j.1461-0248.2010.01477.x

Lane, S. N., N. Odoni, C. Landström, S. J. Whatmore, N. Ward, and S. Bradley. 2011. Doing flood risk science differently: an experiment in radical scientific method. Transactions of the Institute of British Geographers 36(1):15-36. http://dx.doi.org/10.1111/j.1475-5661.2010.00410.x

Liedloff, A. C., E. L. Woodward, G. A. Harrington, and S. Jackson. 2013. Integrating indigenous ecological and scientific hydro-geological knowledge using a Bayesian network in the context of water resource development. Journal of Hydrology 499:177-187. http://dx.doi.org/10.1016/j.jhydrol.2013.06.051

Luizza, M. W., T. Wakie, P. H. Evangelista, and C. S. Jarnevich. 2016. Integrating local pastoral knowledge, participatory mapping, and species distribution modeling for risk assessment of invasive rubber vine (Cryptostegia grandiflora) in Ethiopia’s Afar region. Ecology and Society 21(1):22. http://dx.doi.org/10.5751/ES-07988-210122

Lynam, T., W. De Jong, D. Sheil, T. Kusumanto, and K. Evans. 2007. A review of tools for incorporating community knowledge, preferences, and values into decision making in natural resources management. Ecology and Society 12(1):5. http://dx.doi.org/10.5751/ES-01987-120105

MacKinson, S. 2000. An adaptive fuzzy expert system for predicting structure, dynamics and distribution of herring shoals. Ecological Modelling 126(2-3):155-178. http://dx.doi.org/10.1016/S0304-3800(00)00263-5

Mantyka-Pringle, C. S., T. D. Jardine, L. Bradford, L. Bharadwaj, A. P. Kythreotis, J. Fresque-Baxter, E. Kelly, G. Somers, L. E. Doig, P. D. Jones, K. E. Lindenschmidt, and the Slave River and Delta Partnership. 2017. Bridging science and traditional knowledge to assess cumulative impacts of stressors on ecosystem health. Environment International 102:125-137. http://dx.doi.org/10.1016/j.envint.2017.02.008

Martin, T. G., M. A. Burgman, F. Fidler, P. M. Kuhnert, S. Low-Choy, M. McBride, and K. Mengersen. 2012. Eliciting expert knowledge in conservation science. Conservation Biology 26(1):29-38. http://dx.doi.org/10.1111/j.1523-1739.2011.01806.x

McGregor, S., V. Lawson, P. Christophersen, R. Kennett, J. Boyden, P. Bayliss, A. Liedloff, B. McKaige, and A. N. Andersen. 2010. Indigenous wetland burning: conserving natural and cultural resources in Australia’s world heritage-listed Kakadu National Park. Human Ecology 38(6):721-729. http://dx.doi.org/10.1007/s10745-010-9362-y

Mendoza, G. A., and R. Prabhu. 2006. Participatory modeling and analysis for sustainable forest management: overview of soft system dynamics models and applications. Forest Policy and Economics 9(2):179-196. http://dx.doi.org/10.1016/j.forpol.2005.06.006

Menzies, C. R., and C. Butler. 2006. Understanding ecological knowledge. Pages 1-20 in C. R. Menzies, editor. Traditional ecological knowledge and natural resource management. University of Nebraska Press, Lincoln, Nebraska, USA.

Miller, T. R., T. D. Baird, C. M. Littlefield, G. Kofinas, F. S. Chapin III, and C. L. Redman. 2008. Epistemological pluralism: reorganizing interdisciplinary research. Ecology and Society 13(2):46. http://dx.doi.org/10.5751/ES-02671-130246

Moon, K., and D. Blackman. 2014. A guide to understanding social science research for natural scientists. Conservation Biology 28(5):1167-1177. http://dx.doi.org/10.1111/cobi.12326

O’Hagan, A., C. E. Buck, A. Daneshkhah, J. R. Eiser, P. H. Garthwaite, D. J. Jenkinson, J. E. Oakley, and T. Rakow. 2006. Uncertain judgements: eliciting experts’ probabilities. John Wiley & Sons, Chichester, USA. http://dx.doi.org/10.1002/0470033312

Oguamanam, C. 2008. Local knowledge as trapped knowledge: intellectual property, culture, power and politics. Journal of World Intellectual Property 11(1):29-57. http://dx.doi.org/10.1111/j.1747-1796.2008.00333.x

Olsen, P. M., C. A. Kolden, and L. Gadamus. 2015. Developing theoretical marine habitat suitability models from remotely-sensed data and traditional ecological knowledge. Remote Sensing 7(9):11863-11886. http://dx.doi.org/10.3390/rs70911863

Rajaram, T., and A. Das. 2008. A methodology for integrated assessment of rural linkages in a developing nation. Impact Assessment and Project Appraisal 26(2):99-113. http://dx.doi.org/10.3152/146155108X323605

Rykiel, E. J., Jr. 1996. Testing ecological models: the meaning of validation. Ecological Modelling 90(3):229-244. http://dx.doi.org/10.1016/0304-3800(95)00152-2

Sillitoe, P. 1998. The development of indigenous knowledge: a new applied anthropology. Current Anthropology 39(2):223-252. http://dx.doi.org/10.1086/204722

Sutherland, W. J., R. P. Freckleton, H. C. J. Godfray, S. R. Beissinger, T. Benton, D. D. Cameron, Y. Carmel, D. A. Coomes, T. Coulson, M. C. Emmerson, R. S. Hails, G. C. Hays, D. J. Hodgson, M. J. Hutchings, D. Johnson, J. P. G. Jones, M. J. Keeling, H. Kokko, W. E. Kunin, X. Lambin, O. T. Lewis, Y. Malhi, N. Mieszkowska, E. J. Milner-Gulland, K. Norris, A. B. Phillimore, D. W. Purves, J. M. Reid, D. C. Reuman, K. Thompson, J. M. J. Travis, L. A. Turnbull, D. A. Wardle, and T. Wiegand. 2013. Identification of 100 fundamental ecological questions. Journal of Ecology 101(1):58-67. http://dx.doi.org/10.1111/1365-2745.12025

Tress, B., G. Tress, H. Décamps, and A.-M. d’Hauteserre. 2001. Bridging human and natural sciences in landscape research. Landscape and Urban Planning 57(3-4):137-141. http://dx.doi.org/10.1016/S0169-2046(01)00199-2

Usher, P. J. 2000. Traditional ecological knowledge in environmental assessment and management. Arctic 53(2):183-193. http://dx.doi.org/10.14430/arctic849

Voinov, A., and F. Bousquet. 2010. Modelling with stakeholders. Environmental Modelling and Software 25(11):1268-1281. http://dx.doi.org/10.1016/j.envsoft.2010.03.007

Voinov, A., and E. J. B. Gaddis. 2008. Lessons for successful participatory watershed modeling: a perspective from modeling practitioners. Ecological Modelling 216(2):197-207. http://dx.doi.org/10.1016/j.ecolmodel.2008.03.010

Yager, R. R., and D. P. Filev. 1994. Essentials of fuzzy modeling and control. Wiley, New York, New York, USA.

Zadeh, L. A. 1965. Fuzzy sets. Information and Control 8(3):338-353. http://dx.doi.org/10.1016/S0019-9958(65)90241-X