Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> As methods of study constitute the leading theme of our session, I have chosen as a subject in measurable consonance the method of multiple working hypotheses in its application to its investigation, instruction, and citizenship. <s> BIB001 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Abstract We consider the problem of model selection and accounting for model uncertainty in high-dimensional contingency tables, motivated by expert system applications. The approach most used currently is a stepwise strategy guided by tests based on approximate asymptotic P values leading to the selection of a single model; inference is then conditional on the selected model. The sampling properties of such a strategy are complex, and the failure to take account of model uncertainty leads to underestimation of uncertainty about quantities of interest. In principle, a panacea is provided by the standard Bayesian formalism that averages the posterior distributions of the quantity of interest under each of the models, weighted by their posterior model probabilities. Furthermore, this approach is optimal in the sense of maximizing predictive ability. But this has not been used in practice, because computing the posterior model probabilities is hard and the number of models is very large (often greater than 1... <s> BIB002 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Although bioclimatic modelling is often used to estimate potential impacts of likely climate changes, little has been done to assess the reliability and variability of projections. Here, using four niche-based models, two methods to derive probability values from models into presence–absence data and five climate change scenarios, I project the future potential habitats of 1350 European plant species for 2050. All 40 different projections of species turnover across Europe suggested high potential species turnover (up to 70%) in response to climate change. However variability in the potential distributional changes of species across climate scenarios was obscured by a strong variability in projections arising from alternative, yet equally justifiable, niche-based models. Therefore, projections of future species distributions and derived community descriptors cannot be reliably discussed unless model uncertainty is quantified explicitly. I propose and test an alternative way to account for modelling variability when deriving estimates of species turnover (with and without dispersal) according to a range of climate change scenarios representing various socio-economic futures. <s> BIB003 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Recently, researchers in several areas of ecology and evolution have begun to change the way in which they analyze data and make biological inferences. Rather than the traditional null hypothesis testing approach, they have adopted an approach called model selection, in which several competing hypotheses are simultaneously confronted with data. Model selection can be used to identify a single best model, thus lending support to one particular hypothesis, or it can be used to make inferences based on weighted support from a complete set of competing models. Model selection is widely accepted and well developed in certain fields, most notably in molecular systematics and mark-recapture analysis. However, it is now gaining support in several other areas, from molecular evolution to landscape ecology. Here, we outline the steps of model selection and highlight several ways that it is now being implemented. By adopting this approach, researchers in ecology and evolution will find a valuable alternative to traditional null hypothesis testing, especially when more than one hypothesis is plausible. <s> BIB004 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Bayesian inference is an important statistical tool that is increasingly being used by ecologists. In a Bayesian analysis, information available before a study is conducted is summarized in a quantitative model or hypothesis: the prior probability distribution. Bayes Theorem uses the prior probability distribution and the likelihood of the data to generate a posterior probability distribution. Posterior probability distributions are an epistemological alternative to P-values and provide a direct measure of the degree of belief that can be placed on models, hypotheses, or parameter estimates. Moreover, Bayesian information-theoretic methods provide robust measures of the probability of alternative models, and multiple models can be averaged into a single model that reflects uncertainty in model construction and selection. These methods are demonstrated through a simple worked example. Ecologists are using Bayesian inference in studies that range from predicting single-species population dynamics to understanding ecosystem processes. Not all ecologists, however, appreciate the philosophical underpinnings of Bayesian inference. In particular, Bayesians and frequentists differ in their definition of probability and in their treatment of model parameters as random variables or estimates of true values. These assumptions must be addressed explicitly before deciding whether or not to use Bayesian methods to analyse ecological data. <s> BIB005 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Ecologists are increasingly applying model selection to their data analyses, primarily to compare regression models. Model selection can also be used to compare mechanistic models derived from ecological theory, thereby providing a formal framework for testing the theory. The Akaike Information Criterion (AIC) is the most commonly adopted criterion used to compare models; however, its performance in general is not very well known. The best model according to AIC has the smallest expected Kullback-Leibler (K-L) distance, which is an information-theoretic measure of the difference between a model and the truth. I review the theory behind AIC and demonstrate how it can be used to test ecological theory by considering two example studies of foraging, motivated by simple foraging theory. I present plausible truths for the two studies, and models that can be fit to the foraging data. K-L distances are calculated for simulated studies, which provide an appropriate test of AIC. Results support the use of a commonly adopted rule of thumb for selecting models based on AIC differences. However, AICc, a corrected version of AIC commonly used to reduce model selection bias, showed no clear improvement, and model averaging, a technique to reduce model prediction bias, gave mixed results. <s> BIB006 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Population limitation is a fundamental tenet of ecology, but the relative roles of exogenous and endogenous mechanisms remain unquantified for most species. Here we used multi-model inference (MMI), a form of model averaging, based on information theory (Akaike's Information Criterion) to evaluate the relative strength of evidence for density-dependent and density-independent population dynamical models in long-term abundance time series of 1198 species. We also compared the MMI results to more classic methods for detecting density dependence: Neyman-Pearson hypothesis-testing and best-model selection using the Bayesian Information Criterion or cross-validation. Using MMI on our large database, we show that density dependence is a pervasive feature of population dynamics (median MMI support for density dependence = 74.7-92.2%), and that this holds across widely different taxa. The weight of evidence for density dependence varied among species but increased consistently, with the number of generations monitored. Best-model selection methods yielded similar results to MMI (a density-dependent model was favored in 66.2-93.9% of species time series), while the hypothesis-testing methods detected density dependence less frequently (32.6-49.8%). There were no obvious differences in the prevalence of density dependence across major taxonomic groups under any of the statistical methods used. These results underscore the value of using multiple modes of analysis to quantify the relative empirical support for a set of working hypotheses that encompass a range of realistic population dynamical behaviors. <s> BIB007 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Statistical methods emphasizing formal hypothesis testing have dominated the analyses used by ecologists to gain insight from data. Here, we review alternatives to hypothesis testing including techniques for parameter estimation and model selection using likelihood and Bayesian techniques. These methods emphasize evaluation of weight of evidence for multiple hypotheses, multimodel inference, and use of prior information in analysis. We provide a tutorial for maximum likelihood estimation of model parameters and model selection using information theoretics, including a brief treatment of procedures for model comparison, model averaging, and use of data from multiple sources. We discuss the advantages of likelihood estimation, Bayesian analysis, and meta-analysis as ways to accumulate understanding across multiple studies. These statistical methods hold promise for new insight in ecology by encouraging thoughtful model building as part of inquiry, providing a unified framework for the empirical analysis of theoretical models, and by facilitating the formal accumulation of evidence bearing on fundamental questions. <s> BIB008 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Statistical thinking in wildlife biology and ecology has been profoundly influenced by the introduction of AIC (Akaike's information criterion) as a tool for model selection and as a basis for model averaging. In this paper, we advocate the Bayesian paradigm as a broader framework for multimodel inference, one in which model averaging and model selection are naturally linked, and in which the performance of AIC-based tools is naturally evaluated. Prior model weights implicitly associated with the use of AIC are seen to highly favor complex models: in some cases, all but the most highly parameterized models in the model set are virtually ignored a priori. We suggest the usefulness of the weighted BIC (Bayesian information criterion) as a computationally simple alternative to AIC, based on explicit selection of prior model probabilities rather than acceptance of default priors associated with AIC. We note, however, that both procedures are only approximate to the use of exact Bayes factors. We discuss and illustrate technical difficulties associated with Bayes factors, and suggest approaches to avoiding these difficulties in the context of model selection for a logistic regression. Our example highlights the predisposition of AIC weighting to favor complex models and suggests a need for caution in using the BIC for computing approximate posterior model weights. <s> BIB009 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Observed patterns of species richness at landscape scale (gamma diversity) cannot always be attributed to a specific set of explanatory variables, but rather different alternative explanatory statistical models of similar quality may exist. Therefore predictions of the effects of environmental change (such as in climate or land cover) on biodiversity may differ considerably, depending on the chosen set of explanatory variables. Here we use multimodel prediction to evaluate effects of climate, land-use intensity and landscape structure on species richness in each of seven groups of organisms (plants, birds, spiders, wild bees, ground beetles, true bugs and hoverflies) in temperate Europe. We contrast this approach with traditional best-model predictions, which we show, using cross-validation, to have inferior prediction accuracy. Multimodel inference changed the importance of some environmental variables in comparison with the best model, and accordingly gave deviating predictions for environmental change effects. Overall, prediction uncertainty for the multimodel approach was only slightly higher than that of the best model, and absolute changes in predicted species richness were also comparable. Richness predictions varied generally more for the impact of climate change than for land-use change at the coarse scale of our study. Overall, our study indicates that the uncertainty introduced to environmental change predictions through uncertainty in model selection both qualitatively and quantitatively affects species richness projections. <s> BIB010 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Forecasts of species range shifts under climate change are fraught with uncertainties and ensemble forecasting may provide a framework to deal with such uncertainties. Here, a novel approach to partition the variance among modeled attributes, such as richness or turnover, and map sources of uncertainty in ensembles of forecasts is presented. We model the distributions of 3837 New World birds and project them into 2080. We then quantify and map the relative contribution of different sources of uncertainty from alternative methods for niche modeling, general circulation models (AOGCM), and emission scenarios. The greatest source of uncertainty in forecasts of species range shifts arises from using alternative methods for niche modeling, followed by AOGCM, and their interaction. Our results concur with previous studies that discovered that projections from alternative models can be extremely varied, but we provide a new analytical framework to examine uncertainties in models by quantifying their importance and mapping their patterns. <s> BIB011 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Abstract An important aspect of ensemble forecasting is that the resulting probabilities are reliable (i.e., the forecast probabilities match the observed frequencies). In the medium-range forecasting context, the literature tends to focus on the requirement that, for a reliable ensemble, the ensemble spread should be representative of the uncertainty in the mean, whereas in the seasonal forecasting context, the literature tends to focus on the requirement that, for a reliable ensemble, the ensemble forecasts should have the same climatological variance as the truth. In this note, the authors emphasize that both of these requirements are necessary for reliability and they clarify that a popular calibration method actually achieves both of these requirements. <s> BIB012 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> 1. Department of Ecology and Evolution, University of Lausanne, CH-1015 Lausanne, Switzerland. Ecography, volume 33, issue 6, pages 1015-1027, December 2010. Article first published online: 16 NOV 2010. DOI: 10.1111/j.1600-0587.2010.06338.x Abstract Identifying the geographic distribution of populations is a basic, yet crucial step in many fundamental and applied ecological projects, as it provides key information on which many subsequent analyses depend. However, this task is often costly and time consuming, especially where rare species are concerned and where most sampling designs generally prove inefficient. At the same time, rare species are those for which distribution data are most needed for their conservation to be effective. To enhance fieldwork sampling, model-based sampling (MBS) uses predictions from species distribution models: when looking for the species in areas of high habitat suitability, chances should be higher to find them. We thoroughly tested the efficiency of MBS by conducting an important survey in the Swiss Alps, assessing the detection rate of three rare and five common plant species. For each species, habitat suitability maps were produced following an ensemble modeling framework combining two spatial resolutions and two modeling techniques. We tested the efficiency of MBS and the accuracy of our models by sampling 240 sites in the field (30 sites × 8 species). Across all species, the MBS approach proved to be effective. In particular, the MBS design strictly led to the discovery of six sites of presence of one rare plant, increasing chances to find this species from 0 to 50%. For common species, MBS doubled the new population discovery rates as compared to random sampling. Habitat suitability maps coming from the combination of four individual modeling methods predicted well the species’ distribution and more accurately than the individual models. As a conclusion, using MBS for fieldwork could efficiently help in increasing our knowledge of rare species distribution. More generally, we recommend using habitat suitability models to support conservation plans. Key words : Habitat suitability, spatial predictions, species distribution, public database, model-based sampling, field validation, rare plants, ENFA, GAM. <s> BIB013 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> We briefly outline the information-theoretic (I-T) approaches to valid inference including a review of some simple methods for making formal inference from all the hypotheses in the model set (multimodel inference). The I-T approaches can replace the usual t tests and ANOVA tables that are so inferentially limited, but still commonly used. The I-T methods are easy to compute and understand and provide formal measures of the strength of evidence for both the null and alternative hypotheses, given the data. We give an example to highlight the importance of deriving alternative hypotheses and representing these as probability models. Fifteen technical issues are addressed to clarify various points that have appeared incorrectly in the recent literature. We offer several remarks regarding the future of empirical science and data analysis under an I-T framework. <s> BIB014 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> There has been a great deal of recent discussion of the practice of regression analysis (or more generally, linear modelling) in behaviour and ecology. In this paper, I wish to highlight two factors that have been under-considered, collinearity and measurement error in predictors, as well as to consider what happens when both exist at the same time. I examine what the consequences are for conventional regression analysis (ordinary least squares, OLS) as well as model averaging methods, typified by information theoretic approaches based around Akaike’s information criterion. Collinearity causes variance inflation of estimated slopes in OLS analysis, as is well known. In the presence of collinearity, model averaging reduces this variance for predictors with weak effects, but also can lead to parameter bias. When collinearity is strong or when all predictors have strong effects, model averaging relies heavily on the full model including all predictors and hence the results from this and OLS are essentially the same. I highlight that it is not safe to simply eliminate collinear variables without due consideration of their likely independent effects as this can lead to biases. Measurement error is also considered and I show that when collinearity exists, this can lead to extreme biases when predictors are collinear, have strong effects but differ in their degree of measurement error. I highlight techniques for dealing with and diagnosing these problems. These results reinforce that automated model selection techniques should not be relied on in the analysis of complex multivariable datasets. <s> BIB015 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Model averaging, specifically information theoretic approaches based on Akaike’s information criterion (IT-AIC approaches), has had a major influence on statistical practices in the field of ecology and evolution. However, a neglected issue is that in common with most other model fitting approaches, IT-AIC methods are sensitive to the presence of missing observations. The commonest way of handling missing data is the complete-case analysis (the complete deletion from the dataset of cases containing any missing values). It is well-known that this results in reduced estimation precision (or reduced statistical power), biased parameter estimates; however, the implications for model selection have not been explored. Here we employ an example from behavioural ecology to illustrate how missing data can affect the conclusions drawn from model selection or based on hypothesis testing. We show how missing observations can be recovered to give accurate estimates for IT-related indices (e.g. AIC and Akaike weight) as well as parameters (and their standard errors) by utilizing ‘multiple imputation’. We use this paper to illustrate key concepts from missing data theory and as a basis for discussing available methods for handling missing data. The example is intended to serve as a practically oriented case study for behavioural ecologists deciding on how to handle missing data in their own datasets and also as a first attempt to consider the problems of conducting model selection and averaging in the presence of missing observations. <s> BIB016 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Behavioural ecologists often study complex systems in which multiple hypotheses could be proposed to explain observed phenomena. For some systems, simple controlled experiments can be employed to reveal part of the complexity; often, however, observational studies that incorporate a multitude of causal factors may be the only (or preferred) avenue of study. We assess the value of recently advocated approaches to inference in both contexts. Specifically, we examine the use of information theoretic (IT) model selection using Akaike’s information criterion (AIC). We find that, for simple analyses, the advantages of switching to an IT-AIC approach are likely to be slight, especially given recent emphasis on biological rather than statistical significance. By contrast, the model selection approach embodied by IT approaches offers significant advantages when applied to problems of more complex causality. Model averaging is an intuitively appealing extension to model selection. However, we were unable to demonstrate consistent improvements in prediction accuracy when using model averaging with IT-AIC; our equivocal results suggest that more research is needed on its utility. We illustrate our arguments with worked examples from behavioural experiments. <s> BIB017 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Akaike’s information criterion (AIC) is increasingly being used in analyses in the field of ecology. This measure allows one to compare and rank multiple competing models and to estimate which of them best approximates the “true” process underlying the biological phenomenon under study. Behavioural ecologists have been slow to adopt this statistical tool, perhaps because of unfounded fears regarding the complexity of the technique. Here, we provide, using recent examples from the behavioural ecology literature, a simple introductory guide to AIC: what it is, how and when to apply it and what it achieves. We discuss multimodel inference using AIC—a procedure which should be used where no one model is strongly supported. Finally, we highlight a few of the pitfalls and problems that can be encountered by novice practitioners. <s> BIB018 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Multi-model averaging is currently receiving a surge of attention in the atmospheric, hydrologic, and statistical literature to explicitly handle conceptual model uncertainty in the analysis of environmental systems and derive predictive distributions of model output. Such density forecasts are necessary to help analyze which parts of the model are well resolved, and which parts are subject to considerable uncertainty. Yet, accurate point predictors are still desired in many practical applications. In this paper, we compare a suite of different model averaging techniques by their ability to improve forecast accuracy of environmental systems. We compare equal weights averaging (EWA), Bates-Granger model averaging (BGA), averaging using Akaike’s information criterion (AICA), and Bayes’ Information Criterion (BICA), Bayesian model averaging (BMA), Mallows model averaging (MMA), and Granger-Ramanathan averaging (GRA) for two different hydrologic systems involving water flow through a 1950 km2 watershed and 5 m deep vadose zone. Averaging methods with weights restricted to the multi-dimensional simplex (positive weights summing up to one) are shown to have considerably larger forecast errors than approaches with unconstrained weights. Whereas various sophisticated model averaging approaches have recently emerged in the literature, our results convincingly demonstrate the advantages of GRA for hydrologic applications. This method achieves similar performance as MMA and BMA, but is much simpler to implement and use, and computationally much less demanding. <s> BIB019 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Africa is predicted to be highly vulnerable to 21st century climatic changes. Assessing the impacts of these changes on Africa's biodiversity is, however, plagued by uncertainties, and markedly different results can be obtained from alternative bioclimatic envelope models or future climate projections. Using an ensemble forecasting framework, we examine projections of future shifts in climatic suitability, and their methodological uncertainties, for over 2500 species of mammals, birds, amphibians and snakes in sub-Saharan Africa. To summarize a priori the variability in the ensemble of 17 general circulation models, we introduce a consensus methodology that combines co-varying models. Thus, we quantify and map the relative contribution to uncertainty of seven bioclimatic envelope models, three multi-model climate projections and three emissions scenarios, and explore the resulting variability in species turnover estimates. We show that bioclimatic envelope models contribute most to variability, particularly in projected novel climatic conditions over Sahelian and southern Saharan Africa. To summarize agreements among projections from the bioclimatic envelope models we compare five consensus methodologies, which generally increase or retain projection accuracy and provide consistent estimates of species turnover. Variability from emissions scenarios increases towards late-century and affects southern regions of high species turnover centred in arid Namibia. Twofold differences in median species turnover across the study area emerge among alternative climate projections and emissions scenarios. Our ensemble of projections underscores the potential bias when using a single algorithm or climate projection for Africa, and provides a cautious first approximation of the potential exposure of sub-Saharan African vertebrates to climatic changes. The future use and further development of bioclimatic envelope modelling will hinge on the interpretation of results in the light of methodological as well as biological uncertainties. Here, we provide a framework to address methodological uncertainties and contextualize results. <s> BIB020 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Information theoretic approaches and model averaging are increasing in popularity, but this approach can be difficult to apply to the realistic, complex models that typify many ecological and evolutionary analyses. This is especially true for those researchers without a formal background in information theory. Here, we highlight a number of practical obstacles to model averaging complex models. Although not meant to be an exhaustive review, we identify several important issues with tentative solutions where they exist (e.g. dealing with collinearity amongst predictors; how to compute model-averaged parameters) and highlight areas for future research where solutions are not clear (e.g. when to use random intercepts or slopes; which information criteria to use when random factors are involved). We also provide a worked example of a mixed model analysis of inbreeding depression in a wild population. By providing an overview of these issues, we hope that this approach will become more accessible to those investigating any process where multiple variables impact an evolutionary or ecological response. <s> BIB021 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> We studied the effects of roads on presence of Plateau brown frogs (Rana kukunoris Nikolsky, 1918) and Tibetan frogs (Nanorana pleskei Gunther, 1896) in temporary pools of Sedges dominated wetland area in eastern Qinghai-Tibetan Plateau. The road is seven meter-wide, asphalt-paved with daily traffic rate about 400 vehicles. The temporary pools hold water only in summer with surface area of about 2 m(2). We used logistic regression models, a theoretic information approach, and model averaging to test the effects of distance from road and depth, area and pH of pools on distribution of frogs in terms of presence/absence in 180 small pools located at 10 to 150 m from the road edge. Observed data showed that presence probabilities of both species declined in the vicinity of roads, starting at approximately 100 m away from the road edge. Model averaging based on AICc (Sigma omega(i) = 95% confidence) indicated that both distance from road edge and its quadratic term were important predictors for explaining presence of both amphibians. Model-averaged prediction based on 95% confidence model set also revealed non-monotonic increasing curve relationships between presence probability of both amphibians and distance from road edge, even when other habitat variables were held constant. These results indicated that the road-effect zone for both amphibians extended 100 in on side of the wetland roads along which we sampled. Additionally, the results showed that water depth and water pH of pools positively influenced presence of Tibetan frogs and had highest contribution to the models. In contrast, water depth influenced presence of Plateau brown frogs negatively. It was indicated that environmental variables influence the presence of the two species of amphibians in different ways. <s> BIB022 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> General Strategies.- Regression Models.- Classification Models.- Other Considerations.- Appendix.- References.- Indices. <s> BIB023 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> More diverse biological communities may provide ecosystem services that are less variable over space or time. However, the mechanisms underlying this relationship are rarely investigated empirically in real-world ecosystems. Here, we investigate how a potentially important stabilising mechanism, response diversity, the differential response to environmental change among species, stabilises pollination services against land-use change. We measured crop pollination services provided by native bees across land-use gradients in three crop systems. We found that bee species responded differentially to increasing agricultural land cover in all three systems, demonstrating that response diversity occurs. Similarly, we found response diversity in pollination services in two of the systems. However, there was no evidence that response diversity, in general, stabilised ecosystem services. Our results suggest that either response diversity is not the primary stabilising mechanism in our system, or that new measures of response diversity are needed that better capture the stabilising effects it provides. <s> BIB024 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> AIM ::: Conservation planning exercises increasingly rely on species distributions predicted either from one particular statistical model or, more recently, from an ensemble of models (i.e. ensemble forecasting). However, it has not yet been explored how different ways of summarizing ensemble predictions affect conservation planning outcomes. We evaluate these effects and compare commonplace consensus methods, applied before the conservation prioritization phase, to a novel method that applies consensus after reserve selection. ::: ::: ::: LOCATION ::: Europe. ::: ::: ::: METHODS ::: We used an ensemble of predicted distributions of 146 Western Palaearctic bird species in alternative ways: four different consensus methods, as well as distributions discounted with variability, were used to produce inputs for spatial conservation prioritization. In addition, we developed and tested a novel method, in which we built 100 datasets by sampling the ensemble of predicted distributions, ran a conservation prioritization analysis on each of them and averaged the resulting priority ranks. We evaluated the conservation outcome against three controls: (i) a null control, based on random ranking of cells; (2) the reference solution, based on an expert-refined dataset; and (3) the independent solution, based on an independent dataset. ::: ::: ::: RESULTS ::: Networks based on predicted distributions were more representative of rare species than randomly selected networks. Alternative methods to summarize ensemble predictions differed in representativeness of resulting reserve networks. Our novel method resulted in better representation of rare species than pre-selection consensus methods. ::: ::: ::: MAIN CONCLUSIONS ::: Retaining information about the variation in the predicted distributions throughout the conservation prioritization seems to provide better results than summarizing the predictions before conservation prioritization. Our results highlight the need to understand and consider model-based uncertainty when using predicted distribution data in conservation prioritization. <s> BIB025 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> When developing a species distribution model, usually one tests several competing models such as logistic regressions characterized by different sets of covariates. Yet, there is an exponential number of subsets of covariates to choose from. This generates the problem of model uncertainty. Bayesian model averaging (BMA) is a state-of-the-art approach to deal with model uncertainty. BMA weights the inferences of multiple models. However, the results yielded by BMA depend on the prior probability assigned to the models. Credal model averaging (CMA) extends BMA towards robustness. It substitutes the single prior over the models by a set of priors. The CMA inferences (e.g., posterior probability of inclusion of a covariate, coefficient of a covariate, probability of presence) are intervals. The interval shows the sensitivity of the BMA estimate on the prior over the models. CMA detects the prior-dependent instances, namely cases in which the most probable outcome becomes presence or absence depending on the adopted prior over the models. On such prior-dependent instances, BMA behaves almost as a random guesser. The weakness of BMA on the prior-dependent instances is to our knowledge pointed out for the first time in the ecological literature. On the prior-dependent instances CMA avoids random guessing acknowledging undecidability. In this way it stimulates the decision maker to convey further information before taking the decision. We provide thorough experiments on different data sets. <s> BIB026 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Methodological absences, i.e. when a species is not detected although it is actually present, are known to reduce the prediction accuracy of species distribution models (SDMs). To deal with this problem, we assessed whether a new iterative ensemble modelling (IEM) approach better predicts the spatial distribution of a set of 31 freshwater fish species, exhibiting a wide range of prevalence and methodological absences. Model efficiency was compared using one threshold-independent (AUC) and three threshold-dependent indicators of model predictive performance: the percentage of misclassified sites; the Kappa index; and the True Skill Statistic. We then reconstructed species assemblages from individual species predictions and compared observed assemblages to those predicted using EM and IEM using the Jaccard index. Compared to an EM approach, IEM improved model predictive performance for most difficult-to-detect species. The iterative approach outperformed EM at modelling the distribution of difficult-to-detect species, provided that presence data are representative of the niche of the species. At the assemblage level, the discrepancy between observed and IEM predicted assemblages was significantly lower than that between observed and EM predicted assemblages, showing that IEM can be used to predict the distribution of entire species assemblages. The IEM approach provides a way to consider difficult-to-detect species in species distribution models. <s> BIB027 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> The steady upward trend in the use of model selection and Bayesian methods in ecological research has made it clear that both approaches to inference are important for modern analysis of models and data. However, in teaching Bayesian methods and in working with our research colleagues, we have noticed a general dissatisfaction with the available literature on Bayesian model selection and multimodel inference. Students and researchers new to Bayesian methods quickly find that the published advice on model selection is often preferential in its treatment of options for analysis, frequently advocating one particular method above others. The recent appearance of many articles and textbooks on Bayesian modeling has provided welcome background on relevant approaches to model selection in the Bayesian framework, but most of these are either very narrowly focused in scope or inaccessible to ecologists. Moreover, the methodological details of Bayesian model selection approaches are spread thinly throughout the literature, appearing in journals from many different fields. Our aim with this guide is to condense the large body of literature on Bayesian approaches to model selection and multimodel inference and present it specifically for quantitative ecologists as neutrally as possible. We also bring to light a few important and fundamental concepts relating directly to model selection that seem to have gone unnoticed in the ecological literature. Throughout, we provide only a minimal discussion of philosophy, preferring instead to examine the breadth of approaches as well as their practical advantages and disadvantages. This guide serves as a reference for ecologists using Bayesian methods, so that they can better understand their options and can make an informed choice that is best aligned with their goals for inference. <s> BIB028 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Three flawed practices associated with model averaging coefficients for predictor variables in regression models commonly occur when making multimodel inferences in analyses of ecological data. Model-averaged regression coefficients based on Akaike information criterion (AIC) weights have been recommended for addressing model uncertainty but they are not valid, interpretable estimates of partial effects for individual predictors when there is multicollinearity among the predictor variables. Multicollinearity implies that the scaling of units in the denominators of the regression coefficients may change across models such that neither the parameters nor their estimates have common scales, therefore averaging them makes no sense. The associated sums of AIC model weights recommended to assess relative importance of individual predictors are really a measure of relative importance of models, with little information about contributions by individual predictors compared to other measures of relative importance based on effects size or variance reduction. Sometimes the model-averaged regression coefficients for predictor variables are incorrectly used to make model-averaged predictions of the response variable when the models are not linear in the parameters. I demonstrate the issues with the first two practices using the college grade point average example extensively analyzed by Burnham and Anderson. I show how partial standard deviations of the predictor variables can be used to detect changing scales of their estimates with multicollinearity. Standardizing estimates based on partial standard deviations for their variables can be used to make the scaling of the estimates commensurate across models, a necessary but not sufficient condition for model averaging of the estimates to be sensible. A unimodal distribution of estimates and valid interpretation of individual parameters are additional requisite conditions. The standardized estimates or equivalently the t statistics on unstandardized estimates also can be used to provide more informative measures of relative importance than sums of AIC weights. Finally, I illustrate how seriously compromised statistical interpretations and predictions can be for all three of these flawed practices by critiquing their use in a recent species distribution modeling technique developed for predicting Greater Sage-Grouse (Centrocercus urophasianus) distribution in Colorado, USA. These model averaging issues are common in other ecological literature and ought to be discontinued if we are to make effective scientific contributions to ecological knowledge and conservation of natural resources. <s> BIB029 </s> Model averaging in ecology: a review of Bayesian, information-theoretic and tactical approaches for predictive inference <s> INTRODUCTION <s> Model choice is usually an inevitable source of uncertainty in model-based statistical analyses. While the focus of model choice was traditionally on methods for choosing a single model, methods to formally account for multiple models within a single analysis are now accessible to many researchers. The specific technique of model averaging was developed to improve predictive ability by combining predictions from a set of models. However, it is now often used to average regression coefficients across multiple models with the ultimate goal of capturing a variable's overall effect. This use of model averaging implicitly assumes the same parameter exists across models so that averaging is sensible. While this assumption may initially seem tenable, regression coefficients associated with particular explanatory variables may not hold equivalent interpretations across all of the models in which they appear, making explanatory inference about covariates challenging. Accessibility to easily implementable software, concerns about being criticized for ignoring model uncertainty, and the chance to avoid having to justify choice of a final model have all led to the increasing popularity of model averaging in practice. We see a gap between the theoretical development of model averaging and its current use in practice, potentially leaving well-intentioned researchers with unclear inferences or difficulties justifying reasons for using (or not using) model averaging. We attempt to narrow this gap by revisiting some relevant foundations of regression modeling, suggesting more explicit notation and graphical tools, and discussing how individual model results are combined to obtain a model averaged result. Our goal is to help researchers make informed decisions about model averaging and to encourage question-focused modeling over method-focused modeling. <s> BIB030
Models are an integral part of ecological research, representing alternative, possibly overlapping, hypotheses BIB001 . They are also the standard approach to making predictions about ecological systems  . In many cases, it is not possible to clearly identify a single most-appropriate model. For instance, processbased models may differ in the specific ways they represent ecological mechanisms, without a clear empirical or theoretical reason to prefer one option over the other. Statistical analyses rarely offer a single solution, both because the limited amount of data allows for several plausible combinations of predictors, and because different modelling approaches are available for statistical analysis (e.g., Hastie et al. 2009, Kuhn and BIB023 . Model averaging seemingly solves this dilemma. Proponents of this approach have claimed that calculating a weighted average of the predictions of all candidate models will reduce prediction error through reduced variance and bias (the latter based on arguments described in BIB002 , as well as better represent uncertainty about model parametrisation and structure  , see also Model averaging (typically) reduces prediction errors). For some ecological examples of model averaging, see BIB003 , BIB006 , BIB007 , BIB010 , BIB011 , Le BIB013 , BIB020 , BIB024 , BIB025 , and BIB027 . Evaluating the utility of this approach is complicated by the large number of different methods for model averaging and the subsequent uncertainty quantification of averaged predictions. Several previous reviews on model averaging in ecology and evolution focused exclusively on "informationtheoretical model averaging" BIB004 BIB008 BIB014 BIB015 BIB021 BIB016 BIB017 BIB018 , probably under the influence of the AIC-weighted (Akaike information criterion) averaging popularized by Anderson (2002, Posada and  . Bayesian model averaging has been used less frequently in ecology (for an example see BIB026 , but, for an excellent recent review of this topic in the context of Bayesian model selection, see BIB028 , see also Hoeting et al. 1999 BIB005 BIB009 . However, none of the above covers all available model averaging approaches, together with a general discussion of advantages and disadvantages. Our aim is to provide such a comprehensive review in the light of developments over the last 20 years, summarizing the mathematical reasoning behind model averaging, and offering an intuitive but technically sound entry to the field, illustrated by case studies. We primarily address prediction averaging of correlative models, although most of the points will similarly apply to mechanistic/process-based models (see, e.g., BIB019 Vrugt 2010, Knutti et al. 2010 for reviews in the context of climate and hydrological models, respectively). We do not consider averaging model parameters, because we agree with the criticism summarized in Banner and Higgs (2017): parameters (such as partial regression coefficients) are estimated conditional on the model structure; as the model structure changes, parameters may become incommensurable (see  BIB029 BIB030 , and Appendix S1.1 for short review of the parameter-averaging literature). Instead, our focus is on prediction, and predictive inference (sensu  , as exemplified by model-averaged predictions of species potential occurence for reserve-site selection BIB025 or the effect of roads on occupancy of ponds by frogs BIB022 . Also, we only focus on averaging sets of models that differ in structure, as opposed to mere differences in initial conditions or parameter values (Gibbs 1902, Johnson and BIB012 . The latter case is called "ensemble" in the statistical and physical sciences, while in ecology that term is used more loosely. This review is divided into five parts: first, we present the mathematical logic behind model averaging, and why this alone puts severe constraints on how we do model averaging. Then, in the second part, we review the different ways through which model-averaging weights can be derived, comparing Bayesian, information-theoretic, and tactical perspectives (by tactical we mean heuristic approaches to model averaging that are not explicitly based on statistical theory). This is followed by a brief exploration of how to quantify the uncertainty of model-averaged predictions. Finally, we briefly illustrate model averaging with two case studies, before closing with unresolved challenges, and recommendations.