Systems as diverse as the interacting species in a community, alleles at a genetic locus, and companies in a market are characterized by competition (over resources, space, capital, etc) and adaptation. Neutral theory, built around the hypothesis that individual performance is independent of group membership, has found utility across the disciplines of ecology, population genetics, and economics, both because of the success of the neutral hypothesis in predicting system properties and because deviations from these predictions provide information about the underlying dynamics. However, most tests of neutrality are weak, based on static system properties such as species-abundance distributions or the number of singletons in a sample. Time-series data provide a window onto a system’s dynamics, and should furnish tests of the neutral hypothesis that are more powerful to detect deviations from neutrality and more informative about to the type of competitive asymmetry that drives the deviation. Here, we present a neutrality test for time-series data. We apply this test to several microbial time-series and financial time-series and find that most of these systems are not neutral. Our test isolates the covariance structure of neutral competition, thus facilitating further exploration of the nature of asymmetry in the covariance structure of competitive systems. Much like neutrality tests from population genetics that use relative abundance distributions have enabled researchers to scan entire genomes for genes under selection, we anticipate our time-series test will be useful for quick significance tests of neutrality across a range of ecological, economic, and sociological systems for which time-series data are available. Future work can use our test to categorize and compare the dynamic fingerprints of particular competitive asymmetries (frequency dependence, volatility smiles, etc) to improve forecasting and management of complex adaptive systems.
Adaptive evolution requires that rivalrous goods are consumed by agents, those agents have heritable variation in how they acquire and consume the rivalrous goods, and the fitness of agents increases with the amount of goods consumed [2]. By Lewontin’s listing of the necessary conditions of evolution, a variety of systems can be seen as evolving. Genes in populations, species in a community, companies in a market, and political groups in a society all satisfy Lewontin’s axioms [2]. Canopy space is a rivalrous resource in the multi-species closed-canopy forests. If nothing else intervenes, a competitively superior tree species will dominate the canopy just like a competitively superior gene will become fixed in a population. In economic systems, companies compete over capital, customers, and labor, and a company well-adapted to a market will increase its share of the resources. In social systems, political groups compete over votes and occupied positions of power, and political groups with superior recruitment compared to other groups—either by persuasion, coercion, aggression, or reproduction with vertical transmission of culture—will increase the votes it receives and/or its representation in various positions of power. These generalized competitive systems are examples of “complex adaptive systems” [3,4, 5] and understanding how they evolve can provide insight into the drivers of adaptive evolution [6], diversity maintenance in human and natural systems [7], portfolio construction in a market [8], and problems of recognition and representation in multicultural societies [9]. Much literature has explored the stochastic fluctuations of individual populations (e. g. [10,11]) or asset prices [12] in these systems, and accurate models of the stochastic time-evolution of multi-species systems can enable calculations of the risk of extinction [13], the dynamics of diversity (such as the entropy or evenness of a system), portfolio analysis, and other features of interest. A common stochastic model in which all groups are functionally equivalent, termed “Neutral Theory” in ecology and population genetics, has been used across many systems [14,15,16,17,18]. By “functionally equivalent”, we mean that every agent’s performance in acquiring the rivalrous resource is independent of their group membership. In other words, an organism’s species identity, a company’s strategy or sector, a citizen’s political identity, or a political party’s platform have no impact on their ability to hold or acquire new rivalrous resources. Neutrality is a parsimonious starting point for community modeling because it is based on first principles of random birth and death or acquisition and release of resources that are appropriate for many competitive systems, and, because neutrality does not assume particular traits that distinguish groups and complex interactions between groups, it is invariant to grouping: the populations of neutral species can be aggregated into larger groups whose competition is also neutral. Neutrality is often posed as a null model for multi-species systems because it can be parsimonious to assume, initially, that all species are equivalent. The mathematical tractability of neutral systems has allowed for useful calculations [19,20] that can sometimes accurately describe features of the system. However, despite the mathematical ease, calculations for features such as extinction time or the dynamics of portfolio diversity based on neutrality may be inaccurate for systems with non-neutral dynamics such as positive or negative frequency-dependent selection. Thus, there is a need for powerful and informative tests of neutrality to assess whether or not the dynamics of the competitive system are neutral. In population genetics, tests of neutrality [21,22] have facilitated rapid conceptual and empirical advancements [23], allowing researchers to scan entire genomes for neutral loci and identify loci that have been under selection. Neutrality tests developed in ecological and sociological systems test features of rank-abundance and frequency-abundance distributions [16,24,25]. Many of these existing neutrality tests utilize snapshots of a competitive system, but time-series contain a tremendous amount of data and can enable stronger tests of neutrality. Some work has been done developing and utilizing tools to test whether or not population dynamics are consistent with Neutral Theory [10,26,11]. These tests rely on a particular description of Neutral Theory as the one-step process [27] posed by Kimura and Hubbell, but the proof [28] that the non-zero-sum volatility-stabilized market models [17] converge to neutral drift in relative abundances motivates a broader definition and more general tests of neutrality. Since neutrality is the per capita equivalence between species, it is necessarily relative, not absolute; a population is not neutral per se, but can only be neutral relative to another population. To the best of our knowledge, the existing time-series tests have all analyzed whether or not the variance in jumps in abundance increase linearly or quadratically with the population size prior to the jump, and none have examined the covariance structure of fluctuations in relative abundance. Here, we present, to our knowledge, the first covariance-based neutrality test for time-series data. Our test determines if the covariance structure of a time-series dataset is compatable with a Wright-Fisher process, thus yielding more insight into the nature of non-neutrality than traditional tests of rank-abundance distributions and the volatility of individual populations. Our test utilizes the grouping invariance of neutral systems to isolate and test the covariance between changes is species’ relative abundances over small time intervals, allowing a rejection of neutrality for the entire community considered. We apply our test to 6 metagenomic time-series [29], a time-series of breeding birds across North America [30], and a time-series of market capitalization of companies in the S&P 500 from 2000–2005. We show that even some systems whose rank-abundance distributions appear neutral can exhibit significantly non-neutral covariances between species as detected by our test. Furthermore, our test, based on random groupings of species, illustrates how to analyze the volatility of randomly formed groups to reveal state-dependent volatility that differs from neutrality. The non-neutral state-dependent covariance structure uncovered here can be incorporated to improve our models of community dynamics and calculations of species’ extinction times, portfolio risk, and more.
Large neutral communities are well-approximated by a Wright-Fisher Process (WFP) [31,32]. The convergence of discrete neutral communities to the continuous diffusion model of the WFP is covered in [33], and some numerical methods used for parameter estimation and simulation have been produced by [34]. The WFP is a continuous-state, continuous-time approximation of Kimura’s theory [14], it is an approximation of Hubbell’s neutral theory [15] when speciation rates are negligible over the timescale of interest, and it describes the dynamics of relative abundances of non-zero-sum volatility-stabilized market models [17,28]. Using the WFP as a continuous approximation of large, finite communities simplifies the covariance in the jumps between species’ relative abundances, thereby permitting the analysis below. The WFP models the stochastic time-evolution of relative abundances of n species. Let X t = (X t 1, . . . , X t n) be the vector of relative abundances, the WFP is defined by the Ito SDE d X t = λ ρ - X t d t + σ X t d W t (1) where λ > 0 and ρ > 0. The covariation between relative abundances of different species is given by the elements of Σ = σσT/2, where The deterministic motion, or drift, of the WFP—λ (ρ−Xt) —yields exponential mean reversion like many dynamical systems reverting to a nearby stable equilibrium. The quadratic covariation of the WFP, Σ, captures the stochastic fingerprint of neutrality; it arises from randomly drawing a resource to be freed from its agent followed by randomly drawing one of the remaining agents to acquire that resource with a probability proportional to the agent’s current resource ownership. The family of Wright-Fisher Processes is closed to grouping, meaning that if a multi-species community’s dynamics are governed by a WFP, species can be grouped (e. g. collecting species into genera or higher taxonomic levels) and the dynamics of the resulting, re-grouped community will also be governed by a WFP. We developed a test that is intentionally sensitive to the state-dependent noise of the WFP, allowing researches to test the underlying stochastic model of the random turnover of resources at the heart of neutral competition. Developing a strong test of the state-dependent covariance is not trivial, though, because direct measurement of the covariance of jumps conditioned on the state of the system, Cov [ΔXt|Xt], would require many replicate time points each with the same initial state, Xt, and, even with multiple time points at the same state, the sparsity of the high-dimensional data challenges the accurate estimation and significance testing of the covariance matrix. To circumvent these problems of replicate time points and high dimensionality and develop a strong test the state-dependent noise of the WFP, we find a variance-stabilizing transformation for the WFP that allows a regression-based heteroskedasticity test [35]. To be precise, we are looking for a real-valued function f (Xt) such that for Xt obeying the WFP law in eq (1), lim Δ t → 0 Var f X t + Δ t - f X t Δ t = c o n s t. (3) This approach is conceptually similar to the variance-stabilizing tools used for population fluctuation analyses [10,26,11] which stabilize the variance in jumps of a population size, except our function must stabilize the covariance of jumps between populations, not just the variance. In particular, to have a constant volatility, our function f must satisfy the Hamilton-Jacobi equation, ∇ f T Σ ∇ f = c o n s t. (4) (see S1 Text, part 2, for more details). The grouping invariance of the WFP can be used to intuit and show that there are at least 2n different variance-stabilizing transformations of the WFP, parametrized by a vector a: f a X t = arcsin a 1 X t 1 +. . . + a n X t n, (5) where ai = ±1 for all i. After transforming the data with fa, we need to perform a constant-volatility test. To test the constant volatility of f, we test the homoskedasticity of standardized jumps, ν t = f t + Δ t - f t / Δ t, (6) following regression on ft to eliminate the state-dependent drift. A homoskedasticity test of νt for a single CVT is a test of neutrality for time-series data. However, with 2n different CVTs, one can perform multiple hypothesis tests. For any multiple-hypothesis tests, if the null hypothesis is true, the distribution of P-values is uniform. In this paper, we test the uniformity of the distribution of resultant P-values from homoskedasticity tests of νt for a number of randomly drawn CVTs. Fig 1 illustrates this test for 2,000 randomly drawn CVTs and shows the successful rejection of the WFP for the relative abundances of a mean-reverting geometric Brownian motion. The P-values arising from homoskedasticity tests of different CVTs are not independent. Consequently, a Kolmogorov-Smirnov test of the P-value distribution against a uniform distribution would have a high false-positive rate. To reduce the false-positive rate, we perform a perturbation analysis to generate conservative estimates of cutoffs for the KS statistic at 0. 05 and 0. 005 significance levels. Details of the sensitivity analysis are provided in S1 Text, part 3.
Our method is proven to work and can correctly identify a community of uncorrelated geometric Brownian motions as being non-neutral (Fig 1). Some may claim that neutrality of geometric Brownian motions is more easily rejected by a super-linear relationship between mean and variance in population size over short time intervals, or by analysis of how the variance in a measure of fluctuation of population size Nt, D Δ t = (N t + Δ t - N t) / N t, changes with the size of the time-window Δt [11]. We applied existing tests of variance- and fluctuation-scaling to the volatility-stabilized market model (VSM), d N t i = δ i 2 S t d t + N t i S t d W t i (7) where N t i is the absolute abundance of species i, S t = ∑ i N t i is the community size, and δi > 0. Pal [28] proved that the market shares or relative abundances of the VSM follow a WFP with λ = ∑i δi and ρi = δi/λ, and the grouping invariance of the VSM can be shown. Both the scaling of variance in population size with expected population size and scaling of the variance of DΔt indicate would incorrectly reject the proven neutrality of the VSM, whereas our test does not reject neutrality in the relative abundances of the volatility-stabilized market model (Fig 2). Applying existing tests which analyze the fluctuations of individual populations to the volatility stabilized market model reinforces our claim that neutrality is necessarily relative and that fluctuations in absolute abundance are dubious tests of neutrality. Our test of the fluctuations of relative abundances tests the grouping invariance at the heart of neutrality, regardless of the manner of fluctuations of community size, and correctly identifies the volatility-stabilized market model as neutral where existing tests suggest otherwise. We apply our neutrality test to 8 different datasets. Six of these datasets are sequence-count data of microbial communities [29] from three body sites on two individuals. One dataset is survey of breeding birds across North America from 1966–2014 [30], and one dataset is financial data, obtained from the Center for Research in Security Prices, of the day-end market shares and market capitalization of 451 companies in the S&P 500 from January 1,2000 to January 1,2005. These datasets are long, time-series datasets, many of which have rank-abundance distributions that are decently fit by neutral theory’s expected rank-abundance distribution (see S1 Text for a detailed description of the datasets and fits of neutral theory’s rank-abundance distributions). Our test relies on multiple groups of species. To group the species, we randomly selected ai = ±1, for 4,000 independent groups, we then calculated νt as in eq (6), and used a White test [36] to test the homoskedasticity of νt. The White test performed auxiliary regression with a generalized linear model with a log-normal link function of the form ϵ ^ 2 = β 0 + β 1 f t + β 2 f t 2 + γ (8) where ϵ ^ 2 are the squared residuals from similar quadratic regression of ft on νt. With a 0. 005 significance threshold, neutrality was rejected in all but one dataset (Fig 3). Qualitatively, there is a major difference in the neutrality of gut, tongue, and palm microbial communities. The gut and tongue microbial communities are clearly non-neutral, whereas the palm microbial communities, likely subjected to more frequent disturbances and colonizations, both appeared close-to-neutral, with one of the palm microbial communities having a P-value above the significance threshold. These qualitative differences suggest different probabilistic laws governing the colonization and turnover of these communities, and that some microbial communities may be closer to neutral than others. Rejecting neutrality for these competitive systems motivates further investigation on whether the rejection of neutrality stems from sampling error or from true competitive asymmetries in the system. For the financial datasets, there is no sampling error—the reported values of day-end prices are the true values. For the metagenomic datasets, sampling error for sequence count data could be driving apparent non-neutrality. Fig 4 examines the competitive asymmetry in the male tongue’s bacterial community. Scatter plots of νt vs. ft reveal a downward trend indicative of mean reversion—jumps in ft are positive when ft is above its mean and negative when ft is above its mean (Fig 4A). The mean reversion is regressed out prior to our auxiliary regression, but such strong mean reversion is not apparent in the simulated neutral community of Fig 1. The discrepancy between the mean reversion in the data and the simulated neutral community may be due to a stronger and/or non-linear mean reversion in the microbial system, or it may be due to the relatively long time between time points in the data relative to the turnover rate of the community. Such sparse time-sampling could affect the accuracy of our test, which relies on the assumption that Δt in eq 3 is small. A neutral community may still have mean reversion due to migration from a metacommunity or mutation/conversion rates between classes of agents, and sparsely sampling a time-series of such a neutral community may yield the same downward trend on plots of νt versus ft. To examine if the long time between time points accounts for the perceived non-neutrality from our analysis of Fig 3, we produced surrogate data by simulating neutral communities with similarly sparse time points. Parameter estimation of λ, ρ and WFP simulation is described in S1 Text, part 4. For one particular CVT, we simulated 4,000 independent trajectories to allow the superposition of the νt vs ft scatter plots from the male tongue data over the points from the surrogate data. Much of the strong mean reversion in the data can in fact be accounted for by the sparsity of time points (Fig 4A), but the P-value distribution from constant-volatility tests of 16,000 randomly drawn surrogate CVT simulations is much more uniform than the same distribution from the male tongue, which has many small P-values indicative of consistent state-dependent volatility of ft (Fig 4b). Thus, the non-neutrality of the male tongue dataset is not due to long sampling intervals. The male tongue microbiome deviates from neutrality by having a significantly more β2 > 0 than β2 < 0 for those auxiliary regressions yielding significant heteroskedasticity (Fig 4C). β2 > 0 indicates that the volatility increases farther away from the mean and plotting the residuals, ϵ ^ 2 against the state variable, ft, reveals the form of heteroskedasticity (Fig 4D). A similar significant hyper-abundance of β2 > 0 exists for all datasets considered here (Fig 4E).
Evolution, driven by competition over rivalrous goods or limiting resources, is a phenomenon common to ecology, economics and sociology, and accurate statistical models of how competitive systems evolve can allow us to forecast, manage, and invest in them [2,3, 8,1, 34]. Neutral Theory is a null model of competition which assumes that all players are equal—that a canopy tree fills a gap in the canopy independent of its species’ identity, a dollar finds its way to another dollar independent of who owns the dollar, and a seat in congress is filled by someone independent of the racial, cultural or political traits of the successor or predecessor. It’s been hypothesized that neutrality could arise naturally as a result of competitively inferior species going extinct [37], and thus systems would tend towards neutrality over long periods of time, but the accuracy and generality of Neutral Theory as a dynamical model for a range of competitive systems was unclear. Previous tests of neutrality have relied on tests of the fluctuations of population size. However, the concept of neutrality is necessarily relative: organisms can only be neutral relative to one-another, and so testing the fluctuations of population size against a “neutral” signature only tests very particular models of neutrality with particular fluctuations in community size. Changes in population size are equal to the product of changes in relative abundance and changes in community size, and the essence of neutrality is in the changes in relative abundance, not changes in population size. Neutrality tests based on fluctuations in population size require assumptions about fluctuations in community size, fluctuations which may not be known from the ecological data, data which may be best viewed as compositional data. There are many ways to maintain neutrality in finite-community-size models while modifying fluctuations in community size: at random points in time, there could be deterministic forcing, geometric increases or decreases, or any other fluctuations in community size independent of community composition, followed by a random draw of organisms to die, migrate and reproduce fill the empty spaces in the community; such populations can be neutral despite markedly different probability laws for the fluctuations in population size. In all cases, however, the relative abundances will be neutral, and with neutrality comes a particular signature of competitive symmetry that can be tested. In this paper, we have provided a time-series test of neutral covariance structure yielding a P-value indicative of the compatibility of the data with neutral dynamics. Applying our test to real datasets has revealed a common feature of non-neutrality across a range of ecological and economic systems. Our test is based on the grouping invariance of neutral communities, and this grouping invariance is maintained by a particular covariance structure of volatilities, namely where the volatility, v, of a group, X t i, is v (X t i) = c X t i (1 - X t i) for some constant, c, which can only be invariant to grouping if the covariance between groups i, and j is Σ i, j = - c X t i X t j. We test neutrality by randomly grouping species and testing if the volatility of those random groups is of the form of v (x). A deviation from v can indicate non-neutral covariance structure. If, for instance, instead of a quadratic curve of v, the volatility of random groups in the data follows a bell-shaped curve with positive curvature at the end-points, it could indicate that there is a positive covariance between rare species, possibly due to kill-the-winner effects or due to the relative abundances being driven by fluctuations in the largest populations. Conversely, excessive negative curvature could indicate strong negative covariances between rare species, possibly due to relatively constant populations of abundant species. Future work analyzing the volatility surfaces of random groups can improve our test and allow researchers to quickly isolate particular forms and fingerprints of common, non-neutral competitive asymmetries. Our test assumes stationary migration and volatility. Non-stationarity in migration, such a daily time-series with a large pulse of migrant birds arriving to their breeding grounds, would very likely lead to a rejection of neutrality. However, the same system, sampled over longer time-intervals, may have stationary distributions of migration or volatility that could lead to a failure to reject neutrality. The results presented here are limited to the particular choice of species (the taxonomic scale), resource (trophic scale), and time scale. However, an analysis restricted to a particular scale means that our test can also allow researchers to probe multiple taxonomic, trophic and time-scales to see if there are patterns in which scales are most/least neutral in their dynamics. An alternative grouping of these original OTUs by genera may reveal different results by conditioning the groupings of species on a particular sub-set of possible groups, such as grouping monophyletic clades of OTUs or grouping OTUS with shared metabolic traits, and consequently this test could serve as a tool for evaluating competition at multiple taxonomic scales. The results are also limited to the choice of resource: companies within a sector governed by trust-busting policies which break the neutral symmetry in their market capitalization dynamics may still be neutral in their competition over the ethnic or cultural composition of their labor force. Much like the example of breeding birds presented above, it’s possible that microbes in the gut are not neutral in their short-term fluctuations over the course of a year, but perhaps are neutral over longer time-scales that average out short-term fluctuations in diet and physiological state that are known to have predictable effects on microbial communities [38,39]. The results from a neutrality test also depend on the number of time points sampled. Ultimately, we use a P value as a quantification of the incompatibility of the data with neutral covariance structure. With P values as our measure of incompatibility, the true measure of relative distance from neutrality to any given competitive system, the P value we obtain in the limit of infinte data, will be 0 or 1 (neutral or not). Most, if not all, systems are non-neutral, much like there is no perfect square in the universe, for all square-like objects are not exactly square. Nonetheless, there are many competitive systems in nature, from ecosystems to economies, where neutrality and the grouping invariance it entails is a compelling null model and scientists pursuing hypothetico-deductive progress can use the P value as a means to test the essence of neutrality and, where they reject neutrality, move on to constructing the next-best model. Others may be interested in comparing the strength of non-neutrality across systems, and for those cases the most rigorous comparisons through P values are those comparisons made between systems with the same number of time points. Our test can be applied to any community of competing agents classified into discrete groups for which time-series of relative abundances (or market share, etc) of the groups are available. This test is most effective when the time-series is long and the spacing between samples is short relative to the turnover rate of the underlying resources (trees, dollars, congressional seats). There are many ways to build on our method. Explicit calculations of the drift and volatility over long time-intervals can improve our method for datasets with sparse time points. The dependence of the CVTs may be calculated as copulas allowing the implementation of a more exact goodness of fit test [40], or a perturbation analysis of the Kolmogorov-Smirnov test-statistic under different numbers of species, timepoints, and CVTs can provide quantitative estimation of the P-values, thus allowing more precise comparison of the degrees of non-neutrality across systems. There may be more CVTs that solve the Hamilton-Jacobi equation, and different CVTs might be more specialized at detecting different asymmetries in competitive systems. Neutrality tests of time-series data can help us understand the stochastic time-evolution of competitive systems and facilitate better prediction and management. For bacteria in the gut, for instance, understanding the important non-neutral forces governing the dynamics could allow progress towards the stochastic pharmacokinetics of probiotics [41]. Understanding the predictability of invasions at different taxonomic scales can tell us whether to evaluate metagenomic or ecological datasets based on the species, or whether other taxonomic levels will yield a more informative analysis—perhaps grouping tropical trees into the family Fabaceae, the family Melastomataceae, the genus Cecropia, and all other trees reveals trophic structure of tropical forests and competitive asymmetries that are drowned out by analyses at the species level. Demonstrating that the non-neutrality of portfolios is consistent with rare-species advantages in the Atlas model [8] would have major implications for portfolio design. In all cases, the first step of empirically demonstrating the existence of competitive asymmetries in time-series data can now be done with the test provided here.
From fisheries and forestries to game parks and gut microbes, managing a community of organisms is much like managing a portfolio. Managers care about diversity, and calculations of risk—for extinction or financial ruin—require accurate models of the covariance between the parts of the portfolio. To model the covariances in portfolios or communities which may have some direct or diffuse competition over limiting resources, it helps to start simple with a null model assuming the equivalence of species or companies relative to one another (termed “neutrality”) and then testing whether or not the data suggest otherwise. Researchers in biology and finance have independently entertained and tested neutral models, but the existing tests have used snapshots of communities or the variance of fluctuations of individual populations, whereas tests of the covariances between species can better inform the development of alternative models. We develop a covariance-based neutrality test for time-series data and use it to show that the human microbiome, North American birds, and companies in the S&P 500 all have a similar deviation from neutrality. Understanding and incorporating this non-neutral covariance structure can yield more accurate alternative models of community dynamics which can improve our management of “portfolios” of multi-species systems.
species colonization
medicine and health sciences
ecology and environmental sciences
invasive species
neutral theory
population genetics
social sciences
random variables
covariance
mathematics
tongue
population biology
digestive system
ecological economics
population ecology
economics
probability theory
community ecology
finance
mouth
anatomy
ecology
genetics
biology and life sciences
physical sciences
evolutionary biology
Novel Covariance-Based Neutrality Test of Time-Series Data Reveals Asymmetries in Ecological and Economic Systems