Machine learning approaches are an attractive option for analyzing large-scale data to detect genetic variants that contribute to variation of a quantitative trait, without requiring specific distributional assumptions. We evaluate two machine learning methods, random forests and logic regression, and compare them to standard simple univariate linear regression, using the Genetic Analysis Workshop 17 mini-exome data. We also apply these methods after collapsing multiple rare variants within genes and within gene pathways. Linear regression and the random forest method performed better when rare variants were collapsed based on genes or gene pathways than when each variant was analyzed separately. Logic regression performed better when rare variants were collapsed based on genes rather than on pathways. The common disease/common variant hypothesis has been successful at detecting some small to moderate genetic effects for complex traits in genome-wide association studies, although a substantial proportion of the heritability remains unexplained. The paradigm of common disease/rare variant contributions to the remaining genetic variation is now of interest. New sequencing technologies have made it feasible to determine DNA sequence variations in large numbers of subjects.Machine learning approaches are attractive in terms of handling large-scale data without requiring specific distributional assumptions and are useful for detecting interaction effects of multiple predictors on a trait. The random forest (RF) method and logic regression (LR) are two machine learning methods . The RF In the RF method, the goal is to build a forest of models, or trees, that, when combined, explain the variation of a trait. The RF method uses a random subset of the data and a random subset of the predictors to build a tree that minimizes the out-of-bag (OOB) mean-square error (MSE) in a bootstrap sample. The samples not used to build the tree are called the OOB sample, and they are used to calculate the OOB MSE. At each node in each tree, the RF method makes a random draw from the list of predictor variables. Predictors used or selected in one node may reappear in any other node. The RF method then repeats this process using different random samples of predictors and data to produce a large collection, or forest, of trees. A variable importance score is computed for each predictor in a tree, using permutation to measure how much the OOB MSE for a given tree would increase if this predictor was randomized (made noisy). This is one method among several that can be used to evaluate the predictiveness of the predictors in the tree. The RF method then ranks the predictors based on an average, across all trees, of the variable importance scores. To prevent overfitting, the RF method sets aside OOB samples and calculates the error rate in prediction for its proposed models in each tree using the OOB sample. The RF method does not pick a final model that contains only a subset of the original large number of predictors. The top-ranked predictors in an RF model are considered to be those that are most likely to be truly causal. Users decide the cutoff that determines what proportion of the top-ranked predictors are to be used in follow-up investigations see ,2,4).,4.2,4]).LR is a model-searching method embedded in a regression framework. It uses Boolean operators to construct a flexible network of markers as a model and uses simulated annealing to control the model’s search across all variants ,5. LR seWe evaluate the performance of the RF method, LR, and simple univariate linear regression (ULR) using the Genetic Analysis Workshop 17 (GAW17) mini-exome sequence data. We want to determine whether these methods can detect genetic effects that contribute to the variation of a quantitative trait despite sparseness in the data resulting from low minor allele frequency (MAF) at many genetic variants. Neither the RF method nor LR performs tests of hypotheses in a frequentist framework. Thus power and type I error calculations under the null hypothesis are not relevant for these methods. However, the percentage of simulated data replicates in which either top-ranked predictors in the RF method or final models in LR contain true causal variants (CVs), termed the percentage of replicates (PoR), can be interpreted as a surrogate of power. Alternatively, the percentage of CVs or noncausal variants (NCVs) that are contained in the top-ranked predictors in the RF method, the final models in LR, or the most significant set of tests in ULR for multiple replicates of the simulated data can be used to evaluate whether CVs are more likely to be ranked more highly than NCVs as predictors of Q2. For LR, permutation testing is used to avoid overfitting, and it provides the PoR in which the MSE of the final model is smaller than the MSEs of the models from randomly permuted data sets.We apply the RF and LR approaches to the simulated data after collapsing multiple rare variants (RVs) within genes and within gene pathways. Gene pathways are a set of genes that interact to perform a particular aggregate function within a cell. We use biological knowledge of gene pathways to collapse multiple RVs into single predictors.GAW17 provided mini-exome data  see alsop-values of 0.05 based on the number of variants, genes, or pathways. The tests were ranked by ascending p-values after dropping any NCV that exhibited a correlation greater than 0.6 with any CV (termed an “uncorrelated NCV” [UNCV]). The percentage of both CVs and UNCVs that were ranked in the most significant 10% of tests in at least 5%, 10% and 20% of replicates was calculated. This was repeated for gene-collapsed and pathway-collapsed variants.The uncollapsed, gene-collapsed, and pathway-collapsed data were analyzed with ULR using a trend test with PLINK  and R afWe used Random Jungle, version 1.0.359 , which iIn this study, the main purpose of RF analysis was to reduce the number of variables for further follow-up association analysis based on the rated lists of variables . To evalWe used the LogicReg, version 1.4.8, package in R. Because of software limitations , we separated the analysis by chromosome. Because LR accepts only binary predictors, we coded the collapsed RV predictors as binary predictors and recoded the uncollapsed common variant genotypes as binary predictors using dominant and recessive coding. Each logic tree is composed of multiple binary predictors (leaves) connected by Boolean operators. To accommodate the situation of many causal RVs, we fitted the tree using only the “or” Boolean operator instead of the “and” operator because allowing both in the presence of many rare alleles would lead to model-fitting failures. Multiple logic trees are included in LR as additive linear terms.For gene-collapsed data, we analyzed (1) the RVs and common variants together and (2) only the RVs. To evaluate the effect of model size, we fitted the LR using 1 tree with 3 leaves and 1 tree with 10 leaves for the gene-collapsed data with combined rare and common variants, and for the gene-collapsed data with only RVs we fitted the LR using either 1 tree with 10 leaves or 3 trees with 10 leaves. To demonstrate that overfitting might be a problem for this data set, we conducted permutation tests for the gene-collapsed 3-trees-10-leaves procedure on three chromosomes with CVs  as well as three chromosomes without any CVs  by permuting the trait values and refitting at the same model size. For each chromosome we evaluated whether the MSE for the real data was less than the observed MSE values for 25 permuted data sets.For the pathway analysis, we fitted models using 2 trees with 20 leaves. For each predictor, we calculated the PoR that identified the predictor in the fitted model, denoted as the pick rate . AlthougWe also calculated the percentage of CVs and UNCVs that were included in the fitted model in 5%, 10%, and 20% of replicates for the gene-collapsed and pathway-collapsed variants. For each analysis, we also calculated the MSE for the optimal model in each replicate, averaged this MSE across replicates, and compared this average MSE for chromosomes with CVs versus chromosomes without CVs.p-value of 0.05 in one replicate each (PoR = 0.5%). However, 56 UNCVs were detected at this Bonferroni-corrected p-value (once for each UNCV) across the 200 replicates. In the gene-collapsed data, 2 out of 13 causal genes and 62 out of 2,876 noncausal genes were detected at a Bonferroni-corrected p-value of 0.05. For pathway-collapsed data, 17 out of 167 causal pathways and 47 out of 2,249 noncausal pathways were detected. Table p-values that were ranked in the most significant 10%, with similar findings for gene-collapsed and pathway-collapsed variants.Additional table For the uncollapsed data for the RF analysis, each CV was ranked in the top 10% of predictors in at least 5% of replicates (range 5–24%)  [h2 between 0.011 and 0.017, and the remaining CVs had locus-specific h2 < 0.01. Thus there was little power to detect CVs in this small sample using any statistical method. ULR of uncollapsed data detected only 3 of the CVs in a single replicate each (PoR = 0.5%) and 56 false-positive results at a Bonferroni-corrected p-value of 0.05. This false-positive rate is inflated over the expected rate of 5 per 100 genome-wide analyses. This inflation may be due to only one set of genotypes being used in all 200 replicates, the multicollinearity of sequence data, or other factors related to the simulations. Permutation testing should be used in real data to control type I error. The collapsed analyses had better power and more appropriate false-positive rates.The simulated quantitative trait Q2 was caused by 36 variants with small individual effects on the variation of the trait . One CV h2)   appear more appropriate for RVs with large variance in effect size. In the gene-collapsed data, even with small effect sizes of CVs on Q2, higher proportions of causal genes than noncausal genes were included in the fitted models across multiple replicates. Future extensions to LR software that incorporate all predictors across the genome in a single analysis may improve performance.No interactions between loci were simulated for Q2. However, in reality, genetic elements can work both collectively and separately. Thus machine learning methods may be especially useful in such situations. Finding networks or motifs in the genetic material is an unsolved problem in the literature.We applied the RF, LR, and ULR approaches to a simulated polygenic quantitative trait and sequence data with both common and rare variants. ULR had low power to detect any CVs in this small sample. All three methods ranked larger proportions of CVs than NCVs among the top-ranked predictors in multiple replicates. When using the RF method or LR, we suggest that multiple trees and permutation tests be used for analysis. Our results leave open the question of whether a larger sample could improve RF and LR performance when effect sizes are small. Although not providing tests of hypotheses, machine learning approaches such as the RF method and LR can help researchers elucidate the causes of variation in a quantitative trait by providing a putative set of candidate genes for further study.The authors declare that there is/are no competing interests.YK and QL designed the study, performed the RF and LR analyses, and drafted the manuscript. CDC performed the ULR and drafted the manuscript. HS, JC, CLS, and BP performed quality control analyses and prepared the data for the analyses. AD, JDM and AFW participated in study design and critically revised the manuscript. JEBW designed the study, drafted and revised the manuscript. All authors read and approved the final manuscript..Click here for fileClick here for file


Converging lines of evidence point to the existence of immune dysfunction in autism spectrum disorder (ASD), which could directly affect several key neurodevelopmental processes. Previous studies have shown higher cytokine levels in patients with autism compared with matched controls or subjects with other developmental disorders. In the current study, we used plasma-cytokine profiling for 25 discordant sibling pairs to evaluate whether these alterations occur within families with ASD.Plasma-cytokine profiling was conducted using an array-based multiplex sandwich ELISA for simultaneous quantitative measurement of 40 unique targets. We also analyzed the correlations between cytokine levels and clinically relevant quantitative traits  composite score, Social Responsiveness Scale (SRS) total T score, head circumference, and full intelligence quotient (IQ)). In addition, because of the high phenotypic heterogeneity of ASD, we defined four subgroups of subjects , which encompass common and/or recurrent endophenotypes in ASD, and tested the cytokine levels in each group.None of the measured parameters showed significant differences between children with ASD and their related typically developing siblings. However, specific target levels did correlate with quantitative clinical traits, and these were significantly different when the ASD subgroups were analyzed. It is notable that these differences seem to be attributable to a predisposing immunogenetic background, as no other significant differences were noticed between discordant sibling pairs. Interleukin-1β appears to be the cytokine most involved in quantitative traits and clinical subgroups of ASD.In the present study, we found a lack of significant differences in plasma-cytokine levels between children with ASD and in their related non-autistic siblings. Thus, our results support the evidence that the immune profiles of children with autism do not differ from their typically developing siblings. However, the significant association of cytokine levels with the quantitative traits and the clinical subgroups analyzed suggests that altered immune responses may affect core feature of ASD. Autism spectrum disorders (ASDs) are a heterogeneous group of severe neurodevelopmental disorders characterized by atypical social interactions, impaired communication, and tendency to engage in idiosyncratic, repetitive, or restrictive behaviors, with onset before 3 years of age. ASDs include autistic disorder, Asperger’s syndrome, and pervasive developmental disorder-not otherwise specified (PDD-NOS) . AlthougIn addition to the results from neurobiological research in ASD, highlighting the pathways involved in neural development, synapse plasticity, structural brain abnormalities, cognition, and behavior, converging evidence point to the existence of altered immune function in ASD, which directly affects some or all these neurological processes . SeveralPrevious studies -25 have The study protocol was approved by the Western Institutional Review Board . All parents received a comprehensive description of the study, and gave written informed consent for their children’s participation.Diagnostic and Statistical Diagnostic and Statistical Manual of Mental Disorders, Fourth Revision) )Overall, the present study reports the lack of significant differences in plasma-cytokine levels between children with ASD and in their related non-autistic siblings. Thus, our results support the evidence that the immune profiles of children with autism do not differ from their typically developing siblings . HoweverThe authors have declared that no competing interests exist. Ray Biotech Inc., the commercial funder, had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.MJH, VN and BO-R conceived and designed the experiments. BO-R, VN, SS, and TP performed the experiments. CJS, RDM, SOR, and JK were involved in clinical data collection. VN, MJH, BO-R, and JJC analyzed the data. VN, MJH, BO-R, CJS, AMP, and DWC wrote the paper. All authors read and approved the final manuscript.Z scores and P values are reported.Cytokine levels in children with autism spectrum disorder (ASD) and their related healthy siblings. Data are expressed as median (interquartile range). Data analysis was performed using the non-parametric two paired samples signed-rank test (Wilcoxon). Click here for fileR and P values are reported. Significant results are highlighted in bold.Correlation analyses between cytokine levels and quantitative clinical traits. Data analysis was performed by non-parametric were performed by Spearman’s rank correlation analysis (ρ). Click here for fileR and P values are reported. Significant results are highlighted in bold.Analysis of correlations between cytokines associated with VABS composite score. Data analysis was performed by non-parametric were performed by Spearman’s rank correlation analysis (ρ). Click here for fileR and P values are reported. Significant results are highlighted in bold.Correlation analysis of cytokines associated with full intelligent quotient. Data analysis was performed by non-parametric were performed by Spearman’s rank correlation analysis (ρ). Click here for file


Identification of genetic variants that are associated with disease is an important goal in elucidating the genetic causes of diseases. The genetic patterns that are associated with common diseases are complex and may involve multiple interacting genetic variants. The Relief family of algorithms is a powerful tool for efficiently identifying genetic variants that are associated with disease, even if the variants have nonlinear interactions without significant main effects. Many variations of Relief have been developed over the past two decades and several of them have been applied to single nucleotide polymorphism (SNP) data.We developed a new spatially weighted variation of Relief called Sigmoid Weighted ReliefF Star (SWRF*), and applied it to synthetic SNP data. When compared to ReliefF and SURF*, which are two algorithms that have been applied to SNP data for identifying interactions, SWRF* had significantly greater power. Furthermore, we developed a framework called the Modular Relief Framework (MoRF) that can be used to develop novel variations of the Relief algorithm, and we used MoRF to develop the SWRF* algorithm.MoRF allows easy development of new Relief algorithms by specifying different interchangeable functions for the component terms. Using MORF, we developed a new Relief algorithm called SWRF* that had greater ability to identify interacting genetic variants in synthetic data compared to existing Relief algorithms. Correlating genetic variants with disease is the primary method used for elucidating the genetic causes of common diseases like Alzheimer’s disease and type 2 diabetes mellitus. Compared to common diseases, the genetic underpinnings of Mendelian diseases such as cystic fibrosis and thalassemia are generally simpler, as they are caused by variation at a single genetic locus. In contrast, common diseases are caused by genetic variants at multiple loci, with each locus conferring modest risk of developing disease. The most common genetic variation is the single nucleotide polymorphism (SNP), which is a DNA sequence variation occurring when a single base pair at a specific location in the genome differs among individuals in a population. Due to the rapidly falling cost of genotyping, the volume of genomic data available to researchers is growing very quickly. A key challenge in the analyses of such data is to develop computational methods that efficiently identify complex associations between genetic variants (such as SNPs) and disease.epistasis, wherein the genetic variants interact in a nonlinear fashion in their association with disease. Epistatic effects are particularly difficult for many algorithms to detect, because they may require examination of interacting genetic variants taken together, rather than one at a time. Because the number of potential interactions grows exponentially with the number of variants being examined, it is not computationally feasible to evaluate every potential higher order interaction in high-dimensional data. Many current methods address this problem by using a two stage procedure where they identify a subset of variants associated with disease in a univariate fashion in the first stage, and then evaluate interactions only within this smaller set of variants in the second stage. The attribute weight estimated by Relief has a probabilistic interpretation. It is proportional to the difference between two conditional probabilities, namely, the probability of the attribute’s value being different conditioned on the given nearest miss and nearest hit respectively[The Relief algorithm was first described by Kira and Rendell as a simectively.Relief AlgorithmW[a] = 0 for each attribute aset i = 1 to n dofor si from data at randomselect sample sh and nearest miss smfind nearest hit a dofor each attribute Wi[a] = diff - diffΔW[a] = W[a] + ΔWi[a]end forend fora dofor each attribute W[a] = W[a] / nend fordiff = 0, if si[a] = sj[a]where si[a] ≠ sj[a]= 1, if nearest hit and 1 nearest miss) during each iteration of the outer for loop in the pseudocode above. ReliefF uses 2k neighboring samples (k nearest hits and k nearest misses), and averages their contributions to update the attribute weights is the change in the weight for attribute a based on sample si. The final weight W[a] is simply the average of the individual updates, taken over a set of samples S. That is,n is the cardinality of S. The set S may constitute the entire dataset or a randomly selected subset of the dataset. We also have:where ∆class(si) is the class of sample si and si[a] is the value of attribute a in sample si. In Equation 1, f(distij) is a neighbor weighting function that assigns a weight to a sample sj based on its distance from the sample si. The role of the sum in the denominator in is to normalize the weight update so that W[a] is in the range . In the following sections, we describe each term in Equation 1 in greater detail.Where c and defined in Equation 2 is used to compare the class values of samples si and sj. It expresses the fact that Relief uses attribute mismatches between a sample and a neighbor in arithmetically opposite ways depending on if the neighbor is of the same class (a hit) or of a different class (a miss).The class comparator, denoted by diff , is used to determine the effect of a change along a single dimension (attribute). For nominal attributes such as SNPs, a simple match-mismatch diff function is used as defined in Equation 3. The difference between samples si and sj along dimension a is 0 if si[a] and sj[a] have the same values, and 1 if the values are different. However, other possibilities for the diff function can be considered. For example, samples siand sj are defined to be equal if both samples have one or more copies of the minor allele for attribute a, which corresponds to the dominant genetic model at locus a. The diff function can be defined in a similar fashion to represent the recessive or the heterozygote genetic model.The difference function, denoted by distij. The commonest distance measure for samples with nominal attributes such as SNP data is the Hamming distance, which is defined as the number of attributes at which two samples differ in value, and is shown in Equation 4. Equation 4 also shows that the Hamming distance is simply a sum of the diff function over all attributes. In general, distij need not be defined in terms of diff, and other possibilities for the global distance function can be considered for SNP data. For example, if SNP values are coded as 0, 1 or 2 , the taxicab distance is defined as follows:In order to locate sample neighbors, Relief uses a distance measure, denoted by In general, the global distance measures described above are susceptible to noise. This isf(distij) determines which neighbors count for how much in Relief’s update rule. Neighbor weights are determined as a function of distij, and take values from −1 to 1. A neighbor weight of 1 or −1 means that the neighbor influences the weight update maximally; a neighbor weight of 0 means that the neighbor does not influence the weight update at all. Several previously described variations of Relief differ essentially in the neighbor weighting function. For ReliefF, the neighbor weighting function is defined on a sample-wise basis, yielding a function fi for each sample i:The neighbor weighting function denoted by tihit is the distance of the kth nearest hit to si, and timiss is the distance of the kth nearest miss to si (assuming no ties for the kth hit or the kth miss). For SURF*, the neighbor weighting function is a step function defined globally as a function of a single parameter:where t is a user defined distance threshold . Figurewhere We now describe a new variation of ReliefF called SWRF*. In the MoRF framework, SWRF* differs from ReliefF and SURF* in that it uses a sigmoid neighbor weighting function that is defined as:t and u are parameters that define the center and width of the sigmoid function and distij is the Hamming distance. The value 4 scales the width u and was chosen as described in the next paragraph. This neighbor weighing function takes values close to 1 and −1 for nearby and distant neighbors respectively, and provides a smooth transition of weights for the intermediate neighbors. The star in SWRF* indicates that it uses both nearby and distant neighbors, similar to SURF*. In our experiments, we set t to the mean pairwise distance between samples, and u to the standard deviation of the pairwise distances. Figurewhere t. The width of the sigmoid function is scaled by a factor of 4. The value of 4 provided the best performance among a range of values that included 1, 2, 3, 4, 6, 8, and 16. We evaluated the candidate values for the scale factor using 50 replications of the 200 and 400 sample size datasets. If the scale factor is large (>16) the function becomes narrow and behaves like a step function. Less than 20% of the samples are given weights significantly different from SURF*, and the performance of SWRF* is similar to SURF*. If the scale factor is small (<1), the function becomes too wide and assigns less than 1% of the samples the “full” weight (|f(x)| > 0.95). The sigmoid function loses it characteristic plateau regions, and behaves like a ramp function. A scale factor of 4 assigns full weight to sample pairs that are more than roughly one standard deviation from the pairwise distance mean. This yields an intermediate number of neighbor pairs (~30%) which are assigned full weight.The motivation for using the sigmoid function in SWRF* compared to the step function in SURF* is to avoid the large change in weight that may occur due to a small change in distance as can happen with samples that are close to the distance threshold x-axis of each plot denotes the cutoff percentile (100th to 50th percentile in steps of 1 percentile point) for the rank-ordered list of the 1000 SNPs. The y-axis denotes the fraction of the 500 datasets in which the two epistatic SNPs are present in the list of rank-ordered SNPs above the specified percentile cutoff in the top ranked SNPs. At small heritabilities and sample sizes when the genetic effect is small all three algorithms performed poorly, while at large heritabilities and sample sizes when the genetic effect is large all three performed well. At intermediate heritabilities and sample sizes, SWRF* and SURF* performed significantly better than ReliefF, and SWRF* outperformed SURF*.The success rates of ReliefF, SURF*, and SWRF* on synthetic data are plotted in Figureth percentile cutoff  and intermediate to higher heritabilities . In addition, SWRF* never performed statistically significantly worse than SURF* at any of the MAF, heritability and sample size settings that we tested.TableEfficient identification of interactions in SNP data is an important and challenging computational problem. The Relief algorithms provide a promising first stage method for identifying disease associated SNPs large-scale genomic datasets, even in the presence of nonlinear interactions and small main effects.We have developed a framework called MoRF that abstracts the common features of the Relief algorithms. Several recently developed variations of the Relief algorithm may be viewed as distinct specifications of this framework’s neighbor weighting function. Novel enhanced variations of ReliefF can be developed by specifying new component functions in MoRF.Using the MoRF framework we developed a new variation of ReliefF called SWRF* that uses a sigmoid neighbor weighting function. SWRF* performed significantly better at identifying interacting genetic variants in synthetic data than ReliefF and the recently described method SURF*.k = 10. For SURF*, we set the distance threshold t to the mean pairwise distance between samples. For SWRF*, we set sigmoid function parameters t and u to the mean pairwise distance and to the standard deviation of the mean pairwise distance, respectively.We evaluated ReliefF, SURF*, and SWRF* on a large set of synthetic SNP datasets. For ReliefF, we set http://discovery.dartmouth.edu/epistatic_data/. The data were generated from 70 two-SNP epistatic models with different disease penetrance functions, minor allele frequencies (MAFs) of 0.2 or 0.4, and seven heritabilities ranging from 0.01 to 0.40  that might be expected for a common disease. To study the effect of sample size, from a given model, 100 datasets were generated for each of four sample sizes  where each dataset contains equal number of disease and healthy samples. For a generated pair of epistatic SNP values, a set of 998 SNPs that were assigned random values was appended to simulate SNPs that are non-informative with respect to the disease status. In all, 28,000 datasets were analyzed.We used synthetic SNP data that were used by Moore et al. in evaluating SURF*. The datth percentile implies that the top ranked 50 SNPs (top 5%) are examined for the presence of the two epistatic SNPs. To graphically compare the algorithms, we plotted the power of each algorithm for cutoffs varying from 100th percentile to the 50th percentile.We evaluated the performance of ReliefF, SURF*, and SWRF* on power. For a set of 500 datasets generated from a group of 5 related models, power is a number from 0 to 1 that denotes the proportion of datasets in which both epistatic SNPs are located in the list of rank-ordered SNPs above a specified cutoff. For example, in a dataset of 1000 SNPs, a cutoff of 95th percentile cutoff. To compare two algorithms at this cutoff, we compared the differences between their success rates using used Fisher’s exact test. For a set of 500 datasets generated from a group of 5 related models, the success rate is the number of datasets in which both epistatic SNPs are located in the list of rank-ordered SNPs above the 95th percentile cutoff. This value is represented by the points on the Figurex-axis values of 95. We considered a result to be statistically significant when p < 0.05, and we applied the Bonferroni correction in order to account for multiple hypothesis testing .We also statistically compared the performance of SWRF* with that of SURF* and ReliefF at the 95http://www.GitHub.com, username MattStokes42, project MoRF).The algorithms presented in this paper were implemented in the C# programming language. The program source code and compiled executables are available for download at the open source code repository GitHub (The authors have no competing interests.SHV conceived the study; MES developed, implemented and evaluated the algorithms. Both authors read and approved the final manuscript.


To better understand the power and localization ability of such an evolve and resequence (E&R) approach, we carried out forward-in-time population genetics simulations of 1 Mb genomic regions under a large combination of experimental conditions, then attempted to detect significantly diverged SNPs. Our analysis indicates that the ability to detect differentiation between populations is primarily affected by selection coefficient, population size, number of replicate populations, and number of founding haplotypes. We estimate that E&R studies can detect and localize causative sites with 80% success or greater when the number of founder haplotypes is over 500, experimental populations are replicated at least 25-fold, population size is at least 1,000 diploid individuals, and the selection coefficient on the locus of interest is at least 0.1. More achievable experimental designs  can have power of greater than 50% to identify a handful of SNPs of which one is likely causative. Similarly, in cases where  Quantitative traits are of special interest to biologists. The variation in many traits of medical, agricultural, and evolutionary relevance is due to the concerted action of several genes and the environment. Quantitative trait locus (QTL) mapping has been effective at explaining the majority of the heritability of a trait but is poorly suited to resolving the location of QTL beyond several cM . More reIn systems that have short generation times and that can easily be reared in the lab in large numbers, an alternative experimental approach to dissecting complex traits has been to “evolve and resequence” (E&R) populations of organisms. E&R studies have been performed with both asexual  and sexuTo date, the field of E&R has been almost entirely empirically motivated. Study designs have varied greatly in terms of the number of replicate populations, the population sizes maintained, the number of generations over which the experiment was carried out, and the number of haplotypes in the base population from which selection was initiated. For example, 2 test, and an a posteriori Dunnett test to detect significant allele frequency differences between treatments, whereas Furthermore, there are no agreed upon statistical approaches for analyzing the sets of pooled allele frequency estimates obtained from E&R studies. For example, n), number of founder haplotypes (h), selection coefficient on the CS of interest (s), number of replicated populations (r), and number of generations of evolution (g)  . We termWe observed that the false-positive rate for CR detection (even when using a very stringent criterion of significance) was extremely high using standard single-marker tests under a minority of conditions when ten replicate populations were used. The power to detect CRs was determined primarily by population size, replication, selection coefficient, and number of generations, with an intermediate number of generations being ideal. The power to localize CSs was similar to the power to detect CRs but was strongly affected by the number of founder haplotypes. Achieving a total power to detect CRs and localize CSs of 80% required almost all parameters to be at their ideal values  for the case of a selection coefficient of 0.1, but reasonable levels of power can also be achieved with less costly experimental designs or higher selection coefficients. Our simulations suggest that the experimental designs that could be most effectively utilized for detecting CRs and localizing CSs under the E&R paradigm are not currently widely employed and likely require considerable experimental effort. Still, the parameter space that provides reasonable power levels is not outside the realm of possibility for E&R studies using macroscopic organisms.s = 0 the fraction of cases in which at least one SNP was found to have a P value of less than 10−1, 10−2, 10−3, and so on, through 10−14. We referred to this as the false-positive CR detection rate . It is important that our 1 Mb false-positive rate is essentially zero; otherwise, there is a high likelihood of identifying a false-positive CR somewhere in a genome that is several hundred Mb in size. In a genome, the size of Drosophila melanogaster (122 Mb), the false-positive CR detection rate necessary to achieve a genome-wide false-positive rate of 0.05 is 0.05/122 = 0.00041. This corresponds to approximately 1 false positive in every 2,439 regions tested. To accurately measure the false-positive rate at low values, we generated 10,000 replicate simulations at each Θ where s = 0. With this number of replicate simulations, any Θ with four or fewer false positives has an acceptable error rate. At each Θ, we found the most lenient of our chosen significance thresholds that produced four or fewer false positives and used it in power calculations for the remainder of the experiment . This is a more fair comparison than choosing a single significance threshold that is applied to all Θ because the false-positive rate varies widely between Θ, so that a significance threshold that is reasonable for some Θ is unnecessarily strict for other Θ and would not provide a reasonable estimate of the maximum power achievable in those Θ. Θ, in which an acceptable false-positive rate was not achieved by our most strict significance threshold, 10−14, were discarded. This includes all experimental designs where r = 10, n = 100, h = 100, and g = 500 or 1,000 are simultaneously true. Of the 208 chosen thresholds , the distribution was as follows, with the first item in the list corresponding to 10−1, the second corresponding to 10−2, and so on: 0, 0, 42, 0, 7, 25, 42, 62, 20, 5, 1, 1, 1, and 2. The large number of significance thresholds set to 10−3 corresponds to the Θ in which r = 2; in these Θ, power and false-positive rates are both extremely low, so the selecting of a lenient significance threshold is unsurprising. The mean of the −log10 of the significance thresholds is 6.71.From the perspective of a naïve observer, any given simulated 1 Mb genomic region might or might not contain a CS. To determine the fraction of times that we falsely identified a CR, we calculated for every parameter combination (Θ) where ion rate ; that isP ≤ significance threshold). Because we only used 500 simulations per parameter combination where s > 0, we estimated the amount of error in estimates of power due to limited sampling by finding the 95% confidence interval around each power estimate using binomial sampling. We found that the mean width of the 95% confidence interval for all nonzero power estimates was 4.64%, the standard deviation of these widths was 3.31%, and the range of widths was 0.052–8.94%. The power to detect a CR increased with increasing r, n, and s, slowly decreased with increasing h, and was maximized at g = 500 when s = 0.05 and at g = 100 when s ≥ 0.1 . Below, we disregarded parameter values where CR power with that parameter value was always below 35%; specifically, we disregarded all Θ where s ≤ 0.005, r = 2, n ≤ 100, or the specific case where n ≤ 250 and r ≤ 5.Having controlled the false-positive rate via an individualized statistical threshold, we examined the ability to detect CRs. As in traditional QTL literature, there are two issues at hand. First, is it possible to find an association between genetic features and experimental treatment ? Second, if an association is found, to what level of precision can the polymorphism underlying the trait be localized ? As above, we considered a CR detected if it contained at least one significantly diverged SNP , the median MSM-CS distance is zero, whereas the mean is a nonzero value. We observed a similar pattern in the CS rank . Although selective sweeps are clearly visible in the raw significance scores , the fact that a large majority of the MSMs in most regions with high power have an MSM-CS distance of 0 seems to indicate that a sliding window analysis would be no better than a single-SNP analysis at localizing CSs. Indeed, our attempts to use a sliding window for CS localization by identifying the sliding window with the largest summed −log(p) values in each region produced lower power than single-SNP analyses . Thus, we chose to localize CSs through single-SNP analyses.The goal of an E&R study is CR detection followed by the identification of a CS within the detected CR. To determine most effective method of CS localization, we examined the distance from the most significant marker (MSM) to the causative SNP (MSM-CS distance) in each simulated region in which at least one SNP was significant . A large., 10 kb), a small number of SNPs , or within a small logarithm (base 10) of odds (LOD) drop of the MSM  to have utility, as additional experiments may be capable of identifying the CS. s = 0.1. g is set to 500 in all plots below except where specified for ease of viewing, and because the effect of g on power was relatively small in the parameter space where power is high. The primary factor that affected CS localization was h (the number of founding haplotypes). When h is small, it appears that high linkage disequilibrium results in significant allele frequency divergence at SNPs near the CS, making it difficult for the CS to be differentiated from neighboring SNPs. From h was high. h negatively affected CR detection power, yet positively affected CS localization power. As discussed later, the overall effect of h on power was positive due to the extreme effect of h on CS localization power. In a best case scenario where n = 1,000, s = 0.05, g = 500, and r = 25, an h of four produces an exact location power of only 4.0%, whereas an h of 100 produces an exact location power of 76.4%.We examined several methods of precisely localizing a CS, conditional upon identifying its CR as significant. Most strictly, we may identify a CS as being correctly localized only if the MSM in a CR is the CS. Alternatively, many would consider any analysis that restricts the likely location of a CS to a small region , can be considered the fraction of CRs in which the CS is not correctly localized. At least one of the localization false-positive rates calculated is below 5% in 123 of our simulated Θ, including but not limited to the entire simulated parameter space where g = 100. The range where h = 4 is excluded because the CS localization power conditional upon CR detection in these Θ is less than 80% for all statistics except the within 2 LOD power, and few E&R experiments use only four founder haplotypes. Total power is highest when s, n, h, and r are maximized, and g is at a value of 100. The parameters necessary to achieve at least 80% exact location power for the s = 0.1 case are n ≥ 1,000, r ≥ 25, and h ≥ 500 . This is a sobering result because it is experimentally difficult (in a system like Drosophila) to achieve values of Θ that reach a total exact location power above 80%. On the other hand, in the cases where s ≥ 0.1, the same goal of 80% exact location power is much more achievable: 21 of our simulated Θ, including but not limited to all cases in which s ≥ 0.1, h ≥ 500, r ≥ 15, n ≥ 1,000, and 100 ≤ g ≤ 500 produce a total exact location power greater than 80%. Thus, exact localization requires relatively strict experimental conditions, but strongly selected SNPs are more easily localized. Unsurprisingly, within 10 kb power, top 25 power, and within 2 LOD power were consistently higher than exact location power and were higher than 80% when s ≥ 0.05, n ≥ 1,000, r ≥ 25, g = 500, and h ≥ 32, except in the case where s = 0.05 and h = 100, suggesting that ambitious, yet achievable, experimental designs are capable of localizing CSs to a few dozen or even fewer candidate SNPs.Total power is the product of CR detection power and CS localization power given CR detection. Total power is then the fraction of all CSs that, starting from no prior knowledge about the data, can be detected and localized successfully. n and r when setting up an E&R study because both of these parameters are space and resource limited. Both affect the total power differently: increasing replication (r) improves the number of degrees of freedom during statistical analysis, whereas increasing the population size maintained during the experiment (n) decreases the effect of genetic drift on allele frequencies. Both parameters are subject to diminishing returns as their values are increased. For example, in the case where s = 0.05, g = 500, r = 10, and h = 100, a doubling of n from 250 to 500 increases exact location power from 5% to 27%, whereas a doubling of n from 500 to 1,000 increases exact location power from 27% to 46%. With the same parameters and an n of 500, an increase of r from 5 to 10 increases exact location power from 8% to 27%, but a similar increase in r from 10 to 15 only increases exact location power from 27% to 43%. Because diminishing returns occur, the ideal r and n values for a given laboratory size should be balanced, with the specific values depending on the specific conditions of the experiment . Unfortunately, it is difficult to determine an ideal r:n ratio because multiple costs are involved: the cost of more replicates versus more generations, the cost of sequencing versus rearing, and so on.In many experimental systems, there is a direct tradeoff between g on the total power to detect and localize CSs was small in the parameter space where power was high, so g was omitted from several plots for simplicity. It was apparent that there was a strong interaction between g and s with respect to power. At s = 0.05, an intermediate g (500) appeared to be superior to either high  or low (100) g values in terms of the power to detect CS-containing regions and the total power to localize CSs , but that the total number of fixed alleles continued to increase even when g = 1,000, implying that functional standing genetic variation in fitness was largely exhausted by generation 500, but that drift at linked neutral markers continued to occur. This result seems to confirm that rapid selection and slow drift cause intermediate numbers of generations to be ideal for CS detection and localization.As noted earlier, the effect of lize CSs ; at s = s, r, g, h, and n . We generated a table of total exact location power and the five experimental design variables of interest, then censored it in R to only contain the Θ where 10 ≤ r ≤ 25, 250 ≤ n ≤ 1,000, 32 ≤ h ≤ 500, 0.05 ≤ s ≤ 0.2, and 100 ≤ g ≤ 1,000 to focus on modeling the power curve in the area where power is highest. We then used the lm function in R to fit the linear model below:We used multiple linear regression to attempt to create a model that predicts exact location power as a function of the s, r, h, and n by applying the log10 function to them as this improved the fit of the model. In the limited parameter space examined, the linear model explains 86.9% (adjusted R2) of the variation in total exact location power and has a standard error of 0.07782. Values produced by this formula that are above 1 or below 0 should be assumed to be, respectively, 1 or 0. Although this equation does not take interactions between experimental conditions into account, it produces a relatively accurate power estimate in the aforementioned parameter range.Before calculating the slopes, we modified s = 0.05, 0.1, and 0.2 (the selection coefficients that produced reasonable power levels in the previous simulation), g = 100, 500, and 1,000, and r = 2, 5, 10, 15, and 25. We simulated two different combinations of h and n: 1) the highest power level that we simulated  and 2) a moderate power level . We generated 250 replicate experiments under each of these parameter combinations. In each replicate, one of the CSs was placed at the center of the chromosome, and the others were randomly distributed throughout the chromosome but not within the 1 Mb region surrounding the central CS. The external CSs always had the same s as the central CS. Over the course of the forward simulation, the allele frequencies of all SNPs in the 1 Mb region surrounding the central CS were recorded and used to calculate P values. We analyzed the resulting P values according to the same framework used in the previous simulation. When compared with the single CS simulation, the multiple CS simulations almost universally produced higher CR detection power and lower CS localization power . This result is expected. A larger number of neighboring CSs should increase the average significance of SNPs in the region of interest by increasing the probability that an external CS is adjacent to the focal region. This should increase CR detection power by increasing the probability that at least one SNP will be significant, but decrease total power by decreasing the probability that the CS will be the most significant SNP in the region. For the three cases where the number of external CSs was equal to 5, 25, and 50, the average shortest distances from the focal CS to the closest external CS were, respectively, 2.05 Mb, 0.87 Mb, and 0.68 Mb.We simulated the possible case of a 20 Mb chromosome containing 6, 26, or 51 CSs to test the effect of multiple CSs on CR detection power and total power. Specifically, we simulated supplementary fig. S11, Supplementary Material online) of fitnesses at each simulated Θ in which more than one CS is present. As expected, the distribution of fitness becomes broader as s and the number of CSs increase. The variances that we observed  when the number of external CSs was 25 or larger seem much higher than those observed in natural populations . Indeed, all Θ in which /knockin . Given tSupplementary figure S13, Supplementary Material online, shows the CR detection power and false-positive rate when s = 0.05 at the simulated Θ that most closely match the Θ used by existing studies. The power levels at our chosen significance thresholds and when s = 0.05, conditional on using our modified t-statistic, were 19%, 1.4%, 0, 0, 0, and 0 for s = 0.1, generated using our chosen significance thresholds, were 27.8%, 44.6%, 0, 0, 0.2%, and 0.2%, whereas respective powers for s = 0.2 were 34%, 87%, 0, 0, 20.4%, and 20.4%. Respective significance thresholds chosen by our system  were 10−7, 10−6, 10−3, 10−3, 10−3, and 10−3. For the same studies, we estimate that CR detection false-positive rates were all equal to 0, again conditional upon our statistical test. Although we estimate a high CR detection rate for Our results allow us to reflect on the validity of some of the conclusions drawn in published E&R studies by examining the CR detection power and the false-positive rate that we calculated at the Θ that most closely match published experiments. n of 27–44 should make even the detection of CSs with a selection coefficient of 0.2 difficult in this case, artificial selection usually involves very high selection coefficients that may be high enough to override the force of genetic drift. Despite our simulations suggesting low power and high false-positive rate, several factors prevent the outright dismissal of published studies. First, gene ontology analysis of genes in regions enriched for change in published studies is consistent with the characters being selected upon. For example, the top five enriched gene ontology terms from h is low. In our simulations, when h was ≤32 in a population and power was greater than 0, the average exact localization power across our simulations was only 0.244. Although the selection of a 1 Mb region for our simulations was somewhat arbitrary, we believe it is an appropriately sized region to consider. The selective sweeps that occurred in our simulated populations appeared to extend less than 500 kb from the CS under most circumstances , and few replicate simulations generated an MSM-CS distance greater than 100 kb. Indeed, in the parameter space where s ≥ 0.05, 95% of all simulated regions that contained at least one SNP significant at a 10−8 threshold had an MSM-CS distance ≤ 228 kb. Thus, any SNPs simulated further from the CS would resemble SNPs in neutral regions. Similarly, our decision to use the entire simulated region as a candidate CR instead of using only a limited area  is justified in that the various powers simulated here are virtually unchanged when one compares the power using a 1 Mb region and a 100 kb region. The mean difference between the 1 Mb CR detection power and the 100 kb CR detection power is 0.3%, indicating that nearly all CSs that can be detected can be localized to an area the size of a selective sweep around the MSM; in other words, it is reasonable to conclude that CR detection power = total within 100 kb power.This study makes a number of simplifying assumptions, all of which we believe to be realistic when describing the case of experimental evolution of small populations. Our simulation machinery operates based on the Wright–Fisher model of population genetics, in which the gametes of each generation are aggregated into a gene pool to generate the next generation. The assumptions of this model, which generations are discrete and mating is random, are realistic for experimental evolution. A further assumption is that all heritable variation is additive within and between loci. Although it is certainly true that nonadditive variation exists, the majority of heritable variation is likely additive in nature ; therefoNotably, our goal in simulating 1 Mb regions was not to test the efficacy of the particular CS-detection technique used here; researchers attempting to adapt this technique to empirical use would need to first divide their genome of interest into arbitrary 1 Mb blocks to perform our CR detection step, which would be an unnecessarily arbitrary method of subdividing a genome. Rather, our reasoning for choosing to simulate 1 Mb blocks was to be certain to capture all the genetic change due to linkage to the CS in each simulated region. Our CR detection power is thus an upper bound on the ability of a study with a certain set of experimental parameters to detect the presence of any particular CS. It gives no indication as to the ability of that study to localize that CS, except perhaps to say that if a significant SNP is located, the CS that drove its divergence must be close enough to it to have affected its allele frequency via a selective sweep. Some problematic effects that could occur if our CR detection method were applied as-is to real life data, such as the possibility that a CS could be immediately adjacent to a 1 Mb focal region and could thus drive a SNP to significance in a non-CS-containing region, are not considered further here.P values of multiple insignificant SNPs to determine the significance of a region because there is no simple way to determine the probability of observing any particular set of multiple P values if, as in this case, the P values are not independent. Further, the advantage of a combined P value approach (higher CR detection) would presumably be at its largest in the Θ where linkage disequilibrium is very high, such as when h is low, but such Θ have already been established as having very high CR detection power, so the advantage gained from a combined P value approach would be minimal. On a similar note, we did not attempt to simulate a distribution of selection coefficients across the loci in our simulated genomic regions. Recent studies have raised the question as to whether the majority of heritability for any particular trait is best explained by a small number of mutations of large effect , indicating that plateauing allele frequency trajectories will not severely reduce power. This is evidenced by the fact that, at the Θ where n = 1,000; h = 500, r = 25, and s = 0.05, a near doubling of mean CS allele frequency over all 500 simulation replicates from 52% at generation 100 to 94% at generation 500 only increased total exact location power from 71% to 76%.A final model we did not consider is the possibility of plateauing allele frequencies due to diminishing selection pressure as a phenotypic optimum is approached, as hypothesized in Our simulation indicates that, in spite of their inability to detect CSs of very small effect, E&R studies should be capable of detecting and localizing the majority of CSs of moderate to large effect under conditions that, whereas more labor-intensive than traditional experimental evolution conditions, are still feasible. The effectiveness of the next generation of E&R experiments will depend on their ability to improve upon the experimental designs of the past by using large, well-replicated, initially diverse populations.macs 2000 1000000 -t 0.01 -r 0.1 -i 1000 -s $RANDOM. This command line specifies 1,000 replicate simulations of a sample of 2,000 chromosomes. The locus length is 1 million base pairs mutating at rate θ = 4, Nu = 0.01 per site, and recombining at rate ρ = 4N*(recombination rate) = 0.1 per site, where N is the size of a Wright–Fisher population and u is the mutation rate, per base pair per generation. The mutation parameter was chosen to mimic SNP density in non-African Drosophila melanogaster, and the recombination rate was based on estimates from We simulated replicated experimental evolution using a two-stage approach. First, we simulated 1,000 replicates of a sample of size 2,000 chromosomes from a Wright–Fisher population using the “macs” software  previously used in h haplotypes is equivalent to randomly choosing h haplotypes. These h founding haplotypes were replicated r times, and then r large “base” populations of n diploids each were generated by sampling with replacement from the initial coalescent simulations. A single site was assigned a positive selection coefficient, and experimental evolution was simulated using forward-in-time simulations. The forward in time simulations were carried out with various population sizes (n), numbers of founder haplotypes (h), numbers of replicate populations evolved (r), numbers of generations of experimental evolution (g), and selection coefficients (s). Note that, because of the lack of population structure in these populations, the actual population size should be equal to the effective population size; in a study of real data, the effective population size would be more comparable to the n used here because of nonrandom mating and population size fluctuations. The SNP under selection, or the CS, was always the centermost SNP in the region. The selection scheme was codominant with fitnesses 1, 1+s/2, and 1+s, where s is the selection coefficient on the CS. CSs thus followed the same distribution of initial allele frequencies as all other SNPs in the simulation, consistent with a SNP that is initially neutral, but that is selected upon following a change in environmental conditions. Linkage disequilibrium between SNPs is initially an outcome of the neutral Wright–Fisher sampling process used to generate the h founder haplotypes, and subsequently determined by the details of the forward-in-time simulation. The forward simulations assume no further mutation in the region, and the recombination rate used was 0.025 per diploid per generation assuming that the 1 Mb region is 5% of a “typical” 20 Mb chromosome whose total recombination rate per generation is 0.5.The outputs from macs were used to seed forward-time simulations using the first n diploids), number of founder haplotypes (h = twice the number of founding diploids), number of replicate populations (r), and selection coefficient (s) on the CS. In total, 500 replicate simulations were performed under each of 840 possible combinations of experimental parameters (Θ) (h was larger than n were not simulated because such populations would presumably closely resemble populations in which h was reduced to the level of n. At each number of generations (g) in which allele frequencies were recorded, a modified t-statistic was calculated on arcsine square-root transformed SNP frequencies using 2, 5, 10, 15, or 25 replicate populations. This empirical Bayesian t-statistic , all containing individuals with the same genomic region. Haplotypes were generated based on the initial allele frequencies, and individuals carrying these haplotypes were created. A forward-in-time simulation was then used to keep track of the movement of haplotypes over time with recombination and selection applied. Allele frequencies were calculated and recorded at 100, 500, and 1,000 generations. Although a true E&R experiment would have allele frequency estimation errors that are a complex function of number of individuals sequenced, library preparation methods, average sequence coverage, and variation in sequencing coverage, we chose to simulate the best-case scenario where all allele frequencies are estimated without error. Thus, our estimates of power are likely somewhat optimistic. We performed simulations that varied in population size (ters (Θ) . Combinatatistic  indicateP values were calculated from the modified t-statistic using the pt function in R, using a two-tailed method . A 2-tailed t-test was used to avoid making a priori assumptions about the nature of the two alleles involved at any given locus: because either allele could be beneficial in theory, it is not reasonable to assume that only the allele whose frequency is being tracked could be beneficial. Degrees of freedom were considered to be 2 r − 2 . The threshold for significance was set independently for each Θ by calculating the false-positive rate for every whole-number power of 10 from 10−1 to 10−14 and choosing the most lenient threshold with an acceptable false-positive rate . Power was calculated by finding the fraction out of 500 times that 1) at least one SNP was significantly diverged in a region of interest and 2) a secondary condition was met. These secondary conditions included having the MSM in the region be the CS (exact location power), having the MSM be within 10 kb of the CS (within 10 kb power), having the CS be among the top 25 MSMs (top 25 power), and having the CS's LOD (logarithm base 10 of odds) score be within two of the MSM's LOD score (within 2 LOD power). The power to fulfill the first condition without regard for a second condition was termed “CR detection power.” This diversity of methods of SNP localization allowed us to determine which method would be most reliable under any particular set of experimental parameters. The CR detection false-positive rate was determined by finding the fraction of cases in which a region with an s of zero contained at least one significant SNP. The distance from the MSM to the CS (MSM-CS distance) and the rank of the CS's P value compared with the other SNPs in its region (CS rank) were calculated in every replicate of the 500 replicate simulations per Θ to analyze the distribution of significant SNPs across parameter values.s = 0. 10,000 replicate neutral simulations were performed for each Θ where s = 0. These additional replicates were used to more accurately calculate the false-positive CR detection rate. This high level of replication was only required for the calculation of false-positive rates because the maximum allowable false-positive rate is too small  to be accurately measured with only 500 replicate experiments.An additional 18,000 replicate genome regions were generated in macs and used to seed forward-in-time simulations with no selection under all of our Θ where r), number of haplotypes in the base population (h), population size (n), selection coefficient at the selected locus (s), and number of generations of selection (g). The resulting data sets are not completely independent because the coalescent simulation used to generate the 1 Mb regions used here was only run 500 times , and the resulting 500  genome regions were reused for the 500  replicate experiments for each parameter combination. Further, for any particular combination of n, r, s, and h, the three g values simulated were not entirely independent because the data associated with larger g values were derived from continuing the forward simulations of the smaller g simulations. Values were chosen based on the specifics of the variable: h, n, and g values were chosen based on the levels historically used in experiments of this type. The r values were chosen based on the level of replication commonly used in experimental evolution and the level of replication required for high power. Experimental evolution of sexual organisms is usually carried out with five or fewer replicate populations due to the difficulty of rearing large numbers of populations, but, especially in genomics, statistical significance is difficult to achieve with low r values because the large number of independent comparisons require a strict significance threshold. s values were chosen based on the minimum selection strength necessary for selection to have an effect sufficiently stronger than genetic drift to produce a measurable change in allele frequencies: when the selection coefficient at an SNP is less than ∼1/(2n), genetic drift is a more powerful force than selection  to an s larger than 1/(2n).This simulation produced 2,205,000 semi-independent experimental results. There are 500 pure replicates of each possible permutation of five distinct experimental parameters' values —number oelection , p. 425.http://www.molpopgen.org/Data , as is a commented copy of the scripts used to calculate P values, power, and other statistics. macs is available at http://code.google.com/p/macs/ .The simulation code and all data and the necessary code to recreate the data are available online at Supplementary tables S1 and S2 and figures S1–S13 are available at Molecular Biology and Evolution online .