Fields as diverse as human genetics and sociology are increasingly using polygenic scores based on genome-wide association studies (GWAS) for phenotypic prediction. However, recent work has shown that polygenic scores have limited portability across groups of different genetic ancestries, restricting the contexts in which they can be used reliably and potentially creating serious inequities in future clinical applications. Using the UK Biobank data, we demonstrate that even within a single ancestry group , the prediction accuracy of polygenic scores can depend on characteristics such as the socio-economic status, age or sex of the individuals in which the GWAS and the prediction were conducted, as well as on the GWAS design. Our findings highlight both the complexities of interpreting polygenic scores and underappreciated obstacles to their broad use. Complex diseases like cancer and heart disease are caused by the interplay of many factors: the variants of genes we inherit, the lifestyles we lead and the environments we inhabit, plus the interaction of all these factors. In fact, almost every trait, even how many years we will spend studying, is influenced both by our environment and our genes.To identify some of the genetic factors at play, scientists perform analyses known as genome-wide association studies, or GWAS for short. In these studies, the genomes from many different people are scanned to look for genetic differences associated with differences in traits. By summing up all the small genetic differences, so-called “polygenic scores” can be calculated. When there is a large genetic component to a trait, polygenic scores can be useful predictive tools.But there is a catch: polygenic scores make less accurate predictions for individuals of a different ancestry than those involved in the GWAS, which limits the use of these tools around the world. Mostafavi, Harpak et al. set out to understand if there are other factors in addition to ancestry that could influence the performance of polygenic scores.Using data from the UK Biobank, an international health resource that pairs genomic data and clinical information, Mostafavi, Harpak et al. examined polygenic scores among individuals that share a single, common ancestry. These polygenic scores were used to predict three traits  in individuals and the predictions were then compared to the actual trait values to see how accurate they were. The analysis revealed that even within a group of people with similar ancestry, the accuracy of polygenic scores can vary, depending on characteristics such as the sex, age or socioeconomic status of the individuals.This analysis emphasises how variable GWAS and their predictive value can be even within seemingly similar population groups. It further highlights both the complexities of interpreting polygenic scores and underappreciated obstacles to their broad use in medical and social sciences. Genome-wide association studies (GWAS) have now been conducted for thousands of human complex traits, revealing that the genetic architecture is almost always highly polygenic, that is that the bulk of the heritable variation is due to thousands of genetic variants, each with tiny marginal effects . These fmeasure)  and up tre) (R2) .This genomic approach to phenotypic prediction has been rapidly adopted in three distinct fields. In human genetics, PGS have been shown to help identify individuals that are more likely to be at risk of diseases such as breast cancer and cardiovascular disease . Based oAs recent papers have highlighted, however, PGS are not as predictive in individuals whose genetic ancestry differs substantially from the ancestry of individuals in the original GWAS  included in current GWAS are of European ancestry , PGS areImportantly, factors other than ancestry could also impact the accuracy and portability of PGS. For example, the educational attainment of an individual depends not only on their own genotype, but on the genotypes of their parents, due to nurturing effects , and of Given these considerations, it is important to ask to what extent PGS are portable among groups within the same ancestry. To explore this question, we stratified the subset of UK Biobank samples designated as ‘White British’ (WB) according to some of the standard sample characteristics of GWAS studies: the ages of the individuals, their sex, and socio-economic status. We chose to focus on these particular characteristics because they vary among GWAS samples depending on sample ascertainment procedures. Furthermore, these characteristics have been shown to influence heritability for some traits in a study of a subset of the UK Biobank , raisingAt present, it is difficult to determine the reasons why we see such variable prediction accuracy across these strata and study designs. Contributing factors probably include indirect genetic effects from relatives, assortative mating, varying levels of genetic and environmental variance, GxE interaction effects and perhaps undetected confounding. Nonetheless, our results make clear that the prediction accuracy of PGS can be affected in unpredictable ways by known—and presumably unknown—factors in addition to genetic ancestry.We examined how PGS for a few example traits port across samples that are of similar genetic ancestry but differ in terms of some common study characteristics, such as the male:female ratio (henceforth ‘sex ratio’), age distribution, or socio-economic status (SES). To this end, we limited our analysis to the largest subset of individuals in the UKB with a relatively homogeneous ancestry: 337,536 unrelated individuals that were characterized by the UKB, based on self-reported ethnicities as well as genetic analysis, as ‘White British’ (WB) . In all In all analyses, we randomly selected a subset of individuals to be the prediction set; we then conducted GWAS using the remaining individuals and built a PGS model by LD-based clumping of the associations . To examine the reliability of the prediction, we considered the incremental es e.g., , we alsoAs a first case, we considered the prediction accuracy of a PGS for diastolic blood pressure in prediction sets stratified by sex, motivated by reports that variation in this trait may arise for somewhat distinct reasons in the two sexes . We randWe then considered two other cases, evaluating prediction accuracy in groups stratified by age for BMI—since the UK Biobank participants were enrolled within about a five-year span, differences in age could in principle also be reflective of cohort effects—and by adult SES for years of schooling, using the Townsend deprivation index as a measure; our choices were motivated by prior evidence suggesting that these characteristics of the GWAS influence estimates of SNP-heritability . We with-4 for inclusion of a SNP in the PGS. The choice of how stringent to make the GWAS p-value threshold is important but somewhat arbitrary, with approaches ranging from requiring genome-wide significance to including all SNPs , as expected if the genetic variance is fixed across strata. What we find instead is that the estimated SNP heritabilities for all three traits increase or remain the same with increasing phenotypic variance . Thus, fAnother possibility is that there is an interaction between genetic effects and sample characteristics, for instance that different sets of genetic variants contribute to blood pressure levels in males and females or to BMI across different stages of life.  This explanation is not supported by bivariate LD score regression, which indicates that the genetic correlations across strata are close to 1  ; in thatAnother potentially important factor is that the adjustment for PCs may not be a sufficient control for the different ways in which population structure can confound GWAS results , leadingBeyond sample characteristics such as age or sex, a number of other factors may shape the portability of scores across groups of similar ancestry. Standard GWAS is done in samples of individuals that deliberately exclude close relatives; as implemented, it detects direct effects of the genetic variants, but also any indirect genetic effects of parents, siblings, or peers, effects of assortative mating among parents, and potentially environmental differences associated with fine-scale population structure . Given tTo demonstrate that these considerations are not just hypothetical, we compared the prediction accuracy when the PGS is trained on ‘unrelated’ individuals such as those used in a standard GWAS to one obtained from a sibling-based (or ‘sib-based’) GWAS . In the latter, genotype differences between sibs, a result of random Mendelian segregation in the parents, are tested for association with the phenotypic differences between them. Because the tests depend on phenotypic differences between siblings who, of course, have the same parents, these tests are conditioned on the parental genotypes and hence exclude many of the indirect effects signals that may be picked up in standard GWAS (Appendix 1). Differences between standard and sib-based GWAS are thus informative about the presence of factors other than direct genetic effects .A challenge in this comparison is that the UKB contains only ~22K sibling pairs, ~19K of whom are labeled as ‘White British’ (WB). The siblings are similar to the unrelated individuals in terms of ages, SES distributions and genetic ancestries  and 7 buTo increase power and enable a direct comparison between the two designs, we split the SNP ascertainment and effect estimation steps as follows : we idenWe applied the approach to 20 traits, focusing on traits with relatively high heritability estimates as well as social and behavioral traits that have been the focus of recent attention in social sciences. For the majority of the traits, such as diastolic blood pressure, BMI, and hair color, the prediction accuracies of standard and sib-based PGS were similar , as expeA number of factors could contribute to the differences between prediction accuracies for PGS based on sibs versus unrelated individuals, including confounding effects of population stratification, indirect genetic effects from parents and assortative mating. The relative importance of each factor will vary across traits . For eduThe lower prediction accuracies for PGS based on sib-based GWAS indicate that complications such as assortative mating or indirect effects contribute to the standard GWAS estimates. In the absence of these complications, we ensure that prediction accuracies are comparable by matching the sampling errors of the two approaches . In the Motivated by these considerations, we examined how the prediction accuracy varies when progressively relaxing the GWAS p-value threshold for inclusion of SNPs, that is when including more weakly associated SNPs in the PGS. In . For haiIn summary, the differences between the prediction accuracies of standard and sib-based PGS seen for a number of traits , notablyAlthough the conversation around the portability of PGS has largely focused on genetic ancestries, our results show that prediction accuracy can also differ, in some cases substantially, across groups of similar ancestry—even due to basic study design differences such as age, sex or SES composition. When due only to increased environmental variance, such decreased accuracy may not pose a problem, at least for certain applications. But as we have shown, differences in the degree of environmental variance are not the primary explanation for the patterns we report , and oth2), and 'college completion', and evaluated the prediction accuracy as measured by incremental AUC  , the samOne fruitful way forward may be to study data from related individuals, in which it should be possible to decompose the components of the signals identified in GWAS into direct and indirect effects, the degree of assortative mating and the contribution of residual stratification . Not onlThe UK Biobank (UKB) is a large study of about half a million United Kingdom residents, recruited between years 2006 to 2010 . In addiIn this study, we focused on 408,434 participants who passed quality control (QC) measures provided by UKB; specifically, for whom the reported sex (QC parameter ‘Submitted.Gender’) matched their inferred sex from genotype data (QC parameter ‘Inferred.Gender’); who were not identified as outliers based on heterozygosity and missing rate (QC parameter ‘het.missing.outliers’==0); and did not have an excessive number of relatives in the database (QC parameter ‘excess.relatives’==0). We further selected individuals identified by UKB to be of ‘White British’ (WB) ancestry (QC parameter ‘in.white.British.ancestry.subset’==1), which is a label that refers to those who, when given a set of choices, self-reported to be of ‘White’ and ‘British’ ethnic backgrounds and, in addition, were tightly clustered in a principal component analysis of the genotype data, as detailed in 2, and ‘college completion’ defined based on attainment of a college or a university degree. Disease outcomes were ascertained using self-reported information and/or using the hospital inpatient main and secondary diagnoses coded according to the International Classification of Diseases (ICD-9 and ICD-10). Hypothyroidism, type 2 diabetes, and rheumatoid arthritis were ascertained based on ICD-10 codes of E03.X, E11.X and M06.X, respectively. Myocardial infarction was ascertained based on ICD-9 codes of 410.9, 411.9, 412.9, or ICD-10 codes of I21.X, I22.X, I23.X, I24.1, I25.2 following We focused on 25 traits, including traits with relatively high heritability estimates as well as social and behavioral traits that have been the focus of recent attention in social sciences see . For diaplink v. 1.90b5 , sex, and Townsend deprivation index at recruitment . For SES and age, we divided the sample into four sets: Q1 , Q2 , and in a diverse sample with equal number of individuals from all four groups ; for body mass index (BMI) in Q1, Q4, and in a diverse sample with equal number of individuals from all four groups ; and for diastolic blood pressure in males, females, and in a diverse sample with equal number of males and females . We performed all GWASs using plink v. 2.0 (with the flag --linear), adjusting for sex, age (at recruitment) and first 20 PCs as covariates. PCs are principal components of the genotype data, as provided by UKB, calculated using the entire cohort . For a subset of cases (where GWAS was performed in samples restricted by characteristics described above), we additionally performed association tests using a linear mixed model (LMM) as implemented in BOLT-LMM v. 2.3.2 , defined such that no pairs of individuals are inferred to be 3v. 2.3.2 , using LTo better understand the performance of PGS across the strata (see ‘Possible explanations for the variable prediction accuracy’), we estimated the mean effect sizes of significant SNPs in each of the strata. To avoid overfitting, we first performed an association test in the pooled sample of all strata excluding individuals in the prediction sets and matching the number of individuals per stratum; sample size 293,132 for years of schooling, 272,456 for BMI, and 245,548 for diastolic blood pressure. Then for significantly associated SNPs (LD pruned as described in ‘Polygenic score construction and trait prediction’), we re-estimated the effect sizes in each of the strata in the prediction sets see . We alsoIn addition to above examples, we explored the prediction accuracy for years of schooling when GWAS and prediction sets are stratified based the participants’ number of full siblings. Specifically, we performed GWAS using individuals who had exactly one sibling , and evaluated prediction in two independent samples of individuals who reported having no siblings or having one sibling (sample size 20K for each) see .plink v. 2.0 with the flag --logistic). An important caveat to analyses of disease outcomes recorded during multiple follow-ups is that for ‘age’, we could only consider the age at recruitment in the GWAS; that approach is not ideal, considering that a fraction of individuals died during the course of the study .We also considered five binary disease outcomes stratified by sex. Specifically, we performed GWAS in equally sized samples of males and females for hypothyroidism , type 2 diabetes , rheumatoid arthritis , myocardial infarction  and having been diagnosed with a heart attack or angina or stroke , leaving out 20K samples of males and females for prediction see  and 15. rd degree or closer).We used the genetic relatedness information provided by UKB to infer sibling pairs among the WB samples. Following 'Estimating plink v. 1.90b5’s QFAM procedure with the flag --qfam), and by a standard association test in the Unrelateds-plink v. 2.0). Subsequently, for each set of ascertained SNPs in the Unrelateds-discovery set, two PGS were constructed for the samples in the Unrelateds-prediction set , adjusting for sex, age and first 20 PCs as covariates. For sibling-based association tests we first residualized the phenotypic values on age and sex, and then regressed the sibling differences in residuals on sibling genotypic differences using plink’s QFAM procedure as described above.All standard association tests were performed using We also considered a version of the analysis described above, in which we first residualized the phenotypes on covariates in the pooled sample of all WB individuals, and then ran the pipeline on the residuals without further adjustment for covariates in the GWAS or prediction evaluation. As shown in We wanted to check that given the study design described above, sibling-based and standard PGS perform similarly with respect to trait prediction, under the vanilla model of no population stratification, assortative mating or indirect genetic effects . To thishttp://www.nealelab.is/uk-biobank), namely autosomal SNPs, imputed using the haplotype reference consortium (HRC) panel, which have INFO score > 0.8 and have minor allele frequency > 10−4; we further limited the SNP set to ones that were bi-allelic in the WB sample. As in We randomly selected the causal SNPs from a set of 10,879,183 imputed SNPs, considering that most causal variants are plausibly not directly genotyped on SNP arrays. We used a set of SNPs that passed quality control procedures by the Neale lab . We then used plink’s clumping procedure (with the flag --clump) with LD threshold For all GWAS designs described above, we used p-value thresholding followed by clumping to choose sets of roughly independent SNPs to build PGS. We considered a logarithmically-spaced range of p-values: 10e causal . To evalWe calculated SNP heritability across sex, age and SES groups for diastolic blood pressure, BMI and years of schooling, respectively (as described in the section ‘GWAS by sample characteristics’) as well as genetic correlations across pairs of groups: we first performed GWAS using all unrelated WB individuals in each group. We then used the GWAS summary statistics to perform LD score regression with LD scores computed from the 1000 Genomes European-ancestry samples . In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.Acceptance summary:Polygenic scores are being generated for an increasingly large number of traits, and are beginning to be used outside of a narrow research context. However many questions remain about how and when it is appropriate to use them. One oft-discussed concern is whether polygenic scores are "portable" between ancestry groups. But this study demonstrates that polygenic scores are often not even portable within ancestry groups when stratified on the basis of common demographic parameters. This and other results discussed in the paper raise important questions about the use of polygenic scores, and should be read and considered carefully by anyone designing, carrying out or applying the results of large-scale human genetic analyses.Decision letter after peer review:eLife. Your article has been reviewed by three peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Mark McCarthy as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Paul O'Reilly (Reviewer #3).Thank you for submitting your article "Variable prediction accuracy of polygenic scores within an ancestry group" for consideration by The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.The manuscript in its current form would not, even with minor revision, be suitable for the journal. However, we would consider an extensively revised manuscript that addresses the following key concerns of the reviewers.1) The paper will benefit from a clearer focus; while the authors aim was to simply establish the variable of portability between strata, they went quite further from this with the topics of assortative matting, sex balance, parents' effects, etc. Just achieving their main aim would make this paper very worthy as it points to issues that those already thinking in applying PRS in clinical practice right now need to urgently appreciate.2), than PRS based on any of the strata.2) In focusing on the main aim, a deeper exploration of the causes that drive the lack of portability would increases the informativeness of the study. It was pointed out that the difference in portability across strata might be due to differences in power, sample size, heritability, phenotypic variance, phenotype distribution, influence of environment, insights in how these and potentially others affect portability would be helpful. This should also include a discussion on how is it that different strata can generate even higher predictive power than using the same stratum (in base and target) and why the PRS based on the full sample size, in the provided examples, always has a better "predictive ability" (highest R3) Analysis of disease outcomes, here e.g. (extreme) obesity and hypertension, will be useful to translate findings to more clinical settings, including calculation of ROC-AUC, a standard easy-to-interpret predictive statistic.Reviewer #1:In this study, the authors examine the portability of polygenic scores within a homogeneous, generally healthy population of white British unrelated adults, living across the UK. They show that the PGS performance is influenced by age, sex, socio-economic status and study designs. The reduced accuracy of PGS is not (only) caused by environmental influences, the authors show that also the magnitude of the genetic effects among groups, and indirect effects of assortative mating affect the performance of the PGS.While these findings raise important consideration with reference to the use of PGSs, it would have been interesting if they had gone a little deeper with their analyses; e.g. to also identify the features  of outcomes and covariates that impact the prediction performance the most.2 for sex-specific, age-specific and SES-specific PGSs, the highest R2 (still low) is achieved when the PGS is generated in the full population. Thus, isn't the conclusion then simply that stratification lowers the PGS performance and that a (blunt) PGS, based on the most comprehensive possible population performs best?It seems that for all traits, even though there are differences in RThe authors only considered continuous outcomes for "prediction"; it would be informative to also include dichotomous/disease outcomes (related to the continuous risk factors), such as (extreme) obesity, hypertension, etc. That would also allow calculation prediction statistics such as AUC-ROC , which are easier to interpret in the context of clinical relevance.2<0.1. From experience, it seems that higher LD threshold results in better performance of the PGS. Thus, it might be worth testing higher LD threshold for better performing PGSs.The PGS was generated based on LD-based clumping, but it seems only one threshold was used; rReviewer #2:The manuscript of Mostafavi discusses a timely and important question: how accurate are polygenic risk scores (PRS) derived from GWAS studies and first validated in large, but single cohorts, when predicting risk in populations different to those used in their derivation. Recently GWAS studies have achieved sample sizes of a million subjects. In addition, the UK biobank has released rich datasets close to half a million people genotyped, extensively phenotyped, and followed so far for about 10 years. These resources have spurred the development of PRS for many traits ranging from complex disease onset, quantitative phenotypes, and even to sociological predictions. Furthermore, a number of direct to consumer companies have started to provide their customers PRS for medical traits. Therefore, the urgent question is how reliable are these scores and how well they port to other populations and people around the world.Mostafavi work starts with the recent publications suggesting that PRS port poorly to populations other than European descent (the population object of most of the data available), in particular to African populations. There have been recent calls to expand GWAS and biobank style cohorts to include people from other ancestries around the world. However, Mostafavi analyzes now show that problems with portability are not just evident in different ancestries, but also to different strata from samples from European decent participants. If this is correct, this raises the urgency to hold off implementations of these scores in to healthcare settings until we truly understand how and when these scores can be applied across groups of people.The authors provide a meticulous dissection of the situations when the PRS are not portable within people of the same Ancestry. They perform careful QC of the UK Biobank data, use reasonable methods to adjust the sample strata, perform the GWAS, and construct and test the PRS. The authors state that their main goal was to demonstrate how differences in the composition of the GWAS cohorts affect portability for different traits of interest within the same ancestry, and they achieve this. However, they do a good work exploring the possible factors involved in the reduced portability and leave us with important lessons on how our assumptions about GWAS data may be wrong at times. Their data and simulations add important information on how sex ratios, indirect effects, GxE, assortative mating and other factors confound the data and contribute to poor performance in subsets of the UK biobank subjects. The manuscript is well written and extensive details is provided on the methods and simulations. The main text is accompanied by extensive supplemental results of importance, which are referred at appropriate places in the text.I have a number of questions that I would like the authors consider and when appropriate clarify in their final manuscript:1) Throughout the manuscript the term "portability" is used and metrics that measure the performance of the PRS are used to judge whether portability is feasible or reduced. While this term has been used in prior literature w.r.t to the reduced ability of a PRS developed in cohorts of European descent people when applied to populations of different ancestry, this term is jargon. In statistics and machine learning the term that describes this poor performance is overfitting and the ability of a model, or in this case a PRS, to preform equally well in other samples, is termed generalization. Would it not be "overfitting" the term that more accurately describe the lack of "portability"?2 to measure the reduced generalization of the PRS to the different cohorts' strata, and they concede that is not clear what would be the best metric to use for this analysis. However, many publications use the AUC as a metric to decide the best model and threshold of SNPs to pick and to describe how the PRS augments other predictor bases on covariates or other factors. It is not intuitive how R2 correlates with AUC, and therefore it would be best if the authors try in at least one analysis to compare R2 to AUC.2) The author's use R3) Recent publications have used the LDPred method that uses as input summary statistics rather than genotypes to develop PRS based on UK Biobank data. Some of these have shown generalization to others cohorst. Can the author's comment whether LDpred features distinguished it from the methods used in their analysis conferring an advantage? PRS derived from LDpred also tend to include many more SNPs to achieve a greater AUC. The author's explore P-value cut-offs (and hence number of SNPS) in some analysis, but it was not clear whether this suggested more SNPs increases overfitting. Do the results from the author's suggest that including more SNPs increases overfitting?Reviewer #3:The generalisability of polygenic scores is an important issue given the possibility of them being incorporated in to healthcare soon. This paper makes excellent use of the UK Biobank resource to present two sets of interesting results relating to the predictive power of PRS. However, I'm not sure if the way in which the primary results are framed in relation to portability is quite right.The first set of results appear to highlight how GWAS are better powered in certain subgroups of the population . Years of schooling is better predicted in SES 4 by GWAS on SES 1 than by SES 4 itself – this shows hyper-portability, not limited portability as discussed throughout the article. This is also true for BMI and DBP (to a lesser extent DBP). The bivariate LD Score results showing genetic correlations close to 1 seems to support this being principally a power (rather than portability) issue, in which there are different relative contributions of genetics and the environment across the groups, making some better-powered to capture – largely the same – genetics. Based on the results , I'd have thought that this is most likely due to the environment resulting in convergence and genetics resulting in divergence (at least in the case of BMI and education) – eg. BMI converges to low values in old age, irrespective of genetic predisposition (relatively), while education levels of those in the top SES are forced to be similar due to the qualification requirements of professional jobs. Thus, the highest phenotypic variance, heritability, and thus PRS predictive power for BMI is from the lower age group and schooling from the lowest SES. How specific this pattern of results is to these particular traits is unclear, and certainly there seems to be a somewhat different cause for the DBP results. However, in general I see these results as being analogous to eg. how GWAS on Alzheimer's is perhaps being best-powered to detect common genetic variants when applied to individuals in their 70s, GWAS on blood pressure best-powered when applied to individuals in their 30s, and lung cancer GWAS being optimised when applied to non-smokers.While the downstream impact of variable GWAS power in different sub-groups on PRS prediction is extremely valuable to see here, I think the implications of these results may be missed unless discussed in the context of differential GWAS power in different sub-groups – eg. in relation to ancestry, the consensus is that portability will be optimised by expanding GWAS sampling to non-European populations, but the analogy is not true here – the consequence of these results (at least for these traits) is that portability will be optimised if GWAS sampling is restricted to those sub-groups for which heritability is highest .The second set of results expand on similar analyses by Kong et al., 2018 and Selzam et al., 2019, showing that the indirect effects of familial genetics, and potentially assortative mating, could contribute to standard PRS predictive power. Any impact on portability is not explicitly demonstrated here, but the point is made that differences in culture (family structures etc) could lead to significant differential portability given the strong influence of indirect effects on standard PRS for some traits. These are great results to see, and will add to the literature on the topic of indirect genetic effects nicely.eLife. Your revised article has been evaluated by Mark McCarthy (Senior Editor) and a Reviewing Editor.Thank you for resubmitting your work entitled "Variable prediction accuracy of polygenic scores within an ancestry group" for further consideration by The manuscript has been improved and reviewers were generally satisfied with the changes made. However, while discussing the revisions some remaining issues were noted that need to be addressed before acceptance, as outlined below:1) As sample size is an important determinant of the accuracy of the association effect sizes and thus also of the prediction accuracy, the sample sizes of each stratum needs to be clearly stated. Currently, the use of "unstratified" and "all" is misleading as it would suggest it include the data of strata combined. However, this seems not the case; the "unstratified" or "all" have now been  reduced to match the sample size of the strata, which is not intuitive. After all, when data for multiple strata is available, one would be better off combining all data, rather than using the strata. We suggest to clearly report sample sizes, and to choose a more precise terminology to describe the "unstratified" or "all" group. Including an "all" group, that combined strata, as before, would seem informative to illustrate that sample size trumps population-specificity.2 for DBP in the "mixed GWAS" is just less than half than for the "total GWAS" , while for education there's a 10-fold difference (ie. doubling GWAS N has led to 10x greater R2).. This can be the case , but it would be worth checking that there's no error here because it seems a little surprising.2) One of the reviewers noted difference between the original Figure 1 and the new Figure 1, and wondered whether the data changed (slightly) compared to their previous version because the results are slightly different . It would be worth checking that this is just updated UKB data rather than an error. Furthermore, the RReviewer #1:The authors have addressed all concerns and the revised version is more focused and conveys a clearer message.While they now present the "all" GWAS for the same sample size as for the stratified GWAS, this has not been presented in a very clear way. In the text, L179 they state they use the same sample size, but it would be good to report what this sample size is. Intuitively, one would not expect that an "all" GWAS had the same sample size as a stratified GWAS.I suggest to clarify this in the legends of the figures, and also more clearly in the text.Reviewer #2:Motafavi et al. have updated their manuscript based on reviewer's feedback and answered all the questions and comments posed during the review process of the first submission.In my opinion their rebuttal and responses are satisfactory and their new and improved manuscript reflect their responses to reviewers. The author's maintained the focus of their manuscript in demonstrating that within ancestry there are a number of possible confounders and study design considerations that impact the prediction accuracy of PGS and put in question its immediate implementation in the clinic as some propose. In addition, and without going overboard expanding hugely the manuscript to explore all possible avenues to explain these effects, their perform a number of analyses that reject the simple idea than environment variance among strata is solely responsible for these effects, but in fact more complex phenomena are at play, including indirect effects, assortative mating, etc. These studies would be the starting point for future studies by the authors and others in the field. I thank the authors to indulge us by adding several interesting analyses suggested by reviewers, such as the inclusion of results by LDpred, a method gaining traction, and showing that as more markers are included in the PGS portability may be more difficult. The comprehensiveness of the supplementary material and clarity of explanations are superb.After reading the new material in its entirety (apologies that took some time for me to go over this again), I find it a significant contribution to the field, and I do not have any other significant comments to offer recommending its acceptance for publication.Reviewer #3:I am satisfied with the revisions that the authors have made and have no further comments to make on the manuscript. Nice work!eLife. Your article has been discussed by Ruth Loos, the Reviewing Editor, and Mark McCarthy as the Senior Editor.Thank you for submitting your article "Variable prediction accuracy of polygenic scores within an ancestry group" for consideration by While you have been very responsive to the reviewers comments and concerns, we remain concerned that some of the presentation of findings remains counter-intuitive and could lead to mistaken inference. We would like you to consider one final set of revise your paper to include the following:1) Include strata-combined analyses; i.e. a "full" analysis that combines the stratum-specific sample sizes into one large population. The current study shows that the stratum-specific PRSs perform better than strata-combined PRS, but only if the strata-combined (e.g. men+women) PRS is based on GWAS of the same size as the individual strata .In the very first version of the paper, it was clear that the strata-combined PRS (based on combined stratum-specific data) performs much better than any of the stratum-specific PRS: it seems that the larger sample size overcomes any advantage of stratum-specific analyses (in these examples at least). In response to the reviewers' comments, the authors "reduced" the sample size of the strata-combined analyses to the size of that of the individual strata. However, reducing a strata-combined analyses to the same sample size as the stratum-specific sample sizes is not an intuitive comparison , as stratum-specific implies that these are a subset of the strata-combined. The current version does not address this issue: the labeling of the (sub)groups has been changed, but that has the potential to mislead the reader to infer that stratum-specific analyses will provide the best prediction, whereas a strong predictor is also sample size.2) Discuss the role of sample size in prediction in the Discussion section;i.e. put the impact of stratum-specificity in prediction in comparison to the impact of sample size in the (this relates to the comment above). It is important that readers understand that both contribute to prediction, and that possibly sample size is even more important than stratum-specificity. It would be good if the authors can speculate on any circumstances in which the strata-combined analysis would not provide better overall prediction than the stratum-specific subsets.eLife. Your revised article has been evaluated by Mark McCarthy (Senior Editor) and a Reviewing Editor.Thank you for resubmitting your work entitled "Variable prediction accuracy of polygenic scores within an ancestry group" for further consideration by The manuscript has been improved but there are some remaining issues that need to be addressed before we move to final acceptance. These relate to the discussion about the trade-off between sample size and stratum-specific effects on PRS performance that have been the subject of the recent revision requests. Thank you for adding additional information and figures that address this issue. However, as per the last round of revision requests, we do not feel that this question has been adequately addressed in the Discussion, and that some explicit text on this matter forms an essential part of the paper. As you will have seen from the to and fro over the revisions, this is an issue that has exercised several of the editors and reviewers, and I can’t imagine that it will be any less important to the readers of the paper. Put simply, the question is this: based on the data you have generated, are there any real world situations where researchers would be better off splitting a data set into its component strata (eg by gender) and proceeding on the basis of stratum specific risk scores, rather than using the whole data set? The data in the additional figure, suggest not (at least in the models and situations you tested), which is why this is important to place into context. The manuscript in its current form would not, even with minor revision, be suitable for the journal. However, we would consider an extensively revised manuscript that addresses the following key concerns of the reviewers.1) The paper will benefit from a clearer focus; while the authors aim was to simply establish the variable of portability between strata, they went quite further from this with the topics of assortative matting, sex balance, parents' effects, etc. Just achieving their main aim would make this paper very worthy as it points to issues that those already thinking in applying PRS in clinical practice right now need to urgently appreciate.We agree that establishing variable prediction accuracy--in clinical settings as well as in other areas in which polygenic scores are increasingly deployed--is the most important result in the paper. Assortative mating, indirect effects, GxE and stratification are examined only through the prism of portability. We have tried to emphasize our main focus in the revisions. In particular, to make the relevance to human diseases more explicit, we have added disease traits to our analysis, as well as considering three binary traits .2), than PRS based on any of the strata.2) In focusing on the main aim, a deeper exploration of the causes that drive the lack of portability would increases the informativeness of the study. It was pointed out that the difference in portability across strata might be due to differences in power, sample size, heritability, phenotypic variance, phenotype distribution, influence of environment, insights in how these and potentially others affect portability would be helpful. This should also include a discussion on how is it that different strata can generate even higher predictive power than using the same stratum (in base and target) and why the PRS based on the full sample size, in the provided examples, always has a better "predictive ability" .2.In addition, we have expanded the Discussion of how a different stratum can generate even higher predictive power than using the same stratum in the GWAS and prediction sets. In short, the higher prediction accuracy results from the increased power of the GWAS in the stratum with the largest hBeyond this discussion, we feel that an exploration of the precise factors driving variability in prediction accuracy for the traits we present is both beyond the scope of this paper and somewhat counterproductive to our main goal. In highlighting these particular examples, we wished to highlight the problem of generalizing polygenic scores from GWAS sets to prediction sets within a given ancestry group, rather than to explain the behavior of specific traits. In that regard, a detailed examination of the examples may draw attention away from our desired focus.2 would differ across groups primarily because of differences in the extent of environmental variance. We appreciate that this point could have been easily missed by readers as originally framed and we have therefore elaborated upon it in our revision and hopefully clarified our thinking.Having said that, we do discuss several factors that we believe are of general importance for portability across many traits, including for these examples. Notably, Figure 2A-C show that much of the trends in Figure 1 can be explained by differences in (SNP) heritabilities. In Figure 2D-F, we further show that these results cannot be explained by differences in the extent of environmental variance across strata alone. This conclusion stands in contrast to the common notion in the field  that h3) Analysis of disease outcomes, here e.g. (extreme) obesity and hypertension, will be useful to translate findings to more clinical settings, including calculation of ROC-AUC, a standard easy-to-interpret predictive statistic.We have added analyses on extreme obesity and hypertension, using ROC-AUC as a measure for prediction accuracy (Appendix—figure 13). We find that the  AUC trends for obesity, for example, quantitatively mirror those observed for the prediction of BMI as a continuous trait. We have also added five examples of disease traits with relatively high heritability to the Appendix (Appendix—figure 14); in three of these cases, we again see differences in prediction accuracy depending on the GWAS set.Reviewer #1:In this study, the authors examine the portability of polygenic scores within a homogeneous, generally healthy population of white British unrelated adults, living across the UK. They show that the PGS performance is influenced by age, sex, socio-economic status and study designs. The reduced accuracy of PGS is not (only) caused by environmental influences, the authors show that also the magnitude of the genetic effects among groups, and indirect effects of assortative mating affect the performance of the PGS.While these findings raise important consideration with reference to the use of PGSs, it would have been interesting if they had gone a little deeper with their analyses; e.g. to also identify the features  of outcomes and covariates that impact the prediction performance the most.2 throughout the paper mainly to facilitate comparison with key papers published previously in this area . However, it is not an ideal measure for breaking down these factors because it confounds biases and estimation noise. Furthermore, as noted in our response to the editor, we worry that focusing on the breakdown for the three specific examples may draw attention away from the generality of the concerns that we wish to raise, as these contributions likely vary in nature and magnitude across traits.We agree that a dissection of different factors that influence portability would be of great utility; at the same time such a breakdown is beyond our scope in this paper and might require a different approach. For instance, we use incremental RWe do however discuss several factors that are likely to be generally important for portability across traits, including these examples. Notably, Figure 2A-C show that much of the trends in Figure 1 can be explained by differences in the heritability across strata.2 are mostly due to differences in environmental variance across strata, but as we show in Figure 2D-F, a model with constant genetic variance and changing environmental variance provides a poor fit to the data. As noted in response to the editor, we now clarify this point in revisions of the main text. Given these findings, we hypothesize that there is an interaction between genetic effects and sample characteristics, such that genetic effects are systematically larger in the groups with higher prediction accuracy, even though the effect sizes are highly correlated among strata  .We had initially hypothesized that these differences in h2 for sex-specific, age-specific and SES-specific PGSs, the highest R2 (still low) is achieved when the PGS is generated in the full population. Thus, isn't the conclusion then simply that stratification lowers the PGS performance and that a (blunt) PGS, based on the most comprehensive possible population performs best?It seems that for all traits, even though there are differences in RWe apologize for the previous presentation, which led to confusion. As we detail in the response to the editor, we have modified Figure 1 to address this point (see above).The authors only considered continuous outcomes for "prediction"; it would be informative to also include dichotomous/disease outcomes (related to the continuous risk factors), such as (extreme) obesity, hypertension, etc. That would also allow calculation prediction statistics such as AUC-ROC , which are easier to interpret in the context of clinical relevance.We have now done so, both by dichotomizing continuous traits such as BMI and blood pressure and considering them as binary, and by adding five additional disease traits. Following the reviewers’ suggestions, we have used  AUC to measure prediction accuracy in these binary traits. As can be seen (Appendix—figures 13 and 14), the qualitative conclusions remain.2<0.1. From experience, it seems that higher LD threshold results in better performance of the PGS. Thus, it might be worth testing higher LD threshold for better performing PGSs.The PGS was generated based on LD-based clumping, but it seems only one threshold was used; rOur goal here was not to maximize prediction accuracy, but to show that at a given threshold, it is variable across groups of the same ancestry. Nonetheless, to evaluate the sensitivity of our analysis to the choice of the PGS model, we added an analysis repeating the three examples in Figure 1 with LDpred. We find that LDpred generally outperforms the prediction accuracy of clumping approaches. Most importantly for this paper, we find that the trends across strata remain qualitatively unchanged and are often accentuated when LDpred is used instead of clumping (Appendix—figure 2).Reviewer #2:The manuscript of Mostafavi discusses a timely and important question: how accurate are polygenic risk scores (PRS) derived from GWAS studies and first validated in large, but single cohorts, when predicting risk in populations different to those used in their derivation. Recently GWAS studies have achieved sample sizes of a million subjects. In addition, the UK biobank has released rich datasets close to half a million people genotyped, extensively phenotyped, and followed so far for about 10 years. These resources have spurred the development of PRS for many traits ranging from complex disease onset, quantitative phenotypes, and even to sociological predictions. Furthermore, a number of direct to consumer companies have started to provide their customers PRS for medical traits. Therefore, the urgent question is how reliable are these scores and how well they port to other populations and people around the world.Mostafavi work starts with the recent publications suggesting that PRS port poorly to populations other than European descent (the population object of most of the data available), in particular to African populations. There have been recent calls to expand GWAS and biobank style cohorts to include people from other ancestries around the world. However, Mostafavi analyzes now show that problems with portability are not just evident in different ancestries, but also to different strata from samples from European decent participants. If this is correct, this raises the urgency to hold off implementations of these scores in to healthcare settings until we truly understand how and when these scores can be applied across groups of people.The authors provide a meticulous dissection of the situations when the PRS are not portable within people of the same Ancestry. They perform careful QC of the UK Biobank data, use reasonable methods to adjust the sample strata, perform the GWAS, and construct and test the PRS. The authors state that their main goal was to demonstrate how differences in the composition of the GWAS cohorts affect portability for different traits of interest within the same ancestry, and they achieve this. However, they do a good work exploring the possible factors involved in the reduced portability and leave us with important lessons on how our assumptions about GWAS data may be wrong at times. Their data and simulations add important information on how sex ratios, indirect effects, GxE, assortative mating and other factors confound the data and contribute to poor performance in subsets of the UK biobank subjects. The manuscript is well written and extensive details is provided on the methods and simulations. The main text is accompanied by extensive supplemental results of importance, which are referred at appropriate places in the text.I have a number of questions that I would like the authors consider and when appropriate clarify in their final manuscript:1) Throughout the manuscript the term "portability" is used and metrics that measure the performance of the PRS are used to judge whether portability is feasible or reduced. While this term has been used in prior literature w.r.t to the reduced ability of a PRS developed in cohorts of European descent people when applied to populations of different ancestry, this term is jargon. In statistics and machine learning the term that describes this poor performance is overfitting and the ability of a model, or in this case a PRS, to preform equally well in other samples, is termed generalization. Would it not be "overfitting" the term that more accurately describe the lack of "portability"?We agree that the terminology is important here, but precisely because the problem of portability has to date largely been reduced to a question of population genetics alone , we think it is a good idea to use the same terminology in highlighting the numerous other factors that may affect prediction in samples that differ from the GWAS sample. As we discuss in the text, these factors  differ among continental groups or different genetic ancestry groups in the same country. Therefore, our within-ancestry group analyses pertain to cross-ancestry portability as well.2 to measure the reduced generalization of the PRS to the different cohorts' strata, and they concede that is not clear what would be the best metric to use for this analysis. However, many publications use the AUC as a metric to decide the best model and threshold of SNPs to pick and to describe how the PRS augments other predictor bases on covariates or other factors. It is not intuitive how R2 correlates with AUC, and therefore it would be best if the authors try in at least one analysis to compare R2 to AUC.2) The author's use R2 not because we believe it is in any sense an optimal measure, but to allow for more ready comparisons to previous work. Following the reviewer’s suggestion, we have now added analyses with AUC instead of R2 (Appendix—figures 13 and 14) for a total of eight traits, including five disease traits. The qualitative conclusions are the same; a more in depth examination of how AUC relates to R2 is, in our opinion, beyond the scope of this paper.As the reviewer notes, we use R3) Recent publications have used the LDPred method that uses as input summary statistics rather than genotypes to develop PRS based on UK Biobank data. Some of these have shown generalization to others cohorst. Can the author's comment whether LDpred features distinguished it from the methods used in their analysis conferring an advantage? PRS derived from LDpred also tend to include many more SNPs to achieve a greater AUC. The author's explore P-value cut-offs (and hence number of SNPS) in some analysis, but it was not clear whether this suggested more SNPs increases overfitting. Do the results from the author's suggest that including more SNPs increases overfitting?As noted in our response to the reviewer #1, we added an analysis repeating the three examples in Figure 1 with LDpred. We find that LDpred  generally outperforms the prediction accuracy of clumping methods. Most importantly for this paper, we find that the trends across strata remain qualitatively unchanged and are often accentuated when LDpred is used instead of clumping (Appendix—figure 2).Reviewer #3:The generalisability of polygenic scores is an important issue given the possibility of them being incorporated in to healthcare soon. This paper makes excellent use of the UK Biobank resource to present two sets of interesting results relating to the predictive power of PRS. However, I'm not sure if the way in which the primary results are framed in relation to portability is quite right.The first set of results appear to highlight how GWAS are better powered in certain subgroups of the population . Years of schooling is better predicted in SES 4 by GWAS on SES 1 than by SES 4 itself – this shows hyper-portability, not limited portability as discussed throughout the article. This is also true for BMI and DBP (to a lesser extent DBP). The bivariate LD Score results showing genetic correlations close to 1 seems to support this being principally a power (rather than portability) issue, in which there are different relative contributions of genetics and the environment across the groups, making some better-powered to capture – largely the same – genetics. Based on the results , I'd have thought that this is most likely due to the environment resulting in convergence and genetics resulting in divergence (at least in the case of BMI and education) – eg. BMI converges to low values in old age, irrespective of genetic predisposition (relatively), while education levels of those in the top SES are forced to be similar due to the qualification requirements of professional jobs. Thus, the highest phenotypic variance, heritability, and thus PRS predictive power for BMI is from the lower age group and schooling from the lowest SES. How specific this pattern of results is to these particular traits is unclear, and certainly there seems to be a somewhat different cause for the DBP results. However, in general I see these results as being analogous to eg. how GWAS on Alzheimer's is perhaps being best-powered to detect common genetic variants when applied to individuals in their 70s, GWAS on blood pressure best-powered when applied to individuals in their 30s, and lung cancer GWAS being optimised when applied to non-smokers.We agree that a central factor driving the trends of Figure 1 is likely the differential power (and the precision of the estimates) across GWAS strata. We now further clarify this point in the text. We note further that for the three examples in Figure 1 and Figure 2, the data are poorly explained by assuming that the environmental variance differs across strata but the genetic variance remains the same . We therefore hypothesize that genetic variance is also substantially variable across strata; in other words, that there is an interaction between the genetics and the strata characteristics .While the downstream impact of variable GWAS power in different sub-groups on PRS prediction is extremely valuable to see here, I think the implications of these results may be missed unless discussed in the context of differential GWAS power in different sub-groups – eg. in relation to ancestry, the consensus is that portability will be optimised by expanding GWAS sampling to non-European populations, but the analogy is not true here – the consequence of these results (at least for these traits) is that portability will be optimised if GWAS sampling is restricted to those sub-groups for which heritability is highest .2 estimates.The reviewer brings up a very interesting point. We have expanded the Discussion of how a different stratum can generate higher predictive power than using the same stratum in the GWAS and prediction sets. As the reviewer notes, the higher prediction accuracy is largely a result of the increased power of the GWAS in the stratum with the largest (SNP) hcan matter, and matter just as much as factors like ancestry. But these are just some characteristics about which we had prior knowledge, and it seems likely that there exist many more, unknown characteristics that affect heritability for these traits--and that for some other traits, it will be hard to guess a priori what may matter. We now show an additional example in the Appendix: when we focus on a trait of interest in the social sciences  and follow the study design used in most studies, the prediction accuracy of educational attainment is somewhat higher for individuals with a (single) sibling than individuals without a sibling, when the GWAS is done in individuals with one sibling (see Appendix—figure 12).The implications for how best to conduct a GWAS are not obvious to us, for a number of reasons: one, the three examples we present were “cherry-picked” in the sense that we had prior information about what sample characteristics may matter for the trait . We used them to show that such characteristics Second, as we discuss, factors such as GxE, assortative mating and indirect effects are soaked up into the GWAS estimates--and critically also into the SNP heritability estimates. Thus, the choice of a GWAS sample is about more than detection power; it is implicitly making a choice about all sorts of sample characteristics than may or may not hold true of the prediction set.Given these considerations, it is not obvious to us how to choose an optimal GWAS sample a priori. Notably, as mentioned by the reviewer, the most “diverse” set (with respect to the sample characteristic) is not necessarily the optimal set in which to conduct the GWAS: in the revised version of Figure 1, we now show that after matching the sample size, the performance of a GWAS done on an unstratified sample  is intermediate between the performances of GWAS on stratified samples .The manuscript has been improved and reviewers were generally satisfied with the changes made. However, while discussing the revisions some remaining issues were noted that need to be addressed before acceptance, as outlined below:1) As sample size is an important determinant of the accuracy of the association effect sizes and thus also of the prediction accuracy, the sample sizes of each stratum needs to be clearly stated. Currently, the use of "unstratified" and "all" is misleading as it would suggest it include the data of strata combined. However, this seems not the case; the "unstratified" or "all" have now been  reduced to match the sample size of the strata, which is not intuitive. After all, when data for multiple strata is available, one would be better off combining all data, rather than using the strata. We suggest to clearly report sample sizes, and to choose a more precise terminology to describe the "unstratified" or "all" group. Including an "all" group, that combined strata, as before, would seem informative to illustrate that sample size trumps population-specificity.We agree that the term “all” is unclear, and thank the editors and reviewer #1 for their suggestion. We now use the term “diverse” to refer to the sample that consists of individuals from all strata. We also report the sample sizes used for GWAS in Figure 1 in the main text and the figure caption, in addition to the Materials and methods.2 for DBP in the "mixed GWAS" is just less than half than for the "total GWAS" , while for education there's a 10-fold difference (ie. doubling GWAS N has led to 10x greater R2).. This can be the case , but it would be worth checking that there's no error here because it seems a little surprising.2) One of the reviewers noted difference between the original Figure 1 and the new figure 1, and wondered whether the data changed (slightly) compared to their previous version because the results are slightly different . It would be worth checking that this is just updated UKB data rather than an error. Furthermore, the R2 between our initial submission (without downsampling) and our resubmission (with downsampling) is more pronounced for BMI and years of schooling compared to DBP.For our analysis of DBP by sex we downsampled the “diverse” GWAS by a factor of ~2 (given two sex strata), whereas for BMI by age and years of schooling by SES we downsampled by a factor of ~4 (given four BMI and SES strata). As a result, the drop in RIn addition, since our initial submission we removed 60 UKB participants that withdrew from the UKB, and repeated our analyses. Therefore, slight differences are expected due to random sampling.While you have been very responsive to the reviewers comments and concerns, we remain concerned that some of the presentation of findings remains counter-intuitive and could lead to mistaken inference. We would like you to consider one final set of revise your paper to include the following:1) Include strata-combined analyses; i.e. a "full" analysis that combines the stratum-specific sample sizes into one large population. The current study shows that the stratum-specific PRSs perform better than strata-combined PRS, but only if the strata-combined (e.g. men+women) PRS is based on GWAS of the same size as the individual strata .In the very first version of the paper, it was clear that the strata-combined PRS (based on combined stratum-specific data) performs much better than any of the stratum-specific PRS: it seems that the larger sample size overcomes any advantage of stratum-specific analyses (in these examples at least). In response to the reviewers' comments, the authors "reduced" the sample size of the strata-combined analyses to the size of that of the individual strata. However, reducing a strata-combined analyses to the same sample size as the stratum-specific sample sizes is not an intuitive comparison , as stratum-specific implies that these are a subset of the strata-combined. The current version does not address this issue: the labeling of the (sub)groups has been changed, but that has the potential to mislead the reader to infer that stratum-specific analyses will provide the best prediction, whereas a strong predictor is also sample size.2) Discuss the role of sample size in prediction in the Discussion section;i.e. put the impact of stratum-specificity in prediction in comparison to the impact of sample size in the (this relates to the comment above). It is important that readers understand that both contribute to prediction, and that possibly sample size is even more important than stratum-specificity. It would be good if the authors can speculate on any circumstances in which the strata-combined analysis would not provide better overall prediction than the stratum-specific subsets.We thank the editors for their consideration and input. We have added a Figure (Appendix—figure 1) to reflect the point raised by the editors. We have also added to the main text explicit comments on the higher prediction accuracy for the much-larger sample size of a GWAS combining all strata .Having said that, we agree with previous reviewers’ comments (and revised our manuscript accordingly) on the potential for misinterpretation in including a GWAS with a larger sample size in a main-text figure. Further, in matching sample sizes we mimic the methodology in previous research on polygenic score portability, to highlight the factors other than allele frequencies and linkage disequilibrium that affect polygenic prediction. Not matching would confound the effect of these factors with the effect of sample size.We agree with the editors that the interplay of prediction accuracy with sample size is very important, but see it as orthogonal to the points discussed in the paper. In fact, our analyses explicitly control for sample size effect throughout. We therefore suspect that adding such a discussion might only act to confuse the reader.The manuscript has been improved but there are some remaining issues that need to be addressed before we move to final acceptance. These relate to the discussion about the trade-off between sample size and stratum-specific effects on PRS performance that have been the subject of the recent revision requests. Thank you for adding additional information and figures that address this issue. However, as per the last round of revision requests, we do not feel that this question has been adequately addressed in the Discussion, and that some explicit text on this matter forms an essential part of the paper. As you will have seen from the to and fro over the revisions, this is an issue that has exercised several of the editors and reviewers, and I can’t imagine that it will be any less important to the readers of the paper. Put simply, the question is this: based on the data you have generated, are there any real world situations where researchers would be better off splitting a data set into its component strata (eg by gender) and proceeding on the basis of stratum specific risk scores, rather than using the whole data set? The data in the additional figure, suggest not (at least in the models and situations you tested), which is why this is important to place into context.We have added a paragraph to the Discussion section discussing the implications of our findings for PGS construction, notably on the choice between GWAS samples limited to one stratum versus the union of all strata, and removed the previous discussion of this point from the Results section.


Following publication of our two articles , 2, a crComplex phenotypes present several challenges for genome-wide association studies including the presence of epistasis, pleiotropy, and heterogeneity. We approached these problems in a data-driven fashion to test the hypothesis that the heritability expected from twin studies but unexplained by genetic studies is distributed in heterogeneous partitions of a complex trait, each with distinct genotypic-phenotypic associations. We designed a machine learning algorithm termed PGMRA , 3, 4 toBoth approaches, PRS and PGMRA, rely on genome-wide markers Fig. . HoweverPRS algorithms must reduce phenotypes to a single dependent variable because they use a linear supervised model . In contdiscovery sample and testing it in a new sample assuming sample homogeneity [The validity of the replication procedure used by PGMRA was questioned too . The “googeneity , 10. Homogeneity , 14, 15.Inconsistent results obtained from applying PRS to heterogeneous samples , 17 has R2 from a global linear regression, which additively estimates variance explained by the markers. In the absence of a validated estimator of variance for “sets” of markers [R2 of 10 k-fold is 45.05% with SD 0.049) and confirmed the observed results by bootstrapping . We also found that the estimates of heritability for character in our paper [R2 > 15%) than the 45.67% described above . Complex traits are expected and known to be influenced by multiple genes acting in concert, not independently . Most of


Actinia equina. By exploiting the unusual reproductive system of sea anemones, combined with new molecular data, we investigate the role of both additive (DGE + IGE) and non-additive (DGE × IGE) genetic effects on fighting parameters, the latter of which have been hypothesized but never tested for explicitly. We find evidence for heritable variation in fighting ability and that fight duration increases with relatedness. Fighting success is influenced additively by DGEs and IGEs but we found no evidence for non-additive IGEs. These results indicate that variation in fighting behavior is driven by additive indirect genetic effects (DGE + IGE), and support a core assumption of contest theory that strategies are fixed by DGEs.Theoretical models of animal contests such as the Hawk-Dove game predict that variation in fighting behavior will persist due to mixed evolutionarily stable strategies (ESS) under certain conditions. However, the genetic basis for this variation is poorly understood and a mixed ESS for fighting can be interpreted in more than one way. Specifically, we do not know whether variation in aggression within a population arises from among-individual differences in fixed strategy , or from within-individual variation in strategy across contests. Furthermore, as suggested by developments of the original Hawk-Dove model, within-individual variation in strategy may be dependent on the phenotype and thus genotype of the opponent (indirect genetic effects—IGEs). Here we test for the effect of DGEs and IGEs during fights in the beadlet sea anemone  We show that for beadlet sea anemones, the likelihood of winning a fight depends on both the genotype of the individual and the genotype of its opponent. We find heritable variation in fighting success and evidence that fights last longer between more related opponents. Our results indicate that an individual’s ability to win will vary from fight to fight depending on the genetic identity of its opponent. In many species, the acquisition of resources necessary for survival and reproduction  is reliant upon engaging in and winning agonistic contests. Thus any trait that increases an individual’s ability to win a contest  will enhance its fitness and be subject to positive directional selection. Under such selection, we might naively expect to see among-individual variation in these traits diminish; however, studies of agonistic behavioral traits have found evidence for high levels of among- and within-individual variation e.g., . This raGame-theoretic models have long been used to explore ways in which the balance between the fitness costs and benefits of fighting can lead to maintenance of variation in aggression. The Hawk-Dove game, for example, demonstrates how negative-frequency dependent selection can lead to a stable mix of strategies showing different levels of aggression . Under tAlthough both additive and non-additive IGEs are implicit in classic game-theoretic models of contest behavior, the extent to which they actually do influence fighting behavior has received little empirical attention to date. Specifically, although we are beginning to understand the importance of additive IGEs in maintaining variation in aggression , the rolActinia equina has become a model species for studying fighting behavior . These findings show that A. equina broods can be comprised either 1) entirely of clonemates, 2) entirely of genotypically unique individuals or 3) a mix of clonemates and genotypically unique individuals. Moreover, genotypes can differ within broods but be shared between broods, meaning that genetically identical individuals can experience entirely different developmental environments . This could have important implications for the expression of traits related to fighting. Specifically, if RHP traits are influenced by properties of the developmental environment , genetically identical individuals may express different agonistic phenotypes to a standardized opponent when reared in different broods. Alternatively, if RHP is influenced predominantly by additive or non-additive genetic effects, there may be a high degree of within-brood variation in fighting ability based on genotype.Many species of sea anemone, for example, are capable of reproducing asexually through somatic embryogenesis, by which a genetically identical individual develops from a single cell and is brooded internally in the coelenteron (gastric cavity) of the parent. The beadlet sea anemone behavior , 2018b aActinia equina thus provides an ideal model for studying both genotypic and developmental predictors of phenotype, specifically with respect to contest behavior. While previous work has shown that the likelihood of escalation in these fights is significantly influenced by the genetic relatedness between the fighting pair  were collected from Portwrinkle  between December 2015 and June 2016 and taken back to the lab within 1–2 h of collection. As in previous studies, only anemones of the red/brown color morph were collected as this morph has previously been shown to exhibit higher levels of aggression than anemones of the green/orange morphs found lower down on the shore . Pots containing brood mates were then placed together in a larger tank (23 × 16 × 17.5 cm) and maintained as described above. Juveniles from different broods were kept in separate tanks.One week prior to the experiment, juveniles were removed from their brood tanks and isolated in 120 mL pots covered with mesh allowing individual anemones to be identified (Actinia (<1 cm in diameter) do not possess acrorhagi . Individuals within pairs were randomly assigned as either the focal or opponent individual and placed such that their body columns were touching. Contests were recorded from this initial contact until either 1) one anemone (the loser) moved a minimum of a pedal disc’s diameter  away from its opponent or 2) retracted all of its tentacles for at least 10 min. . If bothcrorhagi , all figcrorhagi , and thuAfter the fight, juveniles were returned to their individual pots and allowed to recover for 7 days before being introduced to a new opponent and fought again. Each juvenile fought an average of 3.8 times as either the focal or opponent (randomly assigned in each fight), resulting in a total of 86 fights between juveniles from 11 broods.At the end of the experiment, juvenile anemones were placed in 1.5 mL microcentrifuge tubes containing 100% molecular grade ethanol and frozen at −20 °C for genetic sampling.In order to ascertain clonal identity and estimate relatedness between fighting pairs, individuals were genotyped using 8 polymorphic microsatellite markers developed by Ecogenics GmbH  as described in Lane et al. (under review).PCR amplifications were performed in 8 μL reactions containing 182 2 μM of each primer, 5 U of HotStarTaq DNA polymerase , 1 μL 10X PCR buffer , 2 mM of dNTPs  and 2.9 μL nuclease-free H2O. A total of 2 μL of 0.5 ng/μL DNA was added to each reaction. Thermocycling was performed on an Alpha Cycler 1 PCRmax thermocycler . Cycling conditions consisted of an initial denaturation step of 95°C for 15 min, followed by 35 cycles of 85 °C for 30 s, 56 °C for 45 s, and 72 °C for 45 s, followed by a final step of 72 °C for 30 min. Eight PCR reactions were carried out per sample (one per primer pair).PCR products were analyzed by Ecogenics GmbH  using an ABI3730 (Applied Biosystems) DNA analyzer with an internal size standard  for accurate sizing. Electropherograms were then visualized using Peak Scanner Software v1.0 (Applied Biosystems) and alleles scored based on amplicon size. The microsatellite sequences used in this paper have been deposited in GenBank  or did not (0)), fight outcome scored from the perspective of the designated focal individual , and fight duration . Fight duration data was only available for fights which ended in a clear outcome . For outcome and duration we standardized traits to standard deviation units and modeled the data with an assumption of Gaussian residuals. Although they cannot truly be Gaussian, visual inspection of residuals from our model of outcome suggests that this approach adequately characterizes the structure of the data. For occurrence, we similarly modeled the observed  data with the same assumption of Gaussian residuals and conducted provisional statistical inference on random effects as described below. While this has the advantage of yielding parameter estimates that are readily interpretable on the observed data scale, the Gaussian assumption is obviously very strongly violated here. We therefore also fitted a parallel series of generalized models for this trait using a logit link and parameter estimation by penalized quasi‐likelihood (PQL) approximation in ASReml  effect of any individual i on the observation cannot depend on i’s designated role. From this, it is necessarily true that Vfocal = Vopponent and the within-individual correlation between focal and opponent effects is +1. The situation is identical for outcome in the sense that only one phenotype can be observed per dyad, but the observation recorded now depends on designation. If individual i wins a context with j, then the outcome is 1 if i is the designated focal, but −1 if j is the focal. This means that Vfocal = Vopponent as before, but the within-individual correlation between focal and opponent effects is necessarily −1  and “permanent environment”  contributions. Random clone effects of focal and opponent were constrained exactly as described for individual effects in Model 1, as were the permanent environmental effects. In Model 3, we added an additional random effect of clone dyad, a factor defined by the interaction of focal clone and opponent clone identities. At least in principle, variance caused by G × G interactions could exist in the absence of additive genetic variance. This scenario would arise in, for instance, a genetically determined paper-rock-scissors game scenario, in which no among-type differences in expected (mean) outcome are predicted , but all observed variance in contest outcome could be explained by the type combination. Thus, we also fitted one further model (Model 4) that included the G × G interaction and additive effects of individual identity (but not the additive clone identity effects).P) under each model and their significance tested by likelihood ratio tests (LRTs). Specifically, LRTs were used to compare 1) Model 1 against a null model (with no random effects) to test the significance of repeatable  variance, 2) Model 2 against Model 1 to test for additive genetic variance in contest behavior and outcome , and 3) Model 3 against Model 2 and Model 4 against Model 1 to test for non-additive (G × G) effects. In each of these comparisons, the more complex model includes a single extra variance component, and we, therefore, conduct LRTs assuming that twice the difference in model log-likelihoods is distributed as a 50:50 mix of Estimated variance components were calculated as a ratio to total phenotypic variance  on outcome; and both size asymmetry  and microsatellite-based relatedness (mean-centered) on occurrence and duration. Statistical inference on these fixed effects was by conditional F tests implemented in ASReml.Finally, for each trait we refitted the model with the preferred random effect structure (see Results) to test hypothesized fixed effects on mean contest behavior and/or outcome. Specifically, we tested fixed effects of Although our primary focus is on testing for IGEs that might arise from opponent genotype (clone identity) within dyadic contests, in principle effects analogous to IGEs could also arise from the “parental environment” provided by the brooding adults. To check this possibility, we fitted a set of similar models to those described above but using brood identity in place genotype in all random effect structures. Note that genotype is not fully confounded with brood identity here as not all juveniles from a single brood are clonemates . There were no significant effects of brood identity on any of the response variables and we do not discuss this possibility further , four were unique to individual broods and the remaining four were shared between broods. The genotypic diversity of the population was 0.37.occurrence. In fact, corresponding variance estimates were bound at zero with variances constrained to be in allowable  parameter space (size asymmetry (coefficient = −0.049 (0.079), F1,128 = 0.32, P = 0.528) or relatedness (coefficient = 0.603 (0.787), F1,116.6 = 0.59, P = 0.445) affects whether or not a fight occurs.Using the models with Gaussian residuals, we found no evidence at all to support additive effects of individual or clonal identity on the binary variable of fight er space . Nor waser space . AIC scoG×G / (VG×G + VR + π 2/3) where π 2/3 is the variance on the underlying logit scale). Although formal inference of the G × G effect is slightly problematic using PQL  the ratio of estimated variance to SE provides a useful guide , F1,128 = 0.31, P = 0.573; relatedness coefficient = 2.725 (3.640), F1,114 = 0.56, P = 0.455).While stressing that statistical inference in these models is compromised by the violation of assumed Gaussian residuals, generalized models yielded similar “null” conclusions. Using a logit link, variances for individual, additive, and permanent environment effects were all bound to zero in all models where included. Under Models 3 and 4, the G × G variance (SE) was estimated as 0.419 (0.552) which yields an estimated clone pair “repeatability” of 0.091 on the link scale .By comparing the log-likelihood value for Model 1 against that of the null model (intercept only), we found evidence of significant among-individual variation in contest outcome, indicating that there are consistent, repeatable differences among individuals in how likely they are to win a fight  and in their effects on the fighting success of others (as the opponent). Under Model 1, additive effects of focal and opponent individual identities explained a combined 40.6% (SE = 15.1%) of the variance . Including additive genetic  effects significantly improved the model  of variance in (log-transformed) contest duration. This among-individual variance was not statistically significant  Thus fights between more evenly sized individuals tend to be of longer duration. There was, however, a significant positive effect of relatedness on fight duration (coefficient = 5.676 (2.259), F1,57 = 6.31, P = 0.015). Thus, as the relatedness of opponents increased, so did the duration of the contest ,  contest .A. equina. Our analysis showed that among-individual variation in fighting success was largely explained by additive DGEs and IGEs (G + G). Thus, the data support an underlying assumption implicit in models such as the hawk-dove game that selection acts on strategies fixed by DGE. In contrast, we found only limited evidence for non-additive G × G interactions (on contest duration) and no evidence of a genetic influence at all on the likelihood of fights occurring. Fight duration did not appear to be explained by direct or indirect genetic effects, and so is not a heritable trait per se. Nonetheless, there was a significant positive relationship between contest duration and relatedness, with fights continuing for longer as the relatedness between two opponents increased. Thus the expressed phenotype is actually contingent on the particular combination of genotypes involved in the dyad, but not because of heritable variation in RHP (which would manifest as variance attributable to DGE and IGE).Agonistic contests involve two or more interacting individuals, and thus, the aggressive phenotypes expressed during contests may be dependent not only on an individual’s genotype (DGEs) but also on the genotype of its opponent (IGEs). In this study, we investigated the role of IGEs in determining the contest behavior of juvenile beadlet anemones A. equina, and fighting success is in part determined by DGEs, fighting ability (RHP) is likely to vary within broods. Furthermore, as we found no effect of developmental environment (brood ID) on fighting success, this variation in RHP should persist regardless of the identity (and perhaps genotype) of the brooding adult. This is particularly interesting for broods which contain a mix of clonemates and unique individuals . Actinia equina fight over the acquisition of territory on the intertidal zone, requiring a sheltered spot to avoid desiccation, and thus possible death, when the tide goes out. Depending on the genotype of the brooding adult and on the genotypes of fostered juveniles, it is possible that the adult’s own genetically identical progeny may be outcompeted by their foreign brood mates. This possibility could be detrimental to the brooding adult’s fitness and thus raises the question as to why adults tolerate foreign juveniles within their coelenteron. One explanation is that adults are unable to distinguish between their own progeny and unrelated young. Although it is generally accepted that adult anemones are able to identify self from non-self  combination of its own genotype, and its opponent’s genotype affecting RHP. Although our results indicate that fighting behavior does not differ significantly between broods, because juvenile genotype varies within broods in non-self , very linon-self , especiaA. equina. This finding also reflects the persistence of linear dominance hierarchies in natural populations. If non-additive genetic effects (G × Gs) were of widespread importance in determining fighting behavior we would expect to see the formation of genetically determined intransitive dominance hierarchies in natural populations , but comparative studies have shown transitive dominance hierarchies to be prevalent and intransitive hierarchies to be rare  for any of the fight parameters measured. As discussed above, detection of G × G interactions requires adequate replication at the level of the genotype and the level of each genotypic combination. As we were unable to genotype individuals prior to engaging in fights , we could not ascertain what the number of replicates would be at each level until after the experiment (an average of 8.29 fights per genotype and 1.75 per genotypic combination). However, if the lack of evidence for G × G were due to power limitations, we might expect the variance explained by G × G to still be substantial, albeit nonsignificant. However, the variance components relating to G × G in our analysis were either bound or very close to zero in all models. Thus, even if our replicate size resulted in power limitations, G × Gs are unlikely to be of biological importance for  be rare . Thus thIn contrast to the results on fighting success, we found no among-individual variation for contest occurrence or duration  nor any significant influence of direct or indirect genetic effects on these parameters. Although selection is expected to favor any trait that increases the likelihood of victory, fighting success is not determined by persistence alone but rather by a functionally diverse suite of traits . FurtherA. equina in which fights were found to escalate more often as the relatedness between opponents increased , although the latter is dependent on a genotype-by-genotype interaction in the sense that it increases with pairwise relatedness. Our results thus demonstrate that persistence and success in sea anemone contests are determined by the genotypes of both opponents and that contest success, but not persistence demonstrates heritable variation. Taken together, these effects on contest outcomes validate underlying assumptions implicit in contest theory (ESS solutions depend on the presence of strategies fixed by DGEs) and explanations for the widely observed presence of linear and transitive dominance hierarchies  in animal societies.arz217_suppl_Supplement_MaterialClick here for additional data file.