Background: Many studies have demonstrated the utility of machine learning (ML) methods for genomic prediction (GP) of various plant traits, but a clear rationale for choosing ML over conventionally used, often simpler parametric methods, is still lacking. Predictive performance of GP models might depend on a plethora of factors including sample size, number of markers, population structure and genetic architecture.Methods: Here, we investigate which problem and dataset characteristics are related to good performance of ML methods for genomic prediction. We compare the predictive performance of two frequently used ensemble ML methods (Random Forest and Extreme Gradient Boosting) with parametric methods including genomic best linear unbiased prediction (GBLUP), reproducing kernel Hilbert space regression (RKHS), BayesA and BayesB. To explore problem characteristics, we use simulated and real plant traits under different genetic complexity levels determined by the number of Quantitative Trait Loci (QTLs), heritability (h2 andh2e), population structure and linkage disequilibrium between causal nucleotides and other SNPs.Results: Decision tree based ensemble ML methods are a better choice for nonlinear phenotypes and are comparable to Bayesian methods for linear phenotypes in the case of large effect Quantitative Trait Nucleotides (QTNs). Furthermore, we find that ML methods are susceptible to confounding due to population structure but less sensitive to low linkage disequilibrium than linear parametric methods.Conclusions: Overall, this provides insights into the role of ML in GP as well as guidelines for practitioners. ANN: artificial neural networkBLUPs: Best Linear Unbiased PredictionsGBLUP: Genomic Best Linear Unbiased PredictionGP: Genomic PredictionMLP: Multilayer PerceptronQTL: Quantitative Trait LociQTN: Quantitative Trait NucleotideRF: Random ForestRKHS: Reproducing Kernel Hilbert SpacingSNP: single nucleotide polymorphismSVM: Support Vector MachineSVR: Support Vector RegressionXGBoost: Extreme Gradient Boosting A GP model is constructed on a reference population for which both genotypes and corresponding phenotypes are known, mostly employing a cross-validation strategy, and applied to related populations with only genotypes known. The total genomic value, estimated from the GP model, is used as a pseudo-phenotype to select the best parents for the next generation(s). In general, phenotypes differ from each other in terms of their genetic complexity, ranging from simple/monogenic to complex/polygenic. These differences impact the potential performance of GP. Complex traits are predominantly governed by a combination of additive and non-additive  allele effects, which makes GP challenging for these traits. The genetic architecture of complex traits is characterized by moderate to large numbers of Quantitative Trait Loci (QTLs) with small to medium effect sizes and no or few large effect QTLs. Moreover, the ratio of additive to non-additive genetic variance may differ even for closely related traits. Besides the actual genetic variance level, its distribution over the genome is also a determinant of the trait architecture. Next to genetic architecture, population structure plays a role as well  model, using a genome-wide set of markers, is an integral component of the genomic selection-based approach.as well : predict but may induce multicollinearity among SNPs. Consequently, SNP selection prior to predictive modelling has been reported to provide superior performance compared to simply using a dense marker set. In contrast, low-density genotyping can miss important SNPs in LD with, or weakly linked to, the QTLs, leading to inferior prediction performance.Technological advances and statistical frameworks used bring new challenges . Genotyp This approach also requires proper translation of prior knowledge on the genetics underlying phenotypes into parametric distributions. Although statistical distributions can help describe genetic architecture, devising a specific distribution for each phenotype is impractical. Therefore, many variations of linear regression were proposed by relaxing statistical assumptions; the main differences lie in their estimation framework and prior assumptions on the random effects . Alternatively, machine learning (ML) offers a more general set of non-parametric methods that can model phenotypes as (non) linear combinations of genotypes. Moreover, these methods can jointly model the problem, e.g. strong learners can be stacked or weak learners can be combined in an ensemble. Examples include Support Vector Machines (SVMs), (ensembles of) decision trees and artificial neural networks (ANNs). No statistical assumptions are required in advance; therefore, these methods should be able to pick up more complex genetic signals that are missed by linear models. The downside is the large amount of data required for learning these models from the data.Statistical genetics approaches have traditionally focused on formulating phenotype prediction as a parametric regression of one or more phenotypes on genomic markers, treating non-genetic effects as fixed or random in a linear equation. The resulting GP models are biologically interpretable but might yield poor performance for complex phenotypes, as linear regression fails to capture the more complex relations.,; however, a clear link between simulated and real phenotypes is often missing, or only a specific breeding population structure is considered. For example, Barbosaet al. compared the performance of ML and statistical methods in a simulated F2 population of 1,000 individuals and 2,010 SNPs using 26 simulated phenotypes. They varied the heritability and number of QTLs and included dominant and epistatic effects. They observed that ML methods performed better at low QTL numbers and hypothesized that a reason for this is that with fewer controlling genes, epistatic interactions are more important. But it is still unclear if this is a general conclusion towards a population with different characteristics e.g. natural populations. Moreover, there are conflicting reports on performance of ML., For example, ANNs have been reported to perform worse in some applications and are comparable to competing methods in others., Ensemble decision tree methods, combining the output of a large number of simple predictors, have proven better for some traits but not for others.– Gradient boosting showed improved performances for many real traits,https://paperpile.com/c/ZyQHHy/b9hH+ha6M but was inferior to random forests on simulated datasets. Furthermore, the impact of population structure and low SNP-QTL LD on the performance of ML methods is still unclear.The performance of ML methods in GP problems has previously been compared using simulated and real phenotypes. Some were found to perform better under non-additive allelic activityp/n), genetic complexity, QTN effect sizes and distributions, additive vs. epistatic heritabilities, sparse vs. dense genotyping and population structure.In this paper, we investigate which GP characteristics  a priori point to a better performance for either traditional statistical approaches or ML-based methods. We compare GP performance of two ensemble methods, Random Forests (RF) and Extreme Gradient Boosting (XGBoost), to that of linear mixed models, GBLUP, BayesA, BayesB and RKHS regression with averaged multi-Gaussian kernels. We focus on typical applications in plant breeding to explore various GP characteristics, including the ratio of the total number of markers to the number of samples (Simulationsc =p/n), were simulated. For the high dimensionality scenarios, sample size was fixed atn = 500, because reference populations of this size are feasible for genotyping and phenotyping in genomic selection studies. Using values ofc = {2, 10, 20, 40, 120}, the number of SNPs varied up top = 60,000 (120*500). Similarly, for the low dimensionality scenarios, the number of SNPs was fixed atp = 500 and sample size was varied up ton = 3,000 to arrive atc = {1, 1/2, 1/4, 1/6}. Subsequently, Quantitative Trait Nucleotides (QTNs) were randomly selected from these simulated SNP sets to generate phenotypes. We selected either 5, 50, 100,p/2 orp QTNs, corresponding to a range of low to high genetic complexity, coupled with a narrow-sense heritability ranging from 0.1 to 0.7. A phenotype with a high number of QTNs and low heritability is more complex than one with few QTNs and higher heritability.In a first experiment, artificial genotypes were simulated, in combination with associated phenotype values. Genotype data was simulated for a diploid population with a minor allele frequency (MAF) of 0.4, using a binomial distribution, where each allele was the outcome of a binomial trial. The genotype dataset was coded as {0=AA, 1=Aa, 2=aa}. We fixed MAF for all SNPs, in order not to incorporate the impact of allele frequencies because MAF of a QTL can impact its heritability estimation and ultimately prediction accuracy of the GP model. Moreover, in this way we observed equal and reasonably statistical power for each SNP during allele effects estimation. To explore GP characteristics , differe Additive polygenic phenotypes were simulated using additive modes of allele effects, as follows:Phenotype datasets were generated using the simplePHENOTYPES v1.3.0 (RRID:SCR_022523) R package.βi describes the effect size of thethi QTN, whereQTNi is a vector containing the allele dosages for thethi QTN for all samples. The residuals (ε) were sampled from a normal distributionN). Three different approaches were used to sample the effect sizes: (i) The narrow-sense heritability (h2) determines the variance of effect sizes distribution: one effectβ is randomly sampled fromN and equally divided among all of theq QTNs, such that each QTN is assigned an effect size ofβi = β/q, referred to as ‘simulations with equal/uniform effects’ in the text. This allows, smaller effect sizes to be generated by increasing the number of QTNs, thereby simulating increasing genetic complexity by lowering effect sizes. (ii) To further explore genetic complexity, we usedN. (iii) As a third case, we sampled all QTN effects randomly from the effect sizes distribution.Here,N. We considered only epistasis, ignoring other factors such as dominance. Adding an additional term for epistasis toFor non-additive phenotypes, broad-sense heritability was set at most to 0.8, so the distribution of residuals ish2e) was set analogous to the additive heritability (h2), such thatH2 =h2 +h2e. Theadditive ×additive epistasis model was used, with only a single pairwise interaction. The epistatic effectβe was sampled fromN and attributed to a single interacting pair of markers  such thatβe =βe1 ×βe2. We sampled this interacting pair from the set of additive QTNs; therefore, each interacting marker will always have some main effect. As for additive phenotypes, we also created epistatic phenotypes with one large effect QTN. The total number of settings , physiological yellows, plant height, test weight (TW) and thousand kernel weight (TKW). This particular dataset was chosen as it contains a fairly large number of genotypes  each genotyped forp = 17,181 SNPs. The impact of population structure, training set size, marker density and its interaction with population structure was assessed in a study by the same authors and GBLUP prediction accuracies were reported to saturate when training set size was greater than 8,000. We used the same settings, with five-fold cross-validation repeated for five times .To compare trends observed in simulations with outcomes obtained with real traits, publicly available wheat genotype and phenotype data were taken from Norman, Taylor.The data was generated from a small-plot field experiment for pre-screening of germplasm containing some genotypes that are sown in multiple plots, thus containing spatial heterogeneity with correlation between closely located plots and imbalance in the number of phenotypes per genotype. Soil elevation and salinity, spatial coordinates and virtual blocks (made available on request by the authors) were taken as covariates:X is then × 4 design matrix for the fixed effects and overall mean,b is a 4 × 1 vector of fixed effects, i.e. soil salinity and elevation;Z is ann × 3 design matrix for non-genetic random effectsu, i.e. range, row and block;Zg is then ×k design matrix for genotypesg for a maximum ofk replicates, andε is ann × 1 vector of residuals. The Best Linear Unbiased Estimates (BLUEs) of genotypes were used for GP; in this way, we take care of the experimental design factors. Note that,Here,Arabidopsis thaliana RegMap panel with known structure, containing 1,307 accessions including regional samples . Additive phenotypes were simulated using narrow-sense heritabilitiesh2 = 0.1, 0.4 and 0.7, with equal effect QTNs. The genotypes, available from theArabidopsis 250k SNP array, were further pruned for LD and minor allele frequency (MAF > 5%) using PLINK v1.9 (RRID:SCR_001757). LD pruning was carried out using a window size of 500 markers, stride of 50 and pairwiser2 threshold of 0.1, using the ‘--indep-pairwise’ command. This implies that a set of markers in the 500-marker window with squared pairwise correlation greater than 0.1 is greedily pruned from the window until no such pairs remain. This dataset will be referred to as ‘STRUCT-simdata’ in the text.,To analyse the influence of population structure on the performance of different GP methods, we used a population of the This should resemble one of our simulation scenarios, i.e. high heritability (e.g.h2 = 0.7) with few QTNs (e.g. 5) of large effect. This dataset will be referred to as ‘STRUCT-realdata’ in the text.,The effect of population structure was also assessed on real data: a genotype dataset of 300 out of the 1,307 RegMap accessions, phenotyped for the sodium accumulation trait with a strongly associated gene. Principal component analysis (PCA) was performed on the allele dosage matrix using the prcomp method in R, with centering and scaling. For random forest and XGBoost, we used these top principal components as additional features in the models.To correct for population structure, we used principal components corresponding to the top ten highest eigenvalues as fixed effects in the models for GBLUP, RKHS regression, BayesA and BayesB.To explore the impact of varying LD between SNP markers and actual QTNs on the performance of GP methods, we used two other datasets: one with real genotypes and simulated phenotypes, the other with real genotypes and real traits.Arabidopsis thaliana HapMap population. Genotype data of 344 out of the 360 core accessions was obtained from Farooq, van Dijk, containing 207,981 SNPs. The phenotypes were simulated using one of the scenarios in the Section ‘Simulations’. The total number of SNPs was kept close to the number of samples and genetic complexity was kept low, to study the impact of SNP-QTN LD only. To this end, we simulated additive phenotypes withh2 = 0.7 and 5 QTNs with equal effects. Linkage disequilibrium between SNPs was calculated as squared pairwise Pearson correlation coefficient (r2) using PLINK v1.9 (RRID:SCR_001757). Input sets of 500 SNPs were selected randomly from pairs with either low LD (r2 ≤ 0.5) or high LD (r2 > 0.9); these two sets were used to train two prediction models using each GP method: one model was trained on the QTNs that were used to generate the phenotype, another on QTN-linked SNPs (closest on the genome) instead of the QTNs themselves, from the low or high LD SNPs pool. To avoid spurious correlations between SNPs in both models, non-QTN-linked SNPs were sampled from a different chromosome. We restricted the sampling of QTNs and the QTN-linked SNPs to chromosome 1, whereas the remaining non-QTN SNPs were sampled from chromosome 2. We refer to this dataset as ‘LD-simdata’ in the text.,For the first dataset, we selected a natural population with minimal structure, balanced LD, genotyped at roughly equal genomic spacing and mostly inbred lines: the 360 accessions in the core set of the This dataset contains recombinant inbred lines (RILs) derived from 40 biparental populations and the set of markers have been extensively selected for the above traits. Moreover, high dimensionality is not an issue as the dataset contains 5,014 samples and 4,235 SNPs. We refer to this dataset as ‘LD-soy’ in the text. A complete list of datasets used in this study has been provided in,For the second dataset, we used three soybean traits  phenotyped for the SoyNam population. Below, we briefly describe the GP methods used in our experiments. For (semi) parametric methods, we used BGLR v1.1.0 (RRID:SCR_022522) with default settings of hyperparameters; for Random Forests, the ranger R package v0.14.1 (RRID:SCR_022521); and for XGBoost, h2o4gpu v0.3.3 (RRID:SCR_022520).A wide range of statistical models have been proposed for GP. Most widely applied are Linear Mixed Models (LMMs), which use whole-genome regression to tackle multicollinearity and high-dimensionality with shrinkage during parameter estimation, employing either a frequentist approach, e.g. restricted maximum likelihood (REML), or Bayesian theory.Parametric modelsGBLUP SNP effects are modelled as random effects that follow a normal distribution with zero mean and common variance, and are estimated by solving the mixed model equation:The genomic best linear unbiased prediction (GBLUP) method uses a Gaussian prior with equal variance for all markers and a covariance matrix between individuals, called the genomic relationship matrix (GRM), calculated using identity by state (IBS) distances between markers for each pair of samples.g is ann × 1 vector of the total genomic value of an individual, captured by all genomic markers;μ is the overall population mean; andε is ann-vector of residuals. The genomic valuesg and residuals were assumed to be independent and normally distributed asg ~N,ε ~N. HereG is the GRM, calculated using the rrBLUP v4.6.1 (RRID:SCR_022519) package in R, providing variance-covariance structure for genotypes andI is the identity matrix. Due to the small number of estimable parameters, GBLUP is computationally fast but the assumption of normality only holds when most effects are close to zero and only a few are larger. The limitation of this approach is that it captures only linear relationships between individuals and assumption of equal variance for all marker effects may not be truly valid for many traits.Here,Bayesian methods e.g. BayesA, using a scaledt-distribution; Bayesian LASSO or BL, using a double-exponential; BayesCπ and BayesBπ, both utilising two-component mixture priors with point mass at zero and either a Gaussian or scaledt-distribution, respectively. To control the proportion of zero effect markers, the hyperparameter ‘π’ was set equal to 0.5, resulting in a weakly informative prior. For simplicity, we refer to BayesBπ as BayesB in the text. The model inβj:Several Bayesian methods with slight variations in their prior distributions have been proposed to model different genetic architecturesμ is the intercept,xj is ann-vector of allele dosages for each SNP andβj is the effect of SNPj out of a total ofJ SNPs.Here,Semi-parametric models,https://paperpile.com/c/ZyQHHy/1oKK+NO1v. We used RKHS regression as a representative semi-parametric model, because it not only employs prior assumptions for random components in LMMReproducing Kernel Hilbert Spaces (RKHS) regression is a general semiparametric method that models pairwise distances between samples by a Gaussian kernel and can therefore better capture nonlinear relationships than GBLUP. In fact, GBLUP is a special case of RKHS regression, with a linear kernelgl ~N; whereKl is the kernel evaluated for thelth component usinglth bandwidth (bl), as described below. This kernel matrixK is used as genomic relationship matrix, whereK = {k } is ann ×n matrix of Gaussian kernels applied to the average squared-Euclidean distance between genotypes:In contrast to the GBLUP model , the RKHk is a covariance function that maps genetic distances between pairs of individualsxi andxj onto a positive real value. The hyperparameterb, called the bandwidth, controls the rate at which this covariance function drops with increasing distance between pairs of genotypes. Tuning this parameter for range of values between 0 and 1 could be computationally inefficient. So, instead of tuningb, we used a kernel averaging method, such that multiple kernels, corresponding to possible bandwidth valuesbl = {0.2, 0.5, 0.8}, were averaged.The kernelEnsemble machine learning modelsRandom ForestThe Random Forest (RF) regressor uses an ensemble of decision trees (DTs) that are each grown using bootstrapping (random sampling with replacement of samples), and a random subset of SNPs. The test sample prediction is made by averaging all unpruned DTs as;x is the test sample genotype using an RFD decision trees, for whichthk tree. An RF has a number of hyperparameters that need to be tuned, for which we used grid search using the caret v6.0.92 (RRID:SCR_021138) R packagehttps://paperpile.com/c/ZyQHHy/GaA1. We used 500 trees in the forest for all analyses and tuned ‘mtry’ and ‘nodesize’ hyperparameters to control tree shapes. The total number of SNPs randomly selected at each tree node, i.e. mtry, was selected from {p/3,p/4,p/5,p/6} and the minimum size of terminal nodes below which no split can be tried, i.e. nodesize, was selected from {0.01, 0.05, 0.1, 0.2, 0.3} times the number of training samples in each cross-validation fold.HereExtreme Gradient Boosting (XGBoost)We used XGBoost, a specific implementation of the Gradient Boosting (GB) method. Similar to the Random Forest, Gradient Boosting is an ensemble method, using weak learners such as DTs. The main difference is that an RF aggregates independent DTs trained on random subsets of data (bagging), whereas GB grows iteratively (boosting) by selecting samples in the subsequent DTs based on sample weights obtained in previous DTs, related to how well samples are predicted already by these previous DTs.Hyperparameters were tuned using a grid search through five-fold cross-validation on each training data fold. We searched over max_depth = {2, 3, 4, 50, 100, 500}, colsample_bytree = {0.1, 0.2, 0.3, 0.5, 0.7, 0.9} and subsample = {0.7, 0.8, 0.9}.Performance evaluationr) between observed phenotypic values and predicted genomic values of the test population. For each model, five repeats of five-fold cross-validation were performed, so in total 25 values ofr were used to compare performances. Statistical comparison between different models was performed by comparing prediction accuracies of each pair of models as a whole, i.e. on all values ofp/n together using Wilcoxon rank-sum test.Model performance was evaluated based on prediction accuracy, which was measured as the Pearson correlation coefficient  was used. To obtain a proxy for additivity of the trait, we assumed that if a trait has a higher proportion of additive variance compared to other traits, estimated with the same model, it will be more additive. To verify this on our simulated dataset scenarios  for episga defines a set of additive genotype effects such thatga ∼N, whereG is the genomic relationship matrix (GRM) calculated as described by VanRaden. Moreover,ge ∼N is a vector of epistatic genetic effect andε is a vector of residuals. Here,E is the GRM (G oG). The ratio of additive genetic variance to the epistaic genetic variance (σ2a/σ2e) was calculated for both the simulated dataset and real wheat traits to assess their relative non-additivity. We tested our assumption on simulated phenotypes , showing simulated amounts of non-additive heritability to indeed be negatively related to empirical additive heritability.Here To investigate the role of underlying characteristics , random forest regressor (RF), extreme gradient boosting (XGBoost) regression trees and a fully-connected feed forward artificial neural network i.e. Multilayer Perceptron (MLP). The simulations covered a variety of trait scenarios (from simple to more complex), as shown inσ2a/σ2e) using). Moreover, the applicability of neural networks/deep learning for GP in the feature space is still limited due to their high tendency toward overfitting under high-dimensionality until they are properly regularized or feature selection is employed.,,Previously, numerous GP methods were tested for different traits of varying genetic architectures using low or high density marker sets, but it is still unclear for which (class of) GP problems applying machine learning (ML) can be beneficial.ristics , we geneML methods perform well for simple traitsh2) equal to 0.7 and a total number of QTNs equal to 5. We then alternatively attribute equal effects to all QTNs, assign a larger effect to the first QTN inMany non-mendelian plant traits are fairly simple, where only one or a few QTLs explain a large proportion of phenotypic variance, called oligogenic traits. If these QTLs are identified by the GP model, prediction performance can be pretty high. In our simulations , this sc) than that of genomic relationship-based methods . The performance of ML methods was slightly poorer than that of Bayesian methods when all QTNs effects were equal  inh2 equal to 0.4. Moreover, similar to simple traits, the other two scenarios, i.e. unequal effect sizes and normally distributed effect sizes, were also simulated. Two QTNs were randomly selected to have a fairly large pairwise interaction effect, corresponding to an epistatic heritabilityh2e equal to 0.4. The results in). Interestingly, when one of the QTN had a larger effect size or was attributed with most of the variance, the Bayesian methods performed on par with ML , performance dropped below that of GBLUP and RKHS as well . Therefore, ML methods are not beneficial for this setting. For epistatic phenotypes however, ML outperformed all methods including the Bayesian methods for all scenarios , with random forests generalizing the best. ML methods are thus best suited for epistatic traits and do not necessarily need large main effects to be present. Note that although RKHS regression has been reported to better capture epistatic relationships between markers, it did not perform well in our simulations; perhaps it needs more careful tuning of the bandwidth of the Gaussian distributions, rather than using multi-kernel averaging or require matching prior allele effects distributions .For additive phenotypes with equal QTN effect sizes, performance of ML methods was poorer than that of Bayesian methods under all scenarios; with an increase in genetic complexity (lowering), Bayesian methods perform comparable to ML methods for both additive and epistatic phenotypes under all simulation scenarios, although RF gave slightly better performance for epistatic phenotypes with large epistatic heritability (for2eh = 0.7) and dimensionality (p/n > 2). This could be because the large effect QTN explains most of the additive variance and is easily picked by Bayes and ML methods, but RF has the added advantage of picking up the nonlinear signal, when main effects got smaller with the increase in number of QTNs. XGBoost gave relatively poor performance, especially at smaller heritabilities (0.1 and 0.4) and largerp/n ratios, while GBLUP and RKHS regression performance was consistently poor in all scenarios.For the phenotypes explained by a large effect QTN and many small effect QTNs , the ensemble ML methods were still superior over BLUPs and comparable to Bayes when effect sizes were sampled from a Gaussian distribution for a small number of QTNs , but the advantage diminishes whenq increases and approaches the infinitesimal model i.e.q = p.For both additive and epistatic phenotypes .Genomic prediction is usually employed on a genome-wide set of markers to yield total genomic value, but the training population size is limited, i.e. a high dimensional problem. This results into more statistical power to detect QTLs with many SNPs in LD but comes with obscured genetic variance when added together. Consequently, it leads to an overestimation of allelic variances or genomic relationships, overfitting on training samples and reduced performance on unseen data. To investigate the susceptibility of different GP methods for this issue, we analysed how prediction accuracy varied depending on the ratio of markers vs samples (c =). This implies that for simple traits having one or few large effect QTNs . Here, for the case when main effects were sampled from a Gaussian distribution, increasing polygenicity is analogous to having many small main effects; so, despite having epistatic effects, performance goes down for all methods. In the nutshell, this shows that the conclusions drawn in Section ‘ML methods perform well for simple traits’ and Section ‘ML methods outperform parametric methods for complex traits’ holds under high-dimensionality.In general, the results with different simulation settings of ‘simdata’ for additive phenotypes show that performance is negatively related to an increase in dimensionality when main effects got smaller due to decreasing heritability or increasing total number of QTNs  and 17,181 markers (c ≈ 1.6). These markers have been selected by strict screening criteria, therefore, many of them could be informative. Insights in the genetic complexity for some of these traits were previously reported in Norman, Taylor and Norman, Taylor. For example, glaucousness was reported to be a simple trait, but grain yield to be more complex. The results inr) were higher for both ML methods when the fraction of additive variance was small (i.e. traits were fairly non-additive) and slightly lower or comparable to both Bayesian and GBLUP/RKHS regression methods otherwise. This is in line to what we observed in our simulations: for simple traits  variances of random effects in LMMshttps://paperpile.com/c/ZyQHHy/ByqH. Parametric and nonparametric ML methods, based on their modelling assumptions and approaches, may be differently sensitive to PS. To assess the impact of population structure on ML methods, we used real genotype data with a known population structure and combined it with both simulated (STRUCT-simdata) and real phenotypes . Only additive phenotypes were simulated, with varying complexity and dimensionality scenarios, as described earlier in Section ‘Simulations’. The STRUCT-simdata contains all 1,307Arabidopsis RegMap accessions. To exclude the impact of multicollinearity among SNPs, only uncorrelated markers were retained after pruning with pairwise squared correlation coefficient , leaving 15,662 SNPs, but keeping the population structure intact . This results in a ratioc =p/n of approximately 12 , a setting comparable to the simulation results presented in.Population structure (PS) is a well-known confounding factor that results in decreased diversity in training populations), average pairwise difference of test accuracies before and after correcting for PS was slightly higher for ML methods (RF: 0.03 and XGBoost: 0.04) than for LMMs . Moreover, the correction resulted into relatively elevated accuracies for the scenarios with larger number of QTNs or low heritabilities. This illustrates that with smaller #QTNs and larger heritabilities , effect sizes per QTN were larger; therefore, confounding due to PS was less of a concern. With the decrease in effect sizes per QTN (increase in #QTNs and decrease inh2), correction became more important for reliable predictions. From this, we can argue that confounding due to PS should be generally corrected for, but particularly for complex phenotypes having low heritability and large numbers of QTNs with small-medium effect sizes.Correction for PS was carried out by including the top ten principal components corresponding to the largest eigenvalues as fixed effects into the mixed model equations or as additional features for ML methods. For the simulated phenotypes  using a subset of the same genotypes dataset. Here, we expected to have at least one large effect QTN for this trait, becauseAtHKT1;1 locus, encoding a known sodium (Na+) transporter, has been reported to be a major factor controlling natural variation in leaf Na+ accumulation capacity. Similar to the outcomes on ‘STRUCT-simdata’, correction for PS increased prediction accuracies of all methods on test data; whereas, GBLUP showed the lowest average difference (∆μ = 0.03) in performance before and after correction  capture their effect, depending on the actual correlation and allele frequencies. Therefore, it is worthwhile to investigate the impact of SNP-QTN correlation levels on GP performancehttps://paperpile.com/c/ZyQHHy/Q1iWE. We used two settings, one with real genotypes and simulated phenotypes (LD-simdata), a second with real genotypes and real traits (LD-soy).The utility of GP in genomic selection is based on the assumption that there are ample markers within a densely genotyped set of markers which are in LD with the QTLs.r2 > 0.9), all methods perform equally well and the SNP-based model predictions are very close to those of the actual QTN based models . On the other hand, for low LD between SNPs and QTNs, there was in general a difference between median prediction accuracies (Δr) of the QTN and SNP-based models . Our results show that when SNPs are highly correlated to QTNs (which is likely the case for densely genotyped markers set and models . This dir2 ≤ 0.5). Here, in contrast to LD-simdata, where we knew the QTNs in advance, we assumed that many SNPs could be linked to QTNs, because ~94% of all markers hadr2 > 0.5. So, we compared two models: one with all markers (the benchmark model), and one with low LD (r2 ≤ 0.5). A similar pattern was observed, as shown inAs a real genotype and phenotype dataset, we used three Soybean traits, i.e. height, time to R8 developmental stage and yield (LD-soy). The motivation was to choose a low-dimensional real dataset with highly correlated SNPs to understand the impact of SNP-QTL LD only. The complete set of markers  had many correlated SNPs, such that only 261 were left with low LD  were more stable under low SNP-QTN LD compared to other methods . Moreover, RF seems to couple good prediction performance with reliability under low SNP-QTN LD.p/n > 1), which is less of an issue with ML, especially with RF .Genomic prediction has long been the realm of parametric methods, but recently nonlinear supervised ML methods have become increasingly popular. Yet literature is unclear on the characteristics of GP problems that warrant application of ML methods. This study fills this gap and concludes that nonlinear tree-based ensemble ML methods, especially Random Forests, can be a safe choice along with traditional methods for simple as well as complex polygenic traits where epistatic allele interaction effects are present. We simulated different scenarios mimicking the reality at a broader level e.g. the case of simple oligogenic traits , where wp SNPs were used always to calculate the kinship matrices, whereas, either 5, 50, 100,p/2 or exactlyp SNPs were chosen as QTNs. Whenq is fairly less thanp, makes the kinship matrix too noisy due to the large number of markers that are unrelated to the phenotype but are used in the calculation of the GRM. Hence, we expect equal accuracies for increasing number of QTNs (q), keeping the other factors  fixed. Figure S5 (Extended Data) clearly illustrates that these methods indeed have constant prediction accuracies with increasingq values, while the accuracies of the other methods drop due to decreasing effect sizes. This further explains that their performance can be improved by removing unrelated markers from the GRM, for instance using biological knowledge about markers.,A wide range of parametric, semi-parametric and nonparametric methods can be used for GP, but it is impractical to test all for a particular application. The choice for a suitable method strongly depends on the GP problem characteristics, described in) that these ML methods had better performances than other ML methods (SVR and MLP), we can argue that these tree-based ML methods are a reasonable choice to conduct GP.The parametric LMM equations can be solved using a Bayesian framework. Bayesian methods define prior SNP effects distributions to model different genetic architectures. Instead of a single distribution for all marker effects (e.g. BRR), it could be defined for each individual marker (e.g. BayesA). Mixture distributions have also been proposed . From the Bayesian alphabet, we used BayesA and BayesB as representatives because the first scenario, i.e. a single distribution for all markers, has been covered by GBLUP. Our results illustrate that these methods outperform GBLUP and RKHS regression when large effect QTNs are present, for both additive and epistatic phenotypes. On the other hand, tree-based ensemble ML methods had either comparable performance to Bayesian methods (for simple traits) or superior performance (for complex traits). Capitalising on the results from Appendix-I  do not employ any statistical prior and learn the association patterns from the data itself, they may be more sensitive to structure, as we found in our simulation results. But this is not clearly evident from the real phenotypes, so we cannot generalize this conclusion from our simulations.The impact of population structure can be dealt with in many ways. Conventionally, principal components of the SNP dosages or genomic relationship matrix are introduced as fixed effects in the mixed model equations.– Instead, many GP problem characteristics  had the largest decrease in test performance under low SNP-QTN LD compared to high SNP-QTN LD. This could be due to the application of penalties on individual marker effects, which shrinks the weak SNP-QTN associations towards zero for each SNP. Therefore, minor improvements in GP methodology leading to overall genetic gain can have high impact. In this study, we used a large  Australian germplasm panel, genotyped with a high quality custom AxiomTM Affymetrix SNP array and phenotyped for multiple traits with varying complexity levels. The authors showed that genomic selection was superior to marker-assisted selection (MAS) by employing GBLUP with two random genetic components (referred to as full-model in their text). Our results clearly indicate that ML can perform well for complex bread wheat traits, e.g. grain yield, yellows, greenness, biomass and NDVI. However, for NDVI, the larger difference between LMMs and ML could be due to low phenotypic variance and heritability for this trait in this dataset. All of these traits except grain yield can be measured using high-throughput automated phenotyping. This is an interesting finding since, with the rapid advances in low cost high-throughput phenotyping systems, attention is shifting towards measuring component traits, e.g. vegetative indices, rather than final yields. ML methods can predict these traits more accurately, as evident from our analysis.Bread wheat breeding has huge impact on worldwide food security and socio-economic development.Based on simulated and real data, we conclude that tree-based ensemble ML methods can be useful for GP for both simple and complex traits. Moreover, these methods can work for both low- and high-density genotyped populations and can be a competitive choice for practical plant breeding. In practice, which method works best depends on the particular problem, i.e. genetic architecture of the trait, population size and structure and data dimensionality. Between bagged (Random Forests) or boosted (XGBoost) decision tree ensemble methods, random forest seems to be a good first choice for GP given their generalization performance. Furthermore, population structure should properly corrected for to obtain stable performance. It would be interesting to investigate to what extent these ML methods can benefit from statistical or prior knowledge-based regularization techniques. The paper descries comparison study of performances between various types of machine learning models and statistical models to find the best model for genomic prediction using SNP data. To compare the prediction performance, simulation was performed with several different ratio of causal and non-causal variants, the number of samples and variants, heritability, effect size distribution, existence of population stratification, LD structure, and different genomic assumptions. In the simulation study, ML methods generally performed better than statistical models.  The paper seems written well and the structure looks good. However, I think that it needs some more improvement, and there are some my suggestions below. First, the paper seems little bit long, so it would be better to reduce the length of paper by putting some introduction of methods to supplement. Also, I think that discussion section can be reduced by merging first, second, and fifth section of discussion.et al. Second, in many prediction model studies, the marker selection step before constructing main model is essential to improve the prediction ability of statistical model. I think that if you perform SNP screening first, parametric models would perform better than current results. See Shigemizu Third, it would be great to know how much time will take for each model development with various settings. I think that RKHS regression takes much longer times than the other methods so that it would be one of the great reason why ML need to be used for efficient prediction modeling. Fourth, in population structure analysis, I think that you need to calculate principal components with only training dataset and projects the PCs for testing dataset with those results.  I don't think authors need to do all of above suggestions to index the paper, but those things should be described in the paper.Is the work clearly and accurately presented and does it cite the current literature?YesIf applicable, is the statistical analysis and its interpretation appropriate?YesAre all the source data underlying the results available to ensure full reproducibility?YesIs the study design appropriate and is the work technically sound?YesAre the conclusions drawn adequately supported by the results?YesAre sufficient details of methods and analysis provided to allow replication by others?YesReviewer Expertise:I am working in statistical genetics area with statistical modeling and machine learning.I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. et al., 2019). As many studies have shown, the best method is trait and scenario dependent. Therefore, I suggest to include a small dose of caution in the conclusions. The paper is perhaps overlong.Authors present an ample study on ML and linear methods for GP. Their conclusion is very clear (RF and XGBoost end to outperform linear methods); However, they overinterpret, - RF and related methods are better when a few variables are of large influence, as multiple studies have shown. It is true that ensemble methods tend to be a safe choice . GBLUP can also include dominance and epistasis , which shows similar behavior across methods, as usually observed. Fig 3: weird some very big differences . et al. (2019)Azodiet al. (2022)Reinoso-Peláez Some review refsIs the work clearly and accurately presented and does it cite the current literature?PartlyIf applicable, is the statistical analysis and its interpretation appropriate?YesAre all the source data underlying the results available to ensure full reproducibility?YesIs the study design appropriate and is the work technically sound?PartlyAre the conclusions drawn adequately supported by the results?PartlyAre sufficient details of methods and analysis provided to allow replication by others?YesReviewer Expertise:Quantitative Genetics, Machine LearningI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. We thank the reviewer for constructive criticism and providing a valuable input to the article that helped us to improve it further. We have tried to address the comments one by one and made some changes to the text accordingly.Reviewer) I suggest to include a small dose of caution in the conclusions. I am not sure a clear rationale on choosing between ML and LMM is lacking, but you do not solve it. Authors) We agree that the choice of a particular method is scenario dependent. We also concur that despite simulating many scenarios, we do not find a definitive rationale for choosing between ML and LMM. Instead, our study could provide some general guidelines that can be helpful when considering this issue. We have revised the conclusion section to clarify this and modified the abstract to avoid over-interpretation.Reviewer) The term 'non-linear phenotypes' is misleading, if it refers to the degree of non-additivity, I am not sure we can tell so easily between linear and non-linear phenotypes.Authors) Thank you for pointing this out. Although we did define non-linear phenotypes as those exhibiting epistatic interactions, we agree the use of this term can be misleading. We have therefore replaced all uses of linear and non-linear phenotypes by additive and epistatic, respectively.Reviewer) The simulation of allele frequencies is completely unrealistic; not detailed on how effects are sampled.Authors) We fixed MAF=0.4 and decided not to incorporate the impact of allele frequencies in our analysis because MAF of QTLs can impact heritability estimation and ultimately prediction accuracies . This allowed us to observe equal and reasonably enough statistical power for each SNP during allele effects estimation. To clarify for the readers, we have also mentioned this in the text now (lines 105-109).N, to simulate three different cases. For the first case, one effectβ was sampled and then allq QTNs were allocatedi = ββ /q, i.e. all QTNs had equal effects. This allowed us to evaluate a trait complexity scenario where effect sizes could decrease with increasing numbers of QTNs. In the second case, we sampled two effect sizes  from the effect size distribution. For the third case, all effects were randomly sampled from the Gaussian distribution. The Methods section now contains a more elaborate description of our approach on lines 130-141. The effect sizes were sampled from a Gaussian distribution ~Reviewer) Table 1: you use up to 60k QTLs?Authors) Yes; for the case in which we had 60k SNPs, one possible scenario was to use 60k QTNs. This was simulated to illustrate the infinitesimal modelling scenario.Reviewer) Table 2, provide also n and p.Authors) Thank you for the suggestion. Table 2 has been updated.et al., 2018Reviewer) GBLUP can also include dominance and epistasis .Authors) Indeed it can, and there are many other variations. We believed a full comparison of these methods to be out of the scope of this manuscript. However we did use RKHS as a general class of BLUPs with a three-kernel averaging scheme  using bandwidth values b={0.2,0.5,0.8}.th' components in equation 6?Reviewer) What are the 'IAuthors) The RKHS method contains three random genetic effects. The ith component is the ith random genetic effect, with a genomic relationship matrix determined by Gaussian kernels corresponding to i = first, second or third value of bandwidth from {0.2,0.5,0.8}. We clarified this on line 294.Reviewer) A clear link between simulated and real phenotypes is often missing?Authors) We agree that a direct link between simulated and real traits is difficult to establish because simulations simplify things to a large extent. A better link could perhaps be established if the same population was utilised for simulated phenotypes. However, the purpose of our simulations was to give a general idea of where RF/XGB could be a potential choice, not for a specific population. We observed these methods work well for the case with a few QTNs with large effect or when the phenotype contains interaction effects. When the trait is highly polygenic and additive, such that the effect sizes distribution resembles a Gaussian , similar performance can be expected for RF/XGB and LMMs.Reviewer) Equation 9, not clear how genetic residual is calculated nor its meaning? The estimated non additive variance? A negative relation by the way of additive and non-additive part cannot be generalized.Authors) We agree that this was not clear and have changed equation (9) to accommodate the additive and epistatic random genetic components explicitly. Thus, the additive component is governed by the additive covariance matrix, whereas the epistatic component has a covariance matrix accommodating a fraction of the SNP interactions, as proposed by the E-GBLUP methodology (Yong Jiang and Jochen C Reif 2015). With both of these random components, we were able to extract the proportion of additive (σ2a) and epistatic (σ2e) variances for a trait and therefore, the σ2a/σ2e, should in principle, be higher when epistatic variance is low and vice versa. We have mentioned this in the text for clarification on lines 349-355. This negative relation was also found in our simulated data . Accordingly, for the real wheat traits, we observed a negative relation between σ2a/σ2e and the maximum difference between LMMs and RF/XGB accuracies. There was a clear separation between the traits, as plotted inFigure R1, with a lower additive:epistatic variance ratio  and those with a higher ratio .Figure R1, we can see that for traits that are predominantly additive and polygenic, LMMs and RF/XGB perform quite similar ; but for traits with a predominant epistatic component, for example, grain yield and biomass, RF/XGB outperform LMM. We agree that this negative relation cannot be generalized, but given our results testing the modified equation (9) on multiple phenotypes of the same population, we believe it to be an extrapolation for the same population. In the aboveReviewer) Out of all scenarios simulated, the most realistic is panel F , which shows similar behavior across methods, as usually observed.Authors) We agree to some extent that F is more realistic for many complex traits, but the scenarios in panels B and C can also be observed, e.g. for oligogenic traits. An example is the sodium accumulation trait , where ML performed well due to the large effect QTN. The panels with equal effects were meant  to illustrate the impact of many small effect QTNs and small narrow-sense heritabilities. We have clarified this point further in the text on lines 658-666.Reviewer) Fig 3: weird some very big differences .Authors) The large difference in prediction accuracy for grain yield and biomass could be attributed to their larger epistatic variance compared to the other traits. For the NDVI trait, phenotypic variance and narrow-sense heritability were low in this dataset, so this should be specific to this dataset only. We have explicitly mentioned this issue on lines 773-775 in the text. All the components are written according to international standards and latest information is cited.  The statistical designs are used according to the demand of hypothesis and nature of data for the interpretation of the results. The protocols are elaborated in this manuscript so that one can repeat these experiments for more interpretations. The needed statistical analyses are conducted for the interpretation of the data. Methods-Data-Simulations: Explain the reason why you selected 500 for the sample size. What was the rationale behind choosing fixed MAF=0.4 and not other than that? In Figure 5B, why does yield have more accuracies than height and R8; whereas, yield is usually considered as a relatively more complex trait than others. The data is submitted in the relevant repository and authors would be available to answer any query from the researchers. The conclusion is to the point based on results and will be guideline for later studies. The Ref#55. Patterson N, Price AL, Reich D: Population Structure and Eigen analysis. PLoS Genet. 2006; 2(12): e190. should be removed because Ref#54 is its follow up study.Is the work clearly and accurately presented and does it cite the current literature?YesIf applicable, is the statistical analysis and its interpretation appropriate?PartlyAre all the source data underlying the results available to ensure full reproducibility?PartlyIs the study design appropriate and is the work technically sound?YesAre the conclusions drawn adequately supported by the results?YesAre sufficient details of methods and analysis provided to allow replication by others?PartlyReviewer Expertise:Plant Breeding and Biotechnology, GeneticsI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. We thank all reviewer for providing a valuable input to the article that helped us to improve it further. We have tried to address their comments one by one and made some changes to the text accordingly.Methods-Data-Simulations: Explain the reason why you selected 500 for the sample size. What was the rationale behind choosing fixed MAF=0.4 and no other than that?et al., 2022 used 457 soybean breeding lines, Sandhuet al., 2021 used 650 spring wheat lines etc. We fixed MAF=0.4 for all SNPs, and decided not to incorporate the impact of allele frequencies because the MAF of QTLs can impact SNP heritability estimation and ultimately prediction accuracies . With this setting, we obtained enough statistical power for each SNP during allele effects estimation (line 105-109). We chose 500 samples since a reference population of this size is common for many genomic selection applications in plant breeding, as mentioned on lines 112-113. For example, NonoyIn Figure 5B, why does yield have more accuracies than height and R8; whereas, yield is usually considered as a relatively more complex trait than others.et al., 2019 from the complete SoyNAM genotype dataset after rigorous feature selection. The motivation was to choose a real, low-dimensional dataset with highly correlated SNPs to understand the impact of SNP-QTL LD only, while benchmark accuracies were also available in that study. Note that in general the accuracy of a genomic prediction model depends on whether and how many of the SNPs causal for a trait are included in the model. It might be that the feature selection performed by Azodi et al. had a stronger impact on height and R8 compared to yield. In any case, our results do match with those of Azodi et al., 2019 , who found lower accuracies for predicting height and R8 than for yield, given this selected subset. We thank the reviewer for pointing this out, and now explicitly mention this in the text on lines 636-638 and 645-646. We used the ~4.2k SNPs extracted by AzodiThe Ref#55. Patterson N, Price AL, Reich D: Population Structure and Eigen analysis. PLoS Genet. 2006; 2(12): e190. should be removed because Ref#54 is its follow up study. Thank you for pointing it out. This reference has now been removed.


V), respiration rate (RR), panting score (PS), body condition score (BCS), hair density (HD), body size (BS), ear size, and respiration efficiency (Reff).Genetic selection based on direct indicators of heat stress could capture additional mechanisms that are involved in heat stress response and enable more accurate selection for more heat-tolerant individuals. Therefore, the main objectives of this study were to estimate genetic parameters for various heat stress indicators in a commercial population of Landrace × Large White lactating sows measured under heat stress conditions. The main indicators evaluated were: skin surface temperatures (SST), automatically-recorded vaginal temperature , RR (0.06 ± 0.01), PS (0.05 0.01), and Reff (0.03 ± 0.01). Moderate to high heritability values were estimated for BCS , HD (0.25 ± 0.05), BS (0.33 ± 0.05), ear area , and ear length . High genetic correlations were estimated among SST traits (> 0.78) and among TV traits (> 0.75). Similarly, high genetic correlations were also estimated for RR with PS (0.87 ± 0.02), with BCS measures (0.92 ± 0.04), and with ear measures (0.95 ± 0.03). Low to moderate positive genetic correlations were estimated between SST and TV (from 0.25 ± 0.04 to 0.76 ± 0.07). Low genetic correlations were estimated between TV and BCS (from − 0.01 ± 0.08 to 0.06 ± 0.07). Respiration efficiency was estimated to be positively and moderately correlated with RR (0.36 ± 0.04), PS (0.56 ± 0.03), and BCS . All other trait combinations were lowly genetically correlated.Traits based on TA comprehensive landscape of heritabilities and genetic correlations for various thermotolerance indicators in lactating sows were estimated. All traits evaluated are under genetic control and heritable, with different magnitudes, indicating that genetic progress is possible for all of them. The genetic correlation estimates provide evidence for the complex relationships between these traits and confirm the importance of a sub-index of thermotolerance traits to improve heat tolerance in pigs.The online version contains supplementary material available at 10.1186/s12711-023-00842-x. Heat stress is a major welfare and production concern in the swine industry that negatively impacts metabolism and physiological mechanisms  of the aAdvances in genetic and genomic selection, nutrition practices, and management have increased sow performance, including larger litter sizes and higher milk production . These cOur hypothesis is that genetic selection based on closer-to-biology indicators of heat stress can capture additional physiological and behavioral mechanisms involved in heat stress response and enable more accurate selection of individuals adapted to challenging climatic conditions. In spite of its importance, physiological indicators of heat stress are not regularly collected in swine herds due to measurement difficulties, need of specific personnel training, and costs. For example, core body temperature would be an ideal metric, however it is difficult and usually invasive to measure. To overcome these difficulties, several approaches to measure body temperature have been developed in recent years, such as intramuscular implants in broilers , 20, rumV), respiration rate (RR), panting score (PS), body condition scores (BCS), hair density (HD), body size (BS), ear measurements, and respiration efficiency (Reff).Genetic selection for more heat-tolerant animals has the potential to improve animal welfare, productive efficiency, and the long-term sustainability of the swine industry, especially considering that the effects of climate change are becoming more severe , 28. TheThe experimental protocol followed ethical principles in animal research  and was approved by the Purdue University Animal Care and Use Committee (Protocol #1912001990).Phenotypes were collected on 1645 multiparous (from parity 2 to 7) lactating sows (Large White x Landrace cross) under commercial settings (lower use of heat abatement strategies), as described by Freitas  and JohnRH), and average hourly DP (AVGDP) for statistical analyses by averaging all recorded points during one hour. The environmental data from each of the 4 data loggers (or in some cases each of the remaining ones in the room when some of them stopped working during the experiment) were used to calculate the averaged climatic variables.For in-barn environmental data, 4 data loggers  per farrowing room were placed in one of 4 quadrants that were mounted at sow height, and dry bulb temperature (DBT) and relative humidity (RH) were recorded every five minutes during the data collection period . Using the in-barn climatic variable record, dew point (DP) was calculated using the Magnus-Tetens equation  as folloES), shoulder skin temperature (TSS), rump skin temperature (TRS), tail skin temperature (TTS), and TV, which were collected on all sows throughout the study. RR was collected by counting flank movements for 15 s at 8:00, 12:00, 16:00, and 20:00 h daily during 4 consecutive days, and multiplied by 4 to calculate breaths per min (bpm)  were subjected to single-trait model comparisons based on the deviance information criterion (DIC) using the THRGIBBS1F90 software  can be posed by ; \docume and rep using thThe theoretical accuracy of the breeding values predicted for each trait and animal was calculated as:as above .ES, TSS, TRS, TTS, TV, RR, EA, and EL) are in Table Descriptive statistics for the continuous traits . AVGtemp presented the best estimates of DIC  and the largest amount of estimated additive genetic variance among the available in-barn climate descriptors. Therefore, AVGtemp was chosen as the covariate for each trait in the subsequent analyses. A quadratic function of AVGtemp was also evaluated as covariates but it presented a worse model fit than the linear covariate of AVGtemp for all traits, except for TSS and TTS. Therefore, a linear function of AVGtemp was included in the model for TES, TRS, TV, RR, and PS, and a quadratic function of AVGtemp for TSS and TTS.Table eff) to 0.40 (EA). SST traits generally had a low heritability, ranging from 0.04 to 0.06. Low heritability estimates of 0.06 and 0.05 were also observed for RR and PS, respectively. Vaginal temperature based on all records  had moderate heritability (0.15). In contrast, TV records developed from TVall had higher heritability estimates. The measures of TV based on repeated records per day for 4 days  and the single time per day measures  had moderate heritabilities, with TV12h having the highest heritability (0.24). The estimates of heritability for single record TV traits  were similar to those for the single time per day measures, with TV12hS being the trait with the highest heritability. The single heritability estimates were equal to 0.25, 0.29, 0.22, and 0.22 for TV8hS, TV12hS, TV16hS, and TV20hS, respectively. Moderate heritability values of 0.25, 0.29, and 0.25 were observed for BCSVis, BCSCal, and HD, respectively. The body size score was the most heritable trait among those studied, with a heritability of 0.33. Regarding repeatability, TES, TSS, TRS, TTS, RR, and PS had the lowest re values , while the traits based on TV had the highest re, ranging from 0.37  to 0.59 (TV8h).The heritability estimates for each trait are in Table ES with TSS), 0.81 , and 0.91 .Estimates of genetic correlations among traits  to 0.88 (TTS with TV20hS). Between all the TV traits, TVall and single measurement records  showed the highest Vall with TTS with an ES, TSS, TRS, and TTS, respectively. A similar pattern of low correlations was observed between SST and PS, with the ES and PS being the highest (0.36), while the other trait combinations had estimates of 0.13, 0.16, and 0.17 . The SST traits were also lowly genetically correlated with BS, with estimates ranging from 0.05 (TSS with BS) to 0.28 (TES with BS). Low Skin surface temperatures had low to moderate positive mentclass2pt{minimVall with the other TV measures, except for TVall with TV20h with an V4days and TV20h for which the V with TV4days had an V8h, TV12h, TV16h, and TV20h) were highly genetically correlated with each other, with values higher than 0.96. The same pattern of high V8hS, TV12hS, TV16hS, and TV20hS), for which the later measurement times had the lowest V8hS with TV20hS), and the earlier measurement times had the highest V8hS with TV12hS, and 0.97 for TV16hS with TV20hS). Vaginal temperature derived based on single records per day and single records had moderate to high V8h with TV8hS, and TV12h with TV12hS, and equal to 0.99 for TV16h with TV16hS, and TV20h with TV20hS. All other combinations of traits had a moderate to high V8h with TV20hS) to 0.98 (TV12h with TV8hS).Vaginal temperature traits were estimated to be positively and highly genetically correlated with each other. High V8h to 0.42 (± 0.09) between RR and TV16hS, and between RR and TV20Hs (± 0.07). Low V4days to 0.87 (± 0.02) between PS and RR. Respiration efficiency (Reff) had positive moderate correlations of 0.41 (± 0.02), 0.36 (± 0.02), 0.56 (± 0.03), 0.56 (± 0.05), and 0.50 (± 0.05) with TTS, RR, PS, BCSCal, and BCSVis, respectively. Negative eff and TVall (− 0.28 ± 0.11). Reff was also negatively genetically correlated with TV measured at 8:00 and 12:00 h, with values of − 0.38 (± 0.07), − 0.20 (± 0.07), − 0.35 (± 0.10), and − 0.17 (± 0.10) for TV8h, TV12h, TV8hS, and TV12hS. However, Reff had positive correlations with TV measured at 16:00 and 20:00 h, with values of 0.19 (± 0.08), 0.19 (± 0.11), 0.05 (± 0.08), and 0.07 (± 0.09) for TV16h, TV20h, TV16hS, and TV20hS.The estimate of Cal and BCSVis. Negative low to moderate Cal and TV measurements, with values ranging from − 0.32 (± 0.08) for BCSCal with TV12hS to − 0.55 (± 0.10) for BCSCal with TV8h. A low negative Cal and SST measures, ranging from -0.08 (± 0.03) for BCSCal with TTS to -0.19 (± 0.04) for BCSCal with TRS. Similar to BCSCal, BCSVis had low negative TS to − 0.12 ± 0.04 with TRS). Negative low to moderate Vis and TV traits . Body size was moderately genetically correlated with BCSCal and BCSVis, with estimated values of 0.60 (± 0.05) and 0.63 (± 0.06), respectively. The RS to 0.28 (± 0.09) for BS with TES. Low V traits, with absolute values ranging from − 0.03 ± 0.08 (with TV12h) to 0.21 ± 0.08 (with TV20hS). For HD, the ES (− 0.02 ± 0.03), TSS (− 0.06 ± 0.04), and TRS (-0.02 ± 0.04), and positive with TTS (0.26 ± 0.04). Low V traits, ranging from 0.08 (for HD with TV12hS [± 0.08] and TV16hS [± 0.07]) to 0.24 ± 0.11 . All other combinations of traits presented low A high mentclass2pt{minimeff had the lowest accuracy (0.30 ± 0.10), and TVall the highest (0.66 ± 0.07). The GEBV for SST traits had similar accuracies, ranging from 0.58 ± 0.07 for TSS to 0.60 ± 0.07 for TES. Among all the TV-based traits, TVall had the highest accuracy (0.66 ± 0.07).The average theoretical accuracy of GEBV was moderately high for all the studied traits Table ; the GEBAdvances in genetic selection in the pig industry have led to increased productivity, which has resulted in sows producing more heat that they must dissipate without too much effort to avoid heat stress. To cope with elevated body temperatures, lactating sows tend to reduce feed intake and milk production, which leads to compromised reproductive performance and growth rate of the piglets , 41, 42.V, which represents the core body temperature, displayed a wide range of heritability estimates (from 0.15 to 0.29) depending on the time of measurement. Gourdine et al. [V evaluated in our study, TVall, which accounts for all TV measures, had the lowest heritabilities. This lower heritability of TVall might be related with the type of model used , which may not be able to account for reduced genetic variation in TV measured during the night period. One alternative way to analyze this longitudinal data may be to use random regression models, where the amount of genetic variation could be estimated for each time-point. The single measure traits, TV8hS and TV12hS, had the highest estimates of heritability  among all the TV and could be used in breeding programs to select individuals that can maintain their body temperature in different weather conditions due to its ease and speed of collection, and the large amount of genetic variation that they account for. Furthermore, the high genetic correlations and Spearman correlations greater than 0.82 between GEBV of the different TV measurements indicate that all TV metrics were able to similarly rank the individuals based on their GEBV, and that in most cases the same animal would be selected based on their high or low GEBV.The genetic variation observed for the traits evaluated in this study indicates great opportunities for genetic improvement of heat tolerance in swine populations. Te et al.  estimatee et al. ), and fre et al. ). RegardIncreasing RR and beginning to pant is one strategy that pigs use to release more heat through evaporative heat loss . Howevereff. As a result, the estimation of Reff may be inaccurate and future studies should consider effects such as feed consumption and milk production in their calculations.Respiration efficiency, which was defined in our study as the efficiency of the animal to reduce its body temperature through increased respiration rate, also provided low heritability estimates, but significantly different from 0. Other factors that were not considered in our study can affect the animal's metabolism, such as reduced feed consumption, reduced milk production, increased water usage, and changes in the prioritization of body reserves, and consequently the animal’s ability to cope with changes in body temperature and influence Re that differed depending on the combination of traits, and for TES it implies that the phenotypic values changed substantially within the day and that it is more responsive to changes in the thermal environment. In addition, SST traits may be substantially affected by cooling systems (evaporative or conductive), water , and air speed [V.The lack of functional sweat glands in pigs results in most of the heat elimination by evaporation to occur in the respiratory tract. Still, the skin represents another mechanism that pigs use for heat loss by increasing blood flow to the skin. However, this mechanism is inefficient during heat stress events, when the ambient temperature might approach or exceed the skin temperature and, thus, instead of losing heat to the environment the skin ends up gaining heat . The lowir speed . TherefoIn this study, body size score was demonstrated to be highly heritable (mean heritability of 0.33), which is consistent with other studies that evaluated similar traits in swine populations. Ohnishi and Sato  in the DGenetic parameters for ear size are scarce in the literature. In the current study, ear size measurements were found to be a highly heritable trait. Potentially, larger ears could facilitate heat loss to the environment since the animal’s skin surface area would be greater compared with a same sized individual with smaller ears . In animBCS measures presented moderate to high heritabilities. Body condition is an essential measure to ensure the right amount of nutrition and consequently production in the herd . Fat andV, it can be concluded that selection on any of the measurement (trait) chosen will result in substantial genetic progress for all the TV traits. Among all the combinations of measurements, those taken at more distant times  had the lowest genetic correlation  compared to measurements taken at closer times . This was expected, as behavioral, physiological, and metabolic changes tend to be observed at different hours of the day, and may require activation of different sets of genes. Events such as feeding increase the metabolic heat produced by an animal's body due to the heat associated with nutrient processing [V traits. Thus, they represent potentially useful traits to be included in a selection scheme for heat tolerance, especially TV measured at noon (both repeated and single day record) due its higher heritability estimate.Considering the estimates of genetic correlations between measurements of Tocessing , 60 and,V-based trait used. In most cases, genetic correlation estimates were low, except when using single TV measurements. The low genetic correlation obtained with repeated measurements shows that SST may not be the best indicator trait for heat tolerance, since, as mentioned earlier, SST can be more affected by several external factors.Skin surface temperatures measured at different locations on the sow’s body were highly genetically correlated (from 0.78 to 0.91). This was expected due to the role of the skin’s surface in dissipating heat , 49. In eff was low, which indicates a weak genetic association of most SST traits with the animals’ efficiency for losing heat. SST also had weak genetic correlation estimates with RR and PS. However, TTS had a higher eff. This genetic relationship might be related to genes that act on both phenotypes that could be further investigated based on genome-wide association studies and multi-omics studies . However, as Gourdine et al. [The e et al.  pointed V and RR and between TV and PS had a similar pattern, with the single TV records presenting a higher Genetic correlations between Teff, meaning that longer ears are genetically related to a better Reff. This relationship might not be directly related to the SST measured at the ear, since the ES was low (0.08), however, it may be related to sensitization mechanisms, where individuals with larger ears can better perceive fluctuations in their body temperature and trigger responses on the hypothalamus and a consequent increase in their respiration. This may be related to the fact that we estimated a positive genetic correlation between BS and EL. Therefore, greater BS is genetically related to greater EL, and these individuals tend to have a higher RR . However, future studies need to further investigate this relationship. Genetic correlations between BS and BCS and between BS and TV, agree with this finding, showing that greater BS is genetically related to greater BCS, which is genetically related to lower TV. Another explanation for the observed genetic correlations between BCS and Reff could be that, as respiratory responses are controlled by thermoreceptors located in the skin and muscles, fatter sows may feel the effects of increased temperature faster, which triggers the reaction of the receptors that increase RR. This would result in a lower core body temperature and better Reff. However, this relationship should be further investigated.Ear size (area and length) was lowly genetically correlated with the majority of the traits evaluated. However, EL had a moderate negative correlation with RV measures might indicate that individuals with more hair cover have a disadvantage for dissipating heat and consequently increase their body temperature under heat stress conditions. As stated previously, the unfavorable genetic relationship between hair cover and thermotolerance was proven true in Senepol cattle [V and understand whether sows with more hair cover have a disadvantage (or not) when dissipating heat through the skin. If this relationship was proven, HD could be used as another indicator for thermotolerance in breeding schemes.The positive (but low) l cattle –54. FurtThis is the first study to genetically investigate phenotypes that are related to physiological responses of sows under heat stress conditions in commercial settings. We report estimates of genetic parameters for traits related to physiological responses that provide important findings to the literature. The heritability and substantial additive genetic variance of the studied traits show the potential of including them in selection schemes to improve heat tolerance in swine. However, economically-important phenotypes that reflect the productive and reproductive performance of sows  need to be collected and the genetic correlations of the studied traits with these economically-important traits quantified, enabling the development of a selection sub-index based on a combination of some of the traits evaluated in this study, in order to improve heat tolerance while maintaining productivity. In addition, a larger number of phenotyped and genotyped animals in other populations is needed to validate the results obtained. This validation is also important in purebred populations, which are usually the populations under intense selection pressure. Furthermore, additional studies should be performed to evaluate the genetic mechanisms underlying the main traits evaluated in this study . Finally, different statistical methods, such as random regression models, should be tested to better account for the genetic and environmental variations throughout the collection period.eff. Vaginal temperature presented moderate heritabilities, with TV measured at 8:00 and 12:00 h being the most heritable, while TVall had the lowest heritability among the TV traits evaluated. Moderate to high heritabilities were observed for BCS measures, HD, BS, EA, and EL, indicating great potential for genetic progress on these traits. Vaginal temperature and Reff are directly related with the animal’s response to thermal adaptation and are key traits to be used in the selection of more heat-tolerant animals. However, such selection for these traits must be balanced against the ease of measurement, genetic correlations with other economically-important traits , and their contribution to the breeding objectives. Furthermore, the results for the genetic correlations provide evidence of the complex relationships among these traits and confirm the importance of developing a thermotolerance selection sub-index to breed for improved heat tolerance in pigs. Finally, additional investigation is needed, including the evaluation of other statistical models  and studies to unravel the genetic background of thermotolerance traits.The estimates of heritability obtained here demonstrate that genetic progress for thermotolerance can be achieved by including the studied traits in selection schemes. Low (but significant) heritabilities were estimated for SST traits, RR, PS, and RAdditional file 1: Table S1. Genetic correlations  and standard deviation  estimations among physiological indicators of heat stress in lactating sows under heat stress conditions.


When evaluating individuals with the same parent and no phenotype by pedigree best linear unbiased prediction (BLUP), it is difficult to explain carcass grade difference and select individuals because they have the same value in pedigree BLUP (PBLUP). However, single step GBLUP (ssGBLUP), which can estimate the breeding value suitable for the individual by adding genotype, is more accurate than the existing method.The breeding value and accuracy were estimated with pedigree BLUP and ssGBLUP using pedigree and genotype of 408 Hanwoo cattle from 16 families with the same parent among siblings produced by fertilized egg transplantation. A total of 14,225 Hanwoo cattle with pedigree, genotype and phenotype were used as the reference population. PBLUP obtained estimated breeding value (EBV) using the pedigree of the test and reference populations, and ssGBLUP obtained genomic EBV (GEBV) after constructing and H-matrix by integrating the pedigree and genotype of the test and reference populations.For all traits, the accuracy of GEBV using ssGBLUP is 0.18 to 0.20 higher than the accuracy of EBV obtained with PBLUP. Comparison of EBV and GEBV of individuals without phenotype, since the value of EBV is estimated based on expected values of alleles passed down from common ancestors. It does not take Mendelian sampling into consideration, so the EBV of all individuals within the same family is estimated to be the same value. However, GEBV makes estimating true kinship coefficient based on different genotypes of individuals possible, so GEBV that corresponds to each individual is estimated rather than a uniform GEBV for each individual.Since Hanwoo cows bred through embryo transfer have a high possibility of having the same parent, if ssGBLUP after adding genotype is used, estimating true kinship coefficient corresponding to each individual becomes possible, allowing for more accurate estimation of breeding value. In this technique, 11.9 ova can be collected and 3.6 fertilized eggs can be produced annually from one cow, allowing livestock geneticists to secure large amounts of phenotypic information on descendant animals in a short period of time [Genetic improvement of domestic animals has developed from a method where animals with good records in phenotype are selected into genetics evaluation methods that utilize all information on the phenotype and pedigree that a certain animal possesses. Best linear unbiased prediction (BLUP) which calculate estimated breeding value (EBV) by correcting the environmental elements that affect the pedigree and phenotype of offspring and siblings of individual animals in question has had a great impact on genetic improvement of livestock . Korean  of time . However of time . Also, S of time  reportedGenome selection, which has been widely used in the livestock industry, recently has had a positive influence on animal breeding through its advantages such as estimation with higher accuracy than traditional breeding methods . Genome All animal care and treatment procedures were conducted in strict accordance with the Animal Ethics Committee of Gyeongsang National University, Korea, and performed in accordance with the Committee's guidelines and regulations .The test population used in this study consisted of a total of 467 Hanwoo cattle from 18 families, provided with individual identification numbers of Hanwoo produced by embryo transfer from GAST, Gyeongsang national university. Among the families with the same parents, 408 Hanwoo cattles from 16 families were finally selected by excluding less than 5 slaughtered animals and those with abnormal carcass grades. The reference population used for the analysis was provided with pedigree, genotype, and phenotype from the BioGreen 21 program (Molecular breeding Program) of National Institute of Animal Science (NIAS), Rural Development Administration (RDA), South Korea. Through the normality analysis, individuals with a slaughter age of less than 26 months or more than 36 months and individuals with abnormalities in carcass grade were found and removed. Finally, 14,225 head were used for the analysis.The phenotype shared by the two reference population involved following the Livestock Grade Determination Standard Detail #2014-4 posted by the Ministry of Agriculture, Food and Rural Affairs (MAFRA), with carcass traits measured 24 hour of refrigeration after the butchery. Carcass weight (CWT) measured as the sum of left and right frozen body weights, while eye muscle area (EMA) was measured by cutting the area between the left and right thoracic vertebrae and first lumbar perpendicular to the vertebra and measuring the area of the last thoracic vertebrae. Back-fat thickness (BFT) was measured by the area that is 2/3 inwards towards the belly along with the EMA. And marbling score (MS) was visually measured by comparing the degree of fat deposit in the muscles of the EMA measurement area with the standard table (1 = devoid and 9 = abundant).To collect the pedigree to be used in BLUP, the individual identification numbers were searched for pedigree at Korea Animal Improvement Association (KAIA) to establish a pedigree tree through whole backtracking, and renumbering took place to arrange into Animal, Sire and Dam. After that, outliers were corrected using the R Software  suitable−6 Hardy-Weinberg equilibrium, finally using a total of 41,564 SNP markers.To collect the genotype to be used for ssGBLUP, high purity genomic DNA was acquired by extracting it from the hair and blood, then genotype was collected using Hanwoo 50K SNP BeadChip Ver.1 and Bovine 50K SNP BeadChip Ver. 2, Ver. 3 . The acquired large-scale genotype was converted to a form suitable for PLINK1.9  and MiniThe EBV, prediction error variance (PEV) and genetic parameters for each trait were estimated by applying the numerator relationship matrix (NRM) constructed with pedigree in the BLUPF90 program . For theY is the vector for the observed value of economic traits, X is the vector for the fixed effect and β is the vector for the estimated value of the fixed effects. Z is the vector for the random effects of individuals and u is the vector for the estimated value of individuals and e is the vector for random errors. E(y) = Xβ, h2 is heritability, Here, −1, GBLUP uses H−1 inverse matrix of H that is based on pedigree and genotype, to estimate GEBV, and the mixed model equation is as follows:While ssGBLUP and PBLUP used the same fixed effect, GRM was established using genotype and NRM that contain the relationship coefficient between individuals to perform the GEBV estimation. Instead of AHere, H is relational matrix based on pedigree and genotype; A is NRM constructed using pedigree, G is GRM constructed using genotype, α is the additive genetic variance of the individual, with R is the variation matrix on residual effect. Especially, G was established using the following relation formula .Here, Z = M-P, where M is n  X m (number of SNP), which is a matrix when converting homozygote , heterozygote (AB), each into −1,0,1. And P is the frequency (pj) of second allele on the jth locus. Defining row j of matrix p as 2(pj −0.5), p becomes the frequency value of expected genotype that averages value of 0.The PEV obtained from PBLUP and ssGBLUP was used to estimate the accuracy EBV and GEBV for each trait using the equation below.Here, Acc is accuracy of the EBV and GEBV. PEV is the prediction error variance of the EBV and GEBV, and 2, 12.7±4.9 mm, and 6.5±2.0 point in test population, 441.1±47.6 kg, 95.8±11.4 cm2, 13.9±4.2 mm, 5.9±1.8 Point in the reference population. The average value of all traits was high in the test population. Looking at the coefficient of variation of the two populations, BFT and MS were significantly higher, indicating that the two traits had a larger deviation than the other two traits.The phenotype of the test and reference population used for the analysis are CWT, EMA, BFT and MS, and the basic statistics are shown in The estimated genetic parameters according to the analysis method was shown in CWT, EMA, BFT, and MS used in the analysis were multiple traits of animal model according to the analysis method, and the genetic and phenotypic correlation between each trait were estimated and shown in 2, number 9, which had a BFT of 1.88 mm, number 17, which had a MS of 1.67 point, and other families had different estimated value. The difference between EBVs obtained from both analysis methods was that standard deviation of EBV did not exist, but this alone is not enough to explain the various phenotype as all individuals within the same family are estimated to have the same EBV. On the other hand, standard deviation exists for GEBV, so it can be assumed that different GEBV have been estimated for each individual. If the carcass grade and GEBV for each parent combination is looked into, the family with the best grade in CWT and EMA is number 11. The family with the best grade for BFT is number 9. And MS is number 15 for GEBV, and number 3 for overall highest carcass grade MS. Families with high scores in all aspects all used Sire 3, family numbers 1 and 8 had half-sib relationships by Donor for Dam 1, and family numbers 10 and 14 for Dam 9, and the GEBV and carcass grades of both populations turned out to be similar. On the other hand, family numbers 2 and 7 had half-sib relationships from their father in Sire 2, and they had major differences in carcass grade of CWT and EMA, and in Sire 1, families number 1 and 3 had different carcass grades in all categories apart from BFT. Through these data, it was found that there were differences in carcass grade by combination of each father and mother.The EBV and GEBV, average value and standard deviation of carcass grade estimated according to analysis method have been categorized by family in The test population is divided up into low relatedness population (RT1), a half-sib family where only one side of the parents is the same (RT2), and a full-sib family that has the same parents (RT3) and the relationship graph used to analyze the kinship coefficient distribution of each population is shown as a boxplot and histogram in 2, 14.41±4.87 mm, 6.10± 1.84 point, respectfully. Lee et al [2, 13.30±5.14 mm, 5.66±1.88 point, respectfully, which matched the numbers of the two earlier studies. The average carcass grade of the top 10% of market price among Hawnoo in 2021 was 449.9±56.1 kg, 106.6±13.10 cm2, 12.40±4.1 mm, and 8.40±0.9 point, which is similar to the data of the test population apart from MS [In order to increase the effectiveness of livestock genetic improvement, it is important to form and select population s with good carcass grades. Lee et al  collecteee et al  collecte from MS . The tes from MS  also shoIn the preceding research study done on genetic parameters estimated using both analysis methods, the research of Cesarani et al  comparesoffspring = 1/2 TBVsire + 1/2 TBVdam + Mendelian sampling term, which means that an offspring’s genetic ability is determined by adding the genetic abilities passed on from the parents and the Mendelian sampling effect. Here, Mendelian sampling measures the genetic variability between siblings, relates to the effect of mixing up genes randomly passed on from both parents, and phenotypic characteristic, record, and other traits of each individual differ due to linkage disequilibrium and heterozygote fluctuation [Differences in genetic parameters estimated using PBLUP and ssGBLUP are based on fundamental differences in genetic models formed upon coefficient matrices used in each analysis method . The simctuation . Therefoctuation . On the ctuation ,27,28.The accuracy of GEBV estimated using ssGLBUP  shows thAfter, In order to increase the accuracy of genetic ability evaluation of the test population, measures such as a reference population that has many relationship connections and has various genetic mutations, density of SNP Chip, and accurate pedigree tree is necessary. Among these factors, the most influential one is the number of offspring from the individual to be tested. The higher the number of offspring, the lower the deviation that estimates the genetic mutation of individuals to be tested, which means lower standard deviation and higher accuracy . If the This research study uses pedigree and genotype of full-sib families, PBLUP, ssGBLUP to estimate accuracy and breeding value, and using the results estimated from both analysis methods, an effective method of analysis for full-sib families has been found. The accuracy of GEBV using ssGBLUP is 0.18 to 0.20 higher than the accuracy of EBV obtained with PBLUP. Since the value of EBV is estimated based on expected values of alleles passed down from common ancestors. It does not take Mendelian sampling into consideration, so the EBV of all individuals within the same family is estimated to be the same value, but GEBV makes estimating true kinship coefficient based on different genotypes of individuals possible, so GEBV that corresponds to each individual is estimated rather than a uniform GEBV for each individual. Since Hanwoo cows bred through embryo transfer have a high possibility of having the same parent, if ssGBLUP which estimates GEBV after adding genotype is used, estimating true kinship coefficient corresponding to each individual becomes possible, allowing for accurate estimation of breeding value. Also, if embryo transplant which shows high efficiency at a short period of time in terms of productivity and genetic improvement is applied to individuals with high carcass grades, the effectiveness of genetic improvement can be maximized through high selection intensity.