The recent emergence of multi-sample multi-condition single-cell multi-cohort studies allows researchers to investigate different cell states. The effective integration of multiple large-cohort studies promises biological insights into cells under different conditions that individual studies cannot provide. Here, we present scMerge2, a scalable algorithm that allows data integration of atlas-scale multi-sample multi-condition single-cell studies. We have generalized scMerge2 to enable the merging of millions of cells from single-cell studies generated by various single-cell technologies. Using a large COVID-19 data collection with over five million cells from 1000+ individuals, we demonstrate that scMerge2 enables multi-sample multi-condition scRNA-seq data integration from multiple cohorts and reveals signatures derived from cell-type expression that are more accurate in discriminating disease progression. Further, we demonstrate that scMerge2 can remove dataset variability in CyTOF, imaging mass cytometry and CITE-seq experiments, demonstrating its applicability to a broad spectrum of single-cell profiling technologies. Recent advances in multi-condition single-cell multi-cohort studies enable exploration of diverse cell states. Here, authors present scMerge2, an algorithm that allows integration of a large COVID-19 data collection with over five million cells to uncover distinct signatures of disease progression. This has enabled the discovery of cell-type-specific knowledge and has transformed our understanding of biological systems. This myriad of single-cell data has prompted the recent creation of data atlases that collate single-cell omics data from multiple studies. Examples of large-scale atlases containing over two millions cells are the Human Cell Atlas which aims to map every cell type in the human body4; atlas of gene expression and chromatin accessibility of 4 million human fetal cells across 15 organs6; the Human Tumour Atlas Network7 and DISCO8, which provides integrated human single-cell omics data across 107 tissues/cell lines/organoids and 158 diseases. These atlases serve as valuable references for the exploration of healthy and diseased cells.Technological advances of large-scale single-cell profiling of genes and proteins, such as single-cell RNA-seq (scRNA-seq)9 and to predict treatment response to cancer10. Such data and studies are expected to rise in the coming years11 in the continuing quest to improve human health. This expected increase necessitates the effective access and joint interpretation of multiple datasets to unleash the power of meta-analysis at single-cell resolution.As single-cell technologies advance, there are an increasing number of studies around the globe that perform multi-condition and multi-sample large-cohort single-cell profiling to examine persisting questions associated with human health. These datasets enable researchers to delve into biological insights of cells under multiple treatment conditions across multiple individuals. For example, to investigate the cell-type-specific cellular mechanism underlying COVID-19 disease severity12 began to investigate atlas-scale integration. Luecken and colleagues investigated 16 popular data integration technologies on 13 data integration tasks with up to 1 million cells. While significant progress has been achieved in batch correction and data integration over the years (including our research), the increasing scale of cohort sizes and the number of related studies for integration has introduced additional scalability challenges. The new challenge for atlas-scale integration is to have a scalable algorithm that can handle a large number of studies, consisting a large collection samples (thousands) and millions of cells. With the exception of Seurat13, SAUCIE14 and Scanorama15, several of these rapid procedures  focus on extracting the joint embedding and do not return adjusted gene expression matrices. With the growing need for sample level analysis, the lack of adjusted expression matrices restricts the utilisation of such integrative results and diminishes their potency and generalizability. As a result, the next generation of atlas-scale integration algorithms should be capable of integrating a large number of studies and producing consensus cell type maps as well as adjusted expression matrix for further downstream analysis. In particular, these methods need to overcome the computational challenge of integrating over a million cells and create adjusted gene expression matrix for all genes for downstream analysis.Last year, benchmarking studiesTo this end, we present scMerge2, a scalable, high-capacity algorithm that allows data integration of atlas-scale multi-sample multi-condition single-cell studies. We achieve this through three key innovations in (i) hierarchical integration to capture both local and global variation between studies; (ii) pseudo-bulk construction to ensure computational scalability; and (iii) pseudo-replication inside each condition to capture signals from multiple conditions. Our scMerge2 algorithm is able to integrate many millions of cells from single-cell studies generated from various single-cell technologies, including scRNA-seq, CyTOF, and imaging mass cytometry. Leveraging pseudo-bulk to perform factor analysis of stably expressed genes and pseudoreplicates, scMerge2 is able to integrate five million cells from a large COVID-19 data collection with over 1000 samples from 20 studies globally within a day. We further demonstrate that the integration using scMerge2 improves the performance of discriminating distinct cell states in COVID-19 patients with varying degrees of severity and facilitates diverse single-cell downstream analyses.22. The integration approach supports diverse integration settings, enabling cross-batch, cross-dataset, and cross-species discoveries. In particular, the semi-supervised aspect of scMerge allows incorporation of prior knowledge facilitated by experimental design.scMerge2 provides a scalable data integration method for the rapid growth of multi-sample, multi-condition single-cell studies. This new extension of scMerge is specifically designed to address unwanted intra- and inter-dataset variation that can overshadow true biological signals between conditions. In our previous study, we introduced scMerge, an algorithm that integrates multiple single-cell RNA-seq data by factor analysis of stably expressed genes and pseudo-replicates across datasets and enhances biological discovery, including inferring cell development trajectoriesWith the rapid emergence of multi-sample multi-condition single-cell studies and the increased number of datasets for integration, our proposed scMerge2 addresses challenges associated with scalability of cells and studies as well as producing analytically ready data (i.e. adjusted expression matrix). This is achieved via three key innovations as illustrated in Fig. We demonstrate the performance of scMerge2 in removing multi-level unwanted variation of multiple scRNA-seq datasets from three aspects. Firstly, to illustrate the effectiveness of the hierarchical integration strategy, we applied scMerge2 to a 200k subset of cells from two COVID-19 studies (Liu and Stephenson) that contain three cohorts/batches within each dataset (See “Methods” for details). We compared the performance of two different scMerge2 settings: scMerge2-h, where we performed intra-study correction before inter-study correction; and scMerge2, where we integrated two datasets (6 batches) in one go. We find that integrating the two studies in a hierarchical manner improves the performance of data integration, especially in terms of revealing the cell type signals k genes or 87 proteins). We find that among all the methods that are able to return the adjusted gene expression matrix, scMerge2 is the most efficient in terms of computational time in all comparisons , which in total have ~1M cells. We benchmarked the scalability of the integration methods by varying the number of cells and the number of features (~1723) through a simulation study. We generated synthetic single-cell datasets with two batches and multiple samples from two conditions using a simulation framework that extended from scDesign3 model24, with known ground truth DS genes  and proportions of DS genes (5% and 10%), we find that scMerge2 substantially outperforms the other two data integration methods that also return adjusted matrices in detecting DS genes  analysis by ref. nes Fig. –S8, illuFinally, we illustrate the robustness of scMerge2 by varying the key tuning parameters of the algorithm, including the number of unwanted variation factors, the number of pseudo-bulk, the ways of pseudo-bulk construction and the number of nearest neighbours. As shown in Fig. To demonstrate the scalability of scMerge2 in integrating multi-sample multi-condition single-cell data, we performed scMerge2 on a COVID-19 data collection of consisting of ~5m cells from 1298 samples  PBMC samples from 20 studies worldwide (See Methods). We considered the cell type annotation refined by scClassify as pseudo-replicates information. We also used a hierarchical integration strategy, where we first performed integration of different cohorts within one study respectively  and also two studies with distinguished sequencing depth, followed by the integration of 13 studies with small number of cells . The pathways enriched with disease severity include hallmark TNFnse Fig. c and arense Fig. c, d, the 28 Fig. d. While One of the key strengths of scMerge2 is its generalizability to data from multiple biotechnology platforms. We illustrate that scMerge2 is generalizable to other single cell modalities including spatially resolved modality and multi-modalities. We start by illustrating that our algorithm is directly applicable to other single-cell single-modal data, using two mass cytometry time-of-flight (CyTOF) datasets as an example. The two datasets (COMBAT (CyTOF) and Geanon (CyTOF)) contain more than 11 million cells in total collected from healthy controls, COVID-19 and sepsis patients, with 18 immune cell populations and activation states. The UMAP plots constructed after integration Fig. a reveal 29, followed by clustering using FlowSOM30, with the number of clusters set equal to the manually annotated cell types in the original study. We find that compared to the original data, the scMerge2 adjusted matrix provides better clustering results that are more consistent with the manual cell type annotation  datasetion Fig. c, with Aion Fig. d. For exion Fig. e.32. As an example, we used j-UMAP that generates joint visualisation of the adjusted multi-modal data32, which further confirms the effective integration of the six batches from the two CITE-seq datasets  data that concurrently measure RNA and cell-surface proteins of the same cell. In this case, we can remove the unwanted variation for each of the two modalities separately using scMerge2. We first examined the quality of data integration using two CITE-seq datasets with six batches and 87 common surface proteins measured  of the adjusted matrix for all 5 million cells in the COVID-19 data sets as well as outputting the adjusted matrix by batches, allowing users to effectively balance computational burden with specific downstream analytical strategies.The type of output extracted from atlas-scale data integration has an important impact on the analytical question of interest. To date, there are three standard types of output from recent atlas-scale data integration (defined as over millions of cells). These are (i) an adjusted gene expression matrix, (ii) a low-dimensional projection of the data, known in machine learning as “embeddings”; and (iii) a unified graph representation. Various methodological approaches may provide one or more of these types of outputs. In general, there are a number of existing approaches that use modern deep learning-based algorithms to achieve fast, atlas-scale integration. Given that single-cell data are ultra sparse high-dimensional datasets, “embeddings” are a natural output since they are effective for joint data visualisation and reduce memory load. However, an embedding output by itself increases interpretability challenges since a low-dimensional representation does not naturally lend itself to the development of interpretable features such as cell–cell interactions or pathway information, which is crucial for downstream case-control studies or multi-treatment analysis. One step towards achieving a balance between generating adjusted expression matrices and appropriate memory usage is to enable selective adjusted output. For example, scMerge2 enables the extraction of a subset of genes . Such analyses are difficult to perform in individual studies due to the limited sample sizes. This challenge can be overcome by merging several datasets.Recent technological advancements substantially extend beyond scRNA-seq, enabling other data modalities  to be profiled in individual cells providing a more comprehensive molecular view of the cellular regulation. For the datasets with multi-modal profiles measured for the same cell (paired data), such as CITE-seq and ASAP-seq, scMerge2 can be applied to integrate data from different batches by either considering each each modality as a separated matrix, or concatenating the data into a single matrix. Currently, the integration illustrated in this paper was done within each modality. In the future, we can incorporate the multi-modal information to better identify the pseudo-replicates of the paired data as well as utilise the higher-order relationship of features to improve the integration performance.In summary, scMerge2 enables atlas-scale integrative analysis of large collections of single-cell data. As the availability of public multi-sample multi-conditional single-cell studies continues to surge, scMerge2 demonstrates its ability to integrate over 5 million cells for further downstream analysis, thereby enabling effective downstream meta-analysis. Notability, when compared to the raw log-normalised data from the outset, we demonstrated that scMerge2 offers a significant improvement in the prediction accuracy rate across all of the main cell types. The merge of large collections of scRNA-seq datasets from several cohorts further enables identification of distinct cell states in COVID-19 patients whose symptoms are of varying degrees of severity. Finally, scMerge2 has the ability to combine the data from millions of cells obtained from a variety of single-cell technologies, such as CITE-seq, CyTOF, and image mass cytometry.getTopHVGs in the scran R Package, using batch information as block information. For data like CyTOF and ADT from CITE-seq data, this step will be skipped and all features will be used in the next step. Next, within each batch, instead of using k-means clustering as in the previous version, we construct a shared nearest neighbour graph on the gene expression of the HVGs, with a default number of neighbours of 10, followed by louvain clustering. This therefore relieves the need of predefining the number of clusters that is required in our previous version.Clustering-based grouping: This is performed by default when no cell type label is used as input. Firstly, the top 2000 highly variables genes (HVG) are selected using 34 and SingleR35) can also be used and the reference dataset can be external datasets with similar cell types to the data to be integrated. This approach unifies cell type annotation across all datasets and eliminates the need for clustering and cell type annotation after data integration. It is noted that this approach is used in the COVID-19 case study to integrate the data collection of 20 datasets.Reference-based grouping: This refers to the use of supervised cell type classification to predict or annotate the cell types using one or more reference datasets. This ensures the cell-type annotations are consistent among datasets. Cell type classification algorithms  assign the cells based on library size; (2) randomly assign the cells into k subsets. Then we gene-wisely take the sum of the counts for each subset and generate the counts data following a negative binomial distribution. While the pseudobulk matrix generated by this strategy is able to maintain the gene mean-variance relationship36, we find that this approach does not improve the quality of data integration in scMerge2 , scMerge2 allows the MNC graph to be constructed within each condition to preserve the biological variation. Note that this strategy can only be used when the batches to be merged have at least one common condition and can only be performed in the condition with multiple batches.Replicates are considered as the samples with similar biological variation across batches. Construction of pseudo-replicates is one of the key steps in scMerge which later are utilised to estimate the unwanted variation from the data. In scMerge, we proposed a five-step procedure to identify pseudo-replicates by clustering on a mutual nearest cluster (MNC) graph, where each node of the MNC graph indicates a group of cells in a batch. scMerge2 follows similar steps as the previous version, but with two major improvements:Zcg be the standardized data, where c = 1, …, C, with C indicates the number of cells from all batches/datasets in total; g = 1, …, G, with G indicates the number of genes. Following the same annotation in scMerge, we formulate ZC×G using RUVIII model asX denotes the matrix of observed factors of interest; p denotes the number of factors of interest; W denotes the matrix of unobserved factors of unwanted variation; α denotes the coefficient of W; k denotes the number of unwanted factors, which is unknown ; ϵ denotes the random error. Following the RUVIII model estimation proposed in refs. 37, the model removes the unwanted variation from ZC×G. In summary, it follows the three steps:α via the first k right singular vectors of Singular Value Decomposition (SVD) on RMZ, where M ∈ RC×N, N indicates the number of types of pseudo-replicates;Step i: estimate W by α, which columns include only the genes that belongs to single-cell stably expressed genes (SEG) with number of genes as Gs ;Step ii: estimate Step iii: adjust the matrix by subtracting the estimated unwanted variation component:The underlying model of scMerge2 is the fastRUVIII model that takes the gene-wise standardized gene expression matrix that is log-transformed and cosine normalised as input. Let α. Instead, we can subsample the data or construct cell-type-specific pseudo-bulk which are informative enough to approximate the full single-cell matrix to reduce the computational burden in estimation of α. Let Z; the row now indicates the number of pseudo-bulk, with dimension Cb. We then construct pseudo-replicates based on the pseudo-bulk matrix Zb to obtain the replicate matrix Pseudo-replicates identification across batches in scMerge2 for more details). We estimate k right singular vectors of SVD on Z to estimate W and adjusted SVD is a computationally intensive algorithm, especially for large matrices like single-cell data. We argue that for Step 1, we do not need the full single-cell data to estimate 33.When we integrate data from different studies, the unwanted variation can come from multiple levels, such as batch effect of samples within each study but also between studies. In this case, a hierarchical integration strategy would be useful to first adjust intra-study unwanted variation effect, and then perform the inter-study data integration. On the other hand, when we integrate a large number of studies, such as the COVID-19 data collection in this paper, starting from correcting the data of a smaller set of studies can be a more efficient way to estimate the parameters of the model to harmonise the datascMerge2 allows users to input a hierarchical tree strategy to perform the data adjustment in a multi-level manner. This data strategy can take a flexible multi-level merging structure. For each level, it can contain multiple collections of batches and data correction can be performed within each collection respectively, with a user-defined batch label. For each collection, it can consist of multiple batches. The data adjusted on the current level will be used as input on the next level.For the COVID-19 200k data collection, we first integrated the the 3 batches within each dataset before integrating the two datasets. For the COVID-19 scRNA-seq data collection, we first performed the adjustment on four datasets that have multiple cohorts  to correct the intra-study unwated variation (where the cohort label is used as batch label) as well as between the two datasets that have very different sequencing depth . Next, we performed the adjustment of the 13 datasets with <200,000 cells. We finally integrated all the 20 studies together, where the study label is used as batch label.logNormCount function from scater39 R package. To unify the cell types from different studies, we performed scClassify to reannotate the cell types based on a 3-level hierarchical cell type tree34, using three distinct reference datasets that were either generated from whole blood (Wilk) or generated by CITE-seq protocol that contains multi-level annotations (Liu and Stephenson).We collected 20 public COVID-19 PBMC and whole blood scRNA-seq datasets (Supplementary Table kRUV = 10 (kRUV denotes the number of unwanted variation) and then performed the integration across two datasets, with kRUV = 10.To benchmark scMerge2 with other methods, we subset 200k cells from the two COVID-19 studies (Liu and Stephenson) as a benchmarking dataset that with 17,446 genes, 87 proteins and 184 samples from 3 conditions  to assess the concordance performance of the adjusted gene expression matrix after data integration. Both of these two studies have three batches within the studies, which allows us to evaluate the hierarchical merging strategy in scMerge2 , where we first integrated the three batches within each batch, with logNormCount function from scater39. In scMerge2, we used all features as negative controls and used kRUV = 3 in both levels in scMerge2-h.The raw antibody derived tag (ADT) counts matrix of each dataset is size-factor standardized and log-transformed using the To evaluate the robustness of the parameters in scMerge2, we further created a smaller subset of data, which is derived from selecting the cells from moderate/mild patients of the Stephenson data from the COVID-19 200k data. The selected subset has 66,967 cells from 58 samples and 17,446 genes where the aim is to integrate three different batches in the Stephenson data.40  and zenodo 10.5281/zenodo.6120249 for data from granulocyte depleted whole blood in COMBAT study41 , which both contain the expression matrix and cell type annotations. To combine the two studies, we manually unified antibody names and the cell type annotations to 18 cell types. The expression matrices were then used as input for scMerge2. Noted that we used all features as negative controls in scMerge2.Two public COVID-19 PBMC CyTOF datasets  and 23 samples from two conditions  as training data. From each sample, we randomly subsampled 400 cells. Only genes that were in the top 2000 highly variable genes and expressed in more than 2% of the cells were included. We further excluded any genes that were originally considered as differential expressed . This resulted in the training data with 9200 cells and 1196 genes from 23 samples. Our simulation framework includes three main steps.Simulation framework. We adopted a simulation framework to generate single-cell multi-condition and multi-sample data with batch effect based on scDesign3fit_marginal in scDesign3 that fits the marginal distribution of each gene using a negative binomial distribution with the mu formula  ~ cell type + sample ID + condition and the sigma formula  ~ 1. Then we used a vine copula to estimate the gene correlation from the real training data.Step 1: Construct a null dataset with no differentially expressed genes by first permuting the condition labels in the training data. We then estimate both cell-type and sample variation in the data using the function Step 2: Introduce the batch effect to the simulated data. Assuming all genes are affected by the batch variation, we drew a vector with length equal to the number of genes from a log-normal distribution with mean p% of genes to be differentially expressed between two conditions . The log fold changes (logFC) vector is simulated from a log-normal distribution, with the mean μlfc and the standard deviation σlfc. In our evaluation setting, we consider a range of logFC values from μlfc = 1.1 to 2 in 0.1 increment and σlfc = 0.43. The direction of the regulation is randomly assigned to each DS genes using a binomial distribution with probability 0.5.Step 3: Introduce the ground truth differential state genes to the simulated data. For each cell type, we randomly select Step 1, the simulated single-cell data is generated from the negative binomial distribution using strategies implemented in simu_new of scDesign3. For each value of logFC, we simulated 18,400 cells , with 5% or 10% differential states genes within each cell types.Lastly, with the fold change of both batch effect and condition effect combined with the parameters estimated in G × C, with S samples and T cell types, we took the gene-wise average of each sample within each cell type, resulting in a G × S matrix for each cell type. We then performed a differential state analysis using the limma-trend algorithm25 on the cell-type specific sample-wise aggregated data using the default parameters.Evaluation metrics and settings—Differential states analysis. To assess the impact of data integration on downstream analytics, we considered the performance of the differential states analysis results on the simulated data. Our evaluation is based on three metrics; false discovery rate (FDR), true positive rate (TPR) and F1 scores. For each log-transformed simulated matrix with dimension Signal to noise ratio: We used ARI and ASW  to evaluate the concordance of clustering results with respect to the cell type labels and the datasets. A desirable data integration method will show a high concordance between the clustering result and known cell type information  and a low concordance between the clustering results and known datasets information (noise refers to batch effect).F1 score.Severity prediction: We aggregated cell-type-specific average expression of each sample to a gene by sample matrix for each cell type. We then used each cell-type specific matrix to predict the sample condition  using support vector machine (SVM) with radial basis function kernel. The prediction performance was evaluated using repeated fivefold cross validation with 20 repeats. We evaluate the prediction performance using 32, where we first performed PCA within each modality, and then j-UMAP was performed to obtain the joint UMAP embeddings of the two modalities.Visualisation plot: For scRNA-seq data, we used Uniform Manifold Approximation and Projection (UMAP) to visualise and evaluate the results of the adjusted expression matrix. For CITE-seq case study, we used j-UMAP to jointly visualise the two modalitiesEvaluation setting for scRNA-seq and CITE-seq data collection.29 and removing all undefined cell types (i.e. cells having cell type annotation as “nan”). Next, considering sample labels as batch information, we applied scMerge2 with settings kRUV = 2, kpseudoBulk = 5, kcelltype = 20, using all markers as negative control genes and highly variable genes. Thirdly, unsupervised clustering was performed on both the unnormalised and scMerge2 normalised datasets using the FlowSOM30 algorithm with 17 clusters. The Adjusted Rand Index (ARI) was used to compared the concordance between this unsupervised clustering with the manually curated cell types in the original manuscript29. The results are visualised using heatmaps showing the average marker abundance in the cell types. Average marker abundance were generated after scaling the marker expression by computing the ratio of the mean of each marker and its standard deviation.Evaluation on IMC data collection. We applied scMerge2 to perform data integration of the 23 samples. This is achieve by first filtering and selecting the data using the 38 markers specified in the original publicationNumber of pseudobulk constructed within each group: 10, 20, 30, 40 and 50Number of neighbours in SNN graph: 5, 10, 15, 20, 25 and 30Ways of pseudobulk construction: Default, Pool-Divide, Pool-Divide (Random)Number of factors of unwanted variation to be removed: 10, 15, 20, 25 and 30Sensitivity analysis of scMerge2. We examined the robustness of the following parameters in scMerge2: the number of pseudobulk constructed; the number of neighbours in SNN graph; the pseudobulk construction strategy and the number of unwanted variation. We performed our sensitivity analysis on the COVID-19 60k data on a number of settings for each of the four parameters as below:Evaluation metrics. We compared against benchmarking methods described in the Section Benchmarking methods.For each setting, we repeat the analysis 10 times with a different seed and assess the concordance performance of the signal to noise ratio using ASW and ARI as evaluation metrics as describe in the Section Adjusted Rand Index (ARI) - Clustering analysis: We used ARI to quantify the concordance of the clustering results with respect to the cell type (ARI (cell type)) and batch labels (ARI (batch)). The clustering results for all methods were derived from first building a shared nearest neighbour from the batch corrected embeddings with a default number of neighbours of 10, followed by louvain clustering. For scMerge2, the batch corrected embeddings were derived from the top 20 PCs of the adjusted gene expression matrix.Average silhouette width (ASW) - Embedding visualisation: We calculated the average of silhouette coefficients for each cell (ASW) by considering two different groupings: cell type (ASW (cell type)) and batch label (ASW (batch)), based on the Euclidean distance obtained from the UMAP embeddings generated from the batch corrected embeddings.R2) for a linear regression model that fitted each of the first 20 principal component with technical variation labels, such as batch, technology and protocol labels. We then calculated the product of the variance explained by each principal component and the corresponding R2. The final PCA score was calculated by summing across the products, which quantify how much the PCs explained the unwanted technical variation.PCA scores: We calculated the coefficient of determination  Seurat. Applying Seurat with canonical correlation analysis set as the reduction method. Version 4.1.1. of the FindIntegrationAnchors was performed with reduction set as “rpca”. Lastly, IntegrateData was performed to obtain the integrated data.(ii) SeuratRPCA. Similar to Seurat (CCA), within each batch, we first found the variable features, with an addition PCA step performed. After integration features were selected, 43. R package batchelor v1.12.3 was used. We ran fastMNN with default parameters to derived both the batch corrected embeddings and adjusted expression matrix.(iii) fastMNN. This is a fast version of the mutual nearest neighbours (MNN) methodrliger v1.0.044 was used. Online integrative nonnegative matrix factorization was performed to obtain the batch corrected embedding following the tutorial , where we first ran selectGenes to select the features, scaleNotCenter to scale the features, and online_iNMF with miniBatch_size = 5000 and max.epochs = 5.(iv) Liger. R package Harmony v0.1.018 was used. The PCA space returned by runPCA of R package scater was used as input, and then HarmonyMatrix was performed with do_pca = FALSE to retain the batch corrected embedding.(v) Harmony. R package scvi v0.16.119 was used. We used the following settings to perform the data integration using model SCVI: the number of layers as 2; the number of latent variables as 30 and the gene likelihood as negative binomial.(vi) scVI. Python package scanorama v1.7.315 was used. We ran correct_scanpy to perform the data integration and gene expression adjustment.(vii) Scanorama. Python package We used two of the COVID CITE-seq dataset (Liu and Stephenson) to evaluate the computational efficiency of the integration methods by varying the number of cells from 5k to 1M in four scenarios: (1) scRNA-seq: 17k genes with 2 batches; (2) scRNA-seq: 17k genes with 6 batches; (3) ADT: 87 proteins with two batches; (4) ADT: 87 proteins with six batches. The running time was measured using 1 core with the maximum number of threads of OpenMP library that can be used by a parallel region is set to 1. The peak resident set size, which is the highest amount of memory used by a process, was recorded to measure the memory usage. All methods were run using a research server equipped with dual Intel(R) Xeon(R) Gold 6148 Processor with 40 cores and 768 GB of memory and we set the running time limit as 10 hours. It is noted that for both Seurat and scMerge2 we evaluated two settings: one that adjusts the expression matrix for full gene matrix and another that only adjusts the top 2000 highly variable genes.26. The top 30 PCs derived from the adjusted expression data were used as input for the algorithm to calculate the DA scores. A range of k values from 50 to 500 was used for the calculation of DA score vector with kNN. We define salient differential abundance (DA) cells as cells with absolute abundance scores >0.8.Differential abundance (DA) analysis was performed on the cells from mild/moderate and severe/critical samples using DA-seqlimma25. We then ranked the genes based on the test statistics. The preranked based gene set enrichment analysis (GSEA) of the selected pathways that are related COVID-19 disease mechanism28  of each sample to a gene by sample matrix for each cell type. Next, we model the aggregated cell-type-specific abundance values across using a linear model with severity and the days since symptom onset as covariates. We account for sample level variability using the limma-trend implementation in the R package All analysis was done in R version 4.1.2. No statistical method was used to predetermine sample size. No dataset listed in Supplementary Table Further information on research design is available in the Supplementary InformationPeer Review FileReporting Summary


Multi-cancer early detection remains a key challenge in cell-free DNA (cfDNA)-based liquid biopsy. Here, we perform cfDNA whole-genome sequencing to generate two test datasets covering 2125 patient samples of 9 cancer types and 1241 normal control samples, and also a reference dataset for background variant filtering based on 20,529 low-depth healthy samples. An external cfDNA dataset consisting of 208 cancer and 214 normal control samples is used for additional evaluation. Accuracy for cancer detection and tissue-of-origin localization is achieved using our algorithm, which incorporates cancer type-specific profiles of mutation distribution and chromatin organization in tumor tissues as model references. Our integrative model detects early-stage cancers, including those of pancreatic origin, with high sensitivity that is comparable to that of late-stage detection. Model interpretation reveals the contribution of cancer type-specific genomic and epigenomic features. Our methodologies may lay the groundwork for accurate cfDNA-based cancer diagnosis, especially at early stages. Despite advances in ctDNA cancer detection, early detection remains difficult. Here, the authors utilise whole genome sequencing of 2,125 patient samples to create a model for early cancer and tissue of origin detection. Circulating tumor DNA (ctDNA) reflects tumor-specific genetic and epigenetic alterations7. Also, ctDNA fragments are physically shorter than normal cfDNA fragments8. Hence, multiple approaches have been applied to detect cancer using these specific characteristics of ctDNA. In addition to targeted approaches relying on deep sequencing of recurrent mutations2, genome-wide methods have also been developed on the basis of DNA methylation patterns7 and genomic fragmentation patterns coupled with copy number variations (CNVs)9 or with chromatin signatures10.Noninvasive screening by cell-free DNA (cfDNA) holds great promise for multi-cancer early detection11 suggested that ultrasensitive monitoring of minimal residual diseases is possible by taking full advantage of the cumulative signal of a large number of mutations. However, this approach only tracks the initial mutation profile of the patient’s tumor tissue, instead of identifying mutations de novo. To date, cfDNA WGS has not been attempted for de novo cancer detection, mainly because of inaccurate variant calling and filtering from cfDNA.Whole-genome sequencing (WGS) has been found to be more sensitive than targeted deep sequencing in detecting low-burden diseases. Zviran et al.12 and Nguyen et al.13 demonstrated that the distribution of passenger mutations, calculated as regional mutation density, serve as a powerful feature in predicting the origin of cancer. In a recent proof-of-concept study, Wan et al.14 utilized mutation signatures derived from cfDNA variants for the purpose of cancer detection. As pointed out in this pilot study, the potential of using cfDNA mutation signatures for de novo cancer detection has to be tested in a much larger cohort of samples. Based on these studies, we hypothesize that local mutation density (LMD) and mutation signatures can make the most of cfDNA WGS data. Specifically, we attempt to use cfDNA WGS for multi-cancer detection with the ‘genome model’ that integrates large-scale reference cfDNA data from our healthy cohort and tumor tissue mutation data from the PCAWG project15. In doing so, we anticipate that tumor-derived mutations can be captured by the genome model when coupled with systematic cfDNA variant filtering and large-scale reference database.Unlike targeted deep sequencing of driver mutations, de novo mutation calling from cfDNA can be of low confidence especially without matched normal control. In this work, to compensate for the lack of matched control, we leverage reference cfDNA sequencing data from a large healthy cohort in the process of filtering out germline mutations and various types of artifacts. This enables us to capitalize on information embedded in a large amount of passenger mutations. Jiao et al.16. Such signatures from promoter NDRs were used to infer gene expression programs of the tissues of origin17 or estimate ctDNA burden18. However, this epigenomic feature has not been explored across the whole genome beyond promoter regions or used for multi-cancer detection by cfDNA. In this work, we attempt to use cfDNA WGS for multi-cancer detection with the ‘epigenome model’ that integrates pan-cancer whole-genome chromatin profiles based on the Assay for Transposase-Accessible Chromatin using sequencing (ATAC-seq)19. To this end, we construct tissue-specific NDR profiles based on ATAC-seq data for TCGA samples of various tumor types.Genomic locality is expected for not only variant density but also for ctDNA burden. cfDNA fragments from nucleosome-depleted regions (NDRs) are more frequently degraded in the blood than nucleosome-protected DNA, resulting in nucleosome footprints that reflect tissue type-specific chromatin architecture20. In our epigenome model, the V-plot is used to visualize the density of fragment length as a function of the location of fragment centers. Convolutional neural networks (CNNs) are used to process the V-plot profiles as three-dimensional images.Previous cfDNA studies have focused on the distribution of read fragments falling across NDRs. In this work, we hypothesize that the sensitivity of cancer detection can be improved by considering fragment length and read distribution simultaneously. In order to incorporate fragment length together with read distribution as a prediction feature, we employ the V-plot that is widely used for the analysis of nucleosome positioning based on ATAC-seq data9. The DELFI algorithm along with other existing methods are used to evaluate our models. We show how our algorithm enables accurate early cancer detection and tissue-of-origin localization by incorporating cancer type-specific profiles of mutation distribution and chromatin organization in tumor tissues as model references.Here, we test the genome model and epigenome model by using cfDNA WGS data for 2125 patient samples of 9 cancer types and 1,241 normal control samples, totaling 3366 samples , ATAC-seq profiles of major PBMC types were utilized. The peak calling and processing pipeline was applied to identify tissue-specific NDRs We trained and evaluated our algorithms in each training cohort using stratified five-fold cross-validation  rather than by the cohort in both the cancer detection model Fig.  left andWe also examined how the epigenome features contributed to the model in terms of attribution. For the cancer detection epigenome model, we averaged the attribution values along the z axis of the CNN input images  Fig. . When plThe same analysis was performed for the epigenome model for cancer localization. A periodic distribution of high and low attribution values was observed according to the distance from the NDR center, more distinctly with the MGI data Fig.  and rathFinally, the attribution values assigned to each feature of the epigenome model were used to cluster the cfDNA samples across the cohorts. As a result, the samples tend to cluster by the label  rather than by the data source for both cancer detection Fig.  left and24. In most cases, especially in pancreatic cancers, diagnostic tests are performed after symptoms arise, and the proper treatment times are often missed. Solutions for this urgent unmet need of multi-cancer early detection and localization are being actively pursued by using cfDNA-based noninvasive cancer screening1.Early detection of cancers of various types is an important part of cancer medicine since cancer has a better prognosis and survival rate when diagnosed and treated earlier11. Ultrasensitive detection of minimal residual disease was made possible by capitalizing on genome-wide cumulative signals when informed by original tumor profiles11. However, de novo detection of cancer, especially at early stages, remains challenging; the sensitivity of the DELFI algorithm for stage I diseases at 95% specificity was reported to be 73%9 but was even lower when applied to our larger cohorts (below 50%). To exploit the full potential of whole-genomic cfDNA screening, we developed an analytical strategy empowered by a large amount of training and reference data from cfDNA and tissue samples. Integrative modeling that incorporates the knowledge of how tumor genomes and epigenomes leave their footprints in cfDNA shows unprecedented accuracy of cancer detection at a sensitivity of 91.1% and 98.1% (at 95% specificity) for stage I cancers on the MGI and DELFI dataset, respectively.Recent studies have demonstrated the power of screening cfDNA at the whole-genome scaleOur study has several implications. First, we demonstrate how integrating large-scale reference datasets including the genome and epigenome data from tumor tissues and the normal cfDNA data for variant filtering can increases the sensitivity of cancer detection. In particular, this work involves an attempt to analyze cfDNA data based on modeling of cancer type-specific LMD and NDR profiles derived from large-scale public data in tumor tissues. Second, we propose genomic and epigenomic features that are effective for cfDNA-based cancer diagnosis. The genomic features based on LMD and mutation signatures, which proved to be useful in previous tissue-based cancer type classification, were applied to cfDNA analysis in this work. Our epigenome model employed the V-plot for the integrative analysis of fragment density and length in NDRs. Third, as feature modeling is critical in developing our models, we provide comprehensive characterization of contributing features instead of leaving the models as black boxes. Most of previous cfDNA-based prediction models examined the properties of input features, but did not investigate which features were actually learned during model training and thus contributed most to prediction. In this work, we not only determined which features played an important role in model development and performance, but also investigated the relevance of the features to the tumor biology in terms of genetic and epigenetic characteristics of cancer.In terms of further improvement in accuracy, we have observed three potential factors that appear to affect the performance of our models. The fact that the genome model generally performs better than the epigenome model may be attributed to the amount of reference tissue data. Increasing the number of cases for ATAC-seq or other epigenome data may provide a more accurate reference for the chromatin architecture. However, the compelling performance of the epigenome model with pancreatic cancer emphasizes the importance of making the most of tumor biology. Another factor is the data size used for model training. In both cancer detection and localization, cancer types with large sample sizes, such as breast and liver cancers, are diagnosed with the highest accuracy. Data size per sample also matters. The overall trend that better performance is achieved on the MGI data compared with the Illumina data implies that higher read depths contribute to prediction accuracy. Hence, when coupled with integrative modeling based on tumor biology, an increasing amount of reference and training data for cfDNA analysis has the potential to realize ultrasensitive early cancer detection and accurate cancer diagnosis.Plasma samples form healthy individuals and patients with breast, hepatic, lung, ovarian, colorectal, pancreatic, bile duct, esophagus, or head and neck cancer were obtained from GC Genome, Samsung Medical Center, Seoul National University Hospital, Asan Medical Center, National Cancer Center, and Yonsei Cancer Center. All the healthy and patient samples were obtained under Institutional Review Board approved protocols with informed consent from all participants for research use at participating institutions . Plasma samples from healthy individuals were obtained if they had no previous history of cancer and negative routine health screening questionnaire. Clinical information for all samples used in model training and evaluation are listed in Supplementary Data From each sample, 10 mL of whole blood was collected in Cell-Free DNA BCT tubes . The collected blood samples were centrifuged to separate plasma and cellular components, and plasma was processed immediately or within 24 h.For sequencing on an MGI platform, cfDNA was extracted from 0.6 mL plasma and eluted in a final volume of 56 μL, using a plasma circulating DNA Kit  according to the manufacturer’s instructions. Extracted cfDNA was processed for library construction starting with a 2–6 ng input, using the MGIEasy Cell-free DNA Library Prep Kit  according to the manufacturer’s instructions. The concentration of the library was quantified using the dsDNA HS Qubit Assay . The size of the library was determined using a D1000 screentape assay with 2200 Tapestation . The sequencing libraries were pooled, and up to 20 libraries per batch were multiplexed.For sequencing on an Illumina platform, cfDNA was extracted from 0.4 mL plasma applied with an automated KingFisher system  and eluted in a final volume of 22 μL, using an Apostle MiniMax High Efficiency cfDNA Isolation Kit  according to the manufacturer’s instructions. Extracted cfDNA was processed for library construction starting with a 0.1–7 ng input, using the Swift 2 S® Sonic DNA Library Kit  according to the manufacturer’s instructions. The concentration of the library was quantified using the dsDNA HS Qubit Assay . The size of the library was determined using a D1000 screentape assay with 2200 Tapestation . The sequencing libraries were pooled, and up to 240 libraries per batch were multiplexed.25. Additional processing was conducted differently for the genome and epigenome model as described in the “Genome model input processing and training” and “Epigenome model input processing and training” sections.The prepared libraries for the MGI cohort samples were subjected to 100 bp paired end runs on a DNBSEQ-G400 sequencing instrument . The libraries for the Illumina cohort samples were sequenced with 100 bp paired end reads using a NovaSeq 6000 S4 Reagent Kit v1.5 . The fastq files were aligned to the human hg19 genome using BWA-MEM 0.7.5a. Duplicate marking was performed with the aligned bam files using the Genome Analysis Toolkit (GATK)26 with the -clip_R1 10 -clip_R2 10 -length 20 options. The reads were aligned to the human hg19 genome using BWA-MEM 0.7.5a. The aligned bam files were processed using the GATK pipeline of data preprocessing for variant discovery, including duplicate marking, indel realignment, and base quality score recalibration25. Because of the low depth and large sample size of the Illumina dataset, duplicates were removed by adding the —REMOVE_DUPLICATES true option when performing duplicate marking, and the low mapping quality reads were removed using the SAMtools -q60 option, instead of performing base quality score recalibration and indel realignment. The Illumina bam files were merged in a batch size of 1000 samples using the merge function in SAMtools, and the merged bam files were subjected to variant calling. Mutect225 was used to call variants in the tumor-only mode with the -flr2-max-depth 1000 -initial-tumor-lod 0.1 options for the merged Illumina bam files and -max-mnp-distance 0 option for the individual MGI bam files. After variant calling, we combined all of the variants called from the merged bam files to construct the Illumina normal panel, and implemented the GATK pipeline of creating a somatic panel of normals to construct the MGI normal panel. Finally, the Illumina and MGI panels were combined to create a normal cfDNA reference panel.The cfDNA libraries of 20,529 normal reference samples were sequenced at an average of 0.3 × 75 bp single ends using NextSeq 500 High Output Kit v2.5  or 0.6 × 100 bp paired ends on DNBSEQ-G400RS . The fastq files were trimmed using the Trim Galore tool in Cutadapt15 (https://dcc.icgc.org/releases/PCAWG/). All samples listed in the PCAWG exclusion list or in the PCAWG microsatellite instability list were excluded. The remaining 2754 samples were used to identify cancer type-specific LMD high and low regions. The LMD was calculated in 1 Mb running windows across the genome for each sample. Cancer types were defined as the organ of origin. Specific LMD high or low regions were identified using quasi-likelihood F-tests on the trimmed mean of the M-value (TMM)-normalized counts27. High or low LMD regions were selected at an FDR of 0.05 and sorted based on the fold change. For each cancer type, the top 25 regions with mutation enrichment and the top 25 regions with mutation depletion were identified as the cancer type-specific high and low LMD regions, respectively.We obtained the variant call dataset for tumor tissues from the ICGC portal of the PCAWG project25. The GATK data preprocessing pipeline includes duplicate marking, indel realignment, and base quality score recalibration. SAMtools with the -f 2 -F 2308 -q 30 options was used to filter out supplementary, unmapped, and low mapping quality reads, leaving only properly paired reads. Additionally, the SAMtools view option was used to extract canonical chromosomes. For variant calling, we ran VarScan opting for at least one variant supporting read, at least 3× depth at a variant position, and an average base quality of at least 3028. After variant calling, single nucleotide variants were extracted while filtering out those in the blacklisted regions29. The variants present in the cfDNA normal reference panel were removed. Additional germline variants were also removed using the dbSNP, 1000 Genomes, HapMap, ExAC, and gnomAD databases. Additional artifact filtering processes were implemented. First, variants with a variant allele frequency of 100% were eliminated. Second, discordant reads were discarded by checking whether the variant allele was found on only the forward or reverse reads. Third, multi-nucleotide variants were discarded. Finally, variants at the ends of a read were removed considering the possibility of sequencing errors. The read end was defined as 10 bp from the beginning or end of each read.The cfDNA bam files were processed using the GATK data preprocessing pipeline for variant discoveryOn the basis of the filtered variants, the LVD was calculated in 1 Mb running windows across the genome for each sample. The number of variants in each bin was divided by the total number of variants in the sample. After discarding regions in which no variant was found in more than 50% of samples, 2,726 genomics regions were left to be used as features of the genome model. For variant-type features, 6 types of single nucleotide substitutions , 48 types of dinucleotides consisting of each substitution and either the 5′- or 3′-flanking base, and 96 types of trinucleotides consisting of each substitution and both the 5′- or 3′-flanking bases were used. The frequency of each of the 150 variant types was computed by dividing by the total number of variants per sample. In total, 2726 LVD values and 150 variant-type frequencies were used as features for the genome model.30.The genome model was constructed using TensorFlow deep neural networks to train the cancer detection and localization models. The genome model consisted of dense layers, batch normalization, activation, and dropout. For the cancer detection model, the output layer employed the sigmoid function and the loss function based on binary cross-entropy. For the tissue-of-origin localization model, the output layer employed the softmax function and the loss function using categorical cross-entropy. Optimization of the genome model was performed by the AdamW optimizer. Because of the imbalanced number of samples, the initial weight of the output layer was modified by calculating the class weight using ScipyBayesian optimization was used with 200 trials to find the best hyperparameter set to maximize the performance on the validation set. The list of the hyperparameters is provided in Supplementary Data 25, and the duplicate reads were removed using SAMtools. Supplementary and low mapping quality reads were removed using SAMtools with the -f 2 -F 2048 -q 30 options, leaving only properly paired reads. Peak calling was performed for the filtered ATAC-seq reads using HMMRATAC21 with the default options or MACS222 with the -shift −75 -extsize 150 -nomodel -nolambda -call-summits -q 0.05 -B —SPMR options. In total, 10,9360 peaks and 24,937 peaks were called by MACS222 and HMMRATAC21, respectively, in GM12878, and 17,5733 peaks and 35,708 peaks were called by MACS222 and HMMRATAC21, respectively, in K562. To identify differential NDRs between the GM12878 and K562 cell lines, the HMMRATAC21 peaks were intersected to find non-overlapping peaks. A total of 13,132 peaks were called as GM12878-specific NDRs, and 23,515 peaks were called as K562-specific NDRs.The GM12878 ATAC-seq data were downloaded from the GEO database, and the SRA file was converted into the fastq format. The K562 ATAC-seq data were downloaded from the ENCODE portal in the fastq format. The ATAC-seq fastq datasets underwent an adapter trimming process using Trim Galore, and were aligned using BWA with the default options. Duplicate marking was performed using GATK425 and SAMtools was performed as described above. The filtered ChIP-seq and MNase-seq reads were extended to 150 bp. To refine nucleosome positioning, the MNase-seq reads were resized to 75 bp from the midpoint of the 150 bp read.The ChIP-seq and MNase-seq data in the GM12878 and K562 cell lines were downloaded from the ENCODE data portal in bam format. To remove duplicates, supplementary reads, and low mapping quality reads, the same filtering process using GATK421. Samples with less than 30,000 NDRs were excluded, leaving 266 of 410 samples for further analyses. The CESC cancer type was excluded because we were left with only one sample. From the ENCODE project, 17 ATAC-seq bam files for two normal tissues (pancreas and ovary) were downloaded. The NDR peaks were called using HMMRATAC21. Among the 17 samples, 15 samples with more than 30,000 NDRs were left for further analyses. In addition, 26 ATAC-seq bam files for five PBMC types  were downloaded from the GEO database. The bam files were merged according to cell types to compensate for low sequencing depths. After merging the bam files, the NDR peaks were called using HMMRATAC21.The ATAC-seq data for tumor and normal tissues or PBMCs were obtained from TCGA, ENCODE, and GEO databases. From TCGA portal, 410 samples of 23 cancer types  were downloaded. Because TCGA ATAC-seq data were aligned to the hg38 genome assembly, TCGA bam files were converted into the fastq format and realigned to hg19 using BWA-MEM. Filtering duplicates, supplementary reads, and low mapping quality (<30) reads were carried out as tested using the cell line data. Using the filtered ATAC-seq reads, the NDR peaks were called using HMMRATAC27 and subjected to the limma-voom algorithm31 of edgeR for the selection of the 10,000 tissue-specific NDRs with the lowest P value. NDR-wise min-max normalization30 was applied for visualization of the tissue specificity of the selected NDRs by a heatmap.Following the peak processing and filtering pipeline, tissue-specific NDRs were identified by using the 25 filtered samples. The NDR peaks called from each sample were merged, and peaks with the strongest signal were selected among overlapping peaks, resulting in 61,5401 meta-peaks. The ATAC-seq reads that were shorter than 150 bp and overlapped with the midpoint of the meta-peaks were counted to construct a read count matrix. The matrix was normalized using the TMM function of edgeR30 was applied to the vector of individual pixels of the V-plot image for each sample.The process of filtering duplicate, supplementary, low mapping quality, and unpaired cfDNA fragments was performed as tested using the cell line data. Each fragment was cut in half centering on its midpoint in a similar manner that MNase-seq reads are processed for nucleosome positioning analysis. For each tissue type, the cfDNA fragments were aligned to the identified tissue-specific 10,000 NDRs, and a two-dimensional V-plot was constructed to plot fragment size as a function of the distance from the NDRs. The y-axis representing fragment size contained 100 1-bp bins ranging from 100 bp to 200 bp. The x-axis, representing the distance from the NDRs, was made up of 250 4-bp bins covering ±1000 bp from the NDR midpoint. Finally, the two-dimensional V-plot for each tissue type was added to the channel axis to create a three-dimensional  V-plot image as input to the epigenome model. To correct for read depth bias, min-max normalization30.The TensorFlow CNN architecture was employed for the development of the epigenome model. The epigenome model was designed with two convolutional layers and one fully connected layer. On the first convolutional layer, a normalized three-dimensional V-plot image was convoluted with 250 kernels, followed by batch normalization, ReLU activation, and dropout. The second convolution layer employed one kernel for convolution followed by batch normalization, ReLU activation, and dropout. Following the convolution steps, the flattened nodes were connected to fully connected layers. For the cancer detection model, the output layer was composed of one node with the sigmoid function, and binary cross-entropy was used for the loss function. For the tissue-of-origin localization model, the output layer was composed of the number of tumor sites with the softmax function, and categorical cross-entropy was used for the loss function. Because of the imbalanced number of samples, the initial weight of the output layer was modified by calculating the class weight using ScipyBayesian optimization was used with 200 trials to find the best hyperparameter set to maximize the performance on the validation set. The list of the hyperparameters is provided in Supplementary Data 9. Briefly, locally weighted scatterplot smoothing (LOWESS) regression analysis was separately applied for short (100–150 bp) and long (151–220 bp) fragments with a span setting of 0.75. The LOWESS estimate of coverage was subtracted from the original measurement to separately obtain residuals for the short and long fragments. The genome-wide median short and long estimates of coverage were added back. The corrected values of total and short fragment coverage were z-score normalized across all bins for each sample. For the input of the fragsize model8, cfDNA fragments were broken down by length, increasing by 2 bp from 0 to 250 bp. The 125 features were normalized using the z score.The whole genome was split into non-overlapping 5 Mb windows. To filter out the blacklisted regions including telomeres and centromeres, the windows containing N nucleotides or with low GC contents were discarded. To construct the cnv model, read coverage at each bin was obtained and normalized using the z score. The input data for the fragpattern model were processed, as previously described32. Hyperparameters, including n_estimators, learning_rate, max_depth, min_child_weight, and colsample_bytree, were searched using a random search method with three-fold cross-validation. The XGBoost model was fitted using the log loss evaluation metrics, and the lowest loss model on the validation set was selected as the final model. The binary:logistic objective function was used for cancer detection, and the multi:softmax objective function was used for tissue-of-origin localization.For training of the cnv, fragpattern, and fragsize models, gradient boosting decision trees were implemented by XGBoostn = 54), pancreatic (n = 34), ovarian (n = 28), colorectal (n = 27), gastric (n = 27), lung (n = 12), and bile duct cancer (n = 26). Following the approval of their Data Access Committee (DAC), duplicate marked bam files of the DELFI dataset were obtained from European Genome-Phenome Archive (EGA). Genome, epigenome, cnv, fragpattern, and fragsize input features were processed using duplicate marked bam as described in the sections “Genome model input processing and training”, “Epigenome model input processing and training” and “Cnv, fragpattern, and fragsize model input processing and training”.We used the DELFI dataset with 1-2× cfDNA WGS of 214 healthy samples and 208 cancer patients to validate our algorithm. Cancer patient samples include breast  for the MGI and Illumina sequencing platforms was partitioned into five groups for the application of the stratified five-fold cross-validation. At each iteration, four groups in the training set were further divided into three training sets and one validation set. Using this method, all samples were given a test prediction score from each model. Using the test prediction score of all training cohort samples, we calculated the ROC-AUC score for cancer detection and the accuracy score for tissue-of-origin localization. The confidence interval for sensitivity was calculated from 1000 bootstrap samplings with replicates at 95%, 98%, and 99% specificity. The cancer localization model was developed by using either all cancer samples in the training cohorts or the cancer samples correctly identified by the combined cancer detection model with 98% specificity. The number of correctly predicted samples was 1188 out of 1359 for the MGI cohort and 644 out of 940 samples for the Illumina cohort.To validate the robustness of the models, validation cohorts were generated using independent batches from the training cohorts. The combined, genome, epigenome, cnv, fragpattern, and fragsize algorithms were used to predict validation cohorts. The average of five prediction values was used as the final prediction score. Using the final prediction score, we calculated the ROC-AUC score. Also, the confidence interval for sensitivity was calculated from 1000 bootstrap samplings with replicates at 95% specificity.33 that considers all gradients from the baseline input to the real input. TensorFlow was used to implement the integrated gradient method. To calculate the attribution scores, a zero array was used as the baseline input. From the baseline to each input array, 50 interpolated arrays were generated. The gradients of each interpolated array were obtained and averaged to calculate the attribution scores.The attribution score, a per-feature score based on the feature’s contribution to the model’s output when making a prediction, was calculated using an integrated gradient methodFrom the cancer detection genome model, cancer samples with a test prediction score > 0.8 and normal control samples with a test prediction score <0.2 were selected from the MGI training cohort and Illumina training cohort, respectively. As a result, 601 cancer samples and 306 normal control samples from the MGI training cohort and 391 cancer samples and 519 normal samples from the Illumina training cohort were used for interpretation analysis. The attribution score was calculated as described above. Positive attribution features of cancer were defined as top 100 regions with positive attribution value in cancer patient predictions and negative attribution value in normal sample predictions. Also, negative attribution features of cancer were defined as top 100 regions with negative attribution value in cancer patient predictions and positive attribution value in normal sample predictions. We tested whether positive attribution features of cancer actually had high LMD values in the PCAWG tissue data, and negative attribution features of cancer had low LMD values in the PCAWG tissue data.For the genome model for cancer localization, cancer samples whose localization was predicted correctly with a test prediction score > 0.6 were selected from the MGI training cohort and illumina training cohort, respectively, for interpretation analysis. As a result, 640 breast cancer, 166 liver cancer, 130 esophageal cancer, 134 ovary cancer, 59 pancreatic cancer, and 34 lung cancer cfDNA samples from the MGI training cohort were selected. 99 liver cancer, 25 esophageal cancer, 53 ovary cancer, 41 pancreatic cancer, and 20 colon cancer were selected from the Illumina training cohort. The attribution score was calculated as described above. To evaluate the attribution of the reference LMD values obtained from the PCAWG data, the sample-wise attribution scores were compared among the high LMD regions, low LMD regions, and other LMD regions of each cancer type. We tested whether high LMD regions had positive attribution whereas low LMD regions had negative attribution, specifically for the PCAWG cancer type matching with the given cfDNA sample but not for the other PCAWG cancer types unmatching with the given cfDNA sample.30 was applied across fragment size to compare its distribution between the normal control and cancer samples. To draw a heatmap, we used normalized attribution values of fragment length and NDR position. Normalized attribution values of the fragment length were processed by averaging the attribution arrays  along axes 2 and 3, and normalized attribution values of the NDR position were processed by averaging the attribution arrays  along axes 1 and 3.From the cancer detection epigenome model, cancer samples with a test prediction score >0.8 and normal control samples with a test prediction score <0.2 were selected from the MGI training cohort and Illumina training cohort, respectively, for interpretation analysis. As a result, 1152 cancer samples and 164 normal control samples from the MGI training cohort were selected. A total of 312 cancer samples and 401 normal control samples from the Illumina training cohort were selected. To display attribution values as 2D image, the absolute value of the attribution array  was averaged along the axes 0 and 2 for the normal control and cancer patients, respectively. To examine the effect of fragment size, the absolute value of the attribution array  was averaged along the axes 0, 2, and 3 separately for the normal control and cancer samples, respectively. Min-max normalizationFor the epigenome model for cancer localization, cancer samples whose localization was predicted correctly were selected from the MGI training cohort and Illumina training cohort, respectively. As a result, 591 breast cancer, 155 liver cancer, 71 esophageal cancer, 109 ovary cancer, 26 pancreatic cancer, 3 lung cancer samples from the MGI training cohort and 117 liver cancer, 20 esophageal cancer, 81 ovary cancer, 79 pancreatic cancer from the Illumina training cohort were used for interpretation analysis. To display attribution values as 2D image, the attribution array  was averaged along the axes 0 and 2 for each cancer type, respectively. To examine the effect of genomic distances from NDRs, the tissue-specific NDRs were classified into NDRs of cancer types matching the given cfDNA sample and those of cancer types unmatched with the given cfDNA sample. The attribution array  for the matched NDRs and the attribution array  for the unmatched NDRs were averaged along the axes 0, 1, and 3 within each cancer type. To draw a heatmap, we used normalized attribution values of fragment length and NDR position. Normalized attribution values of the fragment length were processed by averaging the attribution arrays  along axes 2 and 3, and normalized attribution values of the NDR position were processed by averaging the attribution arrays  along axes 1 and 3.3. The reads with mapping quality > 30 reads were selected to compute read coverage across 1 Mb non-overlapping windows using the readCounter function in the HMMcopy R package3. Next, 417 normal control samples in the MGI cohort and 728 normal control samples in the Illumina cohort were used to create a normal reference dataset for each platform. Tumor fraction was estimated using the ichorCNA algorithm with the platform-specific normal reference dataset.Tumor fraction in cfDNA was estimated by using ichorCNAThe average depth of cfDNA WGS was calculated to perform downsampling. SAMtools depth was used to calculate the average depth of cfDNA WGS. To make the desired depth of cfDNA WGS, required downsampling ratio was calculated by dividing the desired depth by the actual depth. Downsampling was performed using SAMtools view with the -b -h and -s [downsampling ratio] options. Genome and epigenome input features were processed using downsampled bam as described in the sections “Genome model input processing and training” and “Epigenome model input processing and training”.We performed wilcoxon two-sided test for statistical analysis. For study design, we did not use statistical method to predetermine sample size. Also no data were excluded from the analyses. The experiments were not randomized and the investigators were not blinded to allocation during experiments and outcome assessment.Further information on research design is available in the Supplementary InformationPeer Review FileReporting SummaryDescription of Additional Supplementary FilesSupplementary Data 1-10


Twenty hub gene features were further selected based on co-expression analysis to construct the model. The principal component analysis reduced the dimension of the training samples from 20 to 16, since 16 components were needed to retain more than 90% of the original variance. The support vector machine model was able to differentiate between those with major depression from those without in fibromyalgia syndrome patients with an average accuracy of 93.22% based on the expression levels of the selected hub gene features. These findings would contribute key information that can be used to develop a clinical decision-making tool for the data-driven, personalized optimization of diagnosing depression in patients with fibromyalgia syndrome.The prevalence rate of depression is higher in patients with fibromyalgia syndrome, but this is often unrecognized in patients with chronic pain. Given that depression is a common major barrier in the management of patients with fibromyalgia syndrome, an objective tool that reliably predicts depression in patients with fibromyalgia syndrome could significantly enhance the diagnostic accuracy. Since pain and depression can cause each other and worsen each other, we wonder if pain-related genes can be used to differentiate between those with major depression from those without. This study developed a support vector machine model combined with principal component analysis to differentiate major depression in fibromyalgia syndrome patients using a microarray dataset, including 25 fibromyalgia syndrome patients with major depression, and 36 patients without major depression. Gene co-expression analysis was used to select gene features to construct support vector machine model. The principal component analysis can help reduce the number of data dimensions without much loss of information, and identify patterns in data easily. The 61 samples available in the database were not enough for learning based methods and cannot represent every possible variation of each patient. To address this issue, we adopted Gaussian noise to generate a large amount of simulated data for training and testing of the model. The ability of support vector machine model to differentiate major depression using microarray data was measured as accuracy. Different structural co-expression patterns were identified for 114 genes involved in pain signaling pathway by two-sample KS test ( Chronic pain, defined as pain lasting at least 3 months, is a major clinical and socioeconomic burden to both patients and society . FibromySupport vector machine (SVM) is one promising technique for identifying biological markers, which has been widely used to diagnose and classify various diseases, especially depression. Researchers have developed an integrated analytical algorithm consisting of nuclear magnetic resonance-based metabolomics and least squares-SVM to diagnose depression . SVM wasSince pain and depression can cause each other and worsen each other, we hypothesize that pain-related genes can be used to differentiate between those with major depression from those without. In this study, whole genome data analysis was applied to select gene features to construct SVM model combined with principal component analysis (PCA) technique for prediction of depression in patients with FMS. Usually, multiple variables are included in SVM models, leading to a high-dimension, and it is difficult to find the patterns in data with high-dimension. PCA can help reduce the number of data dimensions without much loss of information, and identify patterns in data easily . Our proThe microarray dataset GSE67311, from publicly available Gene Expression Omnibus (GEO) repository database, was used for analysis in this study . The datUsually, pain and depression can cause each other and worsen each other. The pain signaling system consists of transduction, conduction, synaptic transmission, and modulation, and 114 pain-related genes identified from the microarray dataset were summarized for analysis  Julius . We furtAn FMS-specific cutoff point was identified to classify the co-expressed gene pairs into strong and weak co-expression classes. The normal-specific strongly co-expressed pairs were the gene pairs strongly co-expressed only in the healthy individuals, which were disrupted in the FMS group, called disrupted links. The FMS-specific strongly co-expressed pairs were the gene pairs strongly co-expressed only in the FMS group, which were invoked in the FMS group, called invoked links. There were also common weakly co-expressed pairs and common strongly co-expressed pairs. The disrupted links were regarded as the inter-gene linkages maintaining physiological balance in healthy individuals. The invoked links represented the characteristics of the disease and may be the pathogenic alternatives.The sum of absolute Pearson correlation coefficient (|r|) values for each gene in the co-expression network was calculated, defined as Rsum. Genes with the largest Rsum were called hub genes. Throughout the study, Rsum values were used to select hub genes for depression prediction. Cytoscape is an open source software used to visualize complex . ClueGO After a subset of gene features was selected, SVM was used to distinguish depressed and non-depressed FMS patients. PCA is a powerful tool to reduce the number of data dimensions without much loss of information . A classThe 61 samples available in the database cannot represent every possible variation of each patient. One important property of an SVM model to predict depression was the ability to generalize to the data obtained from a clinical site that was not involved in the training of the model. We adopted an additive model of embedding Gaussian noise to improve the capacity for such generalization . This add-dimensional feature vector  constructing the training data set; ii) subtracting the mean from each data dimension; iii) calculating the covariance matrix; iv) calculating the eigenvectors and eigenvalues of the covariance matrix; v) determining the number of principal components; and vi) forming feature vectors for testing samples. Through PCA operation, the information redundancy of gene signals can be removed, making it easier for the model to separate depression from non-depression.L denoted the set of sample categories, and l denoted the total number of sample categories (l = 2). This problem was considered as giving n labeled empirical samples.RD (D = 20), and yi denoted the label of p (p = 20) and Xi was then transformed to a higher dimensional feature space  for classification as shown , only support vectors were considered in SVM. For two linearly separated classes, the training data must satisfy the following two constraints:The margin of separation between two classes was thereby calculated as follows:SV denoted the set of support vectors, To separate depression from non-depression, the problem was to maximize the margin straints . This castraints . By intrormation , the catormation :di=signon (RBF) . Generalgamma and the penalty parameter (C). In the implementation, the precision was observed for 6 splits and 6 repeats. For each iteration, the training dataset (trainSetAll) was split into 6 folds, where 5 sets were served as the trainSet, and the remaining 1-fold was served as the testSet. The prediction accuracy of the model was calculated on the testSet in each iteration. Finally, gamma and C were obtained with the highest cross-validation accuracy. The best gamma and C were then used to train the whole training set, and the final model was generated to predict the whole testing set.Input: trainSetAll, type, kernel, m1, m2Output: bestC and bestGambegin <- KfoldSplit (trainSetAll);bestAccuracy = 0;bestC = 0;bestGam = 0;forc= 2^(-m1):2^(m1)fors = 2^(-m2):2^(m2)   model<- SVMtrain ;    acc<- SVMtest ;     if (acc>bestAccuracy)    bestAccuracy = acc; bestC = c; bestGam = s;      end    endendendTo obtain a more robust model with RBF kernels for the prediction of depression, K-fold cross-validation and grid search were implemented to obtain the optimal values of t-test was applied to exam the statistical significance for Rsum values. Significant differences were found at p-value < 0.05.The co-expression analysis and SVM model construction were conducted using MATLAB R2022a. The statistical analysis was performed by IBM SPSS Statistics 27.0 software. Chi-square test was used to determine if genes were more disrupted or invoked in FMS patients. The normality of the data was checked first. The independent sample p-value < 0.001 for the maximum deviation D = 0.11 > Dcritical = 0.05. The FMS-specific cutoff point, 0.366, was identified at the maximum deviation , indicating the important role of invoked hub genes in the development of FMS  TRPs channels, adjusted p-value < 0.001, 3 involved genes; iii) Sodium channel activity, adjusted p-value < 0.001, 5 involved genes; and iv) Transmitter-gated ion channel activity involved in regulation of postsynaptic membrane potential, adjusted p-value < 0.001, 9 involved genes. Each major group was composed of one or several terms/pathways. The group of transmitter-gated ion channel activity involved in regulation of postsynaptic membrane potential had the most terms/pathways, and the percentage was the highest, 86.05%  Ion , 86.05% . The resIn this simulation, PCA was applied to 20 hub genes with expression measurements for original training samples. To get a view for the dimension of the data, the proportion of the variance within each principal component was shown . The resThe simulated expression profiles and their distributions in 2D PCA space with different values of SNR were shown . The flugamma, C and accuracy was visualized , and lead to overfitting for the training data. For unknown samples, the classification performance would be poor. By contrast, if the value of gamma was smaller, the smoothing effect of RBF was larger. It was difficult to obtain higher classification accuracy as shown  and Ca2+ channels (VGCCs), are the major treatment for chronic neuropathic pain . These patients should then be referred to a clinical psychiatrist for formal diagnosis  and subsequent management for their psychiatric condition. Given that depression is a common major barrier in the management of patients with FMS, and is also often clinically unrecognized, such computer-based interface systems could become very useful in aiding clinical diagnosis and management.In conclusion, we have demonstrated a proof-of-concept pipeline for differentiating between those with major depression from those without among FMS patients. When developed into a proper clinical application, it may contribute to the diagnosis and clinical management for patients with co-existing FMS and depression. These findings would help to develop a clinical decision-making tool for data-driven, personalized optimization of diagnosing depression in patients with FMS.


CNA) for pan-cancer HRD prediction based on these CNA features. CNA features BP10MB[1] (The number of breakpoints per 10MB of DNA is 1) and SS[ > 7 & <=8]  are identified as the most important features in HRD prediction. HRDCNA suggests the biallelic inactivation of BRCA1, BRCA2, PALB2, RAD51C, RAD51D, and BARD1 as the major genetic basis for human HRD, and may also be applied to effectively validate the pathogenicity of BRCA1/2 variants of uncertain significance (VUS). Together, this study provides a robust tool for cost-effective HRD prediction and also demonstrates the applicability of CNA features and signatures in cancer precision medicine.Homologous recombination deficiency (HRD) renders cancer cells vulnerable to unrepaired double-strand breaks and is an important therapeutic target as exemplified by the clinical efficacy of poly ADP-ribose polymerase (PARP) inhibitors as well as the platinum chemotherapy drugs applied to HRD patients. However, it remains a challenge to predict HRD status precisely and economically. Copy number alteration (CNA), as a pervasive trait of human cancers, can be extracted from a variety of data sources, including whole genome sequencing (WGS), SNP array, and panel sequencing, and thus can be easily applied clinically. Here we systematically evaluate the predictive performance of various CNA features and signatures in HRD prediction and build a gradient boosting machine model (HRD Analysis of copy number patterns of cancer genomes helps in devising a machine learning algorithm that can accurately predict homologous recombination deficiency status. In cancer, accurate detection of homologous recombination deficiency (HRD) is of clinical relevance as HRD tumors are reported to be sensitive to poly ADP-ribose polymerase (PARP) inhibitors2, as well as to platinum chemotherapy drugs3.DNA damage is a common hallmark of cancer, and DNA double-strand break (DSB) is the most hazardous type of DNA damage which can be repaired by error-free homologous recombination (HR) pathway4 and the genomic instability score (GIS) are the currently widely used HR status diagnostic schemes. Germline BRCA testing could overlook the epigenetic silencing8, and other pathogenic somatic mutations9, and at the same time strongly relies on the completeness and accuracy of clinical variant annotation databases. GIS, also named as “HRD score”, mainly consists of three SNP array-based methods: telomeric allelic imbalance (TAI)10, loss of heterozygosity (LOH)11, and large-scale transition (LST)12. One disadvantage of HRD score is that the optimal threshold is not consistent in patients with different types of cancer15. Recently, HRDetect17 and CHORD9 have been constructed for HRD prediction, however, these tools need whole genome sequencing (WGS) or whole exome sequencing (WES) data, which is expensive and lacks clinical applicability.Germline BRCA genes testing21. CNA information can be obtained from a diverse type of data, such as shallow WGS, WES, SNP array, and panel sequencing, and could represent a cost-effective type of biomarker for cancer diagnosis and clinical response prediction. However, the efficacy and application of CNA signatures and features in HRD prediction have not been established yet.Copy number alteration (CNA) is an important type of cancer-driving genetic alteration, and CNA signatures and features are emerging cancer-inherent patterns. Recently, patterns and mutational processes of CNA signature have started to be revealedCNA  based on CNA features. HRDCNA model is trained on the CNA data of 1,470 cancer samples generated from WGS or SNP array and is validated with 831 samples generated from WGS, SNP array, shallow WGS, or panel sequencing data. HRDCNA can precisely predict HR status using CNA features data derived from different platforms or different cancer types with consistent cut-off. HRDCNA provides a new direction for effectively validating the pathogenicity of uncertain important variants (VUS) of BRCA1 and BRCA2, and suggests the major genetic basis for human HRD. The applicability of CNA features and signatures in cancer precision medicine is also well demonstrated in HRDCNA.Here we develop a robust HRD predictor HRD9 (PCAWG) and 560 breast cancer samples (SNP array data)16 are collected. To obtain a high-confidence training dataset of HRD, samples with BRCA1/2 deficiency are screened for classifier training. In total, 130 cancer samples with loss-of-function mutations in BRCA1/2 are labeled as HRD, while 1340 samples without inactivation mutations in known HR genes are labeled as homologous recombination proficiency (HRP). Then these samples are randomly split into training and held-out datasets by the ratio of 8:2 for model training and testing respectively. The performance of the HRD prediction model is further evaluated on three independent validation datasets: a total of 633 cancer samples, including 66 breast cancer samples with WGS, 66 breast cancer samples with SNP array sequencing, and 501 pan-cancer samples with panel sequencing  from the pan-cancer analysis of whole genomesTo find the best HRD prediction model, a total of 9 machine learning models (see Methods) are trained using the training dataset, and the performances of each machine learning model are reported as the area under the receiver operating characteristic (ROC) curve (AUC) and the area under the precision-recall curves (PR-AUC) in all datasets  algorithm with these CNA features, and the CNA signatures developed with this method are named “Sig-CNS” here20. Ruben et al. conducted a pan-cancer CNA signature study and reported 17 signatures (named “Sig-CX” here)19. CNA features are countable parameters developed in Wang et al. study20. We compare the performance of three different models using CNA features or using the reported two sets of CNA signatures (Sig-CNS and Sig-CX). AUC and PR-AUC results show that HRD prediction models construct with CNA features exhibit improved performance compared with models built with CNA signatures  strategy is used. Then the relative variable importance is calculatedres Fig. . Then weres Fig. . The BP1res Fig. . The finCNA shows excellent performance in the held-out dataset and also three independent validation datasets  have been reported to contribute to human HRD27. We explore potential BRCAness genes using the cancer genome atlas (TCGA) dataset by comparing the incidence of biallelic pathogenic mutations in HR-related genes28 between patients predicted as HRD and HRP using HRDCNA  is observed to be significantly higher in HRD, and the inactivation of these two genes have been demonstrated in previous studies as the major genetic basis of human HRD31. Patients with biallelic inactivation mutations in the following genes: RAD51 paralog D (RAD51D) (n = 4), BRCA1-associated RING domain protein 1 (BARD1) (n = 3), Chromosome 19 open reading frame 40 (C19orf40) (n = 1) and MRE11 homolog A (MRE11A) (n = 1) are all predicted as HRD, suggesting these genes, especially BARD1 and RAD51D inactivation can be genetic basis of human HRD.BRCAness is functionally defined as a defect in homologous recombination repair that phenocopies loss of BRCA1/2CNA Fig. . The incCNA is applied to 33 cancers in the TCGA dataset, each sample gets an HRDCNA score. The scores widely vary across different cancer types. The highest scores are observed in ovarian cancer, and many cancer types also have subsets of samples with high HRDCNA scores. Some cancer types such as thyroid carcinoma, thymoma, uveal melanoma, and acute myeloid leukemia do not show high HRDCNA scores  with unknown functional consequences. We calculate the HRDres Fig. . Then weCNA based on CNA features. This tool enables accurate and robust HRD prediction using CNA profiles derived from a variety of cost-effective platforms, such as shallow WGS, SNP array, and panel sequencing. Furthermore, the CNA features BP10MB[1] and SS[ > 7 & <=8] are identified as the major contributors to HRDCNA model, and also represent potential biomarkers for HRD. In addition to biallelic inactivation of BRCA1 and BRCA2, PALB2, RAD51C, RAD51D, and BARD1 inactivation are identified as the major genetic basis for human HRD. HRDCNA tool also provides a new direction for effectively validating the pathogenicity of VUS. Carrying BRCA1/2 mutations, earlier stage, and lower levels of residual tumor after surgery have been proven as predictors of better prognosis for patients receiving platinum chemotherapy in ovarian cancer35. Here, survival analysis shows that patients with high HRDCNA scores have significantly improved survival than those with low HRDCNA scores in patients receiving platinum chemotherapy. Over time, the application of HRDCNA would unearth a substantial cohort of patients without BRCA1/2 mutations for chemotherapy using platinum drugs.Here we developed a pan-cancer HRD prediction tool HRDBRCA1 and BRCA2, PALB2 and RAD51C biallelic pathogenic mutations have been demonstrated in previous studies as the major genetic basis of human HRD31. Notably, there is also a relatively high incidence of biallelic alterations in BARD1 and RAD51D in HRD. BRCA1 partners with BARD1 to mediate the initial nucleotide excision of DNA damage and the recruitment of the recombinase RAD5136. BARD1 loss is sufficient to confer an HRD phenotype and significantly increase sensitivity to PARP inhibitors in prostate cancer cells37. Triple-negative breast cancer cells carrying RAD51D K91fs and V200X variants are reported to be vulnerable to PARP inhibitors because they lead to decreased HR function38. Together, our study suggests BRCA1, BRCA2, PALB2, RAD51C, BARD1, and RAD51D biallelic inactivation as the major genetic basis of human HRD, and it provides a novel opportunity for the precision medicine approaches in HRD and expands the population of patients who may benefit from agents targeting HRD.In addition to CNA model is a CNA features-based robust tool for HRD detection. In the future, CNA features or signatures could be combined with artificial intelligence to reveal clinically meaningful CNA fingerprints for cancer precision diagnosis and therapy response prediction, and the HRDCNA model developed in this study could represent one example of these clinically meaningful CNA fingerprints. HRDCNA is freely available as an R package at https://github.com/XSLiuLab/HRDCNA.Collectively, accurate measurement of HR status to find HRD patients has great clinical significance in precision medicine. HRDCNA , which included 1159 samples (WGS data) from PCAWG dataset9, 311 samples (SNP array data) from 560 breast dataset16. In particular, samples that were duplicated with PCAWG dataset in 560 breast dataset have been excluded, so there are no duplicated samples in our datasets. The information about the frequencies of cancer types in the training data for HRDCNA model is shown in Supplementary Table https://dcc.icgc.org/releases/PCAWG/. BRCA1/2 status annotations for this dataset are obtained from the supplementary data in Nguyen et al.9. Somatic copy number data of the 560 breast dataset are downloaded from the department of medical genetics at the University of Cambridge . BRCA1/2 status annotations and mutation data for this dataset are obtained from the supplementary data in Davies et al.16.We have used CNA data from 1470 samples to develop the homologous recombination deficiency (HRD) gradient boosting machine (GBM) model, called HRD9. We select 80% of 1470 samples by random sampling as the training dataset, and 20% as the held-out dataset for model training. Copy number data, HR status information, and survival data for the panel dataset are obtained from the supplementary data in Wen H et al.24. 66 breast dataset is publicly available in the figshare repository provided by de Luca et al.39. Copy number data from ASCAT (array data) are available from 10.6084/m9.figshare.9808496 and ascatNGS  and downsampled WGS data are available from ).130 samples carrying mutations in BRCA1/2 are detected with loss of wild-type allele and labeled as HRD samples, and 1340 BRCA1/2 proficiency samples as HRP samples. A sample with one of the following events in BRCA1/2 is defined as a BRCA1/2 deficient sample: (i) complete copy number loss, (ii) LOH in combination with a pathogenic germline or somatic SNV/indel or structural variations, or (iii) pathogenic SNV/indels or structural variations in both alleleshttps://portal.gdc.cancer.gov/). Allele-specific copy number analysis of tumors is performed using ASCAT2, to generate integral allele-specific copy number profiles for the tumor cells. Biallelic inactivation in HR-related genes information for TCGA dataset is obtained from Riaz et al.28. The mutation data for TCGA dataset are obtained from the GDC Data Portal and Legacy Archive (https://gdc.cancer.gov/). The information on the cohort of 71 TNBC patients in TCGA dataset is derived from the supplementary data in Liao et al.40. The information on the cohort of 80 breast patients is derived from the supplementary data in Davies et al.16. Pathogenicity annotations are obtained from ClinVar (https://www.ncbi.nlm.nih.gov/clinvar/), CancerVar41 (https://cancervar.wglab.org/), CADD42 (https://cadd.gs.washington.edu/), MutationAssessor43 (http://mutationassessor.org/r3/) and PolyPhen-244 (http://genetics.bwh.harvard.edu/pph2/).Somatic copy number data of TCGA pan-cancer dataset are downloaded from genomic data commons data portal .Downsampling of the original normal and tumor BAM files is performed using the samtools (version 1.3)GridSearchCV function in Python package sklearn, and we use the default 5-fold CV strategy. The GBM model has the best performance on the held-out dataset. Thus, GBM is chosen to predict the probability of HRD. Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak base models, usually decision tree. R package gbm is used to implementation of the GBM.To find the best performance model among a multitude of methods, 9 machine learning models including extremely randomized trees (Extra trees), random forest, logistic regression, support vector machine (SVM), eXtreme gradient boosting (XGB), adaptive boosting (AdaBoost), decision tree, K-nearest neighbor (K-Neighbor) and gradient boosting machine (GBM) are trained, the AUC of the held-out dataset is selected as the performance criterion. All the hyper-parameters are determined by the gradient search method provided by the Sigminer20, which is based on the tool SigProfiler46 caller. We pick up 8 signatures for our dataset due to its relatively high stability and low distance ; the breakpoint count per chromosome arm (named BPArm); the copy number of the segments (named CN); the difference in copy number between adjacent segments (named CNCP); the lengths of oscillating copy number segment chains (named OsCN); the log10 based size of segments (named SS); the minimal number of chromosome with 50% copy number variation (named NC50); the burden of chromosome (named BoChr). Then we classified 8 CNA feature distributions into 80 different components20, and each copy number component has a clear biological meaning, for example, BP10MB[1] indicates the number of breakpoints per 10MB of DNA is 1 (Details in Supplementary Table Sig-CNS is identified using R package CNA). Firstly, the HRDCNA model is developed based on 76 significantly different CNA features among 80 CNA features  from the Bernoulli deviance  and the area under the precision-recall curves (PR-AUC) are then calculated by R package precrec.Data between two groups are compared using Wilcoxon rank-sum test  depending on the normality of data distribution. In the survival analysis, the differences between different survival curves are compared using log-rank test, and the cut-off score is 0.14, which is determined based on the youden indexCNA includes 1470 samples with 1186 in the training dataset and 284 in the held-out dataset. 130 samples are labeled as HRD samples, and 1340 samples are labeled as HRP samples. The independent validation dataset consists of 633 cancer samples, including 66 breast cancer samples with WGS, 66 breast cancer samples with SNP array sequencing, and 501 pan-cancer samples with panel sequencing, and 330 of which are HRD samples and 303 HRP samples. TCGA dataset of 33 cancer types includes 10,906 samples. The cohort of 71 TNBC patients in TCGA dataset consists of 71 TNBC samples, 43 of which are HRD samples and 28 HRP samples. The cohort of 80 breast patients consists of 80 breast samples, 7 of which are HRD samples and 73 HRP samples.CNA data to develop HRDFurther information on research design is available in the Supplementary InformationReporting Summary