This parameter allows assessing the number of different pathologies that a certain gen is able to discern. It has been evaluated together with other widely known parameters under assessment of an ANOVA statistical test which corroborated its filtering power when the identified genes are subjected to a machine learning process at multiclass level. The optimal tuning of gene extraction evaluated parameters by means of this statistical test led to the selection of 42 highly relevant expressed genes. By the use of minimum-Redundancy Maximum-Relevance (mRMR) feature selection algorithm, these genes were reordered and assessed under the operation of four different classification techniques. Outstanding results were achieved by taking exclusively the first ten genes of the ranking into consideration. Finally, specific literature was consulted on this last subset of genes, revealing the occurrence of practically all of them with biological processes related to leukemia. At sight of these results, this study underlines the relevance of considering a new parameter which facilitates the identification of highly valid expressed genes for simultaneously discerning multiple types of leukemia.In more recent years, a significant increase in the number of available biological experiments has taken place due to the widespread use of massive sequencing data. Furthermore, the continuous developments in the machine learning and in the high performance computing areas, are allowing a faster and more efficient analysis and processing of this type of data. However, biological information about a certain disease is normally widespread due to the use of different sequencing technologies and different manufacturers, in different experiments along the years around the world. Thus, nowadays it is of paramount importance to attain a correct integration of biologically-related data in order to achieve genuine benefits from them. For this purpose, this work presents an integration of multiple Microarray and RNA-seq platforms, which has led to the design of a multiclass study by collecting samples from the main four types of leukemia, quantified at gene expression. Subsequently, in order to find a set of differentially expressed genes with the highest discernment capability among different types of leukemia, an innovative parameter referred to as  Cancer is one of the worst diseases to the point of being the leading cause of death worldwide, just behind cardiovascular disease. There are many types of cancer, and each of these may have a gene signature allowing to discern healthy people from people suffering from cancer. In this paper, from the use of transcriptomic data, different types of leukemia will be studied in order to find a leukemia gene signature for each of them. Leukemia, together with Lymphoma and Myeloma, is one of the three different existing blood cancer forms. People that suffer from leukemia produce an abnormal number of immature white blood cells, which collapse the bone marrow and inhibit the creation of the rest of vital blood cells for a balanced immune system and healthy blood. There are two main types of leukemia, being each divided into two subtypes:Acute myeloid leukemia (AML): is the most common leukemia in people around 70 years but it has impact in all ages. This malignancy is a heterogeneous group of neoplastic disorders, that are characterized by the proliferation and accumulation of immature hematopoietic cells in the bone marrow and blood. Different genetic factors have been identified that predispose to the development of AML. In this context, the germline predisposition and the existence of haematological disorders antecedent, have been associated with an increased risk of AML , 2.Acute lymphoblastic leukemia (ALL): is the most common leukemia in children. About half the cases are in adults and half in children. ALL is also a very heterogeneous disease, characterized by impaired differentiation and proliferation of immature lymphoid cells in the bone marrow and peripheral blood. However, the prognosis of these patients has improved in the last years, especially in children, leading to cure rates approaching 80% to 90% due to the intensification of treatment, patient stratification based on clinical risk factors, and minimal residual disease (MRD) monitoring , 4.Acute Leukemia appears suddenly and progresses quickly so the treatment has to be urgent.Chronic myeloid leukemia (CML): it is very unusual and affects only 700 people per year. CML, defined as a clonal myeloproliferative disorder, was the first human malignancy associated to a consistent chromosomal abnormality and is characterized by the presence of the fusion oncogene BCR-ABL. Clinical symptoms associated to this disease include hypercellular bone marrow, anemia, platelet dysfunction, and an increase in the number of leukocytes, especially neutrophils and immature myeloid cells , 6.Chronic lymphocytic leukemia (CLL): it is more common in people over 60 years and is very rare in people under 40 years. CLL is a common B-cell tumor, characterized by the gradual accumulation of clonally expanded CD5+ B lymphocytes in peripheral lymphoid organs, secondary lymphoid organs, and bone marrow. It is also a genetic and biological complex disease, and the most commonly used factors to stratify CLL patients are the mutational status of the variable portion of the immunoglobulin gene, the deletion of the chromosome 17p and TP53 gene mutations .Chronic Leukemia: symptoms appears more slowly, maybe in months or even years.In this paper, transcriptomic data will be used in order to find biomarkers for each of the types of leukemia mentioned before. For that, two gene quantification technologies (Microarray and RNA-seq) will be utilized to obtain gene expression signature. Both technologies are briefly described next.Microarray technology has been the most widely used gene quantification technology in the last two decades, until the arrival of Next Generation Sequencing (NGS) techniques. Even nowadays, Microarray is still in use due to the low cost of this technology in comparison with RNA-seq. Microarray was the first technology that managed to measure simultaneously the expression levels of all genes. Furthermore, a lot of Microarray platforms or manufacturers can be taken into account. The most extended Microarray platforms are Affymetrix and Illumina , 10. NevAlthough Microarray has been the best gene quantification technology since the ninety decade, RNA-seq was consolidated as the most powerful and newest technology since the last decade. As a natural evolutionary step in the gene quantification technologies, RNA-seq is gradually replacing the widespread use of Microarray. There exist many manufacturers that work with RNA-seq but, nowadays Illumina leads the RNA-seq sequencing technology market. Although its application was originally intended for genomic transcription study, it also allows achieving a mapping between the levels of transcription and gene expression . Thanks RNA-seq offers an important number of advantages over Microarrays, although the cost of RNA-seq experiments is nowadays still higher than Microarray technology’s:RNA-seq allows detecting the variation of a single nucleotide.RNA-seq does not require genomic sequence knowledgement.RNA-seq provides quantitative expression levels.RNA-seq provides isoform-level expression measurements.RNA-seq offers a broader dynamic range than Microarrays.All these advantages of RNA-seq in comparison with Microarray were clear described by Wang et al. . There aIn this research, a novel pipeline has been addressed with the purpose of taking advantage of both RNA-seq and Microarray gene quantification technologies. Furthermore, this integration has been done at multiclass level, due to the nature of the problem studied. At sight of this novel analysis, a new parameter has been introduced for extracting a set of candidate signature biomarkers, this parameter will be called COVERAGE or COV from now on and it will be used together with the Log-Fold Change (LFC) in order to find differentially expressed biomarkers. This parameter aims to measure the “coverage” that a certain biomarker has over the different diseases analyzed, i.e., the number of diseases it is able to distinguish.Moreover, this research is twofold objective. The first one is the extraction of possible Differentially Expressed Genes (DEGs) that allow to discern among the different forms of leukemia and people who does not suffers from the disease. The second one is to perform a classification stage with the DEGs extracted for the assessment of those genes. Therefore, this study is about a multiclass classification problem in which a set of genes will be selected as long as they are useful to discern among the five classes: four different types of leukemia and healthy subjects. This is a very novel research because most of the studies are dichotomous, addressing only two classes for their researches. To this end, the public repository NCBI/GEO has been used , 19. AllThere are not many researches that combine different platforms or even different technologies with the aim of achieving a possible gene signature –22. HoweThis paper has been structured as follows. This section has shown the introduction and state of the art of this work. Next section explains the methodology followed in this study. It begins by describing the available data series that have been used for this research. Later, the pipeline for processing and classifying the data is shown. Next section exposes the results of the integration and the selection process for obtaining the DEGs. Furthermore, the results of the assessment of the classifiers using those DEGs will be exposed and some plots representing the expression levels of the most important DEGs will be shown. The discussion section underlines the validity of the proposed approach and its utility in the classification of unknown samples that belong to the studied types of leukemia, by using the developed machine learning tool. A biological interpretation of the DEGs selected will be exposed too. Finally, the conclusions section summarizes the most important contributions of this study for leukemia diagnosis and genetic profiling.https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=S.NAME where S.NAME is the name of each series at NCBI GEO shown at For this study, both Microarray and RNA-seq samples have been collected in order to accomplish the integration of a wide range of heterogeneous data. As already mentioned, all series has been downloaded from the NCBI/GEO public database. A comprehensive search has been carried out with the purpose of gathering a notable number of samples belonging to the leukemia states addressed in this work. Furthermore, as all the samples must belong to the same tissue, only samples from cells of bone marrow have been used for both healthy and leukemia samples. With regard to Microarray samples, the two main platforms (Affymetrix and Illumina) have been taken into account. For RNA-seq, samples have been solely collected from the most important sequencing platform known as Illumina HiSeq . FinallyAdditionally, The two main tools used for the development of this study have been R and Matlab . On one In this subsection, the pipeline followed in this study is explained in detail. The pipeline can be split into four steps or parts, as it is shown at The first step is the analysis of each Microarray that will be used in the study. This step is valid for any Microarray technology, and in this work, it will be applied to the Microarrays from Affymetrix and Illumina.arrayQualityMetrics R package, which computes the Kolmogorov-Smirnov statistic Ka between the distribution of each array and the distribution of the pooled data [rma function from affy bioconductor package was used for Affymetrix Microarrays [lumiExpresso function from lumi bioconductor package was used [Firstly, it is very important to carry out a quality analysis of the available samples in the selected series in order to detect and consequently remove any possible outlier. This outliers detection and removal was performed through led data . Secondlroarrays . In the was used . Both fuThe second step in the pipeline corresponds to the RNA-seq analysis. This step follows the same ideas of the Microarray analysis except for a few differences, being the most relevant variation the data extraction process due to the necessity of a whole pipeline only to transform the raw data of RNA-seq into gene expression values.sra-toolkit, tophat2, bowtie2, samtools and htseq have been used in order to obtain the gene count files for each sample [NOISeq R packages [This RNA-seq processing pipeline was proposed by Anders et al. . Startinh sample –41. Oncepackages .cqn package was used [For the quality analysis and outliers detection, the same pipeline followed in Microarray can be used if the gene expression values are available. Furthermore, for the normalization and GC BIAS correction, the function cqn from was used . MoreoveThe Microarray and RNA-seq integrative pipeline merges data both from Microarray and RNA-seq technologies. There are two important steps for an appropriate integration of heterogeneous data: data merging and normalization.merge function from the base R package.The first step, data merging, requires the correct individual pre-processing of all available series in the study. Afterwards, heterogeneous integration was carried out using the normalizeBetweenArrays function was used [The second step in the integration is the normalization of the integrated dataset. To perform this normalization, the was used . This fuwas used .These tasks are essential in order to achieve a correct normalization of the biological data and its subsequent processing , 46. TheWith regard to the batch effect, it was addressed by using the considerations explained by Goh et al.  having iOnce the integration has been properly done, the next step is the DEGs extraction. There are some statistical toolboxes/routines that allow to estimate if a gene has enough statistical significance for being considered a DEG or not.As mentioned before, a multiclass problem is addressed in this study, which means that DEGs have to be valid for discerning among more than two classes. Specifically, five classes are identified in this study, being necessary to find a group of biomarkers with the capability of discerning among these classes in order to achieve multiclass classification. However, it is important to notice that limma, when dealing with a multiclass problem, takes into account only one value of the LFC between two classes, to identify if a gene is relevant or not, omitting the needed consideration of the rest of classes and class comparisons.COVmax. In our present problem including 5 classes, COVmax takes the value of 10 (ten pairs of class combinations). The real potential of this parameter lies in the ability to discover high coverage DEGs, thus allowing us to discern among the maximum possible number of classes.The total number of binary problems (taking one against one class comparisons) that take place in a problem with N classes can be defined as shown in COV as criteria to identify gene signatures implies choosing a coverage threshold, so that a certain biomarker will be selected only if it covers or differentiates at least a certain number of binary class comparisons. A large coverage threshold could be too restrictive, and a small coverage threshold could lead to the selection of too many biomarkers. The determination of an appropriate COV threshold is therefore critical in this problem. A medium size COV representing a trade-off between large and low differentiability in number of class pairs -coverage-  may seem to be a reasonable threshold. The ANOVA test presented later will study the performance of different COV threshold values.Using Finally, all the genes that pass the restrictions imposed by the p-value, LFC and COV restrictions will be selected as DEGs for discerning among the five classes of this study.DEGs extraction is a very sensitive process because an error in the parameters or in the design of the followed pipeline could lead to a mistake in the right identification of the DEGs. In order to assess the DEGs selection, a machine learning process is performed. This process is explained below.Firstly, the DEGs dataset is normalized with median 0 and standard deviation 1. This step homogenizes into the same type of distribution the gene expression values and can suppress the effect of possible remaining outliers due to the bounded range.Although the use of limma for the extraction of DEGs already identifies candidate genes for the classification of the different diseases, the application of a specific feature selection process before the classification step would reduce the size of the DEGs set, thus simplifying the classification and even improving the final accuracy. Precisely, the mRMR algorithm is used in order to obtain a ranking with the most appropriate combination of our DEGs, according to the operation of the algorithm. To create this ranking, mRMR takes into account the relevance and redundancy information among the genes. Using both criteria, mRMR sorts the genes so that they bring largest relevance with respect to the class and, at the same time, they have lowest redundancy among themselves. Therefore, this algorithm will rank in first position the gene that contains the largest amount of information, but the following genes will provide also minimum redundancy (apart from maximum relevance as regard to the class) with respect to the already selected genes.When both, the normalization and feature selection process are done, four different supervised learning algorithms that will be exposed below are applied to an increasing number of selected DEGs, according to the mRMR algorithm. A cross-validation step using k-fold is performed in order to assess the results of the classifiers on the dataset. K-fold cross-validation algorithm iteratively leaves out 1/k data from the training dataset, which are used to assess the classifier when the training is done. Finally, both accuracy and f1-score are calculated using the outcomes from the k assessment processes. This last measure takes into account the grade of classification of each class, not only the total amount of samples correctly classified. The whole machine learning process has been explained so far, but without going into detail of each of the four implemented classifiers: Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Naive Bayes (NB) and Random Forest (RF). These are shortly described next.γ hyperparameter, controlling the generalization capability of the model [SVM: The idea of SVM is the selection of an hyperplane that is equidistant from each class, thus achieving a maximum-margin for the separation of the classes. Moreover, when the hyperplane is defined, only the samples that fall into the frontier are taken into account as training support vector samples. Furthermore, the algorithm tolerates classification errors, which are controlled by the he model , 26. Fink-NN: This algorithm assigns to a new unseen sample, the predominant class corresponding to the k nearest neighbors (most similar samples) using the known labeled data. It is a well-known fast and easy-to-use technique which however may provide a comparable performance to other well-known more complex techniques , 48.NB: A naive Bayes classifier assumes that each characteristic is independent to the rest. Therefore, each of the DEGs contributes in an independent way to the probability of being part of a particular class. A clear advantage of this type of classifiers is that these need a small number of training samples for estimating the necessary parameters for the classification (mean and variance). It is a very fast and efficient classifier for supervised learning problems such as the problem addressed in our study .RF: Random Forest algorithm grows many single classification vector (trees) with the purpose of building a forest of classification trees. For the classification, the algorithm assigns a vector as input to be classified for each tree of the forest. Once that each individual tree performs the classification, a voting system among the trees decides the class having the largest number of votes over all the trees , 50.The ANOVA test plays a very important role in this study. It will analyze and compare the performance of the four classifiers, as well as of different combinations of the hyperparameters LFC and COV for optimal relevant biomarker identification. This step is very useful in order to determine if the classifiers have significant differences among them. Moreover, the test will provide valuable information about which combination of LFC and COV is better for our study and it will bring some light into the optimization of these parameters for further studies.This section will be split into three subsections in order to expose the results in the most clearly and organized way. In the first subsection, the impact of both the LFC and the COV on the final classification results is evaluated performing an ANOVA test. In the second subsection, the first four steps of our pipeline are applied for the extraction of the DEGs. Finally, the last subsection and also the last step in our pipeline is the DEGs ranking process, and the assessment of those DEGs using a machine learning process.The ANOVA test is very important at this point due to two main reasons. Firstly, there are several possible values that both the LFC and the COV could take. Secondly, in this research, four different classifiers are assessed. In view of these considerations, the test can decide if there are statistically significant differences among the classifiers and how the value of both the LFC and the COV could affect final results. The chosen classifier, LFC and COV are the parameters that a priori could cause more impact in the study. Nevertheless, the final number of selected genes after a feature ranking process -mRMR- could also affect final results, so it will also be considered.Therefore, the possibles values that the four factors can take in the test are the following:Classifier: this variable represents the classifier used for the simulation. This classifier can be SVM, k-NN, NB or RF.LFC: this variable represents the Log-Fold Change used in order to extract the relevant genes, taking the values 1, 1.5, 2, 2.5 or 3.COV: this variable represents in a multiclass problem, the number of combination of classes in which a gene is truly relevant, taking the values 2, 3, 4 or 5.NR. GENES: this variable represents the number of genes finally selected with mRMR, which were used as input for the classifier, taking values 10, 20, 30 or 40.From all the possible combinations of the former four factors, a wide range of simulations have to be addressed and evaluated in order to achieve an statistical interpretation of the results. Consequently, the test evaluates how both accuracy and f1-score are affected by the four chosen variables for the test . In the same way that the previous table shows how selected variables affect accuracy, Previous tables showed how the four studied variables affected both accuracy and f1-score. At this point, the impact of each value that the different variables could take will be described through a group of plots. In this sense, In the case of LFC, the plot shows that an increase of LFC does not lead to an accuracy improvement. However, it is really important to note the behavior of COV. As it can be seen, an increase in COV leads to an accuracy improvement, meaning that the new proposed measure is an important criterion in the selection of multiclass biomarkers. Among the values compared, ranging from 2 to 5, an improvement in the final recognition was shown due to the fact that as the value of COV increases more classes are covered by the selected genes.Finally, it is straightforward to expect that larger gene signatures may attain higher accuracy in the identification of the different pathologies studied than shorter gene signatures. This is shown by the NR. GENES variable, whose values range from 10 to 40.In the same way that the previous figure shows how the variables have a direct impact in the accuracy, At sight of the ANOVA test results, only one of the possible combinations of LFC and COV will be taken into account to extract the biomarkers of this study. Specifically, the LFC equal or greater than 2.5 and the COV equal or greater than 5. These values are both the best combination in terms of achieving the best possible results, as can be seen in the ANOVA plots exposed before. Then, for these requirement settings, gene signatures with different NR. GENES will be finally studied, and the detailed results of the four classifiers will be shown.Once the ANOVA test was performed and specific values for both the LFC and the COV were selected, we will proceed to extract the DEGs using these values.However, before presenting the DEGs extracted, the results of the series integration for this study will be shown. In order to perform an integrated analysis of these series, an individual analysis and correction of each series was done. Furthermore, it is necessary to correct the existing differences among the series due to the variety of technologies and platforms present in this study. In this sense, In order to achieve the best coercing among the series after the integration, a joint normalization with quantile normalization is required, with the purpose of obtaining the same dynamical range among the series see .normalizationBetweenArrays function has been accomplished, the dataset is completely ready to perform gene expression analysis. Thus, limma will be used in order to achieve this analysis. Nevertheless, it is important to note that this study is a multiclass gene expression problem. Therefore, it is necessary to use limma with this consideration, avoiding the classical biclass pipeline which limma implements by default.Once normalization using the Once multiclass limma was applied, a list of DEGs were reported. Those genes passed the three imposed restrictions: LFC equal or greater than 2.5, COV equal or greater than 5 and p-value equal or less than 0.001. A total of 42 genes that satisfy these restrictions were returned. Once the DEGs extraction pipeline was done by using the integrated dataset, the feature selection algorithm mRMR was applied, obtaining a ranking of genes in which the most relevant genes would be placed on the first positions within of this ranking. Furthermore, thanks to its operation taking into account the mutual information among the selected genes, this algorithm can also minimize the redundancy among them.Subsequently, the performance of the obtained ranking was evaluated. For that, four different classifiers were implemented and compared. Furthermore, this comparison has been performed for different number of genes  and for both the ACC and the f1-score. As for the simulations of the ANOVA test, a cross-validation process (5-fold) was applied with the objective of providing an estimation of the performance on unseen samples, avoiding overfitting. This restrictive cross-validation process guaranteed a significant representation of the lowest frequent classes  and neutrophils (Azurocidin 1 gene) were also detected in our study. LAPTM4B gene acts downregulating the TGFB1 production in regulatory T-cells . DiffereFinally, both RPS15 and RPS24 genes encode for ribosomal proteins that are component of the 40S subunit. Interestingly, the first one has been found in different studies to appear mutated in chronic lymphocytic leukemia patients, as it lead to impaired p53 stability , 72. TheThe most common mutations associated with AML are in FLT3, NPM1, CEBPA, and TP53 . HoweverAlthough the starting point of CML is well known, other genetic and cytogenetic changes play important roles in prognosis and treatment of this malignancy . In thisThis works introduces an important evolution of the classical multiclass genomic analysis pipeline. Datasets from different technologies and from different platforms have been integrated with the purpose of collecting a higher number of samples due to the lack of RNA-seq samples of leukemia available at public databases. Moreover, this integration ensures the heterogeneity of the study. Furthermore, different types of leukemia series have been selected with the purpose of trying to find relevant biomarkers that allow to discern among the five classes. This study, as far as we are aware, was not performed yet and both, the introduced pipeline at multiclass level and the metrics for the DEGs extraction, are a very novelty step in this field.On one hand, in this multiclass study that considers different types of leukemia, an important new parameter called COV has been used for extracting the DEGs in combination with the classical LFC. This parameter extracts biomarkers that are able to discern one or different classes from the rest using paired combinations. Moreover, the ANOVA test performed has shown that this parameter has been crucial in the development of the study. Therefore, the combination of both the LFC and the COV for multiclass biomarkers selection is an important advance in this field with very promising results.A set of 10 DEGs have been identified as possible genetic signature by using the designed pipeline, and this set of genes has been assessed using a set of machine learning classifiers. On the other hand, the classification results at the multiclass level using the DEGs extracted has shown a high percentage for both the ACC and the f1-score metrics, overcoming the 96% with only a small subset of ten genes. At sight of these results, our DEGs can discern among the five proposed classes and can shape a powerful tool that could be very useful for the clinicians in decision making.Thereafter, the biological study of the small subset of ten genes reveals a strong relationship between nine of the ten genes with the leukemia disease. Concretely, these genes highlighted were related with relevant biological processes such as proliferation, apoptosis or migration among others in peripheral blood leukocytes including B-cells, T-cells and neutrophils. Furthermore, these genes have been previously related to the neoplastic process in the hematopoietic tissue being especially relevant the modulation of the DOCK2 gene which has shown therapeutic implications in leukemic cells.


Identifying epigenetic field defects, notably early DNA methylation alterations, is important for early cancer detection. Research has suggested these early methylation alterations are infrequent across samples and identifiable as outlier samples. Here we developed a weighted epigenetic distance-based method characterizing (dis)similarity in methylation measures at multiple CpGs in a gene or a genetic region between pairwise samples, with weights to up-weight signal CpGs and down-weight noise CpGs. Using distance-based approaches, weak signals that might be filtered out in a CpG site-level analysis could be accumulated and therefore boost the overall study power. In constructing epigenetic distances, we considered both differential methylation (DM) and differential variability (DV) signals. We demonstrated the superior performance of the proposed weighted epigenetic distance-based method over non-weighted versions and site-level EWAS (epigenome-wide association studies) methods in simulation studies. Application to breast cancer methylation data from Gene Expression Omnibus (GEO) comparing normal-adjacent tissue to tumor of breast cancer patients and normal tissue of independent age-matched cancer-free women identified novel epigenetic field defects that were missed by EWAS methods, when majority were previously reported to be associated with breast cancer and were confirmed the progression to breast cancer. We further replicated some of the identified epigenetic field defects. There have been studies in identifying epigenetic field defects , notablyhylation , using mhylation  identifiP-value out of all CpGs in that region. These methods will not be powerful when site-level effects are very small. Alternatively, the distance-based methods accumulate any CpG site-level signals from a gene or a genetic region via the (dis)similarity matrix thus boost the overall association power, making them the ideal methods for detection of epigenetic field defects.Due to the fact that CpG site-level signals for epigenetic field defects may be very small, existing methods based on differences (DM or DV or both) on CpG site-level may not have good power. Standard epigenome-wide association studies (EWAS) that focus on mean signals (EWAS-DM) perform CpG site-level tests to identify differentially methylated CpGs between two experimental groups using standard tests such as a t-test, a regression-based test or its regularized versions , or a noHere, we developed a weighted epigenetic distance-based method to identify epigenetic field defects at gene or genetic-region levels using both DM and DV signals. CpG site-level weights were incorporated in the calculation of (dis)similarity matrix to further boost signals and reduce noises. Specifically, we used original DNA methylation measures to examine DM and centered quadratic methylation measures to examine DV and considered site-level weights based on strengths of site-level DM and DV signals. Simulation studies showed much improved performance of the proposed weighted epigenetic distance-based method over several comparing methods including non-weighted versions and methods that use either DM or DV signals as well as standard EWAS methods. We further demonstrated the performance of the proposed method through an application to the 450K DNA methylation data of normal-adjacent tissue of breast invasive carcinoma (BRCA) patients and normal tissue from independent age-matched cancer-free women from Gene Expression Omnibus (GEO). The proposed method that accumulates weighted DM and DV signals identified genes with epigenetic field defects that were missed by standard EWAS methods and non-weighted distance-based methods. Many of these epigenetic field defects were previously reported to be associated with breast cancer. Further examination confirmed their enrichment in the progression to breast cancer and replicated some of these identified epigenetic field defects.Case-control designs using normal tissue from healthy individuals  and to down-weight noise CpGs in calculating distances between pairs of subjects. Therefore, we define weights for mean and variance signals at CpG site The corresponding We apply distance-based regression originally developed in the field of ecology ,32 to teP-values, we pool pseudo-P-value  correlation among CpGs in a gene with a correlation coefficient 0.5. The detailed information for simulation setup for this scenario is summarized in the Type I error rates are well controlled at the 0.05 significance level in settings with one gene and 10 genes after Bonferroni adjustment for multiple comparisons  using the same GEO BRCA data, we identified 2 DMRs of epigenetic field defects using both mean and variance signals . The twoWe also investigated the relation between the number of CpGs in a gene and the probability that the gene is selected, where we binned genes based on their sizes and calculated the selection probability of a gene in a bin as the proportion of genes identified out of all genes in the bin. We plotted the selection probabilities against gene sizes  and founWe further validated the 21 genes of epigenetic field defects identified by  tissues . In geneAs epigenetic field defects identified in one set of normal vs. normal-adjacent comparison maybe driven by a few ‘outlier’ normal-adjacent samples ,4,21, diP-values, 7 out of the 21 previously identified genes with epigenetic field defects in the discovery analysis using the GEO BRCA data were replicated by DPH3B, NAA35, ANKRD13B, CXCL6, FKBP4, PRSS48 and CFTR. We similarly validated these 7 genes by comparing P-values from the two-sample P-values from the two-sample NAA35 and FKBP4, exhibit progression to tumor. More details of the replication analysis results using At the same 0.0005 threshold for gene-level  samples . All 7 gn = 90). We plotted DNA methylation measures of CpGs in the 7 replicated genes  from the Cancer Genome Atlas (TCGA) project together with their matched tumor samples  under the accession number GSE69914. The 450K DNA methylation data of 18 normal tissue of 18 breast reduction mammoplasty subjects were downloaded from Gene Expression Omnibus (GEO) under the accession number GSE67919. The 450K DNA methylation data of 90 BRCA normal-adjacent and tumor pairs were downloaded from the Cancer Genome Atlas (TCGA) project.Supplementary DataClick here for additional data file.


Patient classification has widespread biomedical and clinical applications, including diagnosis, prognosis, and treatment response prediction. A clinically useful prediction algorithm should be accurate, generalizable, be able to integrate diverse data types, and handle sparse data. A clinical predictor based on genomic data needs to be interpretable to drive hypothesis‐driven research into new treatments. We describe netDx, a novel supervised patient classification framework based on patient similarity networks, which meets these criteria. In a cancer survival benchmark dataset integrating up to six data types in four cancer types, netDx significantly outperforms most other machine‐learning approaches across most cancer types. Compared to traditional machine‐learning‐based patient classifiers, netDx results are more interpretable, visualizing the decision boundary in the context of patient similarity space. When patient similarity is defined by pathway‐level gene expression, netDx identifies biological pathways important for outcome prediction, as demonstrated in breast cancer and asthma. netDx can serve as a patient classifier and as a tool for discovery of biological features characteristic of disease. We provide a free software implementation of netDx with automation workflows. Additionally, human disease‐specific research projects are profiling multiple data types across thousands of individuals, including genetic and genomic assays, brain imaging, behavioral testing, and clinical history from integrated electronic medical records . Computational methods to integrate these diverse patient data for analysis and prediction will aid understanding of disease architecture and promise to provide actionable clinical guidance.The goal of precision medicine is to build quantitative models that guide clinical decision‐making by predicting disease risk and response to treatment using data measured for an individual. Within the next 5 years, several countries will have general‐purpose cohort databases with 10,000 to > 1 million patients, with linked genetics, electronic health records, metabolite status, and detailed clinical phenotyping; examples of projects underway include the UK BioBank  is represented as a patient similarity network PSN; Fig A. Each PWe describe netDx, the first PSN‐based approach for supervised patient classification. In this system, patients of unknown status can be classified based on their similarity to patients with known status. It applies the idea of recommender systems, similar to those used in Amazon or Netflix (“find movies like this one”), to precision medicine (“find patients who don't respond to therapy”). This process is clinically intuitive because it is analogous to clinical diagnosis, which often involves a physician relating a patient to a mental database of similar patients they have seen. As demonstrated below, netDx has strengths in classification performance, heterogeneous data integration, and interpretability.The overall netDx workflow is shown in Fig Each patient similarity network (PSN) is a feature, similar to a variable in a regression model. A PSN can be generated from any kind of patient data, using a pairwise patient similarity measure Fig A and B. et al, In machine learning, the feature selection step identifies the features  with the highest generalizable predictive power. In the case of netDx, predictive features are identified for each class using feature selection. netDx is trained on samples from the class of interest, using cross‐validation Figs  and EV2 et al, Selected features are combined by averaging their similarity scores to produce a single, integrated network for each class. GeneMANIA runs label propagation on each integrated network outward from the query nodes (“+” nodes) to rank all other nodes in the network by similarity to the query Fig . For eacet al, netDx returns the set of selected features for each class, predicted labels for all test patients, and standard performance measures including the area under the receiver operating characteristic curve (AUROC), area under the precision‐recall curve (AUPR), and accuracy Fig C. Class‐http://cancergenome.nih.gov/) via the TCGA PanCancer Survival Prediction project website of Yuan et al (https://www.synapse.org/#!Synapse:syn1710282). These tumor types have been thoroughly analyzed using eight machine‐learning methods, which provide extensive performance results that we can compare to  until we reached a stable AUROC estimate. For all tumor types, netDx demonstrates performance consistently better than, or at par with, other machine‐learning methods  and is not significant for the fourth (LUSC: P = 0.07). Furthermore, the top‐performing netDx model outperforms all eight tested machine‐learning algorithms for kidney, ovarian, and brain cancer . Pairwise comparison of netDx broken down by machine‐learning method shows that netDx has a significantly better median AUROC score in most (75%) cases  cases . PerformN = 348 patients; 154 Luminal A, 194 non‐Luminal A; The Cancer Genome Atlas, et al, Creating a single feature per data type identifies the general predictive value of that data layer but, without further work, does not provide insight into which genes or cellular processes are useful for classification. To better understand disease mechanisms, netDx supports the ability to group gene‐level measures into pathways (gene sets) so that the predictive value of pathway features can be measured. To illustrate this ability, we classified breast tumors as being of the Luminal A subtype or not, using tumor‐derived gene expression (P = 0.013). Top‐scoring pathways included themes of cell cycle progression and checkpoint regulation, DNA synthesis, DNA mismatch repair, and DNA double‐strand break repair . Therefore, top‐scoring features succeed in separating same‐class patients, relative to patients of other classes.Classification performance was high, with an average (SD) AUROC of 0.97 ± 0.01, average AUPR of 0.92 ± 0.02, and average accuracy of 89 ± 3% , and the patient class boundary is visually evident in these features . Pathways that score highly in feature selection and correlate with outcome are good candidates for follow‐up biomarker or mechanistic studies.A common problem with genomic data is relating cohort‐level analysis results, such as the set of affected pathways in Fig et al, et al, P < 2e‐16). These examples demonstrate that when used with pathway‐level features, netDx can provide insight into molecular mechanisms and disease‐related processes that discriminate patient groups. Altogether, our results show that using pathway‐level features can improve classification performance and provide insight into disease mechanisms.As a second case study to demonstrate that netDx identifies pathways consistent with the biology of the condition, we predicted case/control status in asthma using gene expression from sorted peripheral blood mononuclear cells  can improve performance.With the PanCancer benchmark set, netDx performs as well as or better than all machine‐learning methods tested. In the case of KIRC, the basic netDx model outperformed other methods without any additional tuning. Some of this performance gain may be due to the use of regularized regression in GeneMANIA. In the context of predicting gene function, the original GeneMANIA benchmarking tests determined that, relative to unregularized linear regression, regularization improved performance by reducing overfitting . GSEA identified roughly ~1.5 times more pathways . We also propose that users store and publicly share patient similarity networks, useful as features for netDx and other PSN methods, in the NDEx network exchange system . Similarity was defined by Pearson correlation where a data type had more than six measures  and G is the set of all values for the variable . For a set of k variables G={g1,g2,..gk}, where 1 ≤ k ≤ 5, the similarity S' between two patients a and b is defined as the average of normalized similarity for each of the variables: Where eucscale) or Pearson correlation, followed by local exponential scaling  is performed within the cross‐validation loop separately for training and test samples to avoid leaking information from train to test. The lung cancer dataset demonstrated the best performance if the model was also limited to the top clinical variable from lasso (plassoc1).This model design defines features at the level of individual variables  and performs within‐CV prefiltering using lasso regression to rank features by their ability to predict outcome  to create a single network . Visually, the goal is to view similar patients as being more tightly grouped and dissimilar patients as being farther apart. Similarity  is therefore converted to dissimilarity, defined as 1‐similarity. Weighted shortest path distances are computed on this resulting dissimilarity network. The one‐tailed Wilcoxon–Mann–Whitney test is used to ascertain whether within‐class distances are collectively shorter than across‐class distances. To aid visualization, only edges representing the top 40% of distances in the network are included. The weighted shortest path between patient classes (a node set) in the integrated network was computed using Dijkstra's method , NetPath , Reactome , NCI Curated Pathways , and Panther . The compiled set of pathways was downloaded from http://download.baderlab.org/EM_Genesets/February_01_2018/Human/symbol/Human_AllPathways_February_01_2018_symbol.gmt , how many top interactions to include per node (topX), and the upper bound on the number of edges in a network (maxEdges). By default, GeneMANIA sparsifies networks by first excluding edge weights below a cutoff and retaining the top 50 edges per node  improved performance in some models. In the latter instance, we ensured that a network included all input patients (so test patients could be classified) by adding the strongest edge for any patients excluded by the maxEdge criterion. If this edge weight was lower than the network cutoff, it was set to be at the cutoff value. Different tumor datasets performed optimally with different levels of sparsification: KIRC: topX = 30, maxEdges = 6,000; LUSC: topX = 40, maxEdges = 6,000; GBM: topX = 50, maxEdges = 3,000.Sparsification is useful for reducing the number of edges to be considered in input networks, speeding computation, and for reducing noise in the data, as weak correlations are removed. Sparsification uses three parameters: the minimum edge weight to include  in Cytoscape 3.6.1 , we ranked the limma output based on the absolute value of the moderated t‐statistic. A weighted pre‐ranked GSEA  and netDx pathways that consistently had high scores across multiple train/test splits (maxScore ≥ 7). This enrichment map was created to assess overlaps in pathways and themes (where a theme is a group of related pathways). We used the Enrichment Map 3.1.0 app   or those with more than 5 miRNA . The design matrix was a 2 × 2 matrix with 0 in the diagonals and 0.1 in the off‐diagonal. Block.splslda was used to estimate the number of components to use; the most frequent estimate of number of components (median choice.ncomp$WeightedVote) was used as the number of components in the model. A grid‐based search was performed to estimate the number of variables to retain for each omic layer and component. Specifically, tune.block.splsda was used with 10‐fold cross‐validation to ascertain the number of variables to keep (between 5 and 30 variables). DIABLO was used to select variables for each component.We ran DIABLO with a gene‐centered model, following operating procedures specified in the online tutorial (All authors contributed to netDx method development. SP and MAS analyzed the data. SP wrote the netDx software package with contributions from MAS. SH, HK, and RI developed initial versions of netDx. SP and GDB wrote the paper. GDB supervised the work.The authors declare that they have no conflict of interest.AppendixClick here for additional data file.Expanded View Figures PDFClick here for additional data file.Dataset EV1Click here for additional data file.Dataset EV2Click here for additional data file.Dataset EV3Click here for additional data file.Dataset EV4Click here for additional data file.Dataset EV5Click here for additional data file.Dataset EV6Click here for additional data file.Review Process FileClick here for additional data file.


We develop a method, VIPER, to impute the zero values in single-cell RNA sequencing studies to facilitate accurate transcriptome quantification at the single-cell level. VIPER is based on nonnegative sparse regression models and is capable of progressively inferring a sparse set of local neighborhood cells that are most predictive of the expression levels of the cell of interest for imputation. A key feature of our method is its ability to preserve gene expression variability across cells after imputation. We illustrate the advantages of our method through several well-designed real data-based analytical experiments.The online version of this article (10.1186/s13059-018-1575-1) contains supplementary material, which is available to authorized users. Single-cell RNA sequencing (scRNAseq) technique is becoming increasingly popular in transcriptome studies –5. WhileWhile scRNAseq holds great promise in studies with complex cellular compositions, it also suffers from several important technical disadvantages that limit its use in many settings. These disadvantages include low transcript capture efficiency, low sequencing depth per cell, and wide-spread dropout events, to name a few –23. As aSeveral imputation methods have been recently proposed to address the challenges resulted from excess zero values in scRNAseq –27. ScRNWhile existing imputation methods have yielded promising results, they also have important drawbacks. For example, methods such as MAGIC perform imputation based on a low-dimensional space projected from the data, but imputation on a low-dimensional space will likely eliminate gene expression variability across cells and thus abolish a key feature of single-cell sequencing data , 26. As Here, we describe a straightforward, accurate, free-of-tuning, and relatively computationally efficient scRNAseq imputation method, which we refer as the Variability-Preserving ImPutation for Expression Recovery (VIPER). VIPER borrows information across cells of similar expression pattern to impute the expression measurements in the cell of interest. However, unlike some of the previous cell-based imputation methods, VIPER does not perform cell clustering before imputation nor uses only cells that belong to the same cell subpopulation for imputation. Instead, VIPER applies a sparse nongenerative regression model to actively select a sparse set of local neighborhood cells that are most predictive of the cell of interest. The selection of this sparse set of cells is done in a progressive manner, and their associated imputation weights are estimated in the final estimation step to ensure both robustness and computational scalability. In addition, VIPER explicitly accounts for expression measurement uncertainty of the zero values in scRNAseq by modeling the dropout probability in a cell-type-specific and gene-specific fashion. VIPER uses an efficient quadratic programing algorithm that infers all modeling parameters from the data at hand while keeping computational cost in check. A key feature of VIPER is its ability to preserve gene expression variability across cells after imputation. We apply our method and compare it with existing imputation methods in several real scRNAseq data-based analytical experiments. We show that, compared to existing methods, VIPER achieves better imputation accuracy, preserves gene expression variability across cells, recovers gene expression measurements that better resemble the bulk RNAseq measurements in the same cell type, and facilitates more reproducible differential expression analysis.Ci, j as the observed gene expression count for ith cell and jth gene, with i ∈ {1, ⋯, n} and j ∈ {1, ⋯, m}. We denote ith cell and obtained normalized gene expression levels in terms of RPM (reads per million reads) defined as Xi, j as the normalized expression level obtained from RPM values by further performing a log transformation Xi, j = log. For imputation, we examine one cell at a time. For ith cell, we assume that its normalized expression level for the jth gene in expectation can be expressed as a summation of the expression levels of the same gene across all other cellsbi, l is the predictive effect of lth cell on ith cell. Note that we only specify a mean model as we only intend to perform single imputation. Single imputation of the mean does not require a full modeling specification for the response variable Xi, j . Thresholding weights allows us to identify a small set of neighborhood cells for imputation. With hard thresholding and nonnegative weight constraint, our model becomes effectively a sparse nonnegative regression model [lth cell is a neighborhood of ith cell does not guarantee that ith cell is also a neighborhood of lth cell. This asymmetric pattern ensures that the identification of neighborhood cells is optimal for each cell. Once we obtain the parameter estimates Xi, j in turn, we then plug in these estimates to obtain a predicted value We assume that the predictive effects on model . Note thbi, l from the above mean model using ordinary least squares with a quadratic programming algorithm. Specifically, we re-formulate parameter estimation problem in the above model to an optimization problem, where, for each cell i in turn, we aim to obtain a set of ith cell-specific prediction weights  from all other cells by minimizing the sum of square prediction errors forbi, l satisfy the constraint that bi, 1 + ⋯ + bi, i − 1 + bi, i + 1 + ⋯ + bi, n = 1, with the non-zero effects being above a certain threshold of t = 0.001. To optimize the above function, we denote Wi, j = Xi, j − Xn, j, Yl, j = Xl, j − Xn, j, and β = T. The above constrained optimization problem can be re-expressed as a quadratic programming problemIn − 1) × (n − 1)( denotes a (n − 1) by (n − 1) identity matrix, 1n − 1)( denotes a (n − 1)-vector of 1s, and 0n − 1)( denotes a (n − 1)-vector of 0s. We solve the optimization problem using quadratic programming [t = 0.001 to be exactly zero to ensure sparsity. Finally, we re-normalize the non-zero weights to ensure a summation of one.We estimate the predictive effect parameters gramming . Once wei in turn, we apply standard penalized regression model  using a random sampled set of 5,000 genes to identify a set of candidate cells that are predictive of the expression of the ith cell. Among these candidate cells, we apply the quadratic programming algorithm described previously to further identify a set of neighborhood cells for final imputation. Therefore, our imputation method eventually consists of two steps: a lasso/elastic net-based pre-selection step and a quadratic programming algorithm-based fine tuning and estimation step. With two separate steps, our method ensures computational scalability while avoiding model overfitting by sequentially reducing model complexity.While the above quadratic programming algorithm is effective, we find that the algorithm is computationally inefficient and does not scale well to large-scale scRNAseq data sets. To ensure algorithm scalability and avoid model overfitting, we perform a pre-selection procedure to first select a small set of candidate cells that will be eventually selected as local neighbors. Specifically, for each cell Cl, j is due to a dropout event, then it is not an accurate measurement of the true expression level of jth gene in lth cell. Subsequently, we do not wish to use the normalized value Xl, j from a dropout event to impute Xi, j. However, if the zero value of Cl, j comes from low or zero expression level of jth gene in lth cell, then we would want to use the normalized value Xl, j to impute Xi, j. To distinguish between these two possibilities, we estimate an expected expression level for any zero value of Cl, j and use these estimates to perform imputation. The modeling and estimation details for this dropout adjustment step are provided in detail in the Additional file jth gene for all selected neighborhood cells for the ith cell of interest follow a zero-inflated Poisson mixed model, such that Cl, j~pi, jδ0 + PMM. In the model, pi, j represents the dropout probability of jth gene that is specific for all neighborhood cells of the ith cell of interest; δ0 denotes a point mass at zero; Nl is the total read depth for the lth cell; λl, j is the Poisson rate parameter that represents the expression level of jth gene in the lth cell; ψi, j is an over-dispersion parameter that is specific for jth gene and for all neighborhood cells of the ith cell of interest; and PMM denotes a Poisson mixed effects model. Under the zero-inflated Poisson mixed effects model, Cl, j is exactly zero with a dropout probability pi, j and follows an over-dispersed Poisson distribution with probability 1 − pi, j. Our goal is to estimate λl, j, the underlying expression level of jth gene in lth cell, to serve as our final predictor variable for all zero values of Cl, j. To do so, we first estimate all parameters  through an expectation maximization (EM) algorithm based on the selected neighborhood cells for the ith cell of interest. Afterwards, we obtain an estimate of Xl, j to serve as the final predictor variable. Certainly, we use the normalized expression measurement Xl, j directly as the predictor variable if Cl, j is non-zero.Finally, we note that a zero count can be generated by two possible mechanisms: it either comes from a dropout event or represents a low or zero level of gene expression. If the zero value of We refer to our method described above as the Variability-Preserving ImPutation for Expression Recovery (VIPER). We note that the non-statistical term “variability preserving” in the method name refers to the fact that our method is capable of preserving gene expression variance across cells after imputation, as we will show in the following real data-based analytic experiments. The property of “variability preserving” in our method contrasts a few other imputation methods that aggressively reduce variance across cells after imputation . However, we also acknowledge that, just like any existing single-cell imputation method, our method is a single imputation method that suffers from the usual drawbacks when compared to other more advanced imputation methods such as multiple imputation .We examine four scRNAseq data collected from three studies. These three studies include both unique molecular identifier (UMI)-based techniques  and non-UMI-based techniques .Specifically, the first data set is from Grun et al. . It is aThe second and third data are both from Chu et al. . The secThe fourth data is from Shalek et al. . It contWe performed three different experiments using published scRNAseq data. Two experiments (masking and comparison to bulk RNAseq) are described in detail in the “In the first step, for each gene in turn, we randomly sampled gene expression values across all cells based on a multinomial distribution. In this multinomial distribution, the cell-specific probability parameters are set to be the corresponding cell expression proportion in the original data, while the total read count parameter is set so that the down-sampled read depth represents a fixed proportion of the original read depth . The multinomial distribution ensures that the expected gene expression proportion of each cell remains unchanged after down-sampling.The down-sampled data from the first step contains both zero values and non-zero values. After the initial down-sampling step, we further introduce extra dropout events to the non-zero values as a second step of the down-sampling experiment, in order to mimic data generating process of real scRNAseq data. These dropout events are introduced to each of the non-zero values by sampling from a Bernoulli distribution characterized by a dropout rate. The dropout rate is designed using two different strategies. In the first strategy, we set a fixed dropout rate of 0.8 that is independent of the non-zero value from the initial multinomial sampling step. In the second strategy, we model the dropout rate as a logistic function of the non-zero down-sampled value. This logistic function is estimated in the original data for each cell subpopulation separately. Specifically, within each cell subpopulation, for each gene in turn, we obtained the percentage of zero values and the mean of non-zero values across all cells. Treating the zero percentage as outcome and the non-zero means as explanatory variable, we fitted a logistic regression model to establish their quantitative relationship in each cell subpopulation. Afterwards, we used the fitted logistic model to compute a dropout rate for each non-zero value in the data obtained from the initial down-sampling step. With either strategy, we set each non-zero value to be zero with a probability equal to the dropout rate. Therefore, the zero values in the final down-sampled data are either due to low expression values in the original data and the subsequent multinomial down-sampling  or due to the extra dropout events .We examined several down-sampling scenarios based on various down-sampling rates. In each scenario, we performed imputation on the zero values resulting from either multinomial down-sampling or from the additional dropout events. We measured imputation accuracy by comparing the imputed data to the original data across all entries. Note that the zero values that were already present in the original data were not analyzed here because we do not know their “true” values in the original data.F test, QLF), and SCDE [We compared our method with four existing imputation methods. These imputation methods include DrImpute  version” sectionand SCDE   DrImpute, which relies on pre-identified cell subpopulations for imputation ; (2) MAGFirst, we assess imputation performance of different methods in recovering randomly masked non-zero gene expression values. To do so, in each of the four scRNAseq data , we randomly selected a fixed percentage  of non-zero entries in the observed data matrix and masked these values to be zero to generate a new gene expression matrix. We then apply different methods to the newly generated gene expression matrix and compute the correlation between the imputed values and the masked values across all entries as a measurement of imputation accuracy. For each data set, we perform 10 masking replicates and plot the results across these replicates in Fig. Second, we assess the performance of different imputation methods using two down-sampling experiments. The procedure of the down-sampling experiments is described in detail in the “For the zeros due to dropouts, VIPER outperforms most other imputation approaches across the four different data sets and the two different dropout rate settings: it is ranked as the best method in six out of the eight scenarios examined. The only exceptions are Time Course data as well as the Grun et al. data with an expression-dependent sampling rate , where, in the later case, while VIPER outperforms SAVER when the down-sampling rate is high (0.6 to 0.95), it performs slightly worse than SAVER in the presence of a low down-sampling rate (0.5). The performance of our method in other settings is generally followed by SAVER and then scImpute, while DrImpute and MAGIC do not work well. For example, in the Cell Type data, when the down-sampling rate is 0.5, the correlation between the imputed values in the down-sampled data and the truth in the original data is 0.830 using lasso and 0.832 using elastic net by VIPER, while the correlation by DrImpute is 0.572, by MAGIC is 0.266, by SAVER is 0.828, and by scImpute is 0.522. The rankings of different methods also remain largely the same when we use different down-sampling rates or alternative down-sampling strategies.For the zeros due to low expression abundance and multinomial down-sampling, VIPER outperforms all other imputation approaches except for SAVER across the four different data sets and the two different dropout rate settings. VIPER and SAVER were each ranked as the best method in four out of the eight scenarios. In particular, VIPER produces better results than SAVER in three out of four data sets when we use expression-independent dropout rates and produces better results in one out of four data sets when we use expression-dependent dropout rate. The good performance of SAVER in half of the scenarios for imputing zeros due to low expression abundance is presumably due to the fact that the imputed values from SAVER do not differ much from the unimputed data in general (which will become apparent in the next section); thus, the imputed values from SAVER for these zeros due to low expression abundance remain close to zero. The performance of our method and SAVER is followed by either DrImpute or scImpute, depending on the data set, while MAGIC does not work well here. For example, in the Cell Type data, when the down-sampling rate is 0.5, the L1 loss between the imputed values in the down-sampled data and the truth in the original data is 0.405 (for either lasso or elastic net) by VIPER, while the L1 loss by DrImpute is 0.686, by MAGIC is 4.182, by SAVER is 0.514, and by scImpute is 0.857. The rankings of different methods remain largely the same when we use different down-sampling rates.Therefore, consistent with the masking experiments, the down-sampling experiments here also suggest that VIPER is capable of accurately recovering the true expression levels in real data.Third, we assess the performance of different methods by comparing the imputed gene expression values from scRNAseq to the expression values measured by bulk RNAseq in the same cell subpopulation. To do so, we rely on the Cell Type and Time Course data from Chu et al.  that alsTo quantify the performance of different imputation methods in terms of recovering the mean expression level of each cell type, we averaged the expression levels across all cells within a cell subpopulation to obtain an averaged gene expression measurement in the imputed scRNAseq data. For each cell subpopulation in turn, we then computed the correlation between bulk RNAseq and the imputed mean scRNAseq measurements across all genes Fig. . Among tTo quantify the cross-cell gene expression variability in the imputed data sets, for each gene in turn, we computed the coefficient of variation (CV) across cells after imputation and compared it to the CV of the non-zero values before imputation. We contrasted these two CV values for DEC cells in Cell Type data set and stratified the contrast by showing different zero proportions Fig.  or diffeImportantly, CV plots in the other three data sets  display similar patterns  to analyze each subset separately. We then computed the proportion of overlap between the top 100, 200, 500, or 1000 DE genes detected from each subset—and we compute this proportion as Jaccard index, defined as the ratio of the intersection and the union between the top gene lists from the two subsets. We performed the random data split 10 times and show the mean overlap proportions of DE genes detected by SCDE from these replicates in Fig. One thing we noticed in our analysis is that many genes are detected as DE in MAGIC imputed data  from the Cell Type data. Specifically, for each cell type, we display cells based on the top two principal components (PCs) extracted from either the raw data or the imputed data by different methods  is to select genes that have similar expression levels as the gene of interest to perform imputation. Our method can be easily adapted to use neighborhood genes to impute missing data. However, in our experience, we find that using neighborhood cells is often more accurate than using neighborhood genes to perform imputation. To illustrate the difference in accuracy between these two different strategies, we perform a simple analysis using the Grun et al. and Cell Type data from Chu et al. . SpecifiLike all other existing imputation methods –27, we hAdditional file 1:Details of modeling for the dropout event adjustment and method comparison to scImpute. (PDF 200 kb)Additional file 2:Additional figures for performance evalution of VIPER. (PDF 13470 kb)Additional file 3:Tables S1 and S2. Percentage of zero values in the downsampling experiments and computating time comparisons. (PDF 150 kb)


In the Next Generation Sequencing (NGS) era a large amount of biological data is being sequenced, analyzed, and stored in many public databases, whose interoperability is often required to allow an enhanced accessibility. The combination of heterogeneous NGS genomic data is an open challenge: the analysis of data from different experiments is a fundamental practice for the study of diseases. In this work, we propose to combine DNA methylation and RNA sequencing NGS experiments at gene level for supervised knowledge extraction in cancer.We retrieve DNA methylation and RNA sequencing datasets from The Cancer Genome Atlas (TCGA), focusing on the Breast Invasive Carcinoma (BRCA), the Thyroid Carcinoma (THCA), and the Kidney Renal Papillary Cell Carcinoma (KIRP). We combine the RNA sequencing gene expression values with the gene methylation quantity, as a new measure that we define for representing the methylation quantity associated to a gene. Additionally, we propose to analyze the combined data through tree- and rule-based classification algorithms .We extract more than 15,000 classification models (composed of gene sets), which allow to distinguish the tumoral samples from the normal ones with an average accuracy of 95%. From the integrated experiments we obtain about 5000 classification models that consider both the gene measures related to the RNA sequencing and the DNA methylation experiments.We compare the sets of genes obtained from the classifications on RNA sequencing and DNA methylation data with the genes obtained from the integration of the two experiments. The comparison results in several genes that are in common among the single experiments and the integrated ones  and 509 genes that are in common among the different experiments. Finally, we investigate the possible relationships among the different analyzed tumors by extracting a core set of 13 genes that appear in all tumors. A preliminary functional analysis confirms the relation of part of those genes (5 out of 13 and 279 out of 509) with cancer, suggesting to focus further studies on the new individuated ones.The online version of this article (10.1186/s13040-018-0184-6) contains supplementary material, which is available to authorized users. Next Generation Sequencing (NGS) techniques have revolutionized the sequencing of genomes, producing large quantities of DNA and RNA data –4. This In this work, we are going to focus on DNA methylation and RNA sequencing, as these two NGS experiments have been proven to play an important role in knowledge discovery in cancer –25.beta value [DNA methylation is one of the most studied epigenetic changes in human cells. The changes in DNA methylation patterns are crucial in the development of diseases and in many forms of cancer –31. Mostta value , and is ta value , 33.RNA sequencing is a next generation sequencing technique for the analysis of the transcriptome and its quantification. Four main methods for measuring gene expression are used in practice: i) Reads Per Kilobase per Million mapped reads (RPKM); ii) FraNGS data the information extracted from a NGS experiment , e.g., the counts of the reads that map on given list of genes in RNA sequencing. We define NGS meta data the information related to the NGS experiment and the sequenced tissue, e.g., the tissues status , or the sequencing depth. We define NGS data integration the procedure of joining different experiments (possibly extracted from heterogeneous databases) sharing common features  in order to extract knowledge. The aim of integration is to aggregate genomic data in an unique schema that provides querying capabilities for retrieving data from a multitude of heterogeneous experiments and databases. Heterogeneous data are the first problem of NGS, because the structure of data is different in diverse experiments and can be different in diverse databases. Therefore, the term integration in NGS data can have different meanings [In this work, we define meanings . On one meanings  with theIn order to have access to the right resources, it is necessary to define standard schemas of these data to avoid redundant information overlaps. Several efforts have been made on NGS data formats and standards. The authors of  provide gene methylation quantity. We combine RNA sequencing and DNA methylation data of The Cancer Genome Atlas (TCGA) [In this work, we address the issue of combining RNA sequencing and DNA methylation experiments, which have different data schemas containing heterogeneous information. Our aim is to obtain a gene oriented organization of both experiments, and therefore we define a new measure on DNA methylation data called s (TCGA)  and testAdditionally, we analyze the combined data by means of supervised classification algorithms, extracting classification models, which are able to distinguish the samples in two classes  and which are composed of features that represent the genes related to the disease and the different NGS experiment.In cancer research many computational methods deal with classification problems, e.g., disease characterization, prognosis, treatment response of patients, mutation pathogenicity, biomarker prediction, and sample malignancy. A recent effort has achieved good performance in the assignment of disease subtypes and malignancy labels to melanoma images with convolutional neural networks . Furtherhttp://dmb.iasi.cnr.it/camur.php as stand alone software; for a comprehensive description we point the reader to [Among them, we focus on a new supervised learning method that is able to extract more knowledge in terms of classification models than state of the art ones, called Classifier with Alternative and MUltiple Rule-based models (CAMUR) . CAMUR ieader to .In this work, thanks to the application of machine learning algorithms, we show the advantage of combining DNA methylation and RNA sequencing data, i.e., the increase of extracted knowledge resulting in combinations of genes from both experimental strategies. Finally, we study the three types of cancer and identify sets of relevant genes. The intersection of them results in a smaller set of genes that should be considered for further investigation.In this section, we discuss the methods used to combine the genomic experiments  and the classification algorithms used to extract knowledge from them. We start by describing the source where we extract the data.http://gdc.cancer.gov/).The Cancer Genome Atlas (TCGA)  is a proIn TCGA each tissue of DNA methylation is represented with a list of following fields: gene symbol, chromosome and genomic coordinates (where the methylation occurs), and its beta value . RNA sequencing data instead contains information on the RSEM values  measuredn samples (tissues) each one with m features (genes) and a class label (condition), which indicates whether the sample is normal or tumoral. A data matrix is composed by n vectors as Fi = , which represent sample i, where fi,j ∈i=1,...,n; j=1,...,m; fi,c ∈ {normal,tumoral}. When considering RNA sequencing, the rows represent the samples, the columns the genes (except the last that represents the class labels) and the items of the matrix contain the RSEM gene expression values for each gene. The structure of this matrix is shown in Table s methylated sites, l of them belonging to a given gene. For aggregating the methylation quantity at gene level, we consider the sum of the beta values as a measure of the overall intensity of the methylation on a gene. Let aijh be the methylation quantity associated to the sample i with i=1,..,n, to the gene j with j=1,....m, and to the methylated site h with h=1,..,l. Then we have gene methylation quantity. It is worth noting that we consider the beta values of CpG sites with a related gene symbol, i.e., the symbol of the gene where the methylation occurs. If a methylation occurs on other genomic regions it is not considered in our data processing procedure, whose aim is to provide a gene oriented data organization. In Table http://bioinf.iasi.cnr.it/genint. The flowchart that reports the computational steps of the software is depicted in Fig. We create data matrices of RNA sequencing and DNA methylation experiments in the following way. Consider i be the i-th sample, j the j-th gene, with i=1,..,n and j=1,..,m in case of a gene of the DNA methylation experiment, j=m+1,....,z in case of a gene of the RNA sequencing experiment. Furthermore, we have bi,j and vali,j ∈j,i, where the “?” symbol means that there is no value associated to the gene j for the sample i. We release also a software that is able to perform the combination of the experiments, available at http://bioinf.iasi.cnr.it/genint. Finally, we report the main steps of the procedure in Fig. In order to perform our analysis on both gene oriented measures (RSEM for RNA sequencing and gene methylation quantity for DNA methylation) at the same time, we propose a combination of these two experiments by applying an intersection of the matrices on common sample IDs and a union of those not in common (this result trace over the full outer join in SQL language), keeping both experimental data and performing a union of the genes that are present in RNA sequencing and DNA methylation as features. The resulting matrix is shown in Table entclass1pt{minimaThe classification that we perform has the objective of being able to determine a set of rules for each type of tumor , which can define if a tissue is in tumoral or in normal condition.It is worth noting that also in previous works –58 DNA mFor performing the task of knowledge discovery, we used four different classification algorithms that are briefly described in the following.C4.5  is an alRandom Forest  is an enRepeated Incremental Pruning to Produce Error Reduction (RIPPER)  is an alClassifier with Alternative and MUltiple Rule-based models (CAMUR)  is a mulIn this section, we describe the performed experiments to test our method and the results of the classification algorithms applied to the RNA sequencing and DNA methylation data of three cancer types. We extracted all the samples of these experiments from TCGA, considering the Breast Invasive Carcinoma (BRCA), the Kidney Renal Papillary Cell Carcinoma (KIRP), and the Thyroid Carcinoma (THCA). For data extraction we used the TCGA2BED tool  and its J48 (C4.5), -C (confidence threshold for pruning.) 0.1, -M (minimum number of instances per leaf) 1, -U (use unpruned tree) false;JRip (RIPPER), -F (the number of folds for Reduced Error Pruning) 5, -N  1, -O (the number of runs of optimizations) 2 -S (the seed of randomization) 1;RandomForest, -I (number of iterations) 30, -K (number of attributes to randomly investigate) 0, -S (seed for random number generator) 1, -num-slots (number of execution slots) 1.The data matrices of the different experiments and tumors have been analyzed with the above-mentioned classification algorithms  through the use of the Weka software package . For theF-R stands for Recall and P is for Precision. Considering True Positives (TP) objects of a given class recognized in this class; False Positives (FP) objects recognized in a class but not belonging in this class; True Negatives (TN) objects not belonging and not recognized in a given class; False Negatives (FN) objects in a given class but not recognized in that, we can then define In addition, we performed the classifications with multiple rule-based models obtained by CAMUR. Therefore four different classification algorithms were applied on three data matrices  of each considered cancer, resulting in 36 different knowledge discovery analyses. For evaluating the classifiers we take into consideration the F-measure, which is defined as ed above  for its In Table The results obtained on the combined datasets are slightly lower due to the increase in features and missing values that make the job of the classification algorithms harder. In order to clarify this point we also applied the classification algorithms on the combined matrices, deleting the samples for which only one NGS experiments is available. In this way we reduced the missing values and the resulting classification performance (F-measure) improved with all the classifiers .http://bioinf.iasi.cnr.it/genint.The performance of the algorithms are important in order to validate the classification, but the main purpose of the work is to extract more and different genes from diverse experiments. The improvement given by the classification of the combined data is that the resulting classification models do not only consider the genes and their associated values for a single experiment, but both from gene expression and DNA methylation data in a single model, providing multiple related genes. In Table We also investigated if the algorithms misclassify the same samples by comparing the predictions of each one. We found out that only some instances are misclassified by all the three algorithms. Further details are described in Additional file Finally, in order to prove the validity of the extracted models we performed random permutations of class membership for each classification problem and each combination. We tested if our procedure is able to extract meaningful classification models regardless of the class partition imposed on the training set. This would be verified only in the presence of a marked overfitting behavior. For validating our results and the extracted classification models, we applied the procedure to data with random permutations of class labels. This validation test was performed on 100 different random permutations for each classification problem. In particular, we obtain low values of F-measure and we report the resulting averages in Table http://bioinf.iasi.cnr.it/genint. In Table http://bioinf.iasi.cnr.it/genint. In Table We ran more than 2000 classification procedures with CAMUR, obtaining rules, literal and conjuncion lists, feature pairs and literals statistics for each tumor and each considered dataset. Detailed results are described in Additional file This section is organized as follows. Firstly, we provide evidence of proof about our new defined measure called gene methylation quantity. Then, we discuss the correlation among RNA sequencing and DNA methylation. Finally, we focus on the extracted knowledge, provide an overview of it, and, in order to prove the validity of the obtained classification models, we considered two external breast cancer RNA sequencing datasets of Gene Expression Omnibus (GSE56022 and GSM1308330) for identifying the diseased samples.In previous studies, efforts have been made for aggregating DNA methylation at gene level. In  a methylAn interesting problem is to investigate if there is correlation between gene expression and DNA methylation. The authors of –80) addrhttp://bioinf.iasi.cnr.it/genint. In the leaves of the trees the total weight of instances reaching that leaf, and the total weight of misclassified instances are specified. In each leaf a fractional weight representing the instances with a missing value is considered. We can then see how missing values are handled by comparing the Tables We extracted a classification model for each experiment and each cancer with C4.5, resulting in 9 decision trees composed of 26 genes (16 for DNA methylation and 10 for RNA sequencing). We show some examples on the Kidney Renal Papillary Cell Carcinoma (KIRP) data, in Table Finally, we validated the tree-based RNA-sequencing classification models on two external datasets extracted from Gene Expression Omnibus (GSE56022 and GSM1308330), obtaining 90% correct classification.We applied Random Forest to all matrices, extracting 9 classification models, each one composed of 30 trees. The total number of genes obtained is 2301 for RNA sequencing and 2574 for DNA methylation of which 306 are in common between two experiments. Thanks to their combination, we extracted 2471 genes with the execution of this algorithm. As example, we show in Table The Random Forest algorithm is particularly suited for knowledge extraction on combined data (which presents a high number of features), because of its randomized and multiple model extraction.For the validation of the classification models on the external datasets GSE56022 and GSM1308330, we consider all trees generated for the RNA-sequencing experiment, and the samples are classified with an average accuracy of 80%.http://bioinf.iasi.cnr.it/genint.The RIPPER algorithm provides 9 rule-based classification models composed of 38 genes, 22 for DNA methylation and 16 for RNA sequencing. Below we show some examples of the rule-based classification models obtained with the RIPPER algorithm on the Kidney Renal Papillary Cell Carcinoma (KIRP). We show some rules for the DNA methylation dataset in Table The rules in the resulting model obtained from the combined matrix, confirm the added value. In this model we can find the same rule obtained from the single DNA methylation data, enriched with a new rule-based on a feature derived from RNA sequencing data.We also applied the rule-based RNA-sequencing model extracted on the Gene Expression Omnibus datasets, obtaining a correct classification rate of 90% on average.http://bioinf.iasi.cnr.it/genint. In this subsection, we show some example of the rules obtained through the execution of CAMUR on the Thyroid Carcinoma (THCA) data. CAMUR extracts many multiple classification models (available at http://bioinf.iasi.cnr.it/genint), as example we report only those with the highest level of accuracy in Table By running more than 2000 classification procedures, we extracted 15.252 rules composed of 1758 genes from RNA sequencing and 3655 genes from DNA methylation. From those genes 509 are in common in both experiments. The reader may find the gene lists in Additional file The rules for the combined data classification model confirm what is derived from the model for DNA methylation data, and also provide a further classification rule-based on RNA sequencing genes that is not obtained by performing classification on data of single experiment of RNA sequencing. It is worth noting that CAMUR can be successfully adopted for knowledge extraction on combined data (which presents a high number of features), because it is able to extract multiple rule-based models.For the validation of CAMUR we selected ten BRCA rules extracted by CAMUR and used them on the Gene Expression Omnibus datasets, noting that with these rules 80% of the samples are correctly classified, confirming the validity of the extracted models.In this subsection we analyze the large quantity of classification models extracted by CAMUR, focusing on the sets of genes that occur in the rules. We summarize the common genes that appear in the different tumors and experiments with six Venn diagrams, which report the intersections of the genes among the considered tumors and the intersections among each considered experiment.In Fig. http://bioinf.iasi.cnr.it/genint we provide all the gene lists that are in common and not in common for each tumor and experiment. Thanks to the intersections, we are able to detect 509 genes in common among DNA methylation and RNA sequencing experiments. From these genes we extract a subset of 13 genes, which are in common among the different tumors. In order to check biological relevance of the obtained subset of genes, we compared our result with the Entrez Gene database of NCBI [In Fig.  of NCBI , which pgene methylation quantity for denoting the quantity of methylation associates to each gene. After the combination of RNA sequencing and DNA methylation, we proposed the application of supervised analyses. We were able to extract many classification models containing the genes and their quantification values by applying different supervised algorithms . The classifications were performed on all matrices for each tumor, with the objective to obtain models to separate the normal from the tumoral samples. All the classification models have an accuracy greater than 95%. In particular we obtained 9 decision trees with C4.5, 9 rule-based classification models with RIPPER, and 9 classification models (each one composed of 30 decision trees) with Random Forest. Moreover, thanks to the execution of more than 2000 classification procedures with CAMUR, we extracted 15.252 classification models, from which we derived 5413 genes related to DNA methylation and RNA sequencing. 509 genes are in common among the different experiments and 13 genes among the different tumors. Through the NCBI Entrez gene database we performed functional analysis of those genes. We found 279 out of 509 and 5 out of 13 of them already marked as oncogenes. CAMUR was applied in order to extract possible oncogenes and to find new ones. Many of the extracted genes have been already classified as oncogenes, and this confirms that our method is able to identify relevant genes and justifies further analyses [In this work, we described the integration of Next Generation Sequencing experiments by distinguishing four cases: (i) integration of data that represent the same NGS experiments, stored in different databases with different schemas; (ii) integration of data that represent different NGS experiments, stored in different databases and with no standardization of schema allowing the access to them; (iii) integration of data that represent different NGS experiments, stored in the same database and with distinct schemas. We also individuated an ideal case for integration (iv), i.e., when a standardization of the schema is provided in order to permit the interoperability between different experiments, with different schemas in different or same databases. We proposed a method for the combination of two distinct NGS experiments with different data schemas. The NGS experiments considered in this study were DNA methylation and RNA sequencing and were extracted from TCGA. We focused on three forms of tumors, i.e. BRCA, KIRP, and THCA. We defined the data matrices, one for each NGS experiment, with samples in the rows and genes in the columns, and a third matrix for representing the combination of DNA methylation and RNA sequencing samples. In particular the objective of the combination was the creation of data matrices indexed on the genes that are related to both NGS experiments. In the RNA sequencing matrices the items are the RNA-Seq by Expectation-Maximization (RSEM) values that quantify the gene expression, whereas in DNA methylation matrices we defined a new measure based on the beta value, that we call analyses . Indeed,Additional File 1Description of the supplementary material. (PDF 99 kb)Additional File 2Detailed results. (RAR 6710 kb)