RNA sequencing (RNA-Seq) enables the measurement and comparison of gene expression with isoform-level quantification. Differences in the effect of each isoform may make traditional methods, which aggregate isoforms, ineffective. Here, we introduce a variance component-based test that can jointly test multiple isoforms of one gene to identify differentially expressed (DE) genes, especially those with isoforms that have differential effects. We model isoform-level expression data from RNA-Seq using a negative binomial distribution and consider the baseline abundance of isoforms and their effects as two random terms. Our approach tests the global null hypothesis of no difference in any of the isoforms. The null distribution of the derived score statistic is investigated using empirical and theoretical methods. The results of simulations suggest that the performance of the proposed set test is superior to that of traditional algorithms and almost reaches optimal power when the variance of covariates is large. This method is also applied to analyze real data. Our algorithm, as a supplement to traditional algorithms, is superior at selecting DE genes with sparse or opposite effects for isoforms. The availability of massively parallel transcriptome sequencing technology has improved our understanding of the central dogma processes of transcription and translation and the molecular mechanisms of complex diseases . RNA seqThe elementary problem of transcriptomic data analysis is accurate identification of differentially expressed (DE) genes . EmerginCurrently, the idea of a set test, which supposes that some related variables form one set and tests the set, likely overcomes the drawbacks of gene-level measurement methods and the disadvantage of testing isoforms individually. The theoretical basis is that the score statistic of the variance component in a generalized linear mixed model (GLMM) follows the mixture chi-squared distribution with the null hypothesis . The seqConsidering the disadvantages of traditional algorithms in selecting heterogeneous genes and the accessibility of the set test, we apply the idea of the set test to identify DE genes in isoform-level measurements. This study involved three parts. First, the relationship between the expression level of one gene and the phenotype is formulated as a GLMM with the assumption of a Poisson and negative binomial (NB) distribution. Second, we study the empirical and theoretical distributions of score statistics and measure their statistical performance for different parameter settings. Third, we analyze level 3 mRNA-Seq data on lung squamous cell carcinoma (LUSC) from The Cancer Genome Atlas (TCGA). Comparisons with traditional algorithms are described in the ‘Simulations’ and ‘Real data analysis’.Nsubjects in the studied sample, and suppose that for all individuals there is one sequenced gene with p isoforms. For the ith individual, Yi1, Yi2, …, Yip denotes the RNA-seq data of one gene. The vector Y is assumed as a Poisson or NB distribution. With the assumption of independent samples, xi is the covariate of individual i. For example, its value is 0 or 1 in a case-control study. We assume αj to be a random term that follows a normal distribution. Its variance represents the heterogeneity of the baseline abundance of the isoforms. This distinct assumption is the difference between gene-level measurement methods and isoform-level measurement methods. The disease effect is also defined as a random effect. Based on the above assumptions, a GLMM of two random terms is constructed as follows: xi is equal to zero in a case-control study, μ + αj indicates the expression of the jth isoform in the control group, and βj indicates the disease effect.Assume that there are N × p vector consisting of NYi vectors T). μ is also an N × p vector, in which all elements are Np × p matrix in which Ip is the p-dimensional identity matrix, The matrix form is written as follows: β equals the test of its variance component τ. The final null hypothesis (H0) of the model is written as τ = 0. The idea of a score test is used to test τ. Based on the theory of score statistics in a GLMM and the special assumption of this model, the null distribution and parameter estimation of the statistic U is as follows:In fact, the test of the disease effect is performed to test the variance component of the second random effect. As the basic assumption of mixed models, the third term in the above equation follows a normal distribution  . Therefo1−, W and W0 are block diagonal matrixes, and 1− of Poisson and NB distribution is the same, but W and W0 are different. For the Poisson assumption, the product of Δ1− and W is an identity matrix, and W = W0. For the NB assumption, we take the product of Δ1− and W as Φ, which has diagonal elements of ϕ is the scale parameter of an NB. The diagonal elements of the block of W0 are First, the matrix form of the first order derivate of the log-likelihood is as follows: U depends on the first derivate of the log-likelihood function. The formula for the U statistic depends on an assumption. Under the Poisson assumption, U is equal to U is formulated as H0.Second, the statistic Third, the chi-square statistic is shown as A = KTK with a diagonal element of aii. aii consists of a vector, denoted as κi2, κi3 and κi4 be the cumulants of Yi. With the assumption of an exponential family of distributions, the relationships between the three cumulants are as follows: Finally, suppose R ∈ ℝNp×Np be a block diagonal matrix. In each block, the diagonal elements and non-diagonal elements are calculated as C ∈ ℝNp×Np is composed of a p diagonal matrix, Cj, whose diagonal elements are J is an Np vector, and its elements are 1. The difference in variances leads to different expressions for R and C. For the Poisson assumption, the diagonal and off-diagonal elements of R are C are R are C are Let U follows two steps: (a) construction of the empirical distribution by shuffling the label and (b) calculation of the P value of the original statistic U. The estimation of μ and α is similar to the above. To improve robustness, the default setting of the number of permutations is 5,000.The permutation algorithm generates the empirical distribution. The two different assumptions cause different score statistics. For the Poisson assumption, the statistic is U, for identifying differential expression under a range of scenarios. The parameter settings are based on the analysis of real data and references (α (s), the variance of the disease effect (l), the number of isoforms (p), the logarithm of expression levels (d). s shows the heterogeneity of the baseline abundance of each isoform. l refers to the heterogeneity of the disease effect. The other three parameters describe the characteristics of the expression levels. The variance of α varies from 0.25 to 1.75 in steps of 0.75. The variance of β varies from 0.20 to 1.00 in steps of 0.4. The number of isoforms can be 2, 4 or 8. d is set at 0.5 or 2. We let l = 0).We perform simulations to examine the type I error and power of the proposed score statistics, ferences . We assul is equal to 0, the proportion of significant genes is the false positive rate.We compare our approach with three traditional algorithms: DESeq, edgeR and TSPM . The sumgene expression, normalized gene expression, isoform expression and normalized isoform expression. We only used the gene expression and isoform expression datasets. These data are attached as We download the LUSC Batch 101 mRNA sequencing data from TCGA . The samglmer and glmer.nb in the lme4 package fit the mixed models in our algorithm. The function ginv in the MASS package is employed to calculate the generalized inverse of the singular matrix. The packages DESeq and edgeR are both from Bioconductor. In consideration of the potential computing time of the simulations, the doParallel package is used for parallel computation.This algorithm is completed using the Microsoft R Open . The functions This algorithm provides self-adaptation for real data analysis. If the fitness of the NB assumption fails or the dispersion parameter is close to one, the Poisson assumption is used. Due to the application of the idea of the variance component test, our method is referred to isoVCT. The distribution of the score test statistic is fitted via theoretical and empirical methods, which we term isoVCT-The and isoVCT-Emp, respectively. The R program of isoVCT and the simulation code are attached as s, l and p may be unrelated to type I error rates, but ϕ may show an inverse correlation to the type I error rate. isoVCT-The is exactly conservative in specific scenarios. The type I error rate of isoVCT-Emp is close to 0.05, which might mean that the permutation can fit the distribution of U in a small sample size setting. DESeq can control type I errors; however, the type I error rates of edgeR and TSPM are both dispersed, especially when ϕ = 0.5. Generally, isoVCT can control type I error rates around the nominal level for the NB assumption.The results for the empirical type I error for the NB assumption are shown in M exhibits a positive correlation with power; (b) p and s exhibit a negative correlation with power; and (c) l also exhibits a positive correlation with power, but its increase is much sharper than those of p and M. Our results show that isoVCT-Emp is more advantageous than traditional methods, especially when the effect size is small and the effect heterogeneity is large.The results regarding empirical power for the NB assumption are shown in The results of the type I error and empirical power under the Poisson assumption are shown in The results of real data analysis are shown in The results of isoVCT are as follows. First, isoVCT-Emp defines 263 DE genes; of these genes, isoVCT-Emp specifically selected 242 DE genes, and only a small portion of these genes is also found by the other algorithms. Second, the DE genes identified by isoVCT-Emp included those that are identified by isoVCT-The. In general, isoVCT is superior at selecting heterogeneous genes.U statistic in the small sample size setting. The type I error rates of edgeR and TSPM increase in some settings, especially with the NB assumption (l and NB assumption. isoVCT-Emp is superior to other methods in each setting,especially for l = 0.6. The small sample size setting may cause the inverse proportion between p and the power of isoVCT. Nevertheless, the powers of five algorithms are high for the Poisson assumption.Here, we propose isoVCT, a variance component score test for testing the coefficients of covariates to select DE genes. Simulations and real data analysis both suggest that this method is advantageous in selecting weak effects or heterogeneous genes. The results of the simulations are discussed from the following two aspects. First, isoVCT controls type I errors in each setting under any of the distribution assumptions. The type I error rate of the empirical distribution is almost controlled. However, the type I error rate of isoVCT-The is strictly controlled, which is likely caused by an inaccurate estimation of the sumption . Second,CASP7 and STAT6 are related to LUSC (In real data analysis, the Venn diagram of DE genes directly indicates the relationships among the four algorithms. The fact that DESeq and edgeR use the same distribution assumption and the same idea for testing likely results in the number of intersections representing a majority of the DE genes identified by these algorithms. TSPM defines the smallest number of DE genes. However, the use of different ideas leads to the different result of isoVCT. isoVCT-Emp specifically defines 242 heterogeneous or small effect size genes. For example,  to LUSC .Furthermore, isoVCT exhibits three innovations. First, isoVCT derives the empirical and theoretical distribution of the variance component score statistics in the framework of a GLMM and evaluates the performance of score statistics. Second, the random effect α represents the correlation of isoforms in the framework of the GLMM. Third, isoVCT further verifies the effectiveness of the set test in RNA-Seq data and supplies a new view of biological functions with isoform information.α is regarded as a fixed parameter, which may cause isoVCT-The to be quite conservative. For the NB assumption, the power of isoVCT is very low in some settings, because the estimation of the dispersion parameter may be biased. The methods of weighted likelihood and quasi-likelihood likely overcome this drawback in estimating the NB mixed model (However, isoVCT has some limitations. In the derivation of the score statistic, the random term ed model .In conclusion, isoVCT, a supplement to DESeq or edgeR, is powerful and robust at selecting small effect and heterogeneous genes.10.7717/peerj.3797/supp-1File S1Click here for additional data file.10.7717/peerj.3797/supp-2File S2Click here for additional data file.10.7717/peerj.3797/supp-3File S3Click here for additional data file.


Cancer classification based on molecular level is a relatively routine research procedure with advances in high-throughput molecular profiling techniques. However, the number of genes typically far exceeds the number of the sample size in gene expression studies. The existing gene selection methods are almost based on statistics and machine learning, overlooking relevant biological principles or knowledge while working with biological data. Here, we propose a robust ensemble learning paradigm, which incorporates multiple pathways information, to predict cancer classification. We compare the proposed method with other methods, such as Elastic SCAD and PPDMF, and estimate the classification performance. The results show that the proposed method has the higher performances on most metrics and robust performance. We further investigate the biological mechanism of the ensemble feature genes. The results demonstrate that the ensemble feature genes are associated with drug targets/clinically-relevant cancer. In addition, some core biological pathways and biological process underlying clinically-relevant phenotypes are identified by function annotation. Overall, our research can provide a new perspective for the further study of molecular activities and manifestations of cancer. Accurate classification of cancer is the initial and significant step for clinical management since different treatment modalities exist. Traditionally, the classification of cancer is primarily based on the experience or histology. With advances in high-throughput sequencing techniques, the researchers can utilize the expression of tens of thousands of genes simultaneously. Cancer classification based on molecular level is now a relatively routine research procedure. However, the number of genes typically far exceeds the number of the sample size in gene expression studies. This situation is called high-dimensional and low sample size problem3.For the patient to receive appropriate therapy, accurate classification of cancer is crucial in disease treatmentet al.4 proposed a SVM method of Recursive Feature Elimination (RFE) to gene selection by measuring the relative contribution of a gene. Li et al.5 employed maximum relevance minimum redundancy (mRMR) method based on Random Forest algorithm (RF) to predict protein cleavage sites. In Cai et al.’ work6 the authors performed ensemble-based feature extraction method, which incorporates Multi-category Receiver Operating Characteristic (Multi-ROC), Random Forests (RFs) as well as Maximum Relevance and Minimum Redundancy (mRMR) methods, to select molecular signatures. For gene selection, an alternative technique is the regularization method, such as lasso7 1-norm support vector machine8 SCAD9 Elastic Net10 and Elastic SCAD11.To address the problem of high dimensionality, gene selection is one of the important steps for classification modeling. Gene selection is of fundamental and practical interest. To date, many types of gene selection methods were proposed. Guyon 19. Pathways, being a series of interactions among molecules , yield stable sets of functional relationships related with molecular biological activities such as metabolic, signaling, protein interaction and gene regulation processes, which plays an important role in understanding the mechanisms of complex diseases, improving clinical treatment, discovering drug target and biomarker20. Pathway-based method not only reduces the number of dimensions and increases statistical power, but also helps scientists better understand biological mechanisms at the molecular level21. For example, Kim et al.19 proposed standardized pathway-based approach extracting multi-level hierarchical feature vectors, with a basic gene level as well as a second level of pathway markers, to biomarker analysis for discriminating cancer subtypes. Huang et al.17 developed a personalized pathway-based diagnostic modeling framework(abbreviated as PPDMF) which converts omics-level features to pathway-level features using the non-parametric principle curve approach and subjects them to feature selection and machine learning classifications for differentiating different phenotypes.However, the above-mentioned gene selection methods are based on statistics and machine learning, seldom do these methods involve relevant biological principles or knowledge while working with biological data. So, many gene selection methods are prone to over-fitting or poor biological interpretation when applied on biological high-dimensional data. To improve the discriminant capability of features, biological domain knowledge, such as pathways are more often referred to during cancer classification to give more robust and generalizable results22  genes is performed on training set of gene expression profiles. We select differentially expressed genes of each pathway to generate a group of base learners through training SVM, then, we rank all DE pathways with classification accuracy on training set. Secondly, the diversities of top 35 pathway-based base learners with higher accuracy are computed. Selecting classifiers into the ensemble from the top 35 pathway-based base classifiers according to diversity. Finally, integrating the remaining classifiers into the final ensemble learning model23 it is available via the Gene Expression Omnibus (ID = GSE25066) and includes 488 samples of breast cancer patients treated with NAC (antracyclines/taxanes) profiled with the U133A microarray. This dataset compared 99 pathologic complete response (pCR) samples and 389 residual disease (RD) samples (http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE25066). For dataset Liver24 it is one RNA-Seq data set from The Cancer Genome Atlas (TCGA) . The Liver dataset consists of 421 samples obtained from comparing 371 liver cancer samples with 50 normal samples using the Agilent platform. For dataset GSE2019425 it is also a chemotherapy response data and comes from the Gene Expression Omnibus (ID = GSE20194). This dataset compared 56 pathologic complete response (pCR) samples and 222 residual disease (RD) samples (https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE20194). Pathways are come from KEGG database (http://www.genome.jp/kegg/). The total number of known human pathways in the KEGG database is 307. We selected 298 pathways of these containing at least one gene.To evaluate the predictive ability of the here presented model, three publicly available gene microarray datasets are used to carry out analysis. For dataset GSE2506626. In the paper, we performed an exact test on training set of gene expression profile data to find genes that are differentially expressed between different phenotypes. Genes are considered to be significantly differentially expressed if they obtain a p-value <0.05. Then, we obtained a list of genes that are differentially expressed from gene expression profile data. The next step was to map each pathway to the list of differentially expressed genes .Differentially expressed genes between different phenotypes are thought to be fertile sources of stable cancer biomarkers. Hence, filtering out genes that are differentially expressed between different phenotypes is an integral part of understanding the molecular basis of phenotypic variation in cancers27 we took the selected feature sets as input and used SVM as base classifier to discriminate between the two classes of interest. In order to form a baseline measure, we used default parameter settings for all SVM tasks. Next, all experiments were repeated for 100 runs on training set of gene expression profile data and the average accuracy of each base classifier was computed as the final results. Finally, base classifiers were ranked in descending order according to accuracy.The differentially expressed genes of each pathway were selected as classification features for each base classifier, respectively. Since Support Vector Machine (SVM) has been successfully applied to cancer classification using gene expression data28.It is well known that diversity among base classifiers plays an important role in ensemble learning. Ensembles tend to yield better results when there is a significant diversity among the base classifiers. There exist many measures of dependency between classifiers. The most commonly employed traditional measures of diversity adopt the zero-one loss (classification error) function, one of which is the disagreement measure. The disagreement measure estimates the diversity for a pair of classifiers in a form of a ratio between the number of samples for which classifiers disagreed, to the total number of observations. We carried out our work based on the disagreement measure due to its easy interpretation for independence, positive/negative dependences, and calculationhi, hk represents two different base classifiers, respectively. L = {l1, l2, …, ln} be a labeled data set. hi, such that hi recognizes correctly lj, 0 otherwise. The diversity between two binary classifier outputs (correct/incorrect) hi, hk isNab is the number of elements lj of L for which Let N base-learners with higher accuracy were computed by formula (1). Finally, a diversity matrix D with N rows and N columns, which is symmetric was obtained.The diversities of top In the present study, genes of each DE pathway were used as features to train a certain SVM as a base learner. Since diversity plays an important role in ensemble learning, optimal selection of base classifiers was made by taking both accuracy and diversity into account . Finally, the ensemble classifier was constructed by selected base learners see Fig. .Figure 2S denotes ordered DE pathway sets corresponding to feature sets of top N base learners based on DE pathways with higher accuracy. In order to reduce the computational complexity, we defined overall diversity of the i th pathway-based base classifier as OD[i]. OD[i] was calculated as follow:D is a diversity matrix, Let 30. A majority vote based classifier ensemble technique classifies a pattern by letting each member of the ensemble cast a single vote for the correct class and deciding according to democratic rules. In the paper, we combined different base classifiers based on DE pathways and used a majority vote rule. The ensemble decision will be correct if at least T denotes the number of base classifiers. Firstly, we calculated OD[i] of base classifier from S and reordered base classifiers of S as S* in descending order according to OD. Secondly, base classifier based on DE pathway was incrementally added into the ensemble one by one in each iteration step from S*. In each iteration, the average accuracy of each ensemble learning was obtained using 5-fold bootstrap cross-validation with 100 runs (100 × 5).Then, the highest accuracy of the ensemble with m base classifiers was obtained. Let S′ denotes ordered DE pathway set corresponding to top m base learners according to OD. Finally, the ensemble with m base classifiers was optimized according to algorithm 1. If the diversity of two base classifiers is smaller than the diversity threshold value θ, we think one of two base classifiers (or even two) is superfluous. Let S1 denotes that two base classifiers are not removed from S′, S2 denotes that one of the two base classifiers is removed from S′, S3 denotes that another one of the two base classifiers is removed from S′, S4 denotes that both of them are removed from S′. Then, we determined whether it can be removed through average classification accuracy obtained using 5-fold bootstrap cross-validation with 100 runs (100 × 5). When the classification accuracy is equal, the priority option is S4 > S3 > S2 > S1. The procedure was repeated until all base classifiers which can be deleted were removed, and the optimized ensemble with base learners from S″ was finally obtained.Among different voting strategies, the majority voting is considered as a simplest and effective scheme31 is an R/Bioconductor software package that allows users to analyse both RNA-seq and microarray data with very similar pipelines. Among the methods evaluated for differential expression (DE) analysis in ref. To verify our method, we conducted computational experiments on Dataset GSE25066. We evaluated the performance of the proposed ensemble method through five measures: accuracy, precision , sensitivity (True Positive Rate), specificity and F-score which are calculated below:S.The DE genes of each pathway were selected as classification features of each base classifier. We took the selected feature sets as input and used SVM based on balanced training data sets (50 pCR vs. 50 RD) to discriminate between the two classes of interest(random sampling 50 RD samples out of 194 RD). For sample type RD, 50 RD were randomly sampled from 194 RD in every run. The performance of each base classifier was tested on training set using 5-fold bootstrap cross-validation with 100 runs (100 × 5). The average accuracy of each base classifier was computed as the final results. The DE pathways were sorted in descending order according to accuracy. Then we selected top 35 DE pathways into S*. In the case where classifier based on DE pathway was added into ensembles one by one in each iteration step in order from S*. Each iteration employed 5-fold bootstrap cross-validation with 100 runs (100 × 5) and the average accuracy of each ensemble classifier was computed as the final results  base classifiers.The pairwise functional diversities between TOP 35 base classifier based on DE pathways were calculated 100 times with each other one. Taking the average, then, a diversity matrix was obtained. According to algorithm 1, we reordered DE pathways by overall diversity of each DE pathway and put it in θ less than 0.15 were removed based on algorithm 1 and the remaining 9 base classifiers based on DE pathways were combined for ensemble learning . Finally, the accuracy, precision, sensitivity, specificity and F-score were computed for each run and then averaged over runs for ensemble classifier. The average accuracy, precision, sensitivity, specificity and F-score of the proposed method are 68.81%, 68.05%, 71.08%, 66.53%, 69.43% on Dataset GSE25066, respectively  is used for feature selection. To make this method comparable to our method, the SVM model is used for classification. For Dataset GSE25066, the transcriptomics-level data were firstly transformed to pathway-level data by the pathifier algorithm. Since 3 pathways out of 298 contain only one gene, their dysregulation scores could not be calculated. Hence, we obtained a pathway dysregulation score matrix with 488 rows (samples) and 295 columns (features).To assess the validity of the proposed approach, here, two latest methods: PPDMF and Elastic SCAD were investigated in parallel with the proposed method on the same public datasets. The PPDMF hypothesizes that pathway-based omics features can provide more information on biological functions for disease diagnosis. This method is a typical representative of pathway-based method for disease diagnosis. It converts omics-level data to pathway-level data by the pathifier algorithm17. Then, 4 features were selected. A new testing data matrix with 243 rows and 4 columns was generated. Finally, we took the testing data matrix as input and used Support Vector Machine (SVM) to predict patient prognosis. Since dataset GSE25066 is unbalanced between two (RD and pCR) phenotypes in the testing matrix, we balanced the two classes for classification purposes by random sampling 49 samples from the larger collection of testing RD samples. The performance of PPDMF was tested using 5-fold bootstrap cross-validation for 100 runs (100 × 5). Finally, the Accuracy, Precision, Sensitivity, Specificity and F-score were computed for each run and then averaged over runs. The results were obtained with average accuracy of 65.20%, precision of 64.01%, sensitivity of 70.29%, specificity of 60.12%, F-score of 66.64% and half for testing (49 pCR vs. 194 RD). CFS feature selection was applied with 10-fold cross-validation (10-fold CV) in the training matrix and kept the features that were selected ten out of ten times (100%)The Elastic SCAD method is a typical representative of the regularization method for classification and feature selection tasks. It is a penalty function providing an automatic feature selection for SVM classification tasks combining smoothly clipped absolute deviation penalty (SCAD) and ridge penalties. Elastic SCAD provides robust classifiers in sparse and non-sparse situations. For Dataset GSE25066, also since dataset GSE25066 is unbalanced between two (RD and pCR) phenotypes in training dataset and testing dataset, we balanced the two classes for classification purposes with same process as our method in training dataset and testing dataset. In Elastic SCAD, we set the search interval for both parameters to see Fig. .Comparing the other two methods over dataset GSE25066, the results show that the proposed method has the higher performances and performed well on all metrics see Fig. , with avm equal to 7 and 10 for Liver and GSE20194, respectively. Finally, the remaining 5 base classifiers were combined for ensemble learning based on algorithm 1 for the two datasets  following the same evaluation strategy. In the proposed method, we obtained parameter Figures 35. We believe that the feature genes from the ensemble should be related to the targets of these drugs. In this article, the feature gene set of each base classifier from ensembles was mapped to all breast cancer drug targets, which come from DrugBank (https://www.drugbank.ca/). In GSE25066 and GSE20194, we found that pathways corresponding to base classifiers from ensembles were associated with drugs that are used to treat breast cancer, which reflects the pathogenesis of breast cancer. Then, some clinical breast cancer drug targets were identified in the pathways which were selected into ensembles  or residual invasive cancer (RD). There are many drugs, which have similar therapeutic mechanisms, for the treatment of breast cancer. To a certain extent, the therapeutic mechanisms of these drugs can reflect the pathogenesis of breast cancer37 which is an extended gene functional interaction network for Homo sapiens. We find that most of the genes were either directly or indirectly connected to each other, forming some network modules . This analysis demonstrates the power of the proposed method to identify cancer-related pathways, including Wnt signaling pathway, Pathways in cancer and so on (see Figs 38. In addition, through functional annotation clustering, we found that the list of genes was also correlated with biological process of cancer, such as cell death, cell growth, cellular response to chemical stimulus, immune system development and positive regulation of cellular metabolic process (see Fig. To better understand and dissect the complexities of the feature genes from the ensemble model underlying clinically-relevant phenotypes, all feature genes of the ensemble for GSE25066 were mapped on HumanNetsee Figs . These fsee Fig. . Taken t40 With the rapidly increasing amount of pathway information databases, it enables researcher further opportunities to understand biological mechanisms of cancer and its phenotypes, connectivity of diseases, mechanisms of drug action at molecular level, etc. Now, the combination of pathway information and gene expression profiles is becoming a central branch of research for cancer classification. In this context, we propose a robust ensemble learning paradigm, which incorporates pathway information, to predict cancer classification.Integrating prior information of biology, like pathways from databases such as KEGG, has recently been proposed to overcome variability of prognostic signatures and improve their prognostic performanceIn conclusion, the results obtained in this study show that the proposed method presents the merit of acquisition of more informative from pathways. The method has improved the classification performances of disease status and performs robust both when classifiers are trained on different datasets and within cross-validated single dataset, comparing with PPDMF and Elastic SCAD. In addition, our method can provide clues on potential biomarkers, core biological pathways and processes that can help make true rational design a drug target selection method through the integration of experimental observations with underlying cellular regulation and signaling pathway.


Real-time quantitative PCR (qPCR) is a broadly used technique in the biomedical research. Currently, few different analysis models are used to determine the quality of data and to quantify the mRNA level across the experimental conditions.CT and standard curve models were implemented to quantify the relative expression of target genes from CT in standard qPCR control-group experiments. In addition, calculation of amplification efficiency and curves from serial dilution qPCR experiments are used to assess the quality of the data. Finally, two-group testing and linear models were used to test for significance of the difference in expression control groups and conditions of interest.We developed an R package to implement methods for quality assessment, analysis and testing qPCR data for statistical significance. Double Delta Using two datasets from qPCR experiments, we applied different quality assessment, analysis and statistical testing in the pcr package and compared the results to the original published articles. The final relative expression values from the different models, as well as the intermediary outputs, were checked against the expected results in the original papers and were found to be accurate and reliable.The pcr package provides an intuitive and unified interface for its main functions to allow biologist to perform all necessary steps of qPCR analysis and produce graphs in a uniform way. Real-time quantitative PCR (qPCR) is a commonly used technique to analyze the relative gene expression level in the biomedical research. In most cases, small scale experiments are designed to quantify the level of mRNA among experimental conditions. Some advanced machines and optimized protocols have simplified the experiments to a highly efficient one-step process, allowing the effective analysis of a large scale of qPCR data. However, all processes for assessing the quality of the data, performing the analysis and reporting the results are not done in the most uniform way across the literature . DiffereIn this report, we introduce an open source R package for performing quality assessment, modeling and testing for statistical significance of qPCR data in a uniform way. In its current version, the pcr package implement two methods for relative quantification of mRNA expression proposed originally by Arabidopsis thaliana plant treated and control samples (24 samples each), and performed qPCR analyses using MT7 and ubiquitin primers. This dataset was referred to as ct4 and shown the results of the different testing methods that applied in the original paper in CT (Cycle Threshold), also known as Cq, was recorded  and/or a control group (reference group) to quantify the mRNA of interest relative to these references. This relative quantification was sufficient to draw conclusions in most of the biomedical applications involving qPCR. A few methods were developed to perform these relative quantification. These methods generally require different assumptions and models for the analysis. The most common two of these methods were described here in the following sections.CT methods assume that the cDNA templates of the gene/s of interest as well as the control/reference gene have similar amplification efficiency, and also that this amplification efficiency is near perfect. This means that, at a certain threshold during the linear portion of the PCR reaction, the amount of the gene of the interest and the control double each cycle. Another assumption is that the expression difference between two genes or two samples can be captured by subtracting one (gene or sample of interest) from another (reference). The final assumption is that the reference doesn’t change with the treatment or the course in question. The formal derivation of the double delta CT model is described here CT is given by: The comparative CT,q is the difference in the CT (or their average) of a gene of interest and a reference gene in a group of interest. ΔCT,cb is the the difference in the CT (or their average) of a gene of interest and a reference gene in a reference group. The error term is given by: s1 is the standard deviation of a gene of interest and s2 is the standard deviation of a reference gene.And the relative expression by: CT values for each gene are used to calculate an intercept and a slope. Secondly, these intercepts and slopes are used to calculate the amounts of mRNA of the genes of interest and the control/reference in the samples of interest and the control sample/reference. These amounts are finally used to calculate the relative expression in a manner similar to the later method, just using division instead of subtraction. The formal derivation of the model is described here . t-test assumes, in addition, that the input CT values are normally distributed and the variance between conditions are comparable. Wilcoxon test can be used if sample size is small, and those two last assumptions are hard to achieve.Assuming that the assumptions of the first methods are holding true, the simple Two use the linear regression here. A null hypothesis is formulated as following, CT value as explained earlier. So the ΔΔCT is estimated and the null is rejected when ΔΔCT ≠ 0.This is exactly the ΔΔCT value of a gene of interest and a control/reference (ΔCT) and the log input amount. This piece of information is required when using the ΔΔCT model. Typically, the slope of the curve should be very small and the R2 value should be very close to one. A value of the amplification efficiency itself is given by 10−1∕slope and should be close to 2. Other analysis methods are recommended when this is not the case. Similar curves are required for each gene using the CT value instead of the difference for applying the standard curve method. In this case, a separate slope and intercept for each gene are required for the calculation of the relative expression.Fortunately, regardless of the method used in the analysis of qPCR data, The quality assessment can be done in a similar way. It requires an experiment similar to that of calculating the standard curve. Serial dilutions of the genes of interest and controls are used as input to the reaction and different calculations are made. The amplification efficiency is approximated be the linear trend between the difference between the CRAN, the main repository for R packages and can be installed by invoking install.packages in an R (≥3.4.2) session. The package’s source code is also available on github, https://github.com/MahShaaban/pcr along with the development version.The pcr packages is available on   # install  the  pcr  package  from  CRAN install.packages('pcr')    The examples shown in this article are explained in greater details in the package vignette that can be accessed through browseVignette(’pcr’). Moreover, the package documentation provides detailed instruction on the input and the output of each function .The pcr package provides different methods for performing quality assessment, modeling and testing real-time qualitative PCR data through the unified interface of three functions: pcr_assess, pcr_analyze and pcr_test, respectively.plot as TRUE in the pcr_assess function provides the a graphic presentation of the amplification and the standard curves as shown in pcr_assess provides two methods for assessing the quality of qPCR data. These are ‘efficiency’ and ‘standard_curve’ to calculate the amplification efficiency and gene standard curves as described in the methods section. The following code block applies both methods to the dataset ct3, shown in   # load  required  libraries library(pcr) library(ggplot2) library(cowplot) library(dplyr) library(xtable) library(readr)      #  pcr_assess ##  locate  and  read  data fl <− system.file ct3 <− read_csv(fl)  ## make a vector  of RNA  amounts amount <− rep,  each = 3)  ##  calculate  amplification  efficiency res1 <− pcr_assess  ##  calculate  standard  curves res2 <− pcr_assess  ##  retain  curve  information intercept <− res2$intercept slope <− res2$slope    CT values and calculates the relative expression of target genes. ’delta_delta_ct’ performs the ΔΔCT method described previously. The average relative expression of the target gene in the condition of interest is given by the Similarly, pcr_analyze provides two methods to model the    #  pcr_analyze ##  locate  and  read  raw ct data fl <− system.file ct1 <− read_csv(fl)  ## add  grouping  variable group_var <− rep, each = 6)  # calculate  all  values  and  errors  in one  step ## mode == 'separate_tube' default res1 <− pcr_analyze  ##  calculate  standard  amounts  and  error res2 <− pcr_analyze    p-values and confidence intervals on the previous models. Two tests are available of the two-group comparisons; ‘t.test’ and ‘wilcox.test’ to test for the difference of the normalized target gene expression (ΔCT) in one condition to another. Linear regression, ‘lm’, can be applied to estimate these differences between multiple conditions and a reference  ct4 <− read_csv(fl)  # make  group  variable group <− rep, each = 12)  # test  using t −test tst1 <− pcr_test  # test  using  wilcox.test tst2 <− pcr_test # testing  using  lm tst3 <− pcr_test    CT from two kinds of experiments. First, the CT values for the genes of interest and a reference gene in the different experimental groups. Second, the CT values of the same genes from a serial dilution experiment. The amplification efficiency can be calculated by calculating the ΔCT from the first and the slope and the intercept from the second experiment. The ΔΔCT model and the relative expression are then calculated and tested. In the case of the two groups comparison, any of the testing methods can be applied otherwise the linear regression can be used for multiple group comparisons. Alternatively, the standard curve methods can be applied by calculating the amount of cDNA in each sample from the CT values and the individual curves for each gene. These amounts are finally used to obtain the relative expression of the genes of interest.In data.frame, and the package source code follows the tidyverse practices. This package targets the small scale qPCR experimental data and the R users. The interface and documentation choices were made with such users in mind and require no deep knowledge in specific data structures or complex statistical models. Users can go from the raw CT values through different analysis steps to publication graphs in a few simple lines of code.In comparison, pcr provides a unified interface for different quality assessment, analysis and testing models. The different functions take similar inputs to perform different analysis steps which requires minimal formating at the end of the user. The main input and the output are tidy CT methods assume that the PCR reaction has a close to perfect amplification rates. Other methods were proposed to model the data when this assumption is not true (The current version of the pcr package (1.1.0) provides only methods to estimate the expression of genes in a certain condition relative to another. Other methods were proposed for absolute quantification of the copy number of genes in samples . Also, tnot true . We are To sum, the pcr package is an open source R package for quality assessing, modeling and testing real-time quantitative PCR data. The package provide an intuitive and unified interface for its main functions to allow biologist to perform all necessary steps of qPCR analysis and produce graphs in a uniform way.


We evaluate the feasibility of using a biological sample’s transcriptome to predict its genome-wide regulatory element activities measured by DNase I hypersensitivity (DH). We develop BIRD, Big Data Regression for predicting DH, to handle this high-dimensional problem. Applying BIRD to the Encyclopedia of DNA Elements (ENCODE) data, we found that to a large extent gene expression predicts DH, and information useful for prediction is contained in the whole transcriptome rather than limited to a regulatory element’s neighboring genes. We show applications of BIRD-predicted DH in predicting transcription factor-binding sites (TFBSs), turning publicly available gene expression samples in Gene Expression Omnibus (GEO) into a regulome database, predicting differential regulatory element activities, and facilitating regulome data analyses by serving as pseudo-replicates. Besides improving our understanding of the regulome–transcriptome relationship, this study suggests that transcriptome-based prediction can provide a useful new approach for regulome mapping. A map of the activities of all genomic regulatory elements across cell types and conditions would be a tremendous resource. The computational method introduced here predicts genome-wide accessible sites from gene expression data and allows the authors to build a database of regulatory element activities using publicly available transcriptome data. So far, these technologies have only been applied to interrogate a small subset of all possible biological contexts defined by different combinations of cell or tissue type, disease state, time, environmental stimuli, and other factors. A major limitation of these technologies is the difficulty in simultaneously analyzing a large number of different biological contexts. This limitation along with practical constraints such as lack of materials, antibodies, resources, or expertise has hindered their application by the vast majority of biomedical investigators from small laboratories.A fundamental question in functional genomics is how genes’ activities are controlled temporally and spatially. To answer this question, it is crucial to comprehensively map activities of all genomic regulatory elements  and understand the complex interplay between the regulome and transcriptome . Regulome mapping has been accelerated by high-throughput technologies such as chromatin immunoprecipitation coupled with high-throughput sequencing (ChIP-seq)7. However, the interplay between regulome and transcriptome is bidirectional due to feedback9. A systematic understanding of this relationship in the reverse direction, to what extent regulatory elements’ activities can be predicted by transcriptome, is still lacking.Numerous researchers have examined how genes’ transcriptional activities can be predicted using activities of their associated regulatory elements10. Besides creating a more complete picture of the regulome–transcriptome relationship, this investigation also has important practical implications for regulome mapping. Gene expression is the most widely measured data type in high-throughput functional genomics. Measuring expression does not require large amounts of materials and complex protocols, and technologies for expression profiling are relatively mature. Thus, expression data are routinely collected when other functional genomic data types are difficult to generate due to technical or resource constraints. Today, the Gene Expression Omnibus (GEO) database11 contains 200,000+ human gene expression samples from a broad spectrum of biological contexts, as compared to only ~7000 human ChIP-seq, DNase-seq, FAIRE-seq, and ATAC-seq samples. If one can use the ENCODE data to build prediction models and apply them to existing and new transcriptome data to predict regulome, the catalog of biological contexts with regulome information may be quickly expanded  Projectded Fig. . This wiIRD Fig. , to handPredicting DH using gene expression can be formulated as a problem of building millions of regression models, one per genomic locus, to describe the relationship between the DH level at that locus (response) and the expression levels of tens of thousands of genes (predictors). For method development and evaluation, we compiled DNase-seq and exon array  data for 57 distinct human cell types with normal karyotype from ENCODE (Supplementary Data rL). Second, we evaluated a method’s ability to predict the relative activities of the same DHS in different cell types by computing the P–T correlation across different cell types at each genomic locus (cross-cell-type correlation rC). Third, we computed the total squared prediction error normalized by the total DH data variance (τ) to characterize the proportion of data variation not explained by the prediction.For each locus, prediction models were constructed using the 40 training cell types. Prediction performance was evaluated using the 17 test cell types based on three types of statistics . 15 based on 1% of the genome. Both methods yielded similar prediction accuracy, but fused lasso was >105 times slower than The locus-level model, denoted by mentclass2pt{minim19, and the mean DH level of all loci in a pathway can be viewed as the pathway’s activity. Because using the cluster mean reduces the variance of measurement noise, pathway activities are less noisy than locus-level DH measurements. Thus, The pathway-level model, denoted by The final prediction by BIRD for each genomic locus is a weighted average of the locus-level and pathway-level predictions. The intuition is that the pathway activity predicted by mentclass2pt{minimAfter building BIRD prediction models using the 40 training cell types, we evaluated their prediction performance in the 17 test cell types. Below is a summary of the major findings.rL and rC) and substantially decreased the squared prediction error (τ)  in Fig. rC) in Fig. rL on average was much larger than rC (0.82 vs. 0.50 for BIRD). Unlike rL, the distribution of rC for random prediction models was centered around zero . On the other hand, 56 and 20% of loci had rC > 0.5 and >0.75, respectively, indicating that DH could be predicted with moderate to high accuracy for a substantial fraction of loci. By examining the true DH levels measured by DNase-seq in the training and test cell types, we found that multiple factors may influence cross-cell-type prediction accuracy of a locus. First, a subset of loci was not active in any test cell type. For these loci, the true DH levels are essentially noise. The cross-cell-type correlation between the predicted DH levels and random noise is expected to be low. Therefore, these noisy loci are not informative for evaluating the performance of predicting biological variation across cell types. After excluding these noisy loci, we found that loci with low signal range , low signal variability (characterized by coefficient of variation (CV)), or high cell-type-specificity (characterized by the number of cell types in which the locus is active or inactive) tend to have lower rC  , and 74 and 30% of loci would have rC > 0.5 and >0.75, respectively  Fig. . In real25) Fig. . In pracrC for the pathway-level prediction by rC for the locus-level prediction by BIRD . For cell type pairs in the highest similarity quartile, the mean correlation was 0.42 for all loci and 0.55 for differential loci Fig. . For celrL, rC, and τ for BIRD from different partitions were similar. The conclusions remained qualitatively the same when Spearman’s rank correlation was used instead of Pearson’s correlation . Among 10 tested cell types, BIRD and ChromImpute showed comparable cross-locus and cross-cell-type prediction performance. Neither method consistently outperformed the other , which are non-trivial to generate. By contrast, BIRD was based on gene expression data, which are easier to generate and widely available.ChromImpute is a method for imputing one functional genomic data type using multiple other data typesher Fig. . We further Fig. . Note thp-values < 0.001)  predictions gave a sensitivity of 0.43, 0.64, and 0.88, respectively, as compared to 0.27, 0.64, and 0.94 by the true DNase-seq approach, 0, 0.02, and 0.11 by the motif only approach, and 0.09, 0.36, and 0.62 by the mean DH approach Fig. . At thes22 and CENTIPEDE21. Both PIQ and CENTIPEDE use true DNase-seq data and motif information to make predictions. PIQ showed comparable performance with our true DNase-seq method, whereas CENTIPEDE performed worse than BIRD  one would not have without BIRD. We note that the prediction performance presented above does not represent the upper limit one can achieve using BIRD. It is possible to further improve TFBS prediction by incorporating other information  (Supplementary Note Predicted DNase I hypersensitivity database). A user interface is provided for data query, display and download  and H9 human embryonic stem cells. Promoters of these genes are known to be bound by MYC in a cell type-dependent fashion23. FBL is bound in both P493-6 and H9, LIN28A is bound in H9 but not in P493-6, and BLMH is bound in P493-6 but not in H925. PDDB successfully predicted these known cell-type-dependent binding patterns . For this system, exon array samples were established from iPSCs and iPSC-derived neurons, but no corresponding experimental regulome data were available. This represents a typical scenario for which BIRD is useful. Using BIRD, we predicted differential DH and identified 76,495 DHSs with predicted log2 fold change p-value <10−15 by one-sided Fisher’s exact test).To test the application of BIRD to studying differential DH, we analyzed a differentiation system in which human induced pluripotent stem cells (iPSCs) were treated to differentiate into dopaminergic neurons34. For DHSs with predicted log2 fold change For evaluation, we randomly sampled 26 DHSs at different predicted fold change levels  to accelerate the study of regulatory circuitry.rL for BIRD + obs vs. obs-only = 0.82 vs. 0.76). Replacing BIRD predictions with the mean DH profile of 40 training cell types in this analysis (mean + obs) did not yield similar increase in the P–T correlation (rL = 0.76). We carried out the same analyses on 16 test cell types, and BIRD predictions improved signal in 12 of them  or a small replicate number. Both can lead to low-signal detection power. However, if one has gene expression data, BIRD predictions may be used as pseudo-replicates to enhance the signal. To demonstrate, we analyzed DNase-seq data for GM12878 generated by ENCODE. The data had two replicates. We reserved one replicate as “truth” and used the other one as the “observed” data. Applying the BIRD prediction models trained using the 40 training cell types (GM12878 not included), we predicted DH in GM12878 and treated the prediction as a pseudo-replicate. We then estimated “true” DH using either the “observed” data alone (obs-only) or the average of the “observed” data and pseudo-replicate (BIRD + obs). After adding the pseudo-replicate, the correlation between the predicted and true DH increased , BIRD predictions can provide valuable information to guide hypothesis generation, target prioritization, and design of follow-up experiments. When experimental regulome data are available, BIRD predictions can also serve as pseudo-replicates to improve the data analysis. Although predictions in this study were made using exon arrays, BIRD is a general approach and can be applied to other types of gene expression data when training data are available. For instance, in a separate study we observe that applying BIRD to RNA-seq allows one to predict genome-wide chromatin accessibility not only for bulk samples but also for RNA-seq samples generated using small number of cellsOur results have important practical implications for the analysis of existing and future gene expression data. Conventionally, gene expression data are mainly collected to study transcriptome. The method and software developed in this study now allow one to conveniently utilize such data to study gene regulation. By adding a new component to the standard analysis pipeline of expression data, expression-based regulome prediction can bring added value to an enormous number of new and existing gene expression experiments. Given the wide application of gene expression profiling, this will greatly impact how expression data are most effectively used.In our analyses, prediction models were trained for DHSs found in the training data. Thus, one limitation of BIRD is that it will not discover new locations of DHSs when applied to analyze a new gene expression sample. However, the number of cell types with both regulome and expression data continues to increase. As more training data become available in the near future, one can expect that most DHSs in the genome will be covered by the training data, and new DHSs uniquely present in a new sample will account for only a small fraction of the regulome  DNase-seq data for 57 human cell types with normal karyotype were downloaded from the ENCODE in bam format (download link: http://hgdownload.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeUwDnase). The human genome was divided into 200 base pair (bp) non-overlapping bins. The number of reads falling into each bin was counted for each DNase-seq sample. To adjust for different sequencing depths, bin read counts for each sample i were first divided by the sample’s total read count Ni and then scaled by multiplying a constant N  data for the same 57 ENCODE cell types were downloaded from GEO (GEO accession number: GSE19090). In addition, we downloaded 2000 exon array samples from GEO for constructing the PDDB database (GEO accession numbers for these samples are available at PDDB). All samples were processed using the GeneBASE2(SNR) was defined as the difference between the normalized and log2 transformed DH level of the bin in question and the background. Genomic bins with log2(SNR) >2 in at least one cell type were identified and retained for subsequent analyses, and the other genomic bins were excluded. After applying this filtering procedure to the 40 training cell types, 912,886 genomic bins were retained and used for training and testing prediction models for results presented in Figs. The 57 ENCODE cell types were randomly partitioned into a training dataset with 40 cell types and a test dataset with 17 cell types  and the gene expression profile .For a biological sample, let fl(X) is unknown. We train it using X and Y observed from a number of different cell types. The training data are organized into two matrices: a gene expression matrix C is the number of training cell types. Each column of X and Y in a specific cell type. Building the prediction model for each locus l is a challenging high-dimensional regression problem as the dimensionality of the predictor X is much bigger than the sample size of the training data . will be described. Each method has a training component and prediction component. Before training prediction models, we standardize each row of l . This is the Pearson’s correlation between the predicted signals m. The cross-locus correlation measures the extent to which the DH signal within each cell type can be predicted.Cross-locus correlation (rC). This is the Pearson’s correlation between the predicted signals l. The cross-cell-type correlation measures how much of the DH variation across cell types can be predicted.Cross-cell-type correlation (τ). This is measured by the total squared prediction error scaled by the total DH data variance in the test dataset: Squared prediction error . The closeness was defined by the distance between the gene’s transcription start site and the locus center. Using the selected genes l in a new sample is predicted using N  on a randomly selected set of DHSs . The performance for the neighboring gene approach shown in Fig. N. For instance, Supplementary Fig. rC distribution for different N based on the 9128 DHSs. At N = 15, the mean rC reached its maximum. Correspondingly, the rC distribution shown in Fig. N = 15.For each genomic locus 41) to the same 1% of DHSs. However, the best prediction performance of GAM was worse than the best prediction performance of the linear regression . This indicates that using non-linear model did not improve prediction accuracy. Moreover, the computational time required by GAM was substantially longer than linear regression  were applied . On the basis of the clustering result, the gene expression profile k. BIRD will use gene clusters’ mean expression 43, we further reduce the predictor dimension using a fast variable screening procedure: for each DHS locus l, the Pearson’s correlation between its DH signal  using a fivefold cross-validation conducted within the 40 training cell types  when both methods used the same N and found that N = 5 as the optimal value for N (N = 5) to K = 1500 and N = 7) in Supplementary Fig. In 44, fused lasso15, group lasso16, composite minimax concave penal regression (composite MCP)17, linear regression with stepwise predictor selection45 (SPS), k-nearest neighbors46 (KNN), random forest47 (RF), and principal component regression48 (PCR). This benchmark analysis shows that In Supplementary Note H clusters using k-means clustering (Euclidean distance used as similarity measure). Each cluster of genomic loci is viewed as a pathway. Based on the clustering result, the DH profile h. Instead of predicting the DH level K = 1500 and N = 7 was set for both H was set to 1000, 2000, and 5000, respectively.H = L). As using the cluster mean can reduce the variance of measurement noise, pathway activities are expected to be less noisy than locus-level DH measurements. As a result, To produce the final prediction for a genomic locus, BIRD combines the locus-level and pathway-level models through a weighted average. The rationale is as follows. H values through model averaging. Consider making predictions for a sample. Let H values used by l, let l is a weighted averageBIRD adopts this idea and implements it by combining multiple H, the weight l observed in M training cell types. Each locus l is associated with a cluster. Let l in the M training cell types. Define H = L, where Comparisons between the final BIRD prediction for individual genomic loci and the locus-level prediction by rL, rC, and τ were computed to evaluate the prediction performance of each permutation. The average values of these three statistics from the 10 permutations were used to represent the prediction performance of random prediction models.To construct random prediction models, we permuted the cell type labels of DNase-seq data in the training dataset. This permutation broke the connection between DNase-seq and gene expression data. BIRD was then trained using the permuted training dataset, and the trained model was applied to predict DH in the test dataset. The permutation was performed 10 times. The statistics p-values. For instance, to test whether two methods A and B perform equally in terms of rL, the paired rL values from these two methods for each cell type was obtained. Then the rL pairs from all cell types are used for the Wilcoxon signed-rank test. Similarly, to compare methods A and B in terms of rC, the paired rC values for each locus was obtained, and rC pairs from all genomic loci were used for the Wilcoxon signed-rank test. p-values < 10−4 were marked with “*” and p-values < 10−15 were marked with “**” in Fig. τ) since there is only one τ for each method.In order to compare the prediction accuracy of each pair of methods in Fig. In order to evaluate the ability of a method to predict differential DH signals between two sample types, we first computed the difference in predicted DH value  between two cell types at each locus. We then computed the Pearson’s correlation between the predicted DH difference and true DH difference (determined by DNase-seq) for each pair of test cell types. The analysis was applied to all DHSs and differential DHSs respectively. The differential DHSs were obtained by first filtering out loci without significant DH signals  in both cell types and then collecting all remaining DHSs with |True DH difference  between the two compared cell types |>1. To investigate how the prediction performance depends on the similarity level of the two compared cell types, we calculated the similarity between the cell types using the Pearson’s correlation of their DH profiles . We then grouped all pairs of cell types into four strata based on the quartiles of the correlation coefficients. Figure 49 was set up in the UCSC genome browser for visualizing the predicted DH signal.BIRD prediction models trained using the 57 ENCODE cell types were applied to predict DH levels at the 1,108,603 genomic loci for 2000 human exon array samples obtained from GEO. Each GEO sample in PDDB was annotated using its GSE number, GSM number, cell type, cell status, gender and other information such as age. The predicted DH and the annotation data were both stored in PDDB. A track data hubUsing the PDDB user interface Fig. , users chttp://jilab.biostat.jhsph.edu/~bsherwo2/bird/index.php.PDDB is available at The detailed methods for the analysis of factors affecting cross-cell-type prediction accuracy, comparison of BIRD and ChromImpute, analysis of predictors selected by BIRD, predicting TFBSs, regulome prediction using public expression samples in GEO, predicting differential signals in a differentiation system, and prediction as pseudo-replicate to improve regulome analysis can be found in Supplementary Methods and Supplementary Notes.https://github.com/WeiqiangZhou/BIRD. Models trained using the 57 ENCODE cell types have been stored in the software package. With these pre-compiled prediction models, making predictions on new samples provided by users is computationally fast. On a computer with 2.5 GHz CPU and 10 Gb RAM, it took <2 min to make predictions for ~1 million DHSs in 100 samples.The BIRD software and its source code are available at http://jilab.biostat.jhsph.edu/~bsherwo2/bird/index.php). MYC ChIP-seq data in P493-6 cells are available in GEO (accession number: GSE51004). SOX2 ChIP-seq data in H9 human embryonic stem cells are available in GEO (accession number: GSE46837). The other TF ChIP-seq data and DNase-seq data used in this study are available from the ENCODE (http://hgdownload.cse.ucsc.edu/goldenPath/hg19/encodeDCC), and the download links for each analysis are provided in “Methods” and Supplementary Methods.Exon array data used for training and testing BIRD models are all available in GEO , and the dataset or sample accession numbers used for each example are provided in “Methods” and Supplementary Methods. Accession numbers of the 2000 GEO exon array samples used for constructing PDDB are available at the PDDB website (Supplementary InformationPeer Review fileDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Data 4Supplementary Data 5Supplementary Data 6Supplementary Data 7Supplementary Data 8Supplementary Data 9Supplementary Data 10Supplementary Data 11


But their combinatorial patterns are still unclear. To further understand the relationship between histone modifications and gene expression levels, here in this paper, we introduce a purely geometric higher-order representation, The prediction models were learned from regions around upstream 10k base pairs and downstream 10k base pairs of the transcriptional start sites (TSSs) on three species  with five histone modifications . Experimental results demonstrate that the proposed method is more powerful to predicting gene expression levels than several other popular methods. Specifically, our method enable to get more powerful performance on both commonly used criteria, R and RMSE, as high as 1.7% and 11%, respectively.The overall aim of this work is to show that the higher-order representation is able to include more unknown interaction information between histone modifications across different species. In epigenetics, histone modifications like methylation, acetylation, and phosphorylation play critical roles in transcriptional regulation process . Specifiy as a linear combinations of the interactions between individual histone modification features xi and their products xixj [One of challenges in this study is to discover or characterize what chromatin mark combinatorial patterns can affect the process of gene expression, further revealing complex gene expression mechanisms in downstream analysis –13. Thiscts xixj ; Dong etcts xixj ; Zhou etcts xixj .N-way) are frequently introduced to model higher-order interactions in different research fields [To naturally characterize higher-order interactions between different markers, tensor representation , in which each row indicates histone markers while each column represents the bin we combined see Fig. . To makeIn this study, we used the real data sets which are from lymphoblastoid cell lines (LCLs) over three species, namely Human (GSE47991 and GSE19480), Rhesus Macaque (GSE60269), and Chimpanzee (GSE60269), and these data set are all available in Gene Expression Omnibus (GEO). For each species, eight individuals were considered, and for each individual, 26,115 genes were considered in our experiments.(i)We used COVERAGEBED tool  to conve(ii)x+θ) to normalize the data. In order to obtain the optimal parameter θ in prediction phrase, we divided the whole data set into two parts. One-third of data set was used to optimize the parameters θ, then the optimized θ∗ was added to the same modification of the remaining two-thirds of data set to train the prediction model and test their performance.We used logarithmic transformation log2 was proposed by Bro et al. [. Here, N is the number of order for high-order data N=3), and the variable Ii represents the dimensionality of the mode i. The response variable  is the averaged gene expression levels across eight individuals for all marks.Higher-order partial least squares , b is the regression coefficient with respect to T. The MATLAB code of N-way partial least squares is freely available at: http://www.models.life.ku.dk/source/nwaytoolbox.To solve this model, we want to find the optimal x+θk) with respect to the parameter θk (the parameter for kth bin). In order to determine the optimal Y was also logarithm-scaled using the equation log2(Y). A high-order multivariate regression model was developed using the logarithm-scaled training data set and the 10-fold cross validations was used to avoid the over-fitting in training model phrase. Finally, the performance of methods were measured by the Pearson’s correlation coefficient (R) and root mean square error (RMSE).The workflow of our experiments was given in Fig. In our experiments, to avoid the risk of over-fitting to training prediction model and obtain the reliable results, we used 10-fold cross-validation (with 10 random splitting replicates) in which nine parts are used for training the prediction model, while the remaining part for testing the performance of learned model (similar to previous study ).Y is the real gene expression levels, while I1 is the number of genes; μY is the mean of gene expression levels Y.We also compared with other three popular machine learning-based methods, i.e., simple linear regression (denoted as LR), random forest (denoted as RF), and support vector machine regression (denoted as SVR) to make the proposed method scalable. All these methods were implemented in R, and the parameter of random forest (the number of trees) was set as 500, and support vector regression used default parameters in R. The performance of each methods were evaluated by two criteria, namely Pearson correlation coefficient (R) and root mean square error (RMSE) which are formulated by the following equations: h of their distributions, h=3,2,1,0.5,0.4. 10,000 genes were simulated and the distributions of expression levels for each gene were divided into 100 bins to represent the expression levels of corresponding simulated histone. Similar to the PLS model, we also introduced a latent variable Z to simulate the data in simulation experiments.Our first experiments were conducted on a series of simulation data sets. Interestingly, we found that the distribution of each mark is similar to the normal distribution but does not exactly the same as it. Therefore, in this simulation experiments, we simulated the five different histone marks according to their expression levels, which are described the height Z∼N, then εy∼N and Suppose the latent variable The simulation data β. As shown in Fig. Figure Our second experiments were conducted on real data sets. In this section, we investigated the relationship between gene expression level and chromatin features based on three species . The results from the previous work  have shoHerein, we considered the DNA regions around (10k) at the upstream and downstream regions of the TSSs in current study. In our model, the number of factors is an important parameter to affect the performance. To investigate how this parameter affect the performance of proposed method, we check it under two criteria (R and RMSE) with the varies of factors see Fig. . We can The comparison of the results of four regression models over three species were summarized in Table In this paper, we proposed a higher-order representation method for predicting gene expression levels from chromatin state enrichments. The effectiveness of proposed method was validated by a series of simulation and real data sets. Our method can outperforms others, most likely because higher-order representation method can integrate more unknown interaction information than standard representation method. These results again demonstrate that the gene expression levels are strongly correlated with the combination of chromatin markers.Each chromatin state shows specific functional and annotation. Our study provides a way to study genomic annotation via chromatin mark combinations, which can extend the epigenetic functional interpretation of the human genome. Therefore, our further work is to incorporate epigenetic factors into the downstream analysis, such as gene expression analysis , 31, GO


DNA methylation has been identified to be widely associated to complex diseases. Among biological platforms to profile DNA methylation in human, the Illumina Infinium HumanMethylation450 BeadChip (450K) has been accepted as one of the most efficient technologies. However, challenges exist in analysis of DNA methylation data generated by this technology due to widespread biases.Here we proposed a generalized framework for evaluating data analysis methods for Illumina 450K array. This framework considers the following steps towards a successful analysis: importing data, quality control, within-array normalization, correcting type bias, detecting differentially methylated probes or regions and biological interpretation.Minfi and methylumi are optimal choice when analyzing small dataset. BMIQ and RCP are proper to correcting type bias and the normalized result of them can be used to discover DMPs. R package missMethyl is suitable for GO term enrichment analysis and biological interpretation.We evaluated five methods using three real datasets, and proposed outperform methods for the Illumina 450K array data analysis.  DNA methylation is an important epigenetic modification which has shown numerous associations with biological processes and complex diseases such as diabetes, schizophrenia and cancer –4. HowevIllumina Infinium HumanMethylation450 (450K) BeadChip array, which covers over 480K CpG sites and targets 96% of CpG islands in human genome [β-value, indicating the methylation level of one CpG site, can be computed as β=M/(M+U+α) where α is 100 generally. M-value, M=log2(β/(1−β)), the logit-transformed β-value, is another quantity used in following up analysis.Unlike the previous platform Illumina Infinium HumanMethylation27 (27K) BeadChip, in which only one probe type is utilized, the Illumina 450K BeadChip includes two distinct probe types, Infinium I (n=135501) and Infinium II (n=350076) [β-value upward shift for U and downward shift for M  raw (*.idat) data which is the direct output from Illumina iScan system and stores intensities for each probe, ii) *.txt data, which is usually got after simple preprocessing and is easier to access. Both file formats can be handled by R packages: *.idat files can be read by  package  and *.txe [minfi , wateRmeteRmelon  et al. WDataset 1: Illumina 450K dataset from Dedeurwaerder et al. (GEO accession number: GSE29290) [SE29290) . Three sDataset 2: Illumina 450K dataset of fresh frozen head and neck cancer (HNC) samples form GSE38268 [GSE38268 , where tDataset 3: Illumina 450K dataset of level 1 methylation data of TCGA KIRC samples, which are *.idat files containing 160 normal samples and 325 tumor samples, to evaluate the efficiency of different modules.p-value should be filtered out (e.g. > 0.01), while such probes have a β-value of “NA” in *.txt files. It is worth mentioned that different strategies of assigning missing β-value are applied in different modules. In minfi, the β-values are assigned with “NAs” when both M and U intensities are zero, but an additional criteria is that “NAs” will be assigned if either M or U intensities don’t fluoresce above background. Second, there are 850 inbuilt control probes on the 450K array, such as bisulfite conversion I, bisulfite conversion II, extension, hybridization and negative (n=613), which can be used to evaluate the other probes’ intensities. Samples that can’t pass this quality control are excluded in further analysis. Third, probes on Chromosome X or Y should be filtered out to eliminate the impact of sex on differential methylated analysis in many studies. Fourth, Price ME et al. [β-value of such probes is more likely to represent a combination of multiple sites and not the level of initially targeted CpG sites.After the data imported into R, we would evaluate the quality of data. First, probes displaying a high detected E et al.  reportedE et al. , which ilumi R package provides two different methods for eliminating the background. The first one is based on the negative-control probes inbuilt the BeadChip and the second one estimates the background from the density modes of probes intensities. The second one would show bad performance when there are more than two density modes for some sample. The popular methylumi R package [This step includes background correction and color bias adjustment. Background correction methods have been developed by Triche et al.  such as  package  proposed package .The type bias is the one that is most crucial to correct as it is the main source decreasing the data quality. There have been several efforts to develop methodologies to correct the probe type bias because of the differences between Infinium I and Infinium II. Because Infinium I probes are more stable and reproducible across different samples, most methods reduce the bias of Infinium II rather than Infinium I probes.β-value density curves and is therefore less robust when the methylation density distribution does not exhibit well-defined peaks.The first method is called peak-based correction (PBC) , which rβ-values of CpGs form the same biological category should have the same density distribution. They found that the normalization result of using the “relation to CpG” annotation perfectly corrected the bias.Touleimat and Tost  developeβ-values distribution should be the same when the probes have the same number of CpGs. But SWAN also alters Infinium I probe data, which increases Infinium I technical variation, and does not seem to improve the data quality when applied to some datasets [Subset-quantile Within Array Normalization (SWAN)  was devedatasets .β-value <0.25), hemimethylated  and fully methylated . Then it uses a quantile normalization to fit β-values distribution of Infinium II to the corresponding β-values distribution of Infinium I. This method does not depend on unceremonious choices of biological characteristics to be used to normalize data. Thus it seems more suitable than other methods. However, some points appear worse after BMIQ correction.Beta Mixture Quantile normalization (BMIQ)  method dAnother method, called Regression on Correlated Probes (RCP) , uses a While background is important for measuring absolute methylation levels for single sample/condition experiments, we ignore the background here in this analysis since it can be cancelled out when comparing two conditions. As shown by other studies , widely It is also important to remove non-biological variation called batch effects existing between batches and samples. Such batch effects can influence on measurement of global level that could be partially removed through between-sample normalization using principal component analysis.β-value is the default value for methylation measurement, allowing easy biological interpretation. Another type of value, M-value, is used to express the degree of methylation obtained with Infinium. Due to the heteroscedasticity of β-value, the variance of M-value across the methylation range is approximately constant, so the M-value has better statistical properties. The two types of value are used in different methods.As we mentioned above, the main focus of many methylation studies has been on detecting differentially methylated probes (DMPs) or regions (DMRs) associated with a phenotype. The β-value medians of paired samples is higher than 0.2: β-values of paired normal and tumor samples. The 0.2 threshold represents approximately a difference in methylation level of 20% which can be detected by the Infinium technology with 99% confidence [SQN simply considers a probe as DMP if the absolute value of the difference between nfidence . Then thMinfi offers a comprehensive package to analyze Illumina 450K array data, where candidate regions are determined for DMR analysis and locally weighted scatterplot smoothing (LOESS) is adopted to smooth the methylation differences between groups within each determined region. It also can find long-range alterations such as identified hypomethylated blocks [limma [d blocks  based ons [limma  when sams [limma .Generally, DMRs are detected by applying various statistical techniques such as Fisher’s exact test , 27, t-tThere is a long list of significant CpGs to be interpreted after differential methylation analysis. Using the Infinium annotation file, Illumina 450K probes are classified according to their relations to CGIs and to the closest annotated gene. Regarding their relation to CGIs, the probes are classified into four categories: sites located inside a CGI, sites located in the CGI shores (0-2k bp), sites located in the CGI shelves (2-4k bp) and sites located in the “open sea”. As regards their relation to annotated genes, the sites are categorized as inside the promoter, inside the 5’-UTR region, inside the gene body and inside the 3’-UTR region. Then the significant DMPs can be marked with their related genes.Performing gene set analysis is a popular way to understand the affected potential gene pathways. Although gene set analysis is well established in gene expression experimental, the research in methylation data is ongoing in different groups. In Illumina 450K array, the numbers of CpGs associated with each gene ranges largely from 1 to 1299 . Genes wβ-value for three samples in Dataset 1. Then, we computed the standard deviation across the three replicates.We evaluated the different modules of correcting methods including SQN, SWAN, Dasen, BMIQ and RCP methods on Dataset 1, described in Table β-values of Infinium I probes, the standard deviation of Infinium I probes of the two methods stays the same as the raw data. SWAN shows the least ability of reducing the variation among replicates.As seen, SQN, Dasen and RCP can significantly reduce the technical variation Fig.  and the β-value of Infinium I/II probes for different methods. BMIQ, RCP and SQN show similar performance on the sample GSM815136 such that the Infinium I and Infinium II probes have more similar distribution modes with same local maximum values, while Dasen and SWAN underperformed others regarding removing the type bias  are identified uniquely. But it is also shown that there are differences among results of different methods, which maybe caused by the models they applying to normalization. Then the GO term enrichment analysis were analyzed with gometh function in missMethyl package. The GO terms of FDR ≤0.05 of five methods were got (see Table %) GO terms among the five methods and SQN got the most number (n=199) of GO terms of which 33.67% (67/199) identified by only this method.We used the minfi and meffil [meffil package on a computer with 8 Gb of RAM and 4 processors while minfi could not run on the same computer. Minfi took two hours and ∼ 23 Gb memory on a server to import the same dataset.We use Dataset3 to evaluate the performance of d meffil  in term In this study we propose a generalized framework for Illumina 450K data analysis. We evaluate five methods for correcting type bias. Analysis of reducing technical replicates showed that different methods optimized different assessment criterion. The summary table of evaluation is shown in Table The SQN, Dasen and RCP methods could significantly improve the replicated data quality, while SWAN and BMIQ didn’t show improvement of the replicates. SWAN and Dasen didn’t remove type bias as other methods, which might be due to that models they applying cannot fit the distribution of Infinium I well as other methods.When evaluation focused on detecting DMPs, BMIQ and RCP got more overlapped DMPs and credible GO terms than other methods. It is should pointed out that result may vary largely when using different datasets, which will be validated in further work.Minfi has been utilized widely but it cannot handle large dataset on personal computer in our view while the newly package meffil displayed surprising performance.Illumina MethylationEPIC BeadChip  microarrminfi has been updated in the newest version while the wateRmelon still use the previous object constructor. It is should be noticed in case of using different version of them.During the evaluating processing, there were conflicts between R packages, for example, the MethylSet object in wateRmelon and minfi are different because the one in minfi and methylumi are optimal choice to import data and SQN, BMIQ and RCP may be proper to correcting the Infinium I/II bias. R package missMethyl is suitable for GO term enrichment analysis and biological interpretation. In our view, minfi is a proper R package to import data, eliminate background and ENmix package can be used to correct the type bias, then the normalized data should be used in the remaining steps of the framework.It is suggested that the Illumina 450K users should choose proper strategy about importing data, background eliminating, correcting dye bias, correcting the type bias and detecting DMPs or DMRs. When analyzing small dataset,