Translational medicine offers a rich promise for improved diagnostics and drug discovery for biomedical research in the field of transplantation, where continued unmet diagnostic and therapeutic needs persist. Current advent of genomics and proteomics profiling called “omics” provides new resources to develop novel biomarkers for clinical routine. Establishing such a marker system heavily depends on appropriate applications of computational algorithms and software, which are basically based on mathematical theories and models. Understanding these theories would help to apply appropriate algorithms to ensure biomarker systems successful. Here, we review the key advances in theories and mathematical models relevant to transplant biomarker developments. Advantages and limitations inherent inside these models are discussed. The principles of key computational approaches for selecting efficiently the best subset of biomarkers from high-dimensional omics data are highlighted. Prediction models are also introduced, and the integration of multi-microarray data is also discussed. Appreciating these key advances would help to accelerate the development of clinically reliable biomarker systems. In the new era of biomedical research, it is being increasingly recognized by funding agencies and journals that traditional hypothesis-driven research alone cannot provide the rapid and incremental advances needed to change the current clinical practice management for transplant patients, so as to positively impact long-term graft outcomes. In addition, given that organ transplantation is an orphan disease, there are few if any focused efforts for discovery of new immunosuppressive drugs for transplant recipients. In fact, the number of Food and Drug Administration (FDA) approved drugs has been relatively constant to about 20 drugs per year, yet the cost of drug discovery has ramped up ($138 million in 1975 to $1.3 billion in 2006), and the rate of new drug production by a pharmaceutical company generally follows a Poisson distribution and is constant (about 2–3 drugs per year at most) , had their 1-year protocol biopsies profiled against gene-sets selected after loading cyclosporine and tacrolimus on renal proximal tubular cells, as the patients . To bettpatients , and infpatients , after cHigh-throughput technologies have also been expanded to study the role of microRNAs (miRNAs) in graft rejection in peripheral blood  and the The introduction of high-density protein arrays as allowed for the evaluation of serological responses to ~9,000 human full-length proteins on a single slide. This technology was used to understand the differential immunogenicity of different tissue compartments of the transplanted kidney , and ideRecent advances in small molecule identification technologies  have given rise to the application of proteomics, peptidomics, and metabolomics to transplantation. Urine is a rich biofluid source for biomarker discovery in organ transplantation. Shotgun proteomics provides us with a map the entire urinary proteome  in healtThe integration of hypothesis generation by high-dimensional data analysis is poised to fulfill the current unmet needs in organ transplantation, which relates to poor long-term survival despite improvement in short-term outcomes, the need for life-long immunosuppressive medications and their associated morbidities, and the lack of non-invasive markers for monitoring and predicting graft injury, superior to current standards of monitoring. Computational biology expands from the traditional molecular biological method of studying pair-wise interactions into a network-based approach by integrating individual components to model a complex system, thus beginning to understand disease at the level of regulatory pathways in tissues and organs, even in whole organisms, while also accounting for dynamics within regulatory networks. A large number of computational approaches have been developed to generate co-expression networks from protein binding data , functioTo date, a large number of biomarkers have been identified for various post-transplant conditions as markers of an ongoing injury (effect markers) or related to the actual causes of the injury . Development of new drugs that reduce drug toxicity and chronic rejection requires identification of causal markers that can be targeted for novel therapeutics. The use of systems and computational biology is the critical next step for deeper understanding and the identification of causal markers of graft injury in transplantation.One of the reasons for the limited impact of the high-throughput studies in transplantation relates to low number of individuals and samples used resulting in lack of sufficient independent validation. As described in our previous review , searchiData integration can also be performed across different types of molecular measurements. Li et al. integrated antibody-level measurements from a protein array with renal compartment-specific gene expression data  and demofalse negatives and false positives is another challenge for high-dimensional data analysis. A typical microarray experiment produces approximately 50,000 data points per sample. An experiment with 50 samples will produce more than 2.5 million data points. Millions of data points are generated by SNP genotyping platforms for thousands of samples in a typical genome-wide association study. The amount of data generated increases again exponentially for next-generation sequencing population studies. Incorporation of computational skills into the curricula of transplantation training program needs to be a high priority, to arm the next-generation clinician scientist.The efficient selection of biomarkers to limit T-statistic, and correlation to clinical variables , lower in standard errors and p-values than the actual values, and as models they can be complex to develop.Many regression models are available to select a particular set of independent variables and the commonly used methods are stepwise techniques , 37. TraVorlat et al. used stepwise multiple regression and identified B-type natriuretic peptide (BNP) and age as the most important factors in evaluating outcomes after heart transplantation, after evaluating many variables, such as body mass index, age, BNP, norepinephrine dose, gender, and total ischemic time .i = 1, 2, …, n ;j = 1, 2, …, p ;iy = response variable of sample i, βj = coefficient for gene j, j = 1, 2, …, p, and ijx = observation value of sample i and gene j.Lasso (least absolute shrinkage and selection operator)  is a penLasso estimation actually introduces a penalized constraint to minimize the usual sum of squared errors to get solution. This penalization is estimated by s is set to a large number, it does not affect Lasso estimation that actually acts as a usual multiple linear least squares estimates. Then, a large number of genes might be selected as biomarkers. However, if s is small (s ≥ 0), Lasso works as shrunken least squares regression and then only a few genes would be selected as biomarkers. Lasso has several limitations. For example, the gene number (p) is usually large and sample size (n) is small. In this case, at most n genes are selected by Lasso before the model saturates. In addition, Lasso tends to select the biomarker with greater variance  has been commonly applied in transplant , 46, 47.t-statistic formula, where kw × is is the standard error of the numerator. The only modification is to add s0 as a fudge factor to avoid very large statistics for very small standard errors. Thus, PAM is a modified t-statistic to measure the difference between the mean of gene i in class k with the overall mean of gene i. A gene with a statistic of large absolute value discriminates one class from the rest. PAM then selects significant genes by then shrinking the ikd toward zero.It actually looks like This measurement in PAM actually shrinks each gene toward its overall mean cross classes. After this shrinkage, all class centroids become more similar to each other than before. This might not help to improve the overall discriminant accuracy in omics data.Reeve et al. used PAM to select the most significant genes from 186 microarrays to build a classifier system to predict acute rejection. These genes are mostly associated with interferon-gamma-inducible or cytotoxic T-cell associated, such as CXCL9, CXCL11, GBP1, and INDO .To circumvent the PAM limitations, an alternative classifier called ClaNC  has beenix is a vector of gene i, and E is a eigenvalue.Principal component analysis method selects biomarkers based on an eigengene score , score=|A subset of genes can also be selected by correlating gene expression level and clinical variables, such as patient survival or graft loss. This correlation can be measured by univariate Cox proportional hazards scores that derived from Cox regression model. Genes that pass the interquartile range (IQR) filter are considered as significant , 41.Repeated random sub-sampling validation randomly splits all samples into subsets of samples, a training set, and validating set. The training subset is used to fit a model and the validating set is for examining predictive accuracy. A large iteration of splits  is usually run to avoid splitting sample bias. The accuracy is calculated by the average of all iterations. k-fold cross-validation randomly splits total samples into k subgroups with equal size. One out of k subgroups is treated as testing validation and the remaining k − 1 subgroups are used as training data to train models. The whole process is run k times (k folds). Each time, each of the k subgroups is used as validation but each group is used only once. The sensitivity and specificity can be calculated by combined result from each run. We have extensively used these systems of study  against specificity or false positive rate (1-specificity) at various threshold settings. The sensitivity is equal to the proportion of correctly classified positive observations, and the specificity is calculated as the proportion of correctly classified negative observations. Area under the curve (AUC) serves as estimated index of overall accuracy and serves a useful practice to compare different ROCs, and it is usually plotted within ROC curve. The biomarker panels developed for graft rejection and tolerance in recent studies provide ROC curves of >85%  may be associated with a phenotype of long-term  outcomes. Predictive models have been applied to reach this purpose. Although all machine learning models have predictive functions, such as Support Vector Machine , 39 and k classes with prior probabilities πk, then LDA can be defined below:x) and centroid or mean (u).Linear discriminant analysis has been widely applied to transplant biomarker development , 48. LDAp = probability of one phenotype; ix = ith biomarker value, i = 1, 2, …, k.Logit link function can be understood as follows:Y, and then this function can be simply written as Y = β0 + βX, so it can actually be understood as a linear regression. The phenotype variable in logit could be binary value  or multinomial.The left side of logit link function above can be signed as Besides machine learning, other mathematical methods have also been employed to establish quantitative biomarker system. Here, we introduce one, kSORT , based oMany omic experiments have been performed with different platforms by different laboratories. These omics data cannot be treated as a single experiment and a meta-analysis strategy should be applied to integrate these data to get a panel of prioritized genes. Several techniques and theories have been proposed , 53, 54.For combining small set of microarray data with a big data set, normalization can be performed using Lowess  or Loess  later g before cRobust multiarray analysis (RMA) is a pre-processing algorithm that pre-processes microarray data by background correction, quantile normalization, and summarization in a modular way by fitting the normalized data with models. It is widely used, but RMA cannot be used in clinical data directly because these data are normally in small batches, and clinical samples are normally processed individually and separately and they are comparable. Therefore, a modified RMA, frozen robust multiarray analysis (fRMA), has been proposed. fRMA computes and freezes probe-specific effects and variances of a dataset, and with new data sets coming, these precomputed and frozen info are used in concert with those from the new coming microarrays to normalize and summarize the data. Thus, it provides a way to combine data analyzed individually or in small batches .When merging multiple datasets, the raw data  are usually used in order to pre-processing all data  with the same algorithm and the similar criteria. If multiple probes exist for a transcript, the mean of probes is used to represent the expression level of this transcript. The genome annotation IDs  universal for all platforms are usually employed to combine the data.p-value meta-analysis uses Fisher’s method to combine the squares of the p-values as defined below (k = experiment number, ip is devised from each experiment. This p-value combination would generate a list of genes with meta p-value for each gene. Genes with up- and down-regulation is separated into two groups during combining p-value but only one with minimum p-value is selected. The meta p-value can then further be corrected by multiple hypothesis testing to obtain adjusted p-value. The final adjusted p-value is used to prioritize genes. Lower in adjusted p-value ranked in the top.ed below :χ2k2=−2if = fold-changes in sample i;iw = reverse variance of the if.Another prioritized gene method is based on fold-change meta-analysis as defined below :fmeta=(It should be noted that meta-analyses might produce different rank prioritizations of genes differentially expressed across studies, and the final biological relevance of the selected genes also becomes important in the final gene selection.In order to take our understanding of injury mechanisms in organ transplantations to the next level, integration of molecular measurement data from different experiments and different technologies is required. Furthermore, these integrated data need to be analyzed at a global, systems biology level to identify better diagnostic and therapeutic markers. Predictive biomarkers should cover diverse genetic and epigenetic backgrounds. Clinical and pathology-based variables should be considered as confounding variables during biomarker development. Next-gen sequencing will provide much higher resolution than microarrays to get insights into the diversity of injury and patient-specific responses. With new advances in mathematical theories, a new biomarker system may include many variables from different biology aspects, such as genetics, epigenetics, clinical variables, and pathology.Biomarker discovery suffers from difficulties in selecting “noise” form “true biology” as this discovery relies on human studies and human samples, which are inherently associated with sample and tissue variation, and often result from the use of multiple measurement platforms, with experimental methodology variations, all of which challenge the process of robust biomarkers discovery. Hence, publications suggest that a set of biomarker that works well for one center or one set of patients based on well-conducted statistical methods may not work as well for another center or a different set of patients with variable demographics or other previously unrecognized clinical confounders. It is also important to recognize that most of the biomarkers in transplantation are still in development, as they do not have the support of robust prospective clinical trials. The biomarkers also face the challenge of their correlation being based on histology as the “gold standard,” a standard that we know is not perfect, as it underdiagnoses alloimmune injury and is not really a predictive measure. Thus, most of the biomarkers in research are really just “associated” with histologic findings. It is still unclear if using these biomarkers will ever actually improve graft survival as this requires the conduct of large clinical trials with long-term follow-up, making this research very expensive and often untenable in clinical practice; thus, most omic studies are underpowered with it comes to predicting graft loss. Biomarkers often do not dictate the type of therapy, but accurate prediction of immune risk may allow for their use as companion diagnostics for specific drugs or for safe immunosuppression minimization. Biomarkers may also not always obviate the need for a biopsy, as often they do not differentiate between infections such as polyoma and rejection as both are associated with graft inflammation. Thus, there may still be the need for confirmatory biopsies for the cause or type of organ injury.The fragmented and incomplete nature of the existing knowledge bases poses a challenge to achieving these goals, and wider adoption of a policy to submit raw data into public repository should be required by the transplant-related journals. It is also imperative that the next generation of clinician scientists is armed with computational skills that will ensure novel questions continued to be posed and answered, enabled by the proper integration of diverse sources of data.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


We developed a new computational method, robust to biological and technical variability, which identifies significant transcript isoform changes across multiple samples. We applied this method to more than 4000 samples from the The Cancer Genome Atlas project to obtain novel splicing signatures that are predictive for nine different cancer types, and find a specific signature for basal-like breast tumors involving the tumor-driver CTNND1. Additionally, our method identifies 244 isoform switches, for which the change occurs in the most abundant transcript. Some of these switches occur in known tumor drivers, including PPARG, CCND3, RALGDS, MITF, PRDM1, ABI1 and MYH11, for which the switch implies a change in the protein product. Moreover, some of the switches cannot be described with simple splicing events. Surprisingly, isoform switches are independent of somatic mutations, except for the tumor-suppressor FBLN2 and the oncogene MYH11. Our method reveals novel signatures of cancer in terms of transcript isoforms specifically expressed in tumors, providing novel potential molecular targets for prognosis and therapy. Data and software are available at:  Somatic alterations in the genome can give rise to changes in the transcript isoforms expressed in a cell, thereby affecting multiple functional pathways and leading to cancer –4. TheseWe have developed a new computational method that is robust to biological and technical variability and which identifies significant transcript isoform changes that are consistent across multiple samples. The method is based on a rank algorithm that detects consistent reversals of relative isoform expression, and is capable of detecting complex alternative splicing changes and isoform switches. We used this method to analyze more than 4000 RNA sequencing (RNA-Seq) samples from The Cancer Genome Atlas (TCGA) project. Using the consistency of the change in relative abundance across samples, our method provides predictive models with a minimal set of isoform pairs that can separate tumor from normal samples in each cancer type and classifies unseen tumor data with high accuracy. The same approach finds a significant signature for basal-like breast tumors that distinguishes them from other breast cancer subtypes and which includes the tumor-driver CTNND1. Additionally, we are able to detect isoform switches, for which the relative expression change occurs in the most abundant isoform, and are therefore more likely to have a functional impact. These switches can also accurately separate tumor and normal samples; they affect genes in pathways frequently altered in cancer and 10 of them occur in known tumor drivers. Surprisingly, most of these switches are independent of somatic mutations, except for the tumor suppressor FBLN2 and the oncogene MYH11, suggesting that recurrent isoform switching in cancer is mostly independent of somatic mutations. Our analyses show that recurrent transcript isoform changes provide novel signatures in cancer that could potentially lead to the development of new molecular targets for prognosis and therapy.https://tcga-data.nci.nih.gov/tcga/). To assess sample quality, the estimated read-counts per gene were analyzed using Unveiling RNA Sample Annotation (URSA)   and sampThe abundance of every transcript per sample was calculated in transcripts per million (TPM)  from thehttps://bitbucket.org/regulatorygenomicsupf/iso-ktsp.Changes in the relative abundance of the alternative transcripts from a gene reflect a variation of their relative order in the ranking of transcript expression. Accordingly, the problem of finding alternative splicing changes in cancer at the transcript level is equivalent to measuring the consistency of the reversals in the relative expression of transcript isoforms from the same gene. For this purpose, we developed the iso-kTSP algorithm, which applies the principle of consistency of expression reversals –33 to alS1  and P are the frequencies at which the isoform g,iI appears later than, or before, g,jI in the expression ranking of classes C1 or C2, respectively. Our definition of S1 differs from the one used in  are evaluated on the test set. For each k, the accuracy of the model is evaluated against the test set:kopt isoform-pairs as a minimal predictive model  filter on the relative inclusion levels or PSIs of the isoforms, and keeping those pairs with score S1 > 0.5 and with average expression per isoform of >1 TPM across either tumor or normal samples  on the same input data sets (Supplementary Table S1).The performance of the derived isoform-pair models was compared to the performance of the models based on the expression reversals of genes using the kTSP algorithm , which iM with one or more mutations in either isoform from the pair, and the samples S with the isoform switch, a Jaccard index J for the association of these two variables was calculated as:z-score was calculated by comparing its value J to the J values of 100 genes with similar median isoform length. The above analysis was also repeated using only mutations that affect the protein sequence or considering the overlap with genes rather than transcript regions, obtaining similar results (see Supplementary Methods). The mutual information for the association of isoform switches and mutations, and corresponding z-score were also computed (see Supplementary Methods). To measure the association of mutations to isoform PSI values, the distribution of the differences between tumor and normal isoform PSIs was compared between mutated and non-mutated samples using a Mann–Whitney test. On the other hand, to measure the mutual exclusion between isoform switches and protein-affecting mutations, we used the following approach: given the number of samples having an isoform switch and no mutation (n10), and those having a mutation but no isoform switch (n01), a mutual-exclusion score (mx), with values between 0 and 1, was defined as:N is the total number of samples. A z-score was calculated similarly as above (see Supplementary Methods). Further details and data are provided as Supplementary Material.For the purpose of finding associations between somatic mutations and isoform switches, we considered those samples for which we had both RNA-Seq and mutation data from DNA sequencing (Supplementary Table S2). Using these samples, we compared the number of samples with a given isoform switch with the number of samples for which the transcripts involved in the switch overlap mutations. Given the samples For each cancer type, the iso-kTSP algorithm was applied to the paired samples to obtain minimal classifiers to separate tumor and normal samples. This yielded different predictive models for the 9 cancer types  and moreover appears as a single gene model for COAD and is part of the BRCA model Figure . FBLN2 hWe compared the predictive accuracy of our isoform-based models with the accuracy of gene-based models using the reversal of gene expression, as described by the original kTSP algorithm . We founCancers are generally classified into subtypes to facilitate patient stratification for more precise prognosis and selection of therapeutic strategy. In particular, breast cancer classification has been recently refined based on molecular information from multiple sources . We thusFour different subtypes have been defined based on mRNA expression for the LUSC: basal, classical, primitive and secretory; which have different clinical and biological characteristics . We applColorectal cancers have been classified into hypermutated and non-hypermutated, where non-hypermutated tumors have generally worse prognosis . Followi1S > 0.5, which corresponds to selecting isoform-pairs with a change in more than 75% of the samples. Additionally, we kept those cases for which the relative inclusion levels of the two isoforms in the significant isoform-pair anti-correlate, as observed for FBLN2, QKI and other genes . These isoform switches are thus good predictors of tumor samples. Moreover, the majority of the switches involve a change in the encoded protein product Figure : 176  (Supplementary Table S1A). Importantly, none of the isoform-pairs predicted by SwitchSeq that are not present in our set are significant according to our permutation test. That is, permuting the tumor and normal labels there is a non-zero probability that the same SwitchSeq isoform-pair will appear by chance. On the other hand, when we compared our isoform switches  . We conclude that our isoform switches can be independently validated and that they describe changes that are significant as can be distinguished from the variability originating from the heterogeneity of the samples and inter-individual variability.To further validate our results, we compared them with the results obtained applying SwitchSeq to the same starting data sets. When we compared our significant isoform-pairs Figure  with Swis Figure  with theAs splicing changes may be triggered by genetic mutations , we thusSomatic mutations could also affect the magnitude of the splicing change in specific samples. We therefore tested, for each isoform-switch, whether the presence of mutations in a tumor sample is associated with a larger difference of PSI between the pair of isoforms involved in the switch . Among the most significant cases, we found TGM1 and SLK, both in LUSC . These two cases show differences in the distribution of PSI differences in samples with and without mutations , suggesting that somatic mutations may be partly responsible for the differences in the relative abundance of the two transcripts involved in the switch in these genes. However, the proportion of mutated samples is very small to make a reliable comparison and after multiple-testing correction, none of the found cases remained significant. This suggests that, except for a limited number of cases, mutations may not be the main cause of the recurrent splicing switches found in tumors.We thus hypothesized that mutations and isoform switches may occur independently as two alternative mechanisms of functional transformation in cancer. To test this possibility, we measured how frequently mutations that affect the protein-coding region occur in tumor samples without the isoform switch in the same gene by defining a mutual-exclusion score based on the number of samples with no switch but with protein-affecting mutations . We found that in general the mutual exclusion score correlates with the overall proportion of mutated samples . However, the number of samples with both switch and mutation is generally comparable or higher than the number of samples with mutation and no switch, except for the genes TNC and HECW2 in LUSC, for which we find more samples with a protein-affecting mutation and no switch than with switch and protein-affecting mutation . This suggests that both types of alterations, mutations and alternative splicing changes, could potentially contribute to cancer. We conclude that, although there are currently not a sufficient number of mutations that can provide clear patterns in relation to the described recurrent isoform switches, there are nonetheless a few cases for which this association may exist, as described for the genes FBLN2 and MYH11, and there is some evidence of mutual exclusion between protein-affecting mutations and alternative splicing, like for the genes TNC and HECW2.We have described a novel computational method to study consistent alternative splicing changes across multiple samples from two conditions to find predictive signatures with a potential functional impact in cancer. Classification rules are based on the relative expression of a single pair of isoforms per gene, which corresponds to an alternative splicing change between two conditions. Our method provides robust predictive models despite of the variability of transcript isoform expression across multiple samples, as the models are not dependent on parametrizations or on any normalization that would maintain the order in the ranking of isoform expression. This is especially useful for the analysis of RNA-Seq data from multiple samples, since between-sample normalization methods are not yet fully established. Moreover, our method can be used with data from heterogeneous platforms, as long as they provide a meaningful ranking of expression, which is convenient for the re-analysis of public data sets.The application of our method to RNA-Seq data from the TCGA project has yielded classifiers that can distinguish tumor from normal samples, and between specific tumor subtypes, based on isoform changes. When tested on held-out data sets, the predictive models show overall high accuracies, which are comparable to the ones obtained using models based on gene expression patterns and involve different sets of genes; indicating that alternative splicing alterations describe independent cancer signatures, possibly due to cancer-specific splicing regulatory programs. Moreover, although individual isoform-pair rules do not show in general a strong predictive power, in combination they accurately classify tumor samples in the blind test. This suggests that splicing alterations are heterogeneous across samples, but in combination they provide characteristic signatures, similarly to the patterns of somatic mutations . This heOur analysis shows that isoform changes hold sufficient information to separate tumor and normal samples, and specific tumor subtypes, which suggests that they can serve as effective molecular markers, as they would only require measuring the expression of two isoforms per gene for a small number of genes. On the other hand, among all significant isoform changes, we found 244 isoform switches, for which the change occurs in the most abundant isoform, and are therefore more likely to have a functional impact. The predicted switches are validated using an independent computational method and are found to occur in genes from pathways frequently altered in tumors. Additionally, we find that the majority of the switches involve a change of the encoded protein or a change between a protein coding and a non-coding isoform, which suggests that they may have a functional impact. In particular, we predict isoform switches that affect the encoded protein for the cancer drivers CCND3, MYH11, MITF, RALGD5, ABI1, PRDM1 and PPARG, which may have implications for targeted therapy development. Interestingly, the predicted switches not only separate accurately tumor and normal samples, but they also group tumor samples according to cancer type. Thus, although multiple alternative splicing alterations occur in all cancer types, there seems to be distinctive regulatory programs that contribute to cancer-specific phenotypes. The isoform switches provide thus an opportunity to develop experimental strategies based on the detection of tumor-specific protein isoforms. For instance, QKI has a splicing switch in lung adenocarcinoma that cannot be described in terms of simple events and which has better predictive power than well-known splicing changes in other genes, like NUMB. Similarly, we found switches in genes involved in cell communication pathways, including DST and FLNA, which could be used for developing tumor-specific molecular targets with reduced cross-reactivity to other proteins.Our analyses indicate that somatic mutations occurring on exons and splice-sites cannot explain in general the isoform switching patterns. In particular, 99% of the transcripts analyzed appear mutated in less than 5% of the tumor samples, whereas the switches occur in at least 50% of the samples. Despite the lack of somatic mutations, we found a significant association for two cases: the tumor suppressor FBLN2 and the cancer driver MYH11. Although it has been suggested that synonymous mutations in known cancer drivers may contribute to the oncogenic process , splicinIn conclusion, we have derived accurate predictive models based on transcript isoform changes from multiple patient samples and recurrent isoform switches with potential application in molecular prognosis and for the exploration of novel therapeutic strategies. Our analysis of nine cancer types indicate that recurrent changes in splicing may contribute together with mutations and other alterations to explain tumor formation, thereby providing novel signatures for cancer.Supplementary Data are available at NAR Online.


Breast cancer comprises multiple tumor entities associated with different biological features and clinical behaviors, making individualized medicine a powerful tool to bring the right drug to the right patient. Next generation sequencing of RNA (RNA-Seq) is a suitable method to detect targets for individualized treatment. Challenges that arise are i) preprocessing and analyzing RNA-Seq data in the n-of-1 setting, ii) extracting clinically relevant and actionable targets from complex data, iii) integrating drug databases, and iv) reporting results to clinicians in a timely and understandable manner.To address these challenges, we present OncoRep, an RNA-Seq based n-of-1 reporting tool for breast cancer patients. It reports molecular classification, altered genes and pathways, gene fusions, clinically actionable mutations and drug recommendations. It visualizes the data in an approachable html-based interactive report and a PDF clinical report, providing the clinician and tumor board with a tool to guide the treatment decision making process.https://bitbucket.org/sulab/oncorep/), thereby offering a platform for future development and innovation by the community.OncoRep is free and open-source ( Breast cancer is the leading cause of cancer among females making up 23 % of total cancer deaths . It is aHER2-amplified breast cancers, the mTOR inhibitor Everolimus in hormone receptor positive, HER2-negative patients, and the PARP inhibitor Olaparib in patients whose tumors harbor BRCA1/2 mutations [Molecularly profiling breast cancer tumors takes advantage of the genomic characteristics of the tumor to improve the chances of patient response to targeted agents. This enables stratification of patients based on their molecular alterations. Therapies targeting specific genomic alterations have been shown to be effective in treating specific subgroups of breast cancer patients. Examples of targeted therapies include the efficacy of Trastuzumab in utations -10. Howeutations , but alsGene expression profiling using RNA-sequencing (RNA-Seq) is an ideal tool to assess the molecular heterogeneity of breast cancer to inform individualized medicine. It enables the estimation of transcript abundance, the detection of altered genes and molecular pathways, the detection of fusion genes and the reliable identification of genomic variants -15. RNA-To address these challenges, we developed OncoRep, an open-source RNA-Seq based reporting framework for breast cancer individualized medicine. It can be used as part of the reproducible, automated next generation sequencing pipeline Omics Pipe , as a sthttp://pythonhosted.org/omics_pipe/.OncoRep is developed within the open-source software environments R (v3.0.2)  and BiocThe reference cohort incorporated into OncoRep  consists of 947 breast cancer samples and 106 matched tumor normal tissue samples from The Cancer Genome Atlas (TCGA), one normal breast tissue sample from the Illumina body map project (ArrayExpress accession number E-MTAB-513) and 3 normal breast tissue samples from the Gene Expression Omnibus dataset GSE52194. Level 3 gene expression data (raw read counts) were downloaded as provided for the TCGA samples. The normal samples within E-MTAB-513 & GSE52194 have been downloaded as raw sequence data (.fastq files) and processed using STAR aligner  and htseOncoRep proceses a single patient sample by applying a “documentation by value” strategy . This usQuality control (QC) of raw RNA-Seq reads was implemented using FastQC. Basic QC statistics are listed tabularly and linked to the full report generated by FastQC. Post alignment QC included computation of insert size distribution and collecting basic RNA-Seq metrics using functionalities provided by Picard tools.RNA-Seq reads were aligned to the human genome (hg19) using STAR aligner . AlignmeGene expression quantification was done using the htseq-count function within the Python HTSeq  analysisER+ Training n = 600; validation n = 305; number of genes: 26; overall error rate training: 0.065; overall error rate validation: 0.036 PR+ Training n = 600; validation n = 302; number of genes: 28; overall error rate training: 0.133; overall error rate validation: 0.099 HER2+ Training n = 136; number of genes: 12; overall error rate training: 0.139 Subtype Training n = 346; validation n = 100; number of genes: 254; overall error rate training: 0.248; overall error rate validation: 0.218Using prediction analysis for microarrays , predictPathway analysis was implemented using Signaling Pathway Impact Analysis (SPIA) on the list of differentially expressed genes and their log fold changes identified in the patient sample to identify significantly dysregulated pathways using the Bioconductor packages SPIA  and GrapFusion gene identification was implemented using FusionCatcher . FusionCVariant calling was implemented using SNPiR, a highly accurate approach to identify SNPs in RNA-seq data . Basic gA list of all FDA approved compounds was extracted and integrated with information from DrugBank and KEGG Drug databases, which including meta information about gene targets, pathway involvements and type of drug . Altered genes were matched against these data using the meta information to select appropriate drug-gene partners. Furthermore, variants were matched against SNP-drug relationships available from DrugBank and PharmGkb.OncoRep was integrated as an RNA-seq Cancer Report pipeline in Omics Pipe  which hahttp://sulab.org/tools/oncorep-oncogenomics-report/.The HTML report produces interactive tables that are sortable and searchable. They can be exported as CSV files to be viewed in spreadsheet software. Gene descriptors and drugs are linked to the respective databases for easy access to further information. Pathways are visualized and they are annotated with differentially expressed genes. The interactive HTML reports for the 17 analyzed breast tumor samples can be viewed and browsed at The PDF based report is generated in LATE X, making it fully customizable Fig. . The repOncoRep provides quality control of raw RNA-Seq reads using the FastQC tool. Basic QC results are displayed within the HTML report and linked to the detailed FastQC report for further inspection if needed , and the the oncogenic potential prediction depicted as a p-value and expression gain/loss . A new patient sample is classified into one of the groups and the prediction probability is given. Results are presented in tabular manner in the HTML report.OncoRep reports FDA approved compounds that target the discovered differentially expressed genes, variants and pathways in the patient sample. Results are presented in tabular manner in the HTML report. Results are linked to their DrugBank and KEGG Drug entries for further investigation.In this article, we introduce OncoRep, a reporting tool that performs automated processing and interpretation of RNA-Seq raw data from breast cancer patients. Gene expression profiling using RNA-Seq generates vast amounts of data. This requires precise analyses and expert knowledge to generate clinically actionable information. Without expert knowledge, it remains challenging and time-consuming to do even simple data preprocessing and analysis. In a clinical setting, mosly clinically relevant data like actianable targets are needed from the RNA-Seq data. We address this problem by chaining software tools together to integrate them into a single analysis workflow that is able to deliver clinically digestable information within a short time span. OncoRep enables the prospective use of transcriptomic profiles within a clinical setting by performing molecular profiling, assessing altered genes and pathways, identifying mutations and fusion gene transcripts and by providing drug recommendations based on actionable targets to guide the treatment decision making process. This represents a critical first step towards individualized cancer treatment since it provides a reproducible approach in reporting actionable targets and allows for a quick turnaround time for real-time treatment of patients.OncoRep detects altered genes, variants, fusions and dysregulated pathways in a patient’s tumor. The challenge exists to distill this large amount of information into clinically actionable targets. OncoRep draws from several databases and employs several variant filtering and annotation steps to extract variants that are the most biologically meaningful. Integrating these databases and presenting them in a report provides the community with a valuable resource, as many databases are sparsely populated and information is distributed throughout many poorly curated databases and in the primary literature . OncoRepROS1 expression) for which targeted therapies exist. Since these are found in only a small fraction of patients, these treatments would not be used as standard of care, highlighting the importance of this method for identifying individualized treatments. In addition, OncoRep reports fusion genes and evidence exists that fusion genes may be suitable therapeutic targets. For example, Banerji et al. identified a recurrent MAGI3-AKT3 fusion enriched in triple-negative breast cancer that leads to constitutive activation of AKT kinase, which can be targeted with an ATP-competitive AKT small-molecule inhibitor [By distilling and reporting clinically actionable aberrations on an individual level, OncoRep provides researchers and clinicians with a powerful tool for implementing individualized medicine. For example, an OncoRep report for a patient may detect an aberration that is present in a small fraction of patients : Platform independent Programming language: R Other requirements: Omics Pipe (recommendet) License: MIThttps://bitbucket.org/sulab/oncorepOmics Pipe: https://bitbucket.org/sulab/omics_pipeThe R suite: http://www.r-project.org/Bioconductor: http://bioconductor.org/knitr: http://yihui.name/knitr/knitr bootstrap: https://github.com/jimhester/knitrBootstrapFastQC: http://www.bioinformatics.babraham.ac.uk/projects/fastqcPicard tools: http://picard.sourceforge.net/HTSeq: http://www-huber.embl.de/users/anders/HTSeq/doc/overviewFusionCatcher: https://code.google.com/p/fusioncatcherOncoFuse: http://www.unav.es/genetica/oncofuse.htmlSNPiR: http://lilab.stanford.edu/SNPiRSnpEff: http://snpeff.sourceforge.netIntogen: http://www.intogen.orgClinVar: http://www.clinvar.comDrugBank: http://www.drugbank.caCosmic: http://cancer.sanger.ac.uk/cancergenome/projects/cosmicPharmGKB: https://www.pharmgkb.orgThe Cancer Genome Atlas Data Portal: http://tcga-data.nci.nih.gov/tcgaOncoRep:


More accurate diagnostic methods are pressingly needed to diagnose breast cancer, the most common malignant cancer in women worldwide. Blood-based metabolomics is a promising diagnostic method for breast cancer. However, many metabolic biomarkers are difficult to replicate among studies.We propose that higher-order functional representation of metabolomics data, such as pathway-based metabolomic features, can be used as robust biomarkers for breast cancer. Towards this, we have developed a new computational method that uses personalized pathway dysregulation scores for disease diagnosis. We applied this method to predict breast cancer occurrence, in combination with correlation feature selection (CFS) and classification methods.The resulting all-stage and early-stage diagnosis models are highly accurate in two sets of testing blood samples, with average AUCs  of 0.968 and 0.934, sensitivities of 0.946 and 0.954, and specificities of 0.934 and 0.918. These two metabolomics-based pathway models are further validated by RNA-Seq-based TCGA (The Cancer Genome Atlas) breast cancer data, with AUCs of 0.995 and 0.993. Moreover, important metabolic pathways, such as taurine and hypotaurine metabolism and the alanine, aspartate, and glutamate pathway, are revealed as critical biological pathways for early diagnosis of breast cancer.We have successfully developed a new type of pathway-based model to study metabolomics data for disease diagnosis. Applying this method to blood-based breast cancer metabolomics data, we have discovered crucial metabolic pathway signatures for breast cancer diagnosis, especially early diagnosis. Further, this modeling approach may be generalized to other omics data types for disease diagnosis.The online version of this article (doi:10.1186/s13073-016-0289-9) contains supplementary material, which is available to authorized users. Breast cancer is the most frequently diagnosed cancer in women worldwide excluding skin cancer and it is ranked second for deaths among cancer patients . Early dMeeting these criteria, metabolomics has quickly risen as a new method in the cancer biomarker field. As the final products of various biological processes, metabolites hold promise as accurate biomarkers that reflect upstream biological events such as genetic mutations and environmental changes . DiscoveGiven the observation that metabolites and enzymes involved in the same biological processes are often dysregulated together in cancer , we hypohttps://tcga-data.nci.nih.gov/tcga/). Patient characteristics, staging of disease, and other parameters are shown in Table Three data sets are used in this study: two metabolomics data sets from our own group and one RNA-Seq data set from The Cancer genome Atlas (TCGA) breast cancers. The first metabolomics cohort is composed of 132 breast cancer and 76 control plasma samples and the second independent set comprises 103 breast cancer and 31 control serum samples. All samples were obtained from City of Hope Hospital (COH). This study was approved by the institutional review boards of the City of Hope National Medical Center. All participants signed an informed consent before they participated in the study. Additionally, we downloaded TCGA breast cancer RNA-Seq data from 1082 tumor and 98 tumour adjacent normal controls  from theFor the all-stage diagnosis model, we used 80 % of the plasma (106) and 80 % of the control (61) samples as the training data. We employed three testing data sets, including: (1) the remaining 20 % of the plasma (26) and 20 % of the control (15) samples as the first hold-out testing data; (2) the entire 103 breast cancer and 31 control serum samples; (3) a cohort of 98 pairs of age-matched breast cancer TCGA RNA-Seq data. There is no sample overlap between the training and test sets. To train the early stage diagnosis model, we used the stage I (15 samples) and II (37 samples) subsets of the training data in the all-stage diagnosis model described above, in combination with the 61 healthy control samples.Fasting serum and plasma specimens were collected in the morning before breakfast from all the participants. The samples from controls were obtained from healthy volunteers. The breast cancer patients were newly diagnosed and were not recurrent or on any medication prior to sample collection. All samples were placed into clean tubes and immediately stored within 2 h of collection at −80 °C until analysis.Liquid chromatography/time-of-flight mass spectrometry (LC-TOFMS) and gas chromatography/time-of-flight mass spectrometry (GC-TOFMS) were used for the metabolomics profiling of all blood samples in the study. The profiling procedure included sample preparation, metabolite separation and detection, metabolomics data pre-processing, metabolite annotation, and, finally, statistical analysis for biomarker identification. To eliminate batch effects, all of the plasma samples were processed in one batch, as were all of the serum samples. All annotated metabolites from GC-TOFMS and LC-TOFMS data sets were combined and exported to SIMCA-P+ 12.0 software  for multivariate statistical analysis.http://www.ingenuity.com/), FLink  and PubChem [The names of metabolites are standardized by linking them to Human Metabolome Database (HMDB) IDs, with consideration of synonyms. A comprehensive master file was created which contains the mapping information between 310 human metabolic pathways and affiliated metabolites. Pathway and metabolite information were extracted from HMDB , Small M PubChem . Most ofpathifier [pathifier are described elsewhere [We used the R package athifier  to perfolsewhere , 23. BriFor feature selection from the training data, we used the correlation feature selection (CFS) method implemented in Weka  with tenhttps://tcga-data.nci.nih.gov/tcga/) on 23 October 2015 [pathifier algorithm on the RNA-Seq data and applied limma’s differential t-test to compare the pathway level results with our study. For metabolite level analysis, the enzyme (gene) information for featured metabolites was extracted from KEGG and SMPDB. Limma’s differential t-tests were used for calculation of the p values for each enzyme (gene). Barplots were used for comparison between metabolites and the related enzyme (gene) in breast cancer and normal samples.Breast cancer TCGA RNA-Seq data were downloaded from the data portal . Additionally, the metabolites mapped to pathways are included in Additional file http://www2.hawaii.edu/~lgarmire/MetaboloPathwayModel.html.All the input metabolomics data used for this study have been deposited in Metabolomics Workbench . The metabolomics data include newly diagnosed pre-treatment samples comprising 1) 132 breast cancer and 76 control plasma samples and (2) 103 breast cancer and 31 control serum samples. For the two plasma and serum sample data sets, we conducted metabolomics experiments by both liquid chromatography time-of-flight mass spectrometry (LC-TOFMS) and gas chromatography time-of-flight mass spectrometry (GC-TOFMS). According to the power analysis tool in MetaboAnalyst , the stu 132 breaTo analyze the metabolomics data, we have developed a novel computational pipeline that identifies pathway-based biomarkers for blood-based breast cancer diagnosis Fig. . The esspathifier algorithm to convert the raw metabolite-based data matrix to the pathway-based matrix that contains pathway dysregulation scores (PDS). Pathifier is a non-parametric method for dimension reduction, where a one-dimensional principle curve is derived from a cloud of data points in the high-dimensional space. The PDS is a metric for the degree of pathway abnormality per patient and it is the distance on the principle curve from the starting point to the point projected by a particular and individualized pathway [First, metabolites are mapped to their standardized Human Metabolome Database (HMDB) IDs and the pathway–metabolite relationships are summarized in a master file from multiple resources, including HMDB, Kyoto Encyclopedia of Genes and Genomes (KEGG), Small Molecule Pathway Database (SMPDB), IPA, FLink, Recon 2, and PubChem. Next, we used the  pathway , 22. A PWe first investigated the metabolomics-based pathways as biomarkers to predict breast cancers composed of all stages of tumors Fig. . To seleThe resulting metabolic pathway-based diagnostic model performs very well on all three testing data sets, with AUCs of 0.923, 0.995, and 0.9946 in the hold-out plasma testing samples, serum samples, and TCGA RNA-Seq set, respectively Fig. . Moreovep value of 1.01E-25 for the differential test in the metabolomics data and it is also a top-ranked pathway with a p value of 7.40E-9 in the RNA-Seq data.These eight pathway features are listed in the following in descending order with regard to their relevance, as measured by Mutual Information (MI), for diagnosis: taurine and hypotaurine metabolism; glutathione metabolism; methionine metabolism; glycine, serine, and threonine metabolism; phospholipid biosynthesis; propanoate metabolism; cAMP signaling pathway; and mitochondrial beta-oxidation of medium chain saturated fatty acids. Interestingly, none of the pathways has an MI greater than 0.5, indicating the complexity of the disease and the significance of pathways collectively. Among them, taurine and hypotaurine metabolism stands out as the most important pathway MI = 0.386). Hypotaurine is a product of the enzyme cysteamine dioxygenase, which is involved in protecting against oxidative stress and cancer-induced membrane damage , 29. The86. HypotNext, we identified the measurable metabolites in these selected pathways from both plasma and serum samples and measured their average log fold changes in tumor versus control samples  of stage I tumors, we combined the samples in stages I and II as early-stage cancers and constructed a sub-model to diagnose early-stage breast cancer, similar to the previous all-stage diagnosis model. As expected, the pathway-based early-stage diagnostic model performs very well on the training data set, with an AUC of 0.995. Moreover, it also predicts very well on the three testing data sets, with AUCs of 0.905, 0.902, and 0.999 in the 20 % hold-out plasma testing, serum, and TCGA breast cancer samples Fig. . Other mEight key pathways are identified as diagnostic features for early-stage breast cancer detection Fig. , namely p values in Additional file At the metabolite level, some key metabolites are preserved in the early-stage diagnosis sub-model Fig.  comparedMetabolic regulation is elaborately linked to cancer initiation and progression as proliferating cells demand nutrients for energy production as well as synthesis of genetic materials, proteins, and lipids , 14. AltTo evaluate the pathway-based metabolomics diagnosis modeling approach compared with the commonly used metabolite-based approach, we constructed a “baseline” metabolite-based model using exactly the same CFS feature selection and logistic regression steps used in our pathway-based method. Since the AUC values indicate that the early-stage model is less likely to have over-fitting, we used the early-stage breast cancer data to compare the pathway-based and metabolite-based diagnosis models. In the training data set, the pathway-based approach performs slighter better, with an AUC of 0.995 compared with 0.988 in the metabolite-based approach  followed by the protein digestion and absorption pathway . On the other hand, in the metabolite-based model, the most significant metabolite, cysteine (HMDB00192), has a significant p value of 2.22E-9. These results indicate that the top individual pathway feature may have better diagnostic performance than metabolites.US Food and Drug Administration approval of biomarkers requires the demonstration of the biomarker candidate functions . We thusTo investigate the effect of the number of pathways on the performance of the pathway-based model, we conducted sensitivity analysis as exemplified by the early-stage diagnosis model. We randomly selected half (51) of the initial 101 pathways within exactly the same training sample sets and applied the same CFS feature selection criteria with tenfold cross-validation. CFS selects six pathways for the early-stage model , and their units. Here we have demonstrated that the pathway features obtained from metabolomics data have excellent predictive performance in TCGA breast cancer RNA-Seq data, where both the sample sources and technical platform are different from the metabolomics data sets. Moreover, by projecting metabolite profiles onto pathway profiles, metabolomics data can be integrated with other types of omics data, such as RNA-Seq gene expression, DNA methylation, and copy number variation data.Our results demonstrate that taurine and hypotaurine metabolism is the most indicative pathway for breast cancer diagnosis. Taurine, converted from hypotaurine by hypotaurine dehydrogenase, is intricately linked with alanine and glutamate metabolism Fig. . AlthougWe also found alanine, aspartate, and glutamate metabolism together with the malate-aspartate shuttle to be significant pathways in the early-stage diagnosis model. Aspartate is the key metabolite that shows significantly lowered levels in breast cancer blood samples  approach where as many metabolites/genes as possible are recorded. Compared with other omics data types, metabolomics data are much less standardized across different studies and data repositories are lacking , 60. A cWe have successfully developed a new type of pathway-based model that uses metabolomics data for disease diagnosis. Applying this method to blood-based breast cancer metabolomics data, we were able to discover crucial metabolic pathway signatures for breast cancer diagnosis, which may be valuable for diagnostic tests and therapeutic interventions , 62. Fur


The sequencing of the full transcriptome (RNA-seq) has become the preferred choice for the measurement of genome-wide gene expression. Despite its widespread use, challenges remain in RNA-seq data analysis. One often-overlooked aspect is normalization. Despite the fact that a variety of factors or ‘batch effects’ can contribute unwanted variation to the data, commonly used RNA-seq normalization methods only correct for sequencing depth. The study of gene expression is particularly problematic when it is influenced simultaneously by a variety of biological factors in addition to the one of interest. Using examples from experimental neuroscience, we show that batch effects can dominate the signal of interest; and that the choice of normalization method affects the power and reproducibility of the results. While commonly used global normalization methods are not able to adequately normalize the data, more recently developed RNA-seq normalization can. We focus on one particular method, RUVSeq and show that it is able to increase power and biological insight of the results. Finally, we provide a tutorial outlining the implementation of RUVSeq normalization that is applicable to a broad range of studies as well as meta-analysis of publicly available data. The sequencing of the full transcriptome (RNA-seq) has become the preferred choice for the measurement of genome-wide gene expression. Despite its widespread use, several challenges remain in RNA-seq data analysis. One often overlooked aspect is normalization, which is the transformation of values that allows comparisons between samples in a way that eliminates the effects of sources of variability that are not of interest. We refer to those effects as ‘unwanted variation’. A variety of technical and biological factors, collectively known as ‘batch effects’, contribute unwanted variation to genome-wide gene expression data. These factors include differences in amount of RNA, library preparation, equipment, operators, and procedures for sample extraction, preservation, or storage. Proper normalization, or removal of these factors, has been shown to critically impact the analysis of high-throughput data –3. In spin vivo, because their influence is usually unknown. Nonetheless experimental neuroscience studies will be less influenced by unwanted variables than non-experimental systems, such as studies in human samples.The use of RNA-seq to study gene expression is particularly problematic when it is influenced simultaneously by a variety of biological factors in addition to the one of interest, such as: genetic background, time of day, differences in responsiveness between individuals and cell-type heterogeneity. Proper experimental design is highly beneficial but may not be enough when factors contributing unwanted variation are unknown. In this study we look at recently published studies applying RNA-seq technology in the context of experimental neuroscience. This type of study represents a good middle ground to study the effect of unwanted variables in RNA-seq experiments. Unlike other experimental systems, many unwanted variables cannot be controlled when studying gene expression in the brain Our results show that batch effects are prevalent. We demonstrate that commonly used global normalization methods are inadequate. Normalization methods that can account for batch effects have been recently developed. Focusing on one particular method, RUV , we showPublicly available data were downloaded from GEO (see Supplementary Table S1 for details). Data generated in this article are available through GEO (GSE63412).ad libitum throughout the experiment. Each animal was handled daily for 3 days prior to contextual fear conditioning (FC). Handling consisted of manipulation of the animals for 1–2 min per mouse in the same room as the experimental setting without exposure to the context. The conditioning protocol entailed a single 2-s, 1.5-mA foot shock, terminating at 2.5 min after placement of the mouse in the chamber, starting at 10 a.m. (ZT3) daily. Hippocampal dissections were performed immediately following the behavioral treatment, and alternated between FC and control animals. Tissue was collected at 30 min after FC (FC) as well as 30 min after testing for retrieval of the memory (RT). Testing was performed at 24 h after training over a 5-min interval, which is sufficient to induce reconsolidation  were obtained from Jackson Laboratories and housed individually for a week on a 12 h/12 h light/dark schedule with lights on at 7 a.m. (Zeitgeber time (ZT) 0). Food and water were available lidation . The aveArc, Fos and Dusp1 was confirmed by qPCR. Two microgram of RNA was used for library preparation using the TruSeq RNA Sample Prep Kit  with Poly A selection according to the manufacturer's instructions. Completed libraries were size-selected (200–400 bp) on an agarose gel to remove any high basepair fragments, quantified by qPCR , and submitted to the Penn Genome Frontiers Institute (PGFI) sequencing core at UPENN for sequencing. Three libraries were multiplexed per lane  and sequenced on an Illumina HiSeq 2000 using Type 3 chemistry, resulting in an average of 64 million 100 bp pair-end reads per sample. Data are publicly available through GEO (GSE63412). Sequencing data from RNA obtained from hippocampus of six animals sacrificed 30 minutes following object-location memory (OLM) and their corresponding controls (HC) were generated in the Wood lab at the University of California Irvine and are publicly available through GEO (GSE44229) (RNA extraction was performed using Qiagen RNAeasy Microarray Tissue kit. All RNA extractions were performed the same day within a week of tissue collection. Induction of positive controls after FC O GSE4422 (9). ReaSE44229) . Only geP-value >0.8 in all pairwise comparisons between RT, FC and CC in microarray data available through GEO (GSE50423) .All analyses were performed using open source software freely available through the R/Bioconductor project . RLE andSE50423) . The R cEnrichment of functional annotation on differentially expressed gene sets was assessed using the Database for Visualization and Integrative Discovery (DAVID) . Functioin vivo, available in GEO (http://www.ncbi.nlm.nih.gov/geo/). We required that the studies met minimal quality criteria: have an associated PMID, were sequenced with Illumina HiSeq technology at a depth of at least 10 million reads per sample, include at least six samples with a minimum of 2 biological replicates per condition and include gene-level read summaries. A summary of the studies can be found in Supplementary Table S1. We also included our own previously published study following object location memory . All studies used established methodology for RNA-seq data analysis and normalization methods that only correct for sequencing depth, which is standard practice. The studies cover a variety of experimental manipulations used in neuroscience research, such as comparison between knock-out versus wild-type animals, injection of shRNAs to inhibit the expression of a gene, age, induction of neurodegeneration, and learning and memory paradigms.To assess whether unwanted variation is a problem within RNA-seq studies in experimental neuroscience, we re-analyzed studies of the mouse hippocampus The general assumption of most studies is that the experimental manipulation of interest is the main source of variation in the data. The main problem when studying the brain is that a lot of variables cannot be controlled, so this assumption may not hold true. One way to visualize the sources of variation in the data is to use principal component analysis (PCA). The use of PCA for data exploration and quality control is an established practice in genome-wide expression studies. PCA is a statistical procedure that looks for a small set of linear combinations of the original variables to summarize the data losing as little information as possible . These lTo further investigate how normalization of RNA-seq affects the detection of differential expression in the brain, we focused on long-term memory formation, since learning and memory paradigms are particularly problematic Figure . One of P-value of differential expression between CC, FC and RT was >0.8. A full list of negative control genes can be found in Supplementary Table S3. Negative control samples were constructed by computing differences of biological replicates within the same treatment condition as detailed in  and is not sufficient to normalize the samples (both in supervised and unsupervised mode). When manually including 5 surrogate variables in the SVA model, the results are similar to those observed for RUV. Thus, the choice of k is a key factor in achieving proper normalization, whether RUV or SVA are used.The ability of RUVs to restore the expected distribution to RLE and PCA plots is also seen in the OLM data . We also evaluated the ability of SVA and PEER to remove unwanted variation from the FC data . Both methods constitute an improvement over global normalization methods, but only SVA is effective. PEER needs the specification of the number of factors of unwanted variation; both P-values (as we did in this article) or entropy  or when using negative control samples and considering all genes as negative controls . We have previously demonstrated that in fact RUV (and RUVs in particular) is quite robust to the choice of negative control genes . The deg entropy .P-value histograms of DE between CC and FC samples following UQ and RUVs normalization. A satisfactory P-value histogram should contain a sharp peak at zero representing genes with strong DE and a ‘floor’ of values that is approximately uniform in the interval , corresponding to genes that are not DE. Lack of uniformity in the p-value distribution, such as the one observed after UQ, suggests the presence of confounding variables not accounted for in the model. RUVs restores uniformity to the p-value distribution and increases the number of genes identified as DE (the height of the peak at zero). Figure To evaluate the impact of normalization methods on differential expression DE), we analyzed UQ and RUVs normalized data using edgeR ,32. FiguE, we anaTo evaluate the impact of normalization on the quality of DE genes, we collected a set of positive control genes whose expression changes have been previously validated (Supplementary Table S3). Figure The ability to integrate datasets, either within multi-site sequencing projects or to perform meta-analysis of publicly available data holds great promise as a way to increase biological insight while maximizing investment of resources. The challenge of cross-site data integration is that differences in protocols, reagents and operators will produce site-specific batch effects that can obscure the biological signal. To evaluate the ability of RNA-seq normalization methods to deal with multi-sites batch effects, we combined the FC and CC samples obtained in the Abel laboratory with the HC and OLM samples obtained in the Wood laboratory. Figure Step 1 is to explore the distribution of variation in the data both before normalization and following normalization for sequencing depth only , which can be done by constructing the RLE and PCA plots previously shown, using a publicly available package such as EDASeq Figure s EDASeq . If the s EDASeq  and C, asing RUV  or similsing RUV  and D. Sr DESeq2 . Check PWe have illustrated the prevalence of batch effects in RNA-seq studies using examples from experimental neuroscience and shown that traditionally used methods for RNA-seq data normalization are not able to remove them. This problem likely extends to a variety of datasets for which sources of variation are hard to control. These limitations can lead to the identification of a small number of confident changes  and/or to uncovering statistically significant changes that are not due to the treatment, which will not be reproducible . Here, we present a novel application of a recent method for RNA-seq normalization, RUV and show that it is better able to correct for unwanted sources of variation when proper controls are provided. We show that within laboratories, RUV considerably increases the number of genes discovered as differentially expressed. We demonstrate that this increase is indeed an improvement in the detection of true biological signal by showing that it increases the discovery of positive controls, known pathways involved in learning and memory and cross-platform concordance. When integrating data across different laboratories, commonly used normalization methods lead to artifacts that cause housekeeping genes to be inferred as differentially expressed. In contrast, RUV normalization is able to properly correct the biases introduced by integrating data from different sites, allowing for direct comparisons regarding differential gene expression following two different behavioral paradigms. These analyses suggest that both contextual fear conditioning and object location memory induce the same changes in gene expression. Finally, we outline a step-by-step guide on how to detect sources of unwanted variation in the data and apply RUV to remove this variation prior to differentially expression analysis. We hope these guidelines together with all the datasets generated in this article will serve as resource, for both authors and reviewers, to ensure that results obtained using high-throughput sequencing technologies are reproducible and thus truly contribute to the advance of knowledge in science.Publicly available data was downloaded from GEO ,40–45 (s


Variations in sample quality are frequently encountered in small RNA-sequencing experiments, and pose a major challenge in a differential expression analysis. Removal of high variation samples reduces noise, but at a cost of reducing power, thus limiting our ability to detect biologically meaningful changes. Similarly, retaining these samples in the analysis may not reveal any statistically significant changes due to the higher noise level. A compromise is to use all available data, but to down-weight the observations from more variable samples. We describe a statistical approach that facilitates this by modelling heterogeneity at both the sample and observational levels as part of the differential expression analysis. At the sample level this is achieved by fitting a log-linear variance model that includes common sample-specific or group-specific parameters that are shared between genes. The estimated sample variance factors are then converted to weights and combined with observational level weights obtained from the mean–variance relationship of the log-counts-per-million using ‘voom’. A comprehensive analysis involving both simulations and experimental RNA-sequencing data demonstrates that this strategy leads to a universally more powerful analysis and fewer false discoveries when compared to conventional approaches. This methodology has wide application and is implemented in the open-source ‘limma’ package. Second-generation sequencing technology provides researchers with a high resolution and cost-effective tool for surveying the complexity of the transcriptome in both health and disease. RNA-sequencing (RNA-seq) is now a routine tool for studying differential expression, alternative splicing, allele-specific expression and for discovering novel transcripts . In an RSmchd1) has been mutated. In both experiments, one or more samples exhibit increased variation that is likely to be related to RNA quality (e.g. degraded RNA may be unavoidable in studies involving patient tissue) or sample purity. In our experience, sample quality variation is relatively common in RNA-seq data sets, and once identified, the researcher may choose to re-run the suspect samples or, in extreme cases, collect fresh samples. These options may not always be feasible due to resource constraints, therefore developing analytical methods that can address this variability and make the most of the data available would be advantageous.Variations in sample quality is another source of noise that makes the identification of differentially expressed (DE) genes more difficult. Sample-specific variation is often first observed by inspecting plots from a principal components analysis or multi-dimensional scaling (MDS). Figure ad-hoc cut-offs and a trial and error approach. At the other extreme, one could retain all samples in the analysis. In this situation, our ability to distinguish genuine differences between experimental conditions from the noise will be limited by the increased variation present. An intermediate approach would be to analyse the complete experiment but down-weight the observations from more variable samples, thus retaining the maximum degrees of freedom whilst discounting noisy observations.There are currently no methods that can model sample-specific variation in small-scale RNA-seq experiments in a systematic way. One option is to remove samples with higher variability from the analysis. This has the benefit of reducing variation, but comes at the expense of reducing power, which can hamper our ability to detect DE genes. This approach assumes that samples with higher than average variation can be readily identified, which is not always the case. The decision of whether to keep or discard a particular sample is often based on et al. (2006)   has allo. (2006) .We adapt this method to enable the use of quantitative quality weights in RNA-seq data sets to improve the results obtained in the presence of more variable samples. We model this variation using a combined approach that takes into account both global intensity-dependent trends in the variability using ‘voom’  as well 2 counts per million (CPM) so that they are approximately normally distributed. The work of Law et al. (2014) (2(CPM) transformation when combined with weights that take into account the unequal variabilities on this scale can lead to a more powerful analysis compared to methods that assume RNA-seq counts follow a negative-binomial distribution. Assuming normality allows us to model the variances, which we will exploit later. Variance modelling is not currently available for RNA-seq analysis using count distributions.The entry point for our analysis is a matrix of counts that have been appropriately normalized and transformed into log. (2014)  has showyg = T is the vector of log2(CPM) values for gene g, X is a design matrix with full column rank and βg = T is a gene-specific vector of regression coefficients (g in sample j and We begin by assuming a linear model where the systematic expression effects for each gene (or exon or other genomic feature of interest) can be described asficients . The desm’ model  for geneygj are normally distributed and that expression values from different samples are independent, the weighted least squares estimator of βg ist-statistic for testing any particular kth diagonal element of t-statistic has J − K degrees of freedom, and allows genes to be ranked for differential expression based on their P-values from this distribution (adjusted for multiple testing), or testing can be performed relative to a chosen fold-change (FC) (If we assume that the n square  from thenge (FC) .et al. (2006)  (The approach of Ritchie . 2006)  allows umentclass2pt{minim  allows Z for the variance model that reflects this grouping structure.We generalize this approach to allow either individual sample-level variance factors , we begEquation ) are usementclass2pt{minim package . This wo package –19 tool  package  to facil package  programmNo Weights: linear modelling on the full data set where all observations are treated equally in the analysis . This is included to provide a measure of performance under a worst case scenario.Sample Weights: linear model analysis on the full data set with sample weights  only (iEquation ).Voom: linear model analysis on the full data set with observational level weights ( weights  only (i.Voom + Sample Weights: linear modelling with ‘voom’ weights combined with sample-specific weights as described in the previous section (Equation (Equation ).Voom + Block Weights: linear modelling with ‘voom’ weights and block weights as described in the previous section (Equation (Equation ). This amentclass2pt{minimSample Removal: discard more variable sample/(s) from the analysis  and apply linear models with ‘voom’ weights to the reduced data set . For a given data set, the same design matrix (Xg) was used for methods 1–5, whilst for method 6 the design matrix was reduced by the outlier sample/(s). For all methods, correction for multiple hypothesis testing was carried out using the false discovery rate (FDR) approach  was carapproach .et al. . We also generated data for an unbalanced design , 1.5 (150%), 2 (200%), 3 (300%), 4 (400%) or 8 (800%). A null simulation where all samples had equivalent variability (i.e. variance inflation factor = 1) was also simulated to provide an example of a well-behaved data set. The library size of the more variable sample was also varied to be 50, 60, 70, 80, 90 and 100% of the size of the other samples (2 × 107 by default). Null simulations  for experiments with n = 3 samples per group across the range of variabilities listed above were also simulated (sample library size =100% for the more variable sample).Simulated data sets were generated by adapting the approach described in Law et al.  to incluj by a factor k, where k = 1 , 1.2 (120%), 1.5 (150%), 2 (200%), 3 (300%), 4 (400%) or 8 (800%).Baseline expression values were first generated to get a relative proportion of counts for 10 000 genes. Next the proportions were converted into expected counts by multiplying by library size and then multiplying by the desired FCs (1 (non-DE), 1.5, 2 or 4) for the first 200 genes. Negative binomial distributed counts were obtained with the specified mean and dispersion for each observation. To obtain dispersions, a trend was set to be 2(FC) versus average log2(expression level) from these simulations and Figure n = 3 samples per group with equal library sizes for each sample variability setting. The position of the more variable sample  relative to the other samples recapitulates the separation seen in the MDS plots in Figure Figure In total there were 511 simulation settings  that were each simulated 100 times. Prior to differential expression analysis, each simulated matrix of counts underwent filtering to remove genes with fewer than 10 simulated counts across all samples. Code and plots of all results are provided as ‘Supplementary Materials’.EGFR mutation) and gene expression profiles compared to other lung cancer cell lines included in the Cell Line Encyclopedia . Cell lAll mixtures corresponding to the second replicate were split into two equal aliquots. One aliquot was left intact (we refer to this as the ‘good’ replicate), whilst the second aliquot was degraded to increase variation by incubation at 37°C for 7 days in a thermal cycler with a heated lid. The RNA Integrity Number determined using TapeStation RNA ScreenTape (Agilent) was below 5 for the degraded samples and above 8 for the intact samples. This experimental design allowed us to perform an analysis on the regular samples  and compare our results to the analysis that included the degraded samples. Ten microlitres from each replicated mixture (both good and degraded) were used for Next Generation Sequencing library preparation using Illumina’s TruSeq Total Stranded RNA with Ribozero according to the manufacturer’s instructions. Library clustering was performed on a cBot with Illumina HiSeq SR Cluster Kit v4 cBot. Libraries were sequenced as single-end 100 base pair reads at the Australian Genome Research Facility on an Illumina HiSeq 2500 with an Illumina HiSeq SBS Kit, v4. Base calling and quality scoring were performed using Real-Time Analysis (version 1.18.61) and FASTQ file generation and de-multiplexing using CASAVA (version 1.8.2). Reads from FASTQ files were aligned to the human genome (hg19) using Subread (version 1.16.1)  and summ6Smchd1+/+;EμMycTg/+ and Smchd1MD1/MD1;EμMycTg/+ lymphoma cells using Qiagen RNeasy Minikit as per the manufacturers instructions. Libraries were prepared using Illumina’s TruSeq RNA sample preparation kit as per the manufacturers instructions and submitted to the Australian Genome Research Facility for quality control, library preparation and sequencing on the Illumina HiSeq 2000 platform using 100 base, paired end or single-end reads. Base calling and quality scoring were performed using Real-Time Analysis (version 1.17.21.3) and FASTQ file generation and de-multiplexing using CASAVA (version 1.8.2). Reads from FASTQ files were aligned to the mouse genome (mm10) using Subread (version 1.10.5)  samples, with genes ranked according to their FDR (RNA was extracted from 1 × 10 1.10.5)  and summ 1.10.5) . Subsequ 1.10.5)  and ‘lim 1.10.5)  Biocondu 1.10.5)  before lheir FDR . These dSmchd1 has been shown to have a role in the regulation of clustered protocadherins and imprinted genes in diverse tissues including whole embryo, adult brain, embryonic fibroblasts, placenta and malignant and normal B cells . Higher abundance observations tend to be more precise and are rewarded with higher relative weights whilst low abundance observations which tend to be less precise receive lower weights.The first level of variability we deal with is at the observation-level and is related to the abundance of a gene in each sample. Figure ’ method  to derivj is returned for use in the linear model analysis. We allow weights to be either distinct for each sample . Unliket Figure  that samThe purpose of combining sample-specific weights with ‘voom’ weights is to obtain more precise estimates of the gene expression coefficients in the linear model (Equation ) and impn = 3 samples per group, 10 000 genes). Overall the number of errors decreases for all methods as the absolute FC of the TPs increases, which is to be expected, as the problem of distinguishing DE genes from non-DE genes becomes easier for larger effect sizes . Figure The empirical FDRs for each method on the same data using the same cut-off (FDR < 0.1) is also shown in Figure P-values should have an approximately uniform distribution, meaning that the expected proportion of P-values below a certain cut-off should be less than or equal to this value. For this analysis we use the raw P-value cut-off of 0.01 (grey dashed line) and plot the proportion of genes that have a P-value less than this threshold. The sample removal method (black line) depicts results from applying ‘voom’ alone when the noise level is constant and, as we would expect, has a fairly constant type I error rate. Across a range of variability inflation factors for the sixth sample, the new weighting methods approximately hold their size, even when this sample is eight times more variable than the others, being only slightly more conservative than removing the sample altogether. Treating all observations as equal (no weights) or using ‘voom’ on the complete data set becomes gradually more conservative as sample variability increases, reporting around half as many genes than would be expected by chance when the more variable sample is eight times as variable as the other samples.We also examined how the different alternatives control the ‘Type I’ error rate in a simulation where there is no differential expression . Ignoring the simulated variability and treating all observations equally or applying sample weights only delivers the poorest results. We therefore exclude these two alternatives from subsequent analyses. Whilst these simulations were intended to approximate RNA-seq data and reflect the variations observed in practice, we now shift our focus to the analysis of the ‘Control’ and the To simulate variations in sample quality experimentally, we performed a control experiment where particular samples were mistreated by applying temperature . The degraded samples were successfully detected by our combined ‘voom’ and sample-specific weighting method and down-weighted in the analysis Figure . ApplyinTable Smchd1 in lymphoma cell lines. This experiment consisted of three wild-type and four mutant samples . Table Smchd1, the gene that was mutated in this study, with an FDR of 1.98 × 10−5, closely followed by ‘voom’ with block weights (5.14 × 10−5), then ‘voom’ on the full data set (2.66 × 10−4) and lastly ‘voom’ after sample removal (4.28 × 10−3).In this second example, we analyse data from an RNA-seq experiment that aimed to identify genes that are transcriptionally regulated by Smchd1 in previous studies (the Protocadherins and a subset of Imprinted genes) (Smchd1 has been lost (Smchd1MD1/MD1) against wild-type (Smchd1+/+) samples and expect genes directly regulated by Smchd1 to increase in expression under this comparison. The P-values in Table P-values for up-regulation .The two gene signatures that have been reported to be regulated by d genes) –32 were d genes)  gene setWe have shown that modelling heteroscedasticity at both the observational and sample-level can enhance the results of an RNA-seq differential expression analysis. Simulations demonstrated that combining ‘voom’ with sample-specific weights can lead to a more powerful analysis with a low FDR relative to other alternatives, such as either ‘voom’ or sample-specific weighting on their own. The analysis of data from a specially designed control experiment and a gene mutation study that each contained more variable samples showed us that the extra discoveries made by our weighting strategy were likely to be biologically meaningful, as they were either in better agreement with the results obtained from analysing a clean version of the same data set or better able to recover known gene signatures. The weights derived by this approach are propagated through each step of a ‘limma’ analysis, including gene set testing.Although we have demonstrated this approach using simple two group experimental designs with varying sample sizes, in practice more complicated experimental designs can be accommodated just as easily. Our method also allows flexibility in the structure of the sample-level variance model. The default sample-specific mode is recommended in most situations as it offers excellent performance and does not require any special input from the user. Where more information is available, the user is free to specify other configurations, such as a block structure where the more variable samples are given distinct weights relative to the remaining samples. This approach relies on the data analyst being able to identify suspect samples in advance of fitting the model, which may not always be possible. A further possibility that this approach allows that we have yet to explore is the modelling of group-specific differences in variability .We have intentionally focused on the performance of these methods on small data sets, as these are not only the most common in our experience, but also where the most gains can be made by modelling the residuals ‘between’ genes to get a handle on the sample-level variation that is present. For larger experiments, the usefulness of other approaches such as robust methods  that canIt is important to note that the minimum sample size where this approach would be recommended is an experiment with three or more samples per group. Theoretically, the variance model can be fitted in smaller studies with a minimum of two samples per group. When a block model is fitted, the software will return block-wise weights even when some or all of the groups have fewer than three samples. When a sample-specific model is fitted, both samples in a group of size 2 will be assigned equal weight , which may or may not be desirable. In an experiment with three samples in one group and two in a second, combined ‘voom’ and sample-specific weights could be beneficial, so long as the more variable sample belongs to the larger group.One obvious consequence of a more powerful analysis through combining ‘voom’ and sample-specific weights is that it may result in a large number of genes passing a given FDR cut-off. In situations where there is a need to further refine the list of genes, we recommend the user apply a test for a given FC cut-off using the TREAT method .We anticipate that this approach may also be useful for dealing with more variable samples in differential binding analyses for chromatin immunoprecipitation sequencing studies or single cell transcriptomic profiling experiments, when the goal is to summarize over replicate samples and look for differences between experimental groups. Further work will be to adapt this approach for incorporation in edgeR , which cvoomWithQualityWeights function in the open-source ‘limma’ package distributed as part of the Bioconductor project (http://www.bioconductor.org/). A Galaxy tool that includes the option to apply ‘voom’ with sample-specific weights in an RNA-seq differential expression analysis is available from the Galaxy Toolshed at https://toolshed.g2.bx.psu.edu/view/shians/voom_rnaseq. The R code and plots of results for all simulation settings along with the R code to carry out the analyses of the ‘Control’ and ‘Smchd1’ RNA-seq experiments are provided as ‘Supplementary Materials’ at http://bioinf.wehi.edu.au/voomWithQualityWeights/. Experimental data are accessible through GEO series accession numbers GSE64098 and GSE64099.The weighting methods described in this paper are implemented in the