Studies on gene expression in response to therapy have led to the discovery of pharmacogenomics biomarkers and advances in precision medicine. Whole transcriptome sequencing (RNA-seq) is an emerging tool for profiling gene expression and has received wide adoption in the biomedical research community. However, its value in regulatory decision making requires rigorous assessment and consensus between various stakeholders, including the research community, regulatory agencies, and industry. The FDA-led SEquencing Quality Control (SEQC) consortium has made considerable progress in this direction, and is the subject of this review. Specifically, three RNA-seq platforms  were extensively evaluated at multiple sites to assess cross-site and cross-platform reproducibility. The results demonstrated that relative gene expression measurements were consistently comparable across labs and platforms, but not so for the measurement of absolute expression levels. As part of the quality evaluation several studies were included to evaluate the utility of RNA-seq in clinical settings and safety assessment. The neuroblastoma study profiled tumor samples from 498 pediatric neuroblastoma patients by both microarray and RNA-seq. RNA-seq offers more utilities than microarray in determining the transcriptomic characteristics of cancer. However, RNA-seq and microarray-based models were comparable in clinical endpoint prediction, even when including additional features unique to RNA-seq beyond gene expression. The toxicogenomics study compared microarray and RNA-seq profiles of the liver samples from rats exposed to 27 different chemicals representing multiple toxicity modes of action. Cross-platform concordance was dependent on chemical treatment and transcript abundance. Though both RNA-seq and microarray are suitable for developing gene expression based predictive models with comparable prediction performance, RNA-seq offers advantages over microarray in profiling genes with low expression. The rat BodyMap study provided a comprehensive rat transcriptomic body map by performing RNA-Seq on 320 samples from 11 organs in either sex of juvenile, adolescent, adult and aged Fischer 344 rats. Lastly, the transferability study demonstrated that signature genes of predictive models are reciprocally transferable between microarray and RNA-seq data for model development using a comprehensive approach with two large clinical data sets. This result suggests continued usefulness of legacy microarray data in the coming RNA-seq era. In conclusion, the SEQC project enhances our understanding of RNA-seq and provides valuable guidelines for RNA-seq based clinical application and safety evaluation to advance precision medicine. Knowingpression . RNA-seqpression . HoweverSeveral relatively independent but complementary components were included in the SEQC project: the core study with titration reference samples , neuroblAs successfully applied in the MAQC-I project, titration study has proven to be a promising way to assess platform reproducibility and reliability . Similarde novo junctions than Magic and Subread. In total, 2.6 million new splice junctions were discovered by at least one pipeline, however only 32% of them were called by all three methods. This relatively low concordance indicates that it is still challenging to reliably detect de novo splice junctions with current tools. For platform comparison, 97,117 unannotated junctions were discovered in the SOLiD data and 74,561 (86%) of them were concordant with those from HiSeq 2000 data.The influence of gene model, pipeline selection and sequencing platform was assessed in terms of gene detection and splice junction discovery . For genBuilt-in truth was also applied to assess the accuracy of novel junctions to see whether their expression levels followed the titration order and the expected A/B mixing ratio (Truth I and II). Junctions with higher expression levels were usually concordant with the titration truth. Moreover, most of these unannotated junctions could be further validated by qPCR and even 18 of 22 junctions detected by only one pipeline could be qualitatively validated. As the last part of the assessment of gene detection and splice junction discovery, all sequencing data of Samples A–D from each site was pooled together and analyzed. In total, ~44,000 known genes, ~310,000 known exons and ~200,000 splice junctions were consistently detected across any pair of replicate sites, which constituted about 90%, 87%, and 83% of all known genes, exons and junctions.p-value, fold change, and expression-level could significantly reduce the false positives while retaining high sensitivity [For differential expression analysis, applying pipeline-dependent filters on sitivity ,14. By aSimilarly, the built-in truth was applied to evaluate the performance of gene expression quantification. The majority of genes showed correct titration order (Truth I), where highly differentially expressed genes performed the best. Similarly, highly expressed genes performed better in mixing ratio recovery (Truth II and IV). However, the recovery ratio of spike-in controls showed a high variation across platforms and sites, indicating a clear batch effect on reproducibility. To facilitate the usage of the ERCC spike-in for technical performance evaluation of RNA-seq experiments, a software tool named “erccdashboard”  has beenAdditionally, gene expression measurements by different platforms, including TaqMan, Affymetrix Human Genome array, qPCR and RNA-seq platforms were compared. Relative expression levels showed a good agreement among different platforms but substantial variations were observed in absolute expression levels for numerous genes. This clearly demonstrated that relative expression measures were more reliable than absolute quantification. In summary, the core study with titration reference samples has demonstrated that RNA-seq is accurate and reproducible both across sites and across platforms. It has also confirmed the power of RNA-seq for discovering novel junctions. These findings have further consolidated the foundation of RNA-seq as a reliable and versatile tool for studying gene expression and thus will accelerate its application in pharmacogenomics and precision medicine.Study of cancer is the disease area where most of the microarray and RNA-seq data are generated. The primary goal of the neuroblastoma patient outcome prediction component see B was to Gene expression profiles from 498 primary neuroblastoma samples were generated using both RNA-seq and microarray [vs. microarrays) nor the data processing pipeline of RNA-seq could significantly affect the model prediction performance. Additionally, the correlation of performance between internal and external validation for microarray- and RNA-seq-based models was almost identical.To construct and assess predictive models, the whole cohort of 498 neuroblastoma samples was divided into the training and validation set. In total, 360 predictive models were constructed on six endpoints such as patient’s sex (SEX), event free survival , overall survival , favorable disease outcome , and EFS and OS in high risk patients (EFS_HR and OS_HR). These six endpoints were designed to cover a broad range of predictive difficulties, from easy  and medium  to difficult (EFS_HR and OS_HR). Multiple data analysis teams were then involved in the model construction with various data processing pipelines and expression features for RNA-seq. As a result, the predictive performance was largely determined by the endpoints and neither the technology platform , nearest shrunken centroids (NSC), and support vector machine (SVM) were adopted to represent three levels of complexity from the simplest to the most complex. In total, 240,096 predictive models were examined. Across all combinations of mapping and modeling complexities, signature genes derived from one platform can be transferred to build models for the other platform with little loss in prediction performance. However applying predictive models directly from one platform to another would lead to mixed results. For most combinations of mapping and modeling complexities, microarray-based models could accurately predict RNA-seq-profiled samples. On the other hand, RNA-seq-based models were less accurate in predicting microarray-profiled samples and were affected both by the choice of modeling algorithm and the gene mapping complexity. This investigation suggests continued usefulness of legacy microarray data and established microarray biomarkers and predictive models in the forthcoming RNA-seq era.In addition, the SEQC consortium also made a coordinated effort to address the critical challenges in choosing RNA-seq analysis pipelines to achieve improved gene expression quantification and downstream predictive modeling of disease outcome. A comprehensive investigation of 278 representative RNA-seq data analysis pipelines consisting of 13 sequence mapping, three quantification, and seven normalization methods was conducted to assess their impact on gene expression accuracy and precision; sensitivity in detecting low-expression genes; specificity in detecting DEGs; and downstream prediction of disease outcome. Our results reveal that RNA-seq pipelines that produced better gene expression usually resulted in better prediction models. A guideline was then provided for the selection of reliable RNA-seq data analysis pipelines. The manuscript  is curreWith this brief review, we aim to summarize some of the key SEQC studies and major findings to spur interest and more thorough examination of these studies and the data sets. To facilitate further research in RNA-seq assessment and development of analysis methods, the comprehensive data sets, including sequencing data and related microarray and qPCR data, have been documented thoroughly and deposited into public data repositories. As a result, the SEQC data sets have already been utilized for studying data normalization methods , detecti


The M2 isoform of pyruvate kinase (PKM2) plays an important role in aerobic glycolysis and is a mediator of the Warburg effect in tumors. It was previously thought that tumor cells switch expression of PKM from normal tissue-expressed PKM1 to tumor-specific PKM2 via an alternative splicing mechanism. This view was challenged by a recent report demonstrating that PKM2 is already the major PKM isoform expressed in many differentiated normal tissues. Here, through analyses on sixteen tumor types using the cancer genome atlas RNA-Seq and exon array datasets, we confirmed that isoform switch from PKM1 to PKM2 occurred in glioblastomas but not in other tumor types examined. Despite lacking of isoform switches, PKM2 expression was found to be increased in all cancer types examined, and correlated strongly to poor prognosis in head and neck cancers. We further demonstrated that elevated PKM2 expression correlated well with the hypomethylation status of intron 1 of the PKM gene in multiple cancer types, suggesting epigenetic regulation by DNA methylation as a major mechanism in controlling PKM transcription in tumors. Our study suggests that isoform switch of PKM1 to PKM2 in cancers is tissue-specific and targeting PKM2 activity in tumors remains a promising approach for clinical intervention of multiple cancer types. Aerobic glycolysis, wherein cells take up more glucose and produce large amounts of lactate in presence of oxygen, is a key feature of many cancer cells. This altered glucose metabolism is known as Warburg effect . Though PKM gene. The two isoforms differ at 23 amino acid residues as PKM1 mRNA contains exon 9 and lacks exon 10, whereas PKM2 mRNA contains exon 10 and lacks exon 9 [Pyruvate kinase (PK) is a rate limiting glycolytic enzyme that converts phosphoenolpyruvate and ADP to pyruvate and ATP. PKM1 and PKM2 are alternatively spliced products of the same primary RNA transcript, encoded by the s exon 9 . The alts exon 9 .In addition to its role as a pyruvate kinase, PKM2 seems to have other nuclear functions that are independent of its function in cancer cell metabolism. For example, PKM2 can function as a protein kinase to phosphorylate histone H3 and promote tumorigenesis . PKM2, bPKM gene promoter.PKM2 is often considered as the predominant isoform expressed in highly proliferative and cancer cells -23. Howein silico analysis of TCGA RNA-Seq data, we performed experiments to amplify PKM cDNA from multiple human tissues using RT-PCR followed by restriction enzyme digestion to estimate PKM1 and PKM2 composition [To examine PKM1 and PKM2 expression levels in human tumors and in normal tissues, we analyzed the level 3 RNA-Seq data from the TCGA datasets, which contain normalized RPKM values (Reads Per Kilobase per Million mapped reads) for each isoform of all genes . The results on sixteen different types of cancers are summarized by boxplots in Figure position . As showSince the TCGA RNA-SeqV2 did not have normal controls for glioblastoma and ovarian cancer, we next examined TCGA exon array data where normal control were available for both tumors. The level 3 data for exon array were in an output format named FIRMA, which can be considered as a normalized score measuring whether an exon is differentially expressed among a set of samples  . As showSince PKM2 expression was elevated in human tumors, we examined whether this could be due to changes in methylation status of the PKM gene. TCGA level 3 DNA methylation data were analyzed and results from five tissue types including bladder, breast, head and neck, kidney, and lung in which both normal and tumor data were available were shown in Figure p value of 7.78e-5. Kaplan Meier analysis . For DNA methylation data, M values were calculated as log2 ratio of methylated intensity over unmethylated intensity [Level 3 RNA-SeqV2 data , level 3 exon array FIRMA data, level 3 Agilent microarray gene expression data, level 2 DNA methylation data (Infinium Human Methylation 450), and clinical data for multiple cancers were downloaded from The Cancer Genome Atlas (TCGA) data portal (ntensity .Based on PKM gene structure  and refg2μg of RNA was used from different tissue samples obtained from Origene (Rockville MD). Reverse transcriptase was carried out using High Capacity cDNA reverse transcription kit from Applied Biosystems. Semi-quantitative PCR was carried out using GoTaq-green master mix  and PKM primers as described earlier . The amphttp://rana.lbl.gov/EisenSoftware.htm). K-means grouping, Cox proportional hazard regression, and Kaplan-Meier log rank test were done using the R software (http://www.r-project.org).Clustering analyses were done using the Cluster and TreeView software (available at


Cluster analysis is widely used in cancer research to discover molecular subgroups that inform subsequent laboratory investigations and define risk classification criteria for subsequent clinical trials. However, for any data set, there are a very large number of candidate cluster analysis methods (CCAMs) due to the many choices for feature selection criteria, number of selected features, number of clusters to define, etc. Frequently, a specific CCAM is chosen without quantifying the validity of its results in terms of reproducibility or distinctiveness of the reported subgroups.Here, we propose the Dunn Index Bootstrap (DIBS) procedure to quantify the reproducibility and distinctiveness of subgroups defined by many CCAMs. DIBS applies each CCAM to the observed data and many bootstrap data sets obtained by subject resampling. The bootstrap results are used to compute metrics of subgroup reproducibility and distinctiveness of the subgroups defined by each CCAM.DIBS was used to characterize the performance of each of 4,032 CCAMs in the analysis of one RNA-seq, two microarray gene expression, and one methylation array data set from three different cancers. In each example, DIBS identified specific CCAMs that defined subgroups of well-established biological and clinical relevance.


The advent of the NGS technologies has permitted profiling of whole-genome transcriptomes  at unprecedented speed and very low cost. RNA-Seq provides a far more precise measurement of transcript levels and their isoforms compared to other methods such as microarrays. A fundamental goal of RNA-Seq is to better identify expression changes between different biological or disease conditions. However, existing methods for detecting differential expression from RNA-Seq count data have not been comprehensively evaluated in large-scale RNA-Seq datasets. Many of them suffer from inflation of type I error and failure in controlling false discovery rate especially in the presence of abnormal high sequence read counts in RNA-Seq experiments.To address these challenges, we propose a powerful and robust tool, termed deGPS, for detecting differential expression in RNA-Seq data. This framework contains new normalization methods based on generalized Poisson distribution modeling sequence count data, followed by permutation-based differential expression tests. We systematically evaluated our new tool in simulated datasets from several large-scale TCGA RNA-Seq projects, unbiased benchmark data from compcodeR package, and real RNA-Seq data from the development transcriptome of Drosophila. deGPS can precisely control type I error and false discovery rate for the detection of differential expression and is robust in the presence of abnormal high sequence read counts in RNA-Seq experiments.https://github.com/LL-LAB-MCW/deGPS). deGPS is a powerful and robust tool for data normalization and detecting different expression in RNA-Seq experiments. Beyond RNA-Seq, deGPS has the potential to significantly enhance future data analysis efforts from many other high-throughput platforms such as ChIP-Seq, MBD-Seq and RIP-Seq.Software implementing our deGPS was released within an R package with parallel computations (The online version of this article (doi:10.1186/s12864-015-1676-0) contains supplementary material, which is available to authorized users. Next-generation sequencing (NGS) technologies parallelize the sequencing processes and produce millions of short-read sequences concurrently. The advent of the NGS technologies has permitted profiling of whole-genome transcriptomes by RNA-Seq, at unprecedented speed and very low cost. RNA-Seq provides a far more precise measurement of transcript levels and their isoforms compared to other methods such as microarrays .In RNA-Seq experiments, millions of short sequence reads are aligned to a reference genome and the number of reads that fall into a particular genomic region is recorded, as read count data. These regions of interest are annotated as microRNA (miRNA), small interfering RNAs (siRNA), long noncoding RNAs (lncRNA), or messenger RNA (mRNA) in the context of RNA-Seq experiment, here all referred to as transcripts. The read count is linearly related to the abundance of target transcripts . A majorThese unique features contained in RNA-Seq data have motivated the development of a number of statistical methods for data normalization and differential expression (DE) detection. Typical approaches use Poisson or negative binomial (NB) distribution to model count-based expression data. The Poisson distribution is commonly applied to models resulting in counting processes. It has a single parameter, which is uniquely determined by its mean. An important property of the Poisson distribution is that the mean equals its variance. However, read counts show a large variability in RNA-Seq experiments, and their variance is often much larger than their mean . This isTo address the overdispersion problem, several statistical methods including DESeq  and edgeAnother alternative to the Poisson distribution is the generalized Poisson (GP) distribution . The GP There are also some other methods developed for finding DE in RNA-Seq studies, e.g., NBPSeq , TSPM 99, baySeqHere, we propose a powerful normalization method based on GP distribution modeling sequence count data, followed by regular permutation-based DE tests of GP-normalized data. Through comprehensive simulations, our method shows improved results for DE expression, in terms of false discovery rate (FDR), and sensitivity and specificity, in RNA-Seq experiments.https://github.com/LL-LAB-MCW). To speed up permutation tests, deGPS also provides efficient parallel computation using multi-core processors. In Step 1, two different methods based on the GP distribution, namely GP-Quantile and GP-Theta, were developed for normalizing sequence count data. These two GP-based methods differ in parameter estimation and data transformation. Generally, GP distributions fit sequence count data better than NB distributions on transcripts over a wide range of relative abundance in RNA-Seq experiments , including 491 miR-Seq libraries   under alternative hypotheses, we first randomly generated two groups of samples and randomly chose a subset of transcripts. Subsequently, we made two types of changes in the selected transcripts to create DE between two groups. In the “shift” transformation, we added varied quantities  of read counts into the selected transcripts in either group. In the “scaling” and “shift” transformation, we multiplied the read counts of selected transcripts by varied quantities  after applying the “shift” transformation  that are not based on the GP distribution, but are commonly used for high throughput data such as those from microarray ,22. Our In addition to the above data-driven simulation strategy, we also used compcodeR for benchmarking of DE analysis methods . The comTo evaluate different FDR adjustment methods, we introduced R package fdrtool  to furthWe first evaluated type I error and FDR of different methods in datasets simulated from two large-scale RNA-Seq studies, including 491 miRNA and 100 mRNA TCGA samples Fig. . FDR is Six of these methods  use different strategies of data normalization, but use the same DE tests as deGPS. They yield very different type I error and FDR. Only GP-Theta and TMM are able to control both type I error and FDR at the desired level; whereas the other four methods have inflated type I error and/or FDR. These results suggest data normalization has substantial impacts on the performance of DE tests in terms of type I error and FDR.Next, we evaluated TPR  of different methods in these RNA-Seq datasets Fig. . These mWe also observed that data normalization dramatically influences statistical power of DE tests. Although the same DE tests were applied after data normalization, six different normalization methods result in varied TPR. Besides GP-Theta, TMM performs reasonably better than the other four methods.We compared deGPS with other methods in terms of sensitivity and specificity in these two RNA-Seq studies. We thus calculated the receiver operating characteristic (ROC) curve and area under curve (AUC) of different methods to measure their sensitivity and specificity Fig. . For theWe further compared deGPS with SAMseq, DESeq and edgeR using compcodeR. compcodeR is an R package for benchmarking of DE analysis methods, in particular methods developed for analyzing RNA-Seq data . In the In compcodeR-based simulations, both deGPS and SAMSeq consistently control both type I error and FDR and are robust against the occurrence of random outliers in RNA-Seq experiments; whereas DESeq2 and edgeR1 are not able to control type I error and/or FDR in most of scenarios. DESeq is still conservative in terms of type I error, but its ability of FDR control varies among different levels of random outliers and samples. edgeR2 generally performs much better than edgeR1 in terms of FDR control in compcodeR-based simulations. edgeR1 is based on generalized linear model in which regular likelihood ratio test (LRT) is performed; whereas edgeR2 replaces the Chi-square approximation to the LRT statistic with a quasi-likelihood F-test .In terms of TPR and AUC, SAMseq performs slightly better than deGPS, but the difference between these two methods becomes small when increasing sample sizes from 5 to 8 subjects per group  from the same development stages into two groups of 5 subjects each. Each group contained equal number of subjects from the same development stages and thus had similar gene expression profiles. Therefore, we expected that no genes are truly DE when comparing these two synthetic groups. Nevertheless, among 100 simulations, these methods identified DE genes ranging from 16 to 277 false positives per genome scan. deGPS found the lowest number of false positives, whereas edgeR1 found the highest number of false positives. edgeR1 inflates type I error, whereas the other four methods can control type I error at the desired level  are presented in RNA-Seq data, for example, as seen in the above Drosophila RNA-Seq data − 1 and the variance of X is θ(1 − λ)− 3. When λ = 0, GP becomes a Poisson. The parameter θ is the mean for the natural Poisson process. The parameter λ is the average rate of effort that the subjects are making to deviate from the process. A positive value of λ indicates that the subjects are making an effort to accelerate the natural process while the negative one denotes an effort to retard the process [Sequence count data,  process . In the  process . Note th process .The maximum likelihood estimate (MLE) of λ in the GP model  can be owhere We propose two new normalization methods for sequence count data based on the above GP distribution: GP-Quantile and GP-Theta. The GP-Quantile method fits every sample in the data with GP distribution, and maps every read count to the corresponding probability, P(X < x), of the fitted GP. Despite that read counts in every sample are normalized between 0 and 1, the data information may be lost during the GP-Quantile normalization process.In the GP-Theta method, read counts from each sample are divided by the parameter θ of the fitted GP distribution. The MLE of θ is od (TMM) .T-test statistic is conducted in our DE test. To eliminate potential technical noise arising from RNA-Seq experiments, T-test statistics are calculated after normalizations:After the data normalization, a procedure using empirical distributional of x’ and Ny’ are sample sizes of the two groups.where “Var” is the variance function and “Mean” is the mean value of read count of a transcript in the sample. X’ and Y’ are GP-transformed read counts from two groups of samples; NT-test statistics for each transcript in the permutated samples. Due to the abundance of the transcripts, our permutation strategy can produce reliable empirical distributions even with small sample sizes  that are still common in RNA-Seq experiments. The p-values are therefore calculated according to the empirical distribution of T statistics. However, the pooled t-statistics is mixture of a “null group” of statistics corresponding to non DE genes and an “alternative” group corresponding to DE genes. Thus we also include fdrtool [We propose to use empirical distribution of T-statistics to determine the p-values of DE tests. We generate empirical distributions by randomly shuffling the samples into two groups and calculate  fdrtool  to adjusThe estimated variances and thus T-statistics used in permutation tests are probably highly variable due to a typically small sample sizes in RNA-Seq experiments. Instead of the above ordinary T-statistics, regularized T-statistics, implemented in R package st, are also included in our deGPS.https://github.com/LL-LAB-MCW). compcodeR-based simulations can be repeated by the R codes which are available in Additional file The real data analysis of the developmental transcriptome of Drosophila can be found in our released R package–deGPS (The data sets supporting the results of this article are included within the article and its additional files.


Gene expression profiling of tumors is a successful tool for the discovery of new cancer biomarkers and potential targets for the development of new therapeutic strategies. Reliable profiling is preferably performed on fresh frozen (FF) tissues in which the quality of nucleic acids is better preserved than in formalin-fixed paraffin-embedded (FFPE) material. However, since snap-freezing of biopsy materials is often not part of daily routine in pathology laboratories, one may have to rely on archival FFPE material. Procedures to retrieve the RNAs from FFPE materials have been developed and therefore, datasets obtained from FFPE and FF materials need to be made compatible to ensure reliable comparisons are possible.To develop an efficient method to compare gene expression profiles obtained from FFPE and FF samples using the same platform.Twenty-six FFPE-FF sample pairs of the same tumors representing various cancer types, and two FFPE-FF sample pairs of breast cancer cell lines, were included. Total RNA was extracted and gene expression profiling was carried out using Illumina’s Whole-Genome cDNA-mediated Annealing, Selection, extension and Ligation (WG-DASL) V3 arrays, enabling the simultaneous detection of 24,526 mRNA transcripts. A sample exclusion criterion was created based on the expression of 11 stably expressed reference genes. Pearson correlation at the probe level was calculated for paired FFPE-FF, and three cut-off values were chosen. Spearman correlation coefficients between the matched FFPE and FF samples were calculated for three probe lists with varying levels of significance and compared to the correlation based on all measured probes. Unsupervised hierarchical cluster analysis was performed to verify performance of the included probe lists to compare matched FPPE-FF samples.P<0.01, n = 1,432; P<0.05, n = 2,530; and P<0.10, n = 3,351 probes). Unsupervised hierarchical clustering of the 27 pairs using the resulting probes yielded 25, 21, and 19 correctly clustered pairs, respectively, compared to 1 pair when all probes were used.Twenty-seven FFPE-FF pairs passed the sample exclusion criterion. From the profiles of 27 FFPE and FF matched samples, the best correlating probes were identified for various levels of significance (Pearson The proposed method enables comparison of gene expression profiles of FFPE and/or FF origin measured on the same platform. Whole-genome microarray gene expression profiling has become an important tool for the discovery of prognostic and predictive genes for various human cancers ,2. UntilIn breast cancer, the use of gene expression profiles to improve outcomes in defined populations was explicitly not recommended by the Evaluation of Genomic Applications in the Practice and Prevention (EGAPP) Working Group, because the evidence for the validity of the test used was considered to be inadequate . In the In the present study we describe a method to analyze RNA microarray data obtained from matched FFPE and FF samples of patients suffering from various types of cancers. Our method is based on data generated with Illumina’s WG-DASL BeadChips, but the principle can be applied to different gene expression platforms. Besides describing a method to define a minimum gene expression cut-off level, we developed a ruler system to reliably compare gene expression profile data obtained from FFPE and FF samples, and demonstrate its use and validity. The ruler is based on various cut-off values indicating the number of probes with a pre-specified Pearson correlation significance to be considered in the analysis.In total, 70 FFPE and FF samples (28 FFPE-FF pairs + 14 duplicate hybridizations) were available to establish a sample-quality threshold  & Fig 1.P-values  were used for the inclusion of probes from the total set of 24,526 probes. The significance levels of P<0.01, P<0.05 and P<0.10 allowed the inclusion of 1,697, 3,016 and 3,982 probes for analysis, respectively.For each individual probe the expression levels measured in 27 FFPE samples were correlated with the case-matched FF levels. Three Pearson R cut-off P<0.01; 2,530 out of 3,016 probes at P<0.05; and 3,351 out of 3,982 probes at P<0.10. These 3 lists consisted of probes with minimal variation in expression levels between FFPE and FF materials.Next, the slope and y-axis intercept were calculated for each probe included at the individual 3 cut-offs. The mean of all slopes and the mean of all y-axis intercepts were calculated. All outlier probes  were deleted. This yielded 1,432 out of 1,697 probes at P<0.01 list; from 0.66 to 0.99 using 2,530 probes of the P<0.05 list; and from 0.63 to 0.99 using 3,351 probes of the P<0.10 list. The Rs between the expression of all probes  for the 27 FF and FFPE samples ranged from 0.55 to 0.93  was calculated between the expression of the probes of 27 sample pairs. The Rs ranged from Rs 0.74 to 0.99 using the 1,432 probes of the  to 0.93 .P<0.01) showed that 25 out of the 27 pairs clustered correctly together (bottom row andP<0.05) (P<0.10)  , and 19/(P<0.10) . In cont(P<0.10) .P<0.01, 2,285 probes at P<0.05, and 3,937 probes at P<0.10. The Spearman rank correlation coefficients and the clustering improved slightly for some cases, but overall the results matched those of the entire set of 27 sample pairs , 0.84 to 0.99 at P<0.05 , and 0.83 to 0.99 at P<0.10  (bottom row).To study the reproducibility of the analysis of the FFPE samples only, 15 FFPE samples of various tumor types and 2 breast cancer cell lines were measured twice  . HierarcP<0.01 calculated by GenomeStudio for the approximately 30 beads tagged with these probes) in our study was also higher in the FF samples compared with the FFPE samples for 24/27 pairs. Only three FFPE samples had a higher number of reliably measured probes compared with the FF samples.Because most of the available data on RNA expression profiling in the literature has been generated from fresh frozen samples, it is important to establish methods for reliably comparing gene expression profiles derived from FFPE and FF samples. In this study we investigated the effects of tissue handling prior to processing for the RNA expression array technology and calculated the correlation coefficients between the gene expression profiles at various levels of stringency. Gene expression profiles obtained from matched FFPE and FF tumor and normal adjacent lung tissue samples showed that the number of reliably measured probes was slightly higher in FF samples as compared to FFPE samples . The numDDX5) was used to judge the quality of the expression profiles and a threshold based on the expression levels of this gene for discarding samples of poor quality was recommended  . In addiFive μm H&E sections from each sample were prepared before and after sectioning for RNA isolation. To ascertain similar morphology of the sections used for the FFPE and matched FF RNA isolation, the origins of the primary tumor, the percentages of the invasive tumor cells, inflammatory infiltrates and the presence of necrosis were taken into consideration (JMK). Only RNA isolated from sections with similar morphology was included for the matched FFPE-FF comparisons.Total RNA from FF tissue samples was, depending on the size of the sample, extracted from 20–30 sections of 30 μm using RNABee reagent according the supplier's instructions . Depending on the size of the FFPE samples, RNA was extracted from 3 to 6 x 20 μm sections. Following paraffin removal with xylene the high-pure RNA paraffin kit was used according the supplier's instructions . Following isolation, RNA was stored in RNase/DNase-free water at -80°C.The quantity and quality of the isolated RNA was monitored by UV spectroscopy and by examination of rRNA bands after agarose gel electrophoresis. Samples were excluded if the yield did not reach the minimum requirement of 500 ng available for the FF samples and 1,000 ng for the FFPE samples. As an inner-assay control, 500 ng of FF RNA isolated from a pool of several cultured breast cancer cell lines was used. The inner-assay control was used to monitor the assay performance and to evaluate the inter-assay BeadChip variability for each experiment.18 and random nonamer primers. The biotinylated cDNA was annealed to the DASL Assay Pool (DAP) probe groups, which contain oligonucleotides specifically designed to interrogate each target sequence of the transcript. The DAP was annealed to targeted cDNA during a 16 hours temperature gradient (70° to 37°C) incubation. Hybridization of these oligonucleotides to the targeted cDNA site, followed by enzymatic extension and ligation was used to create a PCR template that was amplified with a set of universal PCR primers ; P<0.05 [R2>0.38] and P<0.10 [R2>0.32], respectively). To calculate the accuracy of the associations between the individual probe values of FFPE and FF samples, the slope and the y-intercept were calculated for each probe at the different cut-offs. The mean of all slopes and the mean of all y-intercepts were calculated and all outlier probes  were disregarded.Expression levels, normalized by the ‘Lumi’ package, were used to calculate the correlation coefficient (Pearson R2 correlation) at the individual probe level for all paired FFPE and FF samples  that were left after applying the sample exclusion criterion. Various levels of significance were used (Unsupervised clustering was performed using Cluster 3.0 using the correlation (uncentered) average linkage method and Java Treeview 1.1.6 R2 was used for visualization .S1 Fig(A) Implementing all 70 samples resulted in a distribution with a skewness of -0.663. (B) Following the deletion of 7 samples with a median < 2,208 ) fluorescent expression level for the 11 reference genes the skew was reduced to an acceptable level (skewness of -0.150). The red line represents a normal distribution of the values.(TIF)Click here for additional data file.S2 Fig34 FFPE measurements (17 duplicate FFPE samples) were used to apply the sample exclusion criteria that was established earlier in the study based on a threshold of 2,208. Three FFPE samples with a median reference gene signal below this threshold were of poor quality and were excluded. Because the analysis is based on FFPE duplicate measurements, the duplicate measures of 3 FFPE samples of poor quality were excluded from the analysis. In total, 28 FFPE measurements (14 duplicate FFPE samples) were used to identify the reliably measured probes.(TIF)Click here for additional data file.S1 TableFFPE = formalin-fixed, paraffin-embedded; ACUP = adenocarcinoma of unknown primary.(DOCX)Click here for additional data file.S2 TableP-values . The upper row shows the numbers of probes included for the distinct levels of significance. The number of matched pairs following unsupervised hierarchical clustering is shown in the bottom row.Rows 1 through 12 represent breast cancer paired samples of which the FFPE and FF parts were separately run on the Illumina WG-DASL V3 platform. The right four columns show the Spearman rank correlation coefficient between the expression values of the FF and FFPE materials for various (DOCX)Click here for additional data file.S3 TableRows 1 through 12 represent breast cancer FFPE and FF sample pairs which were separately run on the Illumina WG-DASL V3 platform. The normalization was performed separately for the FF and FFPE samples. The right four columns show the correlation coefficient (Spearman correlation) between the expression values of FF and FFPE materials for various p-values . The upper row shows the numbers of probes included for the distinct levels of significance. The number of matched pairs following unsupervised hierarchical clustering is shown in the bottom row.(DOCX)Click here for additional data file.S4 TableRows 1 through 14 represent the FFPE samples that were profiled twice on the Illumina WG-DASL V3 platform. The right four columns show the correlation coefficient (Spearman correlation) between the expression data of the duplicates for various p-values . The upper row shows the numbers of probes included for the distinct levels of significance. The number of matched pairs following unsupervised hierarchical clustering is shown in the bottom row.(DOCX)Click here for additional data file.


It has recently been shown that significant and accurate single nucleotide variants (SNVs) can be reliably called from RNA-Seq data. These may provide another source of features for multivariate predictive modeling of disease phenotype for the prioritization of candidate biomarkers. The continuous nature of SNV allele fraction features allows the concurrent investigation of several genomic phenomena, including allele specific expression, clonal expansion and/or deletion, and copy number variation.The proposed software pipeline and package, SNV Discriminant Analysis (SNV-DA), was applied on two RNA-Seq datasets with varying sample sizes sequenced at different depths: a dataset containing primary tumors from twenty patients with different disease outcomes in lung adenocarcinoma and a larger dataset of primary tumors representing two major breast cancer subtypes, estrogen receptor positive and triple negative. Predictive models were generated using the machine learning algorithm, sparse projections to latent structures discriminant analysis. Training sets composed of RNA-Seq SNV features limited to genomic regions of origin (e.g. exonic or intronic) and/or RNA-editing sites were shown to produce models with accurate predictive performances, were discriminant towards true label groupings, and were able to produce SNV rankings significantly different from than univariate tests. Furthermore, the utility of the proposed methodology is supported by its comparable performance to traditional models as well as the enrichment of selected SNVs located in genes previously associated with cancer and genes showing allele-specific expression. As proof of concept, we highlight the discovery of a previously unannotated intergenic locus that is associated with epigenetic regulatory marks in cancer and whose significant allele-specific expression is correlated with ER+ status; hereafter named ER+ associated hotspot (ERPAHS).The use of models from RNA-Seq SNVs to identify and prioritize candidate molecular targets for biomarker discovery is supported by the ability of the proposed method to produce significantly accurate predictive models that are discriminant towards true label groupings. Importantly, the proposed methodology allows investigation of mutations outside of exonic regions and identification of interesting expressed loci not included in traditional gene annotations. An implementation of the proposed methodology is provided that allows the user to specify SNV filtering criteria and cross-validation design during model creation and evaluation.The online version of this article (doi:10.1186/s12864-016-2542-4) contains supplementary material, which is available to authorized users. Defining the molecular basis for complex disease at high resolution has become increasingly important for the discovery of actionable drug targets and the improvement of diagnosis and prognosis of cancer patients. To this end, a widespread approach has been through differential gene expression (DGE) analyses that utilize massively parallel high-throughput RNA sequencing (RNA-Seq). RNA-Seq provides a wealth of information beyond gene expression that can be used to characterize the transcriptome, such as alternative splicing via changes in isoform proportions. Notably, it has recently been shown that single nucleotide variants (SNVs) in the genome can be accurately and reliably called from RNA-Seq data as well . This isIt is imperative that SNVs be studied as they molecularly underpin complex disease and phenotype . For exaa priori knowledge of their phenotypic associations is not necessary for their creation. In fact, because variants with unknown clinical associations are included, novel variants and/or genomic regions can be implicated as candidate biomarkers.Furthermore, models created from co-occurring SNV features have also been successful in predicting disease phenotype, such as susceptibility to breast or lung cancer , 52. TheOur proposed methodology seeks to train accurate predictive models on SNV allele fraction (AF) values using sparse projections to latent structures discriminant analysis (sPLS-DA). This approach allows the identification of disease-associated SNVs located in coding regions as well other expressed locations of the genome, such as from intronic, intergenic, and 5’ UTR regions, which are often under-represented in cancer biology literature. The continuous nature of SNV AF features from RNA-Seq data also allows the exploration of several genomic phenomena, mainly allele specific expression (ASE), where one allele is preferentially expressed over the other. However, it is important to note that the proposed methodology does not discriminate these events from other sources of allelic imbalance, such as from differential cell survival leading to clonal expansion or depletion and/or from copy number amplification and deletion. In our view, this inherent naïveté is a strength in that these events can be analyzed concurrently in a whole genome fashion, providing a shotgun approach to biomarker discovery. The identification of disease-associated SNVs can thus inform and limit regions of interest when using more comprehensive approaches, such as differential expression, as well as implicate novel unannotated regions that are often ignored using traditional approaches. Furthermore, SNV calling from RNA-Seq avoids using relatively more expensive technology, such as whole-exome and whole-genome sequencing (WES/WGS), and can be used to investigate variations due to RNA-editing, which have been shown to have prognostic value regarding outcomes in cancer .We demonstrate the effectiveness of the proposed methodology and software pipeline, SNV Discriminant Analysis (SNV-DA), on two datasets. The first of which is relatively small dataset of non-small cell lung cancer (NSCLC) primary tumors from which we sought to classify future recurrence. Lung cancer is the leading cause of cancer-related deaths worldwide, with the subtype NSCLC compromising approximately 87 % of lung cancer cases in the United States and causing an estimated 500,000 deaths per year worldwide , 32. DesThe second dataset included in this analysis is composed of 42 estrogen receptor positive (ER+) and 42 triple negative (TR-) primary breast cancer tumors. Standard targeted therapies for breast cancer rely on the presence of either estrogen, progesterone, and/or Her2/neu receptors in the primary tumor sample . Those tWe show that SNV-DA is able to create multivariate predictive models that accurately predict disease phenotype from variants called from RNA-Seq data and are significantly discriminant towards true sample groupings. We also show that the proposed software pipeline is able to identify and prioritize disease-associative SNVs. Importantly, the utility of SNV-DA is supported by the result that rankings produced by the methodology are significantly different than rankings produced from univariate tests and are enriched within genes with significant ASE. Lastly, we present as proof of concept, the discovery of a previously unknown highly mutated ER+ associated hotspot (ERPAHS), which is associated with epigenetic markers in cancer cell lines and whose expression is significantly upregulated in ER+ primary tumors as well as significantly correlated with identified SNV features.First, SNVs are called from processed RNA-seq files using Genome Analysis Toolkit (GATK) . Calls aThe variant calling and filtering pipeline, SNPiR, has been shown to obtain accurate SNVs with minimal false-positives from RNA-Seq data . For eacThe resulting output is a BED file containing SNVs with their genomic coordinates and allele fractions. RADAR is first used to determine if SNVs are located at RNA-editing sites . The SNVi,j is the allele fraction of the i-th SNV in sample j. Allele fraction, or read-frequency, is defined as the amount of reads supporting the variant allele over the total amount of reads covering that nucleotide position. Read coverages are determined for every SNVM i,j. Those SNVM i,j values that do not reach the threshold read coverage (default 10) are given a non-available (NA) value. Sub-models can then be generated by limiting SNVs to those located in a region of interest, such as exonic positions, and/or by requiring a minimum number of non-zero features.The total set of variants is transformed into a matrix SNVM, where SNVM K features that provides the best discrimination between two classes, ignoring all other features. sPLS-DA thus provides a framework for both feature selection and classification.Predictive models are created using sPLS-DA, which is implemented in the mixOmics R package , 15. PLSK, utilized by sPLS-DA that result in the best predictive performance. For every iteration of 10-fold cross-validation, sub-cross-validations are performed across a range of values for K. For each K, the model is trained on 10-fold sub-training sets and evaluated. The value of K with the best performance for each iteration of the parent cross-validation is then stored. This process is repeated 15 times to more accurately estimate the distribution of optimal Ks from 150 values. The optimal K is then determined as the rounded value of K that corresponds to the maximum of the estimated kernel density of the distribution of selected K’s, as represented in Fig. Nested cross-validations are used to determine the amount of features, p-values reported by DESeq2 will simply be referred to as p-values. Models were trained on subsequent gene expression matrices using the same parameters as those used in the creation of SNV models. For each dataset, the distribution of performance statistics are compared to that of the corresponding SNV model to identify the similarity of performance between the proposed methodology and the traditional approach.To compare the performance of the proposed methodology with traditional gene expression classifiers, models were created using gene expression values as input. For the NSCLC dataset, Bowtie (v1.2.18)  and RSEMK, the model is then evaluated using fifteen 10-fold cross-validations to determine performance via its predictive accuracy, classification sensitivities, and area under the receiver operating characteristic curve (AUC), which seeks to quantify the relationship between true and false positive rates. Though sPLS-DA is able to train a model on features that include NA values, missing data in the test set is not compatible with the resulting model. Therefore, NA values are replaced with the mean of the means of the centered and standardized AF values for each feature within each group in the training set. For example, the mean of the normalized AF values for feature X in group A is averaged together with the mean of normalized AF values for feature X in group B disregarding samples from the test set. This value is then used as a proxy for the missing data in the test set.After the empirical estimation of the optimal value of p-values.To determine if the proposed methodology is discriminant towards the true grouping of disease phenotype, permutation tests are repeated 1000 times to construct the null distribution of model performance  for each model. The true model performance is then compared to this null distribution to determine significance, with a significantly discriminant model outperforming the majority of permutation test models.Otherwise, it could be said that model performance is independent of the true grouping and is, thus, insignificant. For each test, one iteration of a 10-fold cross-validation is used to train and test models with randomly permuted sample group labels using the optimal K that was used in the true model. The number of models with AUC greater than or equal to the true model AUC is divided by the number of tests to determine permutation test K. The selected features are then ranked by the absolute values of their predictive coefficients (or loadings) as determined by sPLS-DA. In order to assay the utility of the proposed methodology, a Friedman rank sum test is used to compare the rankings of selected features to those of traditional approaches — the univariate non-parametric tests, Fisher’s exact and Wilcoxon rank sum.Lastly, to obtain the final set of putative SNV features, the model is trained using all samples and the optimal value of The Fisher’s exact test is implemented by the production of a 2 ×4 table for each SNV locus, where each value corresponds to the number of samples in each group with detectable levels of each allele in , while disregarding samples with sub-threshold read coverage (<10) at that locus. As the presence of an allele is binary in this case, the test only takes into account the differential abundance of the alleles across groups. Whereas, Wilcoxon rank sum test p-values are produced by comparing the distributions of continuous allele fractions and do not directly include information on their differential abundance across samples.p-values can be computed.To determine if the proposed methodology selects SNVs that lie in genes that have significant allele-specific expression, selected SNVs were analyzed using MBASED: a method that combines evidence across multiple SNVs to identify gene-level ASE . Though Finally, the top 15 features selected by SNV-DA are characterized by their relevance to cancer phenotype and are analyzed via hierarchical clustering to visualize the co-occurrence of features.NSCLC is the leading cause of cancer-related mortality in the US. Adenocarcinoma, the most frequent histological subtype, accounts for 40 % of such deaths . RNA samRNA integrity was verified on an Agilent 2200 Bioanalyzer . One hundred to two hundred ng of total RNA was used to prepare RNA-Seq libraries using the TruSeq RNA Sample Prep Kit following the protocol as described by the manufacturer . Three samples per lane were clustered on a cBot as described by the manufacturer . Clustered RNA-Seq libraries were paired-end sequenced with 2 ×100 cycles on a HiScanSQ. Demultiplexing was performed utilizing CASAVA to generate the Fastq files. Each sample produced approximately 25 million reads after sequencing. One sample from the relapse group was removed from subsequent analysis after being identified in our previous study as an outlier based on principle component analyses of expression and alternative splicing . Using nTo further validate our model, we obtained a dataset from the publicly available SRA database (SRP042620), which was provided by Varley et al., 2014 . In theiAfter variant calling and SNPiR post-processing, 96,025 and 213,020 unique variants with read coverages >=10 were found in the NSCLC and breast cancer datasets, respectively. SNV matrices were created by limiting SNVs to those that had at least 3 non-zero values across samples in the NSCLC dataset and 6 non-zero values in the breast cancer dataset. Tables K. For the NSCLC dataset, the classification target was patient relapse within a 3 year period, labeled as Relapse, R, or Disease Free, DF. For the breast cancer dataset, the model sought to classify each sample as being from either the cancer subtype estrogen receptor positive, ER+, or hormone receptor triple negative, TR-.sPLS-DA was used to create models using different subsets of SNVs based on type (as shown in Tables p=0.016), thereby, suggesting that selected SNVs reflect a true biological phenomenon. With the addition of synonymous variants in the model, however, performance reflects that of a failed model. Interestingly, the distribution of AUCs from the nonsynonymous exonic model was significantly better than the distribution of AUCs from the gene expression model . Though some of the other models have AUC values that seem to be better than chance, their performances are not significant based on permutation test values.Table p<0.001), while selecting roughly half as many features. The distributions of AUC values from the intergenic and all-SNVs models were significantly higher than from all other models  - being able to accurately predict TR- samples 96 and 97 % of the time. Interestingly, models with relatively small amounts of starting features  were also able to produce accurate results. Most importantly, the biological significance of these models is supported by the result that all had permutation test p-values <0.001. Of note, the model trained on gene expression features had significantly better performance than all models trained on SNV features , however, the all-SNVs model surpassed the gene expression model when classifying TR- samples , which in this dataset can be considered the experimental group. Furthermore, only one of the top 15 features selected by the gene expression model corresponds to a gene where from a top predictive SNV feature is found: ZNF552.Table ps < 10−16). Figure The rankings of selected nonsynonymous features in the NSCLC dataset were significantly different than univariate rankings from Fisher’s exact and Wilcoxon rank sum tests as determined by Friedman rank sum tests .Additional file Table ps <10−6). When comparing rankings to those produced by Wilcoxon rank sum test, all were significantly different (ps <10−7). The similarity to univariate rankings in the two models is likely a result of a small initial feature set size and/or the types of patterns seen in the data. For example, though the 5’ UTR model produced rankings that were significantly different than rankings from Wilcoxon rank sum test, they were not significantly different than those from Fisher’s exact test (p =0.515), suggesting that predictive power of selected SNVs in this model result more from the differential abundance of AF values (number of nonzeros) than with the differential magnitude of AF values between groups, which Wilcoxon rank sum test seeks to quantify. The distribution of SNVs by region of origin selected during the training of the all-SNVs model is given in Fig. Except for up/downstream and 5’UTR models, the rankings of selected SNV features for each model were significantly different than univariate rankings from Fisher’s exact test .Figure p<10−5, DESeq2); and USP35, amplification of which is associated with significantly worse prognosis in breast cancer patients and is associated with ER- tumors [p<10−5), perhaps providing evidence that conflict with the observation that USP35 amplification is associated with ER- tumors. It is important to note that the 5 SNV features that do not have previous associations with cancer lie in genes that are uncharacterized, 3 of which from the same gene, LRRC56. These SNVs thus implicate genes that may provide future insight into the biology of the different cancer subtypes.Ten of the top 15 SNV features lie in genes that have previous associations with cancer; 6, of which, are located in genes specifically associated with breast cancer (Table - tumors . The latp=0.011), suggesting a possible association of the SNV in the downregulation of the gene. Though not significant, the expression of HFM1 is decreased in samples in which the SNV is present . Furthermore, selected intronic SNVs lie in genes that are enriched with ASE events, 24.04 % of gene/sample pairs .Figure p<10−4). Also of note, several variants lie in genes that were previously implicated in the exonic model: two SNVs in ARHGEF16, one in EML4, and one in LMNTD2.Eleven of the top 15 SNVs are located in genes that have previous associations with cancer Table ; 9 of whp<0.001, 1.71X greater than random mean).Figure Similar to the previously described models, the top 15 SNV features are located in genes enriched with cancer associations, 10 of 15 Table ; 6 of whn=8). Interestingly, RNA-editing sites and SNPs in this gene have been identified in other cancers [Figure  cancers . Another cancers . Intriguestrogen receptor positive associated hotspot (ERPAHS). In fact, 9 of the top 15 selected intergenic SNVs are located in this concentrated region . Interestingly, 36 of these are further concentrated in a 1,635 bp region  defined by a strong H3K27Ac peak and histone methylation peaks  present in chronic myelogenous leukemia cell lines (K562), a moderate peak in human embryonic stem cells (H1-ESC), as well as a DNaseI hypersensitivity site and evidence of CTCF binding  was included in the hg19 RefSeq annotation used by featureCounts during read assignment. Importantly, this region does not overlap the annotated transcripts mentioned above. Strikingly, this region is highly expressed in both ER+ and TR- breast samples and is upregulated in ER+ tumors . Characterization of top performing SNV features from optimally performing models demonstrated that there is an enrichment of SNVs originating from genes previously associated with cancer risk, progression, and survival. That the majority of selected features lie in genes that have strong relationships with the analyzed phenotype supports the use of SNV-DA for the identification and prioritization of novel molecular targets associated with disease phenotype. Furthermore, in the breast cancer dataset, SNV-DA was able to identify predictive SNVs located in ncRNA as well as intronic, 5’ UTR, 3’ UTR, up/downstream, and intergenic regions - locations in the genome that are routinely ignored by whole exome sequencing. One outstanding example was the prioritization of the previously studied SNP, rs11515, located in the 3’UTR of the tumor suppressor CDKN2A, which has clear associations with poor prognosis and risk in different cancers . SNV-DA Because RNA-Seq variant calling is limited to regions of the genome that are expressed, the proposed methodology would not be able to identify variants that result in a marked decrease of expression below a threshold level. However, SNVs called from RNA-Seq data have the added benefit over traditional whole-genome or exome sequencing in that they provide information on the relative amounts of allelic expression, which - as shown - can be implicated in disease or phenotype. Moreover, this methodology can also be used to analyze WGS and WES data to determine the differential abundance of alleles across heterogenous tissues. Furthermore, the developers of GATK have recently outlined best practices for indel calling from RNA-Seq data; therefore, SNV-DA can also be used for the analysis of those variants where special attention is given to feature definitions . Though SNV-DA identified predictive SNVs lying in miRNA and lncRNA, the SNPiR pipeline is not designed for non-standard RNA-Seq methodologies, such as small RNA-Seq. Consequently, the development of novel variant calling methods for these non-standard approaches may produce new avenues of research that are amenable to classification using SNV-DA. A tremendous amount of RNA-Seq data has already been collected in the private and public domain. This data now has the added benefit in that it can be mined for more clinical insights via SNV-DA.https://github.com/Anderson-Lab/SNV-DA. Data transformation is implemented in Python and parallelized model creation, evaluation, and permutation tests are implemented in R. The package allows users to specify cross-validation design, filtering criteria, and values of K to be tested, as well as the automatic creation of supporting figures. The repository also contains variant data used in this study, as well as the necessary code and documentation to run SNV-DA and SNV calling from RNA-Seq data.SNV-DA is freely available at


Salmonella enterica subsp. enterica ser. Montevideo strains. Reference-based approaches for identifying SNPs produced trees that were significantly more similar to one another than those produced under the reference-free approach. Topologies inferred using a core matrix  were significantly more discordant than those inferred using a non-core matrix that allows for some missing data. However, allowing for too much missing data likely results in a high false discovery rate of SNPs. When analyzing the same SNP matrix, we observed that the more thorough inference methods implemented in GARLI and RAxML produced more similar topologies than FastTreeMP. Our results also confirm that reproducibility varies among NGS platforms where the MiSeq had the lowest number of pairwise differences among replicate runs. Our investigation into the robustness of clustering patterns illustrates the importance of carefully considering how data from different platforms are combined and analyzed. We found clear differences in the topologies inferred, and certain methods performed significantly better than others for discriminating between the highly clonal organisms investigated here. The methods supported by our results represent a preliminary set of guidelines and a step towards developing validated standards for clustering based on whole genome sequence data.Comparative genomics based on whole genome sequencing (WGS) is increasingly being applied to investigate questions within evolutionary and molecular biology, as well as questions concerning public health . Given the impact that conclusions derived from such analyses may have, we have evaluated the robustness of clustering individuals based on WGS data to three key factors: (1) next-generation sequencing (NGS) platform , (2) algorithms used to construct a SNP (single nucleotide polymorphism) matrix (reference-based and reference-free), and (3) phylogenetic inference method . We carried out these analyses on 194 whole genome sequences representing 107 unique  The increasing availability and decreasing costs of next-generation sequencing (NGS) has made it possible to apply this technique to investigate non-model organisms and myriad questions within evolutionary and molecular biology. Next-generation sequencing as a means to acquire whole genome sequences (WGS) has also revolutionized applied research, where it is now, for example, being employed to investigate issues related to public health such as pathogenic outbreaks . In thisStx)-producing Escherichia coli O104:H4 associated with contaminated sprouts in Germany; NGS data has been used to better understand differences in virulence  within a neonatal intensive care unit. Such an approach was able to distinguish among isolates that belonged to a single lineage, which was likely impossible using traditional typing methods. As a result, NGS data was shown to rapidly  provide clinically important data  the NGS platform on which the sequences were generated, (2) the method used to detect variant sites and create a SNP matrix, and (3) the phylogenetic inference method used to cluster samples. With regards to NGS platform, it is well documented that performance  differs among them . AlthougSalmonella enterica subsp. enterica Serovar Montevideo isolates sequenced on five different NGS platforms. Many of the isolates were implicated in an outbreak of salmonellosis and the evolutionary relationships of a subset of isolates were investigated within two previous studies  by comparing the topologies produced with the same phylogenetic inference package. We evaluated the influence of sequencing platform by determining the degree to which replicate runs of the same strain clustered together and the number of differences among replicate runs of the same strain on the same platform. We also assessed the influence of three different phylogenetic inference methods by comparing topologies created using the same data matrix. These analyses were performed on a set of  studies .Salmonella enterica ssp. enterica serovar Pomona that served as an outgroup and the others are Montevideo  and the number of SNP differences that should exist among them . For analyses comparing topologies (see below), we were not able to determine the accuracy of the methods but rather focused on the variance among methods where we were working under the assumption that methods that produced more similar trees are superior to methods that result in different topologies.We investigated the performance of four different SNP detection methods, which can be broadly separated into reference-based and reference-free. The primary difference between the two types is that reference-based methods use a genome, either a closed or draft assembly, to which reads are mapped and variant positions are called. In contrast, the reference-free approach determines variant sites in the absence of a reference genome by comparing portions  among all samples.CP007530.1). We chose this as the reference genome as it represented the closest Montevideo environmental strain to the outbreak  and assembled using SMRT Analysis v2.0.1 , which masked bases with Q-scores <20. Under the second treatment, we performed de novo assemblies that were then used as the input for kSNP. We chose an assembler that was well suited for the sequencing platform from which the reads came, which was CLC Genomics Workbench 6.0.5  for the Illumina, SOLiD, and IonTorrent data and Newbler v2.6 for the 454 data. As was the case with the mapping software under the reference-based approach, our objective here was not to evaluate multiple de novo assemblers as that has been done elsewhere   on the tree) to 1 . Generally speaking, the gsi is based on the number of nodes uniting all individuals from the same group divide by the observed number of nodes uniting those members. We calculated the gsi statistic for two different cases: (1) all replicates of a given strain comprise a single group and (2) replicates run on the same platform comprise a group. We chose these two groupings because they allowed us to characterize, as best we could given the dataset, the influence of within and between platform variation. To account for phylogenetic uncertainty, we estimated the weighted gsi value under which 100 bootstrap replicates for each matrix  by phylogenetic inference comparison were analyzed.We used the genealogical sorting index (gsi)  to quantTopologies based on each of the matrices created under the different SNP detection methods were created using three phylogenetic inference methods .We quantified the congruence among topologies using the symmetric difference statistic  i.e., t implemenS. Montevideo   ; Fig. S2Although the performance of NGS platforms and the requisite analysis software has been evaluated, those evaluations were often done focusing on a single step in the process . Here, we have considered the entire process from the sequence platform to phylogenetic clustering. Examination of the full process is important in elucidating best practices to increase the consistency and accuracy of such analyses. We found that within each of the three factors we evaluated there were differences in the results produced and, thus, were able to identify some guidelines for how NGS data should be analyzed for the purpose of clustering.Ideally, different next-generation sequencers would produce identical nucleotide sequences when provided with the same sample. Unsurprisingly however, differences in the chemistry and engineering of different platforms mean they do not agree exactly, but have different error rates and error structure. For example, studies of “benchtop” NGS platforms have repeatedly found that the MiSeq has the lowest single-base error rate followed by Roche’s 454 GS Junior followed by the Ion Torrent PGM ; unlike Given the differences between NGS platforms, we must ask if the differences are significant enough to obscure the relationships among individuals. From our analysis of sequencing runs of the same strain carried out on different platforms and labs, it is clear that replicate runs were not identical  and, conThe analysis of WGS is non-trivial, and there are a number of possible approaches that exist. Here we focused on one possible way that such data could be used: to identify variant sites to construct a SNP matrix from which a phylogeny could be inferred. Some of the approaches , which the results presented here support. However, the primary drawback of a reference-based method is that it will fail to detect SNPs within a region shared among non-reference samples but that is absent in the reference genome either due to using a reference whose genome has not been closed  or the genomic region actually being absent in the reference. In contrast, the reference-free approach will detect SNPs within regions missing from a subset of the runs (depending on the percent presence threshold). This additional detection ability comes at the cost of having to decide what threshold of missing data one should use; we used the 50% majority but other values could include the percent of the dataset that represents the ingroup samples. Without a threshold on the percent of missing data, the reference-free approach may have a much higher false-discovery rate most likely due to sequencing error , which The three phylogenetic inference methods we evaluated did not produce the same topology when analyzing the same SNP matrix . HoweverAn important aspect of our phylogenetic analyses is that the substitution models we employed may not be optimal for the types of matrices we were analyzing within which every site is variable. To our knowledge there have been few studies investigating the effects of applying traditional nucleotide substitution models to such matrices. One such study found that under the GTR + Γ model, phylogenetic accuracy decreased when invariant sites were excluded . AlthougWe have evaluated the robustness of topologies to a number of factors that may differ among analyses on a particularly vexing group of samples that are highly clonal and very closely related. Within the original analyses describing the relationships among 47 of the strains, there were only five unique SNP differences between outbreak and non-outbreak isolates . Within The results presented here indicate it is possible to recover the outbreak structure in a dataset of this type, but care should be taken when deciding on the methods to employ. Methodological choices include: (1) deciding to combine data from different NGS platforms, (2) choosing a method to use for identifying variants, and (3) choosing what phylogenetic inference method should be used. Our results suggest good options for each of these choices and indicate that they allow detection of the biological outbreak signal even among these closely related samples. For example, we found strong bootstrap support for the differentiation of outbreak and non-outbreak samples results when using the non-core matrices constructed using a reference-based method and inferring a phylogeny with either RAxML or GARLI . AlthougThe robustness of phylogenetic inference to data acquisition and analysis methodology is critically important in many studies . Our results illustrate how topologies may differ depending on methodological approaches employed—a particularly important issue when investigating closely related samples. Our results also show that good methods do exist that provide robust results when attempting to differentiate between outbreak and non-outbreak samples. For example, the best choice appears to be the use of a reference-based approach allowing for some missing data in the matrix and constructing trees using a thorough inference program. We believe that these general conclusions are likely to be observed across different datasets  than the one investigated here and future studies will reveal whether that assumption is correct.This work represents our ongoing efforts to build, characterize, and validate our WGS methodology and pipeline for food-borne outbreak investigations. The questions we address are applicable to many in the WGS analysis community, and we hope that the results presented here will contribute to the establishment, through additional investigations, of well-defined pipelines for clustering based on WGS data.10.7717/peerj.620/supp-1Figure S1Click here for additional data file.10.7717/peerj.620/supp-2Figure S2Click here for additional data file.10.7717/peerj.620/supp-3Figure S3x = y).The red line is the expected relationship if the two matrices produced the same number of pairwise differences (i.e., Click here for additional data file.10.7717/peerj.620/supp-4Table S1Click here for additional data file.10.7717/peerj.620/supp-5Supplemental Information 5See README file for further details.Click here for additional data file.


Discovering functional relationships of genes through cell-based phenotyping has become an important approach in functional genomics. High-throughput imaging offers the ability to quantitatively assess complex phenotypes after perturbation by RNA interference (RNAi). Such image-based high-throughput RNAi screening studies have facilitated the discovery of novel components of gene networks and their interactions. Images generated by automated microscopy are typically analyzed by extracting quantitative features of individual cells, resulting in large multidimensional data sets. Robust and sensitive methods to interpret these data sets and to derive biologically relevant information in a high-throughput and unbiased manner remain to be developed.Here we propose a new analysis method, PhenoDissim, which computes the phenotypic dissimilarity between cell populations via Support Vector Machine classification and cross validation. Applying this method to a kinome RNAi screening data set, we demonstrate that the proposed method shows a good replicate reproducibility, separation of controls and clustering quality, and we are able to identify siRNA phenotypes and discover potential functional links between genes.a priori knowledge. PhenoDissim is freely available as an R package.PhenoDissim is a novel analysis method for image-based high-throughput screen, relying on two parameters which can be automatically optimized without  To understand phenotypes and their regulations, it is important to identify key genetic components as well as how they interact. Cell-based screening approaches have been successfully used to monitor the effect of individual gene knockdowns or small molecule treatments, identify key regulators contributing to the assessed phenotype and investigate their interactions ,2. Such Drosophila melanogaster or Caenorhabditis elegans, a systematic implementation for quantitative cellular image data sets is still missing. One key question is how to define quantitatives to represent the phenotype of a given perturbation based on the multidimensional cell feature data sets, before one can identify and potentially cluster phenotypes by similarity.While the analysis of univariate readouts from intensity-based screens has been greatly facilitated by the development of specific algorithms and analysis methods -20, how Previous studies typically first applied a dimension reduction or data transformation method, such as principle component analysis , KolmogoHere, we propose a novel method to measure phenotypic dissimilarity between cell populations in imaging screens, based on cell classification and cross validation. We define the phenotypic dissimilarity between a perturbation and a control, or between two perturbations, as the classification accuracy between the two corresponding cell populations. First, we virtually pool cells from both populations. Then, using Support Vector Machine (SVM) classification, the mixed cell population is classified into two groups based on quantitative cell features. The classification accuracy can be estimated by cross validation with the original cell labels, and defined as the phenotypic dissimilarity. A higher accuracy indicates better separation between the cell populations, thus a larger phenotypic dissimilarity. Evaluated on a kinome-wide RNAi screen for cell morphology , the prohttp://www.bioconductor.org/packages/release/bioc/html/phenoDist.html), a toolbox for data analysis in image-based high-throughput screening.The PhenoDissim method is relatively straightforward to apply on different high-throughput screening experiments, as it has only two parameters for SVM classification: cost and gamma, and parameter optimization can be automated. The method, as well as the quality metrics for evaluation, is implemented in a freely available R/Bioconductor package phenoDist . The screening data set has one negative control (siRLUC) and two positive controls (siUBC and siCLSPN), with four wells of each control per 384-well plate. Figure Phenotypic dissimilarity of all perturbations in the screen to the siRLUC control are plotted in Figure Three example perturbation with distinct phenotypes are shown in Figure In total, 31 siRNA perturbations (averaging two replicates of each gene) showing high phenotypic dissimilarity to siRLUC control (>0.85) indicate morphological phenotypes. With the pair-wise phenotypic dissimilarity for the 31 siRNA samples, we generated a network of phenotypes with nodes representing each phenotype and edges for phenotype dissimilarity between nodes as in Figure We then clustered genes by pair-wise phenotypic dissimilarity of the whole screening set to identify genes that perform potentially related functions. To this end, we averaged the two replicates and generated a 779×779 phenotypic dissimilarity matrix, with each row and each column representing an siRNA treatment. Then hierarchical clustering was performed based on the phenotypic dissimilarity matrix . Grouping genes from the same pathway or family together validates the clustering analysis. Additionally, the MAPK pathway and the NME gene family are clustered together, which may suggest a functional link. This is supported by the previous finding that overexpression of NME represses MAPK phosphorylation, and thus inhibits cell migration and metastasis [Genes from the cluster with the highest number of enriched GO terms . As summarized in Table 2)). We have estimated that with the data set used in this study, every dissimilarity calculation takes about 10 seconds on a 2 GHz Intel Xeon CPU (data not shown). The computation needed for each comparison will be affected by the number of features, the number of cells and the SVM parameters when applied to other data sets.Because PhenoDissim applies classification accuracy as a proxy for dissimilarity, the highest dissimilarity value is 1, or 100% classification accuracy. Thus PhenoDissim will not be able to quantify the differences between two phenotypes if both have 100% classification accuracy from the negative control. Also when treatments generate similar phenotypes but different cellular subpopulation distribution, PhenoDissim might not be able to detect the distinctions. In these scenarios it will need to be combined with other methods. Because SVM classification needs to be performed between each pair of treatments, the proposed method is computationally more intensive than previous methods , which implements analysis methods and quality metrics used in this study. As a toolbox for phenotypic analysis in image-based screening, and quality control of screens and analysis methods, the phenoDist package facilitates testing different analysis methods with various image-based screens, which will help develop accurate and effective data analysis methods, and promote further application of image-based screening.The proposed PhenoDissim method needs minimum parameter optimization and is successfully applied in phenotype identification and clustering in the current kinome RNAi screen. More and diverse image-based screening data sets need to be investigated to evaluate proposed analysis methods. To facilitate screen data analysis in general, we have developed an R package, available through the Bioconductor project  , we collect  and , where xi is a set of feature vectors representing cells from treatment i, xj is a set of feature vectors representing cells from treatment j, and yi can be assigned 1 and yj can be assigned -1 to represent cell labeling. Virtually pooling  and , we can find a classifier y=f, where α is the parameter space of the function. The accuracy of the classification represents the separability of these two cell populations, and thus the phenotypic distance between the two treatments. One can estimate the classification accuracy by performing cross validation defined as CV(α)=L is the zero-one loss function, κ{1,…,N}→{1,…,K} is an indexing function for K-fold cross validation. A support vector machine classifier performs classification in an enlarged feature space as h(x) is the function to map the original features to an enlarged space and 〈,〉 is the dot product operator. We can define the kernel function as K=〈h(x),h(x′)〉. Two kernel functions are most frequently used, linear and radial K=exp(−γ∥x−x′∥2). We evaluated both kernel functions together with other methods (see below), and found that the radial kernel function always performed better than the linear kernel function (data not shown). Thus only the radial kernel function data is presented here. We first performed parameter tuning for cost (C) and gamma (γ)  per plate. For each sample well containing a perturbation, we computed the phenotypic dissimilarity between the sample well and each of the four negative controls on the same plate. There were a total of 779 genes targeted in the human kinome library with two replicates for each gene. To measure reproducibility, we calculated the Pearson correlation coefficient between replicate samples, of the sample phenotypic dissimilarity to each negative control, between replicate samples.MAD is the median absolute deviation, μ is the median and abs is the absolute value.Separation of controls: There were two types of positive controls (siUBC and siCLSPN) and one negative control (siRLUC) in the screening data set, with each control represented by four wells per plate. For positive controls, phenotypic dissimilarity to negative control was calculated the same way as the samples. For negative controls, we computed the phenotypic dissimilarity between each negative control well and the other three negative control wells on the same plate, and averaged the three measurements. The performance of the phenotypic dissimilarity method can be indicated by the separation between negative and positive controls, which can be measured by the robust Z’ factor score, as Gene clustering quality: After averaging two replicates of the same gene, we performed hierarchical clustering of 779 genes, based on their pair-wise phenotypic dissimilarity matrix. For validation, the hierarchical tree was cut into 20 clusters (the number of clusters is determined to maximize enriched GO terms), with genes in each cluster analyzed for gene annotation enrichment. Since most genes were kinases, we used the biological process gene ontology annotation . For eachttp://bioconductor.org/). The presented analysis was performed with R version 2.13 and phenoDist version 1.0.0.We implemented the presented phenotypic dissimilarity method and quality control metrics in an R/Bioconductor package, named phenoDist (The authors declare that they have no competing interests.XZ and MB conceived the idea. XZ carried out the analysis. XZ and MB wrote the manuscript. Both authors read and approved the final manuscript.Supplementary methods.Click here for file