Batch effects are responsible for the failure of promising genomic prognostic signatures, major ambiguities in published genomic results, and retractions of widely-publicized findings. Batch effect corrections have been developed to remove these artifacts, but they are designed to be used in population studies. But genomic technologies are beginning to be used in clinical applications where samples are analyzed one at a time for diagnostic, prognostic, and predictive applications. There are currently no batch correction methods that have been developed specifically for prediction. In this paper, we propose an new method called frozen surrogate variable analysis (fSVA) that borrows strength from a training set for individual sample batch correction. We show that fSVA improves prediction accuracy in simulations and in public genomic studies. fSVA is available as part of the sva Bioconductor package. Genomic technologies were originally developed and applied for basic science research and hypothesis generation . As thesOne of the reasons for the relatively low rate of success is the impact of unmeasured technological or biological confounders. These artifacts are collectively referred to as “batch effects” because the processing date, or batch, is the most commonly measured surrogate for these unmeasured variables in genomic studies . The umbBatch effects are responsible for the failure of promising genomic prognostic signatures , major aWhile a number of methods have been developed for removing batch effects in population-based genomic studies , there iHere we propose frozen Surrogate Variable Analysis (fSVA) as a method for batch correction in prediction problems. fSVA borrows strength from a reference database to address the challenges unique to batch correction for prediction. The fSVA approach has two main components. First, surrogate variable analysis (SVA) is used to correct for batch effects in the training database. Any standard classification algorithm can then be applied to build a classifier based on this clean training data set. Second, probability weights and coefficients estimated on the training database are used to remove batch effects in new samples. The classifier trained on the clean database can then be applied to these cleaned samples for prediction.sva Bioconductor package.We show with simulated data that the fSVA approach leads to substantial improvement in predictive accuracy when unmeasured variables are correlated with biological outcomes. We also apply fSVA to multiple publicly available microarray data sets and show improvements in prediction accuracy after correcting for batch. The methods developed in this paper have been implemented in the freely available m measured features and n samples in the training set, we let Xm×n be the matrix of feature data, where ijx is the value of feature i for sample j. For convenience, we will refer to X as the expression matrix for the remainder of the paper. However, our methods can be generally applied to any set of features, including measures of protein abundance, gene expression, or DNA methylation.The first step in batch correction for prediction problems is to remove batch effects from the training set. In the training set, the biological groups are known. This setting is similar to the population genomics setting and we can use a model for gene expression data originally developed for population correction of unmeasured confounders. If there are jy: ks(⋅) are a set of basis functions parameterizing the relationship between expression and outcome. If the prediction problem is two-class, then p1 = 1 and s1(jy) = 1(jy = 1) is an indicator function that sample j belongs to class one. In a multi-class prediction problem k > 1 and the ks(⋅) may represent a factor model for each class. In matrix form this model can be written \docE is not independent across samples  on the training database in order to identify surrogates for batch effects in the training samples. The training set can be “cleaned” of batch effects by regressing the effect of the surrogate variables out of the data for each feature. Any classification algorithm can then be developed on the basis of the clean training data set.In the training set, the biological classes are known, so X by the corresponding probability weight p2 weighted left singular vectors of X are used as estimates of G. An estimate of p2 can be obtained either through permutation st column of Xclean j′ represents the new clean sample. The classifier built on the clean training data can be applied to this clean data set to classify the new sample.To remove batch effects from the new sample, fSVA uses the coefficients estimated from the training database fSVA requires that a new singular value decomposition be applied to the augmented expression matrix once for each new sample. Although this is somewhat computationally intensive, in typical personalized medicine applications, sample collection and processing will be spread over a long period of time. In this setting, computational time is not of critical concern. However, for evaluating the fSVA methodology or developing new classifiers using cross-validation, it is important to be able to quickly calculate clean expression values for test samples.We propose an approximate fSVA algorithm that greatly reduces computing time by performing a streaming singular value decomposition . The basWX = TUDV. After convergence, the first p2 columns of the matrix V are the surrogate variables for the training set. Since U and V are orthonormal matrices, we can write TV = D−1TWXU. The matrix P = D−1TWU projects the columns of X onto the right singular vectors TV. Pre-multiplying a set of new samples newX by P results in an estimate of the singular values for the new samples: p2 columns of When removing batch effects from the training data, the last step is a weighted singular value decomposition of the training expression matrix TP, is calculated using only the samples in the training set. However, there is only a one-sample difference between the projection calculated in the training set and the projection that would be obtained with exact fSVA. As the training set size grows, the approximation is closer and closer to the answer that would be obtained from the exact algorithm. For smaller databases, there is less computational burden in calculating the exact estimates. However, for large training sets, the computational savings can be dramatic, as described in the simulation below.Estimates obtained using this approximate algorithm are not identical to those obtained using the exact fSVA algorithm. The projection matrix used in the approximation, πiγ and ibπ, each equal to 1 to indicate batch- or outcome-affected, and 0 to indicate otherwise. We also varied the distribution of these probability weights , the correlation of batch and outcome within the new samples matrix is unimportant. To have a representative amount of new samples from each combination of batch and outcome, we found it best to simulate the new samples by leaving the batch and outcome uncorrelated. That way, each of the four test-cases of batch and outcome combinations was represented in 25% of the new samples.We simulated 100 database samples and 100 new samples using the parameters described above. Each sample had 10,000 features. As a control, for each iteration in addition to performing fSVA correction, we performed SVA correction on the simulated database alone, and also performed prediction with no batch correction on the simulated database or new samples.To quantify the effect that fSVA had on prediction, we performed exact fSVA as described above on the simulated database and new samples. We then performed Prediction Analysis of Microarrays (PAM), a commonly used method for classifying microarrays . The PAMsample function in R  website . All excEach of the studies was randomly divided into equally-sized “database” and “new sample” subsets. We SVA-corrected the database subset, and then built a predictive model (PAM) on that corrected data. We then performed fSVA correction on the new samples. After performing fSVA correction, we measured the prediction accuracy of the model built on the database by calculating the number of times that the predicted outcome equaled the true outcome status, divided by the number of samples. This process was iterated 100 times for each study to obtain confidence intervals. This method is virtually identical to the simulation described above.Results from this process can be found below . Five ofBatch effects have been recognized as a crucial hurdle for population genomics experiments . They haWe have demonstrated the power of our approach in both simulated and real gene expression microarray data. However, our approach depends on similarity between the training set and the test samples, both in terms of the genes affected and the estimated coefficients. In small training sets, these assumptions may be violated. Similarly, training sets that show near perfect correlation between batch variables and biological classes represent an extreme case that can not be directly corrected using fSVA. An interesting avenue for future research is the use of publicly available microarray data to build increasingly large training databases for batch removal. We note that when the correlation between batch effects and the outcome is high it is impossible to remove these effects entirely and this may lead to poor prediction performance . Therefosva Bioconductor package (https://github.com/hilaryparker/fSVA.We have discussed here applications of fSVA to microarray data but the methodology may also be useful for other applications where genomic technologies are being applied for prediction or classification. For example, with appropriate transformations, SVA can be applied to RNA-sequencing  or DNA m package . Code an


The selection of therapeutic targets is a critical aspect of antibody-drug conjugate research and development. In this study, we applied computational methods to select candidate targets overexpressed in three major breast cancer subtypes as compared with a range of vital organs and tissues. Microarray data corresponding to over 8,000 tissue samples were collected from the public domain. Breast cancer samples were classified into molecular subtypes using an iterative ensemble approach combining six classification algorithms and three feature selection techniques, including a novel kernel density-based method. This feature selection method was used in conjunction with differential expression and subcellular localization information to assemble a primary list of targets. A total of 50 cell membrane targets were identified, including one target for which an antibody-drug conjugate is in clinical use, and six targets for which antibody-drug conjugates are in clinical trials for the treatment of breast cancer and other solid tumors. In addition, 50 extracellular proteins were identified as potential targets for non-internalizing strategies and alternative modalities. Candidate targets linked with the epithelial-to-mesenchymal transition were identified by analyzing differential gene expression in epithelial and mesenchymal tumor-derived cell lines. Overall, these results show that mining human gene expression data has the power to select and prioritize breast cancer antibody-drug conjugate targets, and the potential to lead to new and more effective cancer therapeutics. Personalized cancer therapies are expected to be more effective than conventional treatment and to minimize detrimental effects on normal cells . DiffereThe computational selection of therapeutic targets is a multifaceted process that generally starts with mRNA or protein quantitative analysis to identify targets that are overexpressed in tumor cells. Depending on the type of target and the chosen drug and delivery strategy, candidates are then prioritized using various approaches –11. Theset al. [t-test [F-test [p-values on sample size [Feature selection is another critical aspect of computational target selection. In a recent review, Saeys et al.  divided  [t-test  or the a [F-test . Limma,  [F-test , 25. Prople size , 27. Anople size  or suppople size . These mple size , adaptedple size , 32, as a.k.a. HER2) [In this study, we have applied feature selection and classification methods to identify candidate therapeutic targets in breast cancer, the most common cancer in women and a heterogeneous disease in nature . Breast a. HER2) . Althouga. HER2)  suggestsa. HER2) , we incla. HER2) . Metastaa. HER2) . The fira. HER2) . Resistaa. HER2) , 41. Mana. HER2) , 43. Hera. HER2) , 45, to Our approach for target selection and prioritization is schematized in Figure k-nearest neighbors (KNN), J48 and naïve Bayes. Accuracy increased as the number of features increased in q-value from Limma, overlapping coefficient of kernel densities and weight of SVM), and at each iteration classifiers were trained on all labeled instances. In total, 36 predictions were made for each sample . Labels assigned with high confidence (>95% of votes) by the ensemble of experts were fed back into the data and used for subsequent feature selection and training of the classifiers. Complete convergence was achieved after 15 iterations. At this point, 5,107 samples were assigned class labels unanimously among experts (100% of votes), and 70 samples were assigned class labels with high confidence (>95% of votes). An additional 82 samples were labeled with reasonable confidence (>75% of votes), and 120 samples were left unlabeled. As a final result, a total of 5,259 samples were labeled, in the following classes: 549 normal, 3,085 luminal, 479 HER2+ and 1,146 triple-negative breast cancer. Among labeled samples, 4,808 (91%) retained original labels assigned using sample annotations and receptor status. Figure Breast samples  were initially assigned to one of four classes: normal, luminal, HER2+ and triple-negative, based on sample annotations and receptor status. Class labels were validated using repeated cross-validation combining three feature selection methods, six classification algorithms and two multiclass classification strategies  is currently commercialized for the treatment of breast cancer . In thisClinicalTrials.gov database [Our list of cell membrane candidates contained one target for which an ADC is already in clinical use and six additional targets for which antibody-drug conjugates are in clinical trials for the treatment of breast cancer and other solid tumors Table . Combinidatabase , 54 revedatabase . In our database , the onldatabase , 57. Clidatabase –60, inclTargeted therapies are currently unavailable for the treatment of triple-negative cancer, and patients in this group have a generally poorer prognosis . Of the et al. [et al. [In this study, gene expression data mining was preferred as a medium for target selection and prioritization because of the near-transcriptome coverage of modern microarray platforms and the public availability of thousands of human gene expression datasets. Mass spectrometry-based proteomics data analysis would, in contrast, provide better estimates of the quantity of interest , although in a lower number of publically available samples, and at a typically lower resolution. Correlation between mRNA and protein levels was found to be relatively poor in a number of studies . This laet al.  quantifi [et al.  recentlyApart from tumor specificity, other factors such as tumor-specific aberrant subcellular localization may influence target selection. The glycoprotein nmb (GPNMB), for example, was characterized with high expression in all breast cancer subtypes versus normal tissues in average, but with relatively high expression in some normal tissues . GPNMB subcellular localization, however, tends to be restricted to intracellular compartments in normal cells, while being enriched on the cell surface in tumor cells . In otheExploratory gene expression analysis was performed to select candidate ADC targets, for further experimental validation at the protein level, in cell line and animal models and ultimately in clinical trials. Any of the genes in our list, EMT-related or not, respects the fundamental criterion for ADC target selection . Although membrane proteins represent more attractive targets for use in ADC internalization strategies, extracellular proteins may also prove useful, given that strategies such as that reported in  are deveRaw microarray data corresponding to 101,334 samples analyzed using the Affymetrix Human Genome U133 Plus 2.0 platform  was obtaMetadata associated with 101,334 samples  and the corresponding 3,643 series  were retrieved using GEOmetadb  and storq-values derived from linear models for microarray (Limma) moderated t-test [k-nearest neighbors (KNN) [Gene expression data was collected for a total of 5,379 breast samples. Receptor status data  was available in sample annotations for 3,500 samples . For each receptor, two locally adaptive kernel densities were estimated in annotated samples , and receptor status in other samples was predicted by assigning the label corresponding to the maximum posterior . Samplesd t-test , 85, thed t-test , 31, andd t-test . Locallyd t-test . The weid t-test . To estid t-test , 89. Clad t-test : baggingd t-test , J48  , naive Brs (KNN) , random rs (KNN)  and supprs (KNN) . For thers (KNN) . An iterrs (KNN)  was usedet al. [Results from Ross et al.  were useTo identify genes specific to or overexpressed in breast cancer, expression profiles in each molecular subtype were compared with gene expression in major organs and tissues were toxicity would likely be a serious concern . An initial filtering was done to retrieve genes with a maximum ratio  of 2, an average ratio  greater than 2 and an average overlapping coefficient , 31 smal


Experimental designs that take advantage of high-throughput sequencing to generate datasets include RNA sequencing (RNA-seq), chromatin immunoprecipitation sequencing (ChIP-seq), sequencing of 16S rRNA gene fragments, metagenomic analysis and selective growth experiments. In each case the underlying data are similar and are composed of counts of sequencing reads mapped to a large number of features in each sample. Despite this underlying similarity, the data analysis methods used for these experimental designs are all different, and do not translate across experiments. Alternative methods have been developed in the physical and geological sciences that treat similar data as compositions. Compositional data analysis methods transform the data to relative abundances with the result that the analyses are more robust and reproducible.Data from an in vitro selective growth experiment, an RNA-seq experiment and the Human Microbiome Project 16S rRNA gene abundance dataset were examined by ALDEx2, a compositional data analysis tool that uses Bayesian methods to infer technical and statistical error. The ALDEx2 approach is shown to be suitable for all three types of data: it correctly identifies both the direction and differential abundance of features in the differential growth experiment, it identifies a substantially similar set of differentially expressed genes in the RNA-seq dataset as the leading tools and it identifies as differential the taxa that distinguish the tongue dorsum and buccal mucosa in the Human Microbiome Project dataset. The design of ALDEx2 reduces the number of false positive identifications that result from datasets composed of many features in few samples.Statistical analysis of high-throughput sequencing datasets composed of per feature counts showed that the ALDEx2 R package is a simple and robust tool, which can be applied to RNA-seq, 16S rRNA gene sequencing and differential growth datasets, and by extension to other techniques that use a similar approach. The objective of many high-throughput sequencing studies is to identify those genes or features that make a significant difference between two or more conditions. These methods are diverse and include RNA sequencing (RNA-seq), chromatin immunoprecipitation sequencing (ChIP-seq), and metagenomic and 16S rRNA gene amplification analysis of microbial populations. All of these study designs share common aspects whereby DNA fragments are incorporated into a library, a small proportion of that library is sequenced on an instrument and the reads from the sequencing run are binned into features that represent an underlying biological entity. The entities can be genes, other expressed or non-expressed genomic features (RNA-seq and metagenomics), operational taxonomic units (OTUs) (16S rRNA gene sequencing) or genomic segments (ChIP-seq). Different statistical models are used to determine differential abundance in each type of study despite the underlying similarity in study design. The RNA-seq field has largely standardized on estimating the variation with the negative binomial . In contDespite current advice to treat high-throughput sequencing datasets as count data ,10, fundTreating high-throughput sequencing datasets as compositions is intuitive if one considers that the primary determinant of the observed sequencing depth is the sequencing platform used and the number of samples that are multiplexed per run. For example, an Illumina HiSeq can currently generate over 200 million reads per lane, a MiSeq approximately 20 million reads per run and an Ion Torrent instrument up to 6 million reads per run. Setting aside confounding effects such as the accuracy of the instrument, clearly the direct comparison of different numbers of reads per category would be erroneous. Thus, we must think of datasets derived from high-throughput sequencing as compositions instead of count data. The purpose of this work is to show that datasets from a wide variety of experimental designs, including RNA-seq, 16S rRNA gene sequencing and selective growth (selex) experiments share a similar underlying structure, and can be analyzed appropriately using methods developed and used for decades in fields as diverse as geology, ecology and paleontology -15.There have been many warnings regarding the use of standard statistical methods that assume the independence of the underlying observations when examining compositional data, the first being given by Karl Pearson in 1896 . These wThe major problem with compositional datasets is that the data points do not map to Euclidean space, but instead to a hyperplane referred to as the Aitchison simplex . AitchisCompositional datasets present several special challenges. The first challenge is that compositional data must be analyzed in a scale-invariant way: that is, the answer should be the same whether the investigator is dealing with proportions, percentages, ppm or per feature values where the total value is constrained to be the sum of the parts . ComposiAitchison realized that the above constraints could be alleviated by investigating the ratios between proportions . He deveA conceptually simple transformation is the centered log-ratio transformation, or clr. Here the read counts for each feature are divided by the geometric mean of the read counts of each feature in the sample, followed by taking the logarithm. The clr has the advantage that there is a one-to-one transformation of all features, allowing changes of all features to be observed. Moreover, if 2 is used as the base of the logarithm, then differences between features represent fold-changes in relative abundance between features, a measure that is natural for molecular biologists, biochemists and other life scientists. Additional transformations have been developed by Egozcue and collaborators that have more robust properties, but which lack a one-to-one mapping of the sample features ,22. We wx=: where the proportional sum is constrained to be 1. We could imagine that we have the counts from an RNA-seq experiment where the last feature is the count of reads mapping to ribosomal RNA. Converting to proportions,By way of example, consider the following set of values: and the difference between elements 1 and 2 is −0.042: a very small difference. If we remove one observation (say the last one), equivalent to what is often done when removing the large number of sequences mapping to rRNA gene sequences from RNA-seq datasets, this changes the dataset tox)= and the difference between elements 1 and 2 is −1.81. If the composition is reduced by removing the last element as before, then the corresponding clr values are , and the difference is unchanged at −1.81. The meaning of this result is that element 1 is 2−1.81 as abundant as element 2. Thus, the same relative difference between these two elements is maintained regardless of which other element, or combination of elements, is removed.Now the difference between elements 1 and 2 is much larger, being −0.263, and the investigator could be led to a different conclusion. In contrast we can consider these same elements as compositions and compute the relative difference between elements using the clr. In the case of the complete vector, the corresponding values (using base 2) are clr and three experimental (E) samples. First, sequences that map to each feature are enumerated and the table of read counts for each feature in each sample is converted to a distribution of posterior probabilities through Monte Carlo sampling from the Dirichlet distribution for each sample:Figure pi,j. The distributions are narrow if the feature and the sample contain a large number of counts and wide if either the feature or the sample has a small number of counts. Each Monte Carlo realization of pi,j is transformed by the clr transformation:An uninformative prior of 1/2 is used to model the frequency of features with zero counts ,29. Thisci,j value between conditions is subjected to both a Welch’s t-test and a Wilcoxon rank test giving two vectors of P values, and each P value instance is then corrected for multiple hypothesis testing using the false discovery rate (fdr) approach of Benjamini and Hochberg  and for selected cells are . This variant is identified as non-signficant by DESeq (fdr = 0.73), but is considered significant by ALDEx2 (fdr <1×10−3). The difference is presumably because the counts are similar in the selected and non-selected conditions (DESeq log2(fold-change)=22.9), but the relative abundances are very different (ALDEx2 diff.btw.q 50=26.2).The last line in Table We conclude that these tools, and by extension the many similar tools that use count-based methods to estimate variation ,36, are et al. [The second dataset was generated by Bottomly et al.  and contet al.  in JanuaP values as differences in scaled counts. What is not often appreciated is that gene expression is itself a limited resource in the cell, meaning that the transcript abundance should also be modeled as proportional [RNA-seq attempts to address the question: ‘What are the differences between gene expression in the samples in two or more conditions?’ It is reasonable to examine gene expression values as fold-change (relative differences) because the law of mass action that governs biochemical reactions depends on the ratios of reactants. For this reason, existing RNA-seq analysis tools report changes in gene expression as fold differences, despite computing ortional . Furtheret al. RNA-seq dataset [P values and to calculate corresponding fdr statistics. Thus, this approach allowed the estimation of statistical significance when the number of samples was small.We examined the Bottomly  dataset  using th dataset , and so  dataset  for a de dataset . This vaet al. [Differential expression in RNA-seq datasets is often visualized globally by Bland-Altman plots , which set al.  is shownThe MA plot in Figure The MW plot panel in Figure t-test for DMC-sampled and clr-transformed data ensures that the genes identified as differentially expressed exhibit a greater gene expression difference between mouse strains than the gene expression variation within either mouse strain. This example shows that the compositional analysis approach works well for RNA-seq datasets.The number of significant features identified by DESeq, baySeq and ALDEx2 is presented in a Venn diagram in the third panel of Figure The third dataset is a 16S rRNA gene sequencing dataset obtained from the Human Microbiome web repository on 10 January 2014 . This daThe Human Microbiome Project dataset contained 23,393 OTUs containing one count or more for any of the 316 samples from the tongue dorsum and 312 samples from the buccal mucosa. This dataset was analyzed with 16 DMC instances rather than 128 because of time and memory constraints, and because the analysis above suggested that 16 DMC samples would provide sufficient selectivity on low-count OTUs. The analysis was completed over all 23,393 OTUs in the dataset in approximately 76 minutes on a computer with 16 Gb RAM and an i7 processor. We observed 755 differentially abundant OTUs that had a Benjamini–Hochberg fdr <0.05, and collected the 53 OTUs that passed that fdr cut-off with absolute effect size >0.6. These 53 OTUs are displayed on the heat map in Figure Actinomyces, Prevotella and Veillonella have an increased relative abundance in samples from the tongue dorsum, while OTUs assigned to Haemophilus and Gemella have an increased relative abundance in samples from the buccal mucosa. Several distinct OTUs assigned to Streptococcus had different relative abundances in both body sites, although, in general OTUs assigned to this genus were over-represented in the buccal mucosa. Together, these results are congruent with those of Segata et al., who observed that these genera differentiate these body sites [The results show that OTUs assigned to the genera dy sites . We conct-test or the non-parametric Wilcoxon test, to identify features that are differentially abundant between conditions in datasets derived from selexes, RNA-seq or 16S rRNA gene segment sequencing. Historically, each of these experimental designs has used a distinct statistical model to determine significance when examining difference between conditions, despite these experiments generating similar types of data comprising large numbers of reads that are binned into one or more categories.High-throughput sequencing is increasingly used to identify differences between datasets composed of DNA or RNA sequences. Here we show that data transformations appropriate for compositional data  can be uThe diversity of methods suggests a situation where the assumptions and parameters have been tuned to give biologically meaningful results with the unintended consequence of making these methods unexpectedly fragile. For example, RNA-seq analysis tools are very sensitive to outliers in the datasets ,32. RecoFor 16S rRNA gene sequencing, although the tools used to determine differential abundance between conditions are still evolving rapidly, no tool treats the data as compositions and ensures sub-compositional coherence. For example, one widely used method  is simila priori information on what constitutes a biologically meaningful difference, there is no clearly demarcated line-in-the-sand that can be drawn between differential and non-differential abundance in any experimental design. Moreover, the relatively high cost of sequencing has led to study designs that often emphasize per sample sequencing depth over biological replication [Demonstration of the superiority of analysis methods is limited by two main factors: the first is that the abundance of a feature in a sample is a continuous variable, and the second is that there is often no objective standard to determine what is differentially abundant. Without lication , despitelication ,47. ThisStandard statistical practice indicates that the analyses should be limited to determining those features with abundance differences between conditions that are reliably larger than the variation within either condition. We and others have developed non-parametric tools to identify those features fulfilling those criteria ,32.The approach used in this paper models the reads per feature as proportions, acknowledging that the total sum of reads for a sample is not itself important. The precision of estimation of the proportional values is determined by taking Monte Carlo instances sampled from the Dirichlet distribution, which takes into account the number of reads per feature and the total sequencing depth. This approach generates narrow distributions when the read count is high per feature and per sample. The proportional data are transformed using the clr transformation, allowing standard statistical tools to be applied to each instance. Summary statistics are then reported as the expected value of the distributions.We show that this approach is generalizable to three completely different experimental designs: a selex, an RNA-seq type experiment and a 16S rRNA gene amplicon-sequencing experiment. For the selex experiment, the ALDEx2 approach identified known active enzyme variants and weeded out inactive variants . ALDEx2 ChIP-seq: Chromatin immunoprecipitation sequencing; clr: centered log-ratio; DMC: Monte Carlo instance of the Dirichlet distribution; fdr: false discovery rate; MA: Mean difference between conditions vs average expression; MW: Mean difference between conditions vs maximum within-condition variance; OTU: Operational taxonomic unit; PCR: Polymerase chain reaction; PPM: Parts per million; RAM: Random access memory; RNA-seq: RNA sequencing; rRNA: Ribosomal RNA; selex: selective growth experiment.The authors declare that they have no competing interests.AF conceived and designed the study, analyzed the data, contributed code and gave final approval for the manuscript. GG conceived and designed the study, analyzed the data, contributed code, wrote the manuscript and gave final approval for the manuscript. JM analyzed the data, contributed code to ALDEx2, revised the manuscript and gave final approval for the manuscript. TM designed the study, generated and validated the datasets and gave final approval for the manuscript. DE designed the study, generated and validated the datasets, revised the manuscript and gave final approval for the manuscript. JR designed the study, analyzed the data, revised the manuscript and gave final approval for the manuscript. All authors read and approved the final manuscript.Excel file containing ALDEx2 output for the Bottomly RNA-seq dataset. Column 1 contains gene features from the mouse genome. Column 2 contains the relative abundance of each feature averaged across all samples. Columns 2 and 3 contain the average relative abundance of DBA/2J and C57BL6J gene expression. Columns 4 and 5 contain the median between- and within-condition differences. Column 6 contains the median effect size. Columns 7 and 8 contain the expected values of P and the associated Benjamini–Hochberg corrected fdr values for Welch’s t-tests and the final two columns contain the expected values for Wilcoxon tests.Click here for fileExcel file containing ALDEx2 output for the buccal vs tongue dorsum comparison. Column 1 contains HMP OTU identifier labels. Column 2 contains the relative abundance of each feature averaged across all samples. Columns 2 and 3 contain the average relative abundance of buccal and tongue OTU sequences. Columns 4 and 5 contain the median between- and within-condition differences. Column 6 contains the median effect size. Columns 7 and 8 contain the expected values of P and the associated Benjamini–Hochberg corrected fdr values for Welch’s t-tests and the final two columns contain the expected values for Wilcoxon tests.Click here for file


Sample- and gene- based hierarchical cluster analyses have been widely adopted as tools for exploring gene expression data in high-throughput experiments. Gene expression values (read counts) generated by RNA sequencing technology (RNA-seq) are discrete variables with special statistical properties, such as over-dispersion and right-skewness. Additionally, read counts are subject to technology artifacts as differences in sequencing depth. This possesses a challenge to finding distance measures suitable for hierarchical clustering. Normalization and transformation procedures have been proposed to favor the use of Euclidean and correlation based distances. Additionally, novel model-based dissimilarities that account for RNA-seq data characteristics have also been proposed. Adequacy of dissimilarity measures has been assessed using parametric simulations or exemplar datasets that may limit the scope of the conclusions. Here, we propose the simulation of realistic conditions through creation of plasmode datasets, to assess the adequacy of dissimilarity measures for sample-based hierarchical clustering of RNA-seq data. Consistent results were obtained using plasmode datasets based on RNA-seq experiments conducted under widely different conditions. Dissimilarity measures based on Euclidean distance that only considered data normalization or data standardization were not reliable to represent the expected hierarchical structure. Conversely, using either a Poisson-based dissimilarity or a rank correlation based dissimilarity or an appropriate data transformation, resulted in dendrograms that resemble the expected hierarchical structure. Plasmode datasets can be generated for a wide range of scenarios upon which dissimilarity measures can be evaluated for sample-based hierarchical clustering analysis. We showed different ways of generating such plasmodes and applied them to the problem of selecting a suitable dissimilarity measure. We report several measures that are satisfactory and the choice of a particular measure may rely on the availability on the software pipeline of preference. Hierarchical cluster analysis has been a popular method for finding patterns in data and for representing results of gene expression analysis . ClusterBefore implementing any statistical analysis of RNA-seq data, normalization and transformation have to be performed. ,5,6. NorAlthough a Gaussian distribution assumption is not required to compute Euclidean and correlation based distances, transformations that convert count data into a continuous and almost Gaussianly distributed variable  could beFinally, instead of using transformations to approximate the data to a pre-specified distribution where available dissimilarity measures perform well, model based methods can be directly used to compute dissimilarity measures .a priori known structure in order to assess the goodness of results from alternative analyses. Parametric simulations, exemplar datasets, and permutation sampling have been used to generate such datasets in clustering analysis of biological data ), 2) DE transcripts + all nonDE transcripts (DE[10%]+nonDE[90%]), and 3) DE transcripts + a random sample of 50% from nonDE transcripts (DE[20%]+nonDE[80%]).We built plasmodes for this dataset by using samples from B6 strain, partitioning them in two groups and adding known effects for selected genes taken from the difference in gene expression with strain D2. periment . To evalSince this dataset did not have a natural sample structure derived from experimental conditions, a structure had to be induced in order to know a priori the expected clustering configuration. From a descriptive multidimensional scaling analysis of the 24 pig samples , we selected 5 dissimilar samples  according to their configuration in the main plane . SynthetDefining a dissimilarity measure and a linkage method are the two key decisions for performing hierarchical cluster analysis. We focused on assessing the adequacy of dissimilarity measures that have been commonly used for clustering gene expression data. We also include a recently proposed dissimilarity measure for RNA-seq count data . As linkraw), ii) after normalizing samples using the median ratio size factor proposed by Anders and Huber ) uses only differentially expressed transcripts, therefore the expected hierarchy should arrange samples in two separate groups according to main treatment labels. Such is the structure obtained utilizing the Poisson (poi) dissimilarity measure (rnr) ([10%]+nonDE[90%]), uses information from differentially (10%) and non differentially expressed (90%) transcripts. As a result, we expected that the dissimilarity measures would tend to represent other aspects of samples in addition to the treatment effect. In concordance with such expected structure, dendrogram obtained using the Poisson dissimilarity (rnr) ([20%]+nonDE[80%]) represents an intermediate case that is useful to further explore the performance of dissimilarity measures because it is enriched in DE genes with respect to scenario 1, but it still conserves 80% of background (nonDE) genes. The dendrogram based on the Poisson dissimilarity  (poi and rnr dissimilarities. measure . Using tre (rnr)  mixed trmilarity  firstly ty (rnr)  did not milarity  presenteity (rnr  did not ctively) . These r[100%], DE[10%]+nonDE[90%] and DE[20%]+nonDE[80%]) are presented in raw) is poorly related to any other dissimilarity measure. Second, Euclidean distance computed on normalized data (rnr) and 1- Pearson correlation dissimilarity (pea) presented medium to high correlations of cophenetic matrices (mean from 0.67 to 0.89) with high variability (standard deviation from 0.13 to 0.29) with each other and low correlation values with other dissimilarity measures. Third, there is a subset comprising 1-Pearson correlation dissimilarity computed on log-transformed counts (plg), 1-Spearman correlation dissimilarity (spe), Euclidean distance computed on transformed counts after applying either a variance stabilizing function (vsd) or a regularized logarithm (rld), and the Poisson dissimilarity (poi). This last group of dissimilarity measures presents high correlations of cophenetic matrices (mean from 0.82 to 0.99) with low to medium variability (standard deviation from 0.01 to 0.2). Only hierarchies obtained with dissimilarity measures from the third group consistently presented the expected natural structure created by design in the plasmode generation process, being rld, vsd, spe and plg the more consistent across all scenarios (Correlations between hierarchies obtained with the eight dissimilarity measure approaches for each of three scenarios (DEraw), Euclidean distances using normalized samples (rnr), Euclidean distances using regularized logarithm (rld) Euclidean distances using variance stabilizing transformation (vsd), 1- Pearson correlation using raw counts (pea), 1- Pearson correlation using counts transformed by logarithm (plg), and 1- Spearman correlation using raw counts (spe), and Poisson dissimilarity (poi). Columns correspond to the three scenarios generated for Bottomly (with different proportion of DE genes) and the MSUPRP dataset. We considered a clustering from a dissimilarity measure to be consistent if hierarchies obtained for different plasmode datasets within each dissimilarity measure were highly correlated and presented a low standard deviation. Clustering based on raw, rnr and pea were generally inconsistent presenting a number of very different hierarchical structures.Mean and standard deviation of correlation between cophenetic matrices of dendrograms (N = 50 plasmode datasets) for each of eight dissimilarity measures: Euclidean distances using raw count data  of correlation between cophenetic matrices  rnr and pea, and 3) poi, rld, vsd, plg and spe. Dendrograms from raw did not agree with dendrograms from other groups. Hierarchies from dissimilarity measures in group 2 presented a medium correlation of cophenetic matrices with high variability (0.69±0.22). Dendrograms from the dissimilarity measures in group 3 presented high correlation values with each other (>0.98) and low variation (<0.01).Correlations between hierarchies for the eight dissimilarity measures are summarized in raw, rnr and pea, whereas clusters were much more consistent (r>0.98) for poi, rld, vsd, plg and spe dissimilarities. Additionally, dissimilarity measures raw, rnr, and pea were poorly correlated with the reference dissimilarity  while poi, rld, vsd, plg, and spe were highly correlated with the reference dissimilarity  the comparison of traditional dissimilarity measures and dissimilarity measures based on discrete count distributions specifically proposed for RNA-seq, and ii) studying the effect of normalization and transformation prior to computing dissimilarities.The Euclidean distance or the Pearson correlation based dissimilarity computed after transforming data is a routine method adopted from microarray gene expression analysis ,2. In faThe eight evaluated dissimilarity measures presented a common pattern of agreement and consistency in recovering the expected sample structure for both plasmode datasets. Dissimilarity measures with high level of agreement between them—correlations between cophenetic matrices with high mean and low standard deviation Figs  and 6—prpoi, rld, vsd, plg and spe dissimilarities, which presented similar level of agreement in both datasets .To assess the adequacy of a dissimilarity measure, both agreement and consistency are important. We showed this with ets Figs  and 6. Hets Figs . A countraw and rnr had low agreement with dissimilarity measures that resemble better the true structure of data, e.g. rld, vsd, spe or plg or poi  was suitable to represent the natural structure of samples even without normalizing data, possibly because it preserves the relative rank relationships, and it is less influenced by skewness and outliers [vsd) and regularized logarithm (rld) approaches consistently retrieved the expected dendrogram structure. Both transformations model the mean-variance relationship across all genes to stabilize the variance of counts across samples [poi) generated dendrograms with the expected structure. This is not surprising considering that read counts are usually assumed to fit over dispersed Poisson distributions[vsd or Euclidean distances on normalized data, but using overdispersed Poisson simulations. Our results are encouraging because we did not use a parametric model to produce similar outcomes.Accounting for the discrete nature of read counts in RNA-seq data is the most important issue to consider when computing dissimilarity measures. For instance, Euclidean distance and Pearson correlation based measures are known to be influenced by scale, skewness and outliers, thus, they may not work well for count data . In suppoutliers  when it  samples ,31. The ributions. Similarributions obtainedrld, vsd, plg, spe and poi were highly consistent under all scenarios with a tendency to diminish consistency as the number of non informative genes increases. Although we focused our comparison on the effect of dissimilarity measures on hierarchical clustering results, the same plasmodes could be used to investigate the effect of other decisions made when performing sample-based clustering as the selection of the hierarchical clustering algorithm per-se or even the effect of pre-filtering transcript according to their level of expression [Sample-based hierarchical cluster analysis can be used as a tool to present results after differential expression analysis or it can be used as an explorative technique for finding patterns in data. In the first approach only informative genes, i.e. differentially expressed genes  are used while in the second approach informative as well as non informative genes  are utilized . As the pression –49. We dGenerating plasmode datasets from experimental data is a reliable tool for evaluating dissimilarity measures in agglomerative hierarchical cluster analysis of RNA-seq data. Depending on the characteristics of the available datasets, several scenarios can be established to compare dissimilarity measures upon a broad spectrum of more realistic conditions than using other simulation approaches. Similar methodologies can be applied to study gene-based clustering as well as other clustering analysis methods.ad hoc functions, as well as model-based dissimilarity for RNA-seq data, were consistent in reproducing the expected sample structure in hierarchical dendrograms.Explorative sample-based hierarchical clustering of RNA-seq data needs as an input a dissimilarity matrix that accounts for the mean-variance relationship of the discrete nature of read counts. Euclidean distance calculated either on data that have been previously logarithm-transformed or regularized with more complex S1 FileThis file contains raw count data (6 columns x 25798 rows) for the five animals used to generate the MSUPRP plasmode datasets.(TXT)Click here for additional data file.S1 TextThis file contains a description of sample preparation, sequencing design and bioinformatics tools used for mapping reads and obtaining the count matrix for MSUPRP experiment.(PDF)Click here for additional data file.S2 TextThis file presents figures describing the plasmode generation process as well as the calculation of reference dissimilarity and reference dendrogram for MSUPRP plasmodes.(PDF)Click here for additional data file.


A major weakness in many high-throughput genomic studies is the lack of consideration of a clinical environment where one patient at a time must be evaluated. We examined generalizable and platform-specific sources of variation from NanoString gene expression data on both ovarian cancer and Hodgkin lymphoma patients. A reference-based strategy, applicable to single-patient molecular testing is proposed for batch effect correction. The proposed protocol improved performance in an established Hodgkin lymphoma classifier, reducing batch-to-batch misclassification while retaining accuracy and precision. We suggest this strategy may facilitate development of NanoString and similar molecular assays by accelerating prospective validation and clinical uptake of relevant diagnostics. The use of molecular technologies in clinical assays for guiding patient diagnosis, prognosis and management is paving the way for precision medicine. Gene expression in particular has been widely used in biomedical research to identify biomarkers and genetic profiles that enable disease diagnosis, (sub) classification, and prediction of prognosis and response to therapy–4. DespiPre-processing, normalization and accounting for systematic sources of variability in gene expression data affect the ability to combine different cohorts for model development and cross-cohort predictions for a single patient; these are key requirements for the development, validation and proof of utility for clinical assays,9. BatchApproaches for BE adjustment in multi-sample data –15 generGenerally, BE refers to systematic variability that may be attributed to RNA extractions, types or conditions of tissue, differences between operating labs and technicians, the use of different batches of reagents, age and storage conditions of assays components, experimental conditions, or hardware versions. Failing to correct for BE, particularly when the signal from the confounder is larger than the biological signal, can result in missed or false discoveries and irreproducible results,17,18. GThe NanoString nCounter technology is a relatively new platform for quantifying RNA that exhibits several advantages over traditional microarray, quantitative PCR (qPCR) and RNA-seq methods. It is simple to use, highly automated, cost- and time-effective allowing quantification of up to 800 targets in a single reaction. Its sensitivity is comparable to that of qPCR–22, withUsing the NanoString nCounter platform, we considered quality control metrics, normalization procedures, and BE adjustment methods for gene expression data obtained with the goal of assembling a protocol suitable for single-patient processing, which can be adopted in clinical settings from start to finish. We assessed both within and between-batch variability, focusing on the latter as many existing studies have already investigated within-batch reproducibility of nCounter data,23,27. DThe NanoString technology is based on single-molecule imaging of color-coded barcodes bound to target-specific probes,10. A Coet al.[Datasets obtained from different NanoString experiments included 161 clinical specimens from two cancer types: Hodgkin lymphoma (HL) and ovarian cancer (OC). For the HL cohort, data was derived from Scott et al., whereinet al. were alsRNA extraction was standardized within each research group (either Hodgkin or ovarian). All samples were reviewed for the presence of tumour. Ovarian frozen specimens were cryosectioned , tissue was dissociated by vortexing in Trizol reagent and then processed using the Qiagen miRNeasy protocol (Qiagen) as per manufacturer’s recommendations. FFPE tissues were sectioned (3 sections at 10 μm) into microfuge tubes. OTTA-sourced specimens were collected by scrapping 2–4 5um section off glass slides. FFPE specimens were de-paraffinized and processed according to the Qiagen miRNeasy FFPE kit (Qiagen) with an elongated 55°C digest period (45 min). Lymphoma specimens, all FFPE derived, were processed similarly with the exception of deparaffinization using Qiagen deparaffinization solution as described previously.For oligonucleotide experiments, standard desalted, single strand, 100-base DNA oligonucleotides were obtained (IDT) and a single pool of DNA oligonucleotides was generated by mixing all oligonucleotides in equimolar concentration corresponding to the sum of targets in both ovarian CodeSets. This resulted in each CodeSet targeting only a subset of the total pool. We refer to the “functional concentration” as the concentration of the subset oligonucleotide pool that is targeted in a given CodeSet hybridization. In the case of Hodgkin lymphoma CodeSets an oligonucleotide pool corresponding only to the prognostic genes  was usedThe manufacturer’s protocol was adhered to with the exceptions noted below. Briefly, capture, reporter, specimen total RNA (or DNA oligonucleotides) were mixed with hybridization buffer and hybridized at 65°C overnight. Sample, wash reagents and imaging cartridge were then processed on the nCounter Prep Station and finally imaged on the nCounter Digital Analyzer. FFPE derived specimens used 200–500 ng of input total RNA, whereas frozen specimens used only the recommended 100 ng. Hybridization times ranged from 12–23 hours and were strictly monitored in selected cohort experiments. See Raw data was assessed using several quality assurance (QA) metrics to measure imaging quality, oversaturation and overall signal to noise. All samples satisfying QA metric checks were log-transformed (base 2) to help with distributional assumptions, and were normalized by subtracting the average expression level of HK genes . Principal Components Analysis (PCA) and Principal Variance Components Analysis (PVCA) were used to measure the impact and assess sources of variability in the normalized data. The minimal set of principal components for PVCA were selR), for precision; the coefficient of accuracy (Ca), to assess the systematic bias; Lin’s concordance correlation[c = R Ca). The degree of BE removal was evaluated using DSC.A series of multi-sample methods were applied to both OC and HL data: batch mean centering (bmc), mean anrrelation was usedXB that were run in batch B to samples in XA that were run in batch A, the reference approach would require that some number of reference samples (R) would be run in both batches A and B, resulting in RA and RB. These reference samples would be used to calibrate between the two batches. Mathematically, we assumed the following standard additive  batch effect model on the normalized and log (base 2) transformed NanoString data[Adjusting for batch effects using reference samples, involved running designated reference samples in every batch along actual samples and measuring the same set of genes. Assuming we have two batches (A and B) and were interested in calibrating samples ring data:XijA=Xii in sample j run in batch A, A be a set of designated reference samples of size k, run alongside actual samples in batch A, we can then writeWhere i, measured in the original sample, is not expressed in the reference sample, then We note in the above equation that if a certain gene A and similarly B prior to combining the two data sets. In certain cases a model may have been developed with data that were not batch corrected and hence future data would need to be calibrated to the training data. This is achieved by adding the difference between the reference to the new data: XB + (RA − RB).If the interest is to combine data obtained from different batches, this can be achieved by subtracting 2< 0.95) and the ability to detect the smallest positive control. Failures were more common in OVO cohort (12/135 failed) vs. HLO (no failure); however, this was expected as dilution experiments were set to stress usability limits. OVCL had a large number of genes that were not expressed  metric and was removed. Samples were flagged as imaging failures if the percentage of lane images FOV obtained was less than 75% of the requested number of fields. This may be affected by either physical problems with the cartridge or oversaturation of probes . Oversaturation occurs when probes compete for physical space on a lane, displacing positive control probes and resulting in loss of linearity while simultaneously probes expressed at low levels become indistinguishable from background noise. Lane oversaturation was determined using the linearity of positive control probes , a quality assurance measure suggested by the manufacturer to monitor oversaturation, was not used, as there were minor inconsistencies between hardware versions. BD also did not correlate well with S/N  to help with distribution assumptions, as needed. NanoString recommends a three-step normalization procedure which involves adjustments relative to positive controls and HK as well as background subtraction . DiffereACTB, CLTC and RPLP0. For OC, HK were chosen from a panel of commonly accepted invariant genes represented on the NanoString Human Reference GX panel, based on their variability from a number of OC studies[RPL19, ACTB, PGK1, SDHA, and POLR1B from highest to lowest median expression respectively. The selection of HK is important; optimal genes should have lower variance across samples and an expression level that spans that of all other genes while remaining in the linear dynamic range of the assay. Moreover, although there is little formal analysis as to the optimal number of genes to use as housekeepers, more is better[HK selection, in the HL CodeSets was kept consistent with published works, using C studies,34. Fiveis better; our curCodeSet variability can challenge the reproducibility of results for any third-party without access to original samples and CodeSet. Technical and true biological (e.g. histology and Epstein Barr Virus (EBV) status) variables were considered in PVCA Figs  and 4.In both OC and HL, the variability associated with CodeSet was ~10%. HL clinical samples showed pronounced CodeSet-dependent shifts in plots of the first three principal components. A significant DSC of 0.21 and 0.26 was found in the HL and OC datasets respectively  Tables  and 4, iCommon multi-sample BE adjustments in gene expression experiments,13,12,15Using reference samples for BE adjustment has been suggested as more effective method for calibration,11, wherPVCA plots were used to compare the variability post-adjustment to the unadjusted data Figs  and 6. Ia = 0.81) appeared to result in a systematic shift from the identity line. If the threshold model is to be used 4/31 cases would be misclassified as low risk in the second CodeSet when in fact they were classified as high risk in the first CodeSet. This can only be attributed to batch effect associated with CodeSet. We used the reference-based strategy, randomly selecting 3 samples and setting them as a reference (any3_ref), to correct this bias, resulting in no misclassifications (a was near perfect (over 0.99). We observed no misclassification 39% of times, 1 misclassification 18% of the times, and a maximum of 2 misclassifications 44% of the times. It should be noted that in all cases misclassifications were attributed to the same two cases that sat very close to the threshold, and slight perturbations in the data shifted their classification.BE can have massive implications on downstream analysis. To assess this impact using the NanoString nCounter data, we evaluated a reference-based approach to correct for BE using a previously developed and published HL prognostic model that useications . To ensuWe have provided empirical evidence that the use of our reference-based BE correction strategy is equivalent to population-based correction methods with the advantage of being better suited to clinical applications. Though multi-sample BE correction and normalization methods may be appropriate for exploratory retrospective studies, as bulk data would likely be available, special care should be taken to ensure that samples are from homogenous populations. One problematic example within the spectrum of OC could be comparing a North American population OC cohort to a Japanese cohort as the latter are known to have a larger proportion of ovarian clear cell carcinoma histotype and lesser proportion of high-grade serous.While in some cases a biological signal will over-power any of the above noted variability, this is near impossible to predict in advance. Careful experimental design is critical to maximize the utility of any study intending to generate a clinically relevant classifier, whether diagnostic, prognostic or predictive. In such studies, planning for a reference-based strategy is highly desirable. Optimally, a reference would be included on each cartridge—this being the smallest “batch”. However, within-lot variability, examined in several recent studies, were found to be highly stable and to yield negligible effects,27; our Variability in gene expression data can be partitioned into two sources: biologiOverall we found lot-to-lot variance on a per-probe basis varied widely and was the single greatest source of identifiable variation. Not all probes varied, however, since we do not have knowledge on batch-manufacture of specific probe sets, it is unknown whether stability is inherent to a gene/probe design or if genes with little difference were from the same manufacturing pool.In validating our reference-based strategy for batch-independent normalization we encountered some unexpected results. Specifically, the poor performance of the synthetic DNA oligonucleotide references was surprising. This may have been due to: i) A substantial fraction of n-1 and other incomplete oligonucleotides known to occur in longer synthesis reactions even with high coupling efficiency. This maSimilarly, in the OC cohort, the cell line-based reference performed poorly. In retrospect, this may have been predictable since many of the genes were not expressed in the cell lines; expression in clinical samples originated from stroma and their absence in cell lines hindered correction.in vitro transcribed RNA strategy appears to be the method employed for NanoString’s FDA approved “ProSigna” assay[Finally, we were unable to test specifically whether synthetic RNA-oligonucleotides had superior performance to DNA-oligonucleotides (or other tested references). Synthetic RNA may be an optimal solution for clinical/commercial assays as it could be recreated within very precise parameters. An na” assay. Howeverde facto gold standard, in vitro transcribed RNA pools, has been defined by FDA approvals of NanoString’s ProSigna assay. This synthetic RNA approach is unfortunately expensive to establish and maintain during the research and development phase. Large pools of high-quality RNA from real biological samples of interest may be a low cost stand-in, could be shared with others continuing/reproducing results, and can be migrated to replacement pools or a long-term, commercial-product solution at a later stage.Advance planning is key to any study and identifying the goals in a NanoString-based experiment is no exception. Should a normalization reference sample be required, then careful selection of such a reference should be made: (1) must act identically to a test specimen in the assay, (2) must be plentiful, even in the initial research setting it should last through several CodeSets worth of reagents—stock should be established early, (3) must adequately represent all genes of interest. Cell lines may be suitable in some applications, though not in the above example. If they are used, it is important to avoid replenishing stock by growing more cells as transcript levels may be affected by culture conditions including confluence, nutrient availability, oxygenations, pH, and handling. A new batch of cells is a “new” reference, albeit with similar characteristics to the original. Any new reference must be migrated into the experimental protocol appropriately. Finally, (4) normal tissue, pooled samples or synthetic RNA pools may be considered—it appears that synthetic DNA oligonucleotide pools are not suitable for this purpose. The current Overall, we found the NanoString gene expression platform to be an easy to use and highly robust technology. Experimental variability is relatively small and can be dealt with using good experimental design and proactive planning, keeping in mind the goals of a given project and the desired reproducibility.S1 FileThis file contains additional extensive detail on all methods and parameters related to processing and establishment of metrics for quality assurance.(DOCX)Click here for additional data file.S2 FileThis file contains additional extensive detail on the method used for normalization of the data using the reference-based strategy and example data comparing our method to the manufacturer recommendations.(DOCX)Click here for additional data file.


To improve the applicability of RNA-seq technology, a large number of RNA-seq data analysis methods and correction algorithms have been developed. Although these new methods and algorithms have steadily improved transcriptome analysis, greater prediction accuracy is needed to better guide experimental designs with computational results. In this study, a new tool for the identification of differentially expressed genes with RNA-seq data, named GExposer, was developed. This tool introduces a local normalization algorithm to reduce the bias of nonrandomly positioned read depth. The naive Bayes classifier is employed to integrate fold change, transcript length, and GC content to identify differentially expressed genes. Results on several independent tests show that GExposer has better performance than other methods. The combination of the local normalization algorithm and naive Bayes classifier with three attributes can achieve better results; both false positive rates and false negative rates are reduced. However, only a small portion of genes is affected by the local normalization and GC content correction. There are two steps in analyzing the RNA-seq reads. The obtained short reads are first aligned to a reference genome or transcriptome, and, in the second step, for a given gene, the numbers of reads are compared between two different samples. The number of short reads mapped onto one gene is the count that is taken as a measure of the expression level of the gene. Many different types of analyses can be applied to the results of short-read alignment, including single nucleotide polymorphism discovery, alternative transcript identification, and gene expression profiling.RNA-Seq is a technology based on next-generation sequencing to determine transcript abundance, transcriptional structure of genes, and posttranscriptional modifications. It is essential to accurately construct genome-wide gene expression profiles in order to interpret the functional elements of the genome, molecular constituents of cells, development of organisms, and mechanism of diseases . RNA-seqBecause of the importance of RNA-seq, many methods have been developed to analyze aligned RNA-seq data to identify differentially expressed (DE) genes over the last four years. They include edgeR , DESeq , Cuffdift-tests . For the training step, the function of “naiveBayes” in the package e1071 is used to train a statistical model based on a training set, in which each gene has three attributes and known status. For the prediction step, the function of “prediction” can calculate the conditional a posteriori probability for a gene based on its feature vector with the three values. The conditional a posteriori probability is returned as a score, which is the probability of the given gene belonging to the class. The training data set and the R code are available for downloading from our website http://sysbio.unl.edu/.Mi  are the mean values of the numbers of aligned reads for a given transcript in all replicates of sample Si . Here, Mi  are values after both the local and global normalizations. The attribute of ARPK for a transcript is calculated as Mi  have the same definition as in FC calculation and TL is the transcript length. The attribute of GCC can be calculated asc is the average GCC of reads mapped to a given transcript in both samples and C is the average GCC of all mapped reads. The GCC of one read is the ratio of the total number of guanine and cytosine to the length of the read. For each gene, the NB model will give a score in  by its three features. The higher the score, the higher possibility a gene is differentially expressed. Here, we also assume that the features of a given gene are independent of its specific biological background.Three attributes are calculated with the following equations. The attribute of FC is calculated as T with coordinates  on a chromosome, the number of reads starting from position ti  is defined as ri. The total number of reads mapped to the transcript is R = ∑ri. As the assumption that ri is an accumulation of random events, ri can be modeled as p(λT) is a Poisson distribution with parameter λT. The unbiased estimate of λT isλT, such as at 97.5% and 2.5% confidence level, respectively, in this paper. With the UCL and LCL, the number of reads for a given position, ri, can be corrected as Generally, reads are positioned randomly along every transcript in RNA-seq . For a gi i = 1,,…, n is plicates , but a P2 fold change), the genes were grouped into three sets: DE, no-call, and non-DE (NDE), with the log2(fold-change) being >1.5  and <0.5, respectively. The expression log2(fold-change) for RT-PCR samples was calculated by the ΔΔCT method ,s before , 39. A rTo study the dependence on the number of replicates, six RNA-seq analysis methods were applied on some subsets of the training data set, from one lane to seven lanes. Results of these methods are shown in An RNA-seq data set for human colorectal cancer generated by Griffith et al.  was used2(fold-change) is >1.5  and <0.5, respectively, for the comparison between cells at a maturing zone (+4 cm above the leaf two ligule) and mature zone . All RNA-seq analysis tools were applied onto this RNA-seq data set, and these 37 genes were ranked by different methods. The distributions of top 10 genes ranked by different methods are shown in In contrast to other RNA-seq analysis tools, GExposer needs a training set, which was a set of RNA-seq data from human tissues for this work. Naturally, one may raise the question whether the performance of GExposer could have potential correlation with the species that is used to generate the RNA-seq data. To test whether the human-data trained GExposer works well on different species, the method was applied to plant (maize) RNA-seq data. The maize RNA-seq data set that compares bundle sheath and mesophyll cells were obtained from the laser-capture microdissected (LCM) samples from the tip of the maize leaf which incorporates two biological replicates . The RNA o2 mutant and K0326Y QPM, were used because it is the most important comparison for the QPM study. Each sample has about 20 million 50 bp long reads made up of kernels pooled from five biological replicates. Reads for maize were aligned against the reference genome (ZmB73-RefgenV2), for the pair-wise comparison between the genotypes, W64 o2 and K0326Y QPM. There is a portion of genes that are assigned with different expression levels when using or not using the local normalization method. We selected seven such genes that have been assigned different fold changes with and without local normalization and applied RT-PCR experiments to validate the real fold change. Since the gene annotations and sequences of B73 are used to design the primers for these genes in W64 o2 and K0326Y QPM, only four genes, GRMZM2G002678, GRMZM2G018193, GRMZM2G096719, and GRMZM2G38846, showed results for RT-PCR experiments. The details of the RT-PCR experiment and primers of these four genes are described in Supplementary Data . For these four genes, their log2(fold-change) measured using the local normalization method and not using it is shown in 2(fold-change) measured without the local normalization step shows higher values, while that measured by GExposer with the local normalization is no more than 0.61. The RT-PCR results support the GExposer analysis with the local normalization. The RT-PCR results of these four genes are shown in  o2 and K0326Y QPM. To understand what causes the difference, read coverage of three exons of GRMZM2G002678 are shown in  o2, but there is no peak in K0326Y QPM. Therefore, this peak artificially inflates the absolute value of fold changes between W64 o2 and K0326Y QPM. With the use of the local normalization method, the adjusted depth is low (at the level of the red dotted line), and this false positive is therefore removed.GExposer was also applied onto the maize RNA-seq data to test the performance of the local normalization method. The RNA-seq data were generated to study differentially expressed genes in quality protein maize (QPM) endosperm tissue . To simp o2 RNA-seq data from the maize endosperm data set as the template to conduct the simulation. For each transcript, the same number of reads as in the template data set was generated by the simulator. Then, all simulated reads were mapped to the same genome with the same parameters as for the W64 o2 RNA-seq data. The fractions of the discarded reads by the local normalization method of each transcript are shown in In order to further evaluate the effect of nonrandomly positioned reads, the real and simulated RNA-seq data sets were compared. The next-generation sequencing read simulator “ART”  was usedGExposer used three attributes: FC, ARPK, and relative GCC. To understand which one in the three scores plays a more important role, each attribute was removed from the system and the same training and test procedures were conducted to the training data set. The results are shown in  o2 data set from the QPM RNA-seq data is shown in The local normalization method mainly focuses on the high peaks of the reads; for a given high peak, its depth will be modified according to the average depth and its standard deviation. This raises one question: do the reads in these peaks have special patterns of nucleotides or GCCs, compared with the sequence background of all reads? If they have a special GCC, for example, some existing correction algorithms for GCC bias –23 couldIn the study, a new bias in RNA-seq data called nonrandomly positioned reads was identified. Our analysis shows that this bias is different from GCC bias. In order to reduce the bias, a local normalization algorithm has been developed and the false positive rate caused by this bias is reduced, which has been validated by the RT-PCR experiments. Moreover, the combination of three attributes, FC, ARPK, and GCC, can achieve better results; both false positive rates and false negative rates are reduced. However, GCC correction is only applied to a very small portion of genes in a whole genome. The model of GExposer was trained by one data set, and there is great potential for machine learning methods to improve the performance in finding DE genes by combining more training data sets from different species. On the other hand, training data from various species could potentially limit the ability of a naive Bayes classifier to identify DE genes.In the Supplementary Material, RT-PCR method used to validate gene expression in maize is described, and the primer sequences used for four maize genes are displayed.