ℍ0, of no difference in expression. A false discovery (type1 error) occurs when ℍ0 is incorrectly rejected. The quality of a detection algorithm is assessed by estimating its number of falsediscoveries, 𝔉. Work involving the second-moment modeling of the z-value histogram  hasshown significantly deleterious effects of intergene expression correlation on the estimate of 𝔉. This paper suggests that nonlineardependencies could likewise be important. With an applied emphasis, this paper extends the “moment framework” by includingthird-moment skewness corrections in an estimator of 𝔉. This estimator combines observed correlation (corrected for samplingfluctuations) with the information from easily identifiable null cases. Nonlinear-dependence modeling reduces the estimation errorrelative to that of linear estimation. Third-moment calculations involve empirical densities of 3 × 3 covariance matrices estimated using very few samples. The principle of entropy maximization is employed to connect estimated moments to 𝔉 inference. Model results are tested with BRCA and HIV data sets and with carefully constructed simulations.Microarray data are used to determine which genes are active in response to a changing cell environment. Genes are “discovered” when they are significantly differentially expressed in the microarray data collected under the differing conditions. In one prevalent approach, all genes are assumed to satisfy a null hypothesis,  This work is motivated by analytical challenges that arise in the use of microarray data to discover genes that are differentially expressed across experimental conditions such as control and treatment. Although the discussion centers around this genomics task, the developed methods are quite general and should be useful in other multiple-testing applications in which there is substantial dependence among test measures, and in which a small sample size may cause significant fluctuations in statistics employed in the testing. The specific aim of this work is to develop a reliable estimator of the The classic and contemporary literature in cell biology, and the more recent literature in genomics (both in print and posted on the Internet), is replete with tutorial information at all levels about cell anatomy and physiology, and genomics, as well as the microarray technology deployed in the present application. A good entry point for accessing information about contemporary developments in the genomics field is the web site of the US National Genome Research Institute . The papℍ0, that the gene is not differentially expressed across experimental conditions. All genes are initially assumed to satisfy ℍ0 in this analysis, and 𝔉 is estimated conservatively. This “all-null” presumption is consistent with this application in which ℍ0 is true for a vast majority of the genes in any experiment. Beyond the gene detection problem, however, this presumption is realistic in many practical applications of large-scale testing in which the prior probability of null cases, say π0, is large, and in which the goal is to identify a small set of interesting “nonnull” cases )𝔉. In principle, the proposed extension admits any order moment, but computational constraints limit the present developments. However, even the single step to a third-moment extension under severe sampling fluctuations is very challenging, and, in spite of this modest modeling enhancement, it is a hard-won extension yielding significantly improved estimates for a range of real and simulated examples  the number of false results. Empirical work below provides cogent evidence that accounting for intergene correlation alone does not sufficiently mitigate the adverse effects of dependency. Recent work by Hu et al.  has showples see .Thus, a central finding of this work is that null statistic histogram approaches can be improved by including third-moment skewness corrections. Advancing the techniques to model higher order dependencies is challenging, but the effort could have a substantial payoff. Errors in gene detection are expensive in financial terms, but the derailing of biomedical research resulting from a false gene discovery could be profoundly costly in many ways. Even modest improvements in genomic techniques are potentially very significant.M”), and statistical expectations. Precise formulations require that probability distributions ordinarily be denoted formally as, for example, pv1,v2 for the joint distribution of RVs v1 and v2, but the more common abusive notation “p” is more expedient in a few cases. The notation p(·) may denote either a discrete or continuous  distribution, depending, of course, on the RV(s) being modeled. The meaning should be clear in context. We deliberately allow this ambiguity because it avoids some notational awkwardness as discrete distributions are fitted with densities. On the other hand, the notation ℙ(A) is used to denote a probability assignment to a measurable event A.Because this paper has a practical aim, we will assume, without comment, “friendly” mathematical conditions such as existence of distributions, measurability, and sure convergence of integrals. Even so, the mathematical developments in this paper necessarily involve extensive notation and we strive for consistency and clarity in its use. Quantities are generally formulated as RVs unless stated otherwise. This excludes obviously deterministic quantities like sequence indices, integers defined in the abstract  element is φ. The correlation matrix of v isi, j) element is φ and S is a diagonal matrix with  element = σvi, the standard deviation of the ith RV, vi.Consider a random vector the term “correlation matrix” will refer to the definition in  correlation matrix. In this case the  element of the matrix is the scalar correlation ℰ{vivj}. In our definition the elements are correlation coefficients, which are, in fact, normalized covariances. One significant implication of this fact is that mean values of RVs have no effect on either matrix. This should be kept in mind in the developments to follow. In this paper, ition in . On the G genes are to be studied using M microarray experiments. The expression values are recorded in an G × M matrix, X = [xgm]. For analytical purposes, the expression quantities xgm are generally RVs. Each of the M microarray experiments takes place under one of two conditions (indexed by k = 1  or  2) such as control and treatment. These two subsets of the data are called treatment groups in the paper. There are Mk samples  in treatment group k. Based on the evidence in X, we seek to identify a “small” number, G∗ ≪ G, of genes that are significantly differentially expressed across the two treatment groups. One widely used strategy  is  is G genG∗ discovered genes is only meaningful to the extent that 𝔉 is very small. Of course, 𝔉 can only be estimated since the state of any gene  is unknown. Strong causal relationships among genes give rise to highly correlated zg scores and greatly complicate the estimate of 𝔉  for all g = 1,2,…, G.z scores are jointly Gaussian to the third order (but not uncorrelated).The xgm [element  of the expression matrix X] denotes (the RV model for) the expression of gene g on microarray m. Let xg• denote the marginal RV for xgm, that is, the model for the expression outcomes of gene g. The realizations of xg• are the elements of row g of an observed X. Then, it is assumed that φ = ρ ≈ ρ for all g, g′ (justified below).Recall that The following assumptions underlie these developments:p using the moments of the z-value histogram. We seek central moments because they facilitate working with the maximum entropy distribution . Then, the z-histogram bin counts areIb(zg) is the indicator function for the event “score zg falls in bin 𝒵b.” It is convenient and computationally efficient to obtain the moments of ribution . The momb with count Yb for some b ∈ {1,…, B}. The mean of count Yb issecond-order joint central moment covariance of the pair , where b may equal b′, iszg and zg′,  ≡ 𝒢 is the bivariate Gaussian density with mean vector 0 = [0 ⋯ 0]T, and covariance matrix ζ≝[ζ1 ζ2]T,ζ1 = zb][ and ζ2 = zb′]. Now, suppose that we can derive an empirical density , fitted to the discrete set of ρ, 1 ≤ g, g′ ≤ G}. This empirical distribution allows the summation over gene indices in , which is stated in the form of a lemma below. This result is similar to those of Owen ℓ≝G(G − 1)⋯(G − [ℓ − 1]),   for  1 ≤ ℓ ≤ G, andLet third joint central moment of a triplet , where two or more indices may be equal, is:δbb′b′′ is the Kronecker sequence over ℤ × ℤ × ℤ.Further, the z scores implies that the joint distribution for each score triplet is specified by a 3 × 3 covariance matrix. Let us denote the covariance  matrix for the z-value triplet  byz-score triplet, Rz is an element of the space—call it ℛ3—of all symmetric positive-semidefinite matrices in ℝ3  ×  3 with element magnitudes no greater than unity. Rz is continuously distributed over ℛ3.The assumed trivariate normality of the z scores. Let qRz(Ξ) be the empirical density of the Rz. This density must be inferred from the observed data. Of practical importance is the fact that, although each Rz is distributed over a subspace of ℝ3  ×  3, the domain of qRz is effectively a three-dimensional manifold of that space because each covariance matrix is unambiguously determined by its three values above or below its main diagonal . The argument Ξ may be thought of as a vector of these three elements , but we will continue to denote it as a matrix as a reminder of its association with Rz. We have, in conjunction with , th, thz scoYb, Yb′, Yb′′), where two or more indices may be equal, is approximated byQρz, G!]ℓ are defined in  it is simply necessary to combine the moments of the corresponding Yb counts. For example,To obtain the moments of  The key quantities in the lemmas for approximating moments are the empirical covariance densities. Obtaining these densities in the presence of severe sampling errors is discussed next.qρz(ξ) from the data. This density is then used to estimate qRz(Ξ). For this purpose, as well as to facilitate the calculations of Lemmas α is found to be sufficient. Because of severe sampling fluctuations, the current methods can recover only X are standardized to mean zero and unity variance. This standardization normalizes output “brightness” among microarrays ; sample covariance between rows (genes) g and g′ of Xo. For convenience, we define the notationqρx(ξ).Let τgg′ is the Fisher-transformed underlying correlation coefficient. Assuming a sampling modelpτ is the distribution of the Fisher-transformed underlying correlation coefficients, we can interpret the histogram of Fisher-transformed sample correlations, say the “pτ(ξ), the statistical correlation density on the scale resulting from the Fisher transform, and the histogram of sampling errors, say, the “ɛ-histogram,” also on the τ-scale. Then, the underlying pτ(ξ) is obtained by deconvolving this density from the convolved pair, 𝒢 fits nicely to the ɛ ~ 𝒢, the estimate of pτ(ξ), say 𝒢. It is this estimate that will serve as the empirical density of the Fisher-transformed correlations, To reduce the variability added by sampling errors, we apply the Fisher transform:operties , and eacalso see ), the noqτ(ξ), we must, in principle, undo the mapping (qρx(ξ), then deduce qρz from qρx. Recall, however, that we assume that the correlation coefficients of the z and xo variables are identical [  is the original notation for the 3 × 3 covariance matrix of a score triplet , and, by extension, . In the present discussion, Rx assumes the role of a 3 × 3 submatrix of Ro, namely, R3o.Finally, substituting n result , we obtaG, the assumption of the inverse-Wishart distribution in (qRx(Ξ) from qρx(ξ). There is no concern for a model of the entire matrix Ro = RXo. Further, single-parameter distributions on a positive definite matrix space are few. The inverse-Wishart distribution is chosen for its useful marginalization property. Of course, a tenuous assumption is not justified by a useful property if it leads to an unuseful procedure. The practical validation of the assumption is manifest in qRx is a worthwhile but challenging endeavor.For large ution in  is not wpRoκ(Ξκ | ν) only up to a proportionality constant. However, for qRx(Ξ)(κ = 3), normalization is straightforward. For κ ≥ 4 it is necessary to resort to Monte Carlo methods which only require densities up to a proportionality constant.Finally, we have derived http://www.egr.msu.edu/~deller/. The methods were tested on two real data sets, both showing a significant amount of intergene covariance but exactly opposite 𝔉 behaviors. Calculations below are for left-sided tail-area parameter δ = −2.5, and center area parameter c = 1 [recall  study of Hedenfalk et al. . These dssianity .G = 7680 genes over M = 8 microarrays with four samples assigned to an HIV infected condition and the remaining four to the control. To produce the test cells, the control cells (CD4 T cell lines) were infected by the HIV − 1BRU virus. The paper reports raw mRNA levels which, like the BRCA data, are converted to logarithms in the present work.In a study of the human immunodeficiency virus (HIV), Van 't Wout et al.  investigC, and the omnibus parameter, α. As is evident in α = 17.77, and for HIV, α = 3.51. The The present analysis reduces an entire expression matrix to two numbers: the center area null count, p per Lemmas G, α, c, δ, and Δ. We set Δ = 0.1. These estimated moments are used to find the maximum-entropy (maxent) distribution 𝔉 and C exhibit strong negative correlation of −0.89, a value similar to that in [𝔉 shows significant positive skewness, which causes C to exhibit negative skewness. This is not surprising as 𝔉 is bounded below by zero, and yet has small mean but inflated variance. The third-moment provides an additional level of detail about the joint behavior of 𝔉 and C.The next step is to compute the moments of  p^ .  Figure that in , Table 1G, the HIV X also exhibits more covariance. The BRCA optimization required 30 iterations, while HIV took ~70 iterations, to converge to an estimated distribution.During the maxent numerical optimization, a 100 × 500 equispaced mesh was found sufficient for the BRCA data; however, for the HIV data the resolution had to be increased to 400 × 2000. This is because, in addition to the larger p(𝔉∣𝒪), where 𝒪 refers to the observed data. Second- and third-moment estimates are shown separately. In the framework of statistical inference such a distribution is the ultimate goal, but this result could later be used for other purposes like computing point estimates and associated confidence intervals.𝔉, then for the BRCA data, third-moment calculations suggest 104 false discoveries versus 79 for the second-moment while the usual zg count falling in the left-sided tail-area is 116. For the HIV data, the third-moment analysis found eight false discoveries compared to 19 for second-moment, while the mean estimator zg count in the left-sided tail area is 46. Clearly, extensive analysis of intergene dependence can lead to very different conclusions from the same data, relative to those of the mean estimate of false discoveries .If the mean of the estimated z histogram.The second-order estimator designed by Efron  found 77ℙ(𝔉/G∗∣𝒪 ≥ γ) ≤ λ as a control measure, as recommended by Lehmann and Romano [γ and λ such that a fair comparison with other error measures is possible; however, for illustrative purposes, we set γ = 0.15 and λ = 0.5. With this constraint, the present approach reports 174 discoveries (108 for second-moment) for the HIV X. This compares favorably with the results of Efron [We note in passing that the availability of d Romano . It is nof Efron , where tz-values we take the usual two-sample t-statistic.Insight is gained by testing the approach on simulated data for which the “correct answer” is known. In the studies below, all cases are null . The goal is to see how well the realized left-sided tail-area count can be estimated from the center count. To maintain realism, we simulate raw mRNA levels. The testing scenario is a “two-group study,” so en route to xgm, of gene g measured by microarray m, be distributed as the Gamma density: For m = 1,…, M,κ) = ∫0∞vκ−1ev−dv is the Gamma function. κ and θ are called the shape parameter and the scale parameter of the distribution, respectively. This distribution is similar to the Gamma-Gamma model used by Newton et al. [κ is common to all genes, while the scale parameters {θg}g=1G characterize varying mRNA levels from gene to gene, but are assumed i.i.d. asgth gene is κθg and variance is κθg2.Let the mRNA expression level, n et al. . In 34)xgm, of κ, κ0, θ0) in  in  and 35)κ, κ0, θG × M) matrix, say Z, of i.i.d. unit normal RVs was used to produce a correlated matrix, Zc  , via the mappingL is the lower-triangular Cholesky factor , then further transformed to simulated expression variables, xgm, through the inverse Gamma c.d.f. as in  of Rz,t method , yieldedf. as in . The resδ = −2.0 and δ = −2.5, respectively. In both cases, c = 1 for the center area , 800 matrices X were processed. On each X the approach was applied in its entirety and no additional knowledge was assumed. The a posteriori mean was used as the final estimate. The usual mean estimator δ = −2.5 and 73 for δ = −2.0, regardless of the particular X.Figures area see . For eacκ, κ0, θ0), the third-moment skewness corrections make the estimation process more accurate. For some of the scenarios third-moment estimates saturate somewhat, but the effect is minor compared to that in the second-order approaches. To the extent that these parameter sets cover a wide range of realistic gene expression data, the practical utility of the proposed approach is evident.Strikingly, for all three parameter sets , then estimate the moments. However, due to sampling fluctuations, the underlying covariance matrix ΣzG×G is ordinarily unattainable; however, pursuing quantities like qρz(ξ) and qRz(ξ) is still possible. Also, as G gets larger  computational demands, as well as the large number of z-score sets required, become prohibitive.It is reasonable to question the necessity of the heavy mathematical machinery of the foregoing sections since it is possible to simulate a number of sets of Permutation calculations, as in [α.When a direct extraction of inter-hypotheses covariance is not feasible, single omnibus parameter models remain useful in that the investigator can still use judgment to intelligently incorporate some form of covariance effect by setting a value of the parameter p resides over support domain 𝒟 as shown in The distribution of interest c, the center area boundary. The selection of c = 1 in this paper is based on the first eigenvector analysis of Efron [𝒵 space.Apart from the numerical parameters Δ (bin width) and the mesh resolution in maxent, the only open choice of parameterization in the present method is of Efron  which suzg covariance does not translate into more extreme covariance between variables 𝔉 and C. In the BRCA data, for example, the coefficient between 𝔉 and C is −0.89, while for the HIV data the covariance drops to −0.75. Further research to gain insight into this behavior would be very useful.One surprising result of this and similar studies is that more inter-𝔉, inter-zg covariance can, in fact, be exploited to increase power by finding a superior ranking of potential discoveries. The recent multiple testing literature has begun to address this possibility.Finally, while covariance was viewed in the present paper as a “destructive factor” in the attempt to estimate  Human subject data used in this study are publicly available and anonymous and are, therefore, exempted from continuing Internal Review Board scrutiny according to US Health and Human Services Policy 45 CFR 36, Subpart A, §46.101 (2.b.4).


The big data moniker is nowhere better deserved than to describe the ever-increasing prodigiousness and complexity of biological and medical datasets. New methods are needed to generate and test hypotheses, foster biological interpretation, and build validated predictors. Although multivariate techniques such as cluster analysis may allow researchers to identify groups, or clusters, of related variables, the accuracies and effectiveness of traditional clustering methods diminish for large and hyper dimensional datasets. Topic modeling is an active research field in machine learning and has been mainly used as an analytical tool to structure large textual corpora for data mining. Its ability to reduce high dimensionality to a small number of latent variables makes it suitable as a means for clustering or overcoming clustering difficulties in large biological and medical datasets.Salmonella pulsed-field gel electrophoresis (PFGE) dataset, lung cancer dataset, and breast cancer dataset, which represent various types of large biological or medical datasets. All three various methods are shown to improve the efficacy/effectiveness of clustering results on the three datasets in comparison to traditional methods. A preferable cluster analysis method emerged for each of the three datasets on the basis of replicating known biological truths.In this study, three topic model-derived clustering methods, highest probable topic assignment, feature selection and feature extraction, are proposed and tested on the cluster analysis of three large datasets: Topic modeling could be advantageously applied to the large datasets of biological or medical research. The three proposed topic model-derived clustering methods, highest probable topic assignment, feature selection and feature extraction, yield clustering improvements for the three different data types. Clusters more efficaciously represent truthful groupings and subgroupings in the data than traditional methods, suggesting that topic model-based methods could provide an analytic advancement in the analysis of large biological or medical datasets. The fingerprint of an isolate is characterized by the presence or absence at designated band locations in PFGE analysis. Classification models were developed to characterize and identify serotype of isolates in outbreak investigation from the analysis of PFGE fingerprint [Recent advances in biotechnology have generated massive amounts of biological and medical data for disease diagnosis/prognosis, unknown compound toxicity prediction, and pathogen identification in outbreak investigation, etc. Identification of pattern and structure among a large number of samples and/or the associated variables requires the development of more powerful statistical methods and data mining techniques. For example, genomic microarray and proteomic technologies are often used to identify genes and proteins that have similar functionality for understanding biological processes or identifying new biomarkers for targeted therapy, etc. -6. Data athogens -9. Pulsegerprint . The FDAgerprint -12.Salmonella or other foodborne pathogens [A dataset can be expressed in a two-way data matrix with rows representing samples and columns representing the measured variables that characterize the corresponding samples. A large dataset may have a large number of samples, such as the PFGE dataset of athogens ,9; or a athogens ,14. The k-means [K-means is the most commonly used non-hierarchical clustering algorithm. In k-means clustering procedures, samples are divided into k partitions or clusters based on a measure of similarity. Unlike the hierarchical clustering, the number of clusters in a k-means analysis must be specified a priori. Simulation studies have shown that k-means algorithms and other non-hierarchical clustering algorithms perform poorly when random initial seeds are used; their performance is improved when the results from hierarchical methods are used to form the initial partition [Cluster analysis is a commonly used data mining technique to explore the relationships among attributes, samples and the relationships between attributes and samples. Clustering algorithms assign samples or attributes to clusters based on their similarity. Cluster analysis can be used as a preliminary method for classification or for finding new classes. Hierarchical clustering tree (HCT)  and k-mek-means  are the artition . Thus, hTopic modeling algorithms are statistical methods that analyze the words of documents to discover the themes that pervade a large collection of documents . The basIn this study, we proposed to apply LDA topic modeling for cluster analysis of large datasets. Three different datasets were selected to represent various types of large biological or medical datasets. These large datasets were transformed into the files of documents on which the LDA algorithm was run and two matrices were generated for each dataset. Three different cluster analysis methods were then applied on the topic model-derived data matrixes of the three datasets, and the most accurate method for each type of dataset was determined. The applications of the topic model on various large datasets provide new approaches to improve the accuracy and efficacy of the subgroup identification and data mining.Salmonella PFGE genotyping data from CDC [Salmonella outbreak-related isolates. The dataset contained 20 most common Salmonella serotypes and about 2,000 isolates for each of 20 serotypes. Each profile used 1/ 0 to represent the presence/absence of the electrophoresis bands, and each of 41,232 profiles was nominated to have 60 bands in the dataset. As a standard typing method used in Salmonella outbreak investigations, PFGE has been used by many laboratories to determine strain relatedness and confirm an outbreak of a bacterial disease by comparing the band profiles [Three large data sets were utilized to evaluate the proposed approaches in this study. The first dataset was the from CDC ,9. It inprofiles ,9. The sThe second dataset was the public lung cancer microarray dataset originally collected from the Gene Expression Omnibus ,26. The The third dataset was the breast cancer microarray dataset originally collected by van 't Veer et al. ; there wSalmonella dataset, the PFGE bands were viewed as the words. Each isolate had its corresponding document consisting of the bands present, which had value 1 in the PFGE dataset. After the data preprocessing, the corpus of the Salmonella dataset contained 41,232 documents, where each document contained at most 60 words.In this step, each isolate/sample was transformed into one document and all documents constituted one corpus. For the In both of lung and breast cancer microarray datasets, the expression value for each variable (gene) was normalized to 0  or 1  based on its median value. Each sample was transformed into one document. The variables with value 1 were considered as the words in the documents. The final corpus of the lung cancer dataset contained 111 documents and each document contained at most 54,613 words. The final corpus of the breast cancer dataset contained 96 documents and each document contained at most 21,907 words.For a given dataset, topic modeling with LDA is utilized to model the relationships between samples and variables. LDA assumes that the dataset is generated by the following process :k φ for each topic from a Dirichlet distribution with hyper parameter β;1. Pick a Multinomial distribution s θ for each sample from a Dirichlet distribution with parameter α;2. Pick a Multinomial distribution z from a Multinomial distribution with hyper parameter sθ.a) Pick a topic n w from a Multinomial distribution with parameter zφ.b) Pick a word D = {D1,..., SD} is formalized asBased on the generative process above, the probability of a given dataset θ  /absence  of all the sample isolates at 60 designated band locations was calculated as the characteristic mean of the corresponding cluster. Then, the complete-link hierarchical clustering algorithm was applied on the Euclidean distance measures to investigate the relationships among the 30 clusters.For the breast cancer dataset, based on the obtained clusters (groups) and survival time information of the samples, the survival package in R was utilized for survival analysis. Specifically, function "survfit" was used to generate the Kaplan-Meier curves  for the Z' denotes the cluster assignments of instances in obtained clustering result, and random variable Z denotes the true class labels, then NMI is defined as follows:Normalized mutual information  was utilI = H(Z) - H(Z|Z') is the mutual information between the random variables Z' and Z, H(Z) is the Shannon entropy of Z, and H(Z|Z') is the conditional entropy of Z given Z' [Where given Z' . The ranIn this study, three large datasets representing different types of large biological or medical were selected to illustrate the applications of topic modeling for cluster analysis. The LDA algorithm transformed the original datasets into the files of documents and generated two matrices for each of the three datasets. Three different topic model-derived clustering methods were applied to the LDA-derived matrices from the three large datasets. After the result comparison (data not shown here), the best-fitting cluster analysis method was selected on the basis of the most biological accuracies for each dataset.Topic model-derived clustering based on highest topic assignment.Salmonella PFGE dataset, as compared to the other two topic model-derived clustering methods . In the resultant Figure Topic model-derived clustering based on feature selection.K-means was then applied to the 100 features for the clustering analysis. The results were compared with two conventional methods, k-means and PCA [k-means algorithm was directly applied to the original 54,613 continuous-valued variables, while the PCA method [k-means algorithm. Table k set as 2, 3 and 4.The topic model-derived clustering based on selection of the highest probability features emerged as the best fitting method for the lung cancer dataset after comparison with the other two methods Table . In this and PCA . The conA method  was firsk set to 4 in k-means, our proposed method gives the best cluster results with only 18 samples misclassified.The fourth and fifth columns in Table Topic model-derived clustering based on feature extraction.k-means was then applied to the 10 derived features.The 96 patients from the breast cancer dataset were best clustered using the method based on feature extraction. The topic number was set as 10 for feature extraction; the k = 3, the 96 breast cancer patients were clustered into three groups, G1 (n = 20), G2 (n = 36), and G3 (n = 40). The Kaplan-Meier survival curves of the three clusters were shown in Figure Survival analysis was conducted to evaluate the clustering results. For k = 2, the 96 patients were clustered into G1 (n = 24) and G2 (n = 72). The patients in G3 from the 3-means cluster analysis  performed all the calculations and data analysis, and wrote the first draft of the manuscript. WZ and JC developed the methods, had the original idea, and guided the data analysis and presentation of results. WZ and JC collected and generated the data. All authors contributed to data verification, approach evaluation, and assisted with writing the manuscript. All authors read and approved the final manuscript.The findings and conclusions in this article have not been formally disseminated by the US Food and Drug Administration (FDA) and should not be construed to represent the FDA determination or policy.Click here for file


RUVnormalize.When dealing with large scale gene expression studies, observations are commonlycontaminated by sources of unwanted variation such as platforms or batches. Not takingthis unwanted variation into account when analyzing the data can lead to spuriousassociations and to missing important signals. When the analysis is unsupervised, e.g.when the goal is to cluster the samples or to build a corrected version of the dataset—asopposed to the study of an observed factor of interest—taking unwanted variation intoaccount can become a difficult task. The factors driving unwanted variation may becorrelated with the unobserved factor of interest, so that correcting for the former canremove the latter if not done carefully. We show how negative control genes and replicatesamples can be used to estimate unwanted variation in gene expression, and discuss howthis information can be used to correct the expression data. The proposed methods are thenevaluated on synthetic data and three gene expression datasets. They generally manage toremove unwanted variation without losing the signal of interest and compare favorably tostate-of-the-art corrections. All proposed methods are implemented in the bioconductorpackage  Such large gene expression studies are often carried out over several years, mayinvolve several hospitals or research centers and typically contain some unwantedvariation. Sources of unwanted variation can be technical elements such asbatches, different platforms or laboratories, or any biological signal which is not thefactor of interest of the study such as heterogeneity in ages or different ethnicgroups.Over the last few years, microarray-based gene expression studies involving a large numberof samples have been conducted , with thUnwanted variation can easily lead to spurious associations. For example when one islooking for genes which are differentially expressed between two subtypes of cancer, theobserved differential expression of some genes could actually be caused by differencesbetween laboratories if laboratories are partially confounded with subtypes. When doingclustering to identify new subgroups of the disease, one may actually identify some of theunwanted factors if their effects on gene expression are stronger than the subgroup effect.If one is interested in predicting prognosis, one may actually end up predicting whether thesample was collected at the beginning or at the end of the study because better prognosispatients were accepted at the end of the study. In this case, the classifier obtained wouldhave little value for predicting the prognosis of new patients.Similar problems arise when trying to combine several smaller studies rather than workingon one large heterogeneous study: in a dataset resulting from the merging of several studiesthe strongest effect one can observe is generally related to the membership of samples todifferent studies. A very important objective is therefore to remove this unwanted variationwithout losing the variation of interest.A large number of methods have been proposed to tackle this problem, mostly using linearmodels. When both the factor of interest and the unwanted factors are observed, the problemessentially boils down to a linear regression. ComBat , or usinIn this paper, we focus on this latter case where there is no predefined factor ofinterest. This situation arises when performing unsupervised estimation tasks such asclustering or principal component analysis (PCA), in the presence of unwanted variation. Itcan also be the case that one needs to normalize a dataset without knowing which factors ofinterest will be studied. Our objective is to correct the gene expression by estimating andremoving the unwanted variation, without removing the—unobserved—variation of interest.The recent work of Section 2.The removal of unwanted variation (RUV) model used by 2.1naiveRUV-2 in If We now discuss how a random rsion of  could imThe naive RUV-2 estimator of or model  if \docuThe only difference with the naive RUV-2 estimator  is the \ty term:  is a rid whereas  is an or2.2k-meansalgorithm over Assuming some structure is known on the unobserved cument}:\documentcument}:\documentThe objective of this joint modeling can be 2-fold: one may still just want to estimateA joint solution for able for . A possites than  where \dFinally, such a joint scheme can be used to build a different estimator of squares  used in 3.We now introduce an alternative estimator of the unwanted variationentclass1pt{minimaentclass1pt{minimaIn practice, one way of obtaining such control samples is to use replicate samples, i.e.,samples that come from the same tissue but which were hybridized in two different settings,say across time or platform. The profile formed by the difference of two such replicatesshould only be influenced by unwanted variation—those whose levels differ between the tworeplicates. In particular, the More generally when there are more than two replicates, one may take all pairwisedifferences or the differences between each replicate and the average of the otherreplicates. We will denote by Such samples can then be used to estimate  the way used conUse the rows of Using Once The extreme case where all genes are used as control genes is of interest, and is discussedin Section 10 of the Finally, this replicate-based correction yields an estimator entclass1pt{minima4.We start with a set of experiments on synthetic data, where we generate the data ourselves.In this case, we are able to measure how well each correction method recovers the truematrix 4.1We generate data according to model . \documeThe columns of ng model . The\doWe compare the performances of ComBat (Independent), Confounded),and Moderate). The normalizedreconstruction error is measured by Table 4.2In the case, where The random effect model yields similar performances as naive RUV-2, slightly worsebecause it uses The replicate-based method obtains a performance similar to that of ComBat.Interestingly, its results are slightly improved by using non-control genes, see Section10 of the Confounded), QN, ComBat and naiveRUV-2 lead to the same reconstruction error as the uncorrected matrix. They actually failfor opposite reasons: the unwanted variation along When Finally, as expected from the discussion of Section 11 of The last column illustrates an intermediate case with moderate confounding. Random effectstill works much better than uncorrected, QN, ComBat and naive RUV-2, and is less affectedby the use of non-control genes. It is worth noting that even if We now summarize our observations on these synthetic experiments. ComBat performs well toremove an observed batch if it is largely independent from the signal of interest. NaiveRUV-2 performs well to remove a batch—observed or not—if it is given good control genesand if the batch is not too associated with the factor of interest. The random effectmodel performs like naive RUV-2 but is much less affected by confounding and poor controlgenes. Iterating the estimation of 5.On real data, we have no way to measure how close we are to the trueThis section presents results on the microarray gene expression datasets studied in 5.1For each of the correction methods that we evaluate, we apply the correction method tothe expression matrix We evaluate two basic correction methods: the replicate-based procedure described inSection Some of the methods we compare require the user to choose some hyperparameters: the ranksNo single rule can be therefore given as to hyperparameter choice and judgment isnecessary each time adjustments are performed without a specified factor of interest. Wesuggest using relative log expression (RLE) plots, clustering with respect to a knownfactor of interest, or known differentially expressed genes with respect to a known factorof interest as positive controls. In this experiment, we compare normalization methodsbased on how well they allow clustering to recover the gender partition, so it would notmake sense to use the same criterion to choose eriasee .Since 5.2This gender study is an interesting benchmark for methods aiming at removing unwantedvariation as it expected to be affected by several technical and biological factors: twomicroarray platforms, three different labs, three tissue localizations in the brain. Mostof the 10 patients involved in the study had samples taken from the anterior cingulatecortex (a), the dorsolateral prefontal cortex (d), and the cerebellar hemisphere (c). Mostof these samples were sent to three independent labs: UC Irvine (I), UC Davis (D) andUniversity of Michigan, Ann Arbor (M).Gene expression was measured using either HGU-95A or HGU-95Av2 Affymetrix arrays with12 600 genes shared between the two platforms. Six of the For the replicate-based method of Section Since most genes function irrespective of gender, clustering by gender gives betterresults in general when removing genes with low variance before clustering. For eachmethod, we therefore apply clustering after filtering different numbers of genes based ontheir variance after correction.Figure entclass1pt{minimaThe gray line of Figure The purple lines of Figure The iterative version in dotted line leads to much better clustering except again whenvery few genes are selected. Figure The green lines of Figure 6.We introduced methods to estimate and remove unobserved unwanted variation from geneexpression data when the factor of interest is also unobserved. One method uses the negativecontrol gene-based estimator of unwanted factors introduced in We systematically compared the proposed correction techniques with state-of-the-art methodson both synthetic and real gene expression data. On synthetic data, we knew what the correctsignal was, and could measure how well each correction method recovered this signal. Whengood control genes were available, the random effect estimator performed much better thanexisting correction methods in the presence of confounding. The replicate-based methodperformed less well than the control gene based one—unless a really large number ofreplicates was available—but was unaffected by poor quality control genes and to largeconfounding level. We were able to verify that both proposed methods provide a bettercorrection even in the case where the factor of interest and the unwanted factors aretotally confounded.On real gene expression data where it did not make sense to define a single correct signalto be recovered, we assessed how well we were able to rediscover by clustering a knownfactor of interest which was unspecified at correction time. Here again, the proposedmethods lead to better reconstruction than existing corrections.Assessing how well each unsupervised correction method works on a new real dataset isproblematic, since the factor of interest is not observed. Clustering with respect to aknown biological factor, like we do with gender, is one option to perform this assessment.Other options include using positive control genes and RLE plots, like we do in the To conclude, our results suggest that it is possible to remove unwanted variation from geneexpression without losing the signal of interest, provided enough controls are available:negative control genes which are affected by the unwanted factors only, or replicatesamples. Together with other researchers in our groups we have also started applying some ofthe methods that we introduce here to RNA-Seq (This work was funded by the SU2C-AACR-DT0409 grant. Funding to pay the Open Accesspublication charges for this article was provided by Australian National Health and MedicalResearch Council Program Grant APP1054618.kxv026_Supplementary_MaterialClick here for additional data file.


Sequencing datasets consist of a finite number of reads which map to specific regions of a reference genome. Most effort in modeling these datasets focuses on the detection of univariate differentially expressed genes. However, for classification, we must consider multiple genes and their interactions.Thus, we introduce a hierarchical multivariate Poisson model (MP) and the associated optimal Bayesian classifier (OBC) for classifying samples using sequencing data. Lacking closed-form solutions, we employ a Monte Carlo Markov Chain (MCMC) approach to perform classification. We demonstrate superior or equivalent classification performance compared to typical classifiers for two synthetic datasets and over a range of classification problem difficulties. We also introduce the Bayesian minimum mean squared error (MMSE) conditional error estimator and demonstrate its computation over the feature space. In addition, we demonstrate superior or leading class performance over an RNA-Seq dataset containing two lung cancer tumor types from The Cancer Genome Atlas (TCGA).http://bit.ly/1gimnss.Through model-based, optimal Bayesian classification, we demonstrate superior classification performance for both synthetic and real RNA-Seq datasets. A tutorial video and Python source code is available under an open source license at The online version of this article (doi:10.1186/s12859-014-0401-3) contains supplementary material, which is available to authorized users. The possibility of genomic phenotype classification arose with the inception of gene-expression microarrays. From the outset, two fundamental problems have frustrated the endeavor: 1) the inaccuracy of microarray measurements, and (2) small samples. Our particular application of interest is classification using RNA-Seq data. Modern RNA-Seq technologies sequence small RNA fragments (mRNA) to measure gene expression, where the number of reads mapped to a gene on the reference genome defines the count data. Given that RNA-Seq data has advantages over microarray data, in particular, more accurate measurement, we still confront the second fundamental problem, which is statistical, not technological: small samples cause re-sampling-based classifier error estimators to be very inaccurate due to excessive variance and lack of regression with the true error  the inac. Since teffective class conditional distributions, which are expectations relative to the posterior distribution of the class-conditional distributions. The closed-form solutions depend on particular models  and the existence of conjugate priors, which can be too constraining for practical applications such as RNA-Seq classification.The situation has been addressed by utilizing prior knowledge via a Bayesian approach that considers a prior distribution on an uncertainty class of feature-label distributions ,7. For eMuch of the statistical literature concerning classification of RNA-Seq data attempts to address differential expression testing, that is, univariate statistical testing on an individual gene basis. These attempts typically model RNA-Seq data via negative binomial ,15 and PIn this paper, we focus on modeling the pipeline that starts with extracting the gene concentrations from the biological samples and their subsequent processing by the sequencing instrument . This isWhereas Dalton and Dougherty have presented a computational method for nonlinear classifiers in the Gaussian model , this stp(X) as the probability measure over the random variable X. p(X) may be a probability mass function, probability density function, or arbitrary probability measure. p(x|y) denotes the conditional probability p(X=x|Y=y). Similarly, following Bayesian convention, we write parameterized distributions by conditioning on the parameter, for instance, p, and posterior expectations by conditioning on the sample, such as E, where Sn and all other values are defined in Section ‘p(θ|Sn).Throughout, we use capital letters to indicate random variables, lower case letters to indicate individual realizations of random variables or indices, bold latin characters for observed vectors, and Greek letters for latent features and parameters. We write n labeled training data points, y∈{0,1} is the class label and . An example of binary classification in a clinical setting might include class 0 and 1 being two types of cancers, or normal and cancerous tissues. Available features would then be the gene or genes that will eventually be used in the designed classifier to assign this label. The feature space  would be the set of possible gene expression measurements for all genes in the feature vector. The labeled training data Sn would be the set of gene expression measurements from samples which had undergone further testing  to identify the type or malignancy of the tissue. Using Sn, we design a classifier ψ that hopefully performs well on the unknown joint feature-label distribution p. In the same clinical example, the classifier ψ could then identify the type of cancer using gene expression measurements alone.Binary classification considers a set of By parameterizing this unknown joint distribution in a model-based Bayesian framework one can derive an optimal Bayesian classifier (OBC) that minimizes the expected error over the space of all classifiers under assumed forms of the class-conditional densities. Specifically, under Gaussian and multinomial class-conditional densities and their corresponding conjugate prior distributions, closed-form solutions for the OBC ,11 and tc and the class-conditional densities p, where a particular value θy∈Θy specifies a single class-conditional density contained in the class of densities defined over the space Θy, which will be a Cartesian product as described in Section ‘θ=∈Θ=×Θ0×Θ1. In the Bayesian classification framework, these values are then treated as random variables, so that we may consider quantities such as the expectation of c, or another random variable conditioned on the value of the parameter vector θ.The parameterization of the feature-label distribution consists of the marginal class probability c and the class-conditional parameters θy. In addition, the posterior of c is assumed known throughout the tree. Figure Figure p(θ|Sn) is the posterior distribution of the parameterized feature-label distribution – posterior to the labeled samples in Sn. Typically, error estimates and the optimal classifier are our primary interest, so that this posterior distribution is traditionally used as a means to compute other quantities and is not of interest by itself.We now examine the tree in more detail. Starting at the far left of the tree, effective class-conditional density is the marginal predictive posterior of the feature vector X conditioned Sn and the class variable Y, The Θy given a class y. The weights in this expectation are the posterior, p(θy|Sn), evaluated at each θy.It gives the distribution of the feature vector using a weighted average over all the parameterized class-conditional densities in ψ is ε=p(ψ(X)≠Y). Given the sample data Sn, ε is a random unknown quantity in the Bayesian framework. The MMSE estimate given in : . We then consider the mean μ and covariance Σ of the gene concentrations as independent quantities for each class y.where  process . For eacy is parameterized by θy=, where d= and λ=,i=1,2,…n,j=1,2,…,D, for n sample points and D total genes. Therefore, θ=, where c=p(Y=0), the prior probability for class 0.The entire MP model is represented in Figure μy is distributed according to a univariate Gaussian. Unless otherwise stated, η is the D dimensional zero vector, ν2=25, κ=10, and S=(κ−1−D)ID. For computational and identifiability reasons, d is fixed to be a vector of normalization constants in order to match the different sequencing depths across all the samples. In practice, d can be approximated by an upper quartile normalization, which has been shown to be effective T andSince designed classifiers typically use very few of the totality of observed genes, only a small fraction of the data is used for classifier design. Similarly to , we can σ2>0, and −1≤ρ≤1. For each class, we need to determine values for five scalar quantities: m,ν2,σ2,ρ, and κ.where μ and Σ. Typically, the number of discarded features F is much larger than the dimensionality D of the classification problem. Therefore, due to computation time, we uniformly sample Fs pairs of features from F and average the resulting runs rather than using all or large groups of discarded features in a single MCMC run. We use the following procedure : Obtain MCMC samples using the feature pair as data and flat priors.μ and Σ.Record posterior averages of For each randomly chosen discarded feature pair (Average over these posterior averages as given by Equations - 19)..19).Using the resulting five hyperparameter estimates, run the final MCMC for classification.Due to the hierarchical design of the MP model, we cannot apply the method of moments in a direct fashion, as did . InsteadFollowing , we use which together with our simplified covariance structure implies κ via The variance of the first diagonal of an inverse-Wishart matrix can be used to solve for μ directly from our posterior, we obtain As we have samples of Fs discarded feature pairs. For the i-th feature pair we obtain the posterior means In order to use Equations -14), we, we14), We substitute the estimates from Equations -19) bac bac19) bOne must keep in mind that the calibration procedure explicitly assumes the MP model. Hence, one can only expect an improvement in the classification performance if the data follow the MP model.To obtain the MP OBC, we approximate the effective class conditional densities in order to minimize the expected error in a pointwise fashion: M samples of θy from the model posterior distributions.where y, and we assume a single class. We do this because the computation can be performed per-class due to the assumed independence between the classes and the marginal probability, p=p(c)p(θ0)p(θ1).For clarity of presentation, we do not consider the class variable θ using the Metropolis Hastings MCMC algorithm we define a proposal distribution p(θ′|θ) to obtain a new value for the class parameters θ′ from the old values θ. We then calculate the acceptance ratio p(θ′|θ)=p(θ|θ′)). The process of proposing and accepting samples from this distribution with the probability R induces a Markov chain. Positivity of the proposal distribution (p(θ′|θ)>0 for any θ) is a sufficient condition for ergodicity of this Markov chain. Furthermore, this Markov chain admits a steady-state distribution equal to our desired posterior distribution p(θ|Sn) [To obtain posterior samples of  p(θ|Sn) .p(xi|θ)=p(xi|λi) owing to conditional independence. From the definition of the prior, From the definition of the likelihood, p(xk|λk)∼Poisson(dk exp(λk)), λ∼ Normal , λg)ID. To examine the effects of feature correlation in IC datasets, we can also generate low-correlation covariance matrices by zeroing the off-diagonal terms. Once the covariance matrix for class y is obtained, location parameters for gene-expression values for each sample point are drawn from the respective multivariate normal distribution λy∼N. Each sample point is then assumed to be normalized through an upper quartile or other suitable method, but in practice any sample-based normalization is imperfect. We reflect this variation by drawing the sequencing depth di from a Uniform distribution, giving the rate of the Poisson process as di expλi. The number of reads for a single gene from a single sample is then drawn from this Poisson distribution. See Additional file Hence, we modify the synthetic data generation procedure in an attempt to produce synthetic datasets more representative of such nonlinear phenomena in biology. In this modified procedure, we allow independent covariance (IC) matrices for the features of the two classes. To generate these covariance matrices, mentclass2pt{minimThe OBC is optimal on average across the space of distributions determined by its prior distributions. To avoid biasing the performance comparison, we draw the classification problem datasets using different distributions than those of the OBC priors. See Additional file c and therefore c must be known in advance. Based upon records from 2006-2010, we have a very accurate estimate, 48,600/141,300≈0.34 [c directly, along with all of the data, in designing the OBC, for classification rules that do not use c explicitly, the separately sampled data must be maximally subsampled to the proper sampling ratio c before applying the classification rule [Ntrn desired samples, the sample subsets will contain round((1−c)Ntrn) and round(cNtrn) for class 0 and 1, respectively. Moreover, holdout error estimation, which we use here, must be properly adapted for separate sampling for all design methods, including the OBC. The holdout estimate is given by We consider a real RNA-Seq dataset composed of level 3, RNASeqV2 data from the Cancer Genome Atlas (TCGA) project. It contains 484 and 470 specimens from lung adenocarcinoma and lung squamous cell carcinoma tumor biopsies, respectively. The samples are mapped read counts against 20531 known human RNA transcripts as generated by the University of North Carolina at Chapel Hill, one of the Genome Sequencing Centers for the TCGA. The data for each cancer type is the result of processing approximately 20 billion reads and the read count files for each are one gigabyte apiece. The problem is to classify the tumor types. Because the class-0 (lung adenocarcinoma) and class-1 (lung squamous cell carcinoma) sample sizes, 484 and 470, are not chosen randomly, we are confronted with the problem of separate sampling. This means that there is no way to obtain a posterior distribution for 300≈0.34 . Whereasion rule . This meectively . We noteectively .While averaging over sample subsets for holdout error estimation, we also average over uniformly, randomly selected gene subsets of size 4. This sampling occurs from low (1-10 average reads per gene) expression genes. We sample from these lower expression genes because we are ultimately interested in classification problems where the delineation between phenotypes is determined by genes with low expression. We used 10,000 for averaging in order to obtain a large enough sample over this feature and sample subset space to achieve repeatable results (data not shown). Computational runtime for each sample and gene subset was similar to the synthetic data.The Additional file To evaluate classification performance, classifiers were trained using 3NN, LDA, and c-support vector machines with a radial basis function kernel . StartinIn the case of independent-covariance data with highly correlated features, Figure The superior performance of the OBC relative to LDA, 3NN, and SVM in Figure σy,ρ} and {S,κ} for the HC and IC cases, respectively, and the particular choices made in Section ‘The larger overall classification errors in Figure μ0=0.0 in Figure μ1 from 0.0 to 1.0 to make the classification problem harder and easier, respectively. Across this range of classification problems, the MP OBC had better classification performance than the other classification methods. In addition, calibrated priors improved performance further, especially for harder classification problems.Still using independent-covariance data, we fixed the mean of class 0 at In Table In Figure O(D3), where D is the number of features. In practice, one would have a small number of data sets and could use parallel computing to devote more time and computing effort for the classification. For example, in timescales on the order of hours on a typical workstation, we have successfully performed classification using 50 genes.The results in Figure The other classification methods compared in this study have smaller computational requirements and can correspondingly handle larger numbers of features given the same available resources. However, for the small sample sizes often available in biology, 50 genes is typically beyond the “peaking” point where most classifiers decrease in classification performance as more features are added (for a fixed number of training samples) . IncidenIn addition, the computational time requirements of classification is typically not a bottleneck in translational medicine given the timescales used in collecting biological data. In these settings, the accuracy of classification is much more valuable than rapid runtimes, and this is the primary advantage of the computational OBC framework proposed in this paper.ipso facto, the MCMC approach to optimal Bayesian classification can be applied, via [We have demonstrated that Bayesian classification can be applied to specific problem domains such as RNA-Seq through statistical modeling and MCMC computation. The resulting classifier provides superior classification performance compared to state-of-the-art classifiers such as SVM with a radial basis kernel. Although we have not discussed error estimation – our interest in the present paper being classification, ied, via ,7 and [1ied, via ,13, to od and determining if additional performance improvements can be made by considering the distribution over d rather than transforming the original data through the process of data normalization. Additionally, more efficient computational techniques could be used to allow for larger feature sizes, including program optimization and utilizing structure in the feature covariance to reduce the size of the parameter space.Future work includes examining the normalization parameter Additional file 1Supplementary Materials. Algorithms and Model Diagnostics. Supporting details including in-depth algorithms and model diagnostic plots and figures are given in a single multi-page PDF.


Several missing value imputation methods for gene expression data have been proposed in the literature. In the past few years, researchers have been putting a great deal of effort into presenting systematic evaluations of the different imputation algorithms. Initially, most algorithms were assessed with an emphasis on the accuracy of the imputation, using metrics such as the root mean squared error. However, it has become clear that the success of the estimation of the expression value should be evaluated in more practical terms as well. One can consider, for example, the ability of the method to preserve the significant genes in the dataset, or its discriminative/predictive power for classification/clustering purposes.http://costalab.org/Imputation/.We performed a broad analysis of the impact of five well-known missing value imputation methods on three clustering and four classification methods, in the context of 12 cancer gene expression datasets. We employed a statistical framework, for the first time in this field, to assess whether different imputation methods improve the performance of the clustering/classification methods. Our results suggest that the imputation methods evaluated have a minor impact on the classification and downstream clustering analyses. Simple methods such as replacing the missing values by mean or the median values performed as well as more complex strategies. The datasets analyzed in this study are available at The online version of this article (doi:10.1186/s12859-015-0494-3) contains supplementary material, which is available to authorized users. The expression level of a gene can be regarded as an estimate of the amount of protein it produces in a given cellular state. Different technologies can be used to measure the expression level of a gene. One of the most important is microarray technology, which allows the simultaneous measurement of the expression levels of thousands of genes .As with many types of experimental data, gene expression data obtained from microarray experiments often contain missing values (MVs) -5. This The simplest way of dealing with MVs is to discard the observations that contain them. However, this method is practical only when (1) the data contain a relatively small number of observations containing MVs, or when (2) the analysis of the complete examples will not lead to a serious bias during the inference ,7. NeithThus, in the microarray setting, instead of repeating the biological experiment (too expensive) or discarding all observations with missing values , many MV imputation methods have been proposed in the literature ,5. MV imk-nearest neighbor (WKNN) 13] is thIn the context of gene expression data, MV imputation methods usually fall into two categories . In the % missing values (MV filtering). Next, we imputed the missing values using each of the five methods. Finally, we applied a non-supervised filter to remove genes with little variation between samples. See Section We analyzed the effects of these imputation methods, as well as the Mean and Median methods, where MVs are replaced with a simple mean or median, respectively, from known values. Given these methods we conducted two classical downstream analyses for cancer gene expression data: classification and clustering. We used the following experimental design to do so. First, we removed all genes with more than 10 Tables We employed several classification and clustering techniques with the cancer tissues from each imputed dataset. Refer to the Methods section for further details on the methodology. We used SVM, kNN, naive Bayesian classifiers (NB), and decision trees (DT) for the classification experiments. The classification error rate, after leave-one-out cross-validation (LOOCV), was used as our evaluation metric , and hierarchical clustering with complete linkage (HC-CL) for the clustering experiments. Partition quality was evaluated with the corrected Rand (cR) index  had an impact in our results, we first tried to run the imputation methods without MV filtering. In this context, the implementation of the different imputation methods failed to deal with attributes with 50% or more of missing data. Then, to successfully run most of the methods we had to remove the attributes with more than 40% of missing values. After that, we applied the non-supervised filter. Next, we performed experiments with SVMs  with the resulting datasets. We also run k-medoids and the hierarchical methods. As for the case of the BPCA, the implementation that we have available failed to run for four datasets. In this context, we performed the hypothesis test without considering it. In all cases, the null hypothesis of equal ranks was not rejected .To help put our results into perspective, the most similar work to ours is that reported in ,17, as mk-means), we used two hierarchical clustering methods, as well as k-medoids. In [We employed steady-state cancer gene expression data, unlike , who useoids. In , the autkNN, and CART. According to their results, imputation algorithms, except for substitution by zero, did not have any demonstrable impact on classification. The work in [The work most closely related to ours regarding classification is that in , who inv work in  was limi work in , we alsoThe fact that gene expression is highly correlated helps to explain why the imputation of expression values close to the original missing value is not required for an analysis based on thousands of genes, such as tissue clustering or classification . This isWe would like to point out that, from a practical point of view, it is common to remove attributes (features) with more than a certain percentage of missing value. As other work in the literature, we set the threshold to 10%. Even in a context of a level of missing values of up to 40%, the statistical test does not point to any significant difference between the imputation methods.Finally, and of large consequence, of the 32 datasets in the benchmarking cancer gene expression data study in , only 12% missing entries for the datasets (MV filtering). Next, we applied the different imputation methods (see details below). Finally, we performed a non-supervised filtering to remove genes with low variation between samples. The datasets analyzed in this study are available at http://costalab.org/Imputation/. See Table We used gene expression datasets from the benchmark study in . More spn be the size of the dataset (the number of samples) and p be the number of attributes (variables) of each instance. k closest neighbors. The MV is replaced by the weighted average of the k-neighbors, where the weights are proportional to the similarity between samples. WKNN uses Euclidean distances as its similarity measure. In our experiments, following the guidelines in [k to 20. This method has a computational complexity of WKNN was proposed in . For a glines in , we set k-nearest genes and estimating the missing values by performing a linear regression on the variable with MVs [k=300. Such a method has a computational complexity of LLS is based on finding the with MVs . In thiswith MVs , we usedc is the number of components.BPCA combines a principal component regression with a Bayesian expectation maximization method to estimate the missing values . The metThis section presents the five imputation methods we evaluated. In what follows, let Besides these three “advanced” imputation methods, we also applied two very simple ones: Mean and Median, which are based on assigning the mean (or median) value of the gene to the respective missing entries . This isentclass1pt{minima results ,17. All  results . The reskNN; (3) NB; and (4) SVM.We explored the impacts of several commonly used data imputation methods on the performance of different classification and clustering algorithms by using 12 actual cDNA datasets in this study. More precisely, in terms of classification, we performed an empirical comparison by using four classical supervised learning algorithms : (1) DT,kNN with k=1 and Euclidean distance, and a DT with pruning) in the current experiments, based on our previous research [All of these learning methods were obtained from the WEKA machine learning package . The claresearch .k-medoids [We performed our cluster analysis evaluation employing three clustering algorithms widely applied in the literature of gene expression data: hierarchical clustering with both complete and average linkage , and k-m-medoids . We used-medoids . To do sFinally, all results were statistically compared with the Friedman statistical test for multiple comparisons and the Nemenyi post-test, at the 95% confidence level, according to the procedures described in , for bot


Based on the isotopic character of the alkanes, it is suggested that diesel fuels involved in the diesel theft case were distinguishable. This manuscript shows that CSIA when used in tandem with multivariate statistical analysis provide a defensible means to differentiate and source-apportion qualitatively similar oils at the molecular level. This approach was able to overcome confounding challenges posed by the near single-point source of origin, i.e., the very subtle differences in isotopic values between the samples.Compound-specific isotope analysis (CSIA) offers great potential as a tool to provide chemical evidence in a forensic investigation. Many attempts to trace environmental oil spills were successful where isotopic values were particularly distinct. However, difficulties arise when a large data set is analyzed and the isotopic differences between samples are subtle. In the present study, discrimination of diesel oils involved in a diesel theft case was carried out to infer the relatedness of the samples to potential source samples. This discriminatory analysis used a suite of hydrocarbon diagnostic indices, alkanes, to generate carbon and hydrogen isotopic data of the compositions of the compounds which were then processed using multivariate statistical analyses to infer the relatedness of the data set. The results from this analysis were put into context by comparing the data with the δ Compound-specific isotopic analysis (CSIA) is a technique that is becoming increasingly popular as a forensic tool to measure stable isotope composition of chemical compounds, such as hydrocarbons, which can be likened to fingerprinting at the molecular level. From a forensic perspective, it is important to infer a link between a sample to a suspected source(s) by obtaining chemical information from both data sets or to differentiate the sample in question from the other samples of known origin, i.e., a population study where an informed assessment can be made by obtaining the profiles of the sample population to provide context for the data. In other words, stable isotope fingerprinting provides the resolution to the question of whether two compounds or substances are distinguishable, i.e., un-related. If the two samples are indistinguishable this supports the hypothesis that they are related. Although variations in stable isotope ratios are generally very small they are robust and have allowed workers in numerous fields to formulate links between samples  to separate and quantify the different molecular species. Crude oils from different sources exhibit different characteristics in terms of the ratios of molecular species. Processing of the oil changes these characteristics, e.g., the high molecular weight pentacyclic terpenes and steranes are generally removed during the refining process whereas the diamondoids (adamantanes and diamantanes) are found in most petroleum products  using stable isotope fingerprints  Control (2 samples); (2) GKT ### (9 samples). The Control samples were taken from the storage tanks located at the ski field in Blenheim  via a high temperature conversion furnace, heated to 940°C and 1450°C for carbon and hydrogen analyses, respectively. The injection mode used was splitless with the temperature set at 300°C. Compounds were separated using a HP-1 GC column . Identification of the analytes was made prior to isotope analysis using GC-FID by comparing the retention time of DRH-008S-R2 hydrocarbon standard solution , which contains 35 n-alkane compounds in chloroform, with that of the samples. The GC analytical conditions for both instruments were set to be the same throughout each run to avoid misrepresentation. The carrier gas for the analysis was helium with a constant flow of 1.5 ml/min. The parameters for the analytical run were as follows: initial oven temperature 50°C, initial hold time of 1 min, temperature ramp at 10°C min to 300°C, and final hold time 4 min.Gas chromatographic and isotope analyses were obtained on at least two aliquots of each sample and each analysis was an average of at least three measurements. Compound specific carbon and hydrogen isotope ratios were determined using a Trace Ultra-gas chromatograph  coupled to a Delta2 or) at the beginning and end of each sample run. While instrumental drift during a batch sample was corrected by injecting a standard mixture  every six samples. The instrument analytical precision for compound specific δ 13C and δ2H analysis was determined to be <0.3 and 3%, respectively, based on long-term repeatability of a control sample. Isotopic compositions of each alkane were expressed as δ values per mil (%) deviation relative to isotopic standard reference materials:R = 13C/12C or 2H/1H. The δ13C were reported relative to the Vienna Pee Dee Belemnite (VPDB), while δ2H values were reported relative to the Vienna Standard Mean Ocean Water (VSMOW) standard. The H+3 factor was determined daily using the standard hydrogen gas introduced through the interface. The H+3 is a byproduct species formed during ion-molecule reaction which can interfere with the measurement of HD, so correction was required for H isotope determination. The mass spectrometer was tuned to ensure that the H+3 factor was less than 10 nA/ppm and the daily variability was <0.1.Instrumental drift during individual sample analysis (~30 min) was corrected by injecting multiple pulses of monitoring gas , SigmaPlot 11.0 (Systat Software Inc.®) and SPSS 16.0 (IBM®). Multivariate statistical analysis of the stable isotope data was performed using PCA and HCA.PCA is a mathematical procedure that converts the possibly correlated original variables into new linearly uncorrelated variables, called the principal components (PCs). This technique also reveals the internal structure of the data and finds the indices which best explains the variance in the data set. PCA also provides the most meaningful parameters which interpret the whole data set, thus reducing the dimensionality of the transformed data and summarize the statistical correlation among constituents with minimum loss of original information . The dendrogram summarizes the clustering process, showing the number of clusters (number criterion) and indicating their proximity in space (distance criterion), thus reducing the dimensionality of the original data. To analyze the data set in this study, HCA was performed using Ward's linkage method. The Ward's method minimizes the total within-cluster variance, and the cluster distances are defined by the squared Euclidean distances as a measure of similarity.HCA is an analysis that builds a hierarchy of clusters by measuring either the distance or the similarity between the objects to be clustered. It is normally used when there are no 13C values of individual alkanes in the same set of samples. The range of carbon isotope compositions of the alkanes within these diesel samples (−33.3 to −25.8%) was comparable to the values determined in previous work  and pristane (−33.3 to −29.2%). There was no significant difference in δ13C values of odd and even numbered alkanes.The ranges of carbon isotope ratios of each individual alkane in Control, GKT and South Island of New Zealand samples are illustrated in Figure 2H values of individual alkanes in Control, GKT and South Island of New Zealand samples. The largest range of δ2H was found in nC12 (−100.0 to −45.5%), nC15 (−96.2 to −58.1%) and nC17 (−97.6 to −59.7%). The wide range of hydrogen isotopic values for these alkanes was in agreement with previous published result  described 50.4% of the variance in the data. The second principal component (PC2) described an additional 19.1% of the variance while the third principal component (PC3) explained further 9.6% of the data variability. Thus, 79.1% of the total variance was explained by the first three components.The forensic evidence provided by stable isotope analysis is circumstantial in that when two samples are indistinguishable that supports, but does not prove, the hypothesis that they are related. The likelihood that they are actually related can only be estimated by measuring control samples and determining how likely two samples could have identical isotopic character by chance. In the present context it is important to establish whether samples obtained from the same source region have similar isotopic characteristics and are distinctive from samples obtained from other source regions. Therefore, it is imperative that control samples be subject to a population study in order for analysts to make an informed assessment of the likelihood. The control samples used here were commercially available diesel samples obtained from around the South Island of New Zealand  that described 40.9% of the variance in the data. The second principal component (PC2) described an additional 17.9% of the variance while the third principal component (PC3) explained further 14.8% of the data variability. Thus, 73.6% of the total variance was explained by the first three components.13C and δ2H values as variables and their contribution to the variance in the data set comprised of the Control, GKT and South Island of New Zealand samples. Component matrix 1 shows the δ13C and δ2H values of the alkane compounds contributed evenly with highly positive coefficients for most of the compounds. This implies that they have equal weighting in the interpretation of the scores for PC1. For PC2, δ2H values of the compounds were mostly negatively correlated to the δ13C variables with more compounds showing higher weightage and contributed more to the interpretation of the scores. In component matrix 3, an opposing trend for δ13C and δ2H values was seen with δ13C variables showing negative correlations to that of δ2H variables.Table These interpretations from PCA were visualized on the scores plots in Figures The stable isotope data of the diesel samples involved in the theft case were also analyzed using cluster analysis to see their association with the diesel samples from the South Island of New Zealand. This relationship was presented in a dendrogram in Figure The forensic discrimination of diesel fuels involved in a theft case utilizing stable isotope fingerprint of hydrocarbon indices was achieved. There were variations in the stable isotopic compositions of alkanes within the diesel samples that can be used to differentiate them. Additionally, a large population of commercial diesel fuels obtained from various areas in the South Island of New Zealand was included in the data set to rule out statistical coincidence as well to put the data into context thus providing an informed assessment of the analysis.Discrimination of the diesel samples into groups was evident by comparing the subtle differences in the stable isotope values of the alkane compounds using multivariate statistical analyses such as PCA and HCA. The main conclusions from these analyses are:PCA showed that the Control diesel samples were strongly correlated with GKT 106 and GKT 202 samples and moderately correlated with the other GKT samples.Diesel samples that were grouped together based on PCA also showed high correlation using HCA.However, GKT 105 samples were shown to be grouped separately from the other Blenheim samples using HCA.PCA and HCA both highlighted the association of Control samples with that of GKT 106, GKT 107, GKT 108, and GKT 202 samples.In our opinion, the most likely explanation is that these diesels share a common history.In summary, this paper provided some context on the Blenheim diesel theft case. The Control samples were found to be chemically related to samples found in possession of the suspect, i.e., they were indistinguishable by measurements of their stable isotope fingerprint. That they were distinguishable from the 45 additional samples collected from around the South Island and that the South Island samples were able to be grouped according to source provided support for the prosecution hypothesis that the samples in possession of the suspect had originated from the theft. When this evidence was presented in court the accused changed their plea to guilty and was convicted of the theft.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.