A fundamental challenge in quantitation of biomolecules for cancer biomarker discovery is owing to the heterogeneous nature of human biospecimens. Although this issue has been a subject of discussion in cancer genomic studies, it has not yet been rigorously investigated in mass spectrometry based proteomic and metabolomic studies. Purification of mass spectometric data is highly desired prior to subsequent analysis, e.g., quantitative comparison of the abundance of biomolecules in biological samples.We investigated topic models to computationally analyze mass spectrometric data considering both integrated peak intensities and scan-level features, i.e., extracted ion chromatograms (EICs). Probabilistic generative models enable flexible representation in data structure and infer sample-specific pure resources. Scan-level modeling helps alleviate information loss during data preprocessing. We evaluated the capability of the proposed models in capturing mixture proportions of contaminants and cancer profiles on LC-MS based serum proteomic and GC-MS based tissue metabolomic datasets acquired from patients with hepatocellular carcinoma (HCC) and liver cirrhosis as well as synthetic data we generated based on the serum proteomic data.%) between estimation and ground truth. By applying the topic model-based purification to mass spectrometric data, we found more proteins and metabolites with significant changes between HCC cases and cirrhotic controls. Candidate biomarkers selected after purification yielded biologically meaningful pathway analysis results and improved disease discrimination power in terms of the area under ROC curve compared to the results found prior to purification.The results we obtained by analysis of the synthetic data demonstrated that both intensity-level and scan-level purification models can accurately infer the mixture proportions and the underlying true cancerous sources with small average error ratios (<7 We investigated topic model-based inference methods to computationally address the heterogeneity issue in samples analyzed by LC/GC-MS. We observed that incorporation of scan-level features have the potential to lead to more accurate purification results by alleviating the loss in information as a result of integrating peaks. We believe cancer biomarker discovery studies that use mass spectrometric analysis of human biospecimens can greatly benefit from topic model-based purification of the data prior to statistical and pathway analyses. Identification of disease-related alterations in molecular and cellular mechanisms may reveal useful disease biomarkers. Discovery of clinically relevant biomarkers has potentially far reaching implications for disease management and patient treatment –4. High-While the capability of high-throughput technology to yield comprehensive profiling and quantification offers a unique advantage in biomedical research, the heterogeneous nature of the biological samples poses a fundamental challenge in data analysis and interpretation. Specimens, such as tumor tissues and human blood, are typically mixtures of cells with distinct states and types, and usually only part of the constituent cell populations is relevant to the biological question of interest , 8. In sExperimental methods for cleaning samples and isolating type-specific constituents are costly and time-consuming. Computational purification methods offer an attractive alternative that is inexpensive and efficient to implement, and can be applied to data already generated without any modifications on experimental procedures. Multiple approaches have been developed to deconvolute gene expression profiles in the past years, varying from linear regression based models , 12 to gIn this section, we introduce topic model-based intensity-level and scan-level purification methods. Assumptions and strategies in the topic models are elaborated. Mass spectrometric datasets from cancer biomarker discovery studies are described.m/z and retention time points. Due to the existence of heterogeneity, multiple constituents in the sample contribute to the observed expression profile. Therefore, we can model the expression profile of a heterogeneous sample t as a weighted mixture of expression profiles of multiple sources, including a cancerous origin γ and non-cancerous contaminants β. The expression distribution for every biomolecule in each of the sources plays a role as a “topic” contributing to the mixed expression profile. Basically, each ion in the observed profile is associated with a specific topic, i.e. a multinomial distribution of ion counts over biomolecules, determined by the corresponding source profile. In this model, expression profiles refer to integrated peak intensities.The LC/GC-MS instruments provide ion intensity values by counting the ions detected at specific The purification procedure can be realized through a set of topic models, which are generative probabilistic models typically applied to text corpora mining. Specifically, each expression profile is characterized by a probability distribution across topics. Topics are probability distributions across biomolecules. These distributions can be inferred based on the analysis of a collection of expression profiles through topic models. These hierarchical Bayesian models are variants of latent Dirichlet allocation (LDA) , a topictd}d=1,⋯,D are coming from the control group {βm}m=1,⋯,M . It is commonly observed that the cancerous tissues are surrounded by adjacent non-cancerous tissues, which are typically used as controls in differential expression analysis. Second, the corresponding cancerous origins {γd}d=1,⋯,D share an average cancer profile γ′. Individual cancerous profile can be treated as a noisy version of the average cancer profile. Third, the average cancer profile γ′ has similar patterns as non-cancerous profiles {β}, except for some sites (biomolecules) which are differentially expressed between case and control groups. Thus, the cancerous profile can be treated as a similar non-cancerous profile with several sites altered.We use a modified topic model to purify the molecular expression profiles in cancer. Basically, three assumptions are made in developing the model. First, the source contaminants in each expression profile {td}d=1,…,D) describes how the profiles {The model also incorporates the following latent variables: the average cancer profile γ′ is sampled from a Dirichlet distribution parameterized by a weighted mixture of non-cancerous profiles. Each pure cancer profile γd together with M contaminants {βm} forms a sample-specific topic panel. The mixture proportion of topics determines zd,n, indicating which source  each ion originates from. We infer the latent variables γ′,{γd}d=1,⋯,D,{θd}d=1,⋯,D, and estimate the parameters using the two-step learning approach developed based on variational EM algorithm  interval. This integration or truncation, however, inevitably brings in variances which interfere with original sample heterogeneity. Therefore, we propose to investigate LC/GC-MS data purification with scan-level measurements based on extracted ion chromatogram (EIC), which preserves scan-level peak shape information. We hypothesize that purification at the scan level leads to more accurate results and offers the opportunity to extend the model to characterize both ion abundance and peak shape.td} (same for {βm}) consists of multiple EIC peaks. Each peak is represented by ion abundances across S scans with a certain elution profile shape After ion tracing and missing value interpolation, we can obtain a list of EICs for each sample. EIC is characterized by its retention time (corresponding to multiple scans), mass value, and ion abundance. In this scenario, the observed data {entclass1pt{minimaxd,n is the ion abundance for nth compound of dth sample; δd,n(s) is a latent indicator to model the missing scans; the chromatographic peak shape is characterized by the exponentially modified Gaussian (EMG) function [ϕ, as described in Eq.  is the random noise. where, function  parameted in Eq. , and ed,td,n corresponds to the shape of the EIC (characterized by ϕ) as well as ion abundance xd,n.We hypothesize that the data heterogeneity in xt, xβ, xγ′, and xγ. We assume error terms in intensity measurements in Eq.  follows a Bernoulli distribution, parameterized by qd with a prior of Beta distribution: The missing scan indicator variable The observed data point therefore follows the distribution: ϕ are considered to have a normal distribution and its detailed priors are described in [xt, xβ, and shape function parameters ϕ). We then treat xt, xβ as observed variables to implement the inference on the first part using the same algorithm [θ, pure ion abundance xγ, shape function parameters ϕ and related parameters. After purification is performed, ion intensity may be calculated by applying peak detection algorithms [γd,n} recovered using Eq.  and liver cirrhosis (control group) –4. HCC iFifteen liver tissues were collected from 10 participants recruited at MedStar Georgetown University Hospital. As shown in Fig. L=101). All profiles were normalized to the mean total-ion-counts at N=1.68×108. It is still not clear how the development of tumor in liver directly affect the alterations in blood. We hypothesize that there are some impacts from cirrhotic constituents contributing to the HCC profile in serum. The contamination may occur in an indirect way through, for example, secreted biomolecules instead of adjacent tissue cells. We apply the purification to remove the influence from cirrhotic contaminants.We acquired 116 proteomic data by analysis of sera from 57 HCC cases and 59 patients with liver cirrhosis recruited from the hepatology clinics at MedStar Georgetown University Hospital. Following depletion and digestion, proteins extracted from sera were injected into a 3000 Ultimate nano-LC system interfaced to LTQ Orbitrap Velos and TSQ Vantage mass spectrometers in untargeted and targeted analyses, respectively. Proteins were identified and quantified by MaxQuant  and Skylλs}s=1,⋯,57, to obtain an average cancer profile γ′, which is close to the real cancerous profile for HCC. (ii) Sample 30 individual pure cancer profiles {γd}d=1,⋯,30 from a Dirichlet distribution, as in  as sources of contamination. Normalize them into form of multinomial distribution. (iv) Combine M cirrhotic profiles with each of the pure cancer profiles to create 30 topic panels, each consisting of M+1=10 profiles. (v) Sample 30 mixture proportions {θd}d=1,⋯,30 from a Dirichlet distribution, as in  and with a larger value assigned to last constituent (cancer origin). This ensures a larger proportion of cancerous component in final cancer profile. (vi) Sample a topic indicator zd,n from θd using  to test on intensity level if the model can reasonably estimate the proportion of mixtures in each of the synthetic profiles and recover the pure cancer profiles underneath; 2) to demonstrate if the scan-level purification model gives more accurate estimation on synthetic data; 3) to investigate the benefits of using these models to purify samples from cancer patients collected in our previous differential analysis studies.γ′∗. Whereas, we expect outputs of sample-specific mixture proportion ϕ∗ from extended model. For synthetic datasets, we compare the estimated proportions of mixtures θd}) used to generate the synthetic data. Estimation error ratio for a single sample is defined in Eq.  and in purified cancer profile versus true cancer profile (orange). The average correlation coefficient increases from 0.986 to 0.999 after purification. The effects of purification are illustrated in Fig. We obtained an average error ratio of mixture proportion % by Eq. . We found 43 proteins with significant change in expression between the two groups. The number of reported significant proteins under the same testing method increased from 43 to 75 after purification. The majority of the proteins identified in original profiles (40 out of 43) remained significant after purification. If purified based on scan-level features, the number of significant proteins also increased to 69, among which 38 and 61 are overlapped with unpurification and intensity-level purification results, respectively.Firstly, in statistical analysis, the most relevant proteins with differential intensities between HCC cases and cirrhotic controls were selected using t-test, and the associated % confidence interval (CI) of the area under each ROC curve. After intensity-level and scan-level purification we respectively achieved an average AUC of 0.793  and 0.811, both higher than 0.706  for original biomarkers. More powerful biomarkers were selected after scan-level purification.Figure 11Finally, we used DAVID  . However, seven metabolites were identified as significant after the profiles were purified. For the adjacent cirrhotic profiles {ψd}d=1,⋯,5, we applied the model to remove contaminations from any neighboring cancerous cells. We expected to observe that the purified adjacent cirrhotic profiles became close to independent cirrhotic profiles. The dissimilarity, defined in (Heterogeneity issue is more intuitive in tissue samples, where the contaminations originate from the neighboring non-homogeneous cells. We first purified the HCC profiles {fined in , betweenIn this paper, we investigate topic model-based inference methods to computationally address heterogeneity issue in samples analyzed by LC/GC-MS. The topic model gives a probabilistic explanation on the corpus of LC/GC-MS based profiles on both integrated peak and scan-level ion intensity levels. The performances of our models in estimating mixture proportion and retrieving underlying true cancer profile are evaluated through well-designed synthetic data. We observed that incorporation of scan-level features gives more accurate purification results by alleviating the loss in information caused as a result of integrating peak intensity values. Through GC-MS metabolomic and LC-MS proteomic datasets we acquired from tissues and blood samples, respectively, we showed the benefit of applying topic-model based purification of the data prior to statistical and pathway analyses. Specifically, we observed improved discrimination between case and control groups and biologically meaningful pathway analysis results. Future studies will focus on cross-validation of the findings either computationally through mass spectrometric data from large-scale cancer biomarker discovery studies or by using ground-truth information from pathology reports and literature survey.EIC, extracted ion chromatogram; GC, gas chromatography; HCC, hepatocellular carcinoma; LC, liquid chromatography; MS, mass spectrum; RT, retention time


L1, SCAD and MC+. However, none of the existing algorithms optimizes L0, which penalizes the number of nonzero features directly.Feature selection and prediction are the most important tasks for big data mining. The common strategies for feature selection in big data mining are L0 approximation for feature selection and prediction with big omics data. The proposed approach approximate the L0 optimization directly. Even though the original L0 problem is non-convex, the problem is approximated by sequential convex optimizations with the proposed algorithm. The proposed method is easy to implement with only several lines of code. Novel adaptive ridge algorithms (L0ADRIDGE) for L0 penalized GLM with ultra high dimensional big data are developed. The proposed approach outperforms the other cutting edge regularization methods including SCAD and MC+ in simulations. When it is applied to integrated analysis of mRNA, microRNA, and methylation data from TCGA ovarian cancer, multilevel gene signatures associated with suboptimal debulking are identified simultaneously. The biological significance and potential clinical importance of those genes are further explored.In this paper, we develop a novel sparse generalized linear model (GLM) with L0ADRIDGE in MATLAB is available at https://github.com/liuzqx/L0adridge.The developed Software The online version of this article (doi:10.1186/s13040-017-0159-z) contains supplementary material, which is available to authorized users. L0 regularized sparse modeling, which penalizes the number of nonzero features directly. L0 is known as the most essential sparsity measure and has nice theoretical properties. However, it is computational impossible to perform an exhaustive search when analyzing omics data sets with millions of features. L0 penalized optimization is known to be NP-hard in general .Integrating multilevel molecular and clinical data to design preventive, diagnostic, and therapeutic solutions that are individually tailored to each patient’s requirements is the ultimate goal of precision medicine. However, the huge number of features makes it neither practical nor feasible to predict clinical outcomes with all omics features directly. Thus, selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions has become an important step toward effective big data mining. Statistical tests or univariate correlation analysis for feature selection ignore the interacting relationship among genes. To evaluate the predictive power of the features, one appealing approach for feature selection is L0 with the L1 norm. L1 is a convex relaxation and loose approximation of L0. Although L1 penalized sparse models t, X=t, and μ=t, so The distribution of Y in GLM is assumed to be from the exponential families with the following probability (density) function: utions Y . The corA(ϕ), and C, we have the simplified log likelihood as follows: Dropping the constants L0 penalized error function to minimize is Hence, β, μi=G(θi) and where entclass1pt{minimawhich is equivalent to the following system: η and E w.r.t. β is Given where ⊘ indicates element-wise division. The Hessian matrix is Vi=V(μi)=B″(θi)=G′(θi), i=1,…,N, and let Let β is The Newton-Raphson iteration for Let Different link functions will lead to different regression models as shown as in Table V(μ). When dealing with big data problems with N≪P,where N is the number of samples and P is the number of parameters, the inverse of a P×P matrix is time-consuming and computational challenging. We proposed an efficient algorithm to calculate the inverse of a much smaller N×N matrix as follows : Other GLMs such as negative binomial, gamma, and inverse Gaussian can be implemented accordingly with a different N≪P, we have a much efficient estimation: So that when L0ADRIDGEA) is implemented in MATLAB are as follows:The adaptive ridge algorithm , respectively. We further discuss that the proposed method is a L0 approximation and converges to L0 when the number of iterations m→∞.The algorithm is easy to implement and very efficient for either small sample size and large dimension or large sample size and small dimension big data problem. The regularized parameter Algorithm justification: Given a high-dimensional big feature matrix XN×P(N≪P) and a threshold γ for the coefficient estimates, L0 rejects all the coefficient estimates below γ to 0 and keeps the large coefficients unchanged. This is the same as defining a binary vector s=t, with the value of 0 or 1 for each feature, where sj=1 if the coefficient estimate for that feature is above the threshold γ, and 0 otherwise. Let S=diag(s) be a matrix with s on its diagonal, we have the selected feature matrix XS=XS. We can build the standard models with the matrix XS, if we know s in advance. For instance, we can estimate the coefficients of a GLM with L2 regulation given XS and Y with S. It is guaranteed that the estimate is 0 for feature j with sj=0. However, in reality we do not know s. Estimating both s and θ is an NP-hard problem, since we need to solve a mixed-integer optimization problem. Comparing Eq. (βnew=(DXtVX+λI)−1DXtZ, it is clear that S is replaced by D and a binary sj is approximated by a continuous L0 approximation.where ring Eq.  with Eq.ring Eq. , βnew= become smaller at each iteration and will eventually go to zero (below the machine ε). We can also set a parameter to 0 if it is below predefined ε=1e−6 to speed up the convergence of the algorithm.Smaller t of Eq. , becauset in Eq. . Note thPoisson Regression: Our first simulation was used to evaluate the performance of our method for high dimensional Poisson regression. The data was generated from Poisson distribution with different sample sizes (N) and dimensions (P). However, only features 1, 5, 10 and the constant term are used to generate the Poisson counts with =. The count Y is generated with Y=Poisson(μ), where mean μ= exp(βX). The proposed method is compared with the glmnet (=, respectively, and the remaining coefficients were set to zero. The score z=Xβ+ε, where ε is the random noise with the signal to noise ratio of 4. Then, the probability y is generated from the logistic function y=1/(1+ez−). Note that y is the true probability instead of binary (1/0) in this simulation. Unlike the previous example, the optimal values of λ in this simulation were selected with the standard 5-fold cross-validation. We divided the λ from λmin=1e−4, to λmax into 100 equal intervals in log-scale, then chose the optimal λ with the smallest test error. The simulation was also repeated 100 times. The computational results were reported in Table L0ADRIDGE outperforms L1, SCAD and MC+ with a substantial margin under the 5-fold cross-validation. Cross-validation is a standard tool for parameter selection in machine learning. L0ADRIDGE achieved the smallest test square root of mean squared error, least absolute biases, the lowest FDR, and highest percentages of identified true models The average number of selected features are 3.33 and 3.41 for the dimensions of 100 and 1000, respectively, which are the closest to the true number of features 3. In contrary, L1, SCAD and MC+ selected unnecessary features. L1 on average identified 17.1 and 50.92 features, and SCAD selected 18.35 and 73.03 features on average for the dimensions of 100 and 1000, respectively, while MC+ performed slightly better, choosing 10.41 and 24.8 features for the dimensions of 100 and 1000, respectively. More impressively, out of 100 simulations, L0ADRIDGE identified the true model 81 and 80 times with different dimensions, while L1 and SCAD could not find the true model once, and MC+ only identified the true model 2 times for the dimension of 100, indicating the super performance of L0ADRIDGE under cross-validation. Finally, L0ADRIDGE is robust. The test square root of mean squared error and other performance measures did not vary much when the dimension increased from 100 to 1000. It is worth noting that our proposed method performs well with the popular statistical model selection criteria such as BIC and cross-validation. Other popular methods such as L1, SCAD, and MC+ select more features than necessary with such criteria. Therefore, many popular packages including the commercial MATLAB usually choose a larger λ one standard deviation above the minimum test error with cross-validation, which is arbitrary and leads to larger bias. To overcome such bias in parameter estimation, some packages re-estimate the parameters with the selected features and standard GLM model. Unlike these methods, our proposed method performed much better without any postprocessing. Finally, the algorithm is very robust with different initialization. With N=100, P=1000 and 100 times of different randomized initialization, we achieved the trMSE of 0.437(±.003), average absolute bias of 0.0763(±.07), ANSF of 3.39(±1.154), PTM of 85% and FDR of 6.9%, which is quite similar to the results with a fixed initialization.Table P-values of less than 0.01 with the training data only. Based on the central dogma of biology, suboptimal debulking is associated with microRNA expression, gene expression, and DNA methylation; gene expression is a function of microRNA expression and DNA methylation; and microRNA expression is regulated by DNA methylation. L0 Logistic regression was used for suboptimal debulking prediction, while L0 penalized Poisson regression was used for gene expression and microRNA expression prediction with FPKM. FPKM, representing fragments per kilobase of exon per million fragments mapped, measures the normalized read counts for RNA-seq. Three-fold cross validation was used for gene selection and validation. We reported the gene signatures with the best predicted area under the ROC curves (AUCs). Molecular signatures that are directly or indirectly associated with suboptimal debulking are shown in Fig. The Cancer Genomic Atlas (TCCA) has generated a large amount of next generation sequencing and other omics data for ovarian adenocarcinoma (OC). In this study, we conducted integrated analysis of RNA-seq, miRNA expression, promoter methylation, and debulking status data from 367 OC patients. There are 342 microRNAs, 13,911 mRNA expression (in FPKM), and 21,985 promoter methylation values available. We first normalized different omics data and screened the debulking associated microRNA, mRNAs, and methylation promoters with the Figure Figure L0 based GLM, which directly penalizes the number of nonzero parameters, has nice theoretical properties and leads to essential sparsity for biomarker discovery. Optimizing the L0 regularization is a crucial, but difficult problem. We have developed an adaptive ridge algorithm (L0ADRIDGE) for approximating L0 penalized GLM. The algorithm is easy to implement and efficient for problems with either an ultra-high dimension and small sample size, or a low-dimension and large sample size. It outperforms the other cutting edge regularization methods including L1, SCAD and MC+ through simulations. When applied to the integration of multilevel omics data from TCGA and the prediction of suboptimal debulking from ovarian cancer, it can identify a panel of gene signatures achieving the best prediction power. We also demonstrate that prediction power of a model with multi-omics data increases substantially, when comparing with a model with one omics data, indicating the importance of big data mining.Biomarkers from multi-omics data may predict disease status and help physicians to make clinical decisions.


The same set of tumour tissue samples were profiled twice using the Agilent microRNA microarrays: once under an ideal experimental condition with balanced array-to-sample allocation and uniform handling; a second time by mimicking typical practice, with arrays assigned in the order of sample collection and processed by two technicians in multiple batches. This paper provides a detailed description of the generation and validation of this unique dataset pair so that the research community can re-use it to investigate other statistical questions regarding microarray study design and data analysis, and to address biological questions on the relevance of microRNA expression in gynaecologic cancer.We set out to demonstrate the logistic feasibility of careful experimental design for microarray studies and its level of scientific benefits for improving the accuracy and reproducibility of data inference. Towards this end, we conducted a study of microRNA expression using endometrioid endometrial tumours ( Systematic variations in experimental handling factors, such as lab technicians and image scanners, can lead to undesirable variations in the data that increase data variability and confound the biological signal of interest3. A typical practice for combating such unwanted variations is to use post-hoc data adjustments such as normalization4–6. An alternative to post-hoc adjustment is to carefully design the experiment, using time-tested statistical principles such as blocking and randomization, to balance handling effects between sample groups of interest and abate their negative impacts on data inference7–9. However, such careful design has received little attention in genomic studies, possibly due to lack of awareness and the perceived level of logistic difficulty in implementing them.Genomic profiling of molecular features such as gene expression levels entails a complex multi-stage experimentn=96) and serous ovarian tumours (n=96) that were primary, untreated, and collected during 2000–2012 at Memorial Sloan Kettering Cancer Center. The same set of tumour tissue samples was profiled twice using the Agilent microRNA microarrays: once under an ideal experimental condition with balanced array-to-sample-group allocation (via the use of blocked randomization) and uniform handling (by an experienced technician in a single processing run); a second time by mimicking typical practice with arrays assigned to samples in the order of sample collection and processed by two technicians in multiple batches. Differential expression between the two sample groups was assessed in the uniformly-handled dataset to serve as a benchmark; it was also assessed in the non-uniformly-handled dataset, both before and after normalization, and compared with the benchmark. Additional datasets were simulated by estimating (1) biological effects for the samples  and (2) handling effects for the non-uniformly-handled arrays  from the paired datasets, and then re-allocating and re-hybridizing the virtual arrays to the virtual samples following various configurations of blocking, stratification, and randomization in the presence of handling effects7.We aim to demonstrate that (1) careful experimental design is logistically feasible in clinical microarray studies and (2) it can provide significant scientific benefits that warrant the planning effort. Towards this end, we conducted a study of microRNA expression in endometrioid endometrial tumours . Fluorescence labelling of the extracted RNAs, hybridization to the arrays, slide washing, and image scanning all followed the manufacturer's instructions. Image data were extracted using Feature Extraction 10.7.3.1 (Agilent). Arrays used for the first study (with careful design) were ordered from the same manufacture batch, and arrays used for the second study were ordered from two separate manufacture batches.In the first study, arrays were assigned to tumour samples using blocked randomization and were processed by an experienced technician in a single run. The data from this study is referred to as Data Citation 1.10. Agilent microRNA microarrays come in 8-plex slides (with 8 arrays on each slide), which serve as blocks. In this study, 24 slides containing 192 arrays were used for the 192 tumour samples, with four arrays on each slide assigned to each sample group. Blocking can balance handling effects between two (pre-specified) sample groups so that they can cancel out in the analysis of differential expression comparing the two groups.− Blocking is the assigning of experimental units in each block of units to sample groups in proportion to group sizes− On each slide eight arrays are arranged in two rows and four columns. In order to avoid any positional effect on the slide, array assignment was further stratified by slide row and column, with equal numbers of arrays on each row and each column assigned to the two sample groups. For a 2 by 4 array slide, there are a total of six possible configurations that allow row and column balance.10. It can likely balance handling effects, with the level of likelihood positively correlated with the sample size. It is particularly useful when the primary outcome of interest is unknown or when there are secondary outcomes of interest.− Randomization is the assignment of arrays to samples in a random manner− Randomization, in combination with blocking, is the allocation of arrays to sample groups with blocking first, and then assigning the arrays allocated to a sample group to samples in that group in a random manner.7.− When implementing the array assignment for our study, we randomly assigned the 24 slides to four repetitions of the six row-column-balanced configurations, and then randomly assigned arrays allocated to a sample group to tumor samples in that groupIn the second study, arrays were assigned to tumour samples in the order of sample collection and were handled by two technicians in five batches (with each batch on a separate date). More specifically, two batches of 40 arrays each were handled by one technician (the same technician who handled the first study), and three batches of size 34, 38, and 40 were handled by another technician. The data from this study is referred to as Data Citation 2.7. The Agilent microRNA array platform includes 10 to 40 replicates for each of the 3,523 markers. The between-replicate variation was very small in our data, which allowed us to use a simple median to summarize the replicates for each marker7.Data pre-processing for Data Citation 1 (which resulted from careful study design) included two steps: (1) log2 transformation, and (2) marker-replicate summarization using the median11: (1) log2 transformation, (2) data normalization using quantile normalization, and (3) marker-replicate summarization using the median. We focus on the use of quantile normalization for the normalization step in this paper, and refer the readers to our previous publication for the use of other normalization methods7.Data pre-processing for Data Citation 2 included three stepsIn addition to the 3,523 markers representing microRNAs, 7 negative control markers and 37 positive control markers were included on the Agilent microRNA array. The data for these control markers were included in Gene Expression Omnibus (GEO) database submission of the two datasets.R 3.2.3. Codes for reading the raw Agilent data files into in R  is also available to provide access to both datasets.Extracted RNAs of tumour samples were checked on their quality based on the RNA Integrity Number (RIN)  and the How to cite this article: Qin, L.-X. et al. A pair of microarray datasets for microRNA expression profiling to examine the use of careful study design for assigning arrays to samples. Sci. Data 5:180084 doi: 10.1084/sdata.2018.84(2018).Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Glioblastomas exhibit widespread molecular alterations including a highly distorted epigenome. Here, we resolve genome-wide 5-methylcytosine and 5-hydroxymethylcytosine in glioblastoma through parallel processing of DNA with bisulfite and oxidative bisulfite treatments. We apply a statistical algorithm to estimate 5-methylcytosine, 5-hydroxymethylcytosine and unmethylated proportions from methylation array data. We show that 5-hydroxymethylcytosine is depleted in glioblastoma compared with prefrontal cortex tissue. In addition, the genomic localization of 5-hydroxymethylcytosine in glioblastoma is associated with features of dynamic cell-identity regulation such as tissue-specific transcription and super-enhancers. Annotation of 5-hydroxymethylcytosine genomic distribution reveal significant associations with RNA regulatory processes, immune function, stem cell maintenance and binding sites of transcription factors that drive cellular proliferation. In addition, model-based clustering results indicate that patients with low-5-hydroxymethylcytosine patterns have significantly poorer overall survival. Our results demonstrate that 5-hydroxymethylcytosine patterns are strongly related with transcription, localizes to disease-critical genes and are associated with patient prognosis. Glioblastomas have distorted epigenomes. Here, the authors compare the genome-wide profiles of 5-methylcytosine and 5- hydroxymethylcytosine in glioblastoma and prefrontal cortex tissue reporting a correlation between these profiles and patients' prognosis. Glioblastoma is the most common and malignant primary intracranial tumour accounting for ∼60% of gliomas in adultsIDH1 and IDH2) genes have been shown to harbour a pattern of DNA hypermethylation at certain promoter regions which results in a glioma CpG island methylator phenotype (G-CIMP)78Systematic molecular analyses have advanced our understanding of glioblastoma pathobiology through the identification of common mutations, structural-based genetic alterations and dysregulated epigenomes346783710While the glioblastoma DNA methylome has begun to be described, additional modifications to DNA that modulate normal gene function have been implicated in disease development15Use of sodium bisulfite (BS) treatment followed by hybridization to the Infinium DNA methylation arrays  is a common method for interrogating DNA methylation at the single base level1919Here, we present the characterization of 5 mC and 5 hmC levels from paired BS and oxBS 450 K assays in thirty glioblastomas. To effectively analyze and interpret this data, we develop and apply a novel algorithm, OxyBSIDH1 and IDH2 wild type. Patient demographic, tumour characteristics and survival data are presented in We adapted the BS–oxBS technology to DNA from 30 glioblastomas and applied a novel algorithm, OxyBS, to obtain genome-wide 5 hmC and 5 mC estimatesn=5). To avoid limitations due to the presence of multiple cell types  in the prefrontal cortex samples we did not compare CpG-specific differences in 5 hmC. However, to provide a global perspective on 5 hmC differences we examined the overall proportion of 5 hmC in total cytosine content  between the two tissues. We observed an average 3.5 fold reduction  in the total 5 hmC content in glioblastoma samples when compared with cortex samples . Again, we did not observe any significant relationships . 5 hmC has been implicated in the DNA demethylation pathway and may suggest functionality of 5 hmC outside simply removal of 5 mC.First, to qualitatively assess the extent of deregulation in the glioblastoma epigenome we compared the total genomic content of 5 hmC among glioblastoma samples with levels of 5 hmC in the prefrontal cortex. Although we were unable to obtain matched disease-free cortex samples in our own cohort, we accessed publicly available prefrontal cortex data using the same BS–oxBS protocol in an independent population of individuals that presented with no evidence of neurological impairment  analysis. The top twenty most highly significant gene sets ranked by fold enrichment are presented in Q-value<0.05, Fisher's exact test, 3435It has previously been observed that 5 hmC is located near transcription factor binding sites (TFBSs) in the mammalian genomen=172)P=5.2E-139, χ2 test, P=2.23E-48, Fisher's exact test, P=9.75E-35, Fisher's exact test, P=6.3E-01, Fisher's exact test, The robust enrichment for 5 hmC sites in super-enhancers and binding sites of proliferation-associated TFs suggests that 5 hmC is strongly associated with regulation of disease-specific gene expression programs. To confirm that 5 hmC marks are generally associated with active expression in glioblastoma we interrogated the entire glioblastoma transcriptome using RNA sequencing expression levels from The Cancer Genome Atlas (TCGA)  using the NanoString nCounter technology. For our candidate gene approach we selected 14 genes with a high proportion of high 5 hmC sites in a given gene region  and a geMGMT . In geneMGMT . In contMGMT . MGMT mepression . OverallIDH1 and IDH2 wild type, we confirmed that one sample was G-CIMP+ as defined in Noushmehr et al.3839P=2.0E-04, Kruskal–Wallis rank sum test), and patient age , but not copy number alterations  and segregated subjects into low and high-total 5 hmC groups based on whether total 5 hmC levels were above or below the median. In a multivariable Cox proportional hazards model adjusted for age at diagnosis and sex, the low total 5 hmC group was not significantly associated with survival (HR=0.94 CI 95% (0.43–2.03), P=0.88, Cox proportional hazards regression) suggesting that the genomic location of 5 hmC is an important consideration in associations with disease progression.Prior publications have identified an association between decreased total 5 hmC (determined by different methods) and poorer survival in glioblastoma patientsIn this study, we leveraged paired oxBS and BS 450 K arrays to delineate the genomic location and abundance of 5 hmC at nucleotide resolution in thirty primary glioblastomas. To the best of our knowledge, the investigation of 5 hmC genomic distribution in glioblastoma is novel. Use of 5 hmC and 5 mC profiling has permitted us to observe that the glioblastoma genome is relatively depleted of 5 hmC when compared with healthy cortex tissue, a finding consistent with studies that used less sensitive techniques37414243Previous efforts to characterize genome-wide DNA methylation in glioblastomas using the 450 K DNA methylation arrays have demonstrated an association between an altered epigenetic state and glioblastoma tumour biology5374243EGFR, PTEN, NF1, PIK3R1, RB1, PDGFRA and QKI possessed high 5 hmC levels across intronic regions and further loss of 5 hmC in tumours may reflect a loss of genome integrityGenome-wide loss of 5 hmC in cancer has been a widely observed, yet poorly characterized biological phenomenon1637et al.222649505152Not considered by previous studies is the possibility that cancer cell conservation of 5 hmC may also be critical for glioblastomagenesis and tumour growth as suggested by Takai In summary, we have generated nucleotide resolution maps of 5 hmC in thirty glioblastomas and linked 5 hmC with gene regulatory regions. Our results demonstrate that the glioblastoma genome exhibits a global loss of 5 hmC compared with healthy prefrontal cortex tissues, but observed regions of conserved 5 hmC implying novel associations between 5 hmC and critical tumour transcriptional programs. Our study also highlights that 5 hmC may, in a manner similar to 5 mC, serve as marker of cellular identity and that genomic patterns reflect cellular states in tissue samples. Thus, methods presented here that defined associations between 5 hmC distribution and genomic features may be more broadly applicable to other diseases as well as characterization of 5 hmC function in non-diseased tissue.Pathologically confirmed fresh frozen glioblastoma specimens from 30 subjects diagnosed at Dartmouth Hitchcock Medical Center  between 2004 and 2012 were identified for study. All subjects provided written informed consent at the time of surgery for use of their tumour specimens in research as approved by the committee for protection of human subjects . Subject demographic, tumour characteristic and survival follow-up were available for study and all tissues accessed were from deceased subjects. This work was performed in accordance with the ethical principles in the Declaration of Helsinki.Tumour DNA was extracted with the QIAmp DNeasy tissue kit (Qiagen) according to the manufacturer's instructions. DNA quantity and quality was assessed with the Qubit 3.0 fluorometer (Life Technologies). Sample DNA was subjected to tandem BS and oxidative BS (oxBS) conversion with an input of 4 μg per sample using the TrueMethyl kit v.1.1 (Cambridge Epigenetix) protocol optimized for Illumina HumanMethylation450 arrays. Prior publications have validated 5 hmC values in human brain tissue by oxBS-independent approaches and have demonstrated that replicate samples treated with BS–oxBS display a high level of reproducibilityIDH1 (R132) and IDH2 (R140 and R172) mutation status using PCR amplification and Sanger sequencing. Briefly, 10 ng of genomic DNA was amplified with primers for each of IDH1 spanning codon 132: F-GGTGGCACGGTCTTCAGAG, R-ATGTGTTGAGATGGACGCCT, and IDH2 spanning codons 140 and 172: F-TTCTGGTTGAAAGATGGCG, and R-GGATGGCTAGGCGAGGAG. Reaction conditions included a denaturation at 94 °C for 2 min followed by 35 cycles of 94 °C for 30 s, 62 °C for 30 s and 68 °C for 1 min with extension at 68 °C for 5 min as previously publishedAll glioblastomas were sequenced for FunNorm procedure available in the R/Bioconductor package minfi (version 1.10.2)SBS, RBS, SoxBS, RoxBS), with Rk representing total signal (unmethylated+methylated) and representing methylated signal ; we used maximum likelihood to fit the data generating model:All data analysis was conducted in R version 3.1.2. Normalization and background correction of raw signals from each of the BS and oxBS converted samples was achieved using the πj>0 , π1+π2+π3=1under the constraints π3<10−16). R code for maximum likelihood estimation as applied to 450 K arrays is available in the R-package ‘OxyBS'. Since the raw intensity data (IDAT) files were unavailable for the prefrontal cortex 5 hmC data set (GSE74368) we processed the samples using the alternative naïve subtraction method outlined in Houseman et al.n=387,617).The resulting estimates parameters (unmethylated proportion), (5 mC proportion) and (5 hmC proportion). Note that this method explicitly disallows negative proportions, although we did observe numerically zero values of 5 hmC bona fide 5 hmC. In our analysis, we defined the locations within glioblastoma that have the 1% highest mean 5 hmC level to be considered high 5 hmC CpGs. Next, each CpG was assigned a classifier for high 5 hmC , putative enhancer regions, open chromatin, Polycomb group target genes, and gene regions  as well as the CpG's relationship to a CpG island . Typically, a Fisher's test is used to test for enrichment of these features; however, we decided to use the Cochran–Mantel–Haenszel test as this approach permits stratification by CpG island probe type, which we demonstrated has a substantial impact of the level of 5 hmC.We moved to delineate the level at which we could confidently call a genomic location as possessing n=24). Candidate gene transcripts of interest are presented in DNMT1),RGMA 5′UTR), (iii) genes with a known relation to glioblastoma pathobiology  and (iv) glioblastoma housekeeping genes. The digital multiplexed NanoString nCounter for custom code set was performed according to manufacturers' instruction with total RNA. Data normalization was performed using the nSolver Analysis software  with initial positive controls used to normalize all platform associated sources of variation and reference gene normalization was performed using housekeeping genes .Tumour RNA was extracted with the RNeasy Mini tissue kit (Qiagen) according to the manufacturer's instructions. Insufficient RNA was available for gene expression from six tumours . RNA quaTo examine whether high 5 hmC CpGs were associated with specific gene sets we used the GREAT software and to query whether 5 hmC was associated with TFBSs in ENCODE we used the LOLA software33The RefFreeEWAS algorithm (R-package RefFreeEWAS) is a method for the reference-free deconvolution that provides proportions of putative cell types as defined by their underlying methylomesEGFR gain, CDKN2A loss, chromosome 7 gain and chromosome 10 loss, we used the Bioconductor package CopyNumber450 K and confirmed copy number gain in samples where EGFR expression was available.To determine common copy number alterations in primary glioblastomas including: We defined regions of the genome  using the UCSC_hg19_refGene file from the UCSC Genome Browser and collapsed 5 hmC and 5 mC around the promoter region using the genomation bioconductor packagen=160 for splicing), all splice events , and software specific filters of Percent Splice In value  of 75% and a minimum range of 10 PSI values to identify variable splicing events across tumours.Level 3 normalized RNAseqV2 expression levels from TCGA were binned into tertiles based upon mean expression across all samples. Glioblastoma mRNA splicing data was downloaded from the TCGASpliceSeq database . The data was downloaded from the database with the following filters: TCGA disease type was set as glioblastoma, all genes were considered, all TCGA glioblastoma samples were considered . Five prefrontal cortex samples from individuals with no evidence of neurological impairment as described in Lunnon et al.et al. (GSE51522)et al. (GSE54792)http://cancergenome.nih.gov/) and processed using the minfi Bioconductor packagehttp://cancergenome.nih.gov/). All remaining data is available within the article, The glioblastoma 5 hmC and 5 mC DNA microarray data that support the findings of this study have been deposited in the Gene Expression Omnibus (GEO) with the accession code GSE73895 .Supplementary Figures 1-8 and Supplementary Tables 1 & 3List of CpGs with the highest levels of 5-hydroxymethylcytosine in glioblastoma ).Complete LOLA results produced from testing enrichment of high 5hmC location for binding sites in 689 different ChIP-seq experiments (ENCODE) against the background of the 450K genomic locations.High 5hmC CpGs collapsed by UCSC Ref Gene Group. The total number of CpGs found in a gene region and the proportion of CpGs in that region determined to be high 5hmC are presented.Normalized gene expression for candidate genes using NanoString nCounter platform. Initial positive controls were used to normalize all platform-associated sources of variation and reference gene normalization was performed using housekeeping genes.Spearman Correlations between NanoString gene transcript variant expression and CpG-specific 5hmC (n=119).Spearman Correlations between NanoString gene expression and CpG-specific 5hmC and 5mC (n=23).


Cancer research continues to highlight the extensive genetic diversity that exists both between and within tumors. This intrinsic heterogeneity poses one of the central challenges to predicting patient clinical outcome and the personalization of treatments. Despite progress in some individual tumor types, it is not yet possible to prospectively, accurately classify patients by expected survival. One hypothesis proposed to explain this is that the prognostic classifiers developed to date are insufficiently sensitive and specific; however it is also possible that patients are not equally easy to classify by any given biomarker. We demonstrate in a cohort of 45 AJCC stage III melanoma patients that clinico-pathologic biomarkers can identify those patients that are most likely to be misclassified by a molecular biomarker. The process of modelling the classifiability of patients was then replicated in a cohort of 49 stage II breast cancer patients and 53 stage III colon cancer patients. A multi-step procedure incorporating this information not only improved classification accuracy but also indicated the specific clinical attributes that had made classification problematic in each cohort. These findings show that, even when cohorts are of moderate size, including features that explain the patient-specific performance of a prognostic biomarker in a classification framework can improve the modelling and estimation of survival. The biological complexity of most disease states limits the performance of prognostic biomarkers –3. ThereHerein we propose one such model capable of retrospectively identifying clinical and pathological features that identify and explain which patients are more likely to be successfully separated according to a good and poor prognosis by gene expression data. By including this identification step in a classification procedure we then show that we can substantially improve the accuracy of predicting prognosis in three cancer datasets; 45 AJCC stage III melanoma patients , 49 stagWe previously observed that there were subsets of AJCC stage III metastatic melanoma patients whose prognosis, death within a year of resection or survival for more than four years after resection, was more easily predicted than others for a variety of biomarkers . This inGiven that the gene expression and clinical data might be detecting different prognostic signals for different patients, two standard methods for integrating gene expression and clinical data were applied to the melanoma data and two additional cohorts in separate cancers: breast  and coloWe then took a different approach, hypothesizing that there would be clinical variables associated with the classification accuracy of the gene expression biomarker. Extranodal spread was identified as the most informative clinical variable for accurate classification of melanoma patient outcome by the biomarker constructed with gene expression data. Figure We therefore propose a new multi-step classification framework that includes modelling of gene expression biomarker performance, and which markedly improved estimation of prognosis in three different cancer types Figure . First, We further analyzed why classification performance in each of the three cancer cohorts could be improved by the multi-step approach. Specifically, we aimed to investigate whether there may simply be multiple signals in the data with one signal overpowering the others. Figure We have shown that by conceptualizing error rates as a measure of classifiablilty, we can identify clinical determinants of this classifiability and then leverage these determinants to construct superior classification models. Importantly, none of the key clinical determinants could have been selected a priori as most likely to confound classification. That is, pre-determining potential confounding variables before an experiment without sufficient evidence and using these to subset a cohort might have unnecessarily reduced power or further confounded the analysis , 11, 12.Our results imply that resources should be invested into both the measurement of gene expression and the collation of detailed clinical information to improve the accuracy of prognostic estimates for cancer patients. Both the clinical and gene expression data contain useful prognostic information, however, these two classes of information have different relevance to different patients. The pivotal message from Figure Interestingly, standard methods for integrating gene expression and clinical data did not perform competitively on these datasets. This was surprising since, in the melanoma cohort, both the gene expression and clinical data classified different patients correctly suggesting that integrating the two would be superior. Nonetheless, the question remains: why did the two integrative approaches not provide more substantial improvements to classification performance in these three datasets? To begin with, since PV is simply a linear combination of the clinical variables and a biomarker built from the gene expression, it might not be capable of capturing the inherent complexity of the data. In contrast, RF is a non-linear classification method built using decision trees making it capable of describing complex models, but despite its use of resampling methods it may still be over-fitting to the data .To conclude, we have demonstrated that by identifying clinical variables that confound gene expression biomarker performance in individual patients we were able to improve the prediction of patient clinical outcome in three cancer cohorts. The value in this finding is that it is not always clear which sources of heterogeneity might be confounding a discriminatory signal within a dataset, and this is especially true in complex diseases such as cancers. Our multi-step procedure identified the clinical variables that were most likely to obscure a potentially useful gene expression signal of interest. It therefore has strong potential to help build models of outcome that have greater translational relevance, including aiding clinicians in determining the most appropriate schedule for clinical follow up of cancer patients, through substantial improvements to their accuracy. There is also potential value in the cost-effectiveness of this approach, which begins by leveraging the clinic-pathologic information that is immediately available to the clinician in order to enhance the value of molecular biomarkers that are measured with more difficulty and at higher cost.BRAF and NRAS mutation status, as described in [Melanoma_GoodPrognosis=22), and patients who died of melanoma within 12 months of resection (nMelanoma_PoorPrognosis =23). We considered the following 11 clinico-pathologic variables in our analysis: patient age (at banking), patient sex, size of nodal metastatic tumor (mm), extranodal spread (present vs. absent), number of metastatic nodes, primary melanoma cell shape , percentage of necrosis and degree of pigmentation. BRAF and NRAS mutation status were identified via the Sequenom OncoCarta v1.0, MelaCarta v1.0 platform followed by MassARRAY25 mass spectroscopy, as previously described [We used the previously reported global mRNA expression profiles from 45 AJCC stage III metastatic melanomas (deposited in GEO Accession Number: GSE54467), and linked detailed clinico-pathologic data, including We used previously reported global mRNA expression profiles of human breast cancer samples together with clinical history   Feature selection, selecting the genes to include in a model, was performed by selecting the one thousand genes with largest fold change; 2) Using the class information, a diagonalized linear discriminant analysis (DLDA)  was traiClassifiers using the clinical data were built using logistic regression in conjunction with step-wise AIC variable selection. While logistic regression was chosen here for illustration, in further application of our multi-step framework any classifier can be used instead. Step-wise AIC was chosen as it is the default model selection choice for many logistic regression packages. Other choices are possible such as using step-wise methods with other information criteria . HoweverWe used a pre-validation approach  to integIntegration of gene expression and clinical information to predict patient clinical outcome was also performed using a random forest  classifiClassification error rates for individual patients were calculated by performing LOOCV of a classification scheme on the gene expression data. In general, such errors could alternatively be calculated using bootstrap, k-fold cross-validation or repeated k-fold cross-validation. Logistic regression was then used, treating the prediction errors as a response and using the clinical variables as explanatory variables. The clinical variable with smallest AIC was selected to divide the patients into two cohorts; those patients who could be classified with the gene expression data (referred to herein as ‘easy-to-classify’) and those who could not (‘hard-to-classify).If a clinical variable was identified as having the potential to explain the heterogeneity in the gene expression data there may me some question as to whether the samples within any of its levels contained signal on a given platform. To test if this was the case, the samples were divided by the levels of the clinical variable . Within these splits, a leave-2-out cross-validated classification scheme was independently performed.http://www.ellispatrick.com/classifiabilitycode.First, the clinical variables were used to predict which samples the gene expression data could and could not classify in the LOOCV. Using this prediction, the samples were divided into ‘easy-to-classify’ and ‘hard-to-classify’ subsets. A new classifier was built on the gene expression data for the patients in the former group while the clinical variables were used to construct a classifier for patients in the latter . This approach was implemented at the level of individual patients wherein for each case the clinical variable was first used to predict whether the gene expression data would be informative and then, depending on the result (hard or easy), the appropriate downstream classifier was then subsequently applied to predict an individual's prognosis class. This process is further illustrated in Figure


In high-throughput molecular profiling studies, genotype labels can be wrongly assigned at various experimental steps; the resulting mislabeled samples seriously reduce the power to detect the genetic basis of phenotypic variation. We have developed an approach to detect potential mislabeling, recover the “ideal” genotype and identify “best-matched” labels for mislabeled samples. On average, we identified 4% of samples as mislabeled in eight published datasets, highlighting the necessity of applying a “data cleaning” step before standard data analysis. With the development of a wide range of high-throughput molecular profiling methods in recent years, there has been significant growth in the number of genetic studies on molecular profiles from genetically different individuals –6, aiminA wrong genotype assignment will seriously weaken the significance testing  in genetreGenotyper, which implements a fast and accurate algorithm to identify samples that are likely to have been mislabeled, based on the measured phenotypes  from a variety of other population types.For sake of simplicity, we will showcase our algorithm on transcriptome data from a population of recombinant inbred lines (RILs). RILs are homozygous individuals that result from repeated self-mating or sibling mating, starting from an Ft = tnew −told < 0, when using the t statistic) for all molecular traits mapping to a QTL near that marker  for all molecular traits mapping to a QTL near that marker  from one sample can actually be used to evaluate the chance that a sample is mislabeled. reGenotyper also provides an estimate of the “ideal” genotype of the potential mislabeled sample and identifies the best-matched labels for each sample, from a collection of potential genotypes. The mathematical details of the algorithm and the significance assessment are provided in the The reGenotyper algorithm works as follows: perturbing the genotype at a particular marker of a correct sample (correct → wrong) will lead to a decreased QTL significance (e.g. Δt marker . This wot marker . This woC. elegans datasets. The recombinant inbred lines (RILs) used in these studies were derived from C. elegans wild-types N2 and CB4856. In the temperature experiment [www.wormqtl.org] [www.wormqtl-hd.org]   or WormQ-hd.org]  web port18 = 3.8×10−6. With increasing dissimilarity between the two genotypes, the performance increases to an AUC of almost 100%, i.e. a perfect detection of all wrongly labeled samples.The ability of reGenotyper to detect wrongly labelled samples will obviously depend on how similar the assumed (wrong) genotype is to the real genotype. We first evaluated this effect through simulation using the area under the (receiver operating characteristic) curve (AUC) approach. Specifically, we studied the performance of the method when the original true genotypes showed different proportions of common marker genotypes with the mistakenly used genotype. The more similar the mistakenly used genotype is to the correct genotype, the more difficult it should be to identify the sample as mislabeled. In C. elegans datasets, as described in the Test Datasets subsection of the Materials and Methods   . This re studies ,26.As we had identified a number of mislabeled samples in the worm datasets affecting the power to detect eQTLs, we applied our algorithm to several publicly available datasets for which the genotype and transcriptome data were available online.C. elegans [Among 208 recombinant inbred advanced intercross lines from  elegans , four sa elegans  and Worm elegans  portals.Among 96 recombinant inbred lines from mouse , two samNone of the yeast segregants  was deteIn each case, a number of additional samples were highlighted as suspicious and could be followed up experimentally when resources permit.The simplest way is removing those unreliable samples from the dataset and performing the further analysis based on the remaining samples. This most conservative approach would remove dubious information from the original data, but it also leads to a decrease in the sample size and a concomitant reduction of statistical power. We performed a simulation study to show an effect of removing the samples. We used the median of the detected QTL significance distribution as a benchmark score. We removed samples in order of their chance of being mislabeled (i.e. according to their mislabeling score). This increased the benchmark score, until the point of removing samples with little evidence of being mislabeled (i.e. correct samples). .A possibly more efficient way of handling the mislabeled data would be to recover the true genotypes for the mislabeled samples. Not discarding molecular profiling information of the mislabeled samples could potentially lead to obtaining even more power for QTL detection. For example, in a simulated genetical genomics experiment . MixupMapper aims to detect pairs of samples for which the genotype information has been mistakenly swapped, but both genotypes have been measured and both samples have been included in the experiment. Thus, this method cannot find mislabeled samples that result from slightly more complex (but still very likely) experimental mix-ups, e.g. the use of wrong samples from a larger collection, or accidental duplicate measurements. Additionally, a Bayesian approach was reported to predict SNP genotypes based on RNA expression data, and then to match the predicted genotype to the observed genotype of individuals in large populations [cis-eQTL across different tissues and therefore requires training data from one or more independent tissues; unfortunately this kind of replication is not available for most studies. Moreover, all of the above methods depend on eQTL, which limits their usage to gene expression studies, whereas our reGenotyper method could in principle be applied to any type of high-throughput phenotype data, including the increasingly popular and powerful genetic analysis of protein and metabolite profiles [C. elegans data) to hours (human data) on a personal computer. However, we recommend use of parallel computing (e.g. cloud services) so that more permutations can be performed increasing accuracy of reGenotyper.To our best knowledge, there is no other tool capable of such a task. Even though there has been numerous attempts to address sample mix-ups detection ,30–33. Aprofiles ,5. The tThe package was built so that detection of sample mix-ups is performed quickly and accurately for large numbers of phenotypes. However, our heuristic “significance change value” approach is only valid if phenotype data is of massively parallel nature. Moreover, the package needs numerous phenotypes with considerable number of highly significant QTLs in order to perform meaningful permutation analysis. This limits reGenotyper usage to the high-throughput molecular studies where measuring thousands of phenotypes is feasible.On the other side of the spectrum are studies generating truly big data. For example, genomic studies in humans utilize tens of millions of SNP markers . PermutiThe accuracy of sample mix-up detection may also be affected by gene—environment interactions, as environmental variation induces major changes in phenotype that might not be linked to genetics. This can result in discovering false mix-ups. On the other hand, measuring phenotypes in multiple environments allows untangling environmental and genetic components of observed phenotypical variation . AnalyziWe aimed at as much automation of the workflow as possible. However, there is still a crucial step that requires user intervention. Samples are marked as mislabeled based on a user-defined threshold. Even though the threshold is statistically well-defined see , the choS1 FigExperiment without a mislabeled sample (a) and with mislabeled samples (b). Grey and red correspond to values of 0 and 1 for the S element value, respectively. The rows represent markers along the genome and the columns represent different samples. (a) S values of 1 (red spots) are scattered in the figure, indicating that no sample has been mislabeled. (b) Vertical red bars indicate that the corresponding samples (columns) are very likely to have been mislabeled. In this simulated experiment, six (~10%) mislabeled samples sharing about 50% common marker genotypes with the correct sample were included, and these were all clearly identified.(PNG)Click here for additional data file.S2 Figa) Comparison of the distribution of Δt for a correctly labeled sample (green) and a wrongly labeled sample (magenta) from a simulated dataset. The wrongly labeled sample and the true genotype are 70% identical. b) Box plot of the area under the receiver operating characteristic curve (AUC) for the reGenotyper algorithm for different scenarios. The x-axis represents ten different situations, i.e. different proportions of common marker genotypes between the mislabeled sample and the true genotype of this sample; in the most challenging scenario, the mislabeled sample shares 95% of its marker genotypes with the true sample. Under each scenario, 50 simulations were carried out. In each simulation, a gene expression dataset of 100 samples with 10% mislabeled samples was simulated.(PNG)Click here for additional data file.S3 FigSample across seven different experiments (rows) from our lab using the reGenotyper algoritm. Different shades of green gradients represent the mislabeling score. Darker green corresponds to higher scores, while magenta indicates that the sample was detected as MS with a score larger than 90%. White indicates that the sample was not selected for use in this experiment.(PNG)Click here for additional data file.S4 FigFigure compares results of QTL mapping on original data (left) and after removal of 5 MS (right). Removal of MS detected by reGenotyper results in sharp increase in number of QTLs shared between two temperatures used in the study (from 93 to 190–111%).(PNG)Click here for additional data file.S5 FigRespectively, 4 out of 208, 2 out of 96 and 0 out of 109 samples show a mislabeling score >0.9 (dashed line).(PNG)Click here for additional data file.S6 FigThe genotype and phenotype data were simulated for 60 RILs in the same way as described in the (PNG)Click here for additional data file.S7 Fig200 simulations were performed and QTL mapping was performed a) before swapping samples to simulate mislabeling, b) after swapping samples and c) after swapping samples and recovering true genotypes using reGenotyper. Plots shows rate of: red 1 –phenotypes with simulated QTL that was mapped; blue 2 –phenotypes without simulated QTL, where no QTL was mapped; green 3 –phenotypes without simulated QTL, where QTL was mapped. After recovery with reGenotyper almost all of the QTLs could be mapped correctly, compared to 77% in case with MS.(PNG)Click here for additional data file.S1 File(DOC)Click here for additional data file.


Though Illumina has largely dominated the RNA-Seq field, the simultaneous availability of Ion Torrent has left scientists wondering which platform is most effective for differential gene expression (DGE) analysis. Previous investigations of this question have typically used reference samples derived from cell lines and brain tissue, and do not involve biological variability. While these comparisons might inform studies of tissue-specific expression, marked by large-scale transcriptional differences, this is not the common use case.Here we employ a standard treatment/control experimental design, which enables us to evaluate these platforms in the context of the expression differences common in differential gene expression experiments. Specifically, we assessed the hepatic inflammatory response of mice by assaying liver RNA from control and IL-1β treated animals with both the Illumina HiSeq and the Ion Torrent Proton sequencing platforms. We found the greatest difference between the platforms at the level of read alignment, a moderate level of concordance at the level of DGE analysis, and nearly identical results at the level of differentially affected pathways. Interestingly, we also observed a strong interaction between sequencing platform and choice of aligner. By aligning both real and simulated Illumina and Ion Torrent data with the twelve most commonly-cited aligners in the literature, we observed that different aligner and platform combinations were better suited to probing different genomic features; for example, disentangling the source of expression in gene-pseudogene pairs.Taken together, our results indicate that while Illumina and Ion Torrent have similar capacities to detect changes in biology from a treatment/control experiment, these platforms may be tailored to interrogate different transcriptional phenomena through careful selection of alignment software.The online version of this article (doi:10.1186/s12864-017-4011-0) contains supplementary material, which is available to authorized users. RNA-Sequencing (RNA-Seq) broadly refers to a family of experimental techniques that give researchers the ability to study the transcriptional landscapes of cells and tissues quantitatively by exploiting high throughput sequencing technology. Currently, the most commonly used sequencing platforms are provided by Illumina, which uses a fluorescence-based paradigm for reading the bases in a nucleotide sequence. One alternative option is provided by Ion Torrent, which is built around the use of pH measurements to read nucleotide sequences. In addition to the distinct sequencing technologies used by these two platforms, there are smaller differences in the types of data they generate. In Illumina data all sequence reads generated during a single experiment have the same lengths, while the lengths of Ion Torrent reads vary. Additionally, the current generation of Illumina instruments can generate sequence reads from both ends of a fragment (“paired-end” reads), while Ion Torrent cannot.Prior studies have compared these two sequencing platforms for various applications including genome sequencing, RNA-Seq, and microbiome profiling –5. TheseFirst, in the intervening years both Illumina and Ion Torrent have released platform updates in the form of improved sequencing chemistry, nucleotide detection, and throughput. With the rapid pace of development in these sequencing technologies, we must continue to compare these two platforms in order to maintain an accurate understanding of their relative performances. This continued re-assessment is of particular importance as researchers adapt both of these sequencing platforms for use in the clinical setting , 7.Second, the previous studies that focused on comparing the Illumina and Ion Torrent platforms for the purposes of RNA-Seq expression analysis used the Universal Human Reference RNA (UHRR) as source RNA , 8, 9. TArguably, one of the most common designs for RNA expression profiling experiments involves a single tissue or cell type, and varying genotypes or treatments. Generally, the transcriptional differences we expect to see between “treatment” and “control” conditions are more subtle than those we see between universal reference samples. Therefore, in order to evaluate the two platforms in the context of this end point, we used the Illumina and Ion Torrent sequencing platforms to assess the effects of IL-1β treatment on the mouse liver transcriptome. We evaluated these platforms at three levels of complexity: 1) individual read alignments and expression quantification, 2) differential expression detection, and 3) pathway-level analysis. Here we seek to determine if/how choosing between the Illumina and Ion Torrent sequencing platforms will affect the biological conclusions a researcher derives from the data, for which concordance at the pathway level is most relevant.n = 5) or saline , and then collected liver samples from these mice four hours after treatment   range for display purposes. (PDF 1099 kb)Additional file 18: Figure S9.GC-content of DEGs. Density plots, for each aligner, of the % GC content for DEGs identified by both platforms (red), Illumina only (blue), Ion Torrent only (green), and non-DEGs (black). (PDF 99 kb)Additional file 19: Table S9.Ensembl biotypes of genes detected by both platforms, or exclusively by one platform. For each aligner, lists the number of and percent of detected genes for each ensembl biotype. These numbers are broken down by those genes detected in both platform, only in Illumina data, and only in Ion Torrent. (XLSX 12 kb)Additional file 20: Figure S10.Mup20 and Mup-ps22. Bargraphs display average expression across samples in each treatment group. Gapdh expression is used as the endogenous control. Error bars display the squared-error of the mean (SEM). (PDF 16 kb)qPCR results for Additional file 21: Figure S11.Mup20 or Mup-ps22. (PDF 110 kb)Using a full dataset to examine platform/aligner interaction. For each sequencing platform, full fastq files from sample 9574 (untreated) were re-aligned using STAR + Bowtie2 and the twelve most popular aligners, according to a survey of the literature. This figure displays the number of uniquely-mapped (top) and multimapped (bottom) reads aligned to Additional file 22: Figure S12.Serpina3c or Serpina3i across all samples were extracted. These data were re-aligned using STAR + Bowtie2 and the twelve most popular aligners, according to a survey of the literature. Additionally, simulated RNA-Seq reads were generated from both of these genes. A) The average number of uniquely-mapped (top) and multimapped (bottom) reads aligned to each gene across the untreated and IL-1β-treated samples. B) qPCR results for Serpina3c and Sperina3i. C) Log2 fold-change differences between the average expression in the IL-1β-treated and untreated samples. (PDF 269 kb)Using simulated data to examine platform/aligner interaction. For each sequencing platform, all reads aligned by GSNAP, or STAR + Bowtie2 to