Motivation: Over the recent years, the field of whole-metagenome shotgun sequencing has witnessed significant growth owing to the high-throughput sequencing technologies that allow sequencing genomic samples cheaper, faster and with better coverage than before. This technical advancement has initiated the trend of sequencing multiple samples in different conditions or environments to explore the similarities and dissimilarities of the microbial communities. Examples include the human microbiome project and various studies of the human intestinal tract. With the availability of ever larger databases of such measurements, finding samples similar to a given query sample is becoming a central operation.Results: In this article, we develop a content-based exploration and retrieval method for whole-metagenome sequencing samples. We apply a distributed string mining framework to efficiently extract all informative sequence k-mers from a pool of metagenomic samples and use them to measure the dissimilarity between two samples. We evaluate the performance of the proposed approach on two human gut metagenome datasets as well as human microbiome project metagenomic samples. We observe significant enrichment for diseased gut samples in results of queries with another diseased sample and high accuracy in discriminating between different body sites even though the method is unsupervised.Availability and implementation: A software implementation of the DSM framework is available at https://github.com/HIITMetagenomics/dsm-framework.Contact:sohan.seth@hiit.fi or antti.honkela@hiit.fiSupplementary information:Supplementary data are available at Bioinformatics online. Metagenomics is the study of microbial communities in their natural habitat using genomics techniques  sequencing data is very challenging. The original sample typically contains genetic material from hundreds to thousands of bacterial species of different abundances  decide if a k-mer is informative or uninformative in the context of a retrieval task, (iii) compute a distance metric using the filtered k-mer frequencies, and (iv) execute these steps fast without explicitly storing the frequency values. To summarize, we introduce methods to (i) estimate the frequencies of a large number of k-mer w over samples k-mers. These are distributions over samples for each k-mer that are computed independently for each k-mer for reasons of computational efficiency.To perform the feature selection or filtering, we first compute Bayesian estimates of the relative frequencies k-mer is the same in every sample, the observed frequencies may differ because of different sequencing depth or coverage in different samples. To tackle this issue, we employ normalization: we normalize the frequency s), which is proportional to the total number of base pairs in a sample, and s) can be interpreted probabilistically as the probability of observing a sequence in the actual sample, assuming every sample had the same number of base pairs to start with, but some have been lost in the processing.Even if the relative abundance of a k-mer counts for speed and ignores any k-mers that are present only once in a sample. The pseudocount from the prior makes up for this missing count. Second, adding pseudocounts assists in playing down the significance of rare k-mers that may appear due to sequencing errors in the filtering step without affecting other k-mers too much.To estimate the relative frequencies, we place a conjugate symmetric Dirichlet prior on the parameters of the multinomial distribution over the observed counts. The common choice of uniform prior distribution corresponds to a Dirichlet distribution with all parameters equal to 1. This yields a posterior mean estimate of the relative frequency values ask-mers, it is crucially important to improve signal-to-noise ratio by focusing on the informative ones. For the unsupervised tasks of comparing the samples, obviously only k-mers that distinguish between the samples are informative. As a concrete example, consider a k-mer that is present in all samples with a similar abundance. It certainly does not give information useful for comparing samples. In the other extreme, if a k-mer is present in one specific sample, but not in any other, it is potentially a spurious k-mer due to sequencing error, and in any case does not help in comparing samples either. On the other hand, if a k-mer is present in some samples, but not all, then it gives information that those samples are similar in a specific sense. Informativeness in this sense can be measured by the entropy H of the distribution of the k-mer over the samples: we filter the k-mers based on the conditional entropiesk-mer is taken into account in distance computation only if the normalized entropy is lower than a certain threshold e. By design information theory terminology higher entropy implies higher information. However, in our context an informative k-mer has low entropy. Also, due to the Bayesian estimation, a spurious k-mer having only small counts will have large conditional entropy and will be filtered out.Finally, given the massive number of potential e varies with datasets. It can be ‘optimized’ in a supervised manner by utilizing a training set where we have labeled samples. In the absence of a labeled set, we suggest taking the ‘average’ of distance metrics computed over the potential thresholds as the final metric. We refer to the final metrics in the two cases as optimized metric and average metric. In our experimental set-up, we randomly make a 50–50 split of a given dataset in training S  or in a distributed manner (i.e. a computer cluster). We review two approaches: k-mer counting  k. In both tools, space efficiency is achieved by keeping most of the hash table on disk. The main drawback with these disk-based approaches is that they are aimed at counting k-mers in a single sample and extending them over to multiple samples is non-trivial. For example, Jellyfish could, in principle, be extended to count k-mers over multiple samples: the authors give a roughly linear time algorithm to merge two or more hash tables. However, the intermediate k-mer counts would need to be stored on disk, which requires significant amount of additional space, and the merge phase is not parallelized  there is no additional disk I/O strain, except reading through the input once. These advantages allow terabyte-scale data analysis on a cluster having limited main memory per node, but a sufficient number of nodes available to facilitate the computation in distributed manner. We extend the DSM framework to be compatible with our definition of informative k-mers (see the above subsection). It allows us to extract the informative k-mers either for a fixed k or over all values of k in feasible time.The decision whether a particular ramework  that cansuffix sorting techniques and on space-efficient data structures for strings  left–right-branching substrings, and the total length of all such substrings is bounded by The DSM framework is based on a client-server model. The clients have one-to-one correspondence to the given samples, each client being responsible for computing the frequencies within the designated sample. The client-side computation relies heavily on  strings : the inpk-mer w, having frequency k-mers can be deduced from the branching k-mers. The second observation guarantees a feasible transmission cost between clients and servers: the upper bound for the concatenation of all left–right-branching substrings also acts as an upper bound for both the server-side running time and the amount of communication needed. The drawback of restricting to left–right-branching substrings is that the informative k-mers that we are able to detect have to appear at least twice in a sample, although this limit may be useful in pruning spurious k-mers introduced by sequencing errors. More detailed explanation and analysis of the DSM framework is given in https://github.com/HIITMetagenomics/dsm-framework.The first observation allows us to reduce the client-side computation to a smaller set of substrings: it is easy to see that if k-mers, we use them to compute the dissimilarity between two metagenomic samples. We consider three dissimilarity metrics that can be computed easily over a large number of k-mers in sequential manner, i.e. one k-mer at a time, and without storing all the k-mer frequencies explicitly. To utilize the natural variance structure of the k-mers—some are more abundant than others—we weight the relative frequencies of each k-mer by their respective total counts, i.e. we utilize the absolute frequencies Having extracted the informative k-mer occurs or not. Given two sets s1 and s2 of k-mers detected as present in two different samples, Jaccard distance measures how many elements are shared between these two sets. Mathematically, it is defined ask-mer is present in a sample if its frequency is >2.We mainly use the simple Jaccard distance that does not consider abundances at all, only whether a We also experiment with two metrics that use the abundance information:Variance-stabilized Euclidean distance: An obvious distance measure between two metagenomic samples s1 and s2 is the Euclidean distance between their respective k-mer frequencies. We consider the distance metrick-mers are extracted. The square root transformation is the variance stabilizing transformation for Poisson distribution—a popular model for quantitative sequencing data.I. Log transformed Euclidean distance: We also consider the same metric but with log transformation, which is a popular approach in document retrieval, i.e.k-mers are present in high abundance in almost every genome, for instance k-mers from the marker gene, and the log transformation reduces their effect in the metric.II. We evaluate the performance of the dissimilarity metric in terms of its performance in the task of retrieving relevant samples given a query metagenomics sample. The ground truth for relevance is either the disease class (disease versus not) or the known body site: samples from the same class are considered relevant.q, the retrieval method ranks the samples in an increasing order of their dissimilarities from q. Given one has retrieved the top (closest) n is defined asaverage precision as,Q is the set of all queries, qm is the number of relevant samples to query q and qR is the set of locations in the ranked list where a relevant sample appears. It is straightforward that a higher MAP implies better performance. To judge if two MAP values are significantly different or not, we employ the randomization test described in P-value. In case two samples share the same dissimilarity from a query sample, we employ the modification suggested in For measuring retrieval performance, we use an evaluation metric which is popular in document retrieval, the mean average precision (MAP)  the same parameters were used for both classes and for the other half of the species the parameters were randomly permuted. For example, given five species the assigned parameters could be:  and  where the parameters for the second and fourth species are the same, but for the other species they were permuted. The exact species and corresponding parameter values can be downloaded from https://github.com/HIITMetagenomics. The resulting datasets are—HIGH-C, relatively easy data with high coverage (10e6 reads/sample); LOW-C, relatively difficult data with low coverage (2e6 reads/sample); MIXED-C, mixed data with half the samples from HIGH-C and the rest from LOW-C to simulate varying sequencing depth; and HIGH-VAR, relatively difficult data with same coverage as HIGH-C but additional noise in the class distributions to simulate more overlap between classes. To elaborate, the relative abundance of species is noise is generated from a symmetric Dirichlet distribution with all parameters equal to 1.To test the method, we simulated four datasets containing samples from separate classes, with the interpretation that samples from the same class are relevant. In all the datasets we have two classes: both classes of samples have the same species composition but different relative abundances. We used MetaSim (MetaHIT (T2D Phase II (Supplementary Table S1). Of 690 samples that passed the QC assessment (http://www.hmpdacc.org/HMASM/), we discarded 255 samples that had <1% of the number of reads of the largest sample.HMP , 435 metWe evaluated the retrieval performance on three human metagenomics datasets:k-mers used by different methods for each dataset are available in the Supplementary Fig. S1.To recapitulate, for MetaHIT and T2D-P2, our goal is to observe if given a positive sample, e.g. from a patient with a particular disease, one can retrieve relevant samples, i.e. with similar disease; whereas for HMP, our goal is to observe if given a sample from a particular body site, one can retrieve relevant samples, i.e. samples from the same body site. For all data, we applied a quality threshold of 30 and ignored any base pairs with quality less than the threshold. Retrieval of samples with similar annotation: We applied the proposed approach and a number of alternatives for retrieval of similar samples from the same dataset and evaluated by how many of the retrieved samples had the same annotation: class label, disease state or body site. A comparison of the obtained MAP values averaged over queries by all positive samples is shown in For the simulated data, the two classes differ only by the relative species abundance; thus, retrieval based on ground truth abundance can be considered to give an upper limit for the performance. For HIGH-C and HIGH-VAR, the proposed method performs closer to the ground truth performance than any other methods, although the difference from ground truth performance is still statistically significant. For LOW-C, the performance of all methods, except the protein family based comparison, drop compared with HIGH-C, whereas for MIXED-C the performance is again close to HIGH-C despite the presence of low coverage samples. This is an encouraging observation showing the robustness of the proposed approach to varying sequencing depths.k-mer features can provide a better separation. Retrieval based on the known protein family performed fairly well, but slightly worse than the proposed approach on MetaHIT. We observe that for MetaHIT, Jaccard metric performs poorly; however, a change of metric to log significantly improves the performance for all methods. Otherwise, all metrics usually work equally well over different datasets.For the real datasets, the proposed approach yielded statistically significantly higher MAP than any of the alternatives . The figures show the complete distribution of average precision values over different queries whose mean is the MAP of k. Thus, the proposed method appears to be a relatively safe choice that does not suffer from catastrophically bad performance on any of the datasets.Effect of the entropy filtering: Next, we evaluated the efficacy of filtering the informative k-mers against retrieval performance without the filtering operation. The results are presented in Supplementary Fig. S3). We observed that entropy filtering usually improved retrieval performance for all tested k-mer lengths when using the ‘optimized metric’, although the improvement might not always be statistically significant. Although ‘average metric’ often provides significant performance, it might not always improve over performance without filtering. Also, retrieval performance of FIGfam may or may not improve with entropy filtering (‘optimized metric’ and ‘average metric’ selected in the same way as other methods).Comparison across different metrics: Finally, we evaluated the retrieval performance over different dissimilarity metrics. We presented the performance using ‘optimized metric’ for different metrics in Supplementary Fig. S4). We observed that the simple presence-/absence-based metric Dcount performed at least as well as abundance-sensitive log and sqrt metrics, except for the MetaHIT data for which the other metrics performed better.k-mer frequencies as feature representation; however, rather than exploring k-mers of a fixed k, we have scanned through all possible k-mers of all possible k’s using DSM, and have proposed appropriate filtering technique to discard uninformative k-mers. Being reference-free, our method is capable of focusing on novel sequence markers, such as ones identifying novel bacterial strains. On the other hand, sensitivity especially on low-coverage species may be lower than using reference-based approaches. Like most read-counting-based approaches, our method can be sensitive to technical variation and changing biases in the sequencing process. For most reliable results, the dataset should be measured as uniformly as possible. The distributed algorithm relies on dedicated cluster hardware having sufficient total amount of main memory. The development of an efficient secondary memory-based algorithm is an important topic of future work. The overall computational requirements scale in near-linear manner as described by In the wake of collecting multiple samples from similar environments, information retrieval for metagenomic samples is expected to become a handy tool in metagenomics research. In this article, we have addressed the problem of retrieving relevant metagenomic samples given a query sample from the same collection. The novelty of the proposed approach is that it is unsupervised, and does not rely on the availability of reference databases. We have suggested employing


High-throughput sequencing technologies, such as the Illumina Hi-seq, are powerful new tools for investigating a wide range of biological and medical problems. Massive and complex data sets produced by the sequencers create a need for development of statistical and computational methods that can tackle the analysis and management of data. The data normalization is one of the most crucial steps of data processing and this process must be carefully considered as it has a profound effect on the results of the analysis. In this work, we focus on a comprehensive comparison of five normalization methods related to sequencing depth, widely used for transcriptome sequencing (RNA-seq) data, and their impact on the results of gene expression analysis. Based on this study, we suggest a universal workflow that can be applied for the selection of the optimal normalization procedure for any particular data set. The described workflow includes calculation of the bias and variance values for the control genes, sensitivity and specificity of the methods, and classification errors as well as generation of the diagnostic plots. Combining the above information facilitates the selection of the most appropriate normalization method for the studied data sets and determines which methods can be used interchangeably. M-values, Upper Quartile, Median, Quantile, and PoissonSeq normalization implemented in R packages edgeR (v3.2.4), DESeq (v1.12.1), EBSeq (v1.3.1), and PoissonSeq 3 (v1.1.2), respectively. The comparison was based on the analysis of three data sets. Two of them are publicly available data sets (Bodymap and Cheung data) and one is the data set (AML data) coming from one of our projects (not published yet). In this paper, we outline a simple and effective method for comparing different normalization approaches and we show how researchers can improve the results of differential expression analysis by including in the normalization step different aspects based on the biology or informatics.In recent years, RNA sequencing (RNA-seq) technology has become a mainstay of biomedical research and is an attractive alternative to microarrays. RNA-seq technologies have several advantages over microarrays, including less noise  , the posIn this section, we describe the normalization methods to be compared and the data sets used in our study. Next, we propose the criteria for comparison of the impact of the normalization methods on the results of DE analysis.M-values (TMM)  using min-max normalization:The square root of the mean square error was used as a measure. The housekeeping genes were selected based on the raw data by using the following formula:Kij denotes read counts for ith control gene from the jth sample, ith control gene and ith control gene, and True Log Ratio is the true value of ith control gene from the definition of HG. The accuracy of a measurement is the degree of closeness of measurements of a quantity to the true value. The precision of a measurement is the degree to which repeated measurements under unchanged conditions show the same results. Based on the definition, each gene, which is considered as HG, should have the same number of counts in each sample, as well as mean value of counts for this gene. Thus, the True Log Ratio is equal to 0 and the formula , Upper Quartile (UQ), Median (DES), Quantile (EBS), and PoissonSeq (PS), described in detail in the Methods section. All methods were tested on three different data sets: Bodymap, Cheung, and AML data. The details concerning data sets are described in The aim of our study was to compare various normalization approaches and outline the workflow that will help to select the normalization method appropriate for a particular data set. We decided to test five of the most commonly used methods of RNA-seq data normalization: Trimmed Mean of P values. Genes that had adjusted P values < 0.05 were selected as differentially expressed.The main purpose of many RNA-seq experiments is to identify genes that are differentially expressed between the compared conditions. Analyzing three mentioned above data sets, we checked the direct impact of the normalization methods described above on the results of DE analysis. After each normalization, we determined the lists of differentially expressed genes (DEGs) using the statistic test from the edgeR package. In the case of each data set, we compared gene expression levels between two types of biological samples and ranked the genes according to adjusted The results of DE analysis can be compared based on the number and content of DEGs. Visualisation of the DEGs is presented in MA plots, available in Supplementary Materials , were generated for additional visualization of DEGs. They present the relationship between base mean and log2FC of the counts. The results show that DEGs in each MA plot after normalization are slightly more spread out compared to the raw data. However, the location of DEG differs depending on the normalization method for AML and Bodymap data sets. In the case of AML data it is also worth pointing out that we can observe more DEGs that are overexpressed than underexpressed.As it is difficult to evaluate which normalization method is more suitable for a data set based on MA plot analysis, more precise verification is necessary. To this end we calculated bias and variance values.Based on details described in the previous section we selected housekeeping genes for which we calculated bias and variance values. Following the idea described in the study , we assuThe sensitivity and the specificity of the normalization methods were investigated using the AML data set, based on our earlier experience with the analysis of microarray data, described in  as well ∗41), 12 (0.75∗16), and 20 (0.74∗27) differentially expressed genes. The results in the table suggest that, in all data sets, PS, DES, and EBS perform better than TMM and UQ.Besides the analytical methods for comparison of normalization methods we suggest using additional determinants that may be helpful for the rejection of the most commonly used methods that evidently fail. It is possible that the normalization methods yield different results for different data sets. Therefore, we suggest the application of the following workflow based on diagnostic plots to determine which normalization method is optimal for a specific data set. Here, we focus only on the AML data set. In The results in ith and jth methods. First we calculate in the th cell a proportion of common detection with respect to the ith method:i ≠ j, Dij is the number of differentially expressed genes commonly detected by the ith method and the jth method, and Di is the number of differentially expressed genes detected by the ith method. Next, we take the average percentage value of common genes between each pair of methods. The most preferable method is the one with the highest number of common DEGs.To compare the number of DE genes and the number of common DE genes found among the normalization methods performed for a particular data set, we generate balloon plots , particular sets of differentially expressed genes were obtained. First, we chose genes common to all six DEG lists. Since we obtained 20 genes in common between six methods, we performed the clustering of the normalization methods based on these common DEGs. Then, for all methods, we ranked these genes, thus obtaining ranking lists of genes. Based on these lists, we computed the distance matrix by using Euclidian distance and plot dendrograms . The denCombining all of the criteria described in the previous sections, we would like to determine which of the five tested normalization methods would be appropriate for the AML data set. In practice, normalization of high-throughput data still remains an important question and has received a lot of attention in the literature. The increasing number of normalization methods makes it difficult for scientists to decide which method should be used for which particular data set. Based on the results presented in this paper as well as in  we can cFurthermore, we find that other conclusions described in  are consIn the study, new coefficients such as bias and variance were proposed as objective criteria for a comparison of normalization methods. In conclusion, our results suggest that, depending on the RNA-seq data structure and the applied method, the influence of normalization differs. However, the presented criteria, in particular bias and variance, can support the choice of normalization method optimal for a specific data set.In Supplementary Materials we provide additional details concerning data sets used in the presented research. Furthermore, we provide here additional results not included in the main text for comparison of normalization methods, that include other sources of variation such as GC content and batch correction.


Escherichia coli using a microplate reader” This article describes reproducibility of a single-step automated β-galactosidase, and the equivalence of its data to the traditional assay (“Experiments in Molecular Genetics”  The Costar microtitre plates were then placed in a FLUOstar Omega Microplate Reader (BMG LABTECH).Samples were grown overnight at 37 °C in LB media. For the traditional assay, samples were treated as described by Miller 2.2600 and OD420 readings taken every 60 s for 1 h (20 flashes per well each cycle). Wells were set to shake at 500 rpm  between readings to ensure homogenisation of the sample and more efficient lysis. A script to run these parameters on a FLUOstar Omega Microplate Reader can be found in The FLUOstar Omega Microplate Reader was set to 30 °C, with OD2.3600 and OD420 data generated by the by FLUOstar Omega Microplate Reader. Miller Units values are expressed as follows:The MARS Data Analysis software package was used to process the OD420 readings were converted into the slope of OD420 over time (OD420/min), multiplied by 5000, and adjusted for the initial OD600 reading at the first time point. The obtained MU values were then compared to those of the traditional assay (The kinetic ODal assay . More de2.4See


In the area of omics profiling in toxicology, i.e. toxicogenomics, characteristic molecular profiles have previously been incorporated into prediction models for early assessment of a carcinogenic potential and mechanism-based classification of compounds. Traditionally, the biomarker signatures used for model construction were derived from individual high-throughput techniques, such as microarrays designed for monitoring global mRNA expression. In this study, we built predictive models by integrating omics data across complementary microarray platforms and introduced new concepts for modeling of pathway alterations and molecular interactions between multiple biological layers. We trained and evaluated diverse machine learning-based models, differing in the incorporated features and learning algorithms on a cross-omics dataset encompassing mRNA, miRNA, and protein expression profiles obtained from rat liver samples treated with a heterogeneous set of substances. Most of these compounds could be unambiguously classified as genotoxic carcinogens, non-genotoxic carcinogens, or non-hepatocarcinogens based on evidence from published studies. Since mixed characteristics were reported for the compounds Cyproterone acetate, Thioacetamide, and Wy-14643, we reclassified these compounds as either genotoxic or non-genotoxic carcinogens based on their molecular profiles. Evaluating our toxicogenomics models in a repeated external cross-validation procedure, we demonstrated that the prediction accuracy of our models could be increased by joining the biomarker signatures across multiple biological layers and by adding complex features derived from cross-platform integration of the omics data. Furthermore, we found that adding these features resulted in a better separation of the compound classes and a more confident reclassification of the three undefined compounds as non-genotoxic carcinogens. This assay requires daily administration of the tested compound to rats or mice of both sexes and close survey of the animals for signs of toxicity and neoplastic lesions (see OECD Test Guideline 451). During preclinical safety assessment, substances showing a carcinogenic potential at a late stage may lead to substantial losses for the pharmaceutical and chemical industry. A mechanistic distinction is made between genotoxic carcinogens (GCs), which form DNA adducts and cause direct DNA damage, as opposed to non-genotoxic carcinogens (NGCs), for which a wide variety of alternative hepatocarcinogenic mechanisms have been described in vivo studies. Most studies published in this field focused on mRNA expression profiling and employed machine learning algorithms or statistical methods to predict the carcinogenic class of compounds based on characteristic expression patterns, called signatures et al. revealed characteristic changes in the expression of mechanistically related genes, which allowed for discriminating GCs from NGCs in male Wistar-Hanover rats et al. proposed two mRNA signatures for the detection of certain classes of NGCs based on Affymetrix gene expression data from Sprague-Dawley rat liver samples, which are deposited at the large toxicogenomics database TG-GATEs Several groups have reported application of toxicogenomics methods for prediction of the outcome of chronic bioassays based on gene expression profiles compiled from short-term et al.In addition to global mRNA profiling data, miRNA, and protein expression data have previously been generated for toxicogenomics applications et al.In general the ability to predict a compound's carcinogenic potential based on short-term expression profiles would be a clear step forward concerning reduction of time, animals, and monetary requirements in chemical and drug development. Building on published toxicogenomics studies that mostly focused on mRNA expression and used individual genes as predictive features, the approaches presented here introduce two novel concepts: first, the integration of omics data across platforms that interrogate different biological layers  and second, the abstraction from individual signature genes to higher-order levels, such as pathway enrichments or molecular interactions. This holistic approach, which integrates multiple global omics approaches, was currently also proposed by Khan The experimental protocol was reviewed and approved by the Institutional Animal Care and Use Committee (IACUC) of the Institute of Toxicology, Bayer, Stilwell, KS, U.S.A, for compliance with the Federal Animal Welfare Act (1988): 7 U.S.C.2131 et seq. as well as the National Research Council's (NRC) Guide for the Care and Use of Laboratory Animals . General clinical observations, including observations for moribundity and mortality, were performed at least daily to monitor the general overall health status of the animals and to minimize suffering. For necropsy, the animals were anesthetized in a CO2 chamber and blood was drawn by cardiac puncture. Finally, animals were exsanguinated by cutting the diaphragm.Male Wistar Hanover rats (Crl:WI[Gl/BRL/Han]IGS BR) from Charles River Laboratories, Inc.  were maintained on certified rodent chow (Purina Mills Certified Rodent Diet 5200) ad libitum in individual suspended stainless steel wire-mesh cages. The animals were kept under controlled temperature (18 to 26°C), humidity (30 to 70%), and lighting (12 h light – dark cycle) and were acclimated for a minimum of 6 days. 8 to 10 week old animals were assigned to dose groups (5 rats/group) by weight using a weight stratification-based computer program. Substances were administered by gastric gavage for up to 14 days (in a volume of 5 ml/kg body weight/day) based on the group mean weekly body weight for each dose group. Test substances were suspended (W/W) in either corn oil, or a 0.5% (W/V) Carboxymethyl Cellulose (CMC)/DI water preparation (5 g CMC/1 liter DI water). To maintain a homogenous suspension during dosing, a magnetic stir-plate was used if needed. Dimethylnitrosamine , C.I. Direct Black , and Cyproterone acetate  were prepared using corn oil as vehicle. Thioacetamide , Wy-14643 , Phenobarbital , Piperonylbutoxide , Dehydroepiandrosterone , Acetamide , Methapyrilene hydrochloride , Methylcarbamate , Diethylstilbestrol , Ethionine , Cefuroxime , and Nifedipine  were dosed using Carboxymethyl Cellulose as vehicle. Diethylstilbestrol and Piperonylbutoxide were administered for 1, 3, and 7 days. All other compounds were dosed for 1, 3, 7, and 14 days. Time points selected for evaluation in this report are listed in To monitor global changes in mRNA expression, biotin-labeled cRNA samples were prepared with a starting amount of 5 µg of total RNA according to the manufacturer's instructions  and hybridized on Affymetrix GeneChip RAE230A arrays. Fluorescent images of the GeneChips were captured with the Affymetrix GeneChip Scanner 3000. Raw data image files (DAT) were converted into CEL files using Affymetrix Microarray Suite (MAS) 5.0 in which the scan data from the 36 pixels per oligo set are summarized as one intensity value. The RAE230A array used in this study contains 15.866 probe sets, corresponding to approx. 5399 annotated rat genes and 10467 expressed-sequence tags (EST).et al.arrayQualityMetrics for R/Bioconductor affy package for R/Bioconductor.For a complete description of the experimental protocol the reader is referred to a former publication from Ellinger-Ziegelbauer 100 ng total RNA was end-labeled using the Agilent miRNA labeling kit (Agilent p/n 5190–0456). End-labeled miRNA samples were purified using Qiagen PCR clean up columns. Labeled miRNA samples were hybridized according to the Agilent miRNA Microarray System with miRNA Complete Labeling and Hyb Kit Protocol G4170-90011 V2.2 October 2009 using reagents contained in the Agilent miRNA labeling kit. Prior to array hybridization, hybridization mixtures were denatured at 100°C for 5 minutes. Hybridization was carried out at 20 RPM at a temperature of 55°C for 20 hours before washing in Agilent Gene Expression Wash Buffer 1 and Agilent Gene Expression Wash Buffer 2 (Agilent p/n 5188–5327). Hybridization, scanning, and image analysis were performed using the Agilent DNA Microarray Scanner equipped with extended dynamic range (XDR) software according to the Agilent miRNA Microarray System with miRNA Complete Labeling and Hyb Kit Protocol G4170–90011 V2.2 October 2009. Agilent Feature Extraction Software v10.7 was used for data extraction from raw microarray image files.arrayQualityMetrics, which supports a wide variety of microarray platforms et al.AgiMicroRna library for R/Bioconductor was used without the initial background correction step, as recommended by Lopez-Romero et al., who recently demonstrated that this normalization technique facilitates a more accurate estimation of miRNA expression levels than the method suggested by the array manufacturer As previously done for the mRNA expression data, we ensured sufficient quality of the raw data by performing quality checks implemented in the R library Protein expression profiling was performed using reverse-phase protein microarrays (RPPAs) on the ZeptoMARK assay platform . Frozen liver tissue (50–80 mg tissue) was weighted into a cryovial and grinded to a fine powder under liquid nitrogen. 8 volumes of lysis buffer were added to the pulverized tissue and lysis was carried out in a rotating mixer for 30 min. Protein concentration of the lysate was determined by a Bradford assay and protein concentration of the lysates was adjusted to 0.3 mg/ml protein. The prepared lysates were used to print RPPAs as described in detail by Pirnia et al. 2-transformed. If the background signal level, which was determined by exclusive use of the secondary antibody, exceeded the combined signal level of both the primary specific and secondary antibodies when used in combination, the corresponding measurement was considered as a missing value. Missing values, which accounted for approx. 1% of the data, were estimated using k-Nearest-Neighbor imputation.For the sake of improved interpretability, the blank-corrected normalized signal levels were divided by the median of the corresponding control group samples incubated with the same specific antibody and subsequently logPredictive signatures were inferred based on the fold changes obtained from multiple omics platforms using recursive feature selection with SVMs (SVM-RFE). SVM-RFE repeatedly trains an SVM on the provided features and assigns a weight for each feature depending on the relevance for the classification To find the optimal signature size, we evaluated the predictive power of signatures of different size for each supervised machine learning method used for classification. Signatures containing 5, 10, 15, 20, and 25 features were extracted and evaluated for their predictive power. Based on the accuracies estimated for each of the differently sized signatures, we used spline interpolation to numerically approximate the optimal signature size.As multiple signatures were extracted for each cross-validation fold and repetition, the individual signatures were merged into a consensus signature. For this purpose, the features were ranked by their average rank in all cross-validation folds and repetitions. The consensus signature was then constructed by selecting the best features until the approximated optimal signature size was reached.As the regulation of gene expression is orchestrated by the interplay of multiple biological layers, an approach that assesses the relevance of each genomic and proteomic feature individually may be insufficient. In order to account for the highly interconnected nature of gene regulatory mechanisms, we conceived a novel feature representation for multi-level omics datasets, which captures characteristic molecular interactions present upon treatment with a specific class of compounds .2(fold changes), to a common scale. Furthermore, the scaling of the data also ensures that each platform contributes equally to the score computed for a certain molecular interaction. All log-ratios x were mapped to the same interval using the linear function As the dynamic range of differential expression varies between platforms interrogating different biological layers, we propose to transform the log-ratios, i.e., loga priori not contained in canonical pathways , their regulatory influence on pathways was modeled based on experimentally validated interactions to known target mRNAs corresponding to pathway nodes. Overrepresented pathways for this combined gene list, derived from deregulated mRNAs, miRNA targets, and proteins, were detected with a hypergeometric test. Given a universe where the union of all pathways contains N genes of which M are in the pathway of interest, and the combined list containing n genes of which m are contained in the pathway, the p-values were computed according to the following formula:The basic idea of this feature type is to generate a more robust representation of the molecular signature of a compound by abstraction from genes to pathways . To this10. Finally, a vector composed of the enrichment scores of each pathway was employed as a numeric feature representation of pathway alterations induced by compound treatment. The higher the enrichment score computed for a certain pathway, the more significant is the overrepresentation of genes from the combined list in this pathway. The feature vectors were constructed based on different sets of canonical pathways extracted from the databases KEGG The p-values were then transformed into enrichment scores, which correspond to the -logcaret library available for R We ensured unbiased parameter tuning and evaluation on independent test compounds by employing a 2×2-fold nested, stratified cross-validation procedure. In order to obtain a more robust estimate of the classification performance, 10 repetitions were performed with different random splits of the data. The prediction accuracy that can be achieved with different feature representations derived from the expression profiles of the compounds was assessed based on the average area under the ROC curve observed for a representative selection of machine learning methods that are prevalent in toxicogenomics applications. We employed linear Support Vector Machines (SVM), Random Forests (RF), Neural Networks (NN), Bayesian Generalized Linear Models (BGLM), and Principal Component Regression (PCR). We included PCR because a good separation of classes was observed in a principal component analysis (PCA) of the data, and because the method achieved comparable accuracy than the other methods on the here evaluated classification problems. We have also analyzed the optimal number of principal components used for classification with PCR . RF, NN,With the goal to identify molecular signatures that may allow early prediction of a carcinogenic risk associated with a compound, we inferred mRNAs, miRNAs, and protein expression signatures correlated with the toxicological classes of the compounds which induced these signatures in the liver of rats treated for up to 14 days. In addition, a combined signature was obtained by merging the signatures extracted from each individual platform. Furthermore, we compiled signatures based on putative molecular interactions (MI features) observed between two molecular layers, as well as based on pathways enriched with genes/proteins that were detected as deregulated (PE features). These two types of cross-platform features  were joiSince in most cases an increased average AUC was observed for the combined signatures, we can conclude that composite signatures derived from multiple platforms, may allow for a more accurate compound classification than the traditionally used single-platform signatures . The onlFor each classifier, consensus signatures were generated by combining the feature rankings inferred from random data splits, which correspond to different repetitions of the cross-validation procedure. The extracted signatures are shown in Gsta5, Aldh1a1Ephx1Akr7a3The most informative mRNA markers for C vs. NC discrimination include various genes known to be involved in cellular responses to carcinogenic exposure, e.g., As has been done for the single-platform signatures, consensus signatures were generated based on the MI and PE signatures extracted for each run. The most informative pathways for the C vs. NC and NGC vs. GC contrasts are depicted in the heatmap in p53, which is a key gene in the cellular DNA damage response, were selected as GC-specific pathway features The top PE features for the discrimination of carcinogens from non-carcinogens were highly specific for carcinogens and showed no enrichment at all in the non-carcinogens . For theGlulDusp1JunSgk1Mgat4bGlul and Jun  features and the pathway enrichment (PE) features.With extensive cross-validation, we evaluated the predictive power of single-platform signatures as well as hybrid signatures complemented with cross-platform features on a dataset containing mRNA, miRNA, and protein expression data for 8 NGCs, 2 GCs, and 2 NCs concerning rodent hepatocarcinogenicity. In addition, all signatures were used to classify three rodent hepatocarcinogens with insufficiently defined mode of actions. The dataset used for the evaluation is publically available from the NCBI Gene Expression Omnibus (GEO) repository under the accession number GSE53085 In comparison to the signatures inferred solely from the mRNA data, we observed an increase in predictive power when combining signatures across multiple platforms for all evaluated class contrasts . In addition, we found that the inclusion of complex cross-platform features capturing molecular interactions or pathway alterations led to a further improvement of the classification performance. In general, the combined single-platform signatures with MI and PE features were superior to the single-platform signatures in terms of prediction accuracy, except for the NGC vs. GC contrast, where the protein signature performed slightly better. However, visual inspection of the informative features by PCA revealed a much better separation of classes for the cross-platform signatures . Furtheret al. and Low et al. consistently reported that the integration of QSAR-based features did not contribute to an increased classification performance In related toxicogenomics studies the accuracy of classifiers trained solely on omics-based features has been compared to models exclusively or additionally provided with features derived from a Quantitative Structure-Activity Relationship (QSAR) approach As an alternative to our complex supervised approach for compound class discrimination, we applied clustering to determine if the expression profile based grouping of the compounds agrees with their annotated carcinogenic class . For thiThe single- and cross-platform signatures inferred with our supervised approach have been used to reclassify the compounds with undefined class, i.e., CPA, TAA, and WY. All three compounds are known to be carcinogenic, but some of the genotoxicity test results were inconclusive In contrast to the standard toxicogenomics approach of inferring signatures from mRNA expression, the combination of single-platform signatures from multiple omics platforms led to improved prediction accuracy. The inclusion of the MI and PE features into these combined signatures offered a true cross-platform integration of the multi-level omics data. These integrated features may yield additional information for supervised learning methods and provide new insights for mechanistic analyses of cross-platform omics data. In addition, the PE features introduce an abstraction from the level of individual genes to higher-order levels, such as complex signaling or metabolic networks. Possibly, this abstraction led to a more robust representation of the complex mechanisms underlying early events during hepatocarcinogenesis in rat. Other relevant mechanistic aspects, which cannot be detected by analyzing each platform individually, may be captured by the here introduced MI features. In consideration of the fact that uncertainty was introduced in the signature extraction process by permitting predicted miRNA-mRNA interactions as potential features, the implication of individual, predictive molecular interactions during hepatocarcinogenesis has to be critically assessed. However, since the focus of this study was the proof of principle that considering multiple molecular layers and interactions between them may increase the reliability of toxicogenomics-based prediction models, the experimental validation and confirmation of the biological relevance of the selected features is beyond the scope of this article.et al.et al.et al.et al.et al.et al.et al., Nie et al., and Uehara et al. (2011), and only one common gene with Uehara et al. (2008): Akr7a3. However, these studies used other microarray platforms, different compounds, and different treatment durations: Nie et al. and Uehara et al. (2008) used a single dosage 24 h setting, Fielden et al. used a 5 day repeated dosage setting, and Uehara et al. (2011) used a longer 28-day exposure. For the C vs. NC signature published by Nakayama et al., we found five genes that are also present in our C vs. NC mRNA signature  were in common with our 22-probe-set C vs. NC mRNA signature Click here for additional data file.Figure S2Cross-validation performance of signatures for compound classification. (A) The receiver operating characteristic (ROC) curves illustrate the predictive power of the signatures inferred for discrimination of carcinogens (C) and non-carcinogens (NC). The left plot illustrates the predictive power of the signature inferred using only mRNA features. The right plot illustrates the predictive power of the signature combining the single-platform signatures for mRNA, miRNA, and protein features as well as the cross-platform molecular interaction (MI) and pathway enrichment (PE) features. For each supervised learning method trained on the extracted signature, one ROC curve has been plotted. The dotted line indicates the chance level, which corresponds to an area under the ROC curve (AUC) of 0.5. The bar plot inside the ROC curve plot indicates the AUC achieved by each classifier. Predictive power was assessed using a 10 times repeated, nested 2×2-fold cross-validation. Prediction scores were scaled linearly to  and subsequently merged across all folds and repetitions to obtain a single ROC curve. (B) Predictive power of signatures for discrimination of non-genotoxic carcinogens (NGC) and genotoxic carcinogens (GC) illustrated as in (A). (C) Predictive power of signatures for discrimination of NGCs and NCs illustrated as in (A).(PDF)Click here for additional data file.Figure S3Heatmap plots of single-platform signatures for NGC vs. GC classification. The heatmaps depict characteristic expression patterns that were observed in livers of rats after exposure to non-genotoxic or genotoxic rodent hepatocarcinogens. A selection of signature molecules is shown for each profiled molecular level: (A) mRNA expression, (B) miRNA expression, and (C) protein expression. In each heatmap, the rows correspond to signature molecules and the columns correspond to differentially liver samples from differentially treated rats. The bold vertical lines separate the NGCs from the GCs. Plotted are the log2(fold changes), where red indicates up-regulation and green indicates down-regulation (see color keys). The color bar on top refers to the carcinogenic compound class (see legend).(PDF)Click here for additional data file.Figure S4Heatmap plots of single-platform signatures for NGC vs. NC classification. The heatmaps depict characteristic expression patterns, which were observed in livers of rats after exposure to non-genotoxic hepatocarcinogens and non-carcinogens. A selection of signature molecules is shown for each profiled molecular level: (A) mRNA expression, (B) miRNA expression, and (C) protein expression. In each heatmap, the rows correspond to signature molecules and the columns correspond to differentially treated rat liver samples. The bold vertical lines separate the NGCs from the NCs. Plotted are the log2(fold changes), where red indicates up-regulation and green indicates down-regulation (see color keys). The color bar on top refers to the carcinogenic compound class (see legend).(PDF)Click here for additional data file.Figure S5Heatmap plot for pathway enrichment. The heatmap depicts overrepresentation of genes involved in relevant pathways among the genes deregulated upon treatment with a certain compound. Pathways relevant for compound classification were selected by SVM-RFE for NGC vs. NC discrimination. The rows correspond to canonical pathways from the databases Reactome (R), KEGG (K), or BioCarta (B) and the columns correspond to samples. The color of each cell refers to the -log10 obtained from a hypergeometric overrepresentation test and indicates the significance of a certain pathway enrichment (see color key). The color bar on top of each heatmap denotes the carcinogenic class (see legend).(PDF)Click here for additional data file.Figure S6Volcano plots of molecular interaction signatures. Shown are volcano plots for expression profiles of the six non-genotoxic carcinogens not shown in 2(fold change) and plotted against its significance which is given by the FDR-corrected log10 obtained from a moderated t-test. Different shapes and colors denote different types of molecules (see legend). Colored edges were used to highlight molecular interactions for which a positive or negative correlation was observed between two platforms. We considered correlations in the expression profiles of miRNAs and their experimentally confirmed or predicted mRNA targets as well as between mRNAs and proteins sharing the same genomic locus. As a formal criterion for a putative molecular interaction, we required that for both interaction partners a 50% increase or decrease in expression was observed relative to the controls.(PDF)Click here for additional data file.Figure S7Principal component analysis of signatures for classification of undefined compounds. (A) Samples are represented based on the mRNA signature for NGC vs. GC discrimination. The corresponding fold changes were PCA-transformed and plotted in a lower-dimensional space spanned by the first two principal components. The color of the spheres corresponds to the class of the administered compounds with respect to their hepatocarcinogenic properties in rats. Clusters of rat liver samples after treatment of rats with compounds of the same class are highlighted by means of transparent polygons. (B) Similar plot as in (A), but based on the miRNA signature for NGC vs. GC discrimination. (C) Similar plot as in (A), but instead of using a single-platform signature for sample representation, all single-platform  signatures were combined. (D) Similar plot as in (A), but instead of using a single-platform signature for sample representation, all single-platform  signatures and the cross-platform pathway enrichment (PE) signature were combined. (E) Similar plot as in (A), but instead of using a single-platform signature for sample representation, all single-platform  signatures and the cross-platform molecular interaction (MI) signature were combined.(PDF)Click here for additional data file.Figure S8Reclassification of undefined compounds. (A) The heatmap displays the confidence scores obtained from five different machine learning methods that were used to classify the undefined compounds CPA, TAA, and WY as either NGC or GC based on the mRNA signature for NGC vs. GC discrimination. The confidence scores are -scaled and correspond to the probability that a certain sample was derived from rats treated with an NGC (see color key). (B) Similar illustration as in (A) obtained from an SVM classifier trained on the miRNA signature. (C) Similar illustration as in (A) obtained from an SVM classifier trained on all single-platform  signatures combined. (D) Similar illustration as in (A) obtained from an SVM classifier trained on all single-platform  signatures and the cross-platform pathway-enrichment (PE) signature combined. (E) Similar illustration as in (A) obtained from an SVM classifier trained on all single-platform  signatures and the cross-platform molecular interaction (MI) signature combined.(PDF)Click here for additional data file.Figure S9Cluster analysis of mRNA, miRNA, and protein expression data. The heatmaps depict the results of a cluster analysis limited to the top 100 differentially expressed (A) mRNAs, (B) miRNAs, and (C) proteins in livers of rats treated with diverse carcinogenic or non-carcinogenic compounds. In each heatmap, the rows correspond to the animals and the columns correspond to the profiled molecules. The colors refer to the log2(fold changes), where red indicates up-regulation and green indicates down-regulation (see color key). The dendrograms that were obtained from hierarchical cluster analysis of both samples and molecules are shown adjacent to the heatmap. The color bar on the right indicates the cluster membership of the samples.(PDF)Click here for additional data file.Figure S10Common genes between our inferred and published signatures for C vs. NC discrimination. The Venn diagram illustrates the common genes between our signature inferred from mRNA data for the discrimination of carcinogens and non-carcinogens and signatures for the same class contrast previously published by Ellinger-Ziegelbauer et al. and Nakayama et al.(PDF)Click here for additional data file.Table S1Signatures for C vs. NC discrimination. The tables list the signatures inferred for the discrimination of carcinogenic and non-carcinogenic chemicals for each signature type  and pathway enrichment (PE) features).(XLS)Click here for additional data file.Table S2Signatures for NGC vs. GC discrimination. The tables list the signatures inferred for the discrimination of non-genotoxic and genotoxic carcinogens for each signature type  and pathway enrichment (PE) features).(XLS)Click here for additional data file.Table S3Signatures for NGC vs. NC discrimination. The tables list the signatures inferred for the discrimination of non-genotoxic carcinogens and non-carcinogens for each signature type  and pathway enrichment (PE) features).(XLS)Click here for additional data file.


By focussing on differences in the mural granulosa cell (MGC) and cumulus cell (CC) transcriptomes from follicles resulting in competent (live birth) and non-competent (no pregnancy) oocytes the study aims on defining a competence classifier expression profile in the two cellular compartments. Design: A case-control study. Setting: University based facilities for clinical services and research. Patients: MGC and CC samples from 60 women undergoing IVF treatment following the long GnRH-agonist protocol were collected. Samples from 16 oocytes where live birth was achieved and 16 age- and embryo morphology matched incompetent oocytes were included in the study.MGC and CC were isolated immediately after oocyte retrieval. From the 16 competent and non-competent follicles, mRNA was extracted and expression profile generated on the Human Gene 1.0 ST Affymetrix array. Live birth prediction analysis using machine learning algorithms (support vector machines) with performance estimation by leave-one-out cross validation and independent validation on an external data set.We defined a signature of 30 genes expressed in CC predictive of live birth. This live birth prediction model had an accuracy of 81%, a sensitivity of 0.83, a specificity of 0.80, a positive predictive value of 0.77, and a negative predictive value of 0.86. Receiver operating characteristic analysis found an area under the curve of 0.86, significantly greater than random chance. When applied on 3 external data sets with the end-point outcome measure of blastocyst formation, the signature resulted in 62%, 75% and 88% accuracy, respectively. The genes in the classifier are primarily connected to apoptosis and involvement in formation of extracellular matrix. We were not able to define a robust MGC classifier signature that could classify live birth with accuracy above random chance level.We have developed a cumulus cell classifier, which showed a promising performance on external data. This suggests that the gene signature at least partly include genes that relates to competence in the developing blastocyst. In order to improve the efficacy of in vitro fertilisation treatment, there is ongoing research to define good non-invasive embryo markers to increase the implantation rate from an average of around 25% achieved with the existing morpho-kinetic markers .The developmental competence of the mammalian oocyte is achieved during folliculogenesis, where cross-talk between oocyte and somatic cells ensures oocyte growth and maturation and the subsequent potential to sustain fertilisation and produce viable embryos . Thus, sSpecific functions of the somatic cells from the pre-ovulatory follicles just prior to ovulation (e.g. progesterone and oestrogen production from the cumulus-oocyte-complex (COC)  and exprTaking the complexity of the preovulatory follicle into account, the aim of identifying single marker genes seems to be less robust than a marker based on the scenario of multi interacting genes, which may be identified by the high trough put techniques such as the expression array. Therefore the present study focus on differences in the MGC and CC transcriptomes between oocytes from competent and non-competent follicles and the aim is to develop classifier profiles in the two cellular compartments in order to be able to select the embryos with the best potential to result in life offspring. To approach the scenario for selecting a competent embryo for transfer in a cohort of embryos, a case control design was chosen matching women’s age and embryo morphology.Thus, in the present study, gene expression profiling of corresponding CC and MGC of individual follicles obtained from women undergoing IVF with elective single embryo Day 3 transfer were associated to no pregnancy after IVF or life birth of a healthy baby.The CC and MGC were collected at the Fertility Clinic, Copenhagen University Hospital, Rigshospitalet, Denmark between February 2009 and June 2010 as a part of a prospective, randomised study ) . The Dan2, regular menstrual cycle, early follicular phase serum FSH levels within 1–12 IU/l and early follicular phase total antral follicle (2–10mm) count ≥ 6 donated CC and MGC. The patients underwent controlled ovarian stimulation in a GnRH agonist protocol as described in [A total of 60 women, scheduled for IVF, aged 25–37 years and with BMI >18 and < 30 kg/mribed in . Oocyte ribed in . To reprFor demographic data t-test and chi-squared tests were used and p<0.05 considered significant.Follicular fluids from all follicles were collected individually in tubes . When an oocyte was obtained, the number of the follicle was recorded and the oocytes were numbered consecutively. Thus, each oocyte could be linked to an individual follicle and hence the correspondent MGC. Immediately after isolation of the COC, the MGC were isolated from the follicular fluid by centrifugation at 400 g in 8 min at room temperature. The pellet was re-suspended in 1 ml HEPES-buffered solution  and placed on a density gradient column  and centrifuged at 400 g for 30 minutes. After centrifugation, the inter-phase containing the MGC was aspirated and the MGC washed twice by mixing with1 ml of the buffer solution followed by centrifugation at 400 g for 8 minutes at room temperature. After the second wash, pellet was re-suspended in 500 μl in a cryo-vial  and stored at -80°C until RNA extraction.Within 1 hour after the oocyte retrieval a part of the cumulus mass was mechanically removed (18G needles) from the COC and transferred to a cryo-vial  with as little fluid as possible and stored at -80°C until RNA extraction.IVF insemination was done with standard IVF procedure at 3 h (±1 h) after oocyte retrieval. All oocytes were followed individually and zygote and embryo quality assessed at 20 h (±1 h), 28 h (±1 h), 44 h (±1 h) and 68 h (±1 h) after oocyte retrieval. The embryo quality evaluation consisted of assessment of cell number and three parameters of embryo morphology: degree of fragmentation, cell stage specific blastomere uniformity and multinucleation .RNA from individual samples was purified using Arcturus PicoPure RNA Isolation kit  according to manufactures instructions but with small modifications. In short: 100 μl extraction buffer was used for initial lysis of each sample, except for samples were sample volume excided 70 μl. For these samples 2 x sample volume of extraction buffer was added. Following lysis and centrifugation, the supernatant was transferred to a 2 ml Eppendorf tube from the kit  where it was mixed 1:1 with 70% ethanol. For some MGC samples with large volume, several transfers and spins were needed to collect all the RNA in the spin tubes. In the end, the RNA was extracted in 12 μl nuclease free water, giving a ~10 μl end volume. The integrity of the RNA was measured using bioanalyzer pico chip  and only samples with a RNA integrity number (RIN) above 7 as analyzed by the 2100 bioanalyzer  were included in the analysis.RNA was amplified and labeled using a pico amplification kit according to manufactures instructions. In short, approximately half of the total RNA from each sample (5 μl of 10 μl) was amplified using the Ovation Pico WTA v.2 RNA Amplification System from NuGEN® Inc.  and biotin labeling was performed with the Encore Biotin Module (NuGEN®). The labeled samples were hybridized to the Human Gene 1.0 ST GeneChip array . The arrays were washed and stained with phycoerytrin conjugated streptavidin (SAPE) using the Affymetrix Fluidics Station® 450, and the arrays were scanned in the Affymetrix GeneArray® 3000 7G scanner to generate fluorescent images, as described in the Affymetrix GeneChip® protocol. Cell intensity files (CEL files) were generated in the GeneChip® Command Console® Software (AGCC) .All samples are MIAME compliant and were handled according to SOP in the Microarray Center. A total of 27 CC and 19 MGC samples were amplified and hybridized to arrays. The 46 samples were submitted to ArrayExpress at EMBL using MIAMExpress. The experiment accession number is E-MTAB-4012.Classifiers were constructed based on expression matrices obtained by Plier normalization . FollowiBased on these data sets, three different classification algorithms  ,20, k-neFollowing the cross validation procedure, we calculated the cross-validation support for each gene showing the percentage of the cross-validation training sets in which each particular gene was selected. The cross-validation support shows the percentage of the cross-validation training sets in which each particular gene was selected, giving the relative strength (weight) of each gene in the signature.For the optimal CC classifier, the predictive probability (estimate of the probability of getting a particular class label) of each samples, was derived from the trained SVM model. This was achieved by transforming the SMV classifier into a probabilistic classifier by calculating the predictive probability of a sample being one or the other type using logit estimates as implemented in the e1071 R library . The preA permutation test was performed in order to determine if the cross-validated misclassification rate of the final CC classification model was lower than expected by chance . In 1000 random permutations of the class label, the entire cross-validation was repeated for classifying the random grouping of the samples. The proportion of the 1000 random permutations that resulted in a smaller or similar cross-validation misclassification rate as obtained with the real data determine the permutation p-value. The statistical significance of the error rate was determined both for the linear SVM, 1-NN, 3-NN and dLDA classifier.www.ingenuity.com). The list of the signature probe sets with p value and fold change was imported into Ingenuity. By application of the Ingenuity Pathway Knowledge Base each gene identifier was overlaid onto a global molecular network linking their functionality to the function of other genes and biological enriched functionality and mechanistic networks of these genes were then generated based on their connectivity and enrichment statistics. A network score was calculated based on the hyper-geometric distribution and calculated with the right-tailed Fisher’s exact test. The score is the negative log of this p-value. The score takes into account, the number of network eligible molecules in the network and its size, as well as the total number of network eligible molecules analyzed and the total number of molecules in the knowledge base that could potentially be included in the network. The score represents the chance of getting a network containing at least the same number of network eligible molecules by chance when randomly picking the number of genes that can be in networks from the knowledge base. The top molecular and cellular functions and networks are shown.The cumulus gene signature was subjected to Ingenuity pathway analysis using IPA Ingenuity software Ingenuity®Systems  was validated on an external data set generated on the Agilent platform using the Whole Human Genome Microarray 4x44K microarray . The fulThe microarray expression profiles were generated in a two-color array with dye-swap of replicate samples to avoid systematic noise in the data related to the choice of dye (cye-5 or cye-3). We downloaded the preprocessed data of normalized unlogged expression values representing one-color channel intensities. The data was used for validation of the gene signature after applying a log2 transformation. Gene symbols were used to select the probes on the 4x44K array that overlap with the unique genes in the 30-gene signature. Due to redundant representation of some of the genes on the Agilent array, the gene signature was represented by 68 redundant mRNA transcripts (representing 25 unique genes) on the Agilent 4x44 platform. For assessing the performance of this gene signature, represented by 25 unique genes, the probe with the intermediate expression value for each of the genes with redundant probes was used to define the 25-gene signature in the external data sets.The external data sub sets  were tested one by one. For each sub set, a test set data object was constructed and the overlapping gene signature was tested using SVM classification with linear kernels. The predictive probabilities were derived and visualized along with the predictive probability of the training set samples.ROC curve characteristics analysis was performed on all data set  with the end point LB or NP in the training data, and blastocyst or embryo of poor quality used in the validation data set. A ROC curve was generated for each partition of the validation data set. The ROC curve was created by plotting the true positive rate  versus the false positive rate . The area under the curve (AUC) indicates the degree of predictive ability of the gene expression, where 0.5 is random chance and 1.0 is perfect predictive ability. The ROC and AUC were produced using the functionality of the R library ROCR.A total of 46 expression profiles were generated, encompassing 27 CC samples and 19 MGC samples, of which 12 CC sample and 9 MGC samples, respectively, were obtained from follicles where the oocyte after implantation gave rise to a live birth of a healthy baby (LB) and 15 CC and 10 MGC samples, respectively, which did not lead to pregnancy (NP). The expression profiles of the CC and MGC samples were analyzed separately. The basic demographic parameters for the women which CC and MGC were analyzed are presented in We constructed a classifier to discriminate between patients where IVF treatment lead to live birth compared with no pregnancy, based on expression profiles on CC or MGC from the pre-ovulatory follicles.We examined different methods for the construction of an optimal classifier of oocyte competence and tested several cut-off values of the test statistics with or with-out fold change and expression above back-ground filtering. Three different classifier models , k-nearest neighbors and SVM with linear kernel function were generated and tested by LOOCV. Overall accuracy, as well as PPV and NPV were derived for both the cumulus and granulosa data separately, as shown in The overall best performance of the classifier models applied on the CC was achieved with PLIER normalized unlogged expression values by the linear SVM model and 3-NN and resulted in a LOOCV accuracy of 0.81 and 0.85, respectively . On the The performance range of the classifier models applied on PLIER normalized MGC with a p-value cut-off of 0.01 was 11% - 37% for dLDA, 3-NN (k = 3) linear SVM . EvaluatThe CC data show high levels of variance between the samples within each group and relatively small overall expression differences between the LB and NP groups. To increase the power of the analysis we applied a filter to firstly exclude un-annotated probe sets and further exclude genes with an average expression below back-ground level  . TraininRerunning the training of the of the linear SVM after filtering for genes expressed above background, the overall accuracy of the SVM with radial kernel was unchanged (81%), when using a signature representing 30 unique genes . The fulROC curve analysis to distinguish between true positive rate and false positive rate of the final linear SVM classifier (annotated probes filtered to exclude probes expressed below background level) resulted in an AUC value of 0.86 indicating a good performance of the CC classifier .We furthermore assessed the strength of each of the 30 genes in the signature by the cross-validation support, which shows the percentage of the cross-validation training sets in which each particular gene was selected . AnalysiIn order to assess whether the misclassification rate was lower than what would be expected by chance we performed 1000 random permutation of the samples. For each permutation the entire LOOCV process was repeated and the proportion of the 1000 random permutations resulting in smaller misclassification rates determined the statistical significance of the classifiers error rate. The significance of the misclassification rate after 1000 permutations was 0.284, 0.002, 0.009, for the diagonal linear discriminant analysis classifier, the 3-nearest neighbors’ classifier and the linear SVM classifier, respectively. All but the d-LDA classifier was statistically significant compared to the random setting.We used the trained SVM classifier to derive the predictive probability of a sample belonging to one class or the other by the use of logit estimates . The preWe tested the cumulus classifier on an external data set published by Feuerstein et al, 2012 downloaded from the public expression profile repository, gene expression omnibus (GEO) at NCBI . The datThe SVM predictions were translated into predictive probabilities and visualized alongside the predictive probabilities of the training data for comparison . In agreDespite the fact that gene classification signatures often can be difficult to interpret with respect to the biology of the underlying disease; and missing biologic interpretation does not warrant poor clinical usefulness of well-established biomarkers , the proMechanistic networks were generated based on sets of connected upstream regulators that in conjunction would result in the gene expression changes observed in the dataset. Two mechanistic networks were generated with network scores above 6. The first network was enriched in functions related to Cellular Movement, Nervous System Development and Function, Cellular Growth and Proliferation with a network score of 24 . Eleven The study demonstrated that is possible to develop a classifier based on selection of 30 genes from whole genome microarray data from CC that predict the chance of pregnancy in an IVF setting with 81%. This classifier proved its value by prediction of the ability of embryos to reach the blastocyst stage in vitro from an independent data set. Therefore this classifier has the potential to be developed into a clinical useful tool if larger studies can confirm the results obtained in this study.In contrast, we were not able to define a robust MGC classifier signature that could classify live birth with accuracy above random chance level.The aim of this study was to use machine learning techniques to build CC and MGC transcriptomic classifiers to predict oocyte competence. Several combinations of microarray data pre-processing  and feature selection were tested during cross validation rounds to derive the best performing algorithm and classifier gene signature –30.Aside from the inherent heterogeneity of the data, we were able to correctly classify LB patients by leave one cross validation based on a 30 gene cumulus cell signature, and the cross validation support data showed that more than half of the genes in the signature were included in 89 percent or more of the cross validation runs, indicating that these genes have a stronger impact on the outcome and hence have a high relative weight in the classifier. We furthermore proved the classifier and the signature to be statistic significant compared to random classification of LB versus NP in 1000 random permutations of the samples as described .The signature was furthermore tested for its ability to predict developmental competence as defined by the ability to reach the blastocyst stage in an external validation data set. The data represented 3 sets of transcriptomes from CC corresponding to oocytes that gave rise to 4-cell embryos on day 2 after fertilisation that either failed (incompetent) or succeeded to develop to full blastocysts (competent) after 5 days in vitro culture . Our claWhereas we observed a varying performance of the different CC classifiers depending on the choice of normalization and classification algorithm applied, all tested combinations of MGC classifiers showed poor performance in discriminating oocytes leading to live birth from oocytes which failed to establish a pregnancy. These data indicate that that gene expression in MGC does not reflect competence. However, the high degree of variability across samples, the small fold differences between the two end-points (NP and LB), combined with the lower number of samples in the MGC data set (19 MGC compared to 27 CC), with the resulting reduced power of the analysis may influence the ability to detect a set of discriminating signature genes during cross validation.Further, explanation for the small expression changes observed in both cell types may be that the true signal of competence is blurred by the massive changes in the follicle originating from the ovulation trigger 34–36 hours before oocyte retrieval and isolation of cells –34. FurtSince the overall aim of the project was to define predictors (genes) of LB and not to detect differentially expressed genes, a significance level of p < 0.01 in a univariate t-test was used for feature inclusion. This significance level is not sufficient significant to exclude false discoveries since no genes were significant after multiple testing correction (data not shown). This is in agreement with a recent study, where no genes were significantly expressed with FDR below 0.05 in CC and MGC of oocytes leading to pregnancy or not . In someAlthough many of the signature genes showed modest level of expression changes in the training data set and do not overlap with other studies –13,38,39P53, BAX, BCL-2 were not related to competence in our data.The 30 annotated genes in the CC classifier signature were subjected to Ingenuity Pathway analysis. Downstream effect analysis suggests increased activity of apoptosis in the CC surrounding oocytes with developmental competence to develop, implant and succeed in birth of a healthy baby. None of the classifier signature genes have previously been shown to be regulated in CC of oocytes determined to be of good quality, mature or developmentally competent by different measures and standards –13,38,43NR2F1, COL4A1, GAL, DCLK1, MRC2, FLNA, SOCS6, RAB33A, FN1, BGN) showed an overlap with genes that previously have been shown to be differentially expressed between corresponding CC and MGC in the preovulatory follicle [COL4A1) was 2–10 fold higher expressed in CC as compared to MGC indicating a specific function of the gene products in the CC compartment in the ovulatory follicle [Of the 30 genes in the classifier, 10 .S1 Fig(PDF)Click here for additional data file.S2 Fig(PDF)Click here for additional data file.S1 Table(PDF)Click here for additional data file.S2 Table(PDF)Click here for additional data file.S3 Table(PDF)Click here for additional data file.S4 Table(PDF)Click here for additional data file.S5 Table(PDF)Click here for additional data file.


The comprehensive transcriptomic analysis of clinically annotated human tissue has found widespread use in oncology, cell biology, immunology, and toxicology. In cancer research, microarray-based gene expression profiling has successfully been applied to subclassify disease entities, predict therapy response, and identify cellular mechanisms. Public accessibility of raw data, together with corresponding information on clinicopathological parameters, offers the opportunity to reuse previously analyzed data and to gain statistical power by combining multiple datasets. However, results and conclusions obviously depend on the reliability of the available information. Here, we propose gene expression-based methods for identifying sample misannotations in public transcriptomic datasets. Sample mix-up can be detected by a classifier that differentiates between samples from male and female patients. Correlation analysis identifies multiple measurements of material from the same sample. The analysis of 45 datasets (including 4913 patients) revealed that erroneous sample annotation, affecting 40 % of the analyzed datasets, may be a more widespread phenomenon than previously thought. Removal of erroneously labelled samples may influence the results of the statistical evaluation in some datasets. Our methods may help to identify individual datasets that contain numerous discrepancies and could be routinely included into the statistical analysis of clinical gene expression data.The online version of this article (doi:10.1007/s00204-015-1632-4) contains supplementary material, which is available to authorized users. The generation of large gene expression datasets presents a logistic challenge that extends from the initial procurement and storage of tissue samples, through laboratory procedures, to bioinformatic data processing and analysis. Although anticipated to be low, little is known about the actual frequency of sample mix-up during this multi-step process. The reasons for sample identity being swapped between individuals are diverse, and these events are difficult to pinpoint retrospectively with absolute certainty. In datasets with roughly balanced frequencies of male and female individuals, it can be assumed that approximately half of the mix-ups will result in sex mislabeling. These cases can be identified by assessment of genes with male- or female-specific expression. Other commonly annotated clinicopathological parameters, such as tumor stage, would also be affected by mislabeling, but the lack of genes that exhibit for instance a reliable stage-specific expression pattern makes the standardized assessment of these parameters unsuitable.XIST and genes located on the Y chromosome for the discrimination between male and female samples in the analysis of single datasets. However, these methods are not generalizable because of the lack of normalization across datasets  a likelihood-based strategy was applied that ensures robustness against outliers .XIST gene (221728_x_at and 214218_s_at), located on the X chromosome, and the other two to RPS4Y1 (201909_at) and DDX3Y (205000_at), respectively, both located on the Y chromosome. XIST is expressed from the inactive female X chromosome and silenced in men. This is illustrated in one NSCLC dataset (GSE31210), with high expression of XIST (221728_x_at) observed in all patients labelled as female  gene expression datasets analyzed on the AffymetrixGeneChip© HG-U133A or HG-U133 Plus 2.0 array . For each sample, sex information and gene expression measurements for 22,277 probe sets were available. Only seven probe sets achieved median male–female classification accuracy above 75 % and only five above 90 %. The top four probe sets were included in the classifier (Table RPS4Y1) across all datasets showed high male–female classification accuracy per dataset, but large overall expression shifts between datasets . First, for each cohort and for each probe set, the expression values were clustered into two groups of low and high values and a normal distribution was fitted to the low expression group, estimating location and scale with robust measures   could be explained by duplicated measurements.p value <0.01; not FDR-adjusted) was first calculated for the original datasets. Removing all unambiguously misannotated samples from the six datasets with misannotations, 12–53 % of the previously significant genes were not significant any more. In contrast, using only the reduced number of samples, the number of newly discovered genes was in the range of 9–39 % of the original number of significant genes , suggesting that sample mix-up in this case did not occur during sample collection and biobanking procedures. Subsequently, the gene expression array analysis was repeated for the misclassified sample and for five additional control samples from the previously analyzed cohort. The pairwise correlation between the new and old misclassified sample was only 0.464, strongly indicating that these two samples were derived from different individuals. In contrast, a striking correlation of 0.993 was detected between the misclassified sample and a sample from one other female patient in the previously analyzed cohort. The high correlation suggests that the mRNA sample from one female patient erroneously had been measured twice in the previous analysis. A second duplicated measurement was detected, with correlation 0.990 between the expression values of two patients with sex label male. In contrast, all correlations of the repeated control samples with the corresponding original measurements were high (correlation coefficients: 0.910–0.987).To elucidate the reason behind the sample mislabeling observed in our own non-small cell lung cancer cohort (GSE37745), one patient annotated as male in the original records and assigned as female by our classifier was re-analyzed. First, new DNA and RNA samples were prepared from the original biobanked tissue specimen. Male sex was then confirmed based on the analysis of STR marker distribution using the AmpFLSTRThe rapidly increasing number of newly published results of microarray and RNA-seq experiments reveals that genome-wide expression data play an important role in translational research