Diagnosing subclinical atherosclerosis is often difficult since patients are asymptomatic. In order to alleviate this limitation, we have developed a molecular prediction technique for predicting patients with atherogenic risks using multi-gene expression biomarkers on leukocytes.We first discovered 356 expression biomarkers which showed significant differential expression between genome-wide microarray data of monocytes from patients with familial hyperlipidemia and increased risk of atherosclerosis compared to normal controls. These biomarkers were further triaged with 56 biomarkers known to be directly related to atherogenic risks. We also applied a COXEN algorithm to identify concordantly expressed biomarkers between monocytes and each of three different cell types of leukocytes. We then developed a multi-gene predictor using all or three subsets of these 56 biomarkers on the monocyte patient data. These predictors were then applied to multiple independent patient sets from three cell types of leukocytes  to predict patients with atherogenic risks.When the 56 predictor was applied to the three patient sets from different cell types of leukocytes, all significantly stratified patients with atherogenic risks from healthy people in these independent cohorts. Concordantly expressed biomarkers identified by the COXEN algorithm provided slightly better prediction results.These results demonstrated the potential of molecular prediction of atherogenic risks across different cell types of leukocytes. Atherosclerosis is the leading cause of death in most countries . In the Tests based on surgical patient samples such as arterial tissue or atherosclerotic plaques are used to diagnose atherosclerosis but are impractical ,5. High Identifying genome-wide biomarkers of various molecular mechanisms relevant to atherogenic risks, we here developed multi-gene prediction signatures for stratifying patients with atherosclerosis using peripheral blood samples. In particular, we addressed two questions in this study: 1) whether subclinical atherosclerosis can be diagnosed by molecular biomarkers in peripheral blood samples, and 2) whether they share certain common molecular expression signatures across different cell types of leukocytes.Four published microarray sets Table  of diffeThree additional data sets of patients were obtained and used as test sets for our multi-gene predictors. The first test set, FH2 (NCBI GEO GSE6088), was from the same FH patients in the training set but with microarray data of different cell, T-lymphocytes [The third test set, ATHERO1 (NCBI GEO GSE9874), was comprised of 15 patients with asymptomatic atherosclerosis and a family history of coronary heart disease and 15 age- and sex-matched subjects with no atherosclerosis and no family history of coronary heart disease as control . Two arrOur model training of atherosclerosis predictors was composed of three sequential analysis steps Figure . The firhttp://www.ingenuity.com; Redwood City, CA). Genes related to inflammation, lipid metabolism process and another metabolic process (including carbohydrate protein and nucleic acid metabolism) and hematological system developments were selected [In the second step, we identified a subset of the above biomarkers known to have functions directly related to atherosclerosis and cardiovascular diseases. Biological functional analysis on these genes was performed by the Ingenuity Pathway Analysis  was the multivariate prediction model construction. Microarray expression values were first standardized within each gene for each set in order to adjust for biased expression distributions of the four independent data sets obtained under different experimental settings. We then applied principal component analysis (PCA) to reduce the high dimension of the multi-gene biomarker space. Top three PCAs were chosen to convey the majority (> 70%) of the information of original genes while their loadings (or weights) reflect the influence of the original genes.We trained our prediction model using the linear discriminant analysis (LDA) technique, a widely-used multivariate classification technique for our prediction modeling. First, 23 human samples in the training set were divided into a low-risk group  and a high-risk group . The LDA is then performed to obtain a linear discriminant function for the two risk classes by maximizing the prediction power or minimizing the overall classification error both for false positives and false negatives with equal weights. The output of the prediction model is the LDA posterior probabilities of atherosclerosis between 0 (low-risk) and 1 (high-risk). Thus, a high predicted score (or posterior probability) of a subject implies a high risk of atherosclerosis whereas a low predicted score suggests a low risk of atherosclerosis.The performance of prediction models was evaluated to examine if the predicted (posterior) probability of atherosclerosis risk could statistically discriminate patients at high risk for atherosclerosis from normal controls in these three test sets. The statistical significance of the difference in scores was first evaluated using the two-sample t-test between high risk and control groups. To define an optimal cutoff value for each predictor, a Receiver Operating Characteristic (ROC) curve was plotted for the predicted scores of atherosclerosis risk and phenotype information on each set. The cutoff value on each ROC curve was then determined by maximizing the Youden index (= sensitivity+specificity-1) . SensitiSensitivity, specificity, PPV, NPV were defined asWe also applied the COXEN algorithm, a previously published bioinformatics technique that can identify concordantly (expression) regulated genes between different cancer types ,16 to imThe initial biomarker discovery step Step 1) identified 363 differentially expressed genes by comparing FH patients and normal controls in the training set FH1 with q-value < 0.01  each gene's expression values across all patient samples Figure . This clWe independently evaluated the performance of our 56-gene predictor of T-cells on the FH2 set Table . To examIn the next application, we used another independent patient dataset FH3 of total white blood cells to test the predictability of the same 56-gene model for WBC. This small cohort consisted of 5 FH patient and 5 healthy controls. As shown in Figure We next applied the identical 56-gene predictor to the ATHERO1 set of asymptomatic atherosclerosis patients. This set included 28 blood macrophage samples from 14 patients confirmed with asymptotic atherosclerosis and 14 healthy controls, both groups with a family history of coronary artery disease. In particular, 14 patients were diagnosed with atherosclerosis by CTA but they had not shown any clinical symptoms with little clinical difference from the control subjects with a family history of CAD. Therefore, this stratification was challenging and we believe it could be similar to difficult cases of diagnosis in clinics.Using the prediction model of these 56 biomarkers on FH1, we tested this multi-gene predictor on macrophage cell data for predicting the risk of asymptomatic atherosclerosis. As shown in Figure While our universal 56-gene model showed consistent prediction performance on the three independent test sets, we further investigated whether these predictions could be improved by using biomarkers that preserved highly concordant expression patterns between different cell types of leukocytes. In particular, we used the COXEN algorithm originally developed to discover concordant expression biomarkers between cancer cell lines and human patients with cancer. Applying the COXEN algorithm, we identified 48, 6, and 25 genes (among these 56 genes) that preserved highly concordant gene expression patterns between patient sets FH1 and FH2, FH1 and FH3, or FH1 and ATHERO1, respectively, with a Pearson correlation association p-value < 0.001  since these two data sets were obtained from the same set of patients. For the other two data sets, the ATHERO1 set from macrophages was found to result in more COXEN genes (25 genes among 56) that were concordantly expressed with those of the FH1 monocyte set than the WBC FH3 set (6 among 56). Thus, macrophage cells appeared to preserve many common gene expression patterns with monocyte cells. On the contrary, WBC, which is a mixture of different cell types, showed less consistent gene expression patterns with monocytes than the other two sets. As shown in Table Gene function analysis by the IPA software  showed 6 of 48 biomarkers of T-lymphocyte cells were directly related to leukocyte activation and a change in morphology and behavior of a leukocyte including lipopolysaccharide binding protein (LBP), interleukin 27 receptor, alpha (IL27RA), mitogen-activated protein kinase kinase kinase 7 (MAP3K7), pallidin homolog (PLDN), signal transducer and activator of transcription 6, interleukin-4 induced (STAT6) and C-type lectin domain family 7, member A (CLEC7A). This clearly implied that T-lymphocyte activation was involved in atherogenesis. 6 genes selected for white blood cell FH3 prediction were phospholipid scramblase 3 (PLSCR3), nischarin (NISCH), bromodomain containing 4 (BRD4), huntingtin (HTT), ribulose-5-phosphate-3-epimerase (RPE) and nuclear receptor subfamily 5, group A, member 1 (NR5A1). Also, the 6 biomarkers were the phosphoproteins of which phosphate groups regulate protein functions by esterifying to serine, threonine or tyrosine. In particular, NR5A1 is known to be a nuclear receptor that regulates the transcription of many genes involved in atherogenesis and PLSCR3 has been proved to be related to atherosclerosis development by animal studies .In this study, we developed multi-gene biomarker models to predict early-stage (asymptomatic) atherosclerosis based on different types of leukocytes. In particular, we have shown a proof-of-concept strategy to predict asymptomatic atherosclerosis by using molecular biomarkers from peripheral blood samples, and the existence of common molecular expression signatures of atherogenic risks across different cell types of WBC. We believe these predictions were possible due to several reasons. First, gene expression signatures from patient blood samples appear to have a high potential to predict atherosclerosis in its early stage since certain molecular changes in blood cells may occur much before the plaque development or serious clinical symptoms. Second, a genome-wide microarray technique enabled us to comprehensibly identify relevant molecular expression signatures of atherogenic risks beyond the information obtained from standard clinical parameters. We also believe our multi-gene predictors could predict the risk of atherosclerosis more accurately than predictors based on single biomarkers or a small number of clinical parameters. Finally, we found many biomarkers of atherosclerosis shared consistent expression patterns across different leukocyte subsets. It will be interesting to further investigate these common biomarkers for their specific roles in the disease.We note that FH1 and FH2 were from the identical set of familial hyperglycemia patients and healthy controls, and possibly that our significant prediction on FH2 is correlated with the use of the identical patient set. However, we believe that our significant prediction on FH2 is not mainly due to the use of the same patient set for the following reasons. First, the molecular data of the two sets were from completely different immune cells---FH1 from monocytes and FH2 from circulating T cells. Since our biomarker discovery and predictive modeling were performed strictly based on monocyte cells of the FH1 set, FH2 is independent of FH1 for its molecular characteristics and data. Second, we observed that our identical prediction model performed considerably better for white blood samples on FH3, a completely independent set of FH patients and controls from the FH1 set. We think this was due to the fact that monocytes are partially included in white blood cells so our monocyte-based predictor presented a better predictability for that set. Therefore, the common molecular information appears to be more important than the use of specific patient set for our training and prediction.We believe our approach can be highly useful for clinical diagnosis of atherosclerosis for the following reasons. First, we used molecular signatures from patients' blood samples that can be conveniently obtained in routine clinical practice. Second, we found that molecular signatures of atherogenic risks exist and are commonly shared among different types of leukocytes so we may be able to choose and further refine diagnosis tests based on one or multiple types depending on their accuracy and clinical applicability. For example, clinical data showed that monocyte cells are involved in very early pathogenetic stages of atherosclerosis . In partThere are several limitations in our current study. First, our primary biomarker discovery and prediction model training were performed by contrasting familial hypercholesterolemia patients against healthy controls. Likely due to this restriction, our prediction was better in stratifying FH patients from healthy controls than general subclinical atherosclerosis patients. When we reversed the role of training and test sets in our preliminary analysis, i.e. used the subclinical atherosclerosis patient set for model training and FH patient sets for independent model test, the prediction results were generally deteriorated, possibly due to the small sample size of the subclinical atherosclerosis patient set (data not shown). Questions regarding whether predictors can perform significantly better if they are trained based on the same disease type of patients and/or same subtype of blood cells requires more careful investigation with a larger number of patient data in a future study. Also, we found that COXEN biomarker discovery and modeling training based on monocyte data was more successful to predict risks based on other cell types than other directions, e.g. T-cell data for training to predict the others. It may be due to the data quality or biological information in the monocyte data which requires further investigation.Even though we did not use patients' outcome information in our COXEN-based predictions, we partially used the molecular information of our test patient data sets in the current study. A more rigorous prediction performance of these predictors should thus be further evaluated using a third patient set from the same cell type of leukocytes. Our current predictors were constructed solely based on molecular data due to the lack of patients' other clinical information in our datasets. However, we believe additional predictive information can be obtained from many clinical parameters of patients such as age, gender , LDL levWe discovered 56 atherosclerosis-related expression biomarkers which showed significant differential expression between genome-wide microarray data of monocytes from patients with familial hyperlipidemia and increased risk of atherosclerosis compared to normal controls. Based on these biomarkers, we developed our multi-gene biomarker model that could predict early-stage (asymptomatic) atherosclerosis across different types of leukocytes. We also applied a COXEN algorithm to identify concordantly expressed biomarkers between monocytes and each of three different cell types of leukocytes---macrophage, circulating T, and whole white blood cells. We then developed three separate multi-gene predictors using the three subsets of these 56 biomarkers using the monocyte patient data as the training set. These individual predictors showed further improved statistical power for predicting early-stage atherosclerosis. We thus believe these predictors can be quite useful for developing diagnostic tools for patients who are at an increased risk of cardiovascular disease.The authors declare that they have no competing interests.FC designed the study, conducted data acquisition, analysis, and wrote the manuscript. EK provided clinical input and investigation directions for atherosclerosis prediction and wrote the manuscript. JL designed the study, guided the analysis, and wrote the manuscript. All authors have read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1755-8794/5/2/prepub


Microarray experiments are becoming increasingly common in biomedical research, as is their deposition in publicly accessible repositories, such as Gene Expression Omnibus (GEO). As such, there has been a surge in interest to use this microarray data for meta-analytic approaches, whether to increase sample size for a more powerful analysis of a specific disease (e.g. lung cancer) or to re-examine experiments for reasons different than those examined in the initial, publishing study that generated them. For the average biomedical researcher, there are a number of practical barriers to conducting such meta-analyses such as manually aggregating, filtering and formatting the data. Methods to automatically process large repositories of microarray data into a standardized, directly comparable format will enable easier and more reliable access to microarray data to conduct meta-analyses.We present a straightforward, simple but robust against potential outliers method for automatic quality control and pre-processing of tens of thousands of single-channel microarray data files. GEO GDS files are quality checked by comparing parametric distributions and quantile normalized to enable direct comparison of expression level for subsequent meta-analyses.13,000 human 1-color experiments were processed to create a single gene expression matrix that subsets can be extracted from to conduct meta-analyses. Interestingly, we found that when conducting a global meta-analysis of gene-gene co-expression patterns across all 13,000 experiments to predict gene function, normalization had minimal improvement over using the raw data.Normalization of microarray data appears to be of minimal importance on analyses based on co-expression patterns when the sample size is on the order of thousands microarray datasets. Smaller subsets, however, are more prone to aberrations and artefacts, and effective means of automating normalization procedures not only empowers meta-analytic approaches, but aids in reproducibility by providing a standard way of approaching the problem.Data availability: matrix containing normalized expression of 20,813 genes across 13,000 experiments is available for download at . Source code for GDS files pre-processing is available from the authors upon request. Using microarrays for hypothesis generation and validation has become routine in virtually every area of biomedical research. However, the majority of generated data is underutilized since publications reporting microarray results often focus on a small subset of the results they feel are most relevant to their research focus, even if other interesting observations may be present in the data. Thanks to the Minimum Information About a Microarray Experiment (MIAME)  to standWhether the meta-analysis is focused on specific experimental types ,7 or is Gene nomenclature poses another serious problem in comparison of different microarray platforms. Gene identifiers, such as GenBank, Illumina IDs, Affymetrix IDs have different underlying annotations and are not directly comparable. Several attempts have been made to overcome those hurdles. Two tools, List of lists-annotated (LOLA)  and ListSeveral papers have been published that thoroughly outline the challenges, opportunities and recommendations for standardization of microarray meta-analysis, discussing the benefits and pitfalls, and comparing methods for data processing ,22-26. Rftp://ftp.ncbi.nih.gov/pub/geo/DATA/SOFT/GDS/ and uncompressed from .gz compression format. Files were selected for processing if the following fields were dataset_sample_organism=“homo sapiens”, dataset_type=”nucleotide” or “gene expression”, dataset_channel_count=”single” or “1”, and dataset_value_type=”count”. This ensures only raw data from one-channel human microarray samples were processed.GEO Datasets (GDS) files were downloaded from ftp://ftp.ncbi.nih.gov/gene/DATA/GENE_INFO/Mammalia/Homo_sapiens.gene_info.gz). Unmapped probes were stored in a file and examined if any of the platforms was unmapped due to absence of its mapping data in the database. Each technological platform has unique set of probes targeting different gene regions. Moreover, some probes recognize particular isoforms of the same genes, such as implemented in Affymetrix and Illumina platforms. Affymetrix uses extensions to its unique IDs, such as “_at” indicating a probe recognizing a unique gene isoform. “_s_at” extension indicates a probe can recognize multiple isoforms of the same gene. Illumina flags its IDs by “I” (a probe recognizes a single isoform) and “A” . Due to aforementioned problems of linking diverse manufacturer’s IDs to unique gene identifiers it is logical to use probes recognizing all isoforms of a given gene. Such probes should have maximum expression value relative to other probes that only recognize individual isoforms of the same gene. Therefore, the maximum expression value was selected from multiple probes targeting the same gene.Probe mapping was done using a relational database, assigning unique Entrez ID identifiers based on gene names and accession numbers downloaded from NCBI . Data for each experiment were sorted and distribution of expression values around maximum was examined. Due to technological imperfections some genes demonstrate extreme saturated measurements at the high end of expression spectrum. Such expression values would negatively influence the following quantile normalization step by introducing artefacts, and should be adjusted. Overall, no more than 0.1% of maximum expression values showed extreme measurements. For each experiment, 0.1% of genes with highest expression values were selected, and a minimum expression value  among them was identified. Expression of these genes was replaced with this flooring value. Each experimental dataset was then adjusted to fit within 0 - 10,000 range.Low expression values in each microarray dataset constitute technical noise that can be approximated with normal distribution . To makeWe explored the data structure in 1) The matrix containing raw expression values; 2) matrix containing scaled to 0-10,000 range data; and 3) matrix containing scaled and quantile normalized data. We performed gene ontology prediction analysis  and calcEquation 1: 	Recall = TP/(TP+FN)Equation 2: 	Precision = TP/(TP+FP)Equation 3: 	F-measure = 2*Precision*Recall/.A total of 2,325 GDS files were downloaded, out of which 587 contained raw gene expression data from human single-channel microarrays. Probe identifiers were mapped to Entrez IDs, totaling 20,814 genes.Single-channel microarray data follows well-defined distribution that can be approximated with a lognormal model ,28. GeneWhile remaining datasets did not contain negative values, raw expression data distribution may show inconsistent intensities at very high expression level . Manual Gene expression matrix from 13,000 experiments contained data from different experimental platforms. Affymetrix platforms delivered highest number of datasets, with 234 datasets done on Affymetrix Human Genome U133A Array (GPL96) followed by 111 datasets on Affymetrix Human Genome U95A Array (GPL91) platform  files work best for that purpose, since they are reassembled by GEO staff and stored in text files with information fields. GDS files are standardized versions of the originally submitted GSE (GEO series) files. GSE datasets lack a standardized structure and will require manual reassembly, which currently lags the production of such datasets and is a reason others have chosen to exclude them as well , [32].One of the primary challenges in standardizing datasets is choosing an appropriate identifier for each gene. Fortunately, many tools exist to convert microarray IDs between database probe names and among the more developed are DAVID [33] and RESOURCERER . One dedSingle-channel microarray data have been shown to exhibit a well-defined distribution of their values ,28. A frData from different technological platforms have different levels of noise and probe intensity and direct comparisons could be misleading without first correcting for these issues. Quantile normalization  is a meaEach experimental platform may suffer from erroneous or artefactual expression measurements, such as negative values, or extremely high expression. Since gene expression can’t be negative, such values, if present, should be flagged. In the present study datasets that passed quality control did not have negative values, which does not guarantee their absence in other datasets that could be added to the total pool in the future. We observed extremely high expression measurements across several datasets . Such outliers may negatively affect performance of quantile normalization by skewing the average distribution in an unpredictable manner. Therefore, we “floored” top 0.1% of the data to decrease outliers’ effect. Although this “flooring” did not noticeably affect co-expression analysis (data not shown) we feel this negligible data treatment is an important precaution against possible technology-related errors.To our surprise, even raw data can be used for predicting gene ontology categories using a global meta-analysis of all available data , althougIn this work we addressed potential pitfalls and problems associated with microarray meta-analysis of large number of disparate microarray experiments  and presThe authors declare that they have no competing interests.JDW conceived of the project. MGD designed, implemented and tested the approach to normalization. Both authors wrote the manuscript. All authors read and approved the final manuscript.


The gCDA procedure was applied to simulated data and to three publicly available microarray datasets. gCDA shows very interesting performance when compared to state-of-the-art classification methods. The software package gcda, along with the real datasets that were used in this study, are available online: http://biodev.cea.fr/gcda/.Integrating gene regulatory networks (GRNs) into the classification process of DNA microarrays is an important issue in bioinformatics, both because this information has a true biological interest and because it helps in the interpretation of the final classifier. We present a method called graph-constrained discriminant analysis (gCDA), which aims to integrate the information contained in one or several GRNs into a classification procedure. We show that when the integrated graph includes erroneous information, gCDA's performance is only slightly worse, thus showing robustness to misspecifications in the given GRNs. The gCDA framework also allows the classification process to take into account as many  These structured priors are usually represented by a graph, called a gene regulation network (GRN) throughout this paper, in which the nodes are the genes and the edges represent interactions between genes. Integrating such a structured prior knowledge into the classification of microarray data is an important bioinformatics research field and has recently been addressed in the literature  - , which is a constrained version of the discriminant analysis The present work is structured as follows: the first section is dedicated to the presentation of gCDA and of the state-of-the-art methods to which it is compared. The second part is devoted to the validation of gCDA on a simulated dataset and three publicly available gene expression microarray datasets The integration of a graph into the classification process of microarray data requires that GRNs describing the dependencies between genes and a given microarray dataset are considered. The structure of these two objects is radically different, and the challenging task we attempt to overcome in this paper is to combine these two sources of information.Let We consider finite, undirected graphs to model GRNs. A graph, The Laplacian of a graph with et al.Rapaport et al. integrate the given graph by adding constraints to the classical SVM optimization problem. These additional constraints encode the fact that two adjacent variables must have close weights in the final model. This idea is further developed by Zhu et al.In a more recent study with et al.The two methods described above are intended to solve the issue that variables connected in a given graph must have close coefficients in the decision function. This type of constraint clearly helps the interpretation of the resulting classifier, but it is not specifically designed to improve the performance of the classification, even if Zhu In a nutshell, we propose to regularize the estimation of the covariance matrix by integrating information contained in the GRN(s). The resulting estimator can then simply be used in the context of DA. As described by DA is a simple, yet very popular, classification method with Considering that there are only two classes, the transformation It was shown  framework: The dataset is randomly split into a training dataset (two thirds), and the rest of the dataset is used as a test dataset. The whole procedure is iterated 100 times. The tuning parameters  to build a covariance matrix,and model the two classes by random multivariate Gaussian variables A comparison of the results obtained from simulated data for gCDA, NB-SVM, RDA, and SVM is presented in To explore the limits of gCDA, we also run the linear version of the method on a simulated dataset containing Additionally, The classification performance of gCDA also depends on the quality of the integrated graph.We empirically observed the evolution of gCDA's performance as a function of the quality of the integrated graph on simulated data. Starting with the graph that was used to simulate the dataset, we generated a set of gradually different graphs by randomly reassigning some of its edges to different vertices. The difference between two graphs is calculated as the number of different vertices between the union of the two graphs and their intersection, which corresponds to the structural Hamming distance To evaluate the performance of gCDA with real data, we chose three gene expression microarray datasets. The characteristics of the three datasets are summarized in When dealing with gene expression microarray datasets obtained from specific tissues and under particular experimental conditions with the gCDA method, two major issues must be pointed out: 1) the graph describing the various interactions between genes is not known and has to be inferred, and 2) differences between the covariance matrices of the two classes should be evaluated. In the results reported for the real data, we investigated pragmatic choices. We selected two recognized approaches to infer the GRNs to be integrated in the classification and we also built GRNs based on reported interactions gathered in the Kyoto Encyclopedia of Genes and Genomes (KEGG) ARACNE ridge.net and the KEGG pathway hsa05200, that pertains to the biological samples we considered. The network was extracted thanks to the Bioconductor library KEGGgraph For the two first kinds of GRN mentioned above, we inferred the graphs based on a dataset independent from the one used for the classification process.These methods impose limits on the set of variables to be considered down to several hundreds. Therefore, we selected a restricted set of genes corresponding to the KEGG pathway for human cancer (hsa05200). To avoid any bias in the classification process, we never used the same dataset to compute the GRNs and to measure the classification performance. We considered a couple of distinct datasets see  correspohttp://genome.ucsc.edu/, March, 2006 (NCBI36/hg18)). For each gene, we chose the probe set whose position is the closest to the transcription initiation site. When several probe sets were selected, the mean value of the measurements was computed.When necessary, we re-annotated the probe sets to associate them with a corresponding specific gene . We used the UCSC database  and to gCDA (RDA) to focus on how much the integrated graph can improve the classification performance. Both the interpretability and the performance of our classifier is clearly not necessarily improved compared to the approach of Rapaport


An important use of data obtained from microarray measurements is the classification of tumor types with respect to genes that are either up or down regulated in specific cancer types. A number of algorithms have been proposed to obtain such classifications. These algorithms usually require parameter optimization to obtain accurate results depending on the type of data. Additionally, it is highly critical to find an optimal set of markers among those up or down regulated genes that can be clinically utilized to build assays for the diagnosis or to follow progression of specific cancer types. In this paper, we employ a mixed integer programming based classification algorithm named hyper-box enclosure method (HBE) for the classification of some cancer types with a minimal set of predictor genes. This optimization based method which is a user friendly and efficient classifier may allow the clinicians to diagnose and follow progression of certain cancer types.We apply HBE algorithm to some well known data sets such as leukemia, prostate cancer, diffuse large B-cell lymphoma (DLBCL), small round blue cell tumors (SRBCT) to find some predictor genes that can be utilized for diagnosis and prognosis in a robust manner with a high accuracy. Our approach does not require any modification or parameter optimization for each data set. Additionally, information gain attribute evaluator, relief attribute evaluator and correlation-based feature selection methods are employed for the gene selection. The results are compared with those from other studies and biological roles of selected genes in corresponding cancer type are described.The performance of our algorithm overall was better than the other algorithms reported in the literature and classifiers found in WEKA data-mining package. Since it does not require a parameter optimization and it performs consistently very high prediction rate on different type of data sets, HBE method is an effective and consistent tool for cancer type prediction with a small number of gene markers. Microarray technology provides wealth of information on expression levels of thousand genes that has been used for diagnostic and prognostic purposes for various types of diseases. The data obtained from microarray measurements leads to understanding of genes that are being regulated under the disease conditions including cancer both in biology and clinical medicine at the molecular level An important characteristics of microarray data is the large number of genes relative to number of samples. This high dimensionality in gene space increases the computational complexity while it usually decreases the accuracy of the classification. This fact brings the necessity of gene selection by ranking or gene reduction for the high dimensional gene space. The relevance of genes in cancer occurrence can be categorized into three classes: Strongly relevant, weakly relevant and irrelevant genes The effectiveness of the selected gene subset is measured by its prediction accuracy or error rate in classification. IN microarray experiments, classification of data is a crucial step for the prediction of phenotype of cells. Different machine learning approaches have been employed to analyze microarray data including k-nearest-neighbors In this work, we show that a systematic and efficient algorithm, mixed integer linear programming based hyper-box enclosure (HBE) approach, can be applied to classification of different cancer types efficiently. We first introduce and establish a consistent classification method for different types of microarray data. Second we provide an optimal set of genes as best diagnostic indicators for different cancer types that gives the highest accuracy in classification. Information gain attribute evaluator, relief attribute evaluator and correlation-based feature selection (CFS) methods are used for gene selection. We conduct experiments using 6 well known cancer data sets including leukemia data set http://datam.i2r.a-star.edu.sg/datasets/krbd/) and Artificial Intelligence Laboratory (http://www.ailab.si/supp/bi-cancer/projections/index.htm) databases  with 12,600 genes. Considering the results given in The selected genes by HBE are serine protease hepsin (X07732), nel-related protein 2 (D83018), ao89h09.x1 (AI207842), Cdk-inhibitor p57KIP2 (U22398), DKFZp564I1663-r1 (AL036744), adipsin/complement factor D (M84526), glutathione transferase 4 (GSTM4) (M96233), DKFZp586K1220 (AL050152), aldose reductase (X15414), ADP/ATP translocase (J03592). In fact, many of the genes that have been selected in this study shown to display different expression patterns in prostate cancer tissues. For example hepsin (a cell surface serine protease) is significantly upregulated in human prostate cancer and it promotes cancer progression and metastasis of prostate This data contains 8 patients having relapsed and 13 patients having non-relapse with measurements of 12600 genes.The genes, selected with HBE are shown to play a critical role in prostate cancers, are human cofactor A protein (AF038952), farnesyl-protein transferase beta-subunit (HUMFPTB), glutamine-fructose-6-phosphate amidotransferase (GFAT) (M90516). The function of heterodimeric enzyme farnesyl: protein transferase (FPTase) 3 is to transfer of a 15-carbon isoprenoid moiety to a C-terminal cysteine of many cellular proteins. The inhibition of farnesyl protein transferase affects for the prevention of proper functioning of the Ras protein that leads to oncogenesis or cancer. In fact, elimination of cancer were reported to increase when prostate cancer cell line is treated with farnesyl-protein transferase inhibitor et al. This data set contains 58 samples from DLBCL patients and 19 samples from follicular lymphoma. The gene expression profiles were analyzed using Affymetrix human 6800 oligonucleotide arrays. HBE method is the most accurate classifier among the classifiers with both 10-CV and LOOCV in the DLBCL data set . From thHBE analysis resulted in high classification accuracy with 6 genes which are DNA replication licensing factor CDC47 homolog (D55716-at), gamma-interferon-inducible protein IP-30 precursor (J03909-at), LDHA lactate dehydrogenase A (X02152-at), CD69 antigen (Z30426-at), SLC (AB002409-at), Rad2 (HG4074-HT4344-at). When the function of those proteins and their role in DLBCL were analyzed, we see that these proteins play a significant role in the progression of DLBCL. DNA replication licensing factor CDC47 is a factor that helps the DNA to undergo a single round of replication per cell cycle. CDC47 is not only necessary for DNA replication and cell proliferation, but also for S-phase checkpoint activation upon UV-induced damage The lymphoma data set contains 47 samples, 24 of them are referred to as germinal center B-like group and 23 are activated B-like group. This data set contains expression of 4026 genes.In the lymphoma data set, HBE method is the most accurate classifier using both ten-fold-cross-validation and leave-one-out cross validation with the accuracies of 96.40% and 97.87% respectively . Supportalgorithm selects the following genes: Deoxycytidylate deaminase (19408), lymphoid-restricted membrane protein (JAW1) (16886), PKU-beta = KIAA0137 = protein kinase (20423), T-cell protein-tyrosine phosphatase (17140), TTG-2 Rhombotin-2 (19238), stress-activated protein kinase (JNK3) (19384), and unknown labeled genes with ids 19288, 19274, 13394. Expression of the following genes are currently being used as markers for lymphoma in clinical diagnosis. Deoxycytidylate deaminase (dCMPase) hydrolyzes dCMP into dUMP, and it is suggested that this gene is a marker of the aggression of human lymphoid malignancies There are four different small round blue cell tumors in this data set: Ewing family tumor (EWS), Burkitt lymphoma (BL), neuroblastoma (NB) and rhabdomysarcoma (RMS). The training set contains 63 samples and the test set contains 20 samples. The cDNA microarrays comprise 2308 genes. Small round blue cell tumors (SRBCT) of childhood are diagnosed using single layer neural network et al. et al. et al. et al. The selected genes with their gene ids in SRBCT classification are Fc fragment of IgG, receptor, transporter, alpha (FCGRT) (70394), transmembrane protein (812105), fibroblast growth factor receptor 4 (784224), ESTs (295985), recoverin (383188). FCGRT encodes a receptor binding the Fc region of monomeric immunoglobulin G. This protein both helps to transfer of immunoglobulin G antibodies from mother to fetus across the placenta, and binds to immunoglobulin G to prevent the antibody degradation The contributions of this work are two-fold. The first contribution is that we implement an effective optimization based classifier that gives very high performance and valuable insight into different type of cancer data sets. Previously it has been shown that our approach was successfully applied to protein folding and drug classification problems. HBE approach does not require parameters to optimize in order to obtain high classification accuracies. This method can be used for different types of data without any modifications. The second contribution is finding of optimal predictor genes that give the highest accuracy in classification. This effort can provide to develop antibody assays for the diagnosis of specific types of cancer and to provide accurate diagnostics by only measuring expression of few genes. We have applied our algorithm on publicly available data sets including leukemia data set, two prostate cancer data sets, two lymphoma data sets and SRBCT data set. In conclusion, mixed-integer programming based hyper-box enclosure approach is robust and effective method for microarray analysis.We performed scaling to all data sets by using the expression ofet al. There are generally three types of approaches in feature (gene) selection: Filters, wrappers, and feature weighting. Filter methods eliminate irrelevant features according to some prior knowledge. Wrapper approaches use machine learning algorithms to evaluate the feature subsets; however, they have high computational complexity when they combined with classification algorithms. Feature weighting methods simply weigh features instead of selecting a subset of features that is a combinatorial problem. We employed information gain attribute evaluator, relief attribute evaluator, and correlation-based feature selection (CFS) from Weka machine learning package Information gain evaluates a feature (gene) by measuring the information gain with respect to the class:Relief attribute evaluator is an evaluating algorithm that rates features (genes in our case) due to these facts: (1) how well their values distinguish among samples of different classes (tumor type in our case) (2) how well they cluster instances of the same class Correlation-based feature selection (CFS) is a fast algorithm that reveals a good feature subset that contains features highly correlated with the class, yet uncorrelated with each other The objective in data classification is to assign samples that are described by several attributes into a predefined number of classes. In this study, we propose HBE algorithm for the classification of cancer types. This algorithm consists of integer programming (IP) and mixed integer linear programming (MILP) based components, and the data points belonging to different classes are discriminated by hyper-boxes. The use of hyper-boxes for defining boundaries of the sets that include all or some of the samples in that set can be very accurate on both two-class  and multi-class (more than two tumor types) problems. If it is necessary, more than one hyper-box could be used in order to represent a class. When two classes are both represented with a single hyper-box respectively, the boundaries of these hyper-boxes may overlap. Thus, two boxes could be constructed in order to eliminate this overlapping. The description of the optimization model is given in This step is used to improve the computational efficiency by determining representative seeds for each class . Seed fiAs it is proven in Uney-Yuksektepe Furthermore, different classification models will always develop different models  for different data sets. In classification problems, benchmark data sets are used in order to compare the results of different methods. As the same benchmark data sets for each tumor problem are used to compare distinct models, the comparisons are unbiased and stable for this study. For instance, the most popular data classification method Support Vector Machines (SVM) will produce different hyper-planes for perturbations in the original data set. Hence, differences between the constructed hyper-boxes of different data sets are not an interesting and problematic issue for data classification.The objective of the IP-Seed problem given in Equation (6) is to minimize the distances from each seed to sample of its group (in-class distances) and maximize the average distances from each seed to the samples that belong to other classes (out-class distances). Equation (7) states that every class must have exactly one seed, and integrality of the decision variable Training part studies are performed on a training data set composed of a number of samples Minimization of the number of misclassified samples in the data set with the minimum number of hyper-boxes is the objective of the mixed integer linear programming model. The objective function isThe lower and upper bounds of the hyper-boxes are determined by the samples that are enclosed within the hyper-boxes. Hence, lower and upper bounds of hyper-boxes are calculated by related constraints. Moreover, the bounds of hyper-boxes exist if and only if this hyper-box is assigned to a class. There exist constraints that ensure the assignment of each data point to a single box and single class Since the MILP model is solved for ‘problematic samples’ only, the ‘non-problematic samples’ are assigned to hyper-boxes in a straight forward manner . We defier-boxes .After the initial gene ranking, the optimal gene set is searched by using Supporting Information S1MILP formulation of the hyper-box enclosure approach.(0.05 MB PDF)Click here for additional data file.Supporting Information S2Source codes, scripts and data files.(0.78 MB ZIP)Click here for additional data file.


An under-appreciated aspect of the genetic analysis of gene expression is the impact of post-probe level normalization on biological inference. Here we contrast nine different methods for normalization of an Illumina bead-array gene expression profiling dataset consisting of peripheral blood samples from 189 individual participants in the Center for Health Discovery and Well Being study in Atlanta, quantifying differences in the inference of global variance components and covariance of gene expression, as well as the detection of variants that affect transcript abundance (eSNPs). The normalization strategies, all relative to raw log2 measures, include simple mean centering, two modes of transcript-level linear adjustment for technical factors, and for differential immune cell counts, variance normalization by interquartile range and by quantile, fitting the first 16 Principal Components, and supervised normalization using the SNM procedure with adjustment for cell counts. Robustness of genetic associations as a consequence of Pearson and Spearman rank correlation is also reported for each method, and it is shown that the normalization strategy has a far greater impact than correlation method. We describe similarities among methods, discuss the impact on biological interpretation, and make recommendations regarding appropriate strategies. The nine methods are: raw log2 transformation (RAW); mean centering (MEA); IQR variance normalization; gene-by-gene adjustment for date and RNA integrity with variance standardization(dr3); date and RNA integrity adjustment followed by mean centering of each sample profile (DRM); further adjustment for the absolute number of lymphocytes, monocytes, neutrophils (LMN), erythrocytes, and platelets; fitting the first 16 principal components to the data (PCA); QNM; and implementation of the SNM procedure to remove technical and blood cell count effects and adjust for a series of biological covariates simultaneously. We document the widespread impact of these methods, draw conclusions regarding similar aspects of their performance, and discuss the implications for interpretation of hypothesis testing.This study reanalyzes peripheral blood transcriptome data from the Center for Health Discovery and Well Being (CHDWB) study of 189 health adults from Atlanta, GA, USA. Gene expression profiles were generated with Illumina HT-12 V3 bead arrays, and the raw data is available at GEO as study accession number GSE35846. We consider here the abundance of transcripts measured with 14,343 probes that are consistently detected across multiple datasets of peripheral blood samples, which for this study were obtained from Tempus tubes  that preserve whole blood RNA. Whole genome genotypes were measured using Illumina OmniQuad arrays. After quality filtering, we retained 34,548 common variants (MAF > 0.05) on Chromosome 6 for the eQTL analyses. All statistical analyses were performed in JMP Genomics v5 .http://www.gibsongroup.biology.gatech.edu/supplementary-data. RAW refers to the average bead fluorescence intensity for each probe obtained directly from Bead Studio without background subtraction, with log base 2 transformation but no adjustment across arrays. MEA refers to mean centering of the RAW profiles for each sample, namely an additive shift on the log base 2 scale that ensures that the mean value is the same for each individual, but the shape and variance of each profile is not adjusted. Technical batch and RNA quality effects were adjusted giving rise to the dr3 profiles, by fitting an ANOVA to each probe with fixed effects of hybridization date and Bioanalyzer RNA Integrity Number (RIN) and then standardizing the residuals to yield z-score gene expression measures . Date has five levels where between 24 and 48 samples (three to six chips each with eight arrays) were hybridized on each of 5 days in a 1 month period in July/August 2010. RIN was categorized with three levels, namely poor quality (RIN < 7), moderate quality (7 < RIN < 8), or good quality (RIN > 8). DRM refers to profiles obtained by mean centering of the dr3 profiles, which ensures that there is no bias in the overall distribution of transcripts with relatively low or high expression in each individual, as expected biologically. The dr3 profiles were subject to an alternate transformation adjusting for blood cell counts, giving rise to the LMN profiles by fitting probe-specific multiple linear regression with counts of Lymphocytes, Monocytes, Neutrophils, Erythrocytes, and Platelets , and retaining the residuals.Normalization was performed as follows. Each of the data files is available from the authors’ website at IQR refers to the InterQuartile Range, namely the distribution of each RAW log base 2 profile adjusted to ensure that the range between the 25th and 75th percentile values is 1 and that these are the same for each sample. This produces more similar variance structure than the MEA transform, while also ensuring that all arrays have similar means. QNM refers to quantile normalization, which is a density-adjusted rank ordering. For each sample, each probe is ranked according to intensity and then the average intensity of each rank is computed. The probe is assigned that average value, resulting in identical overall distributions.Two types of variance transformation were performed. SNM refers to supervised normalization of microarrays and was performed using the R package of that name from Bioconductor . PCA refers to profile residuals after fitting a multiple linear regression with each of the first 16 principal components of expression of all 14,434 probes across the 189 samples. These 16 PC explain 61.5% of the variance cumulatively; all remaining PC explain less than 1% each.The other two normalizations considered here are SNM and PCA. x–y plots of significance as the negative logarithm of the p-value against fold-change in gene expression between the indicated groups.Variance component analysis was performed using the Basic Expression Workflow routine in JMP Genomics. The first five principal components of the total gene expression dataset were computed, and then an average of the proportion of the trait explained by each of these five PC, weighted by their contributions to the total gene expression, was computed. Immuno-informative axes of variation are defined as the first PC of 10 definitive genes for each of seven axes described in . Volcano plots are simply z-score distributions with different means  of the five major blood cell types that a priori are likely to impact global gene expression profiles. The bottom row shows the effect of the more aggressive normalization procedures. PCA objectively removes most of the sources of covariance without regard to the source; the resultant standardized distributions are further mean-centered for all subsequent analyses. The QNM plot shows how QNM forces all samples to the same overall distribution, and clearly the SNM procedure is almost as effective at adjusting both the mean and variance of the distributions, but in a more experimentally  motivated manner.A visual representation of the impact of the normalization on data distributions is provided in Figure In light of the dramatic impact of normalization on the data distributions, it is to be expected that patterns of covariance of gene expression might also be affected. We visualize this in two ways. Figure r > 0.5) across all of the normalization methods with the exception of PCA (which was expected to remove this factor). There are two groups with almost identical PC1 eigenvalues: RAW, MEA, IQR, QNM, and dr3/DRM, LMNEP, SNM. This is as expected, and shows that cell counts have very little impact on the major axis of variation in peripheral blood. Surprisingly, the QNM procedure does not separate its PC2 from PC1 assessed with the other procedures.The potential impact of the normalization procedures on biological inference is perhaps more readily appreciated by evaluating the impact on the major principal components of residual gene expression variation, as shown in Figure The next three PC are correlated to varying degrees with neutrophil, monocyte, and lymphocyte counts in particular. The two methods that adjust for cell numbers, LMN, and SNM, alter PC2 substantially, with the result that strong correlations are observed between their PC3 scores and PC2 with the other methods. Since the minor PC only explain 2–4% of the variance each, slight changes in variance component can change their relative rank, as well as their sign . It is also striking that although PC are by definition orthogonal within a normalization, across normalizations they generally pick up overlapping components of covariance so that a single PC in one analysis can significantly correlate with multiple PC in another analysis: thus, PC4 under DRM evidently captures covariance that contributes to each of PC2, PC4, and PC5 under MEA normalization. It is also important to note that the impact of QNM is biologically difficult to interpret as the PCs show the least similarity with those derived from the other methods. By contrast, SNM removes the cell abundance effects as expected and generally shows a covariance structure that is a composite of fitting cell types and technical factors.Mirroring the changes in correlation structure, normalization can have a dramatic impact on the covariance of the principal components of variation with traits of interest. Here we consider just four: Gender, Ethnicity, and partitioning of BMI into low, normal/overweight, and obese reflecting their proportions in the population, and of Age into four levels, namely young, 40s, 50s, and older. A weighted sum of the proportion of variance of each that is captured by the first five principal components is shown in Table p < 0.00007) after fitting a multivariate ANOVA with AGE, BMI, Gender, and Ethnicity. There is a threefold range in the total number of highly significant associations detected, with the least observed for analysis of the RAW data, and the most overall for QNM. Age associations that are partially correlated with the technical covariates in this sample are not detected after QNM, while SNM facilitates enhancement of the BMI effect, possibly at the expense of Gender and Ethnicity effects. The simple expedient of mean centering is at least as effective as the mild IQR variance adjustment. Also indicated is the impact of failing to adjust for array effects after fitting the technical and cell number covariates (compare the dr3 and DRM rows), since overall profile differences are mildly correlated with the biological factors. Once again, fitting the first 16 PCA completely removes most trait associations. Similar trends are observed at less stringent significance thresholds.Similarly, gene-by-gene modeling of the association between transcript abundance and continuous trait measures is a strong function of normalization. Table z-scores for the latter. All points in the dr3 plot align on a simple curve, since all genes have the same variance after standardization . The correlation in the p-values for both Ethnicity and Gender is high across all eight methods , but note that there is a strong over-estimation of the significance of the Gender effect in QNM relative to SNM  and that many genes are only called significant for Ethnicity with either procedure (red and blue circles). Given the wide variety of significance thresholds adopted for taking genes forward for downstream processing steps such as Gene Ontology analysis, it is not clear whether normalization has as larger effect than simply setting the threshold for inclusion of genes in downstream analysis. The impact will largely be study-specific, but these analyses indicate that it will rarely be negligible.The good news in this analysis is that there is extensive overlap in the most significant transcripts identified after each normalization approach. Figure −8 threshold for genome-wide significance, there are anywhere from 39 to 88 independent associations, depending on the normalization approach. Relaxation of significance thresholds recovers most of the associations detected in the more sensitive analyses. There are few instances where a highly significant eSNP effect observed with one analysis is not seen at least nominally with the others, indicating that the normalization does not generally induce spurious associations. Yet the more than twofold difference in detection rate is striking, and would have potential consequences where the purpose is to compare regulatory profiles across conditions and/or tissues.In order to evaluate the effect of normalization on the ability to detect regulatory influences of locally acting SNPs on transcript abundance, we next performed so-called cis-eSNP analysis  for IQR eSNP associations), but recovers most cis-associations detected by the “better” normalization strategies. It does not however improve on those strategies, since both SNM and QNM uncover more cis-associations both individually and at the probe level. It has been argued that PCA should remove all sources of environmental variance, hence enhancing the genetic contribution to expression variation and leading to the detection of more cis-eSNPs. This however clearly does not happen, but we also note that the number of PCA that are fit can also influence the results, as discussed in , it may be the relative abundance that is of most interest  centering has no effect on the covariance structure, but it is an effective first step toward removing pervasive biases across the transcriptome and demonstrably increases the variance explained by the first five principal components, increases the number of transcript associations with biological covariates, and increases the power of eSNP analysis. Variance transforms such as IQR and QNM can have effects ranging from subtle to substantive: in this particular dataset, QNM produces stronger trait associations, but there is some suggestion that it perturbs the covariance structure in ways that may over-estimate some effects and under-estimate others. Fitting Principal Components may be desirable in situations where technical variance is suspected to obscure relatively weak biological signals , but in many situations the major PC actually capture important biological sources of variance  and so their removal is not advisable.Perhaps more importantly, it is very clear that neither centering nor variance adjustments do anything about confounding effects of either technical or biological covariates: the batch effects of hybridization on different dates continue to account for over a third of the total transcript variance. In an earlier study of peripheral blood profiles  were modeled as fixed categories, with three levels of RIN. There is a danger in over-fitting by binning samples into small classes that may also make the analysis susceptible to outliers, but this can generally be controlled by plotting the principal components as a function of the categorical levels – and in turn, such data visualization can be used to help define the appropriate bins. Second, we also performed linear adjustment for the abundance of five types of contributing blood cell by multivariate regression of each transcript on the five measures. Inspection of the beta coefficients can be used to assess which transcripts are strongly affected by cell abundance, and it is clear that at least a quarter of the transcriptome varies as a function of cell types. Recognizing this, though, underscores the motivation for transcript-specific adjustment: centering and variance transforms treat all transcripts equally, but they are not all equally affected by the technical covariates.z-scores, namely standard deviations of one around a mean of zero for each transcript independently. It is important also to post-process the scores to account for array effects, namely to re-center the residuals, otherwise some arrays systematically have low or high values, which is probably biologically implausible – it is unlikely that any one individual will have expression below the average for all transcripts, for example. The difference between dr3 and DRM shows that this process restores the association of transcripts with traits and the ability to detect eSNPs, and it does so without the concern that associations may be driven more by covariance between biological and technical factors. This mode of adjustment can be extended to include latent, or surrogate, variables (Leek and Storey, When using these procedures, we chose to standardize the residuals so that the abundance measures are in A more computationally demanding, but intuitively appealing, approach is to simultaneously fit the covariates to all probes simultaneously. This is the approach taken by the SNM of microarrays and related procedures (Listgarten et al., p-value distributions following standard normalization procedures, Storey and co-workers (Mecham et al., p-values, and focused the analysis on a consistent set of biomarkers (Desai et al., Nevertheless, it is desirable to have metrics of performance that might guide the choice of the most appropriate normalization strategy. Probably the most useful is repeatability, which can be frustratingly elusive in microarray datasets. In fact, this issue was the motivation for the development of the SNM approach: faced with four parallel trauma datasets that yielded incomparable Another metric could be the consistent detection of expected patterns of covariance. Figure In summary, our consideration of nine different normalization strategies highlights how the first phase of analysis of gene expression datasets can influence various aspects of statistical and in turn biological inference. Similar arguments can be made with respect to other types of high throughput data including RNA-Seq, methylation and chromatin profiling, and comparative genome hybridization. Where the dataset is known to be affected by multiple biological factors, or suspected to be perturbed by hidden variables, it will generally pay to explore and compare diverse strategies, searching both for consistency, as well as differences that may themselves point to interesting biology.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Several groups have employed genomic data from subchronic chemical toxicity studies in rodents (90 days) to derive gene-centric predictors of chronic toxicity and carcinogenicity. Genes are annotated to belong to biological processes or molecular pathways that are mechanistically well understood and are described in public databases.To develop a molecular pathway-based prediction model of long term hepatocarcinogenicity using 90-day gene expression data and to evaluate the performance of this model with respect to both intra-species, dose-dependent and cross-species predictions.Genome-wide hepatic mRNA expression was retrospectively measured in B6C3F1 mice following subchronic exposure to twenty-six (26) chemicals  previously studied by the US National Toxicology Program. Using these data, a pathway-based predictor model for long-term liver cancer risk was derived using random forests. The prediction model was independently validated on test sets associated with liver cancer risk obtained from mice, rats and humans.Using 5-fold cross validation, the developed prediction model had reasonable predictive performance with the area under receiver-operator curve (AUC) equal to 0.66. The developed prediction model was then used to extrapolate the results to data associated with rat and human liver cancer. The extrapolated model worked well for both extrapolated species . The prediction models implied a balanced interplay between all pathway responses leading to carcinogenicity predictions.Pathway-based prediction models estimated from sub-chronic data hold promise for predicting long-term carcinogenicity and also for its ability to extrapolate results across multiple species. Over the past approximately 40 years, only around 600 chemicals have been tested by the NTPs in their two-year cancer bioassay and only about 5–6 times this number have been tested worldwide.There is push to use a broad array of biological data in toxicity testing to improve reliability and provide reasonably quick indications of animal and human toxicity of chemical compounds. The National Toxicology Program (NTP) 's High Throughput Screening (HTS) in vitro results from the various cell lines to the human in vivo context.The goal of toxicity testing is ultimately to protect human health. Even though the two-year cancer bioassay is performed on rats and mice, there is considerable data supporting its use to screen for carcinogens in humans An alternative to the cell-based assays in the HTS and ToxCast programs is the use of short term molecular data  from exposed animals or humans to predict future toxicity or carcinogenicity. Others have used gene expression data based classifiers that would distinguish a toxin or carcinogen from a non-toxin or non-carcinogen A characteristic of gene expression data is the very large features (genes) to samples  ratio. This characteristic contributes to the lack of identification of robust classifiers as demonstrated by In order to gauge quantitative predictive accuracy of a pathway-based prediction model we obtained gene expression data from the livers of mice dosed with 26 chemicals over a period of 90 days. All the 26 chemicals were tested in a two-year bioassay in the same strain of mice dosed via the same respective routes. Thus, the differential liver carcinoma and adenoma rates between treated and control animals from the results of the two-year bioassays are known. Using the data, we derived pathway-based models to predict the differential tumor rates at the end of a two-year bioassay. The use of a pathway-based model would result in a reduction of a model with around 6000 gene features to one with around 200 pathways in the KEGG database 1 mice are summarized in The chemical treatments used in the experiments on female B6C3FTo address the question of species extrapolation, gene expression and tumor data from chemical exposures in Fischer 344 rats were obtained from the literature The pathology review of the treated mice after 90 days of exposure to hepatocarcinogens did not identify liver tumors see . Hence, 2-transformed. The entire set of data sets associated with risk factors for liver cancer is summarized in Genetic risk factors in terms of single nucleotide polymorphisms associated with various human diseases including liver cancer are tabulated on databases like the Genetic Association Database The biochemical pathways used in the analysis in this paper were obtained from the Kyoto Encyclopedia of Genes and Genomes (KEGG) Pathway database 4 randomizations , cN the number of chemicals . iz is given by ijZ is given by Specifically, the poly-3 statistic for a chemical obtained from the two-year cancer bioassays is assumed to be functionally related to the 90 day gene expression-based perturbation of the pathways :The framework of the so-called Super Learner CV.SuperLearner function implemented in the SuperLearner package. Specifically, the predictions for the chemicals in each of the 5 test sets (corresponding to the 5-folds) are derived from an additional 5-fold cross-validation of the remaining 20 or 21 chemicals in the 5 training tests.Five-fold cross-validation was chosen where the chemicals in each of the five folds were fixed (Table S3) and chosen so that distribution of high and low carcinogenicity values in each fold was more or less the same. Predictions for all the data sets were based on this 5-fold cross validation framework. In order to get ‘honest’ predictions for the chemicals with data from mice, because these chemical data are used to make predictions for the very same set of chemicals, two levels of 5-folds cross-validation was implemented using the The two year cancer bioassay carcinogenicity calls of the chemicals used in the mice and rats experiments along with the continuous predictions from the model ) derivedSL.randomForest.1_All in importance for each of the biochemical pathways used in the prediction model. The importance of a pathway is a measure of how much it contributes to increases the accuracy of the predictions.Different prediction models were evaluated in terms of their 5-fold cross-validated risk for fitting the model given by The model derived using the mouse data was used to predict the carcinogenicity in mice, rats and humans for the corresponding data sets. In this manuscript, the predicted values were treated as proxies for the continuous NTP carcinogenicity values in The false positives and false negatives in each of the three species at a level of specificity between 56–75%  is summarized are This paper presents results only for pathway-based prediction models. Gene-based prediction models were also evaluated and they showed a similar performance in terms of the AUC metric (data not shown) The importance measures for each of the pathway-based features are reported in The predicted dose response for the chemicals in The analyses involved in this paper worked with hepatocarcinogenicity prediction models trained using data from mice and extrapolated to data sets from rats and humans. Alternate models derived from rats to predict carcinogenicity in other species could be attempted. This was not attempted here because of lack of sufficient analogous two-year cancer assay data for the rat and human data sets. The model derived here used data for chemicals at their maximum tolerable dose (MTD) levels. It could be argued that responses at lower doses are the ones with the most human relevance. Again this was not attempted here because of lack of sufficient number of chemicals dosed at lower levels. One possibility of understanding chemical-specific low-dose mechanisms would be the availability to a relatively large epidemiological data set of exposed people . The mice have a relatively high background rate of developing HCC. In humans, HCC is thought to arise in a background of chronic inflammation, necrosis and regeneration, fibrosis and extracellular matrix deposition However, in the predictive toxicology context of the manuscript, in spite of the known differences in etiologies, our hypothesis is that gene expression levels of the precancerous lesions are the similar in mice and humans. This hypothesis is validated by the relatively high predictability of human hepatocarcinogenicity using an independently derived predictive model using 90 day gene expression data from B6C3F1 mice. Further, the work in Hoeneroff et al suggests that this hypothesis is not without justification. In their work the authors found similar gene expression profiles in cells from HCCs obtained from mice and human samples. The situation is analogous to the case of azoxymethane induced colon cancer In summary, we rigorously derive and evaluate a biochemical pathway based hepatocarcinogenicity prediction model. Among the set of alternate prediction models , random forests were found to perform the best in terms of cross-validated risk. The model used gene expression data from a given tissue at the end of a short term study to predict the risk for development of tumors at the end of a longer period of time. Specifically, the model is evaluated using gene expression data obtained from the liver of mice treated with a range of 26 chemicals over a period of 90 days. The model with the information on affected pathways derived using these gene expression data had sufficient signal to adequately predict the two-year liver carcinogenicity risk of the same chemicals as evaluated in the National Toxicology Program's two-year cancer bioassay. The fact that the model was developed at the biochemical pathway level allows one to reasonably expect conserved behaviors of the chemicals at the pathway level across multiple species. This belief was validated using the model developed for mice to predict results in humans and rats. This fact was observed in Figure S1Clustergram of transformed p-values ) representing the enrichment of the 216 pathways across the 26 chemicals treatments in mice, was generated using hierarchical clustering with the euclidean distance metric and average linkage to generate the hierarchical trees of pathways and chemicals using the Cluster and Tree view programs .(PNG)Click here for additional data file.Text S1Provides additional description of the microarray experiments, involved in mice and rats, the data for which are used in the analysis. Also provided are details of generation of (DOC)Click here for additional data file.Table S1List of rat hepatocarcinogen treatment groups from Auerbach et al., 2009.(XLS)Click here for additional data file.Table S2List of chemicals in  that were chosen to be in each of the 5 folds in the cross-validation analysis performed.(XLS)Click here for additional data file.Table S3List of various learning algorithms attempted in the SuperLearner package  and the cross-validated risk for the case of pathway-based continuous predictions.(XLS)Click here for additional data file.Table S4The list of pathways used as predictors along with their importance measures as reported by the random forests learning algorithm . The importance measures for each of the pathways are also reported.(XLS)Click here for additional data file.


Finding genes that are differentially expressed between conditions is an integral part of understanding the molecular basis of phenotypic variation. In the past decades, DNA microarrays have been used extensively to quantify the abundance of mRNA corresponding to different genes, and more recently high-throughput sequencing of cDNA (RNA-seq) has emerged as a powerful competitor. As the cost of sequencing decreases, it is conceivable that the use of RNA-seq for differential expression analysis will increase rapidly. To exploit the possibilities and address the challenges posed by this relatively new type of data, a number of software packages have been developed especially for differential expression analysis of RNA-seq data.We conducted an extensive comparison of eleven methods for differential expression analysis of RNA-seq data. All methods are freely available within the R framework and take as input a matrix of counts, i.e. the number of reads mapping to each genomic feature of interest in each of a number of samples. We evaluate the methods based on both simulated data and real RNA-seq data.Very small sample sizes, which are still common in RNA-seq experiments, impose problems for all evaluated methods and any results obtained under such conditions should be interpreted with caution. For larger sample sizes, the methods combining a variance-stabilizing transformation with the ‘limma’ method for differential expression analysis perform well under many different conditions, as does the nonparametric SAMseq method. Transcriptome analysis is an important tool for characterization and understanding of the molecular basis of phenotypic variation in biology, including diseases. During the past decades microarrays have been the most important and widely used approach for such analyses, but recently high-throughput sequencing of cDNA (RNA-seq) has emerged as a powerful alternative  and it hArguably the most common use of transcriptome profiling is in the search for differentially expressed (DE) genes, that is, genes that show differences in expression level between conditions or in other ways are associated with given predictors or responses. RNA-seq offers several advantages over microarrays for differential expression analysis, such as an increased dynamic range and a lower background level, and the ability to detect and quantify the expression of previously unknown transcripts and isoforms -5. The abetween samples in an RNA-seq experiment. First, the sequencing depths or library sizes  are typically different for different samples, which means that the observed counts are not directly comparable between samples. Indeed, even in the absence of any true differential expression, if one sample is sequenced to twice the depth of another we expect all the genes to obtain twice as high count in the first sample compared to the second, and we do not want to confuse such effects with true differential expression. The most straightforward way of approaching the different library sizes is to simply rescale or resample the read counts to obtain equal library sizes for all samples. However, such a normalization is generally not enough. The reason is that even if the library sizes are indeed identical, RNA-seq counts inherently represent relative abundances of the genes. A few highly expressed genes may contribute a very large part of the sequenced reads in an experiment, leaving only few reads to be distributed among the remaining genes , and c(s) ∈ {S1, S2} denoted the condition for sample s. We let the dispersion parameter φgs be the same in the two sample groups, that is, φgs = φg for all s.where φg from those estimated from the real RNA-seq data. We then defined For each gene, we drew a pair of values wg denoted the lower bound on the differential expression between the two groups. In our simulations, we let wg = 1.5 for all g.The parameter P’). Furthermore, we studied the effect of including outliers with extremely high counts. The outliers were introduced in two different ways. For the ‘single’ outlier simulation studies (denoted ‘S’), we selected 10% of the genes, and for each of these genes we selected a single sample for which we multiplied the observed count with a randomly selected factor between 5 and 10. For the ‘random’ outlier simulation studies (denoted ‘R’), we considered each observed count independently, and with probability 0.05 we multiplied a count by a randomly selected factor between 5 and 10. Table To simulate different real situations, we also evaluated the effect of generating the counts for half of the genes using a Poisson distribution (i.e., without overdispersion, simulation studies denoted ‘http://bowtie-bio.sourceforge.net/recount/. The data set contained RNA-seq data taken from 21 samples from two different mouse strains. Also for this data set we filtered out all genes for which the total count across the 21 samples did not exceed 10, which left 11,870 genes in the data set. In the supplementary material, we analyse two other real data sets [In addition to the synthetic data, we also considered a real RNA-seq data set  that we ata sets ,34, downThe authors declare that they have no competing interests.CS and MD contributed to the design of the study, the interpretation of the results and the writing of the manuscript. CS performed the implementation and the numerical experiments. Both authors read and approved the final manuscript.Contains supplementary figures referred to in the text. Here, we also evaluate the effect of selecting different values for the parameters of edgeR and DESeq and evaluate two additional transformation-based methods, and we evaluate the effect of simulating data with different dispersion parameter in the two compared conditions. We also present some comparisons based on data sets with 3 samples per condition. The file also contains information regarding the estimation of the mean and dispersion parameters from real data, and an additional analysis of two real RNA-seq data sets. Finally, it contains sample R code to run the differential expression analysis and estimates of the computational time requirements for the different methods.Click here for file


The nCounter analysis system  is a technology that enables the digital quantification of multiplexed target RNA molecules using color-coded molecular barcodes and single-molecule imaging. This system gives discrete counts of RNA transcripts and is capable of providing a high level of precision and sensitivity at less than one transcript copy per cell.We have designed a web application compatible with any modern web browser that accepts the raw count data produced by the NanoString nCounter analysis system, normalizes it according to guidelines provided by NanoString Technologies, performs differential expression analysis on the normalized data, and provides a heatmap of the results from the differential expression analysis.NanoStriDE allows biologists to take raw data produced by a NanoString nCounter analysis system and easily interpret differential expression analysis of this data represented through a heatmap. NanoStriDE is freely accessible to use on the NanoStriDE website and is available to use under the GPL v2 license. In recent years, RNA expression studies have relied on two major technologies: microarrays and high-throughput sequencing. Although both of these methods have proven their utility in biological assays ,2, each The nCounter allows digital quantification of multiplexed target molecules through the use of color-coded molecular barcodes and a single-molecule imaging system . By provDifferential expression analysis is performed differently for microarrays and RNA-Seq because of distinctions in the underlying technologies. Microarrays measure expression levels using optical detection of fluorescent intensities. Two-sided t-tests are normally used with microarray data as log normalized fluorescent intensity levels are well modelled by the Gaussian distribution. RNA-Seq uses sequence coverage as a measurement of expression, producing discrete rather than continuous data. The use of a discrete distribution like the Poisson, or more appropriately the negative binomial ,10, is aAlthough the technology overcomes major limitations of microarrays and high throughput sequencing, the data produced by the NanoString nCounter is not in a directly usable format and must be compiled, normalized, and analyzed using a variety of statistical methods. Guidelines to achieve this are provided by the company; however, the process is complex and generally requires the use of Microsoft Excel running on a Windows system. Moreover, performing statistical analysis of the data to explore relevant trends and produce differential expression maps requires familiarity with R and associated packages like Bioconductor and DESeq. The use of this software and an understanding of which analyses must be performed and in what order is not part of most biologists' skill sets and so acts as a needless impediment to their effective use of nCounter data.Our NanoString web application, NanoString Differential Expression (NanoStriDE), provides a configurable, extremely simple-to-use interface as a front end to a fully automated analysis pipeline of R libraries and Perl scripts. nCounter data is uploaded directly to the website. Processed data with a differential expression heatmap is made available for download within minutes.NanoStriDe intelligently applies positive and negative corrections, as well as a sample content normalization as per manufacturer guidelines to the raw data. The first stage applied to the data is a correction to positive controls. Each NanoString experiment contains synthetic spike-in controls in the early stage preparatory mix that allow for the correction of sample-to-sample variation due to assay-specific factors such as differences in amount of input material or reagents. The positive correction is calculated byc is count data for a gene in a given sample, m is the mean of the sum of the positive controls across all samples, and s is the sum of all of the positive controls for that given sample. The positive correction is applied to the data only if the t-test or one-way ANOVA was chosen. DESeq and one-way ANOVA  using DESeq ANODEV uses DESeq's built in normalization methods.where After the positive correction is applied, one of four negative correction methods is employed. The negative correction subtracts background noise from the positively-corrected data using counts of sequence tags known to be absent from the assay. The first three choices of negative correction use: (a) the mean of the negative controls for a given sample, (b) the mean of the negative controls summed with 2 standard deviations of the negative controls, or (c) the maximum of the negative controls. For these choices, the negative control is applied byc is the count data and n is one of the values for the aforementioned negative controls. The fourth choice of negative control that can be applied is a one-tailed Student's t-test using the negative controls as one group against all samples of a given gene as the other group, and using this to determine the significance of a given gene with a p-value cutoff. This fourth type of negative correction is applied bywhere c is all count data for a given gene, m is the mean of the count data for that gene across all samples, and p is the p-value cutoff used. If the p-value is at or above the provided cutoff (default 0.05), the counts for the gene are not significant and are set to 0 for all samples. If the p-value is below the provided cutoff, the mean for the count data for the gene are subtracted from the counts for the gene and count values that fall below zero are set to zero. As with the positive correction, the negative correction is applied to the data only if the t-test or one-way ANOVA was chosen. DESeq and one-way ANOVA  using DESeq ANODEV uses DESeq's built in normalization methods.where The normalization employed after the positive and negative correction steps depend on the type of statistical analysis selected. If DESeq or one-way ANOVA  was chosen for the differential expression, then the normalization process used within DESeq is applied to the data and the sample content normalization is not applied to the data as application of the sample content normalization and the DESeq normalization would be redundant. Refer to DESeq for the details of the normalization process applied . If the Option 1: Normalize to the mRNA housekeeping genes. This normalization is applied by using Formula 1, where c is the count data, m is the mean of the sum of the housekeeping genes across all samples, and s is the sum of the housekeeping genes for a given sample.Option 2: Normalize to the entire miRNA sample. The normalization is employed by using Formula 1, where c is the count data, m is the mean of the sums of the counts for each sample, and s is the sum of all of the counts for each sample.Option 3: Normalize to the highest miRNA in an assayed sample. Unlike Option 2, which uses all of the genes, this option normalizes the data to only the highest miRNA counts (default top 75) for a given sample (default sample 1) for the normalization factor.After all of the relevant correction and normalization steps have been applied, statistical tests are applied to determine differentially expressed genes. If the data is assumed to be distributed normally, a t-test (for two conditions) or one-way ANOVA (for three or more conditions) can be performed on the sample content-normalized data to determine statistical significance. If a negative binomial distribution is assumed, DESeq  or a one-way ANODEV  can be used on the DESeq normalized to determine statistical significance. When estimating the dispersion factors in DESeq used for normalization, different parameters are implemented. If only single replicates are used for each condition, the blind method is used to estimate dispersions with a fit-only sharing mode. If any condition has only two biological replicates, the pooled method is used with a fit-only sharing mode. If neither of the previous two conditions is met, a pooled method with a maximum sharing mode is used to estimate the dispersions.Only a subset of genes whose change in expression level are statistically significant are selected for visualization in a heatmap. A user-defined p-value threshold applied to the results from differential expression analysis removes any genes with p-values at or above the cutoff value. This cutoff is applied to either the p-value or to an adjusted p-value as specified by the user. The Bonferroni, Holm, Hochberg, Hommel, Benjamini & Hochberg, and Benjamini & Yekutieli p-value corrections are available as valid adjusted p-value options. A mean cutoff is applied using the subset from the p-value cutoff, removing any genes with a final normalized count mean at or below the cutoff value. If there are two or more remaining genes, a heatmap representative of the differential expression analysis is generated on a log scale across the rows. Refer to Figure NanoStriDE is configured to be easily accessible to biologists who employ nCounter to perform sophisticated experiments, so that they may complete their analyses without specialized training in biostatistics. A user's guide outlining how to use the website as well as details on all options and warnings is available on the NanoStriDE website. The analyzed data from a completed job can be downloaded in a number of stages and formats from NanoStriDE. Corrected raw, normalized data, results of differential expression analysis for all probes, user settings, and customized readme file are available for download in text format.The NanoStriDE web application was developed to assist biologists with performing and interpreting differential expression analaysis from NanoString nCounter system data. The high resolution heatmaps produced by NanoStriDE are suitable for use as figures in publications. If the user wishes to independently perform differential analysis on the data, the normalized data is additionally provided in the output of the completed job. NanoStriDE was designed for ease of use allowing biologists with limited computational experience to perform sophisticated differential expression analysis.Project Name: NanoStriDEhttp://nanostride.soe.ucsc.eduProject Homepage: Operating System: Platform independentProgramming Language: Perl, PHP, ROther Requirements: Modern HTML 5 compliant web browser License: NanoStriDE is freely available under the GPL v2 license and can be found on the NanoStriDE web page on the license page.CDB designed and implemented the web interface and the corresponding backend scripts required to run the pipeline. HJK and MG participated in its design and helped to draft the manuscript. NP conceived of the study, participated in its coordination and helped to draft the manuscript. All authors read and approved the final manuscript.