Classification ranking is a challenging problem, particularly for microarray data, where there is a huge number of possible regulated genes with no known rating function. This study investigates the possibility of making tumor classification more informative by using a method for classification ranking that requires no additional ranking analysis and maintains relatively good classification accuracy.Gene expression profiles based on microarray data are recognized as potential diagnostic indices of cancer. Molecular tumor classifications resulted from these data and learning algorithms have advanced our understanding of genetic changes associated with cancer etiology and development. However, classifications are not always perfect and in such cases the Microarray data of 11 different types and subtypes of cancer were analyzed using MDR , a recently developed boosting-based ranking algorithm. The number of predictor genes in all of the resulting classification models was at most nine, a huge reduction from the more than 12 thousands genes in the majority of the expression samples. Compared to several other learning algorithms, MDR gives the greatest AUC (area under the ROC curve) for the classifications of prostate cancer, acute lymphoblastic leukemia (ALL) and four ALL subtypes: BCR-ABL, E2A-PBX1, MALL and TALL. SVM (Support Vector Machine) gives the highest AUC for the classifications of lung, lymphoma, and breast cancers, and two ALL subtypes: Hyperdiploid > 50 and TEL-AML1. MDR gives highly competitive results, producing the highest average AUC, 91.01%, and an average overall accuracy of 90.01% for cancer expression analysis.Using the classification rankings from MDR is a simple technique for obtaining effective and informative tumor classifications from cancer gene expression data. Further interpretation of the results obtained from MDR is required. MDR can also be used directly as a simple feature selection mechanism to identify genes relevant to tumor classification. MDR may be applicable to many other classification problems for microarray data. Numerous studies have shown that cancer involve accumulated genetic aberrations in the cell. Advances in DNA microarray technology have revolutionized cancer research by enabling, within a given cell population, the simultaneous monitoring of the transcription and complex changes in the expression of thousands of genes during cancer development. This makes rapid genetic analysis for genome-wide cancer studies feasible. Researchers can quickly compare gene expressions between normal and malignant cells, and explore the genetic changes associated with cancer etiology and development. Microarray analysis offers promising avenues to the discovery of both new biomarkers for cancer diagnosis and prognosis and new treatments. Microarray data are being used to categorize tumors on the basis of their molecular profiles, to identify subtypes of tumors, to predict patients' responses to treatment and risk of relapse, and to explore the biological properties of tumors -7.classification ranking) can be useful for further research . For example, some of the genetic abnormalities in malignant cells may be the most important contributing factors for cancer. Classification ranking is a challenging problem, particularly in microarray data, which has a huge number of factors whose relative importance is largely unknown. Most machine learners focus on classification and do not explicitly assess the likelihood of correctness for their class predictions, unless additional analysis is performed.Recent cancer research has applied a variety of machine learning algorithms for tumor prediction by associating expression patterns with clinical outcomes for patients with tumors in various stages ,4,8,9. DThis paper describes a simple microarray data analysis technique for tumor classification ranking. In particular, we apply MDR, our recently developed Multi-Dimensional Ranking algorithm, for analyzing gene expression in various types of cancers including leukemia, lung, prostate, lymphoma, and breast cancers. These data have been used in previous cancer research studies ,3,5-8,10types of human acute leukemia: acute myeloid leukemia (AML) and ALL. The ALL-subtype expression data were used by Yeoh et al. , a list of samples in a partition of the training set. Let di = 1 if a sample si is a member of the target class and di = 0 otherwise. We define rating of L as follows:We now define the rating function for d(n - d)/n where d is the total number of samples that belong to the target class. Note that smaller rating values are better. For example, consider lists L1 = , L2 = , and L3 = . It is clear that L3 is the most desirable since target class members are all towards the top of the list and L1 is the least desirable. This is consistent with the heuristic values: R(L1) = 1, R(L2) = 3/4, and R(L3) = 0.The rating is biased to ranking patterns that have target class members at the top of the list and non-target class members at the bottom. Ratings can range from 0 to A, s, p), where A, s and p represent an attribute, a sorting order type  and a ratio of the partition size to the number of training instances, respectively. Figure Ai represents attribute i of the data set.The output model is a list of triplets of the form (A14 in decreasing order. Next (in round 2), we partition the sample list into two parts with the top part containing 20% of the total number of testing samples. We sort the top partition by attribute A12 in decreasing order and sort the bottom partition by attribute A10 in increasing order.To apply the model to a given set of testing data instances, the list is repeatedly sorted according to the attributes selected by the model. For example, if we use the model in Figure A7 and A11, respectively. The resulting list gives the ranking predictions.Finally, in round 3, we partition the list into three parts: the top and middle parts contain 12% and 18% of the total number of testing instances, respectively. The rest are in the last partition. In the top partition, NULL signifies that no attribute is selected and therefore there is no sorting required. Thus, this top partition remains unchanged. We then sort the middle and bottom partitions in decreasing order by attributes Unlike most machine learners, MDR is specifically designed to produce ranking models. The proposed MDR algorithm is similar to the MartiRank algorithm employed in the ROAM system  in that accuracy (ACC), which is defined to be the ratio of the number of correct predictions to the total number of instances in the test sample. However, ACC does not make use of the nature of the incorrect predictions, which can be useful in many domains including tumor classification.To evaluate the quality of our classification models, we employ various standard performance measures; ideally, different measures provide different insights. In machine learning, modeling involves classification or prediction that associates patterns from data points with classes that express different concepts. One of the most popular performance measures in machine learning classification is positive and negative, as shown in Figure TP (True Positive) represents the number of positive samples correctly classified, whereas FP  represents the number of negative samples incorrectly classified as positive. TN (True Negative) and FN  are defined symmetrically. ACC is computed as (TP+TN)/(TP+FP+TN+FN) and does not take into account the difference between false alarms (type I errors), measured by FP, and missed detections (type II errors), measured by FN. The latter value is particularly important in clinical medicine and defect management. Furthermore, ACC can be especially misleading when the distribution of the sample class is unbalanced. For example, a model can achieve 90% accuracy on a sample in which only 10% of the data points are positive simply by classifying every instance as negative, even though all of its classifications for positive samples are incorrect.Consider, for example, a model that classifies data points into the binary classes: TPR (True Positive Rate) is defined to be the ratio of positives correctly classified to the actual number of positives, and FPR  is defined to be the ratio of negatives incorrectly classified to the actual number of negatives. TPR is also called the hit rate, recall, or sensitivity. FPR is also referred to as the false alarm rate. Similarly, specificity is defined as the ratio of correctly classified negatives to the actual number of negatives. TPR and specificity are independent, i.e., neither one tells us anything about the value of the other. Note that the sum of specificity and FPR is one, and thus, knowing one allows us to calculate the other. Another useful metric is precision (PREC), the ratio of correctly classified positives over the total number of samples classified as positive.As shown in Figure AUC (Area Under the ROC Curve) as an additional performance metric. The ROC curve is a two-dimensional plot between the TPR (Y-axis) against the FPR (X-axis) of the predictions. The values of TPR and FPR range from zero to one. The set of points  defines the ROC space. Given a ranking list of sample predictions, each sample point in the list can be used as a cutoff point. For each cutoff point, the corresponding TPR and FPR can be computed and plotted as a single point in ROC space. Connecting these points gives the ROC curve for the list of predictions. ROC curves can be used to compare predictions. The closer the curve is to the Y-axis (high true positives) and the further away it is from the X-axis , the more accurate the predictions are. When predictions are made by random guessing, TPR and FPR grow linearly yielding an ROC curve, which is a straight line 45 degrees from the horizontal. This is referred to as the no-discrimination line [In this paper we also use ion line .ROC1 and ROC2 are plotted using multiple cutoff points as the model incrementally classifies each testing instance, whereas ROC3 uses a single point to create the ROC curve with a cutoff point where all testing instances have been classified. ROC1 and ROC2 are obtained from the same testing instances but in different orders. Even though all the three classification results have the same accuracy, they have different AUCs. AUC is more sensitive to the order of classification ranking than it is to accuracy.Unfortunately, using ROC curves for performance comparisons does not necessary give conclusive results since one ROC curve may dominate another in one region but be dominated in a different region. According to , the AUCWe use microarray gene expression data for leukemia, lung, prostate, lymphoma, and breast cancers. These are real-world expression profiling data sets that are used in various types of cancer research ,3,5-8,12Table n-fold cross-validation [n-fold cross validation, a data set is randomly partitioned into n approximately equally sized subsets (or folds or tests). The learning algorithm is executed n times; each time it is trained on the data that is outside one of the subsets and the generated model (classifier) is tested on that subset. The estimated accuracy for each cross-validation test is a random variable that depends on the random partitioning of the data. The estimated accuracy is computed as the average accuracy over the n test sets. Typically n-fold cross-validations are repeated several times to assure data randomness, and the estimated accuracy is the average over these repetitions. In this paper, we use the number of folds, n = 10, as is standard practice. For efficiency control, MDR allows a user to specify the number of boosting rounds (T). In this study, we use T = 5 as the number of rounds. A method for selecting the best T is a subject for future research. We ran 10-fold cross validations on all eleven test cases of the six data sets using our MDR ranking algorithm and five other machine learning approaches, which are available on Weka [ZeroR, C4.5, Bayes, k-NN and SVM [k-NN is a nearest-neighbor classifier, for which we specify the number of neighbors to be k = 3 . The suThe authors declare that they have no competing interests.RH designed the study, performed analysis of the micro array data and wrote the draft manuscript. PK implemented the proposed algorithm, performed the experiments and assisted in the analysis of the data. Both authors participated in production of the final version of the manuscript, read it and approved it.


To better understand the molecular programs of normal and abnormal genital development, clear-cut definition of androgen-dependent gene expression patterns, without the influence of genotype , is warranted. Previously, we have identified global gene expression profiles in genital-derived fibroblasts that differ between 46, XY males and 46, XY females with complete androgen insensitivity syndrome (CAIS) due to inactivating mutations of the androgen receptor (AR). While these differences could be due to cell autonomous changes in gene expression induced by androgen programming, recent work suggests they could also be influenced by the location from which the fibroblasts were harvested (topology). To minimize the influence of topology, we compared gene expression patterns of fibroblasts derived from identical urogenital anlagen: the scrotum in normally virilized 46, XY males and the labia majora from completely feminized 46, XY individuals with CAIS.612 transcripts representing 440 unique genes differed significantly in expression levels between scrotum and CAIS labia majora, suggesting the effects of androgen programming. While some genes coincided with those we had identified previously , a significant number did not, implying that topology had influenced gene expression in our previous experiments. Supervised clustering of gene expression data derived from a large set of fibroblast cultures from individuals with partial AIS revealed that the new, topology controlled data set better classified the specimens.Inactivating mutations of the AR, in themselves, appear to induce lasting changes in gene expression in cultured fibroblasts, independent of topology and genotype. Genes identified are likely to be relevant candidates to decipher androgen-dependent normal and abnormal genital development. Androgen receptor (AR) signaling is the key determinant of virilization in male external genitalia development -4. Its iIn a previous study, we reported evidence for androgen-mediated programming of gene expression by comparing genital skin fibroblasts from normal males to those derived from CAIS individuals . HoweverWe had previously identified an AR-dependent gene expression signature by comparing normal male foreskin fibroblasts to those cultured from diverse sites in the genitals of patients with documented CAIS. Subsequent reports that cultured fibroblasts retain topographic transcriptional memory (gene expression signatures that reflect their site of biopsy) led to concerns that the AR-dependent gene expression signature we identified could have been affected by topological differences in the fibroblast samples used. To test for this potential confounder, we repeated microarray experiments on seven independent strains of normal male scrotal fibroblasts  and duplicate samples of four labia majora fibroblasts derived from 46, XY individuals with CAIS due to proven inactivating mutations of the AR-gene. Both fibroblasts were derived from identical anlagen: the labioscrotal swellings. The SAM procedure revealed 612 transcripts representing 440 unique genes that differed significantly in expression level between the groups at a false discovery rate of 0.038 , CBX6 (Chromobox homologue 6), IGFBP5 (Insulin-like growth factor binding protein 5), and EGFR  while several others were no longer identified as significantly different between the data sets, such as TBX2 (T-box 2), TBX5 (T-box 5), BMP4 (bone morphogenetic protein 4), HOXA13 (Homeobox A13), WNT2 , and FOXF2 (Forkheadbox F2). The significant change in the gene lists strongly suggested that topology influenced the gene expression signatures identified in our original series of experiments.To evaluate the relevance of the topology-controlled, AR dependent gene list, we tested its ability to classify 72 microarray experiments performed on fibroblast samples derived from 51 individuals that included normal males, normal females, and individuals with PAIS and CAIS Fig. . StringeClustering separated the 72 experiments into two major subgroups. The righthand major branch included predominantly patients with female external genitalia while most of the patients in the lefthand major branch had normal male or highly virilized external genitalia Fig.  and 2B. Structure within the cluster dendrogram suggested that there was some residual influence of topology on gene expression in the samples. In the righthand "female" branch, fibroblast samples derived from AIS gonads clustered separately from all skin-derived samples Fig. . SimilarWe also wanted to reconsider whether the new topology-independent, AR gene list better classified samples than our previous gene list that did not control for the locations from which samples were harvested. After removing the microarray experiments that were used to define either the previous  or the nWe performed a systematic analysis for enrichment of genes belonging to defined biological processes and cellular pathways using the PANTHER classification system . PANTHERThe AR-dependent gene set showed enrichment for a number of genes related to maintenance and modification of tissue shape and structural identity Tables . Genes tFour transcripts were selected for confirmation of the microarray data by semi-quantitative RT-PCR: TNC (tenascin), FZD8 (frizzled 8), ADAM12 , and CSPG2 . RNA from the labia majora fibroblasts derived from CAIS affected individuals , as well as four samples of the normal male scrotal fibroblasts  were used for analysis. In agreement with the microarray data, TNC, FZD8, ADAM12 and CSPG2 show significantly higher expression in the labia majora fibroblasts compared to the scrotal fibroblasts , as described previously [A total of 51 different primary fibroblast cultures were analyzed  procedure [2 red/green ratios for single genes were centered by the mean in each of the three datasets separately according to a previously published procedure [2 red/green ratios ≥ 1.0 in at least 5 experiments of the sample set of 72 microarrays resulted in 259 transcripts. Microarray experiments and genes were organized by hierarchical clustering, using the Pearson correlation metric and average linkage clustering [To identify genes differentially expressed in homologous tissues of normal male and CAIS female derived fibroblasts, we restricted our analysis to samples derived from the labioscrotal swellings (scrotum or labia majora), that were hybridized on poly-lysine coated slides against Strategene reference RNA data set 2). Only spots with fluorescence signals of 1.5-fold over array background in either the experimental or reference channel were considered. The Cy5/Cy3 fluorescence ratios for all genes in each array were normalized to obtain an average absolute logrocedure  was usedrocedure . Centereustering . TreeVieustering . Supplemustering ,23. For . Only spSAM identified transcripts were loaded into the PANTHER  database. Enrichment of biological processes and cellular pathways in the AR dependent gene list was determined by comparing the number of genes observed in the SAM-list for each process or pathway to the number of genes expected based on the NCBI Homo sapiens reference gene list. PANTHER determines a p-value based on binominal statistics, describing the probability that the number of detected genes occurred randomly, with a p-value < 0.05 indicating significance . Since mSemi-quantitative RT-PCR was performed using FAM (F) labeled probes with dabcyl (DB) quencher. RT-PCR was performed on four samples of normal male scrotal fibroblasts  as well as on four samples of labia majora derived from CAIS patients . Exon-spanning primers and probes  had the following sequences: Tenascin: TNC-forward: ACAgTgggACAgCAggTgACT, Tm: 59.6°C; TNC-reverse: CAggTTgACACggTgACAgTTC, Tm: 59.4°C, TNC labeled probe: F-TACCACAATggCAgATCCTTCTCCACCT-DB, Tm: 66.7°C; Frizzled 8: FZD8-forward: gTgCAgCgAAgggACACTTg, Tm: 61.0°C; FZD8-reverse: TCCTCAgCCAACAgAAATTAACg, Tm: 58.7°C, FZD8 labeled probe: F-AggTTCCCACCCCTTCACAgTgTTgA-DB, Tm: 67.9°C; ADAM metallopeptidase 12: ADAM12-forward: CCTggCACCCCTCAgACC, Tm: 60.9°C, ADAM12-reverse: gTACAAAAAACTCCAACTggAgCTg, Tm: 59.4°C, ADAM12 labeled probe: F-TCCACACCAAgTgCCCAgATCCAC-DB, Tm: 67.0°C; Chondroitin sulfate proteoglycane 2, versican: CSPG2-forward: TgTCTCACgAAgAACAAATgTTTgT, Tm: 58.0°C, CSPG2-reverse: gggTCTCCAATTCTCgTATTgCA, Tm: 59.8°C, CSPG2 labeled probe: F-TgCCATCAgTCCAACggAAgTCATgCT-DB, Tm: 69.3°C; TATA-binding protein (TBP) was used to normalize expression values. TBP primers were TBP-forward: CACgAACCACggCACTg-ATT, TBP-reverse: TTTTCTTgCTgCCAgTCTggAC; TBP-labeled probe: F-TgTgCAC-AggAgCCAAgAgTgAAgA-DB. Reverse transcription was performed using the Ambion RETROscript kit according to the manufacturer's recommendations with minor modifications. 2 μg of total RNA, 0.5 μM dNTPs, 5 μM random decamers and 200 units of MMLV-RT polymerase were used per reaction. Heat denaturation was performed for 3 min at 85°C preceding reverse transcription for 1 hr at 44°C and subsequent inactivation for 10 min at 92°C. C-DNA corresponding to 12 ng of initial total RNA served as template for amplification reactions. 0.4 μM of each primer, 0.17 μM of specific probe and TaqMan Universal PCR mastermix  including the Taq polymerase were added to a final volume of 25 μl. Initial denaturation was performed at 95°C for 10 min followed by 1 min cycling intervals at 60°C using a RotorGene RG-3000 cycler . Differential transcription levels between all different cell lines were calculated according to the ΔΔ-CT method [AR, androgen receptor; AIS, androgen insensitivity syndrome; CAIS, complete androgen insensitivity syndrome; PAIS, partial androgen insensitivity syndromeAll authors read and approved the final manuscript. PMH was responsible for the experimental design, assessment of clinical information on AIS-patients , culturing fibroblasts, RNA-works and microarray hybridizations (data set 1), functional characterization of AR in fibroblast samples, data analysis, and he was the leading writer of the manuscript. UD planned and performed culturing fibroblasts, RNA-works, microarray hybridizations, microarray data analysis and management (data set 2 and 3), and contributed essentially to writing the manuscript. RW helped essentially with the microarray strategies and data analysis and contributed to writing the manuscript. He was also responsible for RT-PCR. ARU was responsible for clinical assessment of patients and contributed to writing the manuscript. In particular, she contributed the 46, XX fibroblasts. JHB performed microarray data analysis and wrote the manuscript. LW was responsible for clinical assessment of several patients, and he obtained many fibroblast biopsies, especially the normal scrotal controls, and contributed to writing the manuscript. SK assessed many patients clinically and contributed to writing the manuscript. HUS was reponsible for clinical assessment of several patients (data set 3), contributed functional characterization of AR in fibroblast samples  and contributed to writing the manuscript. JD was indispensable for microarray data analysis, central data management at Stanford University and the web repository of data. He also helped in creating the figures for the web supplement. FR assessed many patients clinically, took part in data analysis and was one of the writers of the manuscript. OH was an important senior clinical supervisor of genital phenotype assessment, he contributed most of the molecular data of the androgen receptor gene to the study and he was a key person in data interpretation with respect to androgen insensitivity syndrome. JDB was the senior writer of the manuscript. He supervised the experimental strategy, including all statistical issues with respect to the microarray data.Text file to visualize data for figure Click here for fileCluster file to visualize data for figure Click here for fileArray tree file to visualize data for figure Click here for fileGene tree file to visualize data for figure Click here for fileText file to visualize data for figure Click here for fileCluster file to visualize data for figure Click here for fileArray tree file to visualize data for figure Click here for fileGene tree file to visualize data for figure Click here for file


With the rapid development of new genetic measurement methods, several types of genetic alterations can be quantified in a high-throughput manner. While the initial focus has been on investigating each data set separately, there is an increasing interest in studying the correlation structure between two or more data sets. Multivariate methods based on Canonical Correlation Analysis (CCA) have been proposed for integrating paired genetic data sets. The high dimensionality of microarray data imposes computational difficulties, which have been addressed for instance by studying the covariance structure of the data, or by reducing the number of variables prior to applying the CCA. In this work, we propose a new method for analyzing high-dimensional paired genetic data sets, which mainly emphasizes the correlation structure and still permits efficient application to very large data sets. The method is implemented by translating a regularized CCA to its dual form, where the computational complexity depends mainly on the number of samples instead of the number of variables. The optimal regularization parameters are chosen by cross-validation. We apply the regularized dual CCA, as well as a classical CCA preceded by a dimension-reducing Principal Components Analysis (PCA), to a paired data set of gene expression changes and copy number alterations in leukemia.Using the correlation-maximizing methods, regularized dual CCA and PCA+CCA, we show that without pre-selection of known disease-relevant genes, and without using information about clinical class membership, an exploratory analysis singles out two patient groups, corresponding to well-known leukemia subtypes. Furthermore, the variables showing the highest relevance to the extracted features agree with previous biological knowledge concerning copy number alterations and gene expression changes in these subtypes. Finally, the correlation-maximizing methods are shown to yield results which are more biologically interpretable than those resulting from a covariance-maximizing method, and provide different insight compared to when each variable set is studied separately using PCA.We conclude that regularized dual CCA as well as PCA+CCA are useful methods for exploratory analysis of paired genetic data sets, and can be efficiently implemented also when the number of variables is very large. The abnormal behavior of cancer cells is in many cases caused by somatically acquired genetic alterations. Several types of genetic changes, such as fusion genes, mutations, copy number changes and abnormal methylation patterns, have been observed in malignant cells -4. In moThe aim of this paper is to describe two methods for multivariate integrative analysis of paired data, based on Canonical Correlation Analysis (CCA ). Both mCCA is a generalization of multivariate linear regression to the situation where there are more than one response variable. In its classical formulation CCA extracts a pair of features, each being a linear combination of the variables from one variable set, such that the correlation between the features is maximized. The classical formulation of CCA requires invertibility of the sample covariance matrices, making it impossible to apply e.g. to data sets where the number of variables exceeds the number of samples. Moreover it can be severely confounded by collinearities among the variables. To overcome this limitation Vinod  proposedet al. [Several authors have addressed the integration of paired genetic data sets by posing specific questions, e.g. whether there are genes that are differentially expressed in samples possessing a certain copy number alteration, compared to samples without the alteration . Therebyet al.  applied et al.  to a paiet al. -23. CCA-et al. [et al. [et al. [The regularized dual CCA described in this paper includes a ridge penalty on the covariance matrices. In this aspect it is similar to the method proposed by González et al. ,24. When [et al. ,24, beco [et al. . We keep [et al. -22, wher [et al.  is based [et al.  or elast [et al. , are not [et al. . InsteadAnother approach towards constructing a multivariate integrative method keeping the emphasis on the correlation structure while being applicable to very large data sets is to use a classical CCA preceded by dimension reduction by PCA (discussed e.g. by Muller ). This iWe apply regularized dual CCA and PCA+CCA to a paired data set of gene expression and SNP copy number measurements in 173 leukemia patients, representing ten different leukemia subtypes. The results are analyzed first by the relevance of the gene expression and copy number variables to the extracted features, and second by the representation of the samples in the space of the extracted features. Since the extracted features in this paper are used mainly for visualization, we consider only the first two pairs of features from each method. If the features are to be used for a more extensive interpretation, a careful choice on the dimension must be made, which in itself is a non-trivial matter.We begin by splitting the data set into a tuning set and a validation set. The tuning set is used to estimate optimal regularization parameters and extract features, and the validation set is used only for visualization of the results and assessment of the generalization ability of the extracted features. The tuning set consists of two thirds of the samples, to provide a large enough basis for extraction of generalizable features, and the validation set consists of the remaining one third of the samples. The proportion of samples with a specific leukemia subtype is chosen to be similar in the two sets, otherwise the partition is random. The optimal regularization parameters for the regularized CCA are estimated using cross-validation on the tuning set. Using the optimal regularization parameters, we then extract the first two pairs of canonical features from the tuning set, and interpret their biological content using the cross-loadings of all variables with them. Finally, the extracted features are applied to the tuning set as well as the validation set, yielding two-dimensional representations of the samples. From these, we can extract groups of samples which are characterized by the extracted features, and assess the generalization ability of the features to the validation set. We compare the results from the optimally regularized dual CCA and PCA+CCA to those obtained by fully regularized dual CCA, un-regularized dual CCA, a sparse CCA method  and sepaet al. [et al. [τx (for the copy number data) and τy (for the gene expression data). The largest value, τx = 0.9, τy = 0.3. Apparently, the copy number data need more regularization than the gene expression data, which could be anticipated since many copy number variables are highly correlated.The optimal regularization parameters for the two variable sets in the regularized dual CCA are considered to be those maximizing the generalization ability of the first extracted feature pair, which we define as the canonical correlation obtained when the extracted feature pair is applied to a test set. We use a 3-fold cross-validation strategy to estimate this canonical correlation  nor a model with full regularization (τx = τy = 1) yield a high correlation when the extracted features are applied to the test data. The unregularized model extracts spurious features which are very specific to the training data, while the fully regularized model extracts features with high variance, but only moderate correlation. As can be seen in Figure Figure To estimate the generalization ability that could be expected for features extracted from paired data without any truly related components, we permute the samples in the gene expression matrix of the tuning data. We then run the 3-fold cross-validation to estimate the generalization ability of the features extracted with the regularization parameters fixed to the optimal values determined above. The mean value ± SD for τx = 0.9, τy = 0.3. The same 3-fold cross-validation strategy is applied to the tuning data, giving Since the regularization parameters are determined based only on the first pair of extracted features, we estimate the generalization ability also for the second pair of features, with the regularization parameters fixed at For the PCA+CCA, we extract twelve principal components, independently from each variable set, to use as variables in a classical CCA. This choice is motivated by an intention to keep the number of variables for the CCA low, while still extracting enough information from each of the variable sets. The first twelve principal components explain 52% of the variance in the copy number data set, and 58% of the variance in the gene expression data set, and the scree plots are almost flat (data not shown), indicating that the rest of the components mostly contain noise. Applying the 3-fold cross-validation strategy to test the generalization ability of the first two pairs of PCA+CCA features returns the estimates IL13RA1, ZCCHC24 and IGHD. The second pair of features is associated with a large copy number alteration on chromosome 1 and a small change on chromosome 19, oppositely directed. Among the genes with highest cross-loadings with the second feature we note PBX1, SLC27A2 and PSEN2.Given that the first two pairs of features from both regularized dual CCA and PCA+CCA are expected to encode true linear relationships between the copy number and gene expression variables, we interpret the biological content in these by computing the cross-loadings for each variable with the extracted features, based on the tuning samples. The cross-loadings for the copy number and gene expression variables, ordered along the genome, are shown in Figures So far, we have used only the tuning set to determine the optimal regularization parameters and extract two pairs of features from the variable sets. To determine whether there are specific subgroups of the patients that are singled out by the extracted features, and to assess the generalization ability to unseen data, we visualize the samples from the tuning set and the validation set by their coordinates in the extracted features.E2A/PBX1 fusion gene. The redundancy coefficients for the tuning and validation sets are shown in Tables Rx|y) is much larger than the fraction of the variance in the gene expression variables shared by the extracted copy number features (Ry|x). This may indicate that copy number changes are often related to changes in gene expression, captured by the extracted features, while changes in gene expression may have many other causes which are not encoded by the extracted copy number features. For the tuning set, the redundancy coefficients Ry|y and Rx|x are very similar to Ry|x and Rx|y, respectively. This is due to the high correlation between the extracted gene expression and copy number features, implying that each extracted gene expression feature encodes almost the same information as the corresponding copy number feature.Application of the features extracted by dual regularized CCA to the tuning data set results in highly correlated sample representations from the copy number and gene expression data. In fact, the correlation between the representations of the samples by the first feature pair is With PCA+CCA, the canonical correlations for the first two component pairs on the tuning data set are τx = τy = 0 and τx = τy = 1, respectively, on the representation of the validation and tuning samples by their coordinates with respect to the first two pairs of features extracted with regularized dual CCA. In the unregularized case (left panels), although the correlation between the extracted features is very high  is lower with PCA than with fully regularized CCA. The fraction of variance shared by the first feature only is higher with PCA than with any of the other methods, as expected.In this section, we compare the results obtained with the integrative correlation-maximizing methods, i.e. regularized dual CCA and PCA+CCA, to those obtained by applying only PCA to each variable set separately. As before, we extract the first two principal components from the tuning set and apply them to the validation set. Figure et al [With the rapid development of new genetic measurement methods, there is an increasing interest in combining several types of genetic markers measured in the same samples. Previously, several multivariate methods have been applied to this type of data. For microarray data these methods are mostly covariance-maximizing -23 whichet al ,24, but We applied the regularized dual CCA and PCA+CCA to a large paired data set consisting of gene expression and SNP copy number measurements from 173 patients with leukemia. With both regularized dual CCA and PCA+CCA we extracted two pairs of highly correlated features from the gene expression and copy number variable sets. We interpreted the biological content of the features using the cross-loadings of all variables with them. Importantly, we noted that even though the feature vectors extracted with the two methods, and hence the representation of the samples, were quite different the interpretation in terms of the cross-loadings of all variables with the extracted features showed a high degree of similarity.ZCCHC24, IL13RA1 and IGHD, distinguishes a group of patients from the rest. This group consists mainly of patients from the HD50 subgroup. Notably, the copy number effects dominating this feature agree well with those reported in a large study of patients with the HD50 subtype [ZCCHC24, APOOL, HUWE1 and SMAGP genes were found among the top 100 probe sets characterizing the HD50 subtype.Furthermore, we represented the samples, both from the tuning set used to extract the features and from a validation set, by their coordinates in the CCA features and extracted groups of patients with characteristic gene expression and copy number profiles. The first feature pair, characterized by copy number alterations on chromosomes 4, 6, 8, 10, 14, 17, 18, 21 and X and expression changes for genes such as  subtype . Further subtype , probe sPBX1, followed by e.g. SLC27A2 and PSEN2. This feature pair (together with the first pair) clearly separates one group of samples from the rest. Applying the subtype information, this group is seen to consist exclusively of samples from the E2A/PBX1 group. This subtype is indeed characterized by a translocation between chromosomes 1 and 19 which gives rise to the E2A/PBX1 fusion gene. While balanced translocations cannot be detected by SNP-arrays, the 1;19 translocation is in most cases  present as an unbalanced translocation, resulting in gain of 1q and loss of 19p material. Furthermore, many of the genes showing high cross-loadings with the second feature . In sucet al [in situ hybridization, into the following subtypes: T-ALL, hyperdiploidy , E2A/PBX1-positive, TEL/AML1-positive, BCR/ABL1-positive, MLL-rearranged, low hyperdiploidy , hypodiploidy, pseudodiploidy or other [The data set used in this study was generated and first described by Mullighan et al . It inclor other .The gene expression data set was generated using Affymetrix HG-U133A arrays, providing mRNA levels of more than 18,000 transcripts. The copy number data set was generated using Affymetrix Human Mapping 250 K Sty SNP-arrays which give copy number and genotype information for more than 230,000 SNPs.http://hospital.stjude.org/forms/genome-download/request/. The raw intensity signals from the gene expression arrays were normalized using gcRMA [The CEL-files containing raw intensity signals from the Affymetrix HG-U133A arrays and the CEL- and CHP-files containing raw intensity signals and genotype information from the Affymetrix Human Mapping 250 K Sty arrays were downloaded from ng gcRMA  which isng gcRMA  package The CEL- and CHP-files from the SNP-arrays were imported into dChip , where t2 ratios from the copy number data set and the log2-transformed gene expression values for the gene expression probe sets. Before entered into the canonical correlation analysis, each variable is mean-centered and standardized to unit variance across the samples used for feature extraction. Whenever a training set and a test set are used, both sets are standardized using the mean value and standard deviation from the training set. The final data set, to which we apply the integrative methods, consist of 20,021 gene expression probe sets and 238,304 copy number variables.In all analyses, we use the logX ∈ ℝn × N and Y ∈ ℝm × N, where each column in X and Y corresponds to one of the N samples, and each row represents one variable. We let X denote the copy number data matrix and Y the gene expression data matrix, and assume that each variable is centered to have zero mean across the samples. Furthermore, we let Cxx = XXT and Cyy = YYT denote the  sample covariance matrices for the two variable sets, and Cxy = XYT the  sample cross-covariance matrix.Canonical Correlation Analysis (CCA ) is a gewx ∈ ℝn and wy ∈ ℝm such that the empirical correlation between the respective projections onto these weight vectors, i.e. between XT wx and YT wy, is maximized. Hence, we are seeking to maximizeThe objective of CCA is to extract, from the two variable sets, latent features which are most highly correlated. The latent features returned by CCA are linear combinations of the measured variables. More formally, we are searching for weight vectors Cxx and Cyy are invertible, the weight vectors maximizing (1) can be found by solving the eigenvalue problemsSubsequent features can be extracted by maximizing the correlation for projections onto weight vectors uncorrelated to those extracted previously. This formulation of CCA also shows that it is completely symmetric in the two variable sets. If Cxx and Cyy are invertible, it is not uniquely solvable when the number of variables exceeds the number of samples, and it can be severely confounded by collinearities among the variables. Both of these characteristics are common in microarray data sets. In these cases we need to reduce the flexibility of the CCA, thereby increasing the robustness of the feature extraction. We address this issue in two different ways, first by regularizing the CCA explicitly and second by reducing the dimensionality of the data using PCA before applying classical CCA.Since the classical formulation of CCA requires that the sample covariance matrices wx and wy to maximize the penalized correlationWe regularize the classical CCA by adding a ridge penalty to the covariance matrices. This is a commonly used regularization method  tN is the number of samples in the data set. We introduce the N into the expression to render the two terms in each parentheses in the denominator of approximately equal magnitude, at least in the case where the variables within each set are uncorrelated and of unit variance (in which case both Cxx and Cyy are approximately N I). The regularization parameters τx and τy parametrize a whole family of methods, ranging from classical CCA (τx = τy = 0) which extracts maximally correlated features, to PLS (τx = τy = 1) where the extracted features are those showing maximal covariance. The higher value of τx and τy that are used, the more focus is put on the variance of the extracted features. Therefore, using regularized CCA implies a trade-off between explaining the variable sets individually and explaining their correlation structure. Introducing the regularization makes it possible to solve the optimization problem even when the number of variables exceeds the number of samples. Maximal regularization, on the other hand, is not necessarily desirable since in this case relevant features with high correlation between the variable sets may be drowned by features with high variance and only moderate correlation.where Cxx = XXT and Cyy = YYT we work with the matrices XT X and YT Y which are of dimension N × N and hence much smaller. By expressing the weight vectors wx and wy in terms of the sample matrices as wx = Xαx and wy = Yαy, substituting into (3) and noting that the resulting expression is invariant to scaling of αx and αy, we obtain the dual formulation of regularized CCA as the optimization problemIn cases where the number of variables is very large, it can be a computational advantage to solve the regularized CCA problem in its dual formulation . The duaKX = XTX and KY = YTY, this can be restated as the generalized eigenvalue problemUsing the notation αx and αy. The regularized CCA features are then calculated as wx = Xαx, wy = Yαy. In general, as discussed e.g. in [τx = τy = 0) is likely to overfit the data when the number of variables is very large, and should therefore be interpreted with great caution.which is solved to return  e.g. in , unregulτx and τy based on the tuning set, we employ a 3-fold cross-validation technique, with the aim of finding the pair of regularization parameters giving the highest generalization ability of the first extracted pair of features to unseen data. Leave-one-out cross-validation to find the optimal regularization parameter in CCA was proposed by Leurgans et al. [et al. [et al. [i is removed from the training set and with Xi and Yi denoting the parts of X and Y corresponding to group i, we let (Xi)T Yi)T i) onto the extracted features. The estimated test set canonical correlation is then defined asIn order for the extracted features to be of practical use, they should encode characteristics which are in some sense general to the population to which the considered data set belongs. To achieve this goal we determine the optimal regularization parameters by cross-validation. First, the data set is divided into a tuning set, consisting of two thirds of the samples, and a validation set, consisting of the remaining one third of the samples. The tuning set is used to determine regularization parameters and extract features, and the validation set is used only for visualization and assessment of the generalization ability of the extracted features. To choose the optimal regularization parameters s et al. , and the [et al. . More re [et al.  applied et al. [The same objective function was used by Parkhomenko et al.  to estimτx and τy on a grid , and thSince the determination of the regularization parameters is based only on the optimization of the largest test set canonical correlation, we also estimate the generalization ability of the second pair of extracted features, to determine if this can be expected to represent a true linear relationship between the two variable sets. Furthermore, to determine whether the estimated test set canonical correlation is larger than what could be expected by chance only we permute the samples in the gene expression variable set and thereby obtain a paired data set without any true correlations. We then estimate the largest test set canonical correlation for the features extracted from the permuted data using the optimal regularization parameters determined for the original data. To find regularization parameters which are in some sense optimal to all extracted feature pairs, it is possible to change the objective function to include not only the largest test set canonical correlations, but also the subsequent ones. For the data set used in this study, reformulating the objective function to the mean of the first and second test set canonical correlation only has a marginal effect and does not change the interpretations.x ∈ ℝn × s and ∏y ∈ ℝm × s, with s ≪ N, denote the matrices of principal components resulting from applying PCA to each data set separately. Projecting X and Y onto the principal components yields s-dimensional representations such that as much as possible of the variance from each variable set is retained. Furthermore, if X and Y are mean-centered, the same will be true for the projections Another way to overcome the problem with high data dimensionality is to reduce the dimensionality before applying CCA. We use PCA  to reducs principal components, while being advantageous for noise reduction, may result in some highly correlated features of low variance being thrown away. With regularized CCA there is a possibility, depending on the choice of regularization parameters, that such features may be found. The features extracted with PCA+CCA undergo the same cross-validation as the regularized dual CCA features to estimate whether the obtained test set canonical correlations are larger than could be expected by chance only.This approach is intuitively appealing in that we first extract the most variable  features from each data set and then search for the highest correlations between combinations of these. Hence it prevents the CCA from finding high correlations between features of very low variance, which are presumably mostly noise. When using PCA+CCA it is important to be aware that discarding all but the first wx and wy from CCA are sensitive to collinearities among variables, and hence not necessarily a good way of evaluating the individual contribution of each variable to the CCA features. Instead, the cross-loadings of the variables can be used to interpret the extracted features [i'th variable from one of the variable sets with the j'th pair of extracted features is defined as the correlation between the variable and the j'th extracted feature from the other variable set. This means that the cross-loadings indicate the relevance of each variable to each extracted feature pair. Highly correlated variables obtain similar cross-loadings, even though their weights in the extracted features may be different. The relevances of the copy number and gene expression variables are visualized by showing their cross-loadings along the genome, see Figures The weights features . The croTo visualize the representations of the samples, we use their coordinates in the first two pairs of extracted features and show them in a superimposed way . Using tx variable set with respect to the original y variables, indicating the fraction of the variance in the original y-variables shared by the extracted x-features, is denoted by Ry|x, with similar interpretations of Rx|y, Rx|x and Ry|y. Denoting the cross-loading of y-variable i with the j'th feature from the x variable set by cij for i = 1,..., m and j = 1, 2, the redundancy coefficient with the two features is calculated asTo obtain an estimate of the fraction of the variance in the original variable sets which is shared by the extracted features, we calculate the redundancy coefficients ,44. The i with the correlations between the variable and the features extracted from its associated data set, we obtain instead Ry|y. Rx|y and Rx|x are defined accordingly.By replacing the cross-loadings of variable CS designed and implemented the computational analysis, participated in the interpretation of the results and drafted the manuscript. HL pre-processed the data, participated in the interpretation of the results and helped drafting the manuscript. TF and MF contributed to the design of the study, the interpretation of the results and revision of the manuscript. All authors read and approved the final manuscript.Additional file1 contains Supplementary Figures 1, 2, 3 and 4, which show the representation of the samples from the tuning set by the coordinates in the features extracted by the different methods.Click here for fileAdditional file 2τx = 0.9 and τy = 0.3. contains the cross-loadings of the 150 gene expression probe sets showing the highest relevance to the two extracted features from regularized dual CCA with regularization parameters Click here for fileAdditional file3contains the cross-loadings of the 150 gene expression probe sets showing the highest relevance to the two extracted features from PCA+CCA.Click here for file


Critical to the development of molecular signatures from microarray and other high-throughput data is testing the statistical significance of the produced signature in order to ensure its statistical reproducibility. While current best practices emphasize sufficiently powered univariate tests of differential expression, little is known about the factors that affect the statistical power of complex multivariate analysis protocols for high-dimensional molecular signature development.We show that choices of specific components of the analysis  have large and compounding effects on statistical power. The effects are demonstrated empirically by an analysis of 7 of the largest microarray cancer outcome prediction datasets and supplementary simulations, and by contrasting them to prior analyses of the same data.The findings of the present study have two important practical implications: First, high-throughput studies by avoiding under-powered data analysis protocols, can achieve substantial economies in sample required to demonstrate statistical significance of predictive signal. Factors that affect power are identified and studied. Much less sample than previously thought may be sufficient for exploratory studies as long as these factors are taken into consideration when designing and executing the analysis. Second, previous highly-cited claims that microarray assays may not be able to predict disease outcomes better than chance are shown by our experiments to be due to under-powered data analysis combined with inappropriate statistical tests. Microarrays and other high-throughput assaying technologies have generated immense opportunities for discovery spanning the spectrum from basic research to clinical studies statistical reproducibility. The latter can be directly assessed by tests of statistical significance of the produced signatures. These tests are usually permutation based and were introduced in bioinformatics by The reproducibility of gene expression microarrays across laboratories for individual gene expression measurements and the ability to differentiate between disease subtypes are well established in recent studies Although substantial efforts have been invested in studying the statistical power of differential gene expression We start with a theoretical analysis that shows how the choice of four specific components of data analysis protocols for molecular signature development and their statistical testing affects the statistical power to detect predictive signal. We then present a simulation study that demonstrates that depending on choice of the above components even strong signals can fail to be detected with routine sample sizes and that the effects of each component on statistical power are large and compounded. We subsequently test the insights and hypotheses generated by the theoretical analysis and simulation studies with real gene expression data. Specifically, we analyze 7 datasets Fundamental to the assessment of predictive signal of molecular signatures is the choice of error metric that is used to quantify predictivity. An unfortunate frequent practice in the field of bioinformatics to date is to use as classification performance metric the proportion of misclassifications. Discontinuous error metrics such as proportion of misclassifications, sensitivity, and specificity are “improper scoring rules” however, since they impose arbitrary thresholds on predictor models and do not capture the uncertainty in the predictions in principle, there is no biological or mathematical reason why one particular classification method should be better than others” and do not optimize the choice of classifier for the data at hand when conducting statistical testing of microarray gene expression signatures. We will show that this adversely affects the power of their analyses.Statistical power is increased whenever the tested effect size (predictivity in our context) is larger and the variance is smaller (assuming fixed sample size for simplicity). Hence using a classifier that produces the most predictive signature  directly translates to improved statistical power for detecting predictive signal. Statistical machine learning theory proves that different classifiers have different inductive biases , and that a classifier family has to be matched to the characteristics of the domain in order to achieve optimal predictivity  holdout estimator. The holdout estimator is based on splitting the data in two random non-overlapping parts, deriving a signature from the first one and assessing its error in the second one. The holdout estimator is asymptotically unbiased, that is, with infinite test sample it produces an estimate that is the true error in the population. In small samples holdout estimates often deviate from the large-sample value. This variability is reduced as sample size grows Procedures that estimate the generalization error of a signature are called “error estimators”. A commonly used estimator is the repeated 10-fold cross-validation estimatorFrom standard power-size analysis considerations it follows that the lowest-variance unbiased estimator has highest power sampling variance, split variance, testing set size, and internal variance. Sampling variance refers to the uncertainty associated with drawing a random sample of fixed finite size from a population. Split variance refers to uncertainty associated with drawing a random split of training and testing sets from all possible splits of a given sample with fixed training-testing ratio. Testing set size variance refers to variability in error estimates due to finite testing dataset size. Finally, internal variance refers to uncertainty associated with classifier instability  and increases as training set size decreases To see why holdout has higher variance than repeated 10-fold cross-validation, consider that there are several major sources of variance of estimators in practical use. These are: When in the context of error estimation one enforces that both the training and testing sets have the same proportion of events and non-events as the original full data, we will call such error estimation “event balanced”. An important and subtle shortcoming of some data analysis protocols is to not balance the training and testing data, seriously affecting variance, statistical power . For example, in A primary purpose of the simulation experiments is to demonstrate and study the relative importance of the above factors that are hypothesized on the previous theoretical grounds to influence power of complex data analysis of high-throughput data. The simulation uses an idealized analysis in which the data-generating process is known and the true moderate-strength signal is present even in small samples. Such analysis is typical in literature discussing statistical issues surrounding microarray data because knowing the generative model allows a precise characterization of the strengths and limitations of data analysis techniques The left part of The above repeated 10-fold cross-validation AUC estimates in the datasets that had statistically significant signal ranged from 0.67 to 0.76, indicating that even signal with weak strength can be shown in real data to be statistically significant with moderate sample sizes. The U-statistic based confidence intervals for repeated 10-fold cross-validation AUC estimates are provided in the Supporting Information The combined simulated and real data results show very significant differences in the ability of Protocol I and II to detect real signal. This prompted us to investigate further the underlying differences between the two protocols. An unanticipated finding was that a major discrepancy between the two protocols exists in the precise null hypothesis tested: Ideally, one wishes to test the broad null hypothesis “there is no signal in the data”. Rejecting this hypothesis entails that the observed signal in the sample will generalize in the population where the sample is drawn from. There exist several reasons why an observed signal may not be present in the population. First, the available sample may be non-representative of the population. Another reason is that a splitting procedure of the sample into training and testing parts may yield non-representative training or testing datasets (we will refer to this as “bad” split of the data). The previously published procedure for statistical significance testing of the signatures of Protocol I see  section The present work shows that several important components of data analysis for molecular signature creation have significant and compounding effects on probability to detect true signal . Four factors  were investigated by theoretical assessment, simulation study, and application to 7 human microarray datasets.Our findings indicate that the choices made in the data analysis protocol corresponding to the four factors studied can improve power and by extension research efficiency. Increasing study sample size  in simulated or real data with known predictive signal using datasets in which the experimenter varies sample sizes The characteristics of the human datasets analyzed The area under the receiver operating characteristic (ROC) curve (AUC) is calculated by the formula provided in Protocol I C = 100, and a linear kernel Protocol II uses the LibSVM implementation of Support Vector Machines (SVMs) to build molecular signatures with a fixed misclassification penalty parameter n are obtained by sampling without replacement the original dataset (of size N) such that each training set has n/2 subjects with each outcome. For each training set, the testing set is defined as its complement (of size N-n). The molecular signatures are then fitted on the training sets and their classification performance is assessed on the corresponding testing sets. The above procedure is repeated for different training set sizes ranging from 10 to a maximum value which was chosen so that the testing set has at least one subject representing each outcome. Given a distribution of classification performances for each training set size, the corresponding 95% intervals are constructed. The original dataset is considered to contain predictive signal if the upper 95% interval limit is less than 0.5 proportion of misclassifications. Notice that the published description of this method n, or all possible training set sizes, or the majority of them. Thus we examine all three possibilities in the present work.Statistical significance of the molecular signatures in Protocol I replicates the procedure of For Protocol II, we use outcome value-permutation to test in each dataset the null hypothesis of no predictive signal File S1Comparison of proportion of misclassifications with area under ROC curve (AUC)(0.06 MB DOC)Click here for additional data file.File S2Demonstration of pitfalls of non-balanced data(0.10 MB DOC)Click here for additional data file.File S3Details of simulation experiments(0.08 MB DOC)Click here for additional data file.File S4Confidence intervals for repeated 10-fold cross-validation AUC estimates(0.08 MB DOC)Click here for additional data file.File S5Demonstration that Protocol II is not biased(0.09 MB DOC)Click here for additional data file.


A reliable and precise classification is essential for successful diagnosis and treatment of cancer. Gene expression microarrays have provided the high-throughput platform to discover genomic biomarkers for cancer diagnosis and prognosis. Rational use of the available bioinformation can not only effectively remove or suppress noise in gene chips, but also avoid one-sided results of separate experiment. However, only some studies have been aware of the importance of prior information in cancer classification.Together with the application of support vector machine as the discriminant approach, we proposed one modified method that incorporated prior knowledge into cancer classification based on gene expression data to improve accuracy. A public well-known dataset, Malignant pleural mesothelioma and lung adenocarcinoma gene expression database, was used in this study. Prior knowledge is viewed here as a means of directing the classifier using known lung adenocarcinoma related genes. The procedures were performed by software R 2.80.The modified method performed better after incorporating prior knowledge. Accuracy of the modified method improved from 98.86% to 100% in training set and from 98.51% to 99.06% in test set. The standard deviations of the modified method decreased from 0.26% to 0 in training set and from 3.04% to 2.10% in test set.The method that incorporates prior knowledge into discriminant analysis could effectively improve the capacity and reduce the impact of noise. This idea may have good future not only in practice but also in methodology. A reliable and precise classification is essential for successful diagnosis and treatment of cancer. Thus, improvements in cancer classification have attracted more attention ,2. CurreCurrent methods to help classifying human malignancies based on microarray data mostly rely on a variety of feature selection methods and classifiers for selecting informative genes -12. The Lung cancer is one of the leading causes of cancer death worldwide -26, can In summary, together with the application of support vector machine as the discriminant approach and PAM as the feature gene selection method, we propose one method that incorporates prior knowledge into cancer classification based on gene expression data. Our goal is to improve classification accuracy based on the publicly available lung cancer microarray dataset .d, d = 1, 2, ...; (3) radial basis kernel: K = exp{-|x-y|2/σ2 }; (4) Sigmoid kernel: K = tanh [b(x•y)+c], where b, c and σ are parameters. Among these four types of kernel function, radial basis kernel showed best performance according to the results from similar studies [Support Vector Machines (SVM) developed by Cortes & Vapnik  in 1995  studies ,35. The Classification accuracy and the standard deviations of our proposed method (with prior knowledge) were compared with the original one (no prior knowledge) in the training set and test set. The framework of the above mentioned procedures is shown in Figure All the statistical analyses were conducted using R statistical software version 2.80 .The number of genes selected by PAM method varied from 4 to 12 with an average 7.81, and the standard deviation 2.21. The combination of genes selected by PAM is shown in Table After reviewed the full text of literature, twenty-three lung adenocarcinoma-related genes were selected. Then, Table Our proposed method performed better after incorporating prior knowledge Figure . AccuracHere, we considered another situation, if there was an overlap between the two sources of genes, i.e. there existed the multi-collinearity, was there any influence on the performance of classification? Hence, taking into account the effect of overlap seemed natural for the current study. Expression quantity of VAC-β with a coefficient 1, 0.5 and 0.05 which meant complete, strong and minor correlation was added to data set for comparison, respectively. The accuracy in the above situation is 99.12%, 99.28%, 99.23% with the standard deviation 2.04%, 2.04%, 1.93%, respectively  based on the location and protocol used for sample preparation . The comIn summary, a simple and general framework to incorporate prior knowledge into discriminant analysis was proposed. Our method seems to be useful for the improvement of classification accuracy. This idea may have good future not only in practice but also in methodology.PAM: prediction analysis for microarrays; SCRDA: shrunken centroids regularized discriminant analysis; MTP: multiple testing procedure; KNN: k nearest-neighbor classifiers; LDA: linear discriminant analysis; SVM: support vector machine; BP-ANN: back-propagation artificial neural network; SCLC: small cell lung cancer; NSCLC: non-small cell lung cancer; MPM: malignant pleural mesothelioma; ADCA: adenocarcinoma; CV: cross-validation.The authors declare that they have no competing interests.PG conceived the study and drafted the manuscript. PG, DH, MH and BZ retrieved and reviewed the literature. PG and BZ attracted funding. All authors contributed to the writing of the final version of this paper.


In the study of cancer genomics, gene expression microarrays, which measure thousands of genes in a single assay, provide abundant information for the investigation of interesting genes or biological pathways. However, in order to analyze the large number of noisy measurements in microarrays, effective and efficient bioinformatics techniques are needed to identify the associations between genes and relevant phenotypes. Moreover, systematic tests are needed to validate the statistical and biological significance of those discoveries.In this paper, we develop a robust and efficient method for exploratory analysis of microarray data, which produces a number of different orderings (rankings) of both genes and samples (reflecting correlation among those genes and samples). The core algorithm is closely related to biclustering, and so we first compare its performance with several existing biclustering algorithms on two real datasets - gastric cancer and lymphoma datasets. We then show on the gastric cancer data that the sample orderings generated by our method are highly statistically significant with respect to the histological classification of samples by using the Jonckheere trend test, while the gene modules are biologically significant with respect to biological processes (from the Gene Ontology). In particular, some of the gene modules associated with biclusters are closely linked to gastric cancer tumorigenesis reported in previous literature, while others are potentially novel discoveries.In conclusion, we have developed an effective and efficient method, Bi-Ordering Analysis, to detect informative patterns in gene expression microarrays by ranking genes and samples. In addition, a number of evaluation metrics were applied to assess both the statistical and biological significance of the resulting bi-orderings. The methodology was validated on gastric cancer and lymphoma datasets. A typical aim of exploratory analysis of genomics data is to identify potentially interesting genes and pathways that warrant further investigation. There is a critical need to streamline the analysis in order to support continuing advances in high throughput genomics methods such as gene expression microarrays, which measure thousands of genes in a single assay and are the focus of this paper. However, such assays provide noisy and incomplete measurements, which require sophisticated bioinformatics techniques to identify statistically and biologically significant associations between genes and relevant phenotypes of interest.subset of samples, or participate in multiple cellular pathways. A related open issue is how to evaluate the statistical significance of the clusters.Unsupervised analysis techniques cluster data without using prior information on the labels of samples. This enables the discovery of novel histological subtypes. However, a major limitation of traditional clustering algorithms for this task is that they cluster either genes or samples into non-overlapping groups, based on the similarity of gene expression across all samples for gene clustering, or all genes per sample in sample clustering. This limits the ability to find groups of genes that are "co-correlated" across only a In spite of such limitations, there are examples of remarkable biologically significant discoveries. One such case revisited in this paper is the analysis of gastric cancer data . The oriBiclustering, also called co-clustering, is a promising technique proposed for the automated discovery of highly correlated subsets of genes across a subset of samples. The concept of "biclustering" was first introduced by  and has bi-ordering, which is in some respects a member of the family of biclustering techniques. This algorithm is benchmarked against several relevant biclustering algorithms in the literature G n× S nfor G ngenes for S nsamples.Input: A gene expression data matrix ; two cut-off thresholds G θand S θfor selecting genes and samples, respectively.Input: median = 0 and std = 1, then repeat this for each sample across all genes.1. Normalize data: first, for each gene across all samples, normalize to S ⊂ {1, ⋯, Sn}2. Initialization: Select a subset of samples ∅ ≠ G and S stabilise):3. Repeat the Steps a-d below until convergence  ← ⟨gsx⟩s∈S for g = 1, ⋯, Gn,(a) Update gene scores (b) Select genes: h(s) ← ⟨gsx⟩g∈G for S = 1, ⋯, Sn,(c) Update sample scores (d) Select samples: f(g), sample scores h(s), and a bicluster B = , which is determined by applying the cut-off thresholds G θand S θto the scores f(g) and h(s), respectively.Output: The gene scores According to the iterative process of step 3 in the BOA algorithm, we aim to select a submatrix with coherently high gene expression levels across selected samples. In particular, we aim to detect differentially expressed genes between phenotypes. Consequently, it is important to normalize the data in step 1, otherwise, the genes with high expression levels across most samples will dominate the selection procedure. The above algorithm is similar to ISA , which iNote that two further variants of BOA are possible:Under-expressed biclusters - This variant finds significantly down-regulated genes by selecting genes and samples with f(g) and h(s) less than a threshold in steps 3.b and 3.d, respectively.1. Fixed-size biclusters - We could also select G and S of fixed size rather than using the cut-off thresholds in each iteration. The attraction is that the algorithm is guaranteed to converge, which in practice may not always hold for the previous two options. The formal proof of the convergence for this variant is included in the Additional file 2. h(s) to samples and f(g) to genes in a bicluster. The sample score h(s) orders all samples according to the average expression level across a subset of genes in the current bicluster, while the gene score f(g) orders all genes according to a subset of samples. The sample score h(s) relates a bicluster to the clinical annotations of samples, especially for multi-class attributes, such as the cancer progression stage, or continuous attributes, such as the survival time. These are discussed in detail in Section 2.3.An important ability of the BOA algorithm is to assign an ordering score f(g) and sample score h(s) are uniquely determined by the selection of the initial subset of samples S. This is because every selection in Step 3 of Algorithm 1 is deterministic for a given pair of thresholds . Thus, different initializations may result in different biclusters, which are local optima after a few iterations. In order to cover more potential biclusters, we run BOA with 1000 different initial subsets of samples. In every independent initialization, each sample is drawn into the initial subset randomly with probability of 0.2. Some of the 1000 generated biclusters are identical, while others may be very similar to each other but not exactly the same due to local optima. In order to identify robust and distinct biclusters, we then apply a hierarchical clustering algorithm to group similar biclusters into super-biclusters (SBC). We use the Jaccard coefficient on gene sets as a similarity measure between two biclusters. For example, given two biclusters B1 =  and B2 = , the similarity between them is:Super-biclustering is an important feature of our technique, in terms of improving the robustness and stability of the results. For every individual bicluster, the ultimate output of gene score The similarity measure could also be defined on samples, though here we focus on genes, which are the dominant and far more complex dimension to handle in gene expression microarray datasets. Based on the similarity measure in Equation 1, a hierarchy of unique biclusters is constructed. We then use a similarity threshold of 0.5 to extract a few top groups of biclusters, which are super-biclusters, from this hierarchy. For convenience of evaluation, a typical bicluster is selected as a prototype in each resulting SBC. Without loss of generality, we choose the bicluster that covers the most genes in its SBC as the prototype, and so any further numerical evaluation is based on these prototypes when we refer to a SBC.In order to evaluate the statistical and biological significance of the resulting biclusters, we employ three statistical methods and their corresponding p-values.SCS) and Multiple-Class Saturation (MCS) metrics, respectively. The calculation of MCS p-values based on the hypergeometric distribution is given in Equation 2 below.The homogeneity of phenotypes of samples in a bicluster can be evaluated if we are given a prior classification of each sample  as the label. Ideally, each bicluster should be enriched with samples in one or a few more similar classes, e.g., normal or tumor samples. For the purpose of quantification, we use the p-value of the hypergeometric distribution to evaluate the purity of biclusters according to the phenotypes of samples. Previously , a measuq classes {C1, ⋯, qC} and a bicluster B = , the p-value with respect to a group of r classes Given a classification of samples with whereB annotated with any class in k or more samples classified in B.are the numbers of samples in the dataset and in the bicluster C1, ⋯, qC} and compute MCS pfor each bicluster and each combination, so that we could discover any potential associations between gene sets and a group of phenotypes. Finally, we select the subset of classes that corresponds to the most significant p-value for each bicluster in the evaluation in Section 3. Note that the SCS is a special case of the MCS. We compute a p-value with respect to each individual class, and then select the single class that corresponds to the best p-value for each bicluster.In our evaluation, we generate the full set of combinations of all sample classes from {h(s) generated by BOA with any relevant ordering y(s) of all samples based on their biological annotations, e.g., the progression stage of the cancer in the sample. We can test the agreement of samples ordered according to h(s) with this progression score y(s). We use Jonckheere's test statistic [Another method to evaluate the significance of a bicluster is to compare the ordering of all samples tatistic :for this purpose.h(s) (the null hypothesis), this random variable U has an approximately normal distribution. For example, consider that we have an annotation scoring y(s) of samples with respect to q sample classes {C1, ⋯, qC}, which can be numerically ranked, e.g.,For a random scoring i N(1 ≤ i ≤ q) denote the number of samples in class iC, and N denote the total number of samples. The approximate normal distribution of U determined by the random scoring h(s) and the annotation scoring y(s) has the mean:Let and the variancefrom which the p-values can be estimated.Given that each gene's expression in a bicluster is highly similar with respect to other genes in the bicluster, it is expected that the collection of genes as a whole are likely to be involved in some related biological processes. In order to determine this, the structured vocabulary of the Gene Ontology (GO)  is used G n+ Sn), since only averaging operations for computing the gene score f(g) and sample score h(s) are required. Practically, the number of iterations for generating a single bicluster is usually no more than 10, and the number of initializations is 1000 in our experiments.One of the advantages of the BOA algorithm is its efficiency. The time complexity in each iteration is In this section, we analyze the performance of our algorithm on a real gene expression dataset, namely the gastric cancer dataset in . The maiG n= 7383 gene expressions evaluated for S n= 124 human tissue samples. Excluding two singletons, there are six different phenotypes in the data, of which three are subtypes of gastric cancer: 35 diffuse (DGC), 22 intestinal (IGC), 7 mixed (MGC); and the other three phenotypes are pre-malignant conditions: 26 chronic gastritis (CG), 22 intestinal metaplasia (IM) and 10 normal, e.g., non-inflamed mucosa tissue removed during surgery for the gastric cancer. Now we briefly discuss the algorithmic aspects and setup of the experiment.After applying gene filtering as described in , we haveFirst, we generated a set of 1000 initializations, which were 1000 subsets of samples selected by the method described in Section 2.2. The actual number of initializing samples for gastric cancer data ranged from 14 to 41 across 1000 subsets. As described in Section 2.2, each sample is randomly selected with a probability of 0.2 for inclusion in the initial subset of samples. Note that other selection probabilities of 0.4, 0.6 and 0.8 have been tested, but the results were largely insensitive to changes in this parameter.mean = 0 instead of median = 0 for centering the genes and samples. Here, we followed the normalization method used in [median = 0 and normalizing by mean = 0 on the dataset we have studied.Note that in the BOA algorithm, there are other alternative normalization methods that can be used, i.e., using  used in  for the θG, Sθ) = , , , , , , , , , , , which used the same set of initializations.Second, we applied BOA to the gastric cancer data using 11 different pairs of thresholds: , and between 3.4 × 10-27 and 4.0 × 10-14 for the MCS metric. For further analysis we chose a mid-range pair G θ= 5 and S θ= 4.5 for which, additionally, all 1000 initializations of BOA converged. Under this pair of thresholds, the algorithm converged to 49 biclusters, which were further grouped into 8 super-biclusters  and a MCS p= 4.03 × 10-14 with respect to the MCS metric .To show the significance of the resulting biclusters we focus on the most stable super-bicluster generated for the gastric data, labeled SBC7 in Table h(s) change smoothly. Thus, we also used Jonckheere's test to overcome these limitations. We first allocated a "Malignancy Score" y(s) to each sample s following the expert advice: y(s) = 1 for normal, 2 for CG, 3 for IM and finally 4 for any gastric cancer . We then tested the significance of the agreement of the samples ordered according to the h(s) score generated by the BOA algorithm with this progression y(s). For the prototype of SBC7, the malignancy scores show an increasing trend from normal (y(s) = 1) to malignant samples (y(s) = 4) along the ascending ordered gene expression levels, which results in a directional p-value of 5.35 × 10-22.However, there are two limitations of calculating SCS or MCS. First, these measures cannot deal with the case of continuous annotations of samples. Second, the significance of SCS and MCS are affected by the choice of cut-off threshold on samples, especially when the sample orderings χ2 distribution. The p-values were corrected by the process of controlling the False Discovery Rate [For every bicluster, we used the GOstat program  to obtaiery Rate  in our eMore biological details of the gene modules and evaluation statistics for different SBCs are discussed in the next section.As a basis for comparison with our BOA algorithm, we have also tested several existing biclustering algorithms, namely, Cheng and Church's algorithm , SAMBA , biclustCheng & Church's method was chosen as a comparison because it is similar to our BOA method in two aspects. First, they both aim to identify "constant biclusters" . Second,The BOA algorithm is very similar to ISA. However, the main objective of ISA is discerning "co-regulated" gene modules, while the association with phenotype classes (conditions) is not important, whereas it is of prime interest for our medical application. The main formal differences resulting in the different performance are: (i) ISA starts with an initialisation of a subset of genes; (ii) the two sided test is used for the selection of samples; (iii) samples are weighted, with possibly negative weights, so different conditions, say with up-regulated and down-regulated genes, can be joined in the same bicluster. Consequently, ISA aims at generating "constant column" biclusters while BOA's objective is a "constant" bicluster .Figure -22 and 4 × 10-18  in SBC6 of BOA, were also found by the ISA and Gibbs algorithms. However, we have observed that the BOA algorithm usually has better performance than either ISA or Gibbs in terms of Jonckheere's test, in particular, in the case of the evaluation of the "Malignancy Score".Our evaluation of GO annotations for both ISA  and biclTo further validate the performance in terms of SCS and MCS, we applied BOA to a lymphoma dataset , and comIn this section, we focus on validating the biological significance of our findings for the gastric cancer dataset.We first compare the gene modules of the prototypes of the super-biclusters with those reported in a previous study . In thaty(s) and tested the significance of the agreement between y(s) and sample orderings h(s) in Section 3.1. Table h(s) for SBC5 and SBC7 and to a lesser extent SBC3 are very significantly correlated with y(s). More biological relevance is discussed in the Discussion section.In Table Based on the results of our experiments, we now consider the biological significance of our findings. The generated results including the GO and clinical correlation were analysed by expert biologists and clinicians. We quote them to some extent as a proof that the formal data processing protocols as discussed here can lead to the generation of significant biological hypotheses warranting follow-up wet lab experiments. The BOA algorithm has shed new light on preexisting themes in gastric cancer etiology. The resulting bi-orderings represent successive steps in cancer progression and distinct histopathological types of the disease. Specifically, SBC1 represents epithelial morphology, typical to squamous samples; SBC2 and SBC8 are typical intestinal lipid metabolism signatures, observed in intestinal metaplasia pre-malignant samples; SBC3 and SBC4 represent a novel split of the inflammatory signature that in [In this paper we have presented a novel method of bi-ordering genes and samples from microarray data, together with two statistical techniques for evaluating the significance of the generated groupings and orderings of multiple histological samples. In comparison to several existing algorithms in the literature, our method is able to generate highly robust and statistically significant gene modules with respect to sample histological annotations on a gastric cancer dataset. The results of our analysis closely match reported theories of gastric cancer tumorgenesis, and have helped to identify promising hypotheses for further investigation in cancer research. We also show that other biclustering algorithms can also be utilized as a basis of exploratory bi-ordering analysis of genomic data.Fan Shi, under the supervision of Christopher Leckie and Adam Kowalczyk, developed the major part of the algorithms. Geo MacIntyre contributed to the Gene Ontology evaluation of the results. Alex Boussioutas and Izhak Haviv analysed the biological relevance of the results. All authors contributed to the writing of the manuscript, and they have read and approved the final version of the manuscript.Supplement. The Supplement contains the proof of convergence in a variant of BOA algorithm (See 2 for details), and the biological analysis of potential novel observations in the gastric cancer dataset discovered by our method.Click here for filealgorithm implementation. The file of "algorithms.zip" contains the Matlab source code files (.m) implementing the BOA algorithm.Click here for file


Different microarray techniques recently have been successfully used to investigate useful information for cancer diagnosis at the gene expression level due to their ability to measure thousands of gene expression levels in a massively parallel way. One important issue is to improve classification performance of microarray data. However, it would be ideal that influential genes and even interpretable rules can be explored at the same time to offer biological insight.Introducing the concepts of system design in software engineering, this paper has presented an integrated and effective method (named X-AI) for accurate cancer classification and the acquisition of knowledge from DNA microarray data. This method included a feature selector to systematically extract the relative important genes so as to reduce the dimension and retain as much as possible of the class discriminatory information. Next, diagonal quadratic discriminant analysis (DQDA) was combined to classify tumors, and generalized rule induction (GRI) was integrated to establish association rules which can give an understanding of the relationships between cancer classes and related genes..Two non-redundant datasets of acute leukemia were used to validate the proposed X-AI, showing significantly high accuracy for discriminating different classes. On the other hand, I have presented the abilities of X-AI to extract relevant genes, as well as to develop interpretable rules. Further, a web server has been established for cancer classification and it is freely available at  The challenge of cancer treatment is to develop specific therapies based on distinct tumor types, to maximize efficacy and minimize toxicity. Hence, improvements in cancer classification have been paid more and more attention. Recently, microarray gene expression data has been successfully used to investigate useful information for cancer classification at the gene expression level. One of the earliest methods for cancer classification is the weighted voting machine which is based on a linear model . Other mIn addition to classification, another challenge is to extract relevant genes, even creditable and interpretable rules from microarray gene expression data to offer biological insight between genes. Several kinds of rules have been successfully developed in different subjects of molecular biology. In our earlier studies, decision rules based on decision tree algorithms have been effectively extracted from the thermodynamic database of proteins and mutants to explore potential knowledge of protein stability prediction -13. On tTypically, a classification system may achieve high accuracy by non-linear models, but these models are hard to provide rules. In contrast, a rule extraction system is necessary to consider the model interpretability which can provide a pathway to explore underlying relationships among data; however, this restriction often affects the system performance in classification. Hence, a learning model which can provide accurate classification, as well as useful rules, would be ideal. Even so, a relatively few attempts have been made to integrate the two types of systems on microarray gene expression data. In earlier reports, Li et al.  has propIn this paper, I have presented an integrated method (named X-AI) which is based on a three-tiered architecture from the viewpoint of system design of software engineering. Different tests have been carried out on two leukemia datasets for evaluating the performance of X-AI. The obtained results indicated that X-AI is able to perform well on both functions of classification and rule extraction in microarray analysis.I used two different leukemia datasets for the following reasons: (i) both datasets have been analyzed and discussed in many literatures, which is helpful to compare with their results; (ii) the rules extracted from the similar cancer type of datasets could be compared to each other; (iii) the robustness of classification system could be observed by the datasets that are obtained from different experiments; and (iv) the two datasets represent the nature of the binary classification and multi-class problems, which is useful to evaluate the effectiveness of the proposed method for different classification problems.The first acute leukemia data named L1) of Golub et al.  of Golub includes which evolved from research collaborations in the MIT and Harvard communities and made the generated data available to the scientific community.More details of datasets can be found on the web server and in Broad Institute From the viewpoint of system design in software engineering, Yourdon and Constantine  made a mThe three-tiered architecture integrates the tasks of microarray data analysis from the pre-processing to the data mining including classification and rule extraction. Each function layer with independency can be changed internally without affecting other layers. Therefore, this architecture can provide the consistency of data to different components of the same layer, and reduce the interaction between layers as well as between the components of the same layer.The proposed X-AI method primarily implemented the data mining and the data reduction layers of the architecture, and integrated three functions: (i) feature selection, (ii) cancer classification, and (iii) associate rule development  discriminant rule discriminates the class of a feature vector kelihood . For mulμi is the mean of x for the ωi class, Σi the l by l covariance matrix. When the covariance matrices are diagonal, μi and Σi are estimated by corresponding sample quantities. we have effectively utilized it for the analysis of discriminating two- and three-state proteins [where proteins . In thisif antecedent then consequent, GRI applies J-measure quantifies its information content:Generalized rule induction was proposed by Smyth and Goodman , which ap(a) represents the probability of the observed attribute value of a, as a measure of the coverage of the antecedent; p(c) represents the prior probability of the value of c, as a measure of the common of the observed attribute value of c in the consequent; p(c|a) represents an modified probability of observing this value of c after taking into account the additional information of the value of a. For rules with more than one antecedent, p(a) is regarded as the probability of the conjunction of the variable values in the antecedent. Accordingly, a set of optimal rules was then generated by ITRULE algorithm, which calculates J-measures of rules by employing depth-first search over possible left-hand sides.where Here, the genes selected by Chi2 algorithm were considered as the attributes of the antecedent. And the tumor class was the only attribute of the consequent.I considered the classification of the leukemia datasets L1 and L2 as the two-class and three-class problems, respectively. To evaluate the performance of the classification problems, both classification accuracy and misclassified number were calculated along with corresponding number of selected genes.The support and confidence measures were defined to reveal the importance of individual association rule. For a particular association rule, support is the proportion of samples in the dataset that contain the rule antecedent:This measure reveals the comprehensiveness of the rule to the dataset.Further, confidence of the association rule is a measure of accuracy of the rule:n-fold cross-validation, where n is the number of samples in the dataset. Each sample is left out in turn, and the predictor is trained on all the remaining ones. The procedure is repeated for n times to obtain a mean score.The present method was validated by both holdout validation and leave-one-out cross-validation (LOOCV) tests. Holdout validation derives a predictor from the training set, and uses the blind or independent test set to evaluate the predictor. LOOCV is simple X-AI provides a feature selection function to systematically extract the relative important genes for discriminating different classes. In Table In the part of L1, the importance of most genes has been discussed in the study of Golub et al.  also preDifferent tests have been applied to verify the accuracy of the classification function of X-AI. For holdout validation test, it shows the accuracy of 96% and 99% on the test sets of L1 and L2, respectively, using the ten genes as input information. I have also carried out the analysis of classification accuracy along with the corresponding number of genes by holdout validation test. Figure The results show that the classification function performs well in discriminating these different classes when the input information is provided by the feature selector function of X-AI. Namely, the integration of the both functions can be feasible and effective for the binary classification and three-class problems.The performance comparison between X-AI and other methods has also been made on different datasets. The results provide an overall view about the performance of different methods. In Figure k-nearest neighbor (k-NN), SVM and prediction analysis of microarrays (PAM) [Figure ys (PAM) . The accThe function of feature selection did not only reduce the number of input genes, but also improve the efficiency of rule development. It also results in a rational and acceptable number of rules. Based on the genes of Table if the expression of M23197 (CD33) is larger than 401.5, then the sample is classified as ALL. For dataset L1, 29.17% samples contain the antecedent of this rule and all these samples are correctly classified. This rule efficiently reveals the importance of the gene in discriminating between AML and ALL. This finding is in accord with the results of earlier studies [Tables  studies  also gav. The prediction can be made by taking four simple steps (see Figure I have also developed a web server for classifying tumors of acute leukemia and it is freely available at Because X-AI selected two different sets of input genes from two datasets for training the classifiers, it results in two classifiers with different sets of input genes. Users can optionally assign one of both to predict cancer classes. In addition to the cancer classification page, the web server has provided help and reference pages for interested researchers.In this study, I have proposed an integrated method for accurate cancer classification, relevant gene selection, and the associate rule development from DNA microarray data. Applying the concepts of system design, the modules in the present architecture are tight cohesion and loose coupling.Through different tests, the method shows high classification accuracy on two leukemia datasets. In addition, the selected genes and the generated rules are in accord with recent studies. The results suggest that the method can effectively integrate these related functions for the analysis of microarray data.The author declares that they have no competing interests.