Detecting breast cancer at early stages can be challenging. Traditional mammography and tissue microarray that have been studied for early breast cancer detection and prediction have many drawbacks. Therefore, there is a need for more reliable diagnostic tools for early detection of breast cancer due to a number of factors and challenges. In the paper, we presented a five-marker panel approach based on SVM for early detection of breast cancer in peripheral blood and show how to use SVM to model the classification and prediction problem of early detection of breast cancer in peripheral blood. We found that the five-marker panel can improve the prediction performance (area under curve) in the testing data set from 0.5826 to 0.7879. Further pathway analysis showed that the top four five-marker panels are associated with signaling, steroid hormones, metabolism, immune system, and hemostasis, which are consistent with previous findings. Our prediction model can serve as a general model for multibiomarker panel discovery in early detection of other cancers. Traditional methods mostly used for early detection have been regular and periodic self-examination and annual or biannual checkups using mammography and analysis of tissue biopsies. But mammography as a screening tool for early detection has many drawbacks. For example, mammography may not detect small tumors and is often unsatisfactory for younger women, who typically have dense breast tissue. And if a patient does have a suspicious mammogram, a biopsy will probably be done to make the diagnosis. Obtaining tissue biopsies can be difficult for several reasons, including small size of lump, lack of available medical facilities, and patients' reluctance to undergo invasive procedures due to scaring and costs.In recent years, functional genomics studies using DNA microarrays have been shown effective in differentiating between breast cancer tissues and normal tissues by measuring thousands of differentially expressed genes simultaneously –3. HowevTo address these issues, a novel and minimally invasive test that uses easily obtained peripheral blood for breast cancer detection has been developed , 8. For There is a need for more reliable diagnostic tools for early detection of breast cancer in peripheral blood which can achieve high prediction accuracy with as few genes as possible and to reduce the required examination of a large number of genes which increases the dimensionality, computational complexity, and clinical cost of diagnosis . Li estiSupport vector machine (SVM) has several unique characteristics as a research tool for prediction in cancer classification applications. One unique characteristic as a specific type of learning algorithm is that it is characterized by the capacity control of the decision function, the use of the kernel functions, and the sparsity of the solution . The secFor classification and prediction of breast cancer samples, these unique characteristics make SVM appealing as compared with regression-based models or neural network as seen in –13. For There has not been any report until this study that applied SVM to the development of multimarker panels for early detection of breast cancer based on peripheral blood. Based on a neural network approach to multibiomarker panel development for LC/MS/MS proteomics profiles we developed , we propThe peripheral blood data are publicly available through the GEO database with the accession number GSE16443  and werePer sample normalization was performed to normalize for staining intensity variations among samples. All expression data on a sample were normalized to the 50th percentile of log base 2 of all values on that sample. First, log ratio base 2 transformation was used to transform the data. And then for each probe the median of the log summarized values from all the samples was calculated and subtracted from each of the samples.yij using the following:Sj ∈ N, εij ∈ N. Here, μ is the mean expression value, Ti is the fixed group effect , Sj is the random sample effect , and εij is the within-groups errors. All random effects are assumed independent of each other and independent of the within-groups errors εij.We used the ABI Human Genome Survey Microarray Version 2 to manage and map probe IDs. A full factorial model was used to represent the fixed effect and the random effect which are used to account for group and probe. The expression log ratios value is the final quantity that is fit by a separate analysis of the variance (ANOVA) statistical model for each probe as P value of the significance for the group effect. Then we calculated the FDR adjusted P value. Last, we calculated the FDR q value using the Storey-Tibshirani method [q < 0.01) to select genes of which we estimated significant differences in the health and breast cancer samples.Statistical significance was measured by a three-step method. First, we conducted the above linear mixed model to obtain the i method . We chosThe classification problem of breast cancer can be restricted to consideration of the two-class problem without loss of generality . We used a support vector machine- (SVM-) based methods  to develk-fold cross validation on the training group to find the “optimal” parameters for the classifier. Group A is randomly partitioned into k subsamples. For each subsample, a cross section of the data is flagged for use as the validation set, and a new model is created by training on the remaining data which are the training set and not in the subsample. The cross validation process is then repeated k times (the folds), with each of the k subsamples used exactly once as the validation data. The k results from the folds then can be averaged to produce a single estimation. The testing group is used as testing set.For the use of the support vector machine as an appropriate tool for prediction of the breast cancer, a three-way data split is applied for training, validation, and testing. The training set is used for learning to fit the parameters of the classifier. The validation set is used to tune the parameters of the classifier. And the testing set is used only to assess the performance of the fully-trained classifier. We first randomly split the data into two groups: group A (training group) and group B (testing group), with roughly equal size. Then we use the N (N = 5 for five-marker panel) out of all the 42 genes differentially expressed in the training group as inputs to the SVM. In order to find the optimal classifier, we presented an optimization method that measures the area under the curve (AUC) for receiver operating characteristics (ROC). In this scheme, we first train SVM for each combination in the training set with 5-fold cross validation. Then, we measured the AUC for each combination in the validation set. Lastly, the optimal combination C* was determined byC is combination of picking five out of the 42 genes, and V is the validation set of training group.We chose each combination of Fivefold cross validation was used to increase the number of estimates and improve the accuracy of the prediction model by avoiding the overfitting. In 5-fold cross validation, the original sample is randomly partitioned into 5 subsamples. Of the 5 subsamples, a single subsample is retained as the validation data for testing the model, and the remaining 4 subsamples are used as training data. The cross validation process is then repeated 5 times, with each of the 5 subsamples used exactly once as the validation data. The 5 results from the folds then can be averaged to produce a single estimation. The advantage of this method over repeated random sub-sampling is that all observations are used for both training and validation, and each observation is used for validation only once.http://bioinfo.hsc.unt.edu/ipad/) [The Integrated Pathway Analysis Database (IPAD) (u/ipad/)  is used We downloaded from the Gene Expression Omnibus (accession number GSE16443)  the 130 q  value  <0.01. No data from the testing set were utilized in (1) identification of peripheral blood markers or (2) development of the SVM model.We obtained 42 markers in the training group with An SVM model with 5-fold cross validation was built on all 42 markers in the training group. We obtained a high performance  for the training group but a low performance  for the testing group . The resN = 5 for five-marker panel because (1) our pilot study shows that five markers can be enough to achieve a satisfied performance for prediction and classification of cancer [Training of the SVM was performed using radius basis function (RBF) kernels function and five-fold cross validation. Receiver operating characteristic (ROC) curve and area under curve (AUC) were calculated to help evaluating the predictive performance of the SVM. We choose f cancer , (2) pref cancer , and (3)In order to validate our prediction method, we compared the ROCs for the best four 5-marker panel predictions determined by our method with the ROCs for four randomly chosen 5-marker panels from 42 candidate biomarkers . As showIn Pathway analysis shows that the pathways linked with the best four five-marker panels are signaling, steroid hormones, metabolism, immune system, and hemostasis , which aThe confusion matrix and common performance metrics for both the training group and testing group for the best five-marker panel are shown in the k-fold cross validation where the original sample was randomly partitioned into k subsamples and of the k subsamples, a single subsample was retained as the validation data for testing the model, and the remaining k − 1 subsamples were used as training data, our prediction performance actually outperformed them. When we applied an SVM and 5-fold cross validation with our best 5-marker panel to the training group of 34 women with breast cancer and 32 healthy women controls, we obtained a higher performance than these previously published findings  and decrease the size of testing set by simply moving some samples in the testing set to the training group.N markers, there is a limitation for the size of N due to current computational capability. In our talon supercomputer, it would take about 1 hour to calculate all combinations of 5 out of 32 markers and about two weeks to finish the computation of picking 5 out of 100 markers. It is acceptable for us to set the maximum of N to be 100 because in most cases the top 100 markers can be both specific and sensitive in understanding the treatment, diagnosis, and prognosis of cancer and can be limited by setting a reasonable P value threshold.Since our approach enumerates all possible combinations of 5 out of t-test, or SAM. However, ANOVA is a much more flexible and powerful technique that can be applied to much more complex research issues with multiple factors than the other two methods. For example, for the peripheral blood data, we should take into account two factors: (1) the fixed group effect  and (2) the random sample effect . In this case, ANOVA method is more efficient than multiple two-group studies analyzed via t-test or SAM, because with fewer observations we can gain more information.An ANOVA statistical model is used for identifying differentially expressed genes between cancer and normal samples. For a simple two-group comparison, we would get the identical result if we were to compare the two groups using ANOVA, In this work, we use the support vector machine (SVM) for classification, which is in general believed to outperform the other classification methods such as the logistic regression (LR) and the artificial neural networks (ANN) , 20, becWe developed an integrated computational approach that addressed a challenging multipanel biomarker development problem in the early detection of breast cancer in peripheral blood. The approach that we used combined simple statistical filtering of ANOVA with an optimization model of SVM. The approach automatically learned nonlinear relationships between features and outcomes to generate predictive models, which achieved AUC = 0.7879 performance with a sensitivity of 63.64% and a specificity of 74.19% in the testing data set of 33 women with breast cancer and 31 healthy women controls. The SVM combined with the AUC optimization method is capable of identifying the optimal combination of multimarkers for performance comparable to that of conventional medical decision support systems. We believe that this computational approach works well with early detection of breast cancer in peripheral blood and can provide general guidance for future molecular medicine multimarker panel discovery applications in other diseases. In the future, we will follow up with biological experiments to validate these biomarkers with our collaborators.


Formalin Fixed Paraffin Embedded (FFPE) samples represent a valuable resource for cancer research. However, the discovery and development of new cancer biomarkers often requires fresh frozen (FF) samples. Recently, the Whole Genome (WG) DASL  assay was specifically developed to profile FFPE tissue. However, a thorough comparison of data generated from FFPE RNA and Fresh Frozen (FF) RNA using this platform is lacking. To this end we profiled, in duplicate, 20 FFPE tissues and 20 matched FF tissues and evaluated the concordance of the DASL results from FFPE and matched FF material.We show that after proper normalization, all FFPE and FF pairs exhibit a high level of similarity (Pearson correlation >0.7), significantly larger than the similarity between non-paired samples. Interestingly, the probes showing the highest correlation had a higher percentage G/C content and were enriched for cell cycle genes. Predictions of gene expression signatures developed on frozen material  showed a high level of concordance between FFPE and FF matched pairs. Interestingly, predictions based on a 60 gene DASL list (best match with the 70 gene signature) showed very high concordance with the MammaPrint® results.We demonstrate that data generated from FFPE material with the DASL assay, if properly processed, are comparable to data extracted from the FF counterpart. Specifically, gene expression profiles for a known set of prognostic genes for a specific disease are highly comparable between two conditions. This opens up the possibility of using both FFPE and FF material in gene expressions analyses, leading to a vast increase in the potential resources available for cancer research. Tissue samples collected during surgery as well as biopsies are often fixed in Formalin and embedded in Paraffin (FFPE). Molecular genomics assays on archived FFPE blocks, together with clinicopathological information, can provide critical insights into a heterogeneous disease like breast cancer, especially considering the fact that FFPE samples are the most widely available source of tissue material for which long-term clinical follow-up data are recorded. The ability to perform gene expression profiling on these samples will enable many prospective and retrospective studies to be performed facilitating the association of expression profiles with clinical outcomes Microarray expression analysis using FFPE tissues has been problematic as the retrieval of RNA from FFPE material is challenging DNA-mediated Annealing, Selection, Extension and Ligation) with the specific aim of overcoming the technical limitations that are associated with microarray-based analyses of FFPE samples. The DASL assay employs priming with random hexamers in the cDNA synthesis stage and therefore does not depend on poli-A/oligo-dT based priming. Moreover, the assay requires only short target sequences (about 50 nucleotides) for query oligonucleotide annealing implying that also degraded RNA can be quantified Recently, Illumina has developed an innovative assay called DASL , developed from FF samples, on FFPE samples in order to assess how well the FFPE material captures the biological information contained in these signatures. Lastly, we evaluate whether there are any specific biological characteristics that are correlated with the detectability of a transcripts extracted from FFPE material.Twenty one breast cancer patients diagnosed in 2008 at the Netherland Cancer institute were selected for this study based on the availability of both FFPE  and FF (Fresh Frozen) material. For these patients, we collected 21 individual FFPE blocks and 21 individual snap frozen diagnostic biopsies. Sample characteristics are reported in the RNA from FFPE material was extracted using the High Pure RNA paraffin kit (Roche), which is the method recommended by Illumina for DASL applications RNA from FF material was extracted using the RNeasy Mini Kit (QIAGEN). Two 5-µm sections, which were cut before and after sectioning for RNA isolation, were stained with haematoxylin and eosin and reviewed by a pathologist to determine the tumor cell percentage. Only samples with ≥50% tumor cells were used. From the biopsy 15 sections of 30 µm were cut and placed in a 2 ml reaction tube containing 1 ml RNAzol B reagent (Biogenesis). The tissue was homogenized using a rotor-stator homogenizer and 200 µl of chloroform was added to the solution. After a centrifugation at 13000×g (4°C) for 15 minutes the upper aqueous phase containing RNA was transferred in a new vial. From this point on the manufacturer's protocol was followed, including an on-column DNase digestion for eliminating any DNA contamination. For each FF biopsy two independent RNA isolations were performed, in order to have two biological replicates per sample.Depending on the tissue availability, we tried to match FFPE and FF tissues with similar tumor cell percentage, although differences of up to 10% in tumor cell percentage could be present.The RNA concentration of the 84 samples (21 FFPE and 21 FF samples done in duplicates) was measured using the NanoDrop™ 2000 (Thermo scientific).The RNA integrity of the FF samples was evaluated with the 2100 Bioanalyzer (Agilent) using the RNA 6000 Nano LabChip, following the manufacturer's protocol. The RNA Integrity Number (RIN) of the samples was above 8 in all cases except for one sample (in both replicates RIN<6), which was excluded from the subsequent analysis together with the FFPE counterpart.GAGGCCGTGTACACCAAGAT, R ATCTGTTGCCGTAGGTCAGG, F GCAACGAGCTGGTGATCC, R AGAAGACGTCCAGGATGAGG. Because the four oligonucleotides were added at the same time in the reaction mix, four combinations of forward and reverse primers were possible, allowing the amplification of four different length fragments. 300 ng of RNA was used as a template for reverse transcription (30 min at 50°C), followed by activation of the HotStarTaq polymerase (15 min at 95°C), 10 cycles of PCR , 30 cycles of PCR  and final 7 min extension at 72°C. PCR products were visualized on 2% agarose e-gels (Invitrogen). GeneRuler™ 100 bp DNA Ladder  was used to determine product size. If, at the minimum, the PCR products of 123, 145 and 177 bp were visible on the gel, the quality of the RNA was considered acceptable for proceeding with the analysis.The RNA integrity of the FFPE samples was determined by amplifying different length fragments ), of the glucose-6-phosphate dehydrogenase (G6PD) gene using the One-Step RT-PCR kit (Qiagen). The following 5′-3′ primers were used to amplify G6PD (RefSeq ID: NM_000402): F The protocol above was developed in the diagnostic laboratory of the Pathology department at the Netherlands Cancer Institute and is routinely used for testing the quality of the RNA derived from FFPE material. In light of that, we opted for this approach, although Illumina would suggest a TaqMan Real Time PCR analysis for the FFPE RNA quality control th percentile (p95) of the probe intensities on the array, were used for evaluating the quality of the hybridization. Because the p95 value represents a measure of the fluorescence intensity distribution of the data, a low p95 (p95<2500) corresponds to poor-quality hybridizations. Detection p-values were computed using several hundred negative controls to determine gene expression detection limits.The Illumina Whole genome-DASL  assay was derived from the Human Cancer Panel DASL Assay All the microarray data are MIAME compliant and have been submitted to ArrayExpress (E-TABM-1081).Probes with an Illumina detection p-value below 0.05 in at least one of the samples (n = 90)  were kept for the analysis, corresponding in total to 21205 probes. We assumed that a significant detection of a probe in at least one sample is sufficient to show the detectability of the probe in the DASL assay.http://www.illumina.com/software/genomestudio_software.ilmn). For each normalization type we performed the calculation either with background subtraction or without background subtraction. We ran the analyses using all probes that passed the p-value filter or using a subset of them, termed the ‘informative probes’. For selecting the most informative probes we applied the following procedure: 1) we log2 transformed the normalized absolute intensity fluorescence value of each probe; 2) selected, for the FFPE and FF samples separately, the 20% highest variance probes, and 3) took the union of the two probe groups (FF and FFPE).Data were normalized across the arrays using three different normalization methods available in the Genomestudio software package - simple scaling normalization , cubic spline normalization and rank-invariant normalization  of FFPE samples and their own replicates, as well as for FF samples and their replicates. We also measured the correlation between paired FFPE and FF samples. In order to show the specificity of the correlation between replicates of the same sample and between the FFPE/FF paired samples, we also report the average correlation of each sample against all other non-replicate/non-pair samples. The box plots  display After applying the p-value filter of the probes, excluding Sample 69, we ended up with 21178 probes (instead of 21205 probes including sample 69). The number of informative probes equals 5444.http://www.r-project.org/). Heat maps were generated using the R statistics gplots package after simple scaling normalization and median centering of log2 transformed signal intensities on FFPE and FF samples separately. The distance between two samples in the heat map was calculated as one minus their Pearson correlation coefficient.Correlation analyses were performed in the statistical language R (http://linus.nci.nih.gov/BRB-ArrayTools.html) and MeV 4.5.1 (http://www.tm4.org/mev/). Significance Analysis of Microarrays (SAM) Unsupervised clustering analyses on FFPE samples were performed with BRB Array-Tools 3.8.1 (s) s<0.33 small correlation, 0.33<rs<0.67 medium correlation, rs>0.67 large correlation.Immunohistochemistry for ERα , PgR (Progesteron Receptor) and HER2 , and additional chromogenic in situ hybridization (CISH) for HER2 was performed and scored as described previously We used the Single Sample Predictor (SSP) developed by Hu and colleagues We mapped the 128 Affymetrix probes of the GGI signature We mapped the 70 genes previously reported http://www.biomart.org/): transcript length, number of transcripts, probe start position measured from the 3 and 5 prime ends of the transcript respectively and transcript G/C content. We performed a Gene Ontology enrichment analysis using the DAVID Gene Functional Classification Tool 6.7 For each probe that passed the p-value filter (n = 21178) the Pearson correlation coefficient between the log2 median centered intensities in the FFPE samples and in the FF counterparts  was calculated. After ranking the probes based on their correlation value, we selected the top 1000 most highly correlating probes and the bottom 1000 least correlating probes. All calculations were conducted in the statistical language R. We extracted the following features for each transcript from the BioMart database  from 20 Formalin Fixed Paraffin Embedded (FFPE) tissues and 20 matched Fresh Frozen (FF) tissues and hybridized to the DASL microarrays. Five FFPE samples and their FF counterparts were arrayed in duplicate . In summary 45 FFPE and 45 matched FF samples  plus oneAll 90 samples hybridized on the arrays passed the p95 and average signal quality control see . FFPE avWhen considering the raw log2 transformed intensities, FFPE samples showed a comparable distribution of signal intensities compared to the FF samples although the FFPE intensities were slightly lower on average, as was seen in the p95 analysis.The average number of probes detected above background  in FFPE samples was comparable with the number of probes detected in the matched FF samples (16518±718 in FFPE versus 17418±462 in FF samples) although lower on average  . Of the To assess the reproducibility of the WG-DASL assay for FFPE samples, we evaluated the gene expression profiles of 45 FFPE samples which included 15 samples in duplicate  and five samples in triplicate . Unsupervised hierarchical clustering of the FFPE samples only, with the 5444 most informative probes showed that, as expected, all replicates clustered together, , showingIn order to assess the performance of the WG-DASL assay we compared the gene expression profiles of the 20 matched FFPE and FF pairs.We used simple scaling normalized and median centered log2 transformed intensities. The reason for median centering the data is related to the difference in tissue types we are comparing. Besides an overall bias in the signal intensity, which is corrected for by the simple scaling normalization procedure, additional probe specific bias may exist. By median centering each probe separately for each group (i.e. FF and FFPE), part of this bias can be eliminated. In effect, the median intensity value of each probe in each group is shifted towards zero which corrects for consistently higher or lower signal in either group. A similar strategy has been used previously in a different context, to adjust for platform bias The heatmap of the distance measures of all FFPE and FF samples (n = 89) was generated using all probes that passed the p-value filter (n = 21178) . In thisAs can be seen in Next we performed differential gene expression analysis in order to determine whether the results generated with FFPE expression data were similar to the ones obtained with FF expression data. First we identified the two- three- and four-fold differentially expressed probes  in FFPE and FF paired samples. Fold difference was computed with respect to the average signal of all the samples from the same tissue-type. Signal intensities of the two biological replicates of each sample were averaged. The percentage overlap between FFPE and FF paired samples of two-, three- and four- fold up-regulated probes was similar, in contrast to the number of down-regulated probes that showed more variability . The aveAs a representation of common group versus group differential gene expression analysis we subsequently identified differentially expressed probes between histological grade 1 (n = 9) and histological grade 3 (n = 16) tumors see  for bothIn order to evaluate if the FFPE microarray data correlated with the immunohistochemical data for the single gene markers ERα , PgR (Progesteron Receptor) and HER2  we recovered the absolute normalized intensities of the corresponding genes, ESR1, PGR and ERBB2 from the array for the FFPE samples (n = 45). Three probes represent ERBB2 on the array; there is one probe specific for ESR1 and one for PGR.1 = 0.82, p-value <0.01; r2 = 0.57, p-value <0.01; r3 = 0.81, p-value <0.01) , the PGR array intensity also showed a significant correlation with the PgR IHC percentage . All three probes specific for ERBB2 significantly correlated with the HER2 CISH-IHC status although two of them showed a higher correlation coefficient (re <0.01) .We applied three gene known expression signatures developed from fresh frozen samples (the Hu molecular subtype signature Next we classified the samples in high risk or low risk based on the Genomic Grade Index (GGI), calculated as reported in When we performed an unsupervised hierarchical clustering (HCL) on all samples with the 110 DASL probes, the samples grouped in two main clusters, enriched in low GGI risk and high GGI risk samples . AlthougWhen we compared the GGI classification with the histological grade we found on average a good concordance, in agreement with the results from the clustering analysis . All fouLastly we calculated a 60-gene index as derivative of the 70 gene prognosis signature for the FFPE and FF pairs. The 60-gene index represents the 60 genes matching the 70 genes previously reported The 1000 probes showing the highest (top 1000) and lowest (bottom 1000) correlation between FF and FFPE samples were identified as described in the First we evaluated if there was a tendency for highly correlating probes to also be high intensity probes. As can be seen in In order to identify any specific biological characteristics associated with the most and least correlating probes, we evaluated different features of the corresponding transcripts. We evaluated the following features: 1) probe start position from the 3 prime or 5 prime ends of the transcript; 2) transcript length; 3) number of transcripts associated with the probe and G/C content. G/C content showed the strongest difference between the top 1000 (or 500) and bottom 1000 (or 500) probes . InteresSince better correlating transcripts between FFPE and FF matching samples are likely less affected by the degradation, we wanted to evaluate if a difference in detectability could be related to the biological function of the transcript. Therefore, we performed a Gene Ontology (GO) analysis to determine whether the top and bottom correlating transcripts differed in terms of their functional annotation as represented by the three classes in the Gene Ontology: biological process, molecular function or cellular component categories. The top 1000 probes showed enrichment in biological categories mainly related to the cell cycle such as cell cycle phase, M phase, mitosis, nuclear division. The bottom 1000 probes revealed, on the other hand, a broader spectrum of biological categories: DNA repair , RNA processing  or transport .Gene expression profiling of FFPE samples represents great potential for translational cancer research. Unfortunately this type of analysis has proven to be problematic because of the poor quality of RNA extracted from FFPE. In this study we showed that expression data with sufficient quality for further genomic analysis can be generated from FFPE material using a whole genome expression assay specifically designed for degraded RNA, the DASL Whole Genome assay.In our analysis we selected 20 FFPE blocks from 2008 from a single pathology department, in order to reduce the chance of failure due to the age of the tissue block or differing tissue processing methods. Block age, RNA extraction method and optimal concentration for a successful DASL assay, were previously evaluated There is a high reproducibility of FFPE gene expression profiles using 20 biological and technical replicates derived from the FFPE samples as was evident from unsupervised cluster analysis. The Pearson correlation between biological replicates was greater than 0.90, comparable to the correlation coefficient previously shown for technical replicates DASL array data and Immunohistochemical data of prognostic markers used in the clinical setting  significantly correlated with the corresponding gene expression values when we considered the FFPE samples.Different approaches used for comparing matched FFPE and FF tissues showed on average a good degree of comparability. Direct comparisons of paired FFPE and FF gene expression profiles considering a subset of most informative probes and using normalized, median centered data, yielded an average Pearson correlation of 0.7, in accordance with what was previously reported with the same assay We observed a good overlap (>50% relative to the FFPE samples) between differentially expressed genes detected in FFPE and FF pairs, using the histological grade (grade 1 versus grade 3) as the class label and this overlap tended to increase when considering higher fold changes (from two until eight fold). This finding is in accordance with previously reported results with a different assay With the application of three known gene signatures developed from fresh frozen material on our FFPE series using proper adjustments, we showed that it is possible to extract meaningful biological information from partially degraded samples, with respect to fresh frozen intact samples treated as reference. All FFPE and FF samples, with the exception of one case, were predicted to have the same molecular subtypes. The majority of the pairs were classified as either high risk or low risk over all FFPE and FF replicates using the DASL GGI signature. Although some discordance was observed when we applied the DASL GGI classifier, it is interesting to that, except for one case, all samples clustered into two main clusters, in accordance with their expression profile risk.2 = 0.94) and, most intriguing, the prognosis prediction made using 60 DASL gene index showed a trend in agreement with the prediction of the diagnostic test MammaPrint In the 70-gene analysis the degree of concordance between matched FFPE and FF samples was high . In addition, most likely, real differences exist between FFPE and FF results and these differences are due to the different protocols used. However, expression profiles are highly consistent when using the same protocol (>0.9 Pearson correlation). Because of this high within-protocol consistency, FFPE material could be used to discover new signatures and these could readily be applied to new FFPE samples. What this study shows is that using proper normalization methods, signatures derived from FF material with sufficient signal can be accurately translated to an FFPE derived signature. This implies that “the prognostic signal” is present in both the FF and the FFPE material and as such, the reverse must also be true. That is, a signature discovered in FFPE material can be translated to its FF counterpart when the appropriate normalization steps are taken into account. This opens up great opportunities given the large amounts of FFPE stored material and the associated clinical data.When we investigated the biological characteristics associated with the top and bottom correlating transcripts (between FFPE and FF matched samples) the G/C content showed to be higher in the top correlating genes. Our finding could suggest that a higher GC content is associated with a better detectability in FFPE material and therefore to mRNA stability. Although this remains a hypothesis, it seems to be consistent with what previously observed that most of the mRNAs that harbor coding region instability elements happen to be GC-poor Interestingly the top correlating transcripts showed enrichment in Gene Ontology biological categories related to cell cycle processes. Considering the idea proposed in Taken together the results of this investigation demonstrate that the DASL assay, which was specifically designed for partially degraded RNA from FFPE material, combined with a proper way of data processing, allows one to obtain reproducible and usable expression data from FFPE material. While the proposed normalization procedures compensate, for a large part, for the systematic bias between FFPE and FF material, a biologically induced difference between the two is evident. Particularly the function and the GC-content of a particular gene appear to play a role in the stability of the mRNA of that gene in FFPE material. Future studies making use of FFPE material should carefully consider these factors and take great care to apply proper normalization procedures as this can have a big impact on the results. FFPE gene expression data contains biological information comparable to the data generated from the intact counterpart that make them a valuable and useful resource for prognostic risk assessment and other genome wide analyses.Figure S1Box plots of the Pearson correlation coefficient between FF replicates (A), FFPE replicates (B) and FFPE/FF pairs (C) using three normalization procedures: simple scaling (ss), rank-invariant (ri) and cubic spline (cs) and carried out for either with background correction (bgc) or without background correction (nobgc). The box plots were generated using all probes that passed the p-value filtering  or using the informative probes (highvar). The number of informative probes using ss normalization without bgc is equal to 5480, with bgc is equal to 5284. The number of informative probes using ri normalization without bgc is equal to 6351, with bgc is equal to 6323. The number of informative probes using cs normalization without bgc is equal to 5388, with bgc is equal to 5664. In Panels A, B we report the correlation distribution of the comparison between real replicates (replicates) and between non-replicate samples (random_replicates). In Panel C we report the correlation distribution of the comparison between real pairs (pairs) and between non-paired samples (random_pairs). The circles in the graph are outlier samples indicated with the array_ID (see y_ID see . The leg(TIF)Click here for additional data file.Figure S2Number of probes on the array with negative values (na) after application of different normalization methods. The x-axis displays the normalization method  and the dataset used . The y-axis represents the number of probes. The box plots represent the distribution of the number of probes with negative values for each option evaluated. If present, outlier samples are indicated with a black circle. The total number of probes in the dataset is represented by the blue circles.(TIF)Click here for additional data file.Figure S3Quality control of the DASL gene expression data. (A) 95th percentile of the fluorescence intensity in FFPE and FF paired samples. On the x-axis we report the Pair_ID (1-20) and on the y-axis the 95th percentile (P95) of the fluorescence intensity. (B) Number of detected probes above background in FFPE and FF paired samples. On the x-axis we report the Pair_ID (1-20) and on the y-axis the number of probes significantly detected above background .(TIF)Click here for additional data file.Figure S4Heat maps of the distance measures (1-Pearson correlation coefficient) of the FFPE and FF samples (n = 89) using the informative probes (n = 5444). (A) Heat map using median centered log2 normalized data. Displayed are the distance measures of the FFPE and FF samples using all probes for calculating the distance sample by sample. Data are normalized with a simple scaling normalization without background correction and then median centered per probe, separately for FFPE and FF samples. Distances range from 0 (minimum distance) to 1.2 (maximum distance) as shown in the top left panel. The paired samples are indicated with a number from 1 to 20 and are color coded. (B) Heat map using non-median centered log2 normalized data. Displayed are the distance measures of the FFPE and FF samples using 5444 for calculating the distance sample by sample. Data are normalized with a simple scaling normalization without background correction. Distances range from 0 (minimum distance) to 0.6 (maximum distance) as shown in the top left panel.(TIF)Click here for additional data file.Figure S5Comparison between the Immunohistochemistry (IHC) marker status  and the gene intensity on the array  for the FFPE samples (n = 45). (A). Dot plot of ER/ESR1: The x-axis reports the IHC percentage of staining of ER; the y-axis reports the absolute intensity of the gene ESR1 on the array. (B) Dot plot of PgR/PGR: The x-axis reports the IHC percentage of staining of PgR; the y-axis reports the absolute intensity of the gene PGR on the array. (C) Dot plots of HER2/ERBB2: The x-axis reports the IHC-CISH (Chromogenic In Situ Hybridization) status of HER2; the y-axis reports the absolute intensity of the gene ERBB2 represented by three different probes on the array .(TIF)Click here for additional data file.Figure S6Unsupervised hierarchical clustering of all FFPE and FF samples (n = 89) with 110 DASL probes matching with the Genomic Grade Index (GGI) gene list. The clustering is performed using Pearson correlation and average linkage. The paired samples are color coded (pair_ID). Samples were classified as low risk or high risk based on the GGI score (see (TIF)Click here for additional data file.Figure S7Density distribution of genewise correlation between FF and FFPE paired samples versus average expression of all probes in FFPE (A) samples and in FF samples (B).(TIF)Click here for additional data file.Figure S8Distribution of genewise correlation between FFPE and FF paired samples. (A) Density against the Pearson correlation distribution of all probes in FFPE and FF paired samples. (B) Frequency against the Pearson correlation distribution of all probes in FFPE and FF paired samples.(TIF)Click here for additional data file.Figure S9Expression distribution of the top and bottom 500 correlating probes in FFPE samples (A) and in FF samples (B). Density distribution of top 500, bottom 500 and all probes are displayed. Similar plots were obtained using the top 1000 and bottom 1000 probes (data not shown).(TIF)Click here for additional data file.


A central goal of RNA sequencing (RNA-seq) experiments is to detect differentially expressed genes. In the ubiquitous negative binomial model for RNA-seq data, each gene is given a dispersion parameter, and correctly estimating these dispersion parameters is vital to detecting differential expression. Since the dispersions control the variances of the gene counts, underestimation may lead to false discovery, while overestimation may lower the rate of true detection. After briefly reviewing several popular dispersion estimation methods, this article describes a simulation study that compares them in terms of point estimation and the effect on the performance of tests for differential expression. The methods that maximize the test performance are the ones that use a moderate degree of dispersion shrinkage: the DSS, Tagwise wqCML, and Tagwise APL. In practical RNA-seq data analysis, we recommend using one of these moderate-shrinkage methods with the QLShrink test in QuasiSeq R package. In the last five years, groundbreaking new RNA sequencing (RNA-seq) technologies have considerably improved studies in genetics that previously relied on microarray technologies. RNA-seq technologies have several advantages over microarrays, including less noise, higher throughput, and the power to detect novel promoters, isoforms, allele-specific expression, and a wider range of expression levels. So it is not surprising that RNA-seq has become ubiquitous in experiments that investigate the regulation of gene expression across different conditions, such as levels of a treatment factor, genotypes, environmental conditions, and developmental stages.In a typical RNA-seq experiment, reverse transcription and fragmentation convert each RNA sample into a library of complementary DNA (cDNA) fragments, or tags. Next, a sequencing platform, such as the Illumina Genome Analyzer, Applied Biosystems SOLiD, Pacific Biosciences SMRT, or Roche 454 Life Sciences, amplifies and sequences the tags. After sequencing, a subsequence within each tag, called a read, is recorded. After the resulting collection of reads, or library, is assembled, the reads are mapped to genes in the original organism’s genome. The number of reads in a library mapped to a gene represents the relative abundance of that gene in the library. The investigator typically assembles all the read counts of multiple libraries into a table with rows to indicate genes and columns to indicate libraries. Please consult references by Oshlack, Robinson, and Young A central goal of RNA-seq experiments is to detect genes that are differentially expressed : i.e., ones for which the average number of reads differs significantly across treatment groups. Improving the detection of differentially expressed genes opens new ways to control organisms at the molecular level, advancing fields like agriculture engineering, personalized medicine, and the treatment of cancers, contributing to social welfare.Some of the most popular new statistical methods that detect differentially expressed genes from RNA-seq data rely on the negative binomial (NB) probability distribution. If a random variable, Cameron and Trivedi In an RNA-seq dataset, the number of reads, It is common practice to include in the model the library-wise normalization factors, With the above preliminaries taken care of, we now turn to the main issue of this article: the estimation of the dispersion parameters, First, we briefly review several popular dispersion estimation methods . We also touch on some popular tests for differential expression. Next, using a simulation study that draws fundamental information from real datasets, we compare the practical effectiveness of the methods in terms of the accuracy and precision of the point estimates and the effect on the performance of tests for differential expression. In the results and discussion sections, we discuss the distinguishing features of the most successful dispersion estimation methods.Studies by Wu, Wang, and Wu In RNA-seq data analysis, we could apply methods based on counts for each gene separately to estimate model parameters, such as the Quasi-Likelihood (QL) method reviewed below. However, in RNA-seq data, there are typically tens of thousands of genes, but only a few counts per gene with which to estimate gene-specific parameters, a typical example of a “large The QL method estimates a dispersion parameter independently for each gene. This method The MLE, The tagwise dispersion estimate, The QL method estimates tens of thousands of dispersion parameters, but uses only a few read counts to compute each estimate. More sophisticated techniques make use of a larger number of read counts to estimate each As explained by Robinson and Smyth The wqCML method is implemented in the R package, edgeR, designed by Robinson, McCarthy, and Smyth and available at bioconductor.org The wqCML method only applies to completely randomized designs with two treatment groups. In the APL method, McCarthy, Chen, and Smyth To estimate the Like the wqCML and APL methods, the DESeq method by Anders and Huber Anders and Huber As with the APL method, there are three variations on the DESeq method giving different ways to shrink the dispersions. The no-shrinkage variation transforms the estimated raw variance parameters directly into the estimated gene-wise dispersions without any shrinkage. The “Trended” variation performs a regression of the raw variance parameter estimates on the estimated means, and then computes the estimated dispersions from the fitted values on the trend.  The “Maximum” variation computes the maximum of each raw variance parameter estimate and its fitted value on the trend and then computes the dispersion estimates from these maxima. This last method of dispersion shrinkage is conservative, allowing overestimation of the dispersions, but guarding against underestimation in order to avoid false positives in tests for differential expression. The DESeq method is implemented in the R package, DESeq, available at bioconductor.org.The Bayesian paradigm more naturally accommodates the notions of “borrowing information” and “shrinkage” than the Frequentist paradigm. Hence, the DSS method by Wu, Wang, and Wu Here, the gamma distribution is parameterized in terms of its mean, With a model specified and all the parameters estimated, we can test for the differential expression of genes. The simulation study in the next section uses the following five recently-proposed testing methods. The first two tests, found in the edgeR and DESeq packages, extend Fisher’s exact test to data following negative binomial distribution. The next three tests, developed by Lund, Nettleton, McCarthy, and Smyth This article presents a simulation study that puts the featured dispersion estimation methods to the test. We first generate pseudo-datasets for which the true negative binomial dispersion parameters and truly differentially expressed genes are known. Then, we apply the featured dispersion estimation methods and testing methods to the pseudo-data, compare the results to the truth, and measure the performance of the dispersion estimation methods in terms of point estimation and performance in testing for differential expression.http://bowtie-bio.sourceforge.net/recount/The simulation study uses two real RNA-seq datasets to generate the pseudo-datasets. The “Pickrell dataset” (Gene Expression Omnibus accession number GSE19480) comes from a study by Pickrell, Marioni, Pai, Degner, Engelhardt, et al. The top two panels of For each one of the real datasets (Hammer or Pickrell), we first compute the two quantities, Randomly select 10,000 genes from one of the real datasets (Hammer or Pickrell) without replacement. The corresponding 10,000 pairs of Randomly select simulated gene Set the log fold change across treatment levels, Compute the true mean expression level Randomly draw the pseudo-count of each simulated gene Each gene in the pseudo-dataset should have at least one read to be included in the following analysis. Hence, if the pseudo-counts of simulated gene Note that our method of choosing true mean and dispersion pairs builds an empirical dispersion-mean relationship into the simulated data. As Some pseudo-datasets were generated from the Hammer dataset, while others were generated from the Pickrell dataset. In addition, the number of libraries per treatment group varied from pseudo-dataset to pseudo-dataset. Hence, six “simulation settings”, given in Our simulation procedure was configured such that within each pseudo-dataset, the library sizes do not vary systematically. So when analyzing the simulated data, it would be reasonable to set all the library-wise normalization factors, Anders and Huber’s method assigns each library-wise normalization factor, The methods implemented in edgeR With 30 pseudo-datasets generated for each of 6 simulation settings, we apply each dispersion estimation method to each pseudo-dataset, and we use the dispersion estimates to test for the differential expression of genes. We use the true dispersions, the knowledge of which genes are truly differentially expressed, the dispersion estimates, and the test results to compare the dispersion estimation methods. We assess the quality of the methods in terms of the accuracy and precision of point estimation and performance in tests for differential expression.The overall quality of any point estimator can be measured in terms of its mean squared error. For each pseudo-dataset and each dispersion estimation method, we calculate the mean squared error of the transformed estimated dispersions,Here, the Wu, Wang, and Wu Overall, the results for the Hammer-generated pseudo-datasets (simulation settings IV-VI) naturally group the dispersion estimation methods into three categories. The first group includes Maximum DESeq and the no-shrinkage DESeq methods. These methods produce the largest MSE, which is not surprising because the Maximum DESeq method is conservative and designed to obtain larger dispersion and because the no-shrinkage DESeq method applies a naive dispersion estimation technique for each gene independently. The next category includes the QL method alone, which performs better than the Maximum and no-shrinkage DESeq methods, but worse than others when the number of libraries is small (simulation setting IV). The other methods all perform similarly and form a group of MSE-best methods. This demonstrates that shrinkage indeed helps improve the point estimators by borrowing information across genes. However, too much shrinkage is detrimental, as the Common methods perform slightly worse than their Trended and Tagwise counterparts.Parameter estimation is more challenging for the Pickrell-generated pseudo-datasets (simulation settings I–III) than the Hammer-generated pseudo-datasets (simulation settings IV–VI) because the counts are lower, dispersion is larger in general , and disAlthough useful for determining the overall quality of a point estimator, the MSE heuristic is only a single scalar computed for an entire dataset. It is also important to consider the way that estimation error varies with the magnitude of the true dispersions. Wu, Wang, and Wu According to Since the detection of differentially expressed genes is the major goal of most RNA-seq experiments, it is vitally important to measure and compare the direct impact of the dispersion estimation methods on the detection of differentially expressed genes, which is why the pseudo-datasets are generated such that each simulated gene is known to be either differentially expressed or equivalently expressed. Using this knowledge and the p-values obtained from the tests for differential expression, a receiver operating characteristic (ROC) curve is constructed for each pseudo-dataset/test for differential expression/dispersion estimation method combination.An ROC curve is a graph of the true positive rate (TPR) of the detection of differentially expressed genes vs the false positive rate (FPR). In practice, we define FPR and TPR to be functions of the significance level, In this study, we use the area under each ROC curve (AUC) as a relative measure of the quality of a test, where a high AUC indicates relatively good test performance. Here, each AUC is computed only for FPR Overall, the three tests in the QuasiSeq package are less affected by dispersion estimation than the edgeR and DESeq exact tests. These tests introduce gene-wise quasi-likelihood dispersion parameters to the negative binomial model, and the new parameters absorb some of the variability that would otherwise manifest solely in the negative binomial dispersions. The practical upshot is that all three QuasiSeq tests are relatively robust under both noisy data and poor estimation of negative binomial dispersions. The QL test is an extreme case, with little change in its AUC boxplots across the dispersion estimation methods, because it does not apply any special constraints to the quasi-likelihood dispersions.  Unfortunately, the QL test also performs the worst among the five tests overall, making it a poor choice in practice despite its otherwise useful robustness. The QLSpline test is better than the QL test, and the QLShrink test is better still.The rankings of the dispersion estimation methods are similar among the edgeR exact test, the DESeq exact test, and QLShrink test. Specifically, the DSS, Tagwise wqCML, and Tagwise APL – i.e., the moderate-shrinkage methods – are the best. Not only do these dispersion estimation methods perform well relative to other methods, but they are also extremely close to the true dispersions in terms of AUC. Across all six simulation settings, when combined with moderate-shrinkage methods for dispersion, the best tests for differential expression are the edgeR exact test, the DESeq exact test, and the QLShrink test. These methods for testing perform roughly equally well, and they are better than other combinations of tests and dispersion estimation methods. In some cases, this difference in AUC between the two tiers of tests is dramatic. With the addition of the gene-wise quasi-likelihood dispersion parameter to the negative binomial model, the QLShrink test is more robust to changes in dispersion estimation method than the either of the exact tests, which is most noticeable for the Pickrell-generated pseudo-datasets (simulation settings I-III). In practice, we recommend using the QLShrink test because we expect the addition of quasi-likelihood dispersion parameters to make the QLShrink test more flexible than the edgeR and DESeq exact tests under departures from the negative binomial model.It is challenging to estimate negative binomial dispersions from RNA-seq data due to the “small These moderate shrinkage methods outperform the others in all five featured tests for differential expression. However, these tests do not perform equally well. The edgeR exact test, DESeq exact test, and QLShrink test outperform the other two. Furthermore, the performances of the edgeR and DESeq tests depend highly on the dispersion estimation method chosen, while the addition of a gene-wise quasi-likelihood dispersion parameter gives the QLShrink test extra robustness under the choice of dispersion estimation method. We expect this same flexibility to help the QLShrink test perform especially well under departures from the negative binomial model, so we recommend using the QLShrink method with either the DSS, Tagwise wqCML, or Tagwise APL method in practice.Interestingly, the ranking of dispersion estimation methods according to MSE is not the same as the ranking according to AUC. A notable example is the Maximum DESeq method, which performs poorly in terms of MSE, but often performs extremely well in tests for differential expression. This behavior may result from the intentional overestimation of the dispersions, which contributes to a high MSE, but guards against false positives. In addition, methods with similar MSE may have very different AUC. For example, the Trended APL and Tagwise APL methods yield similar MSEs in simulation setting V, but the Tagwise APL method performs much better than the Trended APL method in the edgeR test . We do nSimulation Code S1R code used to generate the pseudo-datasets and conduct the analyses. The software requirements can be found in (R)Click here for additional data file.


Non-negative matrix factorization is a useful tool for reducing the dimension of large datasets. This work considers simultaneous non-negative matrix factorization of multiple sources of data. In particular, we perform the first study that involves more than two datasets. We discuss the algorithmic issues required to convert the approach into a practical computational tool and apply the technique to new gene expression data quantifying the molecular changes in four tissue types due to different dosages of an experimental panPPAR agonist in mouse. This study is of interest in toxicology because, whilst PPARs form potential therapeutic targets for diabetes, it is known that they can induce serious side-effects. Our results show that the practical simultaneous non-negative matrix factorization developed here can add value to the data analysis. In particular, we find that factorizing the data as a single object allows us to distinguish between the four tissue types, but does not correctly reproduce the known dosage level groups. Applying our new approach, which treats the four tissue types as providing distinct, but related, datasets, we find that the dosage level groups are respected. The new algorithm then provides separate gene list orderings that can be studied for each tissue type, and compared with the ordering arising from the single factorization. We find that many of our conclusions can be corroborated with known biological behaviour, and others offer new insights into the toxicological effects. Overall, the algorithm shows promise for early detection of toxicity in the drug discovery process. The aim of this work is to highlight the usefulness of a recently proposed extension to the technique of non-negative matrix factorization (NMF) by demonstrating its promise for early detection of toxicity in the drug discovery process. In particular, we (a) show that any number of related datasets can be treated simultaneously with this approach, (b) deal with practical issues that arise when the algorithm is applied to real datasets, (c) demonstrate its use with a new large scale microrray dataset, and (d) interpret the results from a biological perspective.NMF seeks to represent a large complex dataset in terms of smaller factors. The name covers many algorithms. Each approximates a non-negative matrix as the product of two or more smaller non-negative matrices, by attempting to minimise some objective function. Lee and Seung We analyse gene expression data describing the molecular changes in four tissue types due to different dosages of an experimental pan-peroxisome proliferator-activated receptor (pan-PPAR) agonist PPM-201, provided by Plexxikon. PPARs have attracted great interest as potential therapeutic targets for diabetes The material is organised as follows. In Section we describe the simultaneous NMF algorithm and outline our approach for using the output to order and cluster a dataset. Section describes the mouse microarray data, and the NMF results that arise when we treat it as a single dataset are given in Section . This is followed in Section by the analysis of the data split into four datasets corresponding to the known tissue types; liver, kidney, heart and skeletal muscle. In Section we compare the gene clusters from Sections and , and Section discusses the results. Given The values in Having iterated up to some stopping criterion and produced the factorizations, we use them to bi-cluster the data. Each sample is assigned to the cluster for which it has the largest value in the gene cluster and vice versa. In reordering the data for easy visualisation we organise the rows and columns by cluster number (assigned arbitrarily) and sort the elements within each cluster from the appropriate sample/gene set, with the largest value at the bottom/right of that cluster. Given that the second factor is common to all the factorizations, it produces a matching ordering of the columns of the data.Because the result depends on the choice of initial condition, and because the choice of We apply these techniques to mouse gene expression data quantifying changes in four different tissue types following administration of different dosages  of an experimental pan-PPAR agonist. The study design and clinical chemistry results are summarised in Nine wild type mice (strain: C57BL/6J) were randomly divided into three groups; - Group-I, II and III. PPM-201 in the vehicle base was administered daily for 14 days at 6 mg/kg body weight dose rate to each mouse in Group-II and at 20 mg/kg body weight dose rate to each mouse in Group-III while the mice in Group-I received only the vehicle base. On 15th day, the mice were sacrificed to harvest blood, heart, skeletal muscle, liver and kidney tissues for clinical chemistry, microarray and histopathology analysis. In the clinical chemistry analysis, alanine aminotransferase , aspartate aminotransferase , creatinine kinase , blood urea nitrogen , creatinine (The dataset has been approved by the GEO curators and assigned the accession number GSE31561.The in vivo procedures undertaken during the course of this study (Ref: CXR0631) were subject to the provisions of the United Kingdom Animals (Scientific Procedures) Act 1986. The study was approved by the CXR Biosciences Local Ethics Committee and complied with all applicable sections of the Act and the associated Codes of Practice for the Housing and Care of Animals used in Scientific Procedures and the Humane Killing of Animals under Schedule 1 to the Act, issued under section of the Act.First, the samples are treated as a single dataset, with thirty six samples and 45037 genes, hence the data matrix Based on Because we know the origin of the samples, we can confirm that the algorithm has put the heart samples in cluster one, the skeletal muscle samples in cluster two, the liver samples in cluster three, and the kidney samples in cluster four. The exact ordering of the samples is shown in Given that the factorization has been performed for The test in Section indicates that the basic NMF factorization approach can deliver biologically meaningful results—separating the twelve samples by tissue type. But the failure to order correctly within tissue type according to dosage motivates the use of the multiple dataset generalization introduced in Section , where the four tissue types are treated as separate sources of information across a common set of mice. Intuitively, we would expect to add value to the data analysis by building known biology into the algorithm in this way. In this section, we therefore factorize the four new datasets simultaneously. This is similar to the test in Section in the sense that it produces a single ordering for the mice, but it has the potential to add extra information by providing four different, tissue-level, gene orderings. We thus have The objective function and the consensus measurements are shown in Given these measurements we consider the four-way simultaneous factorization for Our aim now is to test the results from the novel multi-way NMF algorithm used in Section in order to see whether they (a) show consistency and (b) add value to the results in Section from standard NMF. We know that the four simultaneously factorized datasets correspond to the four clusters of samples that were discovered in an unsupervised manner from the single factorization of the full dataset. It could therefore be conjectured that the most influential genes in the first factorization will appear as influential genes in the four-way simultaneous factorization for that dataset, but less so for the other datasets.Our comparisons involve four reference sets. For illustration, we chose an arbitrary threshold of one hundred; that is, we consider the top one hundred most influential genes from the four clusters in the first factorization shown in We would like to demonstrate the utility of the factorization method by using the gene clusters obtained in our analysis to understand tissue specific effects of the experimental drug, PPM-201. Of course, we are not claiming that this is an exhaustive analysis of the effects of PPM-201. We analysed the gene clusters for pathways enrichment and gene ontology enrichment using DAVID The common genes between the top one hundred most influential probe-sets in the four tissue specific clusters and the top one hundred most influential probe-sets in the clusters formed by 4-way simultaneous factorization of the split dataset were also analysed for enrichment of pathways, gene ontology and toxicity functions using DAVID and IPA. The factorization and reordering of the dataset as a whole set  successfobe-sets  and 8 fr cluster  with mos cluster  pathway,Splitting the dataset into four on the basis of tissue types and simultaneous non-negative factorization of them gave us the added reassurance of clustering the samples according to the dosage groups . The clupathways . ACTC1, Interestingly, 49 probe-sets in the B3 genes . This suunctions  which haWe have demonstrated that multi-way simultaneous nonnegative matrix factorization can be usefully applied to the case of multiple datasets—here, for what we believe to be the first time, more than two large scale matrices were treated. The results were shown to be consistent with, and to add value to, standard nonnegative matrix factorization of the whole dataset.In summarizing our biological findings, we first note that the roles of the three different isoforms of PPARs - PPAR-Overall, our aim here is to establish a proof of principle for the approach of simultaneously analysing multiple, related large datasets. We therefore focused on a dataset where clear-cut validation is possible. However, we note that the technique is very general, and therefore opens up many new opportunities in data-driven computational biology. In particular, it can be applied to heterogeneous sources of data; for example, generated by different laboratories or experimental methodologies. We are currently pursuing this approach in the study of colon cancer.


Using hybrid approach for gene selection and classification is common as results obtained are generally better than performing the two tasks independently. Yet, for some microarray datasets, both classification accuracy and stability of gene sets obtained still have rooms for improvement. This may be due to the presence of samples with wrong class labels (i.e. outliers). Outlier detection algorithms proposed so far are either not suitable for microarray data, or only solve the outlier detection problem on their own.et al., 2006) based on same synthetic datasets. MFMW-outlier gave better average precision and recall values on three different settings. Lastly, artificially flipped microarray datasets were created by removing our detected outliers and flipping some of the remaining samples' labels. Almost all the ‘wrong’  samples were detected, suggesting that MFMW-outlier was sufficiently powerful to detect outliers in high-dimensional microarray datasets.We tackle the outlier detection problem based on a previously proposed Multiple-Filter-Multiple-Wrapper (MFMW) model, which was demonstrated to yield promising results when compared to other hybrid approaches . To incorporate outlier detection and overcome limitations of the existing MFMW model, three new features are introduced in our proposed MFMW-outlier approach: 1) an unbiased external Leave-One-Out Cross-Validation framework is developed to replace internal cross-validation in the previous MFMW model; 2) wrongly labeled samples are identified within the MFMW-outlier model; and 3) a stable set of genes is selected using an L1-norm SVM that removes any redundant genes present. Six binary-class microarray datasets were tested. Comparing with outlier detection studies on the same datasets, MFMW-outlier could detect all the outliers found in the original paper , and the genes selected after outlier removal were proven to have biological relevance. We also compared MFMW-outlier with PRAPIV (Zhang  Classification is one of the major goals in microarray data analysis Gene selection is another major goal in microarray data analysis; it is not only necessary for efficient classification, but also important for biomarker identification et al. applied SVM on microarray datasets with reduced set of genes. Samples which have been consistently misclassified in all tests are identified as suspects et al. proposed a method based on Akaike's Information Criterion to detect outliers in the colon data et al., outliers are detected using support vector machine (SVM) in a re-validation framework et al. introduced the misclassification probability which is estimated for each sample in the training set et al.et al. modified the CL-stability approach. Their goal was to detect outlier samples and automatically correct them, and their proposed method was called Fast Outlier Samples Detection (FOSD) Outlier detection is a process to search for samples that do not obey the general rules of the majority portion of the data of the same class. Many outlier detection algorithms have been proposed, yet most of them The aims in all the above studies are to design an outlier detection algorithm on their own. A distinctive feature of our proposed MFMW-outlier framework is that outlier detection is integrated within a proposed hybrid approach. This ‘three-in-one’ approach performs gene selection, classification and outlier detection simultaneously, which is particularly suitable for high-dimensional microarray datasets.Our aim was to identify any wrongly labelled samples present in a high dimensionality data, such as microarray. Six benchmark binary-class datasets on cancers were selected for evaluation using the algorithm proposed. With the help of synthetic datasets, the effectiveness of MFMW-outlier could be demonstrated despite the absence of ground truth information for which samples are outliers.Microarray datasets. The six chosen binary-class datasets, all generated using Affymetrix chips, were: LEU, COL, BRE, LYM, PROS and LUNG. They were pre-processed according to the instructions published in the original paper. In addition, each sample was normalized to have mean zero and unit variance. Synthetic datasets. Synthetic datasets were more reliable as the class labels for all samples were known. Experimental results obtained from these datasets could therefore reflect the true performance of the proposed algorithm. In a recent study on detecting outliers in microarray data et al.Our proposed integrated MFMW-outlier approach is built on top of the MFMW model proposed in The optimal number of genes selected in the multiple-wrapper step in MFMW-outlier is determined using an L1-norm SVM, whereas those selected in earlier approach was based on a threshold, i.e. a fixed number of genes.The entire gene selection and classifier procedure of MFMW-outlier is built within a fully unbiased LOOCV framework  in our present approach, as opposed to applying LOOCV only to the classification part only in Identifying wrongly labelled samples is made possible through the use of external LOOCV, as each sample is left out completely from start, including the gene selection process.Since the present approach is based on the MFMW model developed in . Let The same three filters were used as in MFMW. One hundred and fifty genes are selected by each filter and the three gene lists are then combined by taking their union.The same three wrappers were used in MFMW. Please refer to  Suppose the set of samples In N-MFMW model, the number of genes selected at each level of the wrapper part is ‘optimized’ using L1-SVM. Instead of taking one/all gene(s) at the same level, the number of genes selected is based on the The entire N-MFMW process is repeated for every sample We now propose to incorporate outlier detection into the N-MFMW model with external LOOCV, as shown in Finally, the LOOCV accuracy and the gene stability measure FOO can be computed based on a reduced dataset that has been cleansed of outliers. The set of genes can then be ranked according to their FOO scores from the largest to smallest, with a cut-off threshold set for FOO values. The complete algorithm that integrates all three components of gene selection, optimization of classification accuracy and outlier detection, is given below.MFMW-outlier – Outlier Detection under N-MFMW with External LOOCV:Start with Define Training set Apply ‘Multiple Filters’ algorithm to kM to test sample Apply Repeat steps 2–4 for all If there are no samples marked as outliers, proceed to the next step, otherwise remove all samples Output:LOOCV accuracy based on test results for each FOO of genes in Set of outlier samples.k-NN will perform much faster than neural network, genetic algorithms or other classifiers that require optimization of lots of parameters.The proposed MFMW-outlier is a significant modification of the MFMW model proposed in An external LOOCV is then performed to evaluate the performance of the classifier. More importantly, the use of external LOOCV in the N-MFMW framework allows a natural outlier detection and gene set stability evaluation. LOOCV is chosen instead of other model estimation methods (like bootstrapping) as a score can be assigned to each sample specifically for the determination of whether it is an outlier or not. This score can be easily obtained if only one sample is left out each time. When multiple samples are left out , each time certain portions of samples are used in testing, and the final score for outlier detection will be dependent on the results from several runs. There are two disadvantages of doing so: 1) including several samples together as a test set will obscure the signal of an outlier (if any); 2) additional steps are required to combine the scores from different runs.Though cross validation should be normally performed to evaluate the performance of a classifier, in previous microarray studies, the most common misuse of validation involves selecting genes using the full dataset, called internal cross-validation, rather than performing gene selection from scratch within each loop of the cross-validation We first applied MFMW-outlier on the six microarray datasets. These selected datasets are the most ‘well-studied’ ones in the microarray community. The first three datasets used in our current study LUNG data, no outlier was detected.LEU data, the only outlier being detected in every algorithm is Sample 66. COL and BRE.Comparison with other proposed outlier detection methodologies on microarray datasets were made COL and BRE datasets.In both At the end of MFMW-outlier, a set of stable genes was selected. LEUCST3 is related to AML MGST3 was linked indirectly with GSTM1 according to BioGraph By BioGraph The relationship of MYB with acute leukemias has been shown COLVIP has been shown to regulate the growth of colonic adenocaricinoma cells BREDSC3 expression was down-regulated in more than half of breast cancers ETV1, also known as ER81, was shown to be collaborated with the oncoprotein HER2/Neu to activate Smad7 transcription in breast cancer cell lines LYMTransgenic mice overexpressing HMGA1 gene was shown to develop natural killer cell lymphomas The updated annotation for JTV-1 is AIMP2 gene, which is shown to have protein interaction with RARS gene, which has direct relationship to mir16-1 by BioGraph PROSHPN is a potentially important candidate gene involved in prostate cancer susceptibility NELL2 mRNA expression was predominantly localized in basal cells of the epithelium in situ hybridization analysis of hyperplastic prostate specimens LUNGAccording to BioGraph By Biograph Similarly, SERPINH1 is interacting with CD9 gene, which is a disease causing gene of non-small cell lung carcinoma Besides microarray datasets, MFMW-outlier was also evaluated upon using three synthetic datasets  50 times. We report how accurate we were able to detect the six artificially labelled samples, with comparison to PRAPIV It's almost impossible to know which samples are wrongly labelled in typical microarray datasets. According to microarray studies on Colon cancer N outliers, denote p as precision and r as recall, we expect that there are COL and 0.41 FP and 0.50 FN in BRE, which corroborate with the results shown in COL and 0.58 FP and 0.41 FN in BRE, which is over optimistic when compared to the real values shown in If an algorithm reports n = 200 genes for each filter. We are also interested in how classification performance changes when different number of filters and wrappers are used in our MFMW-outlier model. Consider varying the number of filters in the model. If too few filters are used, inadequate genes of dissimilar characteristics are selected. If too many filters are used, some of the selected genes across the different gene lists are redundant in nature. To investigate how many filters should be included, we check if the biological significant genes discussed in the previous session are all present in all gene lists produced by different filters. The gene lists are obtained by setting The following conclusions can be drawn from results in SNR and TS select almost the same set of genes, except for CST3 in LEU, and hence if SNR is not included as one of the filter, this gene would be missed outSimilar for the case of NELL2 in PROS, and so TS is an important filter.There are two genes: ETV1 in BRE and HMGA1 in LYM which are only selected by AUC, but not by the other two filters. This suggests that AUC is an important filter.Next, we investigate if the number of wrappers  used in the MFMW-outlier model would result in variations in the classification performance of the model. By the basic idea of the multiple wrapper approach, there has to be more than two wrappers. On the other hand, using too many wrappers of similar nature does not provide more information for the decision process. Therefore, the number of wrappers employed in the following experiment varies from two to four. In addition to the three wrappers we used, the extra wrapper chosen is Naïve Bayes (NB). This wrapper is selected because it is one of the most popular classifiers used in microarray experiments. In each experiment, 200 genes are first selected by each of the three filters. Experiments with different number of wrappers are then performed on the LYM dataset.4C2 = 6 experiments. For each experiment, we select the final gene sets with the smallest values of ‘k-NN’, ‘WV+SVM’ and ‘WV+NB’. All three models use an equally small set of eight genes for perfect LOOCV performance. Although the two models ‘k-NN+SVM’ and ‘k-NN+NB’ both select eight genes in the final gene set, the variations in terms of selected genes are too large to allow one to decide on which gene set should be chosen finally. Therefore they are not selected as the best models.Using genes selected by three filters, 4C3 = 4 experiments built using three wrappers. The best set of wrappers of size three is ‘WV+k-NN+SVM’, which is the same as the one we presented earlier. Also, MFMW-outlier models built by using four wrappers are not as good as that built by using three wrappers. Hence, for the LYM dataset, using three wrappers for the MFMW-outlier is most appropriate. Similar results (details not shown here) have been obtained for other datasets. Note that the size of gene set selected using MFMW-outlier based on three wrappers is six, which is smaller than that  selected using two wrappers. As a smaller set of genes is able to give the same biased LOOCV accuracy, we would recommend using the three wrappers ‘WV+k-NN+SVM’ in MFMW-outlier model.Using genes selected by three filters, The proposed method can be extended to the case of multiclass setting involving datasets with more than two classes. The straight forward way is to build a classification model for each class that separates this particular class from the remaining classes. This is a one-versus-all (OVA) classification approach. Another possibility is to train the classification model for every pair of classes in the multiclass dataset. This is a one-versus-one (OVO) classification approach. The challenge for the latter method is that an outlier detected in an OVO model may not be a true outlier, as it may be a sample that belongs to a class other than the two classes used for building the OVO model. We would therefore recommend building MFMW-outlier in an OVA manner. Other than this, integrating MFMW-outlier with other multiclass methods like error-correcting-codes approach will require more efforts, as such approaches require the design of codes for classification beyond the usage of just binary-class classifiers.The main contribution of this paper is to integrate outlier detection into an existing hybrid model, while making two significant modifications to the hybrid model to address issues on optimal gene selection and the problem of bias in internal cross validation. The new ‘three-in-one’ MFMW-outlier model can handle gene selection, sample classification and outlier detection simultaneously. MFMW-outlier was evaluated using both microarray and synthetic datasets. All results showed that we were able to detect the outlying samples present in high dimensional data. When comparing with ‘ground truth’ obtained from original paper, we were able to detect all the mislabelled samples, whereas other methods may result in some FP and FN. The fact that the selected genes were biologically confirmed was a strong indication that we have removed the wrong samples correctly.http://people.pcbi.upenn.edu/~yyee/MFMW-outlier/.To conclude, we have demonstrated the feasibility of integrating outlier detection into a hybrid model. The model is shown to have very high robustness with respect to the number of outliers in the dataset. We have implemented the proposed algorithm in MATLAB® and the software is available at


RNAseq and microarray methods are frequently used to measure gene expression level. While similar in purpose, there are fundamental differences between the two technologies. Here, we present the largest comparative study between microarray and RNAseq methods to date using The Cancer Genome Atlas (TCGA) data. We found high correlations between expression data obtained from the Affymetrix one-channel microarray and RNAseq (Spearman correlations coefficients of ∼0.8). We also observed that the low abundance genes had poorer correlations between microarray and RNAseq data than high abundance genes. As expected, due to measurement and normalization differences, Agilent two-channel microarray and RNAseq data were poorly correlated (Spearman correlations coefficients of only ∼0.2). By examining the differentially expressed genes between tumor and normal samples we observed reasonable concordance in directionality between Agilent two-channel microarray and RNAseq data, although a small group of genes were found to have expression changes reported in opposite directions using these two technologies. Overall, RNAseq produces comparable results to microarray technologies in term of expression profiling. The RNAseq normalization methods RPKM and RSEM produce similar results on the gene level and reasonably concordant results on the exon level. Longer exons tended to have better concordance between the two normalization methods than shorter exons. Gene expression analysis is essential for biomedical research. Expression profiling is the simultaneous measurement of the cellular concentration of different messenger RNAs. Microarrays have been the most popular high-throughput gene expression profiling technology for several decades. Recently, the introduction of RNAseq technology has had a revolutionary impact on the field of expression research. RNAseq refers to the use of next-generation sequencing (NGS) technologies to sequence cDNA in order to get information about a sample's RNA content. Compared to the microarray technology, the RNAseq method offers several distinct advantages. First, the detection range of RNAseq is not limited to a set of predetermined probes as with the microarray technology, so RNAseq is capable of identifying new genes. Second, the resolution of a microarray is limited to the gene level for most arrays and the exon level for specially designed exon arrays. On the other hand, RNAseq can detect expression at the gene, exon, transcript, and coding DNA sequence (CDS) levels. Finally and most importantly, RNAseq can detect structural variants such as alternative splicing and gene fusion. With the maturity of NGS technologies, the price of RNAseq has become comparable to microarrays. The competitive price and additional genomic information make RNAseq an attractive alternative technology for expression profiling. Some researchers have predicted the inevitable replacement of microarray by RNAseq There are two standard microarray detection paradigms: one-channel and two-channel, also known as one-color and two-color detection. Two-channel microarrays are hybridized with cDNA from a pair of samples to be compared, with one sample labelled with fluorescent Cy3 at a fluorescence emission wavelength of 570 nm (green), and the other with Cy5 at a fluorescence emission wavelength of 670 nm (red). After labelling, the two samples are mixed and hybridized to one microarray. The microarray is then scanned for fluorescence intensity. Gene expression identified from a two channel microarray is often represented as a ratio of Cy3/Cy5. In one-channel microarrays, usually only Cy3 is used and only a single sample is hybridized to one microarray. Thus, two-channel microarrays do not truly reflect the abundance levels of a gene transcript but rather the relative abundance between two samples. A study The Microarray Quality Control (MAQC) project has shown that there is a high level of intra-platform consistency across test sites and inter-platform concordance in terms of genes identified as differentially expressed by microarray methods TCGA is a massive, comprehensive, and collaborative project to catalogue genomic data for over 20 types of cancers by the National Cancer Institute (NCI), the National Human Genome Research Institute (NHGRI), and 27 institutes and centers of the National Institute of Health (NIH). Gene expression profiling is one of the major components of genomic data collected by TCGA. However, use of the gene expression profiling data in TCGA is complicated by the fact the gene expression data reported was obtained through a mixture of microarray and RNAseq technologies. Fortunately, many of TCGA's samples have had gene expression quantified using both technologies. This provided a good opportunity to study the repeatability and concordance of gene expression profiling between the two technologies on a large scale.From TCGA, we collected expression data on 4747 samples over 14 cancer types . Out of 4747 samples, 2250 were expression profiled using Agilent G450A_07 arrays, 1269 were expression profiled using Affymetrix HT_U133 arrays, and 4064 were expression profiled using RNAseq. The overlap between Agilent and Affymetrix arrays was 1134 samples, the overlap between the Agilent array and RNAseq was 1662 samples, and the overlap between Affymetrix array and RNAseq was 699 samples. 9(C/NL), where C is the number of reads mapped to the gene, N is the total number of reads mapped to all genes, and L is the length of the gene. An alternative form of RPKM is Fragments Per Kilobase of transcript per Million mapped reads (FPKM) Level 3 released gene level expression data for microarray, gene and exon level expression data for RNAseq were downloaded for 14 cancers from TCGA. The data processing and quality control were done by Broad Institute's TCGA workgroup. For microarray data, gene level normalization was performed by using Robust Multi-array Average (RMA) Both RPKM and RSEM results were generated using SeqWare pipeline Consistencies between the RNAseq and microarray data were tested using Spearman's correlation instead of Pearson's correlation due to two reasons: 1. RMA normalization uses log2 transformation for microarray data, for RNAseq data, log transformation is impractical due to the large number of zeroes that often are reported in this method. 2. Pearson's correlation is heavily influenced by outliers, and RNAseq data is heavily skewed. In addition to raw expression correlation, the directionality and agreement of the significantly differentially expressed gene list between the two technologies are also important measurements of concordance. Breast cancer is the only cancer type in TCGA that collected expression data using both RNAseq and microarray on 53 tumor-normal paired samples. Using breast cancer data, we identified significantly differentially expressed genes based on Benjamini-Hochberg adjusted p-value and fold-change between the tumor and normal samples using paired t-tests on the microarray data. Previous studies have shown that edgeR Microarray and RNAseq methods are different in terms of the technology used to quantify gene expression. Microarray methods measure the intensity of fluorescence, which reflects the corresponding gene expression level, while RNAseq methods measure the read count, which also reflects the abundance of the gene product. The expression values of microarray data after RMA normalization are on a log2 scale. The count data of RNAseq, on the other hand, is not usually normalized using the quantile normalization method because a log-transformation does not provide a variance-stabilization of the data as it does for the  microarray data. However, in a study by Dillies et al. The majority of cancer types in TCGA used the Agilent two-channel array for expression profiling. Three types of cancer, GBM, LUSC and OV  due to the more manageable price.With microarray technology slowly fading into history, the huge amount of microarray expression data collected by researchers over the last decade is still very valuable for data mining. The majority of the microarray data have been organized and stored in publicly available databases. The Gene Expression Omnibus (GEO) contains expression data on 848,178 samples across 2720 datasets, and ArrayExpress contains 988,372 assays across 34,148 experiments. Those microarray databases are still providing important support and information for researchers across the world every day.Our results show that two-channel microarray expression data should not be directly compared to RNAseq expression data due to different detection schemes. One-channel microarray has very good concordance with RNAseq expression data. Reasonable agreement was observed between the significantly differentially expressed gene lists identified by two-channel microarray and RNAseq techniques; however, out of the significant genes identified by both RNAseq and two-channel microarray, about 1.2% genes had an opposite direction of the fold change. This inconsistency could be caused by random error, normalization differences or heterogeneity in samples. In the scenario of observing such inconsistency, it would be extremely difficult to decide which method to trust, especially when the difference is caused by sample heterogeneity. Thus, we recommend running multiple analysis methods such as bioconductor package LIMMA We have presented the largest microarray and RNAseq expression comparison study performed thus far. Given the large sample size, our results would be more definitive than previous studies on this subject. There is no denial that RNAseq is replacing microarray at a rapid pace. However there are still researchers who have yet to make transition from microarray to RNAseq. Our study provides definitive evidence that RNAseq can indeed replace microarray in term of expression analysis. Furthermore, our study also shows that results derived from microarray data are trust worthy. The huge amount of microarray data accumulated over last 10 years stored in repositories such as GEO and ArrayExpress can still serve as excellent data mining resources.Table S1Shared samples between gene expression technologies in TCGA.(XLSX)Click here for additional data file.Table S2Inenuity pathway analysis summary for microarray results.(PDF)Click here for additional data file.Table S3Inenuity pathway analysis summary for RNAseq results.(PDF)Click here for additional data file.