ABCB1, ABCB11, ABCC1, ABCC10, BAD, BBC3, BCL2, BCL2L1, BMF, CYP2C8, CYP3A4, MAP2, MAP4, MAPT, NR1I2, SLCO1B3, TUBB1, TUBB4A, andTUBB4B was 78.6% accurate in predicting survival of 84 patients treated with both HT and CT . Accuracy was lower (73.4%) in 304 untreated patients. The performance of other machine learning approaches was also evaluated at different survival thresholds. Minimum redundancy maximum relevance feature selection of a paclitaxel-based SVM classifier based on expression of genes BCL2L1, BBC3, FGF2, FN1, and TWIST1 was 81.1% accurate in 53 CT patients. In addition, a random forest (RF) classifier using a gene signature  predicted >3-year survival with 85.5% accuracy in 420 HT patients. A similar RF gene signature showed 82.7% accuracy in 504 patients treated with CT and/or HT. These results suggest that tumor gene expression signatures refined by machine learning techniques can be useful for predicting survival after drug therapies.Genomic aberrations and gene expression-defined subtypes in the large METABRIC patient cohort have been used to stratify and predict survival. The present study used normalized gene expression signatures of paclitaxel drug response to predict outcome for different survival times in METABRIC patients receiving hormone (HT) and, in some cases, chemotherapy (CT) agents. This machine learning method, which distinguishes sensitivity vs. resistance in breast cancer cell lines and validates predictions in patients; was also used to derive gene signatures of other HT  (tamoxifen) and CT agents  used in METABRIC. Paclitaxel gene signatures exhibited the best performance, however the other agents also predicted survival with acceptable accuracies. A support vector machine (SVM) model of paclitaxel response containing genes  We analyzed stable4 and linked unstable genes in pathways that determine their disposition. This involved investigating the correspondence between 50% growth inhibitory concentrations (GI50) of paclitaxel and gemcitabine and gene copy number, mutation, and expression first in breast cancer cell lines and then in patients1. Genes encoding direct targets of these drugs, metabolizing enzymes, transporters, and those previously associated with chemo-resistance to paclitaxel (n=31 genes) were then pruned by multiple factor analysis (MFA), which indicated that expression levels of genesABCC10,BCL2,BCL2L1,BIRC5,BMF,FGF2,FN1,MAP4,MAPT,NKFB2,SLCO1B3,TLR6,TMEM243,TWIST1, andCSAG2 could predict sensitivity in breast cancer cell lines with 84% accuracy. The cell line-based paclitaxel-gene signature predicted sensitivity in 84% of patients with no or minimal residual disease  based on variants present in genes involved in the transport, biotransformation, or disposition of a drug. We have applied a supervised machine learning (ML) approach to derive accurate gene signatures, based on the biochemically-guided response to chemotherapies with breast cancer cell linesSVM (Support Vector Machine) learning: Previously, paclitaxel-related response genes were identified from peer-reviewed literature, and their expression and copy number in breast cancer cell lines were analyzed by multiple factor analysis of GI50 values of these lines2  and sigma (which controls the flexibility and smoothness of Gaussians)8; these parameters determine how strictly the SVM learns the training set, and hence if not selected properly, can lead to overfitting. A grid search evaluates a wide range of combinations of these values by parallelization. A Gaussian kernel selects the C and sigma combination that lead to the lowest cross-validation misclassification rate. A backwards feature selection (greedy) algorithm was designed and implemented in MATLAB in which one gene of the set is left out in a reduced gene set and the classification is then assessed; genes that maintain or lower the misclassification rate are kept in the signature. The procedure is repeated until the subset with the lowest misclassification rate is selected as the optimal subset of genes. These SVMs were then assessed for their ability to predict patient outcomes based on available metadata (seehttp://chemotherapy.cytognomix.com.lines2 . Given tRF (Random Forest) learning: RF was trained using the WEKA 3.79 data mining tool. This classifier uses multiple random trees for classification, which are combined via a voting scheme to make a decision on the given input gene set. A grid search was used to optimize the maximum number of randomly selected genes for each tree in RF, where k (maximum number of selected genes for each tree) was set from 1 to 19.Augmented Gene Selection: The most relevant genes (features) for therapy outcome prediction were found using the Minimum Redundancy and Maximum Relevance (mRMR) approach10. mRMR is a wrapper approach that incrementally selects genes by maximizing the average mutual information between gene expression features and classes, while minimizing their redundancies:if corresponds to a feature in gene seti,C)S, II as the base feature set. These genes were selected  based either on their known involvement in paclitaxel metabolism, or evidence that their expression levels and/or copy numbers correlate with paclitaxel GI50 values. mRMR and SVM were combined to obtain a subset of genes that can accurately predict patient survival outcomes; here, we considered 3, 4 and 5 years as survival thresholds for breast cancer patients.For this experiment, we used a 26-gene signature , the weighted average of precision and F-measure, the Matthews Correlation Coefficient (MCC) and the area under ROC curve (AUC).11Predicted treatment response for each individual METABRIC patientThe predicted and expected response to treatment for each individual METABRIC patient for each analyses listed inClick here for additional data file.Copyright: © 2017 Mucaki EJ et al.2017Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).The performances of several ML techniques have been compared such that they distinguish paclitaxel sensitivity and resistance in METABRIC patients using its tumour gene expression datasets. We used mRMR to generate gene signatures and determine which genes are important for treatment response in METABRIC patients. The paclitaxel models are more accurate for prediction of outcomes in patients receiving HT and/or CT compared to other patient groups.SVMs and RF were trained using expression of genes associated with paclitaxel response, mechanism of action and stable genes in the biological pathways of these targets . Pair-wiThe RF classifier was used to predict paclitaxel therapy outcome for patients that underwent CT and/or HT . The besThe best overall accuracy and AUC (sensitivity and specificity) for CT/HT patients using mRMR feature selection for SVM predicting outcome of paclitaxel therapy was obtained for CT patients with 4-year survival . Outcome6. Further, the accuracy of the SVM models tested with CT and HT-treated patients was significantly higher for (HER2)-positive patients  compared to (HER2)-negative patients .MAPT expression , Progesterone Receptor (PR), HER2, Lymph Node (LN) and PAM50 subtypes. To assess model co-variate accuracy, predictions described inDiscovery andValidation datasets, respectively, as training and test sets and repeated the previous experiments. In this scenario, the performance of the model was poor (slightly better than random). This occurred because the gene expression distributions of many of the paclitaxel-related genes in our signature were not reproducible between these two sets . The mRMR+SVM CT-models performed well .We assessed the separateValidation dataset, theDiscovery set was split into two distinct parts, consisting of 70% of the patient samples randomly selected for training, and a different set of 30% of samples for testing. This procedure was repeated 100 times using different combinations of training and test samples, and the median performance of these runs is reported . In addition, we evaluated the performance of the model derived from the Discovery set on a different set of patients treated with paclitaxel5. These results suggest that the aforementioned issue withDiscovery training andValidation testing was primarily due to a batch effect, rather than to overfitting.To evaluate the paclitaxel models without relying on theeported . We alsoodel12 . In mostsensu stricto, the initial paclitaxel gene set used for feature selection was the same as in our previous study1. Predictions for the METABRIC patient cohort, which was independent of the previous validation set5 used in Dormanet al.1, of the either same (SVM) or different ML methods (RF and SVM with mRMR) exhibited comparable or better accuracies than our previous gene signature1.While not a replication studyThese techniques are powerful tools which can be used to identify genes that may be involved in drug resistance, as well as predict patient survival after treatment. Future efforts to expand these models to other drugs may assist in suggesting preferred treatments in specific patients, with the potential impact of improving efficacy and reducing duration of therapy.1. The predictions made by our method are based on the level of an individual drug. Genomic information has been shown to correlate with tumor therapy response in previous studies17. From these studies, analytical methods have been used to develop gene signatures for chemotherapy resistance prediction5, subtypes (PAM50), and metastatic risk stratification . We also examined the method exhibiting the best performance in the Sage Bionetworks / DREAM Breast Cancer Prognosis Challenge18, which was also phenotype-based, however it produces outcome signatures based on molecular processes rather than the cancer drugs themselves. While interesting and informative, the results cannot be directly compared. Our approach may be useful for selecting specific therapies in patients that would be expected to produce a favorable response.In this study we used METABRIC dataset to predict outcome for different survival times in patients receiving hormone (HT) and, in some cases, chemotherapy (CT) agents. We used published literature and various machine learning methods in order to identify optimal subsets of genes from a biologically-relevant initial gene set that can accurately predict therapeutic response of patients who have received chemotherapy, hormone therapy or a combination of both treatments. The SVM methodology has been previously shown to outperform randomized gene setsThe data referenced by this article are under copyright with the following copyright statement: Copyright: © 2017 Mucaki EJ et al.Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).Patient data: The METABRIC datasets are accessible from the European Genome-Phenome Archive (EGA) using the accession number EGAS00000000083 (https://www.ebi.ac.uk/ega/studies/EGAS00000000083). Normalized patient expression data for theDiscovery (EGAD00010000210) andValidation sets (EGAD00010000211) were retrieved with permission from EGA. Corresponding clinical data was obtained from the literature6. While not individually curated, HT patients were treated with tamoxifen and/or aromatase inhibitors, while CT patients were most commonly treated with cyclophosphamide-methotrexate-fluorouracil (CMF), epirubicin-CMF, or doxorubicin-cyclophosphamide.10.5256/f1000research.9417.d14986411F1000Research: Dataset 1. Predicted treatment response for each individual METABRIC patient, The authors have addressed all previous comments.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. The authors have addressed all the concerns raised from the previous review. A minor comment for the batch effects is given in the follows. As batch effects are expected for heterogeneous datasets, the direct application of prediction model built on the discovery dataset to the validation dataset would be incorrect and usually result in poor performance. To show the usefulness of the gene signatures while minimizing the batch effects, the authors might consider to run cross-validation on the validation dataset alone using the gene signatures obtained from discovery dataset.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Discovery dataset (Tables 4 and 5), but using theValidation dataset alone for training and testing. We found that this analysis had a similar performance level as the analysis reported in the main manuscript (Tables 4 and 5). There are exceptions. The mRMR+SVM gene signature developed using “CT-only” patients at a 5-year threshold was much less accurate using theValidation data. However, the “CT-only” subset of theValidation dataset is small (N=31), and thus variability is not unexpected. Overall, this analysis suggests that the cross-validation issue was indeed mostly due to batch effects.Thank you for your suggestion. As recommended, we have repeated the 70/30% cross-validation analysis performed in the manuscript (Tables 4 and 5) with the same genes obtained from theThe following sentence was written in the main text which describes this result:Discovery genes, we also trained a separate model using theValidation data, and tested this data by 70/30% cross-validation  .”“Starting with the same set of The authors were very responsive to the previous round of reviews, including more robust cross-validation and cross-study validation and comparison with other classifiers. Particular concerns remain that the author’s conclusions that it is inappropriate to perform cross-study validation due to batch effects are incorrect, particularly since this challenging task is essential to assess overfitting and for clinical translation of classifiers. In addition, the conclusion was insufficiently revised to place their classifier in the context of the broader literature in this field.MethodsAbbreviations SVM and RF must be spelled out as Support Vector Machine and Random Forest on first use. This was not addressed in the revised methods section.ResultsThe authors did perform a robust cross-study validation, as requested in the previous review. We agree this is challenging, due in part to batch effects as reported in this manuscript. However, such cross-study validation is essential to assess the accuracy of classifiers. It is also essential to have translation of genomic signatures into the clinic, where even different assays may be used. To address these concerns the authors must do the following: (a) Remove the sentence “This heterogeneity indicates that it is inappropriate to test our gene expression signatures derived by one of these datasets using the other dataset.” (b) Discuss the importance of cross-study validation, challenges in this application, and potential of overfitting of suggested by these results.et al., (2012) “Nearly all oestrogen receptor (ER)-positive and/or lymph node (LN)-negative patients did not receive chemotherapy, whereas ER-negative and LN-positive patients did. Additionally, none of the HER2+ patients received trastuzumab. As such, the treatments were homogeneous with respect to clinically relevant groupings.” Therefore, the previous criticism #12 remains. Covariates such as ER/HER2/LN or PAM50 subtypes must be included in a table describing the sample cohorts remains. In addition, accuracy must be computed separately for these co-variates or included in the machine learning model.The author’s response that specific therapies were not provided in METABRIC is incorrect. According to CurtisConclusionet al., 2002) into clinical assays based upon expression of smaller numbers of genes.The discussion is insufficient. It still lacks sufficient context of existing genomics classifiers in the literature. The discrepancy between their algorithm and clinical assays is confusing in revised sentence “Unlike Mammaprint and Oncotype Dx tests, this model focuses on predicting survival prediction based on gene expression in the tumor, presumably before or during drug therapy.” As written, it appears to disregard the long history of predicting clinical outcome from gene expression involved in developing these classifiers from gene expression data .ResultsComment 2. The authors did perform a robust cross-study validation, as requested in the previous review. We agree this is challenging, due in part to batch effects as reported in this manuscript. However, such cross-study validation is essential to assess the accuracy of classifiers. It is also essential to have translation of genomic signatures into the clinic, where even different assays may be used. To address these concerns the authors must do the following: (a) Remove the sentence “This heterogeneity indicates that it is inappropriate to test our gene expression signatures derived by one of these datasets using the other dataset.” (b) Discuss the importance of cross-study validation, challenges in this application, and potential of overfitting of suggested by these results.Response: In regards to this point:(a) This sentence has been removed, as requested.et al., 2013). However, this dataset is not publically available and requires Ethics Board / IRB Review which we did not believe to be worth the effort. Instead, we performed cross-study validation on the gene expression of 310 breast cancer patients made publically available by Hatziset al. (2011).(b) To address concerns regarding potential overfitting of our models, we cross-validate the acquired models to a non-METABRIC data set (from an independent study). In the Sage Bionetworks / DREAM Breast Cancer Prognosis Challenge, cross-study validation was performed using the “OsloVal” data set, which consists of gene expression and copy number data from 184 breast cancer patients , where the threshold for resistance was set to 3-years and 4-years. The “CT 3-year” model performed well predicting responsive patients (74.2% accuracy), while the “CT 4-year” model performed better predicting non-responsive patients (75.1% accuracy). The “CT 4-year” model outperformed the “CT 5-year” model for both sensitive and resistant patient data sets.Random Forest and mRMR+SVM models which used hormone-treated patients (“HT” and “CT+HT”) were much less accurate compared to the “CT-only” models, and predict patients a large percentage of patients from the Hatzis data as sensitive.In the main manuscript, we have replaced the removed sentence from (a) and have written the following:BCL2L1; Supplementary file 2). Furthermore, these gene expression differences also affect the performance of these methods when these datasets were combined . We considered the possibility that the Discovery model might be subject to overfitting. We therefore performed cross-study validation of the Discovery set-signature with an independently-derived dataset (319 invasive breast cancer patients treated with paclitaxel and anthracycline chemotherapy5). The mRMR+SVM CT-models performed well .”“Cross-study validation allows for the comparison of classification accuracy between the generated gene signatures. The observed heterogeneity in gene expression highlights one of the many challenges of cross-validation of gene signatures between these  data from the same study exhibit drastic differences :181re1. doi: 10.1126/scitranslmed.3006112.Margolin AA,et al. 2011. A genomic predictor of response and survival following taxane-anthracycline chemotherapy for invasive breast cancer. JAMA. 305, 1873–1881.Hatzis, C.,+ patients received trastuzumab. As such, the treatments were homogeneous with respect to clinically relevant groupings.” Therefore, the previous criticism #12 remains. Covariates such as ER/HER2/LN or PAM50 subtypes must be included in a table describing the sample cohorts remains. In addition, accuracy must be computed separately for these co-variates or included in the machine learning model.Comment 3. The author’s response that specific therapies were not provided in METABRIC is incorrect. According to Curtis et al., (2012) “Nearly all oestrogen receptor (ER)-positive and/or lymph node (LN)-negative patients did not receive chemotherapy, whereas ER-negative and LN-positive patients did. Additionally, none of the HER2Random Forest and mRMR models are shown to be consistently more accurate in predicting ER+, HER2- when treated with hormone therapy (both “HT” and “CT and/or HT” categories), when compared to ER- and HER2+ patients. The PAM50 basal subtype is consistently low in accuracy when testing patients treated with hormone therapy. This is most likely partially influenced by the RF and mRMR models for ‘HT’ to more often predict patients as sensitive, combined with the fact that ER+ and HER2- patients were more likely to response to therapy. It is important to note that the accuracy of predictions by RF and mRMR with patients treated only with chemotherapy was fairly consistent across all available subtypes (+/- 10% accuracy). MAPT expression (which is present in the paclitaxel model) segregated with PAM50 luminal and basal subtypes. For this model, the accuracies of these subtypes are nearly identical to the accuracy of the entire subset.SVM paclitaxel models performed significantly better with HER2+ patients  in HER2- patients  when tested on patients treated with both hormone and chemotherapy. In Dorman et al (2016), it was stated that Text describing these results can be found in the third paragraph of the results.Response: Thank you for the clarification regarding patient treatment. As a response, we have added an additional supplementary table which breaks down the accuracy of our models by subtype . In the main text, we note that accuracy of most models are consistent between subtypes (+/- 10% deviation in accuracy). Subtypes with less than twenty individuals were ignored due to its small sample size. The following deviations in accuracy were noted:ConclusionComment 4. The discussion is insufficient. It still lacks sufficient context of existing genomics classifiers in the literature. The discrepancy between their algorithm and clinical assays is confusing in revised sentence:“Unlike Mammaprint and Oncotype Dx tests, this model focuses on predicting survival prediction based on gene expression in the tumor, presumably before or during drug therapy.” As written, it appears to disregard the long history of predicting clinical outcome from gene expression involved in developing these classifiers from gene expression data  into clinical assays based upon expression of smaller numbers of genes.Response: We have removed the indicated sentence, which we agree was insufficient to the comment from the previous iteration of this article: “Must be discussed in the context of existing genomics classifiers for breast cancer ”.et al. (2016), which described some of the methodology for initial gene selection that this study was based on, these contributions  are  well-referenced, including the history of the prediction of clinical outcome from genomic status:We in no way meant to ignore the long history of predicting clinical outcome from gene expression (as well as other genomic factors). A discussion on this topic was not included in earlier submissions as it initially had an imposed word length limit (upon first submission). We did, however, reference other articles which do discuss this topic. In Dormanet al., 2003; Glinskyet al., 2005; Hatziset al., 2011; Maet al., 2004; Rajputet al., 2013; van't Veeret al., 2002).“Previous studies have derived associations between the genomic status of one or more genes and tumor response to certain therapies  do not take into account multiple mechanisms of resistance or assess interactions between multiple genes. ABC transporter overexpression has long been shown to confer resistance, but enzymatic or functional inhibition has not substantially improve patient response to chemotherapy .Correlations between single gene expression and tumor resistance (Duan®), subtypes (PAM50), and efforts have been made to predict chemotherapy resistance . “Multi-gene analytical approaches have previously been successful in deriving prognostic gene signatures for metastatic risk stratification :5,12-16. From these studies, analytical methods have been used to develop gene signatures for chemotherapy resistance prediction5, subtypes (PAM50), and metastatic risk stratification .”“Genomic information has been shown to correlate with tumor therapy response in previous studiesComment 5. Based on the previous review, the authors include context with other predictions of the METABRIC data in the response to the reviewers. This must also be included in the Conclusion to assess the relevance of their findings in the literature.Response: We have added the indicated text from the previous ‘response to the reviewers’ (modified) to the Conclusions:17, which was also phenotype-based, however it produces outcome signatures based on molecular processes, rather than the cancer drugs themselves. While interesting and informative, the results cannot be directly compared.”“We also examined the method exhibiting the best performance in the Sage Bionetworks / DREAM Breast Cancer Prognosis ChallengePlease note that the majority of entries in the DREAM project were not fully curated and only exist as source code. Analyzing these files to determine what methodology was attempted by these groups is beyond the scope of our study. A description of the second place of the METABRIC phase of the DREAM challenge is provided in the link below. This link describes how the METABRIC data is trained using a bipartite graphing as input for linear models, boost models, and RankSVM. While they state that RankSVM was the least successful between the three methods, it does not appear that this particular study has been published to the literature. As a result, we cannot fully review their results, and thus cannot be compared to our methodology in the main manuscript.https://sagesynapse.wordpress.com/2012/11/01/breast-cancer-challenge-team-pitttransmed-places-second-for-metabric-phase-of-the-challenge/ This study proposed prediction methods using SVM and RF classifiers with mRMR selected feature sets from cell line data and demonstrate its prediction ability for outcomes from METABRIC patient cohort. The classifiers with good prediction performance show the usefulness of combining domain knowledge with feature selection techniques. However, some details essential for reproducibility and interpretation are missing.What are the values of parameters for SVM and RF classifiers and the methods for parameter selection (by default or other selection methods)?The development and evaluation of models for patient data are not clear. Whether the models were trained using partial data from METABRIC or only leave-one-out cross-validation was applied? If cross-validation is the case, then what is the model offered at the online server because there will be more than one models created, and whether the cross-validation is involved in the feature selection process that often leads to an overestimation of the performance. For the case of training on partial data, both training and test performance are essential information for evaluating the robustness of models.Since some of the datasets are highly imbalanced, the numbers of positives and negatives, as well as sensitivity and specificity are more important than accuracy for interpreting the results as a high accuracy with a low AUC could be the result of all positive/negative predictions on an imbalanced dataset. Listing all the information along with the accuracy and AUC will help the interpretation of prediction performances.   Required information is listed in the following.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Comment 1:What are the values of parameters for SVM and RF classifiers and the methods for parameter selection (by default or other selection methods)? Response: The parameter values for these classifiers have been added to the Tables 1-5. In regards to parameter selection, the first paragraph of the methods now describes C and Sigma selection as a grid search to find the values with the lowest cross-validation misclassification rate. Similarly for RF, a grid search was used to optimize the maximum number of randomly selected genes for each tree (second paragraph of Methods section).Comment 2: The development and evaluation of models for patient data are not clear. Whether the models were trained using partial data from METABRIC or only leave-one-out cross-validation was applied? If cross-validation is the case, then what is the model offered at the online server because there will be more than one models created, and whether the cross-validation is involved in the feature selection process that often leads to an overestimation of the performance. For the case of training on partial data, both training and test performance are essential information for evaluating the robustness of models. Response: We obtained new results for both RF and mRMR+SVM models when we use discovery set as training set and validation set as test set, the performance of the model was poor. After more investigation we found that there happened to be a large variation between gene expression of 26 targeted genes between discovery and validation set (please see Supplementary Dataset 2). Hence, building any classifier using discovery and validation set as training and test set in their current forms will result of poor performance, since the training and test sets are vastly different. However, we did carry out another experiment on discovery set solely and used 70% of data for training and remaining 30% for test the performance of the model. The results have been added to the manuscript (Tables 4 and 5).Comment 3: Since some of the datasets are highly imbalanced, the numbers of positives and negatives, as well as sensitivity and specificity are more important than accuracy for interpreting the results as a high accuracy with a low AUC could be the result of all positive/negative predictions on an imbalanced dataset. Listing all the information along with the accuracy and AUC will help the interpretation of prediction performances.  Response: As previously mentioned, we have added more performance measures including MCC and AUC. They have been added Tables 1-5 of the manuscript. The methods require further clarification to distinguish differences between this study and the previous study as well as the parameters of the machine learning algorithms.Accuracy in the results must better distinguish results on independent test and training sets.Classifiers must be put in the context of other existing genomics classifiers used in breast cancer and/or previously published in Mammaprint data.  This study develops SVM and RF algorithms built upon previously learned gene signatures of therapeutic response to breast cancer. The algorithms are applied and compared to predict patient survival under different treatment conditions in METABRIC data. The analyses and comparisons are robust and this study provides a useful assessment of biologically-driven classifiers. The three major areas that require improvement before the article is indexed are as follows, and described in further detail below.Title and Abstract  AcceptableArticle contentMethodsAbbreviations SVM and RF must be spelled out as Support Vector Machine and Random Forrest on first use in Methods.SVM learning subsection ofMethods requires clarification to distinguish which of these methods were developed in the previousMolecular Oncology publication and which were developed as part of this publications.Writing inDetails about the SVM learning algorithm are included in the caption to Figure 1, but must also be included and completely described in text for the corresponding section of the methods.and sigma. It is also unclear whether this greedy search is implemented by the Matlab functionfitcsvm or uses custom code developed by the authors.No equations are provided to describe the role of the parameters C  ResultsNeed to specify whether reported accuracies are computed with leave-one-out cross validation or 9-fold cross validation (described in Methods).Ideally, given the size of METABRIC data they would be calculated on independent training (first 1000 patient samples) and training (last 1000 patient samples) datasets.AUC must be computed separately for discovery and validation sets (Table 2).It is unclear whether the previous validation set described in the sentence “Predictions for the METABRIC patient cohort, which was independent of the previous validation set” refers to a validation set used in this publication or the previous publication.Covariates such as ER/PR or PAM50 subtypes must be included in a table describing the sample cohorts. Accuracy must be computed separately for these co-variates or they must also be included as co-variates in the machine-learning model.et al., BMC Genomics, 2013) and/or survival curves reported in the literature.Ideally accuracy would be compared to existing breast cancer classifiers .Results must be put in context with other predictions on METABRIC data, e.g., outcomes from the DREAM contest.  Data AcceptableI have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Comment 1: The methods require further clarification to distinguish differences between this study and the previous study as well as the parameters of the machine learning algorithms.et al., 2016. Response: The first paragraph of the Methods describes Support Vector Machine learning, which has been greatly expanded upon. Differences in SVM methodology between the two studies are indicated there (i.e. a Gaussian kernel was used instead of a linear kernel). All other feature selection methods described in the manuscript  were not used in Dorman The parameters for machine learning algorithms have been incorporated in the manuscript, and can be found in the footnote section of each data table.Comment 2: Accuracy in the results must better distinguish results on independent test and training sets. Response: The Validation dataset showed a distinct overall expression profile from the Discovery set, possibly due to batch effects, which are well known. We added another experiment to the manuscript by splitting the Discovery set into Training and Test sets. The model was trained using 70% of the data and then tested using the remaining 30% of data as test set. We repeated this procedure 100 times and took the median as the final performance result. The results are presented in Tables 4 and 5 of the manuscript.Comment 3: Classifiers must be put in the context of other existing genomics classifiers used in breast cancer and/or previously published in Mammaprint data. Response: We have added two sentences in the second paragraph of the “Results and Discussion” section which describes the comparison of our gene signature to those from MammaPrint and Oncotype Dx. Pair-wise comparison of these three signatures show that they are nearly independent of one another.MethodsComment 4: Abbreviations SVM and RF must be spelled out as Support Vector Machine and Random Forest on first use in Methods. Response: We thank the reviewer for this suggestion. It has been addressed in the Methods section of the manuscript.  Comment 5: Writing in SVM learning subsection of Methods requires clarification to distinguish which of these methods were developed in the previous Molecular Oncology publication and which were developed as part of this publications. Response: This is now clarified within the first paragraph of the Methods section in the manuscript. The SVM classifier was adopted from previous Molecular Oncology publication, while the feature selection method has been developed as part of this publication.Comment 6: Details about the SVM learning algorithm are included in the caption to Figure 1, but must also be included and completely described in text for the corresponding section of the methods. Response: Thanks for the reviewer’s suggestion. This description of the SVM learning algorithm has been moved from the Figure 1 legend and integrated into the first paragraph of the methods section.Comment 7: No equations are provided to describe the role of the parameters C and sigma. It is also unclear whether this greedy search is implemented by the Matlab function fitcsvm or uses custom code developed by the authors. Response: A brief description of the role of each parameter has been added to the first paragraph of the methods section of the manuscript. Readers are also now directed to a reference  if more detail is desired. The greedy search, also called sequential backward feature selection, was implemented as a script by our lab in MATLAB. It is not a MATLAB function. This is clarified by changing a few words in the first paragraph of the methods section: “A backwards feature selection (greedy) algorithm was designed and implemented in MATLAB in which…”et al. 2016), while the feature selection method has been developed as part of this publication. Moreover, as described above, the SVM classifier was adopted from previous Molecular Oncology publication . Response: All SVM models described in the manuscript used leave-one-out cross validation except one, and this is clearly indicated in Table 1, and is now commented on in the methods. A 9-fold cross-validation was used to build a model using 735 patients who were treated with Chemotherapy and/or Hormone therapy, as leave-one-out cross validation of this many patients took an unreasonably long time to complete (it exceeded 3 weeks on a dedicated I7 Intel processor).Comment 9: Ideally, given the size of METABRIC data they would be calculated on independent training (first 1000 patient samples) and test (last 1000 patient samples) datasets. Response: We obtained new results for both RF and mRMR+SVM models using Discovery patient set for training and Validation set for testing, however the performance of the model was poor. After further investigation, we found that there were large differences between gene expression levels of the 26 model signature genes in the Discovery versus Validation sets  regardless of patient status . Hence, building any classifier using discovery and validation set as training and test set in their current forms will result of poor performance due to this source of heterogeneity. To address this issue, we did carry out another experiment based on data from the Discovery patient dataset alone; using 70% of data for training and remaining 30% for testing, the performance of the model was significantly better. We speculate that the discrepancy between the expression distributions in the Discovery and Validation sets were the result of batch effects. The results have been added to the manuscript .Comment 10: AUC must be computed separately for discovery and validation sets (Table 2). Response: We have included additional performance measures to Tables 1-5, including Area Under Curve (AUC).Comment 11: It is unclear whether the previous validation set described in the sentence “Predictions for the METABRIC patient cohort, which was independent of the previous validation set” refers to a validation set used in this publication or the previous publication.et al. (2013), which was used as a validation set in Dormanet al. (2016), not this publication. We have modified this sentence to clarify the issue. Response: This sentence is referring to breast cancer patient data from HatzisComment 12: Covariates such as ER/PR or PAM50 subtypes must be included in a table describing the sample cohorts. Accuracy must be computed separately for these co-variates or they must also be included as co-variates in the machine-learning model.et al., 2012). The public METABRIC dataset (or the corresponding publication) does not provide the specific therapies used to treat individual patients. Had they done so, it would have made sense to look at these covariates. Response: Even with the subtype as covariant, it is not possible to perform the analysis the reviewer requested. Certain therapies are definitely more effective in particular subtypes . Subtype and pathway specific responses to anticancer compounds in breast cancer. Proc Natl Acad Sci US A109:2724-2729. Reference: Heiser LM, Sadanandam A, Kuo WL, Benz SC, Goldstein TC, Ng S, Gibb WJ, Wang NJ, Ziyad S, Tong F,Comment 13: Ideally accuracy would be compared to existing breast cancer classifiers  and/or survival curves reported in the literature.et al., BMC Genomics, 2013) as per reviewer’s suggestion and the results are presented in Table 6 of the manuscript. Response: The proposed method has been compared against the K-TSP . Response: We have added text to both the second paragraph of the “Results and Discussion” paragraph and to the conclusion of the paper.Comment 15: Results must be put in context with other predictions on METABRIC data, e.g., outcomes from the DREAM contest.et al., 2013) was phenotype-based, finding signatures for molecular processes that are disregulated in METABRIC, rather than responses to the cancer therapies themselves. While this is an interesting prediction method, the results cannot compared to our approach. The gene signatures that we have derived contain components of many different pathways. Response: An important distinction to note in regards to our methodology is that the predictions are based on the genes known to be associated with the response to specific drugs used to treat breast cancer. In the DREAM contest, the method with the highest METABRIC score :e1002920.


FecB gene has been discovered as an important gene in sheep for its high relationship with the ovulation rate, but its regulatory mechanism remains unknown. In the present study, liquid chromatography-mass spectrometry (LC-MS) and gas chromatography-mass spectrometry (GC-MS) techniques were adopted to detect the metabolic effects of FecB gene in follicular fluid (FF) and ovarian vein serum (OVS) in Small Tail Han (STH) sheep. ANOVA and random forest statistical methods were employed for the identification of important metabolic pathways and biomarkers. Changes in amino acid metabolism, redox environment, and energy metabolism were observed in FF from the three FecB genotype STH ewes. Principal component analysis (PCA) and hierarchical clustering analysis (HCA) showed that metabolic effects of FecB gene are more pronounced in FF than in OVS. Therefore, the difference of the metabolic profile in FF is also affected by the FecB genotypes. In Spearman correlation analysis, key metabolites , cysteine-glutathione disulfide, γ-glutamylglutamine, and 2-hydrosybutyrate) in ovine FF samples showed a significant correlation with the ovulation rate. Our findings will help to explain the metabolic mechanism of high prolificacy ewes and benefit fertility identification.The  FecB by the Committee on Genetic Nomenclature of Sheep and Goats  3 . FF samp3 ,52,53. C® system , and centrifuged. The resulting supernatants were analyzed by UPLC-MS/MS in a positive and negative ion mode  and by GC-MS .Metabolite extraction and detection were performed at Metabolon Inc.  as previously described . FF and m/z), and chromatographic data  on all molecules present in the library. Extraction of raw data, peak identification, and QC were performed using Metabolon’s hardware and software according to previously published methods . CompounPeaks were quantified based on the area under the curve. The normalized data for each compound were corrected into the medians of data equaling to 1.00 and making other data point proportionate. The normalized data points were also beneficial for data visualization. http://cran.r-project.org/).The chi-square test was used to analyze categorical variables such as the ovulation rate and the follicle number difference. One-way ANOVA was adopted to analyze continuous variables of ovulation follicle diameter and the levels of biochemicals. To assess the relationship between metabolic changes and ovulation rates, the Spearman correlation was calculated using scaled imputed log-transformed metabolite levels and ovulation rate values . Principal component analysis (PCA) and hierarchical clustering analysis (HCA) were used to obtain a high-level view of metabolomic datasets. PCA is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principle components. HCA assesses sample similarity by grouping a “hierarchy” of clustered samples and grouping metabolically similar samples close to one another. Random forest was employed to determine key biochemicals and to differentiate classification groups. Statistical analyses mentioned above were performed with the Array Studio, version 7.2  and R version 3.0.2 (FecB mutation (homozygous and heterozygous) in relation to non-carriers differed in a number of metabolic readouts with a more pronounced metabolic impact of the FecB gene in FF than in OVS . In FF, changes in amino acid metabolism relating to different rates of protein biosynthesis might affect the growth of the developing oocyte. Differences observed in the biochemicals involved in redox homeostasis reflected a higher oxidative pressure of rapidly dividing cells in BB animals, which indicated a greater antioxidant capacity in FF. Additionally, some key metabolites in ovine FF samples were found to significantly correlate with ovulation rate. Collectively, similar to the ovulation rate, the differences in the metabolic profile in FF is also affected by the FecB genotypes.The results from theses global metabolomics studies comparing ewes carrying


Next-generation sequencing technologies have recently been used in pharmacogenomic studies to characterize large panels of cancer cell lines at the genomic and transcriptomic levels. Among these technologies, RNA-sequencing enable profiling of alternatively spliced transcripts. Given the high frequency of mRNA splicing in cancers, linking this feature to drug response will open new avenues of research in biomarker discovery. To identify robust transcriptomic biomarkers for drug response across studies, we develop a meta-analytical framework combining the pharmacological data from two large-scale drug screening datasets. We use an independent pan-cancer pharmacogenomic dataset to test the robustness of our candidate biomarkers across multiple cancer types. We further analyze two independent breast cancer datasets and find that specific isoforms of IGF2BP2, NECTIN4, ITGB6, and KLHDC9 are significantly associated with AZD6244, lapatinib, erlotinib, and paclitaxel, respectively. Our results support isoform expressions as a rich resource for biomarkers predictive of drug response. Altered mRNA splicing features in many cancers, but it has not been linked to drug response. Here, with their meta-analytic framework, the authors analyse pharmacogenomic data to identify isoform-based biomarkers predictive of in vitro drug response, and show them to frequently be strong predictors. The overarching goal of these seminal studies was to identify molecular features predictive of drug response (predictive biomarkers). Consequently, the GDSC and CCLE investigators were able to confirm a number of established gene–drug associations, including association between ERBB2 amplification and sensitivity to lapatinib, and BCR/ABL fusion expression and nilotinib. They also found new associations such as SLFN11 expression and response to topoisomerase inhibitors, thereby supporting the relevance of cell-based high-throughput drug screening for biomarker discovery. However, the biomarkers validated in preclinical settings are still largely dominated by genetic  as opposed to transcriptomic (gene expression) features. Therefore, there is a need for further investigation of transcriptomic markers associated with drug response in cancer.Cell lines are the most widely used cancer models to study response of tumor cells to anticancer drugs. Not only have these cell lines recently been comprehensively profiled at the molecular level, but they have also been used in high-throughput drug screening studies, such as the Genomics of Drug Sensitivity in Cancer (GDSC)6. However, it is well established that genes undergo alternative splicing in human tissues7, and changes in splicing have been associated with all hallmarks of cancer8. Despite the major role of alternative splicing in cancer progression and metastasis8, only a few small-scale studies have reported associations between these spliced transcripts  and drug response or resistance14. These limited, yet promising associations support the potential relevance of isoform expression as a new class of biomarkers predictive of drug response. Among the mRNA expression profiling technologies, high-throughput RNA-sequencing (RNA-seq) enables quantification of both isoform and gene expression abundances at the genome-wide level. Recent studies have highlighted the advantages of RNA-seq over microarray-based gene expression assays19. In particular, microarray-profiling platforms are limited to pre-designed cDNA probes and they depend on background levels of hybridization. They also suffer from limited dynamic range probe hybridization. As the detection of transcripts and genes using RNA-seq is based on high-resolution short-reads sequencing instead of probe design, they have the potential to overcome these limitations17.The vast majority of pharmacogenomic studies investigated the association between gene-specific mRNA abundance and drug sensitivity22. As part of CCLE, the Broad Institute of Harvard and MIT released RNAseq profiles of 935 cancer cell lines through the Cancer Genomics Hub , whereas Genentech deposited RNA-seq data for 675 cell lines on the European Genome-phenome Archive (EGA) as part of their Genentech Cell Line Screening Initiative (gCSI)22. Two other initiatives used RNA-seq to profile panels of 70 (GRAY3) and 84 (UHN21) breast cancer cell lines. The availability of these valuable datasets offers unprecedented opportunities to further explore the transcriptomic features of cancer cells and study their association with drug response.Recent initiatives have profiled hundreds of cancer cell lines using Illumina RNA-seq technologyIn this study, we explore the genome-wide transcriptomic landscape of large panels of cancer cell lines to identify isoform-level expression features predictive of drug response in vitro. On the basis of our new metaanalytical framework combining the GDSC and CCLE pharmacogenomic data for biomarker discovery, we show that isoform-level expression measurements are more predictive of response to cytotoxic and targeted therapies than are gene-level expression values. We tested the accuracy of our most promising isoform biomarkers in an independent pan-cancer dataset, gCSI, and two breast cancer datasets, GRAY and UHN. We validated four isoform-based biomarkers predictive of response to lapatinib, erlotinib, AZD6244 (MEK inhibitor), and paclitaxel, indicating that isoforms constitute a promising new class of biomarkers for cytotoxic and targeted anticancer therapies.25, providing us with the opportunity to discover biomarkers generalizable to multiple measures of drug sensitivities. We identified a wide range of statistically significant biomarkers for each drug , which used the same pharmacological assay as CCLE. We first selected the significant isoform-based biomarkers in our training set that were predictive in breast cancer cell lines (see “Methods”). We assessed the predictive value of these biomarker candidates in GRAY and tested whether these isoform biomarkers were significantly more predictive than their corresponding gene expression  from those used in the training and pre-validation sets. We first cultured cell lines to check their doubling time in a course of 120 h , NECTIN4 (ENST00000368012), and ITGB6-001 (ENST00000283249) isoforms were associated with sensitivity to AZD6244, lapatinib, and erlotinib, respectively  isoform is associated with lack of sensitivity to paclitaxel . This resulted in validation of 2 out of 12, 2 out of 5, 6 out of 11, and 11 out of 35 biomarkers for AZD6244, lapatinib, erlotinib, and paclitaxel, respectively  in GRAY and UHN, two independent breast cancer pharmacogenomic datasets , which differs from the assays used in the training and pre-validation datasets  /extracellular signal-regulated kinase (ERK) pathway, this suggests that IGF2BP2 could be a putative biomarker of sensitivity for compounds acting upstream of the MAPK-ERK pathway and targeting MEK1/2 such as AZD6244.Given that we and others have shown that the choice of pharmacological assay may influence drug sensitivity measurementsP2) Fig. , which cHER2/neu and epidermal growth factor receptor (EGFR) pathways, and is a FDA-approved drug for breast cancer. In our study, NECTIN4 isoform expression (ENSG00000143217) has been associated with sensitivity to lapatinib  gene is associated with lack of sensitivity to paclitaxel in breast cancer cell lines , and 14,650 pseudogenes. Expression values were computed as the log2(FPKM+1)45, where FPKM represents the number of fragments per kilobase per million mapped reads units that control sequence length and sequencing depth55.The RNA-seq reads were aligned to the Ensembl Genome Reference Consortium release GRCh3850  of the cell lines treated by the drug of interest; XG represents a vector of size N × 1 of log2 normalized FPKM values for the expression of gene G across all the cell lines; GI is all the isoforms of gene G; IG is a vector of size N × 1 of log2 normalized FPKM values for each isoform of G across all the cell lines. The effect size of each association is quantified by βG and βI, which indicate the strength of associations between drug response and the molecular feature of interest, adjusted for tissue type. To estimate standardized coefficients from the linear model, the variables Y and XG and IG are scaled . The null model  of all of its isoforms is estimated with Eq.  is selected for further analysis  method59.To determine the most predictive isoform for each gene the predictive value  with the same direction of association (sign of the coefficient β) as the training sets. Similarly to the discovery phase, we selected the validated biomarkers by statistically comparing the concordance index distribution of the isoform-based and gene-based models to concordance indices of the null model (tissue type only) based on the bootstrap procedure using a one-sided Wilcoxon-signed rank test  with the same direction of association (sign of the coefficient β) as the training sets. To select the validated biomarkers whose isoform expression is significantly more predictive than the corresponding overall gene expression, we estimated the concordance index distribution of the isoform-based and gene-based models using the bootstrap procedure and compared these distributions using a two-sided Wilcoxon-signed rank test  wells value from all wells and computed the standard deviation and coefficient for each triplicate. Data points with coefficient or standard deviation > 0.2 were discarded. All the individual treated well values were normalized to the control well values. We used the PharmacoGx50 package to fit the curves using a logarithmic logistic regression method to estimate the AUC sensitivity values. Raw and processed pharmacological data are available through our PharmacoGx platform under the UHNBC PharmacoSet.To test whether the predictive value of the isoform-based biomarkers validated in GRAY was robust to the use of a different pharmacological assay, we leveraged a collection of 84 breast cancer cell lines recently used to investigate gene essentiality in breast cancer molecular subtypesgithub.com/bhklab/RNASeqDrug). A detailed tutorial describing how to run our pipeline and reproduce our analysis results is available in the GitHub repository. Our study complies with the guidelines outlined in refs. 62.Our code and documentation are open-source and publicly available through the RNAseqDrug GitHub repository . The GRAY dataset is available from EGA (EGAS00000000059 and EGAS00001000585) and ArrayExpress (E-TABM-157 and E-MTAB-181). The RNA-seq data for the UHN dataset are available on the NCBI Gene Expression Omnibus (GSE73526), whereas the drug sensitivity data are available in PharmacoGx (UHNBreast).The pharmacogenomics data used in this study are publicly available through our PharmacoGx platformSupplementary InformationPeer Review fileDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Data 4Supplementary Data 5Supplementary Data 6


One important task using gene-expression data is to identify a small subset of genes that can be used to build diagnostic classifiers particularly for cancer diseases. Microarray based classifiers are not directly applicable to RNA-Seq data due to its discrete nature. Overdispersion is another problem that requires careful modeling of mean and variance relationship of the RNA-Seq data. In this study, we present voomDDA classifiers: variance modeling at the observational level (voom) extensions of the nearest shrunken centroids (NSC) and the diagonal discriminant classifiers. VoomNSC is one of these classifiers and brings voom and NSC approaches together for the purpose of gene-expression based classification. For this purpose, we propose weighted statistics and put these weighted statistics into the NSC algorithm. The VoomNSC is a sparse classifier that models the mean-variance relationship using the voom method and incorporates voom’s precision weights into the NSC classifier via weighted statistics. A comprehensive simulation study was designed and four real datasets are used for performance assessment. The overall results indicate that voomNSC performs as the sparsest classifier. It also provides the most accurate results together with power-transformed Poisson linear discriminant analysis, rlog transformed support vector machines and random forests algorithms. In addition to prediction purposes, the voomNSC classifier can be used to identify the potential diagnostic biomarkers for a condition of interest. Through this work, statistical learning methods proposed for microarrays can be reused for RNA-Seq data. An interactive web application is freely available at  In molecular biological studies, gene-expression profiling is among the most widely applied genomics techniques to understand the role and the molecular mechanism of particular genes in conditions of interest . Recent Identifying the relevant genes across the conditions  is a common research interest in gene-expression studies. One major task is to detect the minimal set of genes which gives the maximum predictive performance for the diagnostic purpose of samples in medicine. A particular interest is the cancer classification based on the simultaneous monitoring of thousands of genes .For microarray studies, a great deal of machine learning algorithms have been proposed and applied for gene-expression based classification. However, these algorithms cannot be directly applied to RNA-Seq data, since the type of the data is entirely different. In contrast to the continuous data format of microarrays, RNA-Seq data are summarized with nonnegative and integer-valued counts, which are obtained from the number of mapped sequencing reads to genomic regions of the species of interest.For the classification purpose, there is still less advancements for RNA-Seq data until recently. Recently, variance modeling at the observational level (voom) method has been proposed to open access microarray based methods for RNA-Seq analysis . The vooIn this paper, we introduce voomNSC sparse classifier, which brings two powerful methods together for the classification of RNA-Seq data: voom and the nearest shrunken centroids (NSC) algorithm . For thi 1.to extend voom method for RNA-Seq classification studies; 2.to make NSC algorithm available for RNA-Seq technology.We also made available the diagonal discriminant classifiers  to be abWe organized the rest of this study as follows. In the ‘Materials & Methods’ section, we present the underlying theory of voomDDA classifiers and detail the experiments. In the ‘Results’ section, we give the results of simulation and real dataset experiments. We discuss and conclude our study in the ‘Discussion’ and ‘Conclusion’ sections. Extended information about the methods background, experiments, results and software source codes are available in p × n dimensional count data matrix, where p refers to the number of features and n refers to the number of samples. Input data may consist of either xgi raw or X, is a p × n dimensional, filtered and non-normalized count data matrix.In this section, we detail the methodology of voomDDA classifiers. We assume that the input data is a Xi. be the library size for sample i. We start by calculating the log-counts-per million (log-cpm) values using the Firstly, we get the voom estimates, i.e., log-counts and associated precision weights, as described in wgi, we take advantage of the delta rule, linear models and lowess smoothing curves. We assume a linear model between the expected size of the log-cpm values and the class conditions as follows: Small constant values 0.5 and 1 in the formula are used to avoid taking the logarithm of zero and guaranteeing that βg corresponds to a vector of regression coefficients to be estimated. These coefficients are the log-fold-changes between class conditions  as follows: A piecewise linear function zgi, and the associated precision weights, wgi, will be used in the model building process of voomDDA classifiers.The log-cpm values, ik, …, ik+1 − 1 belong to class k, k ∈ 1, …, K, nk is the number of samples in class k and we set iK+1 = n + 1. Let kth class, kth class and gth gene can be calculated as follows: First of all, we assume that genes are independent of each other in building classification rules. Let gth gene in class k can be calculated as follows: The weighted variance of Using these weighted statistics, we define voomDLDA and voomDQDA classifiers, which are extensions of DLDA and DQDA classifiers for RNA-Seq data with the weighted parameter estimates. voomDLDA assumes that the gene specific weighted variances are equal across groups and uses the weighted pooled covariance matrix in modeling class-conditional densities k.Discriminant rules for voomDLDA and voomDQDA classifiers are given below: zg∗ should be obtained after normalizing and transforming x∗ based on the distributional parameters of the training dataset.A new test observation Suppose that the training dataset is normalized using the DESeq median ratio normalization method. Then the size factor of a test observation, X.∗ be the library size for the test observation. Then, we calculate TMM normalization factors as follows: xg∗, xgr > 0.If we use the trimmed mean of M values (TMM) normalization method, then a reference sample which is selected in the training set, will be used for the normalization of the test set. Let x∗ can be calculated as: In the voom transformation, log-cpm values for xg∗ in the formula.If a normalization  is initially applied, then the normalized values The RNA-Seq data is high-dimensional just like the microarray data. Hence, one obtains very complex models from voomDLDA and voomDQDA classifiers. Here, we present the voomNSC algorithm to overcome this complexity and obtain simpler, more interpretable models with reduced variance. voomNSC, which incorporates both log-cpm values and the associated weights together into the estimation of model parameters by using the weighted statistics, is an extension of Tibshirani et al.’s , 2003 NSdwgk be the weighted difference scores, between weighted centroids of kth class and overall weighted centroids: sw0 is a small positive constant added to the denominator of sw0 is calculated from the median value of swg across genes.Like the NSC algorithm, voomNSC aims to identify the most important gene subset for class prediction. Briefly, the standardized class specific weighted gene expression means are shrunken towards the standardized overall gene expression weighted means. The genes shrunken to zero are eliminated and a voomDLDA classification model is built with the remaining genes. Mean expressions are also called centroids. Let These weighted difference scores can be considered as the voom extension of the “relative differences” mentioned in dwgk is shrunken towards zero using the soft-thresholding shrinkage method. This method is equivalent to lasso. Using soft-thresholding with a certain threshold parameter λ, weighted shrunken differences can be obtained as follows: Next, each After shrinking λ will lead to obtaining sparser models by eliminating most of the genes from the class prediction. When g, among all classes, the weighted centroids will be same across the classes. Hence, this gene will not contribute to the class prediction.Increasing λ is very important on the model sparsity. Increasing λ will lead to obtaining sparser models. However, the predictive performance of obtained models might be decreased dramatically. Small values of λ, on the other hand, might increase the accuracy of classifiers, yet it may increase the complexity of the models. Thus, it is necessary to select λ that yields both accurate and sparse results. λ parameters for the cervical dataset  for comparison, since they are the only algorithms proposed for RNA-Seq classification. We also applied the diagonal discriminant classifiers , after transforming the data to make it more similar to microarrays. SVM (support vector machines) and RF (random forests) algorithms are also considered due to their good performances in microarray based classification studies. Implementation details of each algorithm, including voomDDA classifiers are given in below:PLDA1: The data are normalized using the DESeq median ratio method. Normalized count values are taken as input to the PLDA algorithm. A five-fold cross validation is performed to identify the optimal ρ tuning parameter. A grid search (number of searches: 30) is applied and the sparsest model with the minimum misclassification error is selected to optimize ρ. The PLDA is applied with the optimum ρ using PoiClaClu R package . NSC is applied using the R package pamr  is made to determine the optimal sigma and cost parameters. The radial basis function is used to allow for nonlinear decision boundaries in the SVM. SVM is applied using the R package caret  is applied to determine the optimal threshold value.voomNSC2: The raw read counts are directly used as the input for the voomNSC algorithm. All other procedures remain same with voomNSC1.voomNSC3: The TMM method is applied to normalize the data. The normalized data is used as the input for the voomNSC classifier. Other procedures are same with voomNSC1 and voomNSC2.voomDLDA1: The DESeq median ratio normalization is applied to the data and the normalized data is used as the input for voomDLDA classifier. The proportions of class sample sizes are used as the class prior probabilities.voomDLDA2: The raw count data are not normalized and directly used as the input for voomDLDA classifier. Other procedures are same with voomDLDA1.voomDLDA3: The TMM method is used for normalization. Other procedures are same with voomDLDA1 and voomDLDA2.voomDQDA1: The DESeq median ratio normalization is applied to the data and the normalized data is used as the input for voomDQDA classifier. The proportions of class sample sizes are used as the class prior probabilities.voomDQDA2: The raw count data are not normalized and directly used as the input for voomDQDA classifier. Other procedures are same with voomDQDA1.voomDQDA3: The TMM method is performed for normalization. Other procedures are same with voomDQDA1 and voomDQDA2.To evaluate the performance of the developed algorithms, we performed a comprehensive simulation study. Four real datasets were also used to illustrate the applicability of voomDDA classifiers and assess their performance in real experiments. All experimental R code is available in p × n dimensional matrix) under 648 scenarios using negative binomial distribution as follows: μgi corresponds to ggsi, egk is the differential expression probability for each of the p = 10,000 genes among classes, and ϕg is the dispersion parameter. For a given yi = k, xgi has mean μgiegk and variance si is the size factor for each sample and simulated identically and independently from gg refers to the total number of counts per gene and also simulated identically and independently from k, then egk is set to 1. Otherwise, σ is set to 0.10 or 0.20 in simulations. Of the total p = 10,000 genes, 500, 1,000 and 2,000 genes with maximum variances are selected. We added a small constant (ε = 1) to count values of each simulated data to avoid taking the logs of zero in the following analysis.We simulated data .dispersion parameter; , genes having near zero variances in the training set are filtered in this step. For this purpose, two criteria are used for filtering: (i) the ratio of the most frequent value to the most frequent second value is higher than 19 (95/5); (ii) the number of unique values divided by the sample size is less than 10%. The genes with near zero variance are filtered from the test set as well.Variance filtering: Next, a second filtering is applied to keep only the informative genes in the model. In the training set, 500, 1,000 and 2,000 genes with maximum variances are selected and other genes are filtered from both training and test sets. In this step, count data are normalized using the DESeq median ratio method and transformed using vst transformation. The genes are sorted in decreasing order based on their variances. The count values of the selected genes were fetched again for further analysis.Normalization: After filtering steps, the datasets are normalized to adjust the sample specific differences using the DESeq median ratio method or TMM method depending on the selected classification method. Note that the size factors required for the normalization are calculated from the unfiltered raw dataset. The datasets are not normalized for voomNSC3, voomDLDA3 and voomDQDA3 classifiers. Since training and test sets should be in the same scale and be homoscedastic relative to each other, the normalization of test datasets is made based on the information obtained from the training datasets. Therefore, each test sample should be independently normalized using the same parameters calculated from the training set as described in the previous section.Transformation: After normalization, several transformations are applied to the data to estimate the mean and the variance relationship of the data. Normalized count data are converted into a continuous scale using this mean and variance relationship. Since PLDA1 and NBLDA are count-based classifiers, the transformations are not applied for these classifiers. VoomDDA classifiers use the voom method inside the algorithm for transformation. A power transformation is applied for PLDA2 classifier. The rlog transformation is performed for other classifiers, due to its capability of accounting for variations in sequencing depth across samples  dataset is an RNA-Seq dataset that is obtained from The Cancer Genome Atlas (TCGA) (s (TCGA) . TCGA iss (TCGA) .Lung cancer dataset: Lung cancer is another RNA-Seq dataset provided from TCGA platform. Same options as RCC data were used in the download process. The resulting count file contains the read counts of 20,531 transcripts of 1,128 samples. The dataset has two distinct classes including lung adenocarcinoma (LUAD) and lung squamous cell with carcinoma (LUSC) with 576 and 552 class sizes, respectively. These two classes are used as class labels in our analysis.Batch effects of TCGA data sets including renal cell cancer and lung cancer types were removed using the Combat function in the SVAseq package  prior toA similar procedure is followed with the simulation study. The data are randomly split into two parts as training (70%) and test (30%) sets. Near zero filtering is applied to all datasets except Alzheimer, since low counts were already filtered by the authors of the study . Next, 2We repeated the entire process 50 times, since cervical and Alzheimer datasets have relatively small sample size. The test set errors may differ for different train/test splits. Seed number is set between 1 to 50 in the analysis steps. In the results section, summary statistics are given across these 50 repeats.To assess the performance of classifiers, we used three criteria: (i) sparsity, (ii) accuracy and (iii) computational cost. We simply assessed the sparsity of each model by calculating the sparsity which is the number of selected genes in each model, or relative sparsity, with the ratio of the number of genes selected in each classification model over a total number of genes. A model with the lower number of genes in the decision rule is considered as the sparser model. We calculated misclassification errors as the accuracy measure of each model in the test set. Due to the high-dimensionality of the RNA-Seq data, it is possible to encounter the overfitting problem. This means a classification method may perfectly classify the training set, but may not perform well in the test set. Since correctly predicting the class labels of new observations is the major purpose in real life problems, we randomly split each dataset into training and test sets. All model building processes are applied in the training set and the performance assessment is performed on test sets. Misclassification errors are calculated from predicted test observations. A model with less misclassification error is considered as the more accurate model.In case of unbalanced class sizes, misclassification error may lead to problems measuring the actual accuracy. Here, we used the balanced misclassification error as evaluation criterium: , the probability of differential expression (egk) and standard deviation (σ). Odd numbered figures give the accuracy results, while the even numbered figures give the sparsity results. Note that the sparsity results are only given for sparse classifiers . All figures are given in the same format in the same matrix layout. Each figure displays the effect of sample size (n), the number of genes (p′), dispersion parameter (ϕg) on the accuracy and the sparsity of classification models. Axis panels give the results for sample size, ordinate panels give the results for dispersion parameter. Each panel demonstrates the error bars for each classifier on classification performance. Classifiers are displayed in axes; evaluation measures are displayed in ordinates in each panel. Each measure is in the range , where lower values corresponding to more accurate or sparse models. Error bars are generated from the arithmetic mean and 95% confidence level of each performance measure in 50 repeats. Black, red and green bars correspond to the results for 500, 1,000 and 2,000 genes, respectively.Misclassification errors and sparsity results for the simulation scenario 1, PLDA2) and voomNSC  classifiers, unless the data are overdispersed. Increasing the number of genes, mostly provides less accurate results for other classifiers. However, this relation may change in some scenarios where the sample size and the standard deviation increased. VoomNSC and PLDA classifiers mostly produce sparser results depending on the increase in the number of genes. This situation is quite opposite for the NSC algorithm in most scenarios.As can be seen from the figures, an increase in the sample size leads to an increase in the overall accuracies, unless the data are overdispersed. This relation is more distinct for very slightly overdispersed scenarios. However, this increase does not affect the amount of sparsity. The number of genes has a considerable effect on both accuracy and sparsity. Including more genes into classification models, mostly leads to more accurate results for PLDA  and in To provide the applicability of the developed approaches, a user-friendly web application is developed with the shiny package of R. This tool is an interactive platform, which can be accessed through Users can upload either miRNA or mRNA based gene-expression data to identify the diagnostic biomarkers and to predict the classes of test cases. Two example datasets are also available in the web-tool for the implementation of the developed approaches . After uBiomarker discovery and classification are crucial in medicine to assist physicians and other health professionals in decision-making tasks, such as determining a diagnosis based on patient data. With the use of the capabilities of next-generation sequencing technology, detecting the most relevant genes  associated with a condition of interestand developing a decision support system for clinical diagnosis will enable physicians to make a more accurate diagnosis, develop and implement personalized, patient centered therapeutic interventions and improve the life quality of patients by better treatments.In this study, we presented a sparse classifier voomNSC for classification of RNA- Seq data. We successfully coupled the voom method and the NSC classifier together by using weighted statistics, thus extended voom method for classification studies and made NSC algorithm available for RNA-Seq data. We also proposed two non- sparse classifiers, which are the extensions of DLDA and DQDA algorithms for RNA-Seq classification. We designed a comprehensive simulation study and also used four real miRNA and mRNA sequencing datasets to assess the performance of developed approaches and compare their performances with other classification algorithms. We obtained good results in both simulated and real datasets. In particular, voomNSC is able to find small subset of genes in an RNA-Seq data and provides fast, accurate and sparse solutions for RNA-Seq classification.2 gives compatible results with voomNSC classifiers in terms of classification accuracy. However, voomNSC provides sparser models, which is crucial for simpler, interpretable and low variance models. In real datasets, the accuracy results of the classifiers were comparable with each other. However, again the voomNSC classifiers provided the sparsest solutions.We compared our results with both the count based RNA-Seq classifiers and the microarray based classifiers after rlog transformation. To the best of our knowledge, count based classifiers are the only developed approaches in the literature for RNA-Seq data analysis. Using microarray based classifiers; we were able to see the effect of the voom method in classification studies. We selected the rlog transformation for microarray based classifiers, since it is good at accounting for the differences in library size. It also stabilizes the variances more accurately than a simple logarithmic transformation. In the simulation studies, the provided precision weights of voom method led to both more accurate and sparse models than obtained with microarray based classifiers. PLDA2 and voomNSC classifiers give comparable results in model accuracy. We believe that this superiority originates from the robustness of the voom methodology. This method empirically estimates the mean–variance relationships from the datawhile both PLDA and NBLDA aim to specify the exact probability distribution of counts instead. Precision weights allow us to make use of the normal distribution, since its mathematical theory is more tractable than the count distributions . In this case, the DESeq median ratio or TMM methods can be applied before model building in order to obtain sparser results. In other cases, all three voomNSC classifiers provided very similar results in both model accuracy and sparsity. This result is consistent with We also demonstrated the use of the voomNSC algorithm in diagnostic biomarker discovery problems. In the cervical dataset, voomNSC identified 14 miRNAs as biomarkers with misclassifying two out of 58 samples. We conclude that the PLDA algorithm with power transformation and voomNSC classifiers may be the sparse methods of choice, if one aims to obtain accurate models for RNA-Seq classification. SVM and RF algorithms are the overall winners in nonsparse classifiers. When sparsity is the measure of interest, voomNSC classifiers should be the preferred methods. Along with its accurate and sparse performance, the voomNSC method is fast and applicable to even very large RNA-Seq datasets. Besides the prediction purpose, the voomNSC classifier can be used to identify the potential diagnostic biomarkers for a condition of interest. In this way, a small subset of genes, which is relevant to distinguishing the different classes, can be detected. These genes can then be investigated for further, such as discovering additional genes which have interactions with these genes. We leave extending this model with considering the known biomarkers as a follow-up research study.http://www.biosoft.hacettepe.edu.tr/voomDDA/.We believe that this study may contribute to other studies in proposing the voom extensions of powerful statistical learning classifiers including SVM, RF, etc. We also recommend extending this approach for other types of statistical analysis methods such as clustering analysis. These generalizations may allow users to analyze both microarray and RNA-Seq data with similar workflows and provide comparable results. For the applicability of the proposed approaches, we developed a user-friendly and easy-to-use web-based tool. This tool can be accessed through 10.7717/peerj.3890/supp-1File S1Click here for additional data file.10.7717/peerj.3890/supp-2File S2Click here for additional data file.10.7717/peerj.3890/supp-3File S3Click here for additional data file.10.7717/peerj.3890/supp-4File S4Click here for additional data file.10.7717/peerj.3890/supp-5File S5Click here for additional data file.


Nearest shrunken centroids (NSC) is a popular classification method for microarray data. NSC calculates centroids for each class and “shrinks” the centroids toward 0 using soft thresholding. Future observations are then assigned to the class with the minimum distance between the observation and the (shrunken) centroid. Under certain conditions the soft shrinkage used by NSC is equivalent to a LASSO penalty. However, this penalty can produce biased estimates when the true coefficients are large. In addition, NSC ignores the fact that multiple measures of the same gene are likely to be related to one another. We consider several alternative genewise shrinkage methods to address the aforementioned shortcomings of NSC. Three alternative penalties were considered: the smoothly clipped absolute deviation (SCAD), the adaptive LASSO (ADA), and the minimax concave penalty (MCP). We also showed that NSC can be performed in a genewise manner. Classification methods were derived for each alternative shrinkage method or alternative genewise penalty, and the performance of each new classification method was compared with that of conventional NSC on several simulated and real microarray data sets. Moreover, we applied the geometric mean approach for the alternative penalty functions. In general the alternative (genewise) penalties required fewer genes than NSC. The geometric mean of the class-specific prediction accuracies was improved, as well as the overall predictive accuracy in some cases. These results indicate that these alternative penalties should be considered when using NSC. Nearest shrunken centroids (NSC) is one of the most frequently used classification methods for high-dimensional data such as microarray data , 2. NSC L1 penalty  of Tibshirani [L∞-norm penalized NSC (ALP-NSC) and adaptive hierarchically penalized NSC (AHP-NSC). They showed that the two NSC methods have better performance than the original NSC in terms of misclassification error rate and the number of variables with nonzero centroids. However, ALP-NSC requires an exhaustive search to find an index set satisfying certain condition. If no such indices exist, quadratic programming must be employed to estimate the parameters. AHP-NSC requires an iterative procedure to estimate the parameters, and this increases the computational burden as the number of genes increases.Wang and Zhu  showed tbshirani . They obWhile Wang and Zhu  sought tAnother way to improve NSC is to modify the way to select an optimal threshold as in Blagus and Lusa . They imIn this article, we proposed the methods that improve NSC through alternative shrinkage of the class centroids. Like Wang and Zhu , we usedBlockwise additive penalties, which were discussed in Antoniadis and Fan , were shIn the Methods section, we described the penalized least squares framework for general shrinkage methods using the model of Wang and Zhu , which ixij be the gene expression for the jth gene of the ith sample . There are K classes and each sample i belongs to one of K classes, that is i ∈ Ck, where Ck is the set of sample indexes belonging to class k ∈ {1, …, K}, and nk is the number of samples for class k. The average expressions for the jth gene in the kth class and over the entire data set are Adapting the framework of Wang and Zhu , let xijsj is the pooled within-class standard deviation for the jth gene:sj + s0 can be used instead of sj to prevent the genes with low expression levels from having large sj values, where s0 is a small constant. The statistic dkj, in Tibshirani et al. [jth gene comparing class k to the average of the other classes.Letzik = 1 if sample i belongs to the class k, and 0 otherwise, μkj is a parameter to be estimated and εij is an independent error term that has variance i belongs to class k. For a fixed gene index j, μkj is a deviation from the overall mean, so we have the constraint that i belongs is denoted by k(i) ∈ {1, …, K}. By multiplying AT denotes the transpose of a vector or matrix A. The design matrix W =  is an np × Kp matrix, where Wl is a np × 1 vector that corresponds to the lth element of the vector μ for l = 1, …, Kp. If an index l belongs to a class index k and a gene index j, then nk(i) elements of Wl are nk(i) samples belonging to class k(i). This implies that W has only one non-zero value and the rest of the elements are zero. This is because each yij takes only one μkj, and this implies l ≠ h). Thus, W is orthonormal. Note that μ0 = WTy* is the least squares estimator for μ, and let W is orthonormal, a form of the penalized least squares is given by [pλ(⋅) = λp(⋅) is a penalty function and A = T. The problem of minimizing μkj is equivalent to minimizing it componentwise. By ignoring Let pλ(|μkj|) = λ|μkj| [μkj is given byz+ is the positive part of z. The LASSO solution is equivalent to the soft shrinkage estimate [μkj under alternative penalties are presented in the next subsection.= λ|μkj| , and theestimate . The resk asπk = nk/n is a prior probability estimate for class k. The shrunken mean and the discriminant score depend on the shrinkage method used, hence the choice of shrinkage method affects class prediction and gene selection. Finally, the classification rule is given byTo predict the class of a new sample a > 2. This penalty function gives smaller penalties on larger coefficients. The resulting estimator for μkj isHere, we described several shrinkage methods that are possible alternatives to soft shrinkage. The first order derivative of the SCAD penalty function  is defina is close to 2, then SCAD behaves like a hard shrinkage estimate when estimating μkj.If a > 0. The resulting solution isThe adaptive LASSO penalty function , which ia = 0 and is similar to the nonnegative garotte when a = 1 [The adaptive LASSO solution is equivalent to soft shrinkage when en a = 1  . Thus, they overcome the tendency of soft shrinkage to produce biased estimates.As mentioned previously, these shrinkage methods are known to have oracle properties under some mild conditions . In our subsequent analysis, six values of the tuning parameter a were examined for each ALT-NSC and genewise shrinkage method:  for the adaptive LASSO penalty,  for the SCAD penalty and  for the MCP penalty. For each method, thirty values of λ were considered. For the case when there are ties among the CV prediction accuracies or g-means, we chose the parameters resulting in a smaller number of genes.While soft shrinkage has one tuning parameter μj = T denote a K × 1 mean vector for the jth gene. Further let Here we extend the shrinkage methods discussed in the previous subsection to genewise inference. Let μkj, we penalize the vector μj. Using the fact thatW, Note that instead of penalizing The solution to et al. [Using the result of Antoniadis et al. , the solpλ(⋅), we can derive genewise shrinkage methods under diverse penalty functions. Note that the problem of solving Since When the LASSO penalty is employed,a > 2. For the adaptive LASSO penalty, the resulting solution isIf the SCAD penalty is used,a > 1.For the MCP penalty,The thresholding rules of the genewise shrinkage methods are determined by a, λ) of the (genewise) althernative shrinkage methods to be those that maximize the geometric mean of the class-specific prediction accuracies.Adpating the idea of Blagus and Lusa , we consThroughout the remainder of this manuscript, we will refer to the genewise version of each method by adding “G” to the beginning of its abbreviated name. Moreover, when the tuning parameters are determined by the geometric mean, we will add “GM-” to the beginning of the name. For example, “GADA” refers to the genewise version of adaptive lasso, and “GM-GADA” referes to “GADA” whose tunnig parameters are determined by the geometric mean.In this section, we conducted simulation studies to compare ALT-NSC, GEN-NSC, and the GM versions of ALT-NSC and GEN-NSC to conventional NSC. We examined the overall prediction accuracy (PA), geometric mean (g-mean), area under the curve , sensitivity (SEN) and positive predictive value (PPV). SEN is the number of detected important genes divided by total number of important genes. PPV is the number of detected important genes divided by total number of genes the method selects. As in Dudoit et al. , we presn1 = nπ1 and n − n1: MVN and MVN, each had a dimension of p = 2500. μ1 was equal to 0 for all genes and μ2 was 0.5 for 100 genes and 0 for the rest of genes. The differentially expressed (DE) 100 genes were randomly selected. As in Guo et al. [Σ was a block diagonal matrix with each diagonal block Σρ having an auto-regressive structure and alternating in sign. The block size was 50 × 50 and there were 50 blocks, which gave a total of 2500 genes:ρ took values of 0.5 and 0.9, indicating sparse and dense correlation blocks, and π1 took values of 0.5 and 0.8, corresponding to class-balance and -imbalance.In a two class classification scenario, we generated two classes from multivariate normal distributions with sample sizes, o et al. , Σ was aπ1, π2, π3) = : MVN, MVN and MVN, each of which had the same dimension as the previous scenario. Ninety differentially expressed genes were randomly selected and those DE genes had mean vectors of . We used the same Σ as in the first simulation. We let γ take the values of 0.5 and 0.1 to study how the effect size of DE genes is related to the performances of the classifiers.The three class classification scenario is very similar to the previous one. We generated three classes from multivariate normal distributions with the fixed proportions , only the ALT-NSC had better PA and g-mean, but GEN-NSC and GM methods showed no improvement. Under the very small effect size, all the classifiers performed very similarly in terms of PA, but their performance varied with respect to g-mean. MCP had the highest g-mean, and the other penalty functions gave zero as the median quartile of g-mean. Secondly, GM significantly improved g-mean for all the penalty functions, and the amount of the improvement was greater when the genewise penalties were used for the sparse block matrix. Finally, gene selection was also improved by GM: SEN increased and PPV stayed at almost the same value, compared to the corresponding methods based on the cross-validation prediction accuracy criterion.Simulation results for the three-class scenario have been presented in Tables In this section, we applied conventional NSC and the proposed methods (ALT-NSC and GEN-NSC) to four real microarray data sets. The main characteristics of the four microarray data sets are presented in The Gravier et al.  data seta, λ) as the values that give the maximum of 5-fold CV prediction accuracy or g-mean under the training data set. We compared prediction accuracy, g-mean, AUC (only for the Gravier data set) and the number of selected genes.We randomly split each data set into a training set and a test set with 33% of the data allocated to the test set. This process was iterated 100 times. We chose optimal tuning parameters  or prediction was easy (Yeoh data set), only gene selection was improved by the alternative penalty functions. In the data set with the moderate number of classes (Pomeroy data set), g-mean was improved by the alternative penalty functions. When the data set had very large number of classes (Ramaswamy data set), using the genewise penalty functions reduced the performance.In many applications, it is desirable to develop classifiers that use the smallest possible number of genes. For example, one may wish to use an RT-PCR assay to discriminate between different types of tumors or to determine the prognosis of a patient with a given tumor type. Such an assay will be prohibitively expensive if the expression levels of more than a handful of genes are needed. Thus, a classification method that produces comparable accuracy to another method using fewer genes would be considered superior in these situations. Hence, the fact that our proposed methods consistently use fewer genes than conventional NSC represents a significant advantage of our methods even if prediction accuracy is not always improved. MCP would be very useful in real applications becacuse they have shown to select the most reliable parsimonious gene set with competitive predictive accuracy.Both simulation and real data studies showed that our proposed methods produced greater improvement compared to conventional NSC in the data sets with three or four classes, but not in data sets with very large numbers of classes. When the number of classes is large, the sample size per class is usually small, and this affects the efficiency of shrunken mean estimators. By the virtue of the oracle property, ALT-NSC can produce more efficient estaimtes of the shrunken means, which yields better performance on both prediction and gene selection. Genewise shrinkages also improve the NSC classifier by combining the related genes in the same class, producing more accurate estimates when the size of the class is small (which commonly occurs when the number of classes is large). Clearly, the genewise penalty (GEN-NSC) shrinks a mean estimator toward zero faster than the non-genewise penalty (ALT-NSC), as shown in the simulations and the real data study. Appropriately fast shrinkage will be able to remove noisy genes effectively. However, one observes that when the number of classes is large, such as the Ramaswamy data set, the amount of shrinkage produced by the genewise penalty is so large that NSC loses some prediction accuracy. Thus, applying GEN-NSC to data sets with too many classes may not be recommended when the objective is to maximize predictive accuracy (rather than minimize the number of selected genes).The performance of the methods can be affected by heterogeneity of gene expression, and this heterogeneity happens when the variances of genes differ by groups. This was pointed out by Tibshirani et al.  and was S1 RscriptThis file contains the R functions that implement ALT-NSC and GEN-NSC.(ZIP)Click here for additional data file.