In silico analysis indicated that the content of our panel was capable of accurately segregating bladder cancers from several public datasets into the clinically relevant basal and luminal subtypes. On a technical level, our bladder cancer panel yielded robust and reproducible results when analyzing formalin-fixed, paraffin-embedded (FFPE) tissues. We applied our panel in the analysis of a novel set of 204 FFPE samples that included non-muscle invasive bladder cancers (NMIBCs), muscle invasive disease (MIBCs), and bladder cancer metastases (METs). We found NMIBCs to be mostly luminal-like, MIBCs to include both luminal- and basal-like types, and METs to be predominantly of a basal-like transcriptional profile. Mutational analysis confirmed the expected enrichment of FGFR3 mutations in luminal samples, and, consistently, FGFR3 IHC showed high protein expression levels of the receptor in these tumors. Our bladder cancer panel enables basal/luminal characterization of FFPE tissues and with further development could be used for stratification of bladder cancer samples in the clinic.In the age of personalized medicine stratifying tumors into molecularly defined subtypes associated with distinctive clinical behaviors and predictable responses to therapies holds tremendous value. Towards this end, we developed a custom microfluidics-based bladder cancer gene expression panel for characterization of archival clinical samples.  Bladder cancer is the fifth most common malignancy worldwide, with close to 400,000 newly diagnosed cases and ~150,000 associated deaths per year . Approxiet. al. described subsets of MIBCs that were transcriptionally reminiscent of breast cancers, including a luminal subtype characterized by a high frequency of activating FGFR3 mutations and a favorable outcome, and a basal subtype that exhibited a low frequency of FGFR3 mutations and was associated with a poor prognosis [Transcriptionally defined classes of bladder cancers were identified and have been shown to be associated with distinctive clinical behaviors and responses to therapies –15. Accorognosis . Consistrognosis . The sigrognosis , 16, 17,In silico, we demonstrate that the content of our panel is capable of distinguishing between basal and luminal bladder cancers. We employ our panel in the analysis of FFPE tissues from a novel cohort of 204 tumors and show that NMIBCs from our sample set are mostly luminal-like, carry FGFR3 mutations at high frequency, and express high protein levels of the receptor tyrosine kinase. On the other hand, MIBCs from our cohort are mixed with respect to luminal- and basal-like status, and the expression patterns seen in METs are consistent with a more basal-like transcriptional state.In this study, we introduce a novel microfluidics-based panel that is custom designed to measure the expression of bladder cancer-relevant genes in FFPE tissues. TP53, genes involved in the regulation of cell differentiation and development, and epithelial-mesenchymal transition (EMT) genes [Here, we introduce a microfluidics-based bladder cancer gene expression panel that is optimized for the analysis of FFPE tissues. Our panel is comprised of 90 unique genes that were selected to capture key attributes of bladder cancer biology –31, as wT) genes –31  classifier  was traiet. al.  and Damret. al. , and thiet. al.   to evaluate the performance of each of the assays, redesigning primers when necessary, to achieve linear standard dilution curves for all tests . During Next, we evaluated assay variability by calculating coefficient of variation (CV) over the three replicate measurements that were obtained from our quality control experiments for each assay and sample combination. CV was calculated as the ratio of standard deviation over the mean expression measurement. The average CV for all assays on the panel was under 5% except for the MYC assay, which had an average CV of 13% when analyzing cell line- and FFPE tissue-derived RNA . Notably2 range = 0.91–0.98). Furthermore, we ran replicate FFPE-derived RNA samples on different days and observed a high degree of data concordance for each of the genes on the panel (2 range = 0.98–0.99). The results from our qualitative and quantitative assessments suggest that all assays on the panel, with the exception of the MYC test, were robust and yielded reproducible data when analyzing both high quality uRNA and lower quality RNA from FFPE tissues.We also assessed data reproducibly when running high quality uRNA and lower quality RNA from FFPE tissues on different runs from different days. We noted a high degree of chip-to-chip data reproducibility for each gene on our panel when we ran uRNA control samples as part of seven independent runs . We observed a significantly higher fraction of NMIBCs in the luminal-like compared to the basal-like group (P = 0.0001), consistent with the notion that NMIBCs from our cohort are mostly luminal-like in nature. MIBCs were represented in both basal- and luminal-like groups; however, a significantly higher proportion of MIBCs from our series was observed in the basal-like group (P = 0.0001). Interestingly, almost all METs clustered under the basal-like arm of the hierarchical tree, suggesting that the majority of the METs in our cohort exhibited a basal transcriptional profile (P = 0.0001).We next established a method for calculating basal/luminal similarity scores using transcriptional signatures from our FFPE samples. We began by median-centering the expression of all probes corresponding to the genes on our panel in public datasets and calculated an average gene expression value in basal and luminal groups, as assigned by Choi and Damrauer and colleagues , 14 Fig and S6B.datasets . We thendatasets . Hierarcdatasets . StatistFGFR3 in ~60–80% of the cases, and a much lower frequency of ~15% has been reported in MIBCs and METs [FGFR3 and other cancer-relevant genes in samples from cohort using a custom allele-specific PCR panel [FGFR3 mutations, a significantly higher fraction than the <20% FGFR3 mutation rate we observed in samples from the basal-like group (P<0.0001). Mutations in FGFR3 have been reported to drive higher expression levels of the RTK [P<0.0001). We did not observe significant associations between basal- or luminal-like status and any of the other mutations on our panel  had failure rates of >15%; however, even the worst performing assay (RPS6) had a success rate of 48%, indicating that these assays were still informative in about half of the samples. We excluded the MYC assay from further analysis because it was the worst performing test on the panel with a 96% failure rate, and it also exhibited a CV of 22% when we analyzed high quality uRNA samples and a CV of 13% when using RNA from FFPE tissues. When we assessed data reproducibly we observed a high degree of data concordance for the assays on our panel for uRNA replicates from independent runs. Furthermore, a high degree of data concordance was observed for each of the assays when we ran replicate FFPE-derived RNA samples on different days. Our qualitative and quantitative assessments suggest that our panel can provide robust and reproducible results when analyzing archival clinical samples.et. al. showed that NMIBCs from their cohort were mostly basal in nature [FGFR3 gene and expression levels of the RTK by IHC provided molecular confirmation of the luminal and basal status of our samples, by virtue of the high frequency of FGFR3 mutations and high expression levels of the protein that was detected in luminal samples and the low frequency of mutations and protein expression levels that was observed in the basal samples [We utilized our panel in the characterization of a novel cohort of 204 FFPE tissues and were able to identify samples that were transcriptionally similar to the recently described basal and luminal bladder cancer subtypes , 14. By n nature . The bas samples , 14, 19.This study provides proof-of-concept that our microfluidics-based bladder cancer panel can robustly classify FFPE tissues into the clinically-relevant basal and luminal disease subtypes. Further validation efforts on additional archival tissues will be required before this panel can be implemented in the clinic. By making the panel publically available at this point in time we provide the field with the opportunity to apply this rapid, robust, and cost effective approach in the basal/luminal characterization of FFPE clinical samples. This approach offers additional important advantages over global transcriptional profiling for basal/luminal classification, including relatively simple data analysis and suitability for analyzing FFPE-derived RNA, which is often the only available substrate when working with clinical specimens.The bladder cancer panel is built on a commercially available microfluidics platform, and this work is not intended for marketing or commercialization of any product. Selection of gene content was based on a comprehensive review of the literature and prioritization of pathways and related genes that have been shown to be involved in bladder cancer development , 26, 30 http://www.ncbi.nlm.nih.gov/geo/) using the GEOquery R library (version 2.34.0). Expression data was normalized using median polish [et. al. [Analyses were performed using the R statistical language (R version 3.2.2) with the Bioconductor library   we performed 5-fold cross-validation, training a diagonal linear discriminant analysis (DLDA) classifier  using alThe Damrauer discovery dataset  . The clinical samples were of a ~80%/20% male/female ratio, and the proportions of NMIBCs , MIBCs  and metastases were 45%, 30%, and 26%, respectively  following approval of the Ethics Committee of Saint Petersburg City Clinical Oncology Hospital and appropriate confirmation of written informed consent, or from The MT Group  following IRB approval  after de-paraffinization with Envirene, as described previously . RNA wasPIK3CA, EGFR, KRAS, NRAS, HRAS, FGFR3, MET, BRAF, KIT, AKT1, FLT3 were detected using mutation specific qPCR as described previously [Mutation analyses were carried out on genomic DNA extracted from macro-dissected FFPE tissues using the QIAamp FFPE kit  after de-paraffinization with Envirene . Mutatioeviously .IHC was performed on 4-μm thick FFPE tissue section using the Ventana Discovery XT Autostainer platform . For the detection of FGFR3, the slides underwent pretreatment using CC1 extended antigen retrieval followed by anti-FGFR3, clone 15C3 (Genentech), primary antibody diluted to 1 μg/mL and incubated for 60 minutes at room temperature. For amplification of FGFR3 signal, an unconjugated rabbit-anti-mouse linker antibody  was applied at 1 μg/mL and incubated for 32 minutes at room temperature. This was followed by an anti-rabbit-OMNIMAP-HRP kit . Sections were counter stained with hematoxylin, dehydrated, cleared and cover-slipped for viewing.Basal/luminal similarity scores for FFPE samples were determined by first deriving basal/luminal expression profiles from the public Damrauer discovery data set, as previously described . Next, bS1 Fig(TIF)Click here for additional data file.S2 FigNumber of classifier genes ranging form 10–80 was used to train a DLDA classifier on 80% of samples from each public data set, and misclassification rates were calculated on the remaining 20% of the samples. Procedure was repeated 5 times through cross-validation to calculate average misclassification rates and to estimate standard errors.(TIF)Click here for additional data file.S3 FigMutation data based on the Choi study is also provided.(TIF)Click here for additional data file.S4 Fig(TIF)Click here for additional data file.S5 FigStandard deviations over the mean expression values was calculated for each of the assays on the panel.(TIF)Click here for additional data file.S6 Fig(A) Heatmap showing unsupervised clustering of genes and samples from public data sets, as well as NMIBCs, MIBCs, and METs from a novel FFPE tissue cohort. White blocks represent genes that are not found in the respective data sets. (B) Basal profile (red bars) and luminal profile (blue bars) for bladder cancer panel genes calculated from the Damrauer discovery samples. Mean-centering and unit variance normalization was applied to the log-transformed expression values, and mean log normalized expression levels were calculated independently for basal and luminal sample groups to form the final profiles.(TIF)Click here for additional data file.S1 Table(XLSX)Click here for additional data file.S2 Table(XLSX)Click here for additional data file.S3 Table(XLSX)Click here for additional data file.


Despite wide applications of high‐throughput biotechnologies in cancer research, many biomarkers discovered by exploring large‐scale omics data do not provide satisfactory performance when used to predict cancer treatment outcomes. This problem is partly due to the overlooking of functional implications of molecular markers. Here, we present a novel computational method that uses evolutionary conservation as prior knowledge to discover bona fide biomarkers. Evolutionary selection at the molecular level is nature's test on functional consequences of genetic elements. By prioritizing genes that show significant statistical association and high functional impact, our new method reduces the chances of including spurious markers in the predictive model. When applied to predicting therapeutic responses for patients with acute myeloid leukemia and to predicting metastasis for patients with prostate cancers, the new method gave rise to evolution‐informed models that enjoyed low complexity and high accuracy. The identified genetic markers also have significant implications in tumor progression and embrace potential drug targets. Because evolutionary conservation can be estimated as a gene‐specific, position‐specific, or allele‐specific parameter on the nucleotide level and on the protein level, this new method can be extended to apply to miscellaneous “omics” data to accelerate biomarker discoveries. Treatment outcomes were recorded as complete response or resistance to induction therapy. Each patient was measured on 40 clinical covariates describing demographic, cytogenic, mutation status, and the results of several standard blood tests. Proteomic data were available for each patient sample obtained prior to treatments. The proteomic features represent levels of 231 total or phosphorylated proteins, focusing on proteins involved in apoptosis, cell cycle, and signal‐transduction pathways. Seventy‐nine of these proteins have confirmed roles in oncogenesis and cancer progression , as annotated by the Cancer Gene Consensus list in the COSMIC database  Challenge . This dataset consisted of 401 patients who were diagnosed with prostate cancer and received prostatectomy  of each position in a human protein sequence. Given a human protein, we retrieved multiple sequence alignments of its orthologs in 46 species available from the UCSC Genome Browser  was estimated as the average r over all positions (R=1L∑i=1Lri).Using the Fitch algorithm . The selected biomarkers are then used to build a predictive model . It can be achieved by employing (i) a deliberately designed weighting schema, (ii) an effective feature selection algorithm, and (iii) a robust classification model.2.3.1R) as the evolutionary weight (WE). For clinical covariates, there is no meaningful score of evolutionary conservation. Because clinical features tend to have higher predictive power than molecular features in general . In the presence of multiple classes, other statistical tests such as F test can be used. p values from these tests were transformed via negative logarithm (−log(p)) and used as the statistical weight (WS). For each feature i, the final weight was the sum of evolutionary weight and statistical weight (Wi = WEi+ WSi). In this study, we assumed equal contribution of evolutionary conservation and statistical association to the final weights. However, their relative contributions can be adjusted based on the understanding of a specific cancer phenotype.Because fast evolutionary rate indicates low conservation, we used its reciprocal . We then transformed this feature matrix by multiplying Wi for each feature. This weighted feature matrix fijw was subjected to feature selection  is equivalent to the following problem:where In equation (2), a larger penalty is imposed on features with a small weight. Consequently, the solution will favor the features with a large weight.x requires the selection of the most appropriate regularization parameter (λ), which dictates the number of features selected . To reduce such dependence, we employed a stability selection method. In particular, 100 bootstraps were performed to identify features that are consistently selected in more than 50% of runs of the algorithm with different λ values.In equation (1) and (2), the calculation of 2.3.3fij was used to avoid biases. The classification model was a random forest with 50 trees, as implemented in the TreeBagger function in Matlab (version R2013a). While we chose to use random forest for classification, other linear or nonlinear algorithms can be employed as well.A classification model was constructed with selected features Fig. C. In thi2.3.4To avoid bias caused by the imbalance of class size  and area under the receiver operating characteristic (AUROC) to assess the predictive accuracy of a model. These two parameters are robust to the imbalance of class size, and thus commonly used and well documented , in which composite weight, only evolutionary weight, only statistical weight or no weight was used during feature selection, respectively. The rest of the algorithm was kept the same. Our results showed that Me+s achieved the highest performance, with up to 11.0% increase on balanced accuracy and 0.102 increase on AUROC as compared to other models , reflecting high noise level in proteomic data. Therefore, algorithms that solely rely on statistical associations to choose biomarkers from “omics” data may suffer from over‐fitting, as reported by other studies as well  = 4.95, p = 4.5 × 10−5) and 4% higher AUROC values (paired t(19) = 4.08, p = 3 × 10−3) than M0. The improvement became insignificant when the complexity of a model increased and reached 40 genes in each submodel. While the best performance of Me+s is similar to that of M0 , M0 used twice as many features as Me+s (number of features included in each submodel: 40 vs. 20).In this study, we applied the evolution‐informed modeling and evaluated its performance by followed a strict cross‐validation procedure. Specifically, we randomly chose 80% of the samples for training and used the other 20% for independent testing. This procedure was repeated 10 times, and the averages of balanced accuracy and AUROC values estimated from the test datasets were reported. For each iteration, we built an Mest Fig. A,B. If oe+s model and in the top‐performing M0 model, respectively. Most genes (80%) in Me+s were also present in M0, while M0 contained 217 additional genes , t(396) = 3.11, p = .002, Fig. t test on log, t(499) = 3.30, p = .001, Fig. e+s have higher conservation (t test on log(evolutionary rate), t(150) = −5.33, p = 10−7) and stronger statistical association , t(144) = −10.1, p = 10−18). They are significantly enriched in biological processes that have been previously implicated in metastasis and tumor progression, such as DNA repair, cell cycle, and DNA metabolism (Table S2).We further examined genes used in models with the best performance. Summarized over all submodels, 128 and 319 unique genes were included in at least one submodel in the top‐performing Mnes Fig. C. Comparvalue), t99 = 3.304As one of the leading causes of morbidity and mortality in the modern world, cancer has become a major problem in public health. Accurate prediction of a patient's response to treatment and prognosis can greatly assist clinicians to choose appropriate therapy and help improve patient care. High‐throughput biotechnologies have generated a large amount of “omics” data that can be used for this purpose. However, the high noise level in these data impairs the usage in identifying reliable biomarkers. Further, the number of samples tested in an “omics” study is usually several orders of magnitudes smaller than the number of molecular features measured, which makes traditionally derived statistical models prone to overfitting. In fact, our analysis showed that statistical scores tended to describe random error or noise instead of the true underlying relationship in omics data. Consequently, these models are hard to interpret and lack generalization capability.To reduce the noise, we studied the possibility of using evolutionary conservation to prioritize functionally important genes as predictive biomarkers. Evolutionary selection at the molecular level is nature's test on functional impact of genetic elements Kimura, . CompareGene expression profiles and protein expression profiles modeled in this study are molecular changes downstream of genomic alterations. Genomic aberrations play critical roles in carcinogenesis and fuel tumor heterogeneity in and between patients. Such high molecular heterogeneity forms the foundation of diverse clinical outcomes and other cancer phenotypes, as well as makes hunting of cancer driver mutations very challenging Heng, . Our preAnother aspect of cancer evolution is subclonal evolution within a tumor (Greaves & Maley, Meanwhile, biomarker discovery shall not leave out clinical covariates that have been associated with cancer treatment outcomes in numerous studies. One difficulty we encountered in incorporating clinical covariates in evolution‐informed modeling was the calculation of meaningful and distinctive priorities for them. Currently, we rely on statistical weights computed from the training data, which do not reflect the rich domain knowledge. In the future, we will consider deriving scores from meta‐analysis, which may serve as a better surrogate of prior knowledge aggregated from existing studies. By integrating multisource omics data and clinical features and comparing evolutionary contributions and statistical contributions to clinical outcomes, we will gain new insights into the causes of cancer formation and progression.This study was conducted using publicly available data that can be downloaded from the DREAM Challenge website and the Gene Expression Omnibus database. Click here for additional data file.


This paper focuses on the feature gene selection for cancer classification, which employs an optimization algorithm to select a subset of the genes. We propose a binary quantum-behaved particle swarm optimization (BQPSO) for cancer feature gene selection, coupling support vector machine (SVM) for cancer classification. First, the proposed BQPSO algorithm is described, which is a discretized version of original QPSO for binary 0-1 optimization problems. Then, we present the principle and procedure for cancer feature gene selection and cancer classification based on BQPSO and SVM with leave-one-out cross validation (LOOCV). Finally, the BQPSO coupling SVM (BQPSO/SVM), binary PSO coupling SVM (BPSO/SVM), and genetic algorithm coupling SVM (GA/SVM) are tested for feature gene selection and cancer classification on five microarray data sets, namely, Leukemia, Prostate, Colon, Lung, and Lymphoma. The experimental results show that BQPSO/SVM has significant advantages in accuracy, robustness, and the number of feature genes selected compared with the other two algorithms. Nowadays, cancer has been one of the most common lethal factors for human beings. Missed and mistaken diagnosis sometimes makes people lose the best chance for appropriate treatments. Therefore, more auxiliary measurements are needed to promote the accuracy of cancer diagnosis and clinical test combined with medical ways –4. With It was Golub et al. who first employed gene expression data for cancer classification . They prCurrently, there are two categories of the methods of obtaining feature genes for cancer classification based on gene expression data, namely, feature transformation methods and feature selection methods. By definition, feature transformation refers to a way of transforming the original feature attributes into a new set of features that represent the original features to the greatest extent but reduce the dimension as much as possible in order to achieve the purpose of dimension reduction. This means that the new features are low-dimensional features with similar classification abilities. Feature transformation methods for cancer classification by using gene expression data include principal component analysis (PCA) , kernel Feature transformation methods can indeed reduce the dimension for gene expression data and can eliminate the “curse of dimensionality” phenomenon due to large number redundant genes so that they can help to establish effective cancer classification models. However, the new features obtained by feature transform property no longer have the original biological meaning; that is, the methods destroy the biological information of the original gene expression data, which makes it impossible to determine the target genes associated with the cancer. For this reason, feature gene selection methods have attracted more attention.t test [The feature gene selection uses an optimization algorithm to select a subset of the genes, which has the most classification information, from the original gene microarray data. The most commonly used feature gene selection methods can be divided in to filter, wrapper, and embedded ones. Filter algorithm is independent of the subsequent learning algorithm but uses some criteria for scoring gene subsets, which measure the contribution of the genes to classification. Such methods generally use SNR , t test t test , the cort test , mutual t test , relief t test , informat test , or Fisht test . ObviousDifferent from filter methods, wrapper methods combine gene selection and classification method and use training accuracy of the learning algorithm to assess the subset of features to guide gene selection. Such methods include the sequential random search heuristics , random Embedded methods are extension of wrapper approaches and undertake feature selection in the process of classifier training, without dividing the data set into a training set and a validation set. Typical embedded algorithms include decision tree  and artiIn this work, we propose a new method, which couples a binary quantum-behaved particle swarm optimization with SVM approach, to select feature gene subset from cancer microarray data. In order to prove the advantages of BQPSO/SVM, we also implement two other algorithms, BPSO/SVM and GA/SVM. The BPSO and GA used in this work are both the original version. These two algorithms or improved ones were used in this case by other scholars early in –32. All This paper is structured as follows. In δ potential well is employed to sample around the personal best points and then introduced the mean best position into the algorithm [PSO algorithm is a population-based evolutionary search technique, which was firstly proposed in . Social lgorithm –37.X1(1011001010) is the first particle and X2(0010010110) is the second one; they all have two substrings (two decision variables), and the distance is defined as the Hamming distance between two binary strings; namely,dH(·) is the function to get Hamming distance between X1 and X2, which is the count of bits different in the two strings; the distance is seven in i is represented as Xi = , and it has D decision variables, and Xid refers to the dth substring (dth decision variable) of the position of the ith particle. Given that the lengths of Xid and Xi are ld and l, respectively, then we can get equation as follows:Based on our previous work in , in thismbest) position of all particles is determined by the states of the bits of all particles' pbest. In detail, for jth bit of the mbest, if 1 appears more often than 0 at the jth bit of all pbest, the jth bit of mbest will be 1; otherwise the bit will be 0. However, if 1 and 0 have the same frequency of occurrence, the jth bit of mbest will be set randomly to be 1 or 0, with probability 0.5 for either state. The function for obtaining mbest is called mbest = Get_mbest (pbest). The pseudocode of the function for obtaining mbest is given in In the BQPSO, the mean best , and this definitely make Pi lay between pbestid and gbestd as well. The function getting mbest in BQPSO is called Pi = Get_P .Update equation of the particle position in the original QPSO is given by e BQPSO,  can be wdH(·) is Hamming distance, b must be an integer, which is the reason for the use of function ⌈·⌉. New string Xi is obtained by the mutation of Pi with the probability computed byBecause ld is the length of substring Xid. Function getting Xid is denoted as Xid = Transf . The transformation of Transf  is described in In , here lmbest (pbest), Get_P , and Transf .The BQPSO can be summarized as Get_Support vector machine proposed in  is a tecw‖ so that the problem of constructing the hyperplane is converted to the following optimization problem: a > 0 is known as the Lagrange coefficient. Solving the Lagrangian dual of the problem, one obtains a simplified problem:Theoretically, SVM is able to correctly classify any linearly separable data. Consider the data with two classes, which can be expressed ast satisfyyiw·xi+b≥problem: min ϕw=12In many bioinformatics problems the number of features is significantly larger than the number of samples. In order to improve the classification or to help to recognize interesting features in noisy environments, tools for reducing the number of features are indispensable. The hybrid BQPSO/SVM approach proposed in the following contributes especially in this sense.t-test on the data, order the genes by p value ascending, and get 50 top-ranked genes from all. Afterwards, most of the noisy data have been removed. These 50 genes comprise the whole search space of the BQPSO algorithm for gene selection.First of all, the data should be preprocessed. Normalization of data must be conducted so as to eliminate the impact of the dimensionless on the classification. Then we need to take traditional For the BQPSO used in this work, the swarm sizes for the BQPSO and BPSO were set to be 20 and the population size for GA was also 20. Each particle has just one decision variable, and thus the dimension of the particle is just one. The length of the particle is 50, so every particle is a binary string with length of 50, and 1 represents that this gene is chosen and 0 is not. Feature gene selection and cancer classification based on hybrid BQPSO/SVM algorithm can be described as the procedure in Xi is a binary string representing a gene subset in BQPSO/SVM, the evaluation of each particle is executed by the SVM classifier to assess the quality of the represented gene subset. The fitness of a particle Xi is calculated employing a leave-one-out cross validation (LOOCV) method to calculate the accuracy of SVM trained with this subset. In leave-one-out cross validation, one of all samples is evaluated as test data while the others except this one are used as training data, repeated until all samples have been used as test data. The classification accuracy of LOOCV is the average accuracy of n times classifying, if the data set has n samples. The evaluation function is described in α and β are weight values and set to 0.6 and 0.4, respectively, for the purpose of controlling that the accuracy value takes precedence over the subset size, since high accuracy is preferred when leading the search process. The target here consists of maximizing the accuracy and minimizing the number of genes (feature_number). For convenience (only maximum of fitness), the second factor is presented as 50/feature_number.Since a particle http://linus.nci.nih.gov/~brb/DataArchive_New.html. More details of these five data sets are showed in Tables There are several DNA microarray data sets from published cancer gene expression studies. Five of them were used in this paper. They are Leukemia data set, Prostate data set, Colon data set, Lung data set, and Lymphoma data set. All of them were taken from the BRB-ArrayTools in  with URLBQPSO/SVM approach was implemented on MATLAB, along with BPSO/SVM and GA/SVM. The SVM classifier used in these three approaches is based on the LIBSVM library in . For theAll experiments were carried out using a PC with Windows OS and a Pentium Dual-Core 2.60 GHz CPU, with 2 G of RAM. BQPSO/SVM, BPSO/SVM, and GA/SVM algorithms on five cancer related microarray data sets were independent executed 25 times over each data set, in order to have statistically meaningful conclusions as these three algorithms are stochastic search methods.The parameters used in BQPSO, BPSO, and GA algorithms are shown in Depending on the results of the experiments, we made analysis of results focusing on the performance and robustness, as well as the quality of the obtained solutions providing a biological description of most significant ones. We conducted the experiments for BPSO/SVM and GA/SVM in order to demonstrate the advantage of the proposed BQPSO/SVM without any other factors affecting, since in our work all these three algorithms are operated in exactly the same hardware and software environment and with the same data sets and parameters.Next, we compare BQPSO/SVM with BPSO/SVM and GA/SVM. Since these three algorithms are running in the same environment, parameters, and data sets, the results are absolutely comparable. The performance comparison shows that, compared to BPSO/SVM and GA/SVM, BQPSO/SVM has an obvious advantage. In terms of the correct rate, the search capability of BQPSO/SVM is stronger than the other two competitors.The purpose of feature selection in our work is to find small subsets with high classification accuracy. In Besides the quality of the algorithm, its ability to generate similar or identical results when executed several times is also important. One of the most important norms in assessing any proposed algorithm is robustness. It is particularly important for metaheuristics which are employed in this work. The standard deviation (std. dev.) in Among the genes listed in The second selected gene of colon data set is R87126 . The isoform B of R87126 serves as a tumor suppressor and is well known as a component of the cytoskeletal network .Finally, the best subsets of genes were found for each data set. We add up all subsets having the highest accuracy and list the selected genes. For Colon data set, the top 5 genes with the highest selection frequency of each microarray data are presented in In this paper, a hybrid technique for gene selection and classification of high dimensional DNA Microarray data was presented and compared. This technique is based on a metaheuristic algorithm BQPSO used for feature selection using the SVM classifier to identify potentially good gene subsets and is compared with the BPSO and GA. In addition, genes selected are validated by an accurate leave-one-out cross validation method to improve the actual classification.All three approaches were experimentally assessed on five well-known cancer data sets. Results of 100% classification rate and less than average 11 genes are obtained in most of our executions. The use of preprocessing method has shown a great influence on the performance of proposed algorithm, since it introduces an early set of acceptable solutions in their evolution process. Continuing the line of this work, we are interested in optimization of BQPSO/SVM in order to discover new and better subsets of genes using specific Microarray data sets.


The same framework also provides a new approach to batch correction, correcting batch effect (correctBATCH), which we have shown to be a better approach to traditional PCA-based correction. We demonstrate the utility of these methods using two different examples  by merging gene expression data from different studies after diagnosing and correcting for batch effects and retaining the biological effects. These methods, along with conventional visual inspection-based PCA, are available as a part of an R package exploring batch effect .Genome projects now generate large-scale data often produced at various time points by different laboratories using multiple platforms. This increases the potential for batch effects. Currently there are several batch evaluation methods like principal component analysis , and sometimes they fail to reveal all of the underlying batch effects. These methods can also lead to the risk of unintentionally correcting biologically interesting factors attributed to batch effects. Here we propose a novel statistical method, finding batch effect ( If this systematic bias is not removed, its effect can mask important biological differences , at worst resulting in misleading inferences and conclusions.Batch effect refers to technical variation or non-biological differences between measurements of different groups of samples. Although batch effect can be reduced by good experimental design, it is difficult to completely eradicateComBat), an empirical Bayes method for batch correction on each gene2; distance-weighted discrimination (DWD), which employs support vector machines (SVMs) to find a hyper-plane separating the batches3 (both ComBat and DWD were used by us in multiple instances7); FAbatch, an extension of ComBat with batch-specific latent variables that is only suitable when the outcome of interest is known and binary8; mean-adjustment of microarray data by batches using prediction analysis of microarrays - PAM9; gene standardisation by z-score10; cross-platform normalisation (XPN), which is based on fitting a block linear model on clusters of features and samples from the different datasets to be merged11; and finally, PCA/singular value decomposition (SVD), which searches for directions of maximal variance associated with batch effect in the data space and removes them13. The main drawback of PCA is that, if batch effect is not the greatest source of variability PCA fails as a batch correction method3. In addition, we are going to show that correcting batch effect by completely removing principal components (PCs) affected by batch can result in loss of essential non-technical information, as this variability may not be exclusively due to batch effect. At present ComBat is the standard method and it has been shown to outperform most of the available batch correction approaches1. Therefore, we have used ComBat to compare to our exploBATCH method.Many approaches have now been developed to remove batch effects from high-throughput genomic profiling datasets. Common methods include: combating batch effect  the mixture score, which uses a k-nearest neighbour-based distance metric to assess how samples from different batches mix15; b) the skewness divergence score (skewdiv), which measures the distributional differences between data from different batches8; c) average minimal distance to the other batch (avedist) which uses Euclidean distances to measure separation between batches8; and d) the Kullback-Leibler divergence score (klmetr), which assesses variability within and between batches16. However, none of these methods provide a formal statistical test to evaluate the presence of batch effect(s) in the data.There are few metrics available to investigate batch effect including: PVCA) is another batch evaluation method that identifies sources of variability in data17. Specifically, PVCA is a multi-step method that initially reduces data dimensionality using PCA followed by estimating the variability associated with batches using a linear mixed model fitted on each PC17. Finally, PVCA derives the proportion of variability associated with batch effect using the estimated batch variability from the linear mixed model and eigenvalues associated with each PC from PCA1. Although this method has been successfully applied to compare the performances of different batch correction methods, it has the following main limitations in diagnosing batch effects: (i) it involves multiple batch evaluation steps, which reduces statistical power; (ii) there is no standard approach for selecting the optimal number of PCs associated with the data; and (iii) it does not use a formal statistical test to assess the significance of the batch effects. Hence, there remains a need for methods that perform formal statistical testing to significantly evaluate/diagnose the batch effect(s) before and after batch correction.Principal variation component analysis (exploBATCH) based on PPCCA, which we originally developed to discover metabolites associated with cancer phenotypes18. Since the PPCCA framework allows for incorporation of covariates into traditional PCA, findBATCH (within exploBATCH) employs PPCCA to evaluate and detect the presence of significant batch effects by statistically testing if the samples are distributed according to batches in the principal subspace. Reese et al. developed guided PCA (gPCA), an extension of traditional PCA, to discover batch effects in high-throughput genomic data14. However, although gPCA provides a permutation-based formal statistical test of batch effect, it is a global test on all the PCs. Instead, findBATCH statistically tests every PC for the presence of batch effect. Furthermore, we have developed a new method correctBATCH (a part of exploBATCH and again based on PPCCA) for batch correction that subtracts the batch effect on each affected PC to recover the batch-corrected expression data. We evaluate this package using examples from breast  and colorectal cancer  and normal sample gene expression profiles.Here we propose a new batch evaluation and correction approach called explore batch effect  in the data. The findBATCH function selects the optimal number of probabilistic (p)PCs 18 associated with PPCCA and exploits variability associated with the batch variable to quantify and test the effect of batch(es) in the data. The findBATCH function computes 95% confidence intervals (CI) around the estimated batch effect on each pPC. Those pPCs with 95% CI values not including zero are considered to have significant batch effect. Finally, if one or more batch effects exist, correctBATCH subtracts the effect of batch on each affected probabilistic principal component (pPC) to recover the batch-corrected expression data. ComBat (standard approach for batch correction in genomics1) was included as part of exploBATCH, however, any other batch correction method can be implemented within exploBATCH when required. Both findBATCH and correctBATCH are implemented in R statistical software as exploBATCH. The exploBATCH output includes: (i) Forest plots, showing estimated batch effect(s) with corresponding 95% CIs to identify pPCs significantly associated with batch; (ii) PCA and PPCCA plots before and after batch correction for visual inspection; and (iii) batch-corrected expression data using correctBATCH or ComBat. Overall, exploBATCH provides a framework for formal statistical testing to assess and quantitate the batch effect(s), which also allows for batch correction.The main challenge in merging different datasets is detecting and correcting for systematic bias  without distorting important biological effects. The different steps involved in evaluating batch effect(s) in exploBATCH in detecting, estimating and correcting for batch effect. We also compared results of exploBATCH to other commonly used methods – ComBat and gPCA. Initially, we sought to merge gene expression data  generated from primary human breast tumors from three different studies of 70 samples . It can be clearly seen in Fig. 3. Consequently, findBATCH was applied as a formal statistical test to detect the presence of batch effect in this pooled dataset, by determining the lower dimensional representation of the data affected by batch effect. The BIC plot in Fig. findBATCH, which allow us to quantify batch effect and perform formal statistical tests) are used to assess the effect of batch on each of the five pPCs. Fig. gPCA14, and it showed a p-value less than 0.001, representing the presence of significant batch effect in these pooled dataset, which is consistent with findBATCH analysis  had very low correlation with correctBATCH (Pearson correlation coefficient = 0.35) and ComBat (Pearson correlation coefficient = 0.26) corrected data, Supplementary Figure In order to correct batch effect in this pooled dataset, we applied our ods Fig. , also co.96 Fig. . HoweverexploBATCH can be used to: (i) statistically test  and quantitate the presence of batch effect using findBATCH, and (ii) correct for batch effect using correctBATCH.Overall, this example demonstrates how exploBATCH can be used to disentangle biological variability from batch variability.When clustering gene expression data containing a mixture of different samples , tumors typically cluster away from the normal samples; here we refer to this as the “normal/tumor biological effect”. If experiments are not carefully designed, it can be difficult to distinguish biological effects from batch effects. In this example, we demonstrate how 22 and GSE2387823; Affymetrix GeneChip® Human Genome U133 Plus 2.0 Array) consisting of 52 and 58 samples, respectively,  were pooled together. Whilst all of the 52 samples from GSE18088 were CRCs, 24 of 58 GSE23878 samples were from normal tissues and the rest were tumor samples . Here the aim is to assess whether (i) findBATCH can distinguish batch effect from biological effect; and (ii) the correction of batch effect using correctBATCH retains the normal/tumor biological effect.Two gene expression datasets  as covariates in the PPCCA model. The presence of the normal/tumor variable allowed us to assess if batch correction using either correctBATCH or ComBat retained biological effect. The optimal number of pPCs for this dataset was nine . Hence, batch and biological effects are entangled in the first two PCs, which makes PCA-based batch correction challenging.Although Fig. correctBATCH, which removes the effect of batch in each pPC, as well as ComBat to assess if the two methods retain biological effect in the data after batch correction. The results of batch correction using these two methods are shown in Fig. findBATCH confirmed no significant batch affect . The performance of correctBATCH and ComBat was comparable with high Pearson correlation coefficient of 0.95  from findBATCH was associated with the normal/tumor biological effect, as shown in Fig. correctBATCH and ComBat-based batch effect correction of the same data showed additional three pPCs  allows for correction of batch effect without distorting important biological structures in the data.Overall, this example demonstrates that findBATCH and correctBATCH, within the package exploBATCH were applied successfully to the two gene expression datasets  to diagnose and correct for batch effect, respectively. The correctBATCH framework allows for removal of batch effect(s) in genomic data without compromising biological effect, provided that the experiments are designed to properly distinguish between batch and biological effects.Here, to our knowledge for the first time, we establish a method to evaluate or diagnose batch effect(s) in genomic data at the level of individual PCs. Our method allows for both visual inspection and formal statistical testing of batch effect(s) before and after batch correction. The two methods, 19, GSE1378720, and GSE2359321, are human breast cancer samples. The gene expression profiles for these breast cancer samples were performed using Affymetrix GeneChip® Human Genome U133 Plus 2.0 Array. The replicate samples were removed from GSE23593 as indicated in GEO Omnibus leading to a total of 18 samples. The samples were pre-processed and normalized using robust multi-array average (RMA)24 using R and Bioconductor25. One sample (GSM346904) from GSE13787 had normalized unscaled standard error  median score greater than 1.05, which was removed, leading to a total of 22 samples. GSE12763 had 30 samples. A single probe with highest variation was selected for those genes with multiple probes before merging different datasets. In addition, those probes with gene name not annotated by the HUGO gene nomenclature committee (HGNC)28 were removed.All the datasets used are publicly available. The three datasets in the first example, GSE1276322 and GSE2387823; Affymetrix GeneChip® Human Genome U133 Plus 2.0 Array) with 53 and 59 samples, respectively. Two samples, one each from GSE18088 and GSE23878, had NUSE median score greater than 1.05, hence were removed from analysis. All 52 samples from the GSE18088 data were primary CRC tumors whilst 24 of 58 GSE23878 samples were normal samples and rest of the samples were matched tumor samples. Again, each of the datasets was pre-processed and normalized using RMA as described above.For the second example on colorectal cancer, we chose two gene expression data sets (GSE18088yi = (yi1…yip)T, taken on a large number of p correlated variables (i.e. genes) and corresponding phenotypes (i.e. covariates), xi = (xi1…xil)T, recorded on a sample i. PPCCA18 can be used to model the relationship between the expression data matrix Y = (y1…yn)T and covariates X = (x1…xl)T where n is the number of samples and l is the number of covariates plus an intercept term. High-dimensional data point yi is modeled as a linear function of the corresponding low-dimensional probabilistic principal component (pPC) score ui = (ui1…uiq)T , whilst the pPC score, ui, is modeled as a linear function of covariates xi, plus some unexplained additional sources of variation ξi = (ξi1…ξip)T and εi = (εi1…εiq) T, respectively, where, q « p. The PPCCA model can be written as follows,W is a p × q loadings matrix, μ is a p dimensional mean vector of the data and β is a q × l regression coefficients matrix quantifying the relationship between the pPC score ui and covariates xi. The pPC score, observed errors, and the pPC errors are assumed  to be from multivariate normal distribution (MVN), ui ~ MVNq , ξi ~ MVNp  and εi ~ MVNq , where I is an identity matrix and σ2 is the residual variance. For a more detailed description of the PPCCA model see Nyamundanda et al.18.Suppose we have measurements, exploBATCH R package, to quantitate and correct batch effect, are based on the PPCCA model18. Firstly, the PPCCA model is applied to detect batch effect in the dataset. This is achieved by carrying out a formal statistical test to determine if samples are distributed according to batches in the principal subspace defined by the PPCCA model. Since the PPCCA allows for incorporation of covariates in PCA, the batch variable(s) can be tested if it is significantly associated with any of the pPCs from PPCCA using the following test statistic created under the null hypothesis of no batch effect,βbk is the regression coefficient that quantifies batch effect b on kth pPC, and SE is the corresponding standard error. If this test statistic is significant (5% significance level) for any of the pPCs it confirms the presence of batch effect in the data. Secondly, the effect of batch is removed on those pPCs significantly associated with the batch variable of interest using PPCCA (implemented as correctBATCH within exploBATCHR package) as follows,uak and uck is a vector of scores of kth pPC affected and corrected for batch effect, respectively, whilst xb is the variable defining batches. Finally, correctBATCH recovers the batch effect corrected expression data by using the PPCCA model to predict the observed data but conditioning on the scores, u = , where uc is scores of corrected and uu is uncorrected pPCs.Approaches in exploBATCH, Rcpp29 packages (such as RcppArmadillo30 and RcppEigen31), which allow calling C++ functions in R, were adopted in exploBATCH to fit the PPCCA model from MetabolAnalyze18 package. R packages such as, foreach32 and doParallel33, were implemented in exploBATCH to allow for multi-threading when selecting the optimal number of pPCs and estimating parameter uncertainty. This drastically improved the computational efficiency of exploBATCH, compared to MetabolAnalyze package that includes PPCCA. The other information and bottlenecks associated with exploBATCH implementation are available in the supplementary information .Other Supplementary information


Building a universal genomic signature predicting the intensity of FDG uptake in diverse metastatic tumors may allow us to understand better the biological processes underlying this phenomenon and their requirements of glucose uptake.n = 71) of metastatic tumors including some of the most frequent histologies, with matched PET/CT quantification measurements and whole human genome gene expression microarrays, was used to build the signature. Selection of microarray features was carried out exclusively on the basis of their strong association with FDG uptake (as measured by SUVmean35) by means of univariate linear regression. A thorough bioinformatics study of these genes was performed, and multivariable models were built by fitting several state of the art regression techniques to the training set for comparison.A balanced training set (n = 13) obtaining a performance within the 95% CI of that obtained in the training dataset (RMSE = 0.645). Significantly overrepresented biological processes correlating with the SUVmean35 were identified beyond glycolysis, such as ribosome biogenesis and DNA replication (correlating with a higher SUVmean35) and cytoskeleton reorganization and autophagy (correlating with a lower SUVmean35).The 909 probes with the strongest association with the SUVmean35 (comprising 742 identifiable genes and 62 probes not matched to a symbol) were used to build the signature. Partial least squares using three components (PLS-3) was the best performing model in the training dataset cross-validation  and was validated further in an independent validation dataset  contains supplementary material, which is available to authorized users. Another use of this technique that is gaining more acceptance in oncology is the assessment of early metabolic response to antineoplastic agents in advanced and metastatic tumors [8F FluoroAt the molecular level, FDG uptake has been related mainly to aerobic glycolysis, but a full picture of the different biological pathways involved in this process is currently lacking. While the core molecular machinery of glycolysis is widespread in all tumors, the intensity of FDG uptake is quite variable among different tumor histologies and even among the same tumor histotypes according to specific tumor characteristics , 4. SomeAs metastases are the main cause of cancer-related death, a growing interest in metastatic cancer has been recently spurred on by a more thorough characterization of the genomic landscape of these tumors . Hence, The purpose of the present study was to build a genomic signature able to predict FDG uptake intensity in a diverse population of metastatic tumors, by using an unbiased gene expression profiling not limited to a predefined set of genes, but rather using whole human genome gene expression microarrays. To achieve this goal, a different methodology from that used previously in other signatures, that were trained on a single tumor type , 15, wasA deeper knowledge of the metabolic pathways beyond glycolysis involved in FDG uptake might contribute to establish the usefulness of FDG PET/CT in indications such as the evaluation of early metabolic response with different targeted therapies.The conditions that patients should meet to enter this study were (a) a diagnosis of metastatic tumor  with a baseline FDG-PET/CT in order to evaluate the extent of disease and at a later point treatment response, (b) a fresh frozen tumor biopsy taken at the same metastatic location in which FDG uptake was measured for a gene expression microarray, that was performed within a maximum interval of 8 weeks of the FDG-PET/CT, (c) the patients had not received chemotherapy treatment in the 3 weeks prior to the inclusion in the study, and (d) patients in whom no active tumor could be identified by FDG-PET were excluded from the study. Seventy-one cancer patients, seen between July 2010 and July 2015 at Hospital Quironsalud Torrevieja , met these requirements and were retrospectively evaluated. In 3 of these patients, more than one microarray studies had been performed several months apart, but only the first one was included in this study. No other restrictions applied to the patients entering the study on the basis of sex, age, histology of the tumor, or previous treatments. Informed consents for the obtention of the diagnostic-therapeutic biopsy and for undergoing FDG-PET/CT in the patients included in this study were obtained. Approval of this study by the Institutional Review Board of Hospital Quironsalud Torrevieja  was also obtained.These 71 patients comprised the training set used to build the predictive genomic signature. A balanced proportion of some of the most frequent tumor histologies (eight tumor types comprising from 5 to 9 patients) along with a group of miscellaneous tumor types constituted this training set (see Table ≤ P1 − 1.5 × (P3 − P1); or≥ P3 + 1.5 × (P3 − P1)where P1 was the 25% percentile and P3 was the 75% percentile.Among the remaining 13 patients of the validation set, one had a value of FDG uptake just below the patient with the lowest limit in the training set, although it was not a low outlier as defined here. We called this patient sample an “influential observation.” This term is taken from the regression argot and used here with a similar meaning: how different would it be a model prediction if we were to exclude this observation. After logarithmic transformation, this observation was just a little outside the limit of the low outlier boundaries. Nevertheless, in spite of being aware of the limitations of including such a patient  for achieving an accurate prediction, we did not exclude her from the validation set in order to study the effect on the predictive accuracy of the signature of excluding this influential observation from the validation set.All patients fasted for at least 6 h prior to imaging, and pre-examination blood glucose levels were obtained. Patients were injected with 444 MBq (12 mCi) pyrogen-free 18F-FDG. Imaging was performed 90 min (± 10) later on a Biograph 6 Hi-Rez . Whole body PET/CT scanners were acquired in accordance with the HQT PET protocol. CT data was used for attenuation correction  and X-ray contrast medium was injected . All images were iteratively reconstructed using post-emission transmission attenuation-corrected datasets  5.0 mm; iterations 4; subsets 8).FDG uptake in the biopsied location was quantified. Individual tumor VOIs (volume of interest) were automatically drawn threshold-based, one for each patient. A standard VOI analysis tool provided with the scanner was used to calculate the different quantitative parameters obtained . We did not correct for partial volume effect based on the resolution of our Siemens FDG-PET scanner (< 5 mm.), considering that the minimum diameter of all the lesions studied were at least threefolds the FHWM (> 1.5 cm). The following parameters of FDG quantitation were obtained (as defined below): SUVmax, SUVmean35, SUL, SUVglu, MTV (metabolic tumor volume), TLG , and tumor to background index (T/B).The protocol followed for the obtention of the matched biopsies is the usual one at our institution and has been previously published . Total RSUVmean35 was defined based on our previous study (unpublished data) as the SUV mean in a thresholded VOI . To calculate T/B index, two identical circular ROIs (region of interest) 50% in size to corresponding VOIs were centered on the area with maximum uptake tumor localization and on the tumor-free neighboring area respectively.SUL  was calculated as follows:LBM (lean body mass) was calculated according to the formula of Janmahasatian et al. \document2 (kg/m2).BMI (body mass index): weight/heightglu (SUV corrected for the blood glucose level) was obtained as follows [SUV follows :\documenMTV was calculated as tumor volume in centimeter cube contained in the 35% thresholded VOI. TLG was calculated as (SUV mean) × (MTV).r = 0.976, p < 0.001). Given the concordance between SUVmean35 and SUVmax and to avoid redundancies, SUVmean35 was chosen. Moreover, we selected the SUVmean35 as the dependent (or outcome) variable to be predicted as the parameter representing FDG uptake because of the higher intrinsic uncertainty associated with the calculation of SUVmax and also for the better reproducibility of SUVmean35 in accordance to our experience (unpublished data). Among these three quantification parameters mentioned above, SUVmean35 was also preferred because its calculation has demonstrated greater inter- and intra-observer reproducibility, in agreement with reports recommending the use of the SUV mean in quantifying the biological effects on tumor response [Among the different FDG quantification parameters mentioned above, a thorough descriptive statistic was carried out in the 71 patients belonging to the training set. This preliminary analysis showed that SUVmax, SUVmean35, and SUVglu had certain linearity and a data distribution close to a normal distribution as demonstrated by normality tests (Shapiro-Milk and Kolmogorov-Smirnov) and QQ-plots. However, the remaining parameters obtained  did not follow a normal distribution and were not linear. We reasoned that it was convenient to choose a representative parameter that followed a normal distribution and that showed certain linearity as some methods making use of principal components are known to have a better fit to this kind of data. Hence, we preferred to use either the SUVmax, SUVmean35, or SUVglu as continuous dependent variable (the response or outcome variable). As expected, there was a very good correlation between SUVmean35, SUVmax, and SUVglu. The Pearson correlation coefficient was highest and most significant between SUVmean35 and SUVmax , the SUVmean35 underwent a base 2 logarithmic transformation. The transformed data would be used in the elaboration of the predictive model. It is important to notice that the log-transformed SUVmean35 values from the 71 patients in the training set did not contain any outlier. To improve readability, the term SUV instead of SUVmean35 was used throughout the manuscript.p ≫ N) is the selection of the most relevant features to the response. The algorithm of supervised principal components suggested by Hastie et al. [superpc.cv and superpc.plotcv from the superpc library (created by one of the authors of [A key factor to the good performance of predictive models containing a higher number of features (probes in our case) than observations (patient samples)  from thplsVarSel.All the 909 probes selected were included to build the predictive model considering each one of them individually as predictor (independent variable). Thus, the statistical models tested were allowed to assign the coefficients  and complete linkage. A heatmap was generated with the gplots library from R.Hierarchical clustering with the selected 909 probes was performed with the function For a correct interpretation of the results presented in this work, it is worth noticing that no patient in this study had a SUV = 0 (or “negative”). Thus, when we speak below (and throughout this work) about genes  positively or negatively correlated with the SUV, what is implied is that positive refers to a higher and negative to a lower SUV. In other words, all the genes and biological processes studied here have a clear relationship with the SUV.https://david.ncifcrf.gov/) was used to study the biological processes overrepresented among the signature selected genes. To include additional biological processes less represented in DAVID, another public resource used was the Consensus Pathway Database, release 31 (http://ConsensusPathDB.org) as a complement.The DAVID Bioinformatics Resources 6.7 (https://string-db.org). The threshold used to establish the edges (interactions) among the nodes (proteins) of the PPI networks was 0.7 (“high confidence”). Two subnetworks were studied separately. One built using the genes with positive and another with those with negative correlation with the SUV (according to the sign of their regression coefficients). In addition, hierarchical clustering using a fastgreedy algorithm (done separately in the two subnetworks) was carried out with the libraries STRINGdb (http://www.bioconductor.org) as an API to the STRING database and igraph from R in order to assign membership in the two subnetworks obtained. The study of network characteristics such as those related with centrality and connectivity was done with the igraph library from R in the two subnetworks obtained.For the study and identification of potential protein-protein interaction (PPI) networks among the genes selected for the predictive signature, all the genes in each subnetwork (positive and negative correlation with the SUV) were first mapped to their respective protein products using the bioinformatics resource STRING 10.0  from R. Default parameters of this method were used as described by Barbie et al. [http://software.broadinstitute.org/gsea/msigdb/collections.jsp) was used. The scores obtained with ssGSEA for each patient and each signature used in the training set were then pairwise correlated independently with the corresponding transformed SUV values of each patient (Pearson correlation), and the corresponding correlation coefficients and probabilities were obtained for each signature of the C2 subset. To gain further insight into some specific findings obtained with the C2 subset of MSigDB v5.1, other subsets from this database were also used such as the H, C5, and C6 subsets. Using the same ssGSEA methodology described, we also used the 10 genesets containing highly selective and specific genes for 10 different cell populations. These genesets have been validated extensively in thousands of different human solid tumors  to estimate the abundance of immune and non-immune cells and have also been shown to have a good correlation with immunohistochemistry [Gene Set Enrichment Analysis (GSEA) was done using the method single-sample GSEA (ssGSEA) as implemented in the library e et al. . This mehemistry .p > 0.05 for all variables shown). The only difference with the training set was the inclusion of two aggressive locally advanced primary tumors in the validation set: a patient with a pancreatic adenocarcinoma and another with a biliary duct carcinoma.The characteristics of the patients (demographics and quantification data) and biopsies are shown in Tables p = 0.001). The SUV averages of the clusters were significantly different on account of significant differences between the higher average SUV of C1 samples versus the average SUV of the remaining clusters . The C2 vs C5 comparison was found close to significance by t test (p = 0.076). Therefore, this unsupervised methodology is indeed able to discriminate clusters of patients with statistically significant average SUV values. Furthermore, none of the major tumor types in this series was grouped in a single cluster . We found that as a group in our training set, lung cancers (7 patients) had an average SUV value significantly higher than most of the other major tumor types. However, lung cancers were evenly distributed in three different clusters. The remainder of the most represented tumor types had average SUV values that were not significantly different among them, and nevertheless, they were distributed in a minimum of two or more clusters. In addition, no statistically significant differences were found between the average SUV values of the metastatic tumors located in the liver (liver biopsies) and the rest of metastatic locations (t test p = 0.34). Likewise, the samples coming from liver metastases were widely distributed among the five clusters. Overall, these results point to the suitability of these genes as building blocks of a multivariable model to predict the SUV. As a control, hierarchical clustering using the same methodology was also applied to the training set with all the filtered unselected probes to identify five clusters. However, the SUV averages of the clusters identified with all the unselected probes were not significantly different by one-way ANOVA (p = 0.357), as shown in Table Hierarchical clustering was performed in order to check whether the selected probes (the 909 most strongly correlated with FDG uptake as measured by the SUV) were able to discriminate different groups of patients in the training set according to the SUV values and not to other clinical or pathological data. In Fig. Tables q ≤ 0.1). Regarding the genes with positive correlation with the SUV, this database unveils biological processes such as scavenging by class A receptors, DNA replication, and its regulation. Other relevant processes identified are those related to the immune system: PD1 signaling, antigen processing and presentation, CD4 T cell receptor signaling, downstream TCR signaling, phosphorylation of CD3, and TCR zeta chains among others. Previous reports have shown that a high glucose uptake is required for T cell activation [We also checked the Consensus Pathway database with the same genes . In common with DAVID, the regulation of the actin cytoskeleton scores high. However, a relevant contribution to this regulation can be envisaged in the identified biological pathways related to the small GTPases RHO, RAC1, and CDC42 as they are known potential controllers of dynamic processes affecting the cytoskeleton such as the formation of stress fibers (RHO), lamellipodia (RAC1), and filopodia (CDC42) as well as membrane ruffling (RAC1). E-cadherin signaling, integrin, integrin-linked kinase signaling, and focal adhesions also seem relevant to cell adhesion processes. Muscle and smooth muscle contraction processes cannot be overlooked as they may correlate with some of the cytoskeleton changes and with motility. Eukaryotic translation termination is also worth mentioning. Another potentially relevant group of biological processes are those related to common downstream signaling by different growth factors particularly through RAS and the RAF/MAPK cascade and last but not the least, signaling by the VEGFA-VEGFR2 pathway. Several of the processes mentioned may in fact occur in the tumor microenvironment, like those related to angiogenesis.Although less statistically significant than the aforementioned biological processes, those related to the energetic metabolism of carbohydrates were apparent: glycolysis, pentose phosphate cycle, and insulin-mediated glucose transport. In common with DAVID, protein processing and N-linked glycosylation were also apparent. As far as the genes with negative correlation with the SUV were concerned, a deeper biological insight could be obtained from the Consensus Pathway database . This means that there were more interactions among the proteins in each of the clusters considered than would be expected by chance alone in a random set of proteins of similar size extracted from the genome, suggesting a functional cooperation among them. Some potentially relevant clusters are shown in Additional file To identify in each subnetwork (positive and negative correlation with the SUV) modules of relevant functional  interactions assigning membership to each of the interacting proteins, we applied a fastgreedy clustering algorithm, disregarding the proteins with no interactions. After applying the procedure, we selected those clusters containing at least three proteins in each of the two subnetworks. Ten clusters of these characteristics were isolated in the subnetwork of proteins with positive correlation with the SUV and 16 clusters in the subnetwork of those with negative correlation. All the isolated clusters were highly significant in their PPI enrichment value, as defined by the authors of  (the clup < 0.05) are reported in Additional file r = − 0.3827), reorganization of the cytoskeleton , cell adhesion , and angiogenesis  were identified as negatively correlated with the SUV in a statistically significant manner. Another significant gene set identified as positively correlated with the SUV that is worth mentioning is the reactome_facilitative_na_independent_glucose_transporters (r = 0.2496). ssGSEA indeed identified statistically significant relevant signaling pathways missed by the other methods used above. Particularly relevant, the activation of c-MYC is positively correlated with the SUV in some of the genesets of the C2 subset of MSigDB studied . To strengthen this relationship, we also used ssGSEA with the C6 (oncogenic signatures) and the H  subsets of MSigDB v5.1. We found that the single geneset related to upregulation of MYC of the C6 subset, MYC_UP.V1_UP, was significantly associated (p = 0.031) with the SUV (r = 0.25) and that HALLMARK_MYC_TARGETS_V2 was also borderline significantly associated with the SUV . The hallmark subset contains just an additional HALLMARK_MYC_TARGETS_V1 geneset which, although not significant, it was also positively correlated with the SUV (r = 0.12). Overall, this in-silico results on MYC targets appear to suggest that MYC targets are positively associated with higher SUV levels, although experimental confirmation would be required for specific tumor types. MYC upregulation is related to the metabolism reprogramming in cancer cells, influencing a variety of aspects [In order to characterize further the biological and signaling pathways related to the uptake of FDG as measured by the SUV, we performed single sample GSEA (ssGSEA) in our whole filtered training dataset as explained in the “ aspects . Overallr = 0.29, p value = 0.013). Cytotoxic lymphocytes comprise T cells and NK  cells. This correlation appears to suggest that there is a trend towards a recruitment of these cytotoxic lymphocytes in tumors with a higher uptake of FDG. The other finding of interest was a borderline statistically significant association of the abundance of endothelial cells in tumors with lower SUV values . This result is in agreement with the findings reported above, linking angiogenesis with lower SUV values.As mentioned above in the section “n = 71) by tenfold cross-validation (CV) repeated five times (tenfold CV × 5), selecting the best parameters that for each model minimize the RMSE and maximize R2. Fifty resamples per model were generated with their respective RMSE and R2 values as metrics of performance. A summary of the results obtained is shown in Table R2 values, a t test or a Wilcoxon test (both with Bonferroni correction) were used respectively, and the results are also shown in Table R2), and there were no statistically significant differences between them. For the final selection of the model, we also took into account the number of components needed to achieve the lowest RMSE between the two best performing models. Hence, PLS requiring three components (PLS-3) was preferred over PCR (which required 18) on the basis of the statistical principle of parsimony.After the selection of features and the study of their biological meaning, we fitted and compared four different models: partial least squares (PLS), principal components regression (PCR), support vector machine (SVM), and random forest (RF) to the training set n = 1 by tenfn = 13). As a more realistic estimate of the goodness of fit in an independent validation set, we performed a tenfold CV in the training set  was 0.645 which nevertheless is within the 95% confidence interval of the tenfold CV × 5 used to select the best model (see Table n = 12) would be 0.454 and quite similar to the mean RMSE value estimated by tenfold CV × 5 (0.443 as seen in Table n = 13) and without (n = 12) the influential observation using the 909 probes of the original signature, we next tested the performance and stability of the PLS-3 signature by both reducing and increasing the number of probes.The characteristics of the patients of the validation set along with the actual measured and predicted SUV values (by PLS-3) are shown in Additional file n = 13) and without (n = 12) the influential observation is shown. All signatures were first trained by PLS-3 in the training set, and then, the model generated was tested in the validation set.In addition to the 149 and 249 probe signatures (only with positive regression coefficients) and the 201 and 301 probe signatures (only with negative regression coefficients) commented in Additional file n = 13) than in the same validation set without it (n = 12). Therefore, it is not advisable to include such observations in future testing of the predictive signature. Further, a trend towards a degradation of performance is apparent in the tested signatures with an increasing number of probes in the full validation set. However, a clear picture emerges by seeing the performance RMSE results of Fig. n = 12). The signatures containing exclusively probes with positive regression coefficients (the 149 and 249 probe signatures) perform worse than all the remainder (RMSE = 0.59 and 0.57 respectively). The signatures containing exclusively probes with negative regression coefficients (the 201 and 301 probes signatures) perform the best in the validation dataset (RMSE = 0.39 and 0.38 respectively) and perform better than the results obtained with signatures containing a mixture of probes with both positive and negative regression coefficients. Moreover, the performance of the original signature (909 probes) is not degraded by either increasing (the 0.8 and 0.9 threshold signatures) or reducing (the 1.1 and 1.2 threshold signatures) the number of probes. The RMSE for the original PLS-3 signature with 909 probes (1.0 threshold) is approximately the same (between 0.45 and 0.46) as the 0.8 (2248 probes), 0.9 (1461 probes), 1.1 (547 probes), and 1.2 (323 probes) threshold signatures. Only the 1.3 threshold signature (174 probes) has a worse performance (RMSE = 0.52).It was noticeable that all tested signatures perform worse in the full validation set containing the influential observation . These results show that the selection of probes according to VIP values is not enriched in probes matched to a symbol as compared with the whole 909 probe signature, and therefore that unmatched probes seem to also have a similar relevant contribution to the PLS-3 model as matched ones with the same characteristics of VIP values.As a measurement of importance of the PLS-3 probes, we estimated the VIP values of the third component of our PLS model, which are useful to evaluate the relative contribution of each probe to the model. In Additional file Just to mention a few putative relevant genes among the probes matched to a symbol, TEK  scores high (VIP = 2.06). TEK is a kinase that is expressed in endothelial cells and is involved in angiogenesis. Other selected probes are also related to signaling pathways involving angiogenesis such as NRP2, HGF, and FGFR1. EDNRA and EDNRB have also been found expressed in endothelial cells in different human tumors and a role in angiogenesis and cancer metastasis has been reported , 35. It n = 13) might have been a limitation of our study. Notwithstanding the PLS-3 signature has also been thoroughly validated by tenfold CV × 5 in a balanced training set (n = 71). Tenfold CV × 5 was accurate in estimating the performance of the signature in the independent validation set. The PLS-3 signature has a high stability, as shown by similar performance in the validation set using a wide number of probes  through a bioinformatics analysis of the signature genes. The knowledge acquired in this study could be of use to design specific targeted therapies based on SUV values or on its variation in response to antineoplastic treatments.MDK) [MDK is weakly correlated with lower SUV values in our training dataset, and it is known that this heparin-binding factor is also secreted by several cancer types  [As could be expected, the different bioinformatics methods used  in this study show some degree of overlapping in the biological processes identified, but they are complementary as each one also finds some relevant unique biological processes not identified by the others. This makes worthwhile the combined use of all of them for a thorough view of the biological landscape of FDG uptake. There seems to be among the biological processes correlating with lower SUV values a preponderance of biological processes related with the tumor microenvironment and its interaction with tumor cells . Of special interest are the processes related to angiogenesis and specifically neolymphangiogenesis. Recently, it has been shown that in primary melanomas, distant lymph nodes and organs may increase their lymphatic vessel density as a pre-metastatic niche to favor and promote distant metastases through the tumor secretion of midkine (encoded by MDK) . Consistrcinoma) . Hence, Among the biological processes correlated with higher SUV values, the predominant biological processes have to do mainly with the tumor compartment  and also with the immune system. These data suggest that as metastatic tumors evolve through the acquisition of new mutations towards more advanced stages of the metastatic process, which seem to require a higher glucose uptake, they also generate neoantigens capable of inducing effective T cell responses which may compete with the tumor for a higher glucose uptake.To the best of our knowledge, only a few previous reports have characterized signatures predicting FDG uptake from microarray data , 15, 39.n = 25) with NSCLC with the metagenes most significantly associated with the FDG uptake features. The range of accuracies reported in their study cohort (as defined in (14)) was from 0.725 to 0.875. Using this metric, the higher accuracy values the better performance of the model tested (maximum around 1). When using the same metric (the accuracy) in our dataset (instead of RMSE), we obtained values of 0.95 in our training set and 0.78 in our validation set. These values compare favorably with those reported by Nair et al. [The other two studies on signatures predicting FDG uptake in non-small cell lung cancer (NSCLC), which included regression studies, were from the same group , 15. In n = 63), but not in their validation cohort (n = 84). This signature predicted the SUVmax by means of a linear regression of 15 metagenes, comprising 508 genes.Only one of the signatures reported in  was founn = 850) with availability of distant metastases-free survival data (DMFS) [p = 0.001). However, in agreement with Nair et al. [We wondered whether the genes selected for the elaboration of our predictive signature were able, in an unsupervised manner, to separate patients with different prognosis. For this purpose, we performed hierarchical clustering in a previously published large microarray series of primary breast cancer patients (a (DMFS) . The clur et al.  in NSCLCIn summary, we obtained and validated PLS-3 predicting accurately FDG uptake intensity in different metastatic tumors. The PLS-3 genes allowed us to understand better the biological processes underlying the different requirements of FDG uptake in such tumors. FDG-PET might help in the design of specific targeted therapies directed to counteract the malignant biological processes more likely activated in a tumor as inferred from the SUV and also from its variations in response to antineoplastic treatments.Additional file 1:Supplementary Methods. (DOCX 29 kb)Additional file 2: Table S4.Complete list of the 909 probes selected for the generation of the multivariable model along with their correspondent regression coefficient. (DOCX 110 kb)Additional file 3: Table S6.Detailed tumor histologies of the patients in the training and validation datasets. (DOCX 17 kb)Additional file 4: Table S1.Consensus Pathway Database, release 31 (http://ConsensusPathDB.org)). (DOCX 33 kb)Biological processes related to the signature genes  subnetworks obtained from the signature genes in the STRING 10.0 PPI database. Figure S2. Goodness of fit of PLS-3 in the training set. a) Goodnes of fit including Pearson correlation of measured vs predicted SUV values b) Residuals of third component. No pattern is apparent in the residuals distribution c) Estimated goodness of fit after 10-fold CV. (ZIP 242 kb)Additional file 6: Table S2.p < 0.05) (DOCX 31 kb)Correlation coefficient (CC) on SUV of ssGSEA scores with the C2 subset from the MSigDB v5.1 in the training dataset  SUV values. (DOCX 13 kb)Additional file 8: Table S5.List of PLS-3 probes with VIP values equal or greater than 1 along with their regression coefficients. (XLSX 34 kb)


Precision oncology uses genomic evidence to match patients with treatment but often fails to identify all patients who may respond. The transcriptome of these “hidden responders” may reveal responsive molecular states. We describe and evaluate a machine-learning approach to classify aberrant pathway activity in tumors, which may aid in hidden responder identification. The algorithm integrates RNA-seq, copy number, and mutations from 33 different cancer types across The Cancer Genome Atlas (TCGA) PanCanAtlas project to predict aberrant molecular states in tumors. Applied to the Ras pathway, the method detects Ras activation across cancer types and identifies phenocopying variants. The model, trained on human tumors, can predict response to MEK inhibitors in wild-type Ras cell lines. We also present data that suggest that multiple hits in the Ras pathway confer increased Ras activity. The transcriptome is underused in precision oncology and, combined with machine learning, can aid in the identification of hidden responders. Way et al. develop a machine-learning approach using PanCanAtlas data to detect Ras activation in cancer. Integrating mutation, copy number, and expression data, the authors show that their method detects Ras-activating variants in tumors and sensitivity to MEK inhibitors in cell lines. Precision oncology matches cancer patients to specific therapies based on genomic evidence, but it has benefited only a relatively low proportion of cancer patients to date . While cKRAS and NF1 function, respectively  PanCancerAtlas has released multi-platform genomic measurements across thousands of tumors from 33 different cancer types . With thectively . Furtherectively . An unsuectively . Here, wKRAS, NRAS, or HRAS mutations or through NF1 loss-of-function events, cells increase their translational output, and unchecked cellular proliferation occurs , copy number, and mutation data from the PanCanAtlas. We apply the method to Ras genes and demonstrate that our method can detect Ras activation pan-cancer. The classifier also identifies NF1 phenocopying events in TCGA and prioritizes Ras wild-type cell lines that respond to MEK inhibitors. Manually curated oncogenic variants in Ras pathway genes were assigned higher classification scores than variants with unknown significance. Our method can be applied to other cancer-associated genes and pathways as well. For example, the DNA Damage Repair PanCanAtlas analysis working group (AWG) applied this approach to detecting tivation .We developed a machine-learning approach to detect aberrant pathway activity in tumors. The method integrated RNA-seq, copy number, and mutation data. The models were trained using tumors from TCGA PanCanAtlas, with a complete set of these measurements, which included 9,075 tumors across 33 different cancer types. The method is based on a logistic regression classifier framework regularized with an elastic net penalty. We used RNA-seq as a measurement describing the expression state of a tumor and trained the classifier to detect downstream gene expression patterns consistent with aberrant pathway activity . The algKRAS, HRAS, and NRAS mutations and copy number gains (see KRAS mutations were widespread in PAAD (72%), COAD (45%), rectum adenocarcinoma , and LUAD (31%), while NRAS mutations were common in SKCM (31%)  . We perfCM (31%) ; Data S1In the classifier, to enforce a more balanced class representation and to reduce performance metric inflation , we usedOverall, the classifier showed high performance, with an area under the receiver operating characteristic (AUROC) curve above 84% and an area under the precision recall (AUPR) curve above 63% in the cross-validation and testing sets . For theThe Ras classifier consisted of automatically learned gene weights, or importance scores. Training with an elastic net penalty resulted in a sparse classifier, with only 185 genes contributing to classification. Genes and covariates with weights above zero can be interpreted as being upregulated in tumors with activated Ras, while negative-weight genes are characteristic of tumors with wild-type Ras . The fulERRFI1 contributed to predicting tumors with activated Ras. ERRFI1 is a tumor suppressor of various receptors in the Ras pathway  pathway. For instance, high expression of  pathway . The topocytomas . The secthe time .KRAS, NRAS, and HRAS and 11 other Rasopathy genes from the gene expression matrix , MRPS35, ERGIC2, and CMAS, all of which are located on chromosome 12p near KRAS, which may indicate artifacts of common copy-gain events and be a result of low sample size in the positive-copy-only set   . The set1.16e–2) . Though KRAS, HRAS, or NRAS) from Ras wild-type cell lines (p = 6.35e–36). Of the 393 cell lines predicted to be wild-type, 357 were labeled wild-type . However, only 153 of 344 cell lines that were predicted to be Ras mutated were labeled Ras mutant (precision = 44.5%). In total, 510 of 737 (69.2%) cell lines were predicted correctly. In this case, the low precision could indicate either that the classifier failed to generalize or that the classifier successfully identified phenocopying events, which were negatives from the point of view of evaluations but also what we aimed to capture.Next, we applied our Ras classifier to RNA-seq profiles from 737 different cell lines from the Cancer Cell Line Encyclopedia (CCLE) with matched expression and mutation data  Figure . The RasBRAF, a well-characterized oncogene downstream of Ras genes  (BRAF mutations (29.3%). The remaining false-positives indicated either tumors incorrectly assigned or tumors that harbored other phenocopying variants. Next, we tested CCLE pharmacological response data to determine whether Ras classifier scores were predictive of sensitivity to MEK inhibitors. We observed a strong correlation of the Ras classifier scores with sensitivity to two MEK inhibitors, selumetinib (AZD6244) and PD-0325901  . Of all -0325901 . The cor2 = 26.1, degree of freedom 1, p = 3.2e–7. Therefore, our classifier detected signal at variant level resolution. We provide mean classifier scores for all nucleotide  . Converscleotide  and amincleotide  Ras variNF1-loss events particularly well in CNS tumors . Performance was comparable to that of NF1 classifiers built using cancer-type-specific and pan-cancer models , as compared to NF1-specific classifiers   . In thisanalysis .KRAS, NRAS, and HRAS . In clinical trials, selumetinib did not increase overall survival in KRAS mutant advanced non-small-cell lung cancer (NSCLC) patients . The Cancer Genome Atlas will provide instructions on how to access publicly available data.Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Casey S. Greene  when training the models to prevent the model from potentially relying too heavily on dosage-specific effects of these genes instead of the downstream response to their activation. We also removed the samples with the highest mutation burden to remove potential false positives. We defined these samples based on five standard deviations above the log10 total non-silent somatic mutation count per sample. Because we were interested in a balanced training set based on aberrant gene events, we further filtered samples to include only cancer-types with greater than 15 target gene events and a proportion of negatives to positives no less than 5%.We integrated Illumina RNaseq, multi-center mutation calls (MC3), and GISTIC2.0 copy number threshold calls from The Cancer Genome Atlas (TCGA) PanCanAtlas project to classify aberrant pathway function . We downX) to predict gene status (Y) , and loss of function and deep copy number losses for the tumor suppressors (NF1). For example, if a tumor had a deleterious mutation or copy number amplification in one of these genes, we considered the Ras status equal to one.Using this data, we trained a supervised elastic net penalized logistic regression classifier with stochastic gradient descent . Our mod (Y) see . To conti) has a Ras event given the sample’s RNaseq measurements (iX). In order to achieve the objective, the classifier learns a vector of coefficients or gene-specific weights (w) that optimize the following penalized logistic function.The objective of the classifier is to determine the probability a given sample .Where α and l = 0.15 and α = 0.1) and assessed performance on training, testing (held out 10% of data) and across 5-fold cross-validation intervals. In 5-fold cross-validation, the data are partitioned into five even sets . Four of the folds, called training intervals, are used to construct the model. The model is then evaluated on the fifth fold, which is called the evaluation fold. The reported training performance comes from the folds used for training, while the cross-validation performance uses the evaluation fold. Therefore, performance on cross-validation intervals are the predictions reported on the training set samples when they were included in the internal cross-validation evaluation fold. The full model is reported in We trained the Ras classifier using optimal hyperparameters  curve. We also compared gene specific classifiers built using pan-cancer data to classifiers trained independently using only data from individual cancer-types. In these cases, each cancer-type specific model was optimized individually. We compared how the pan-cancer model performed on individual cancer-types compared to individual cancer-type optimizations. Additionally, we cataloged the performance of the Ras classifier to predict We determined the robustness of the classifier by evaluating performance under various input features and prediction tasks. We evaluated potential inflation of performance metrics by training a null model on a randomly shuffled input gene expression matrix. We did not shuffle the covariate information or the y matrix. Performance on the random shuffling of genes, while maintaining the same ratio of Ras mutations, provides insight into how the model would be expected to perform in a scenario lacking Ras activation signal. We also performed the same shuffling and classifier testing procedure as internal negative controls in every pan-cancer model and report ROC/PR curves and AUROC/AUPRs in each figure.KRAS, HRAS, and NRAS from the model , we trained models with the previously described procedure with the input gene expression matrix without dropping Ras genes. We also tested a classifier after dropping 14 genes from the Expanded RASopathy Panel (Genetic Testing Registry). The genes included BRAF, CBL, HRAS, KRAS, MAP2K1, MAP2K2, NF1, NRAS, PTPN11, RAF1, SHOC2, SOS1, SPRED1, and RIT1. For the two previous comparisons, we compared the learned gene expression coefficients to the classifier trained in To assess value added in combining mutation and copy number data in the prediction task , we trained pan-cancer classifiers with the same procedure described above to predict Ras mutations and Ras copy number gains separately. The combined model presented here is the same model trained in We performed a differential expression analysis using the limma Bioconductor package . We adjuWe applied the Ras classifier to two independent cell line datasets. The first dataset was generated by s is the classifier prediction, w is the gene weights, and X is the corresponding subset cell line gene expression matrix.Where https://portals.broadinstitute.org/ccle/data . Data were accessed from cle/data .We used the PanCanAtlas Pathways Working Group definition of 38 core Ras pathway genes . We obtahttps://github.com/greenelab/pancancer) for full software version details. We evaluated all classifiers using AUROC and AUPR. The AUROC is a metric describing the overall trade-off between true positive and false positive rates, while the AUPR measures precision against recall for a given classifier. An AUROC of 0.5 constitutes random guessing. We describe specific filtering steps for each analysis in various places in the Method Details section of the STAR Methods. We describe overall sample and gene filtering in the Training subsection. We discuss additional gene filtering for evaluating all alternative genes in the Evaluation subsection. We set random seeds in all computational analyses in order to preserve reproducibility. We performed independent t tests with unequal variances when comparing classifier scores for curated variants versus variants of unknown significance per Ras pathway gene. We performed the same test comparing CCLE cell line Ras classifier scores for Ras wild-type versus Ras  mutant samples and for Ras wild-type, BRAF wild-type versus Ras wild-type, BRAF mutant. Using the up to 388 cell lines with both gene expression and pharmacology data measured, we fit linear regression models comparing drug activity versus Ras classifier scores for all 24 drugs to Ras wild-type and Ras mutant cell lines individually. Using a Bonferroni adjusted p value (0.05/(24 * 2) = 0.001), we implicated two high correlated drugs (AZD6244 (Selumetinib) and PD-0325901). Selumetinib was tested on 387 cell lines while PD-0325901 was tested on 388 cell lines. We also used a chi square test for proportions of Ras mutations annotated as COSMIC variants in true positives compared to false negatives with a null hypothesis that both sets of samples have the same proportion of COSMIC variants.We performed all machine learning model training, testing, and evaluations using sci-kit learn (version 0.18.1) with python 3.5.2 . We prochttps://github.com/greenelab/pancancer. There, we provide instructions to replicate the computing environment, download versioned data, and all scripts to reproduce the entire analysis pipeline. The pipeline is modular and amendable to generate classifiers and predictions for any combination of genes, pathways, and TCGA PanCanAtlas cancer-types. The source code has been deposited to Zenodo at https://doi.org/10.5281/zenodo.1186801.All analytical results can be reproduced using the code available at 1234567