Recently, Raman Spectroscopy (RS) was demonstrated to be a non-destructive way of cancer diagnosis, due to the uniqueness of RS measurements in revealing molecular biochemical changes between cancerous vs. normal tissues and cells. In order to design computational approaches for cancer detection, the quality and quantity of tissue samples for RS are important for accurate prediction. In reality, however, obtaining skin cancer samples is difficult and expensive due to privacy and other constraints. With a small number of samples, the training of the classifier is difficult, and often results in overfitting. Therefore, it is important to have more samples to better train classifiers for accurate cancer tissue classification. To overcome these limitations, this paper presents a novel generative adversarial network based skin cancer tissue classification framework. Specifically, we design a data augmentation module that employs a Generative Adversarial Network (GAN) to generate synthetic RS data resembling the training data classes. The original tissue samples and the generated data are concatenated to train classification modules. Experiments on real-world RS data demonstrate that (1) data augmentation can help improve skin cancer tissue classification accuracy, and (2) generative adversarial network can be used to generate reliable synthetic Raman spectroscopic data. According to the American Academy of Dermatology Association (AAD), the daily number of diagnosed skin cancer cases in the US is approximately 9500 and one in five Americans is estimated to develop skin cancer in their lifetime3. Although surgical removal is the optimal method for skin cancer diagnose and treatment, current in situ methods can hardly differentiate cancer from normal skin2. In addition, the surgical process is time-consuming and patients may suffer from heavy financial burden4. In contrast, the vibrational modes of molecules can be easily and correctly analyzed with Raman Spectroscopy (RS) technique, which is able to detect differences in the molecular structures of proteins, lipids and pigments of both tumor and normal tissues6.Skin cancer, one of the most common cancers across the world, accounts for more than 40% of global total cancer cases, in which the top three skin cancer types are basal cell skin cancer (BCC), squamous cell skin cancer (SCC), and non-melanoma skin cancer (NMSC)7.Raman Spectroscopy (RS) is a non-destructive in situ spectroscopic chemical analysis technique that provides detailed information about chemical structure, phase and polymorphism, crystallinity and molecular interaction. RS is an inelastic light scattering technique producing scattered photons either lower in energy (Stokes) or higher in energy (anti-Stokes) than the exciting photons. The energy shifts of the photons correspond to the energies of vibrations of molecules in the sample, thus providing detailed chemical structure information revealing chemical compositions of cells and tissues8, where spectral classification is used to classify BCC based on tissue samples from 55 patient, among which logistic regression classifier using five canonical spectral features obtained from rank-reduced multiclass linear discriminant analysis outperforms the rest classifiers. Previously, we also employed Principal Component Analysis (PCA) to differentiate non-melanoma skin cancer (NMSC) from normal skin with the combination of RS and high-powered 9, the Raman spectra are collected from both Using RS and machine learning for skin cancer detection has been studied previouslymentclass2pt{minim11. Discrete wavelet transform (DWT) has been applied to separate shot noise, fluorescence’s baseline and informative Raman peaks by decomposing the spectrum into a set of wavelet coefficient and scale coefficient13. Other advanced methods adapted based on DWT such as adaptive lifting wavelet transform (ALWT) are also proved effective in removing noise from Raman spectrum14. In our previous study9, samples were processed with laser ablation, as shown in the top row in Fig. Despite of the promising properties, RS suffers from weak signals due to inherent noise. Shot noise and fluorescence’s baseline are the most commonly found noise in RS signals. The shot noise is the results of the unavoidable statistical nature of light while the fluorescence’s baseline can mask Raman bands with a higher amplitude. Although several denoising method have been proposed, noise can hardly be effectively removed from the signal to limit the negative impact of the noise, which will damage the integrity of the Raman spectrum16. The data scarcity, combined with high data dimensionality, present difficulties for deep learning algorithms.Another challenge in RS based skin cancer detection is that the data are often ill-posed, because Raman spectrum has a background resulted from skin fluorescence, both the spectra and correlation can be introduced with variance. Apart from that, the number of frequency components in Raman spectrum is normally The above challenges motivate our research to use Generative Adversarial Network (GAN) based data augmentation method to increase the sample size for RS based skin cancer detection. GAN is a method to generate data resemble to the training data, using two deep networks, the generator, and the discriminator. Candidates are generated by generative network while the discriminative network evaluates. Through the iterative generation and evaluation process, new data, with the same statistics as the original data set, are generated. Examples of GAN generated RS samples are shown in the bottom row in Fig. Using data augmentation, new samples for each category of the original data are generated in order to improve the performance of the downstream classifiers. For data classification part, a deep convolutional neural network (CNN) is designed as the core module to compare with other baseline models. We employ CNN as the classifier for RS cancer tissue classification, mainly because that CNN has unique convolutional filters to explore correlation of the signal and learn patterns to differentiate signals between different classes.Deep learning to tackle RS sample scarcity In cancer diagnosis, due to privacy and other restrictions, very few valuable samples are available for training reliable models. Our research proposes solutions to tackle this challenge by applying generative adversarial network to generate synthetic RS samples. We validate and compare the effectiveness of this approach vs. other baselines.RS sample augmentation approaches We design two sample augmentation approaches, balanced data augmentation and stratified data augmentation, to evaluate how augmented samples should be integrated for learning accurate models.Deep learning for RS cancer tissue classification By leveraging sample augmentation, we propose a deep learning based framework and compare its performance using a variety of deep learning models, including CNN and Long Short Term Memory Networks (LSTM), for Raman spectroscopy cancer tissue classification.In this paper, we address the data scarcity of RS cancer tissue classification by using deep learning based data augmentation and classification. This study has three main contributions as follows:17. Recent studies reported that Raman spectroscopy (RS) is beneficial to diagnose and study the evolution of human malignancies both in vitro and in vivo18. It has been widely proved that Raman spectroscopy can be applied to distinguish skin cancers from normal skin tissues for accurate medical diagnosis19.Raman spectroscopy has been considered as a promising non-destructive optical technique characterizing the tissue at the molecular level20 evaluated the application of an integrated real-time system of RS for in vivo skin cancer diagnosis. The performance, in terms of ROC (Receiver Operating Characteristic) curve, can be dramatically increased to 0.879 by using a primary module with generic discriminant analysis in lesion classification. Lieber et al.17 measured Raman spectrum of 21 suspected non-melanoma skin cancers and detected lesions from normal skin yielding 9121 with Confocal Raman spectra obtained from various skin depths. The reserach of Fox et al.9 shows that RS classification accuracy is not negatively affected by the ablation process and also beneficial to tumor border demarcation. A hybrid fluorescence Raman approach or a non-linear Raman technique can be used to reduce imaging time.Several researches have utilized RS to diagnose skin cancer. Harvey et al.21. Therefore, deep learning methods have been integrated in our study.Due to weak Raman intensity caused by the poor scattering efficiency, PCA analysis or Neural Network is preferred to distinguish skin cancer tissue from normal tissueAnalysis and interpretation of biological signals are highly intricate research tasks. Deep learning extracts signal’s features automatically from raw data with a better performance when amounts of data are available for learning, while traditional machine learning methods to understand and translate biological signals are based on hand-engineered features. Biosignals and deep learning are often utilized to solve specific application, such as health status monitoring, emotion detection, analysis and classification of human gestures, diagnosis for illness and so on. Among all applications, we focus on the support for diagnosis using deep learning.22 with CNN. It achieved similar performance with all tested experts’ judgment on the identification of common cancers and the identification of the deadliest skin cancer. Budak et al.23 proposed an end-to-end system based on fully convolutional network (FCN) and bidirectional long short term memory (BiLSTM) for detections of breast cancer. A five-fold cross-validation technique was considered to calculate accuracy metric, which showed that their proposed method was better than those preliminary reported results. Mahbod et al.24 used three pretrained deep models, namely AlexNet, VGG16 and ResNet-18, as deep feature generators. The extracted features were sent to support vector machine classifiers, yielding an area under ROC curve of 83.83An image of a skin lesion was successfully transformed into the probability distribution of clinical dermatoses by Andre et al.25. A skin lesion style-based generative adversarial networks (GANs) model is proposed in26 and proved to be effective for generating skin lesion images with high resolution and abundant diversity. Compared to the prior CNN model, classification indexes like accuracy, sensitivity, specificity, average precision and balanced multi-class accuracy have been improved by 1.627. The F1 density of the classifier model trained with GAN-based augmentation is 0.95, outperforming the models trained with traditionally augmented images with an F1-score of 0.92. There is sufficient evidence to prove that by using GAN-based enhancement methods, deep learning classification models have better performances than using traditional enhancement methods.All the detecting systems by using deep learning mentioned above are based on large amounts of data. In terms of augmenting image sets, GAN is widely applied to overcome the sensitivity of synthetic data samples for the cancer data classification9, which consists of three tissue categories: BCC , SCC (squamous cell carcinoma) and NORMAL. Using RS process, each sample is represented by 1608 dimensions, denoting the frequency of the Raman shift  to classify each sample into respective category, and also comparatively study other rival methods, such as logistic regression (LR), support vector machines (SVM).Our framework introduces a novel generative adversarial network based medical data augmentation for Raman Spectroscopy cancer tissue classification, as shown in Fig. Figure 28. As shown in the lower left dashed rectangular box in Fig. G is to generate fake samples from the latent vector z. The generator can be thought of as analogous to a team counterfeiters, trying to produce fake samples and try to induce the discriminator to give the generated sample a higher score.The generator D is analogous to the police, which tries to discriminate between the original data and the generated samples.The discriminator The generator and discriminator are running in an adversarial way to improve each other. Specifically, the discriminator D tries to learn the original data and guides the generator by sending feedback about the generated synthetic samples. The generator G learns from the feedback and tries to generate new samples which are very close to the original data. The discriminator D is trained to maximize the probability of distinguishing original samples from samples generated from the generator G (i.e. correctly predict whether a sample is generated or not) The loss function of the discriminator D can be expressed as:In order to solve the small sample size problem, we propose to use Generative Adversarial Network (GAN) to investigate data augmentation. Generative Adversarial Network(GAN) has been implemented to synthesize high quality data for adding training data in several studiesG is trained to minimize G can be expressed as:Simultaneously, the generator D and G are trained by a two-player minimax game with the loss function:In other words, In Fig. From the data augmentation module, we can obtain a set of new samples for each category of the original data, which will be used to enhance the performance of the classifier. In our classification module, we employ a deep convolutional neural network (CNN) as the core module.d-dimensional feature corresponding to the ith sample. For each sample k kernel to produce a new feature.w is a filter, and b is a bias term. This filter is applied to each possible window of features in the i-th sample to produce a feature map Let r filters. So we can obtain a feature matrix i-th sample, with In order to obtain multiple features, our classification model uses i-th sample.ith sample.After the convolution process, we apply 1D Max pooling to the feature map matrix i-th sample, The classification loss Algorithm 1 lists the detailed procedures of the proposed algorithm for RS based cancer tissue classification. Algorithm 2 lists the RSDA data augmentation module, which is used in Algorithm 1.k steps of optimizing discriminator D and one step of optimizing generator G. Firstly, we maximize the loss function of discriminator (Step 3–4). After k steps, we then minimize the loss function of generator (Step 6–7). After the training process, we add the generated new samples to the original dataset to expand the training dataset to learn convolutional neural networks for classification.Given a noise prior Balanced data augmentation , NORMAL and SCC (squamous cell carcinoma), each contains 36, 63, and 50 RS samples, respectively. Each sample has 1608 dimensions, representing the wavelength numbers of the Raman shift  activation functions. Furthermore, each fully-connected layer is followed by a Batch Normalization. The discriminator consists of one fully-connected layer with 1608 hidden units and two fully-connected layers with 64 hidden units, using LeakyReLU activation functions.Evaluation metrics The skin cancer tissue classification is a multi-class classification task, whose evaluation measure is commonly the Accuracy metric. However, when datasets suffer from class imbalance, it goes less reliable. Therefore, in addition to the Accuracy metric, we add Macro-F1 and Area Under Curve (AUC) metrics. For all experiments, we use leave-one-out cross validation on total samples throughout the experiments.LR is a Logistic Regression model, which directly feed the sample features into the Softmax classifier to determine the category.PCA_LR first uses Principal Component Analysis (PCA) to reduce the input features to 100 dimensions, and then trains logistic regression classifier for classification.LR_SVM is a support vector machine classifier learned in the original feature space.PCA_SVM uses PCA to reduce input features to 100 dimensions, then trains SVM classifiers for classification.SVM_MLP is a Multilayer Perceptron network, which consists of multiple fully connected layers.MLP_LSTM is a long short-term memory neural network capable of learning features from long sequence inputs.LSTM_CNN uses a convolutional neural network to learn sample representations by sliding windows on sample features.CNN_SMOTE uses a convolutional neural network to learn sample representations from the augmentation of the original and synthetic data. The data augmentation method uses Synthetic Minority Oversampling Technique (SMOTE)29. The SMOTE method is based on the k nearest neighbor sample points of each sample point, and randomly selects N neighboring points to multiply the difference by a threshold in the range of  to achieve the purpose of data enhancement.DL_We implement following baselines for comparisons to demonstrate the effectiveness of our proposed model.r and their width/size k are set to 120 and 5, respectively. The classification module of LR has one fully-connected layer with the number of hidden units 3, which is activated using Softmax. For classification module of SVM, the Maxiter of LinearSVC is set to 500. The MLP classification module has three fully-connected layers activated by Rectified Linear unit (Relu), and the hidden units of each fully connected layer are set to 100, 32, and 3 respectively. The LSTM classification model consists of an LSTM layer with 100 hidden units and three fully-connected layers with 10, 500, and 3 hidden units, respectively. For each network, we use a fixed learning rate For training, we use 500 epochs for all models. For data augmentation module training, we use the Adam optimizer with the initial learning rate of 0.0002 to train the GAN model, and then generate a certain number of augmentation samples for each class. In GAN, the generator is composed of one fully-connected layer with 100 hidden units and three fully-connected layers with 64 hidden units, which are activated using Rectified Linear unit (ReLU). Furthermore, each fully-connected layer is followed by a Batch Normalization. The discriminator is composed of one fully-connected layer with 1,608 hidden units and two fully-connected layers with 64 hidden units, activated using LeakyReLU. For classification module of CNN, the number of convolution filters Most deep network models, such as CNN and LSTM, outperform traditional machine learning models, like LR and SVM. This demonstrates that deep learning methods, even with limited training samples, can learn better hidden representations of samples. This is because that deep learning can better leverage correlation in the RS signals to learn patterns, whereas LR and SVM take raw RS spectra as features, where feature correlation will deteriorate the classifier performance.Models with data augmentation  increased sample density; (2) increased sample diversity; and (3) increased resemblance between synthetic vs. genuine samples. All of which have helped learn better decision boundaries for separation.For increased sample density, data augmentation can generate more samples, similar to the training data, and help increase the training set density. With a higher density, the classifiers can often learn more precise boundaries, compared to sparse data. This explains why data augmentation often outperforms classifiers learned from original sample set.For increased sample diversity, deep learning data augmentation is essentially different from bagging (which duplicates training samples) or SMOTE (which uses linear space conversion to create new features). The non-linear transformation used in deep learning allows more diverse samples to be generated.For increased resemblance between synthetic vs. genuine samples, the adversarial learning process (between generator and discriminator) ensures that synthetic samples are very similar, but not identical, to the original training data. This increases the resemblance of synthetic sample set to the original training set.In this paper, we proposed to use deep learning for Raman Spectroscopy based cancer tissue classification, by using data augmentation method to increase the number of training samples in order to enhance downstream classifiers for better feature representation learning from RS signals and better classification. To achieve the goal, we proposed a novel generative adversarial network based skin cancer tissue classification framework with two major components: (1) Data augmentation module, and (2) Data classification module. The former employs a Generative Adversarial Network for data augmentation to obtain a sufficient number of data, and the latter uses five different approaches for classification. Our study validates different ways of data augmentations, including balanced data augmentation to add same number of samples to each class and stratified data augmentation to preserve class distributions. Experiment results show that GAN can be successfully utilized for augmenting data and generating synthetic samples resemble to the original samples. Moreover, the proposed DL_DA  obtains the best performance to separate BCC, and SCC cancer, from normal samples.Due to resource constraints, our study is only based on a rather small RS sample set. We are working towards obtaining additional RS samples from different sources for validation. Using other data augmentation approaches, such as contrastive learning, for feature learning from RS data is also a future direction of our study.


Prostate cancer has very varied appearances when examined under the microscope, and it is difficult to distinguish clinically significant cancer from indolent disease. In this study, we use computer analyses inspired by neurons, so-called ‘neural networks’, to gain new insights into the connection between how tissue looks and underlying genes which program the function of prostate cells. Neural networks are ‘trained’ to carry out specific tasks, and training requires large numbers of training examples. Here, we show that a network pre-trained on different data can still identify biologically meaningful regions, without the need for additional training. The neural network interpretations matched independent manual assessment by human pathologists, and even resulted in more refined interpretation when considering the relationship with the underlying genes. This is a new way to automatically detect prostate cancer and its genetic characteristics without the need for human supervision, which means it could possibly help in making better treatment decisions.Prostate cancer is a common cancer type in men, yet some of its traits are still under-explored. One reason for this is high molecular and morphological heterogeneity. The purpose of this study was to develop a method to gain new insights into the connection between morphological changes and underlying molecular patterns. We used artificial intelligence (AI) to analyze the morphology of seven hematoxylin and eosin (H&E)-stained prostatectomy slides from a patient with multi-focal prostate cancer. We also paired the slides with spatially resolved expression for thousands of genes obtained by a novel spatial transcriptomics (ST) technique. As both spaces are highly dimensional, we focused on dimensionality reduction before seeking associations between them. Consequently, we extracted morphological features from H&E images using an ensemble of pre-trained convolutional neural networks and proposed a workflow for dimensionality reduction. To summarize the ST data into genetic profiles, we used a previously proposed factor analysis. We found that the regions were automatically defined, outlined by unsupervised clustering, associated with independent manual annotations, in some cases, finding further relevant subdivisions. The morphological patterns were also correlated with molecular profiles and could predict the spatial variation of individual genes. This novel approach enables flexible unsupervised studies relating morphological and genetic heterogeneity using AI to be carried out. Prostate cancer is the second most commonly diagnosed cancer and the fifth most common cause of cancer death in men worldwide . AlthougThis challenge has been addressed for other types of cancer through the use of computational methods and artificial intelligence (AI). Classical machine learning approaches included training a support vector machine with ST breast cancer datasets to identify cancer regions . Deep coThus, we hypothesize that morphological features of prostatectomy H&E sections extracted automatically using a CNN previously trained on prostate needle biopsies could identify histologically and genetically relevant sub-regions inside the tissue in an unsupervised manner. We base our study on the data produced and presented in . It is iAfter radical prostatectomy in a patient with adenocarcinoma, seven H&E-stained tissue sections were selected around the two main tumor foci in the prostate, as shown in We started from a pre-trained Gleason grading ensemble CNN developed by Ström et al. . The ensThe resulting workflow is presented in ant maps .In order to aggregate all the networks’ descriptions, we again applied UMAP dimensionality reduction to reduce the The same tissue sections were also manually/visually graded by two independent pathologists, and their annotations are shown in The automatically obtained color maps of morphological heterogeneity in This suggests that the method defines morphologically coherent regions, described only by three deep morphological features, that are comparable to the consensus of two histopathologists’ annotations.We additionally wanted to perform clustering and therefore embedded the re Model d. To setre Model .Automated clustering by Gaussian Mixture clustering, as shown in It is sometimes the case that the slight differences observable by the feature map representation were coarser when committing to a specific cluster. For instance, in section 1 of the figure, we see that the equivocal benign region was clustered with the adjacent stromal region. In section 2, the low number of clusters loses the differences between ISUP2 and ISUP4 cancer. Furthermore, a similar effect is observed for section 3, where the region associated with ISUP2 cancer becomes much larger. For section 4, we observe an additional benign region defined around the ISUP1 cancer. Interestingly, in section 5, the separation of the ISUP4 region into two clusters persists, as does the unsure region in section 7. In section 6, some of the regions visually graded as ISUP4 were missed.In order to study whether our automatically obtained morphological clusters were genetically relevant, we analyzed whether they corresponded to different gene expression factor signatures. The factors were obtained from spatially-resolved gene expression, as described in ,9. Note Here, we refer to section 5 of the figure, where a region manually marked as containing ISUP2 cancer was automatically divided into two morphological sub-clusters, as shown in While carrying out inverse analysis, we further found that the expression of individual genes forming a factor d is tighFurther interpretation of specific examples are presented in the Discussion.We finally wanted to compare feature extraction using our network pre-trained on prostate needle biopsies to feature extraction using a network pre-trained on the ImageNet dataset . The results of feature extraction with a network pre-trained on ImageNet (while applying the same workflow as described above) are shown in This could be considered to define the lower sensitivity limit. Applying the network pre-trained on prostate needle biopsies on this dataset could be considered to be too specific and rigid, while a network pre-trained on ImageNet was not dedicated enough to find meaningful regions. The corresponding analogy would be asking a lay-person to grade the section: they would only define regions that were stained similarly, without the regions necessarily being relevant.The current study combines previous knowledge of morphological and genetic integration, overcoming important challenges suggested in the field. This flexible workflow allows one to extract morphological features in any desired dimensionality depending on the task at hand. To avoid misleading associations between very high-dimensional morphological and genetic information, both are first summarized using state of the art methods: UMAP dimensionality reduction and factor analysis, respectively.Despite the models being blind to ST data, the correlation of morphological descriptors with gene expression were comparable with previous studies which trained a model specifically to predict individual gene expression . LookingFor instance, in section 3 in As we showed, in section 5, the ISUP2 region was separated into two morphological clusters. The lower region related to factor 10 that is composed of genes such as H2AFJ, shown to relate to tumor progression in other types of cancer  or TRGC1The case of the region on top of the ISUP4 in section 7 is also worth mentioning. The histopathologists agreed that one could not confidently label the region as benign or cancerous. The network also created an additional cluster for this region which, through factor analysis, we found is related to the spatial expression of DEPDC1, which was found to be linked to tumor growth and cell proliferation in prostate cancer .This creates the possibility to automatically annotate tissue slides combining morphological and transcriptomic information. Additionally, this workflow can be used for annotation by first describing the regions and then exploring the spatially resolved gene expression inside. The ST techniques alone are not directly mappable to the morphology , and theIn a clinical setting, this approach could be used to help histopathologists annotate tumor samples by looking at the defined unsupervised regions, reducing the time needed for manual annotations, especially in the assessment of boundaries. The expert histopathologist intra-observer variability in grading prostate cancer is well known . With thSimilar approaches can be used for other types of cancer. One could use challenge-winner networks in biology, extract their penultimate layer and use them as good surrogates of morphological features, instead of the current trend of using generalist pre-trained networks or networks trained on own data . We furtFuture work could also seek to relate the morphological features extracted by the model to the presence of specific cell populations, without the need to retrain with single cell data. Recent work on ST proposes that transcriptomic regions can be mapped to distinct cell types ,27. ThusHowever, in order to validate this kind of bold idea, some limitations should be overcome. First, a limited number of slides from a single patient was used in this study, due to the cost of ST. However, the sections were chosen so as to include different combinations of benign, stroma, and low- and high-grade cancer tissue.Additionally, staining differences across slides and normalization is still challenging in automated histopathology . We therFinally, the tissue patch size was selected to match that of the network we used for feature extraction, and was large compared to the ST spots. A smaller patch size would not have captured as much of the context, and it should be noted also that the histopathologists had access to the whole tissue view at visual grading. The drawback of the large patch size is that it produces a smoothing effect.To conclude, we presented a workflow to repurpose a state-of-the-art AI system in order to define morphologically and genetically relevant sub-regions and find the associations between them. This continues to prove the close relationship between both levels of heterogeneity in prostate cancer and also supports the use of networks pre-trained on histopathology data for tumor grading in other applications. This is a new way to automatically detect prostate cancer without the need for direct visual inspection and, by also detecting genetic characteristics, raises the possibility that AI approaches such as this could differentiate cancer that needs treatment from indolent disease that does not.The H&E sections, together with the Visium ST data, were obtained upon petition and described in detail by Erickson et al. .Annotations with few spots were merged with other annotations. Nerve, fat, vessel, and inflammation, which made up very few of the annotated spots, were merged with stroma. Section 1 contained a region that the histopathologists considered equivocal benign, shown in The slides were digitized using a Hamamatsu C9600-12 scanner with NDP.scan v.2.5.86 software . The pixel size at full resolution (s), rotation (r) and translation (t):The ST coordinates We used the ensemble of deep convolutional networks presented by Ström et al. . It is aHowever, for our study, we needed to repurpose the system to be able to apply it to the prostatectomy slides and define the sub-regions. Using the ST spots as the center, we extracted In order to obtain the morphological information for each of them, we fed the patches into the ensemble and extracted the 2048 penultimate layer activations for each of the 30 models, yielding a total of At this point, we wanted to reduce the On the one hand, for visualization, we used UMAP to reduce the dimensionality from 300 to 3 morphological descriptors per spot. This allows one to easily represent the ST spots of a section in a three-dimensional space. Additionally, assigning red, green, and blue (RGB) intensity values to each dimension, we could represent a color map on top of the original section, to observe the captured heterogeneity.On the other hand, for clustering, we used UMAP to reduce the dimensionality from 300 to 10 morphological descriptors per spot. We experimentally confirmed that clustering with more than 10 features per spot did not yield novel morphological features in each section, suggesting that the features were starting to become redundant. That is, reducing the dimensionality to more than 10 features produced spatial maps which were linear combinations one of the other.The 10 features per spot were clustered using Gaussian Mixture clustering. To set the number of clusters in every section, the Calinski–Harabasz  score usTo numerically compare the spatial overlap of the obtained clusters with the histopathologists manual annotations we constructed a Dice index matrix for each annotation To check whether the clusters obtained by unsupervised clustering of morphological features were also genetically meaningful, we performed a joint analysis with gene factors. We used the gene factors obtained from the gene factor analysis technique ,9, whichFor each of the resulting factors, we selected the top genes with the most similar spatial pattern to the factor and calculated the Pearson correlation between the standardized gene expression and the morphological feature that contributed most for the corresponding cluster, as detailed in We compared the results of clustering the 10 final UMAP components for section 5 into five clusters using Gaussian Mixture, K-means and spectral clustering . For comhttps://tissuumaps.research.it.uu.se/).All analyses were made on a local cluster using Python v. 3.6.9. We used Openslide v. 3.4.1  to acces


Dataset GSE39058 was miswritten as GSE39055, 42 samples was miswritten as 47.In the original article, there was an error. RESULTS, Identification of Seed Genes Based on theCoefficient of Variation in OSA, 1:A correction has been made to . First, we used the GSE39058 dataset, which included 47 samples, as a training set. In this dataset, each patient sample included detailed clinicopathologic information and survival status. The coefficient of variation (CV) of each probe was calculated for all samples, and the probes with a CV >20% were considered to have the largest degree of variation among all OSA samples and were selected as the seed probes. Then, 309 probes were obtained and mapped to 308 unique genes. Next, we completed an unsupervised clustering analysis of the 42 samples by using expression profiling of the 309 probes obtained in the previous step. As shown in Figure 2A, the OSA samples were divided into 2 groups, and there were significant differences in gene expression levels between the groups. Survival analysis was then used to compare the outcomes of the groups: no significant difference was observed between them (log-rank test p ≥ 0.05) .In the present study, we established a 2-gene signature for the prognostic prediction of OSA The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated.


The diagnostic and prognostic value of miRNAs in cutaneous melanoma (CM) has been broadly studied and supported by advanced bioinformatics tools. From early studies using miRNA arrays with several limitations, to the recent NGS-derived miRNA expression profiles, an accurate diagnostic panel of a comprehensive pre-specified set of miRNAs that could aid timely identification of specific cancer stages is still elusive, mainly because of the heterogeneity of the approaches and the samples. Herein, we summarize the existing studies that report several miRNAs as important diagnostic and prognostic biomarkers in CM. Using publicly available NGS data, we analyzed the correlation of specific miRNA expression profiles with the expression signatures of known gene targets. Combining network analytics with machine learning, we developed specific non-linear classification models that could successfully predict CM recurrence and metastasis, based on two newly identified miRNA signatures. Subsequent unbiased analyses and independent test sets  using our prediction models resulted in 73.85% and 82.09% accuracy in predicting CM recurrence and metastasis, respectively. Overall, our approach combines detailed analysis of miRNA profiles with heuristic optimization and machine learning, which facilitates dimensionality reduction and optimization of the prediction models. Our approach provides an improved prediction strategy that could serve as an auxiliary tool towards precision treatment. Cutaneous malignant melanoma (CM) is the most dangerous, heterogeneous and with a strong propensity to metastasize, skin cancer. Until today, the American Joint Committee on Cancer (AJCC) staging is the dominant synthetical index to CM prognosis and has been recently revised to its eighth edition, which improves staging, prognosis, and risk stratification . NotablyPrecision medicine approaches exploring-omics data have emerged to identify diagnostic and prognostic biomarkers, which can lead to early disease detection, the better understanding of the underlying biological mechanisms and the application of individualized treatment protocols to patients with CM. Towards this goal, several studies have provided information using or combining genomics, transcriptomics and epigenomics analyses ,7,8. BasThrough the advancement of sequencing methodologies, noncoding RNAs (ncRNAs) have emerged as among the most promising biomarkers for monitoring CM progression and recurrence, with miRNAs being the most well-studied class. Since their discovery, specific miRNAs have been recognized as dependable markers that help monitor cancer progression or regression. In CM, several different and highly conserved miRNAs exist which have been correlated with essential developmental processes and serve as diagnostic or prognostic markers, either alone or in combination with other types of biomarkers, based on their ability to target at the same time more than one mRNAs, as means of post-transcriptional gene expression regulation . The enrHerein, we provide a comprehensive review on the role of specific miRNAs that contribute to the deregulation of gene expression in CM. In a next step, the value of specific miRNA signatures with possible role on recurrence and metastasis were examined in correlation to specific gene signatures that are currently known to play role in CM. After analysis of the TCGA-derived CM-related miRNASeq data with a statistical, bioinformatics and network analytics approach, we observed two distinct miRNA signatures, one strongly associated with metastasis and one strongly associated with recurrence. Incorporation of the TCGA mRNA transcriptomics profiles and analysis of the miRNAs’ putative targets gave us a better insight into the roles of the miRNAs of the two signatures. The resulted miRNA signatures were further analyzed with a multi-objective optimization ensemble classification method that was trained to predict CM recurrence and metastasis . The traCollectively, the current analysis underlines the importance of rationalized integration of data from multiple sources which could provide novel and accurate means to prognosis of CM progression, recurrence and metastasis that could also apply for virtually any type of cancer. In addition, it highlights the importance of machine learning classification models for precision medicine approaches, and the opportunity to identify elusive, so far, important new biomarkers.Several studies have established miRNAs as important diagnostic, prognostic and therapeutic markers in almost all known cancer types, including CM and other skin cancers . The regA thorough search in public repositories like Gene Expression Omnibus, TCGA and dbGaP for miRNASeq data deriving exclusively from CM biopsies resulted in 10 available datasets . The TCGBased on the accumulated sequencing data, recent efforts focused on the generation of miRNA signatures with the contribution of artificial intelligence (AI) tools to improve the prediction of metastasis, overall survival or recurrence of patients with CM and/or other cancer types. More specifically, an 11-miRNA signature was reported to discern primary melanoma from common nevi using formalin-fixed paraffin-embedded (FFPE) samples and microarray or miRNASeq experiments . A similPredicting the overall CM patient survival is also of vital importance and can be clinically used for optimal treatment management. Several studies attempted to correlate specific miRNA expression profiles with the overall survival of CM patients, albeit with different outcomes  that is strongly correlated with survival prediction [STK26, KCNT2, CASP12) with potential value as predictors for accurate diagnosis and prognosis of CM [KRT9, KBTBD10, DCD, ECRG2, PIP, SCGB1D2, SCGB2A2, COL6A6, HES6), which could predict the clinical outcome [DSG3, DSC3, PKP1, EVPL, IVL, FLG, SPRR1A, SPRR1B) with significant predictive value for metastatic transformation of CM [ALDH1A1, HSP90AB1, KIT, KRT16, SPRR3, TMEM45B) to predict metastasis was identified using publicly available transcriptomics data from biopsies of CM patients, using an AI approach and scanning a genome-wide protein-protein interaction network, followed by shortlisting of the most influential and most differentially expressed genes. The gene signature showed 87% classification accuracy and was further validated on an independent transcriptomic data set and highlighted the importance of AI tools for a more accurate prediction approach [JUN and AXL expression with poor prognosis and response to immunotherapy [The use of NGS for transcriptomics analysis and several elaborate bioinformatics tools have correlated the expression of several genes with CM progression and metastasis  was identified and a prognostic signature of 13 genes for metastatic CM was reported  [Most recently, studies using CM mouse models have led to the generation of similar gene signatures with potential use as prognostic tools for CM progression, recurrence, and metastasis. In more aggressive melanomas, a set of 4 upregulated genes  ,76. In a1, SIX1) ,77. ThisSPRR1B, KCNT2, CRABP2, TRIM29, LTA4H and PDE3B. Similarly with the miRNA signatures mentioned above, most research groups were limited to datasets of a few genes and followed different methodologies in an attempt to generate gene-specific signatures, leading to significant variations between gene clusters, suggesting that a more collective approach is needed.Genes reported in more than one signature include To gain insight into the contribution of miRNAs reported so far in melanoma recurrence and metastasis, we analyzed data from the TCGA-SKCM project, which is the largest collection of integrated biological and clinical data from 470 melanoma patients. miRNA-Seq data are available for 448 patients . For 4 oSubsequent analysis led to the identification of characteristic miRNAs in tumor metastasis. Among the 452 samples, 353 originated from metastatic tumors and 97 from primary tumors. The same analysis steps see . Our anap-value < 0.05) gene ontology enrichment in terms such as melanoma and melanogenesis. Furthermore, these genes are involved in several important biological processes and molecular functions, such as transcriptional regulation and signaling, a finding that provides the basis for further experimental validation.In a next step, the analysis was enriched by incorporating TCGA RNASeq data from the same samples to identify genes that negatively correlated with the miRNA patterns using the Spearman correlation method with Rho < −0.25. Subsequent target prediction analysis between the signature miRNAs and negatively correlated mRNAs resulted in two sets of putative miRNA gene targets  as input see .To test whether incorporating clinical characteristics into the analysis would improve the results of recurrence prediction, the inclusion of various demographic and staging related AJCC characteristics was also tested. The best prediction model chose to include the following features in the miRNA signature: sample type, AJCC T stage and AJCC stage. As a result, we observed increased predictive performance to 96.51% accuracy, 96.67% specificity and 98.38% sensitivity when using cross validation and 85.38% accuracy, 88.35% specificity and 92.86% sensitivity in the external test set . Thus, pWe report, herein, the diagnostic and prognostic potential of miRNAs as reliable biomarkers in CM and discuss their functional correlation with previously reported gene expression signatures. Incorporation of such signatures in clinical practice could assist clinicians to take more informed decisions. However, different studies based on different data, or even based on the same data but using different bioinformatics tools result in inconsistency regarding specific signatures. Overcoming these limitations requires larger patient cohorts, multi-omics data instead of single-omics, more elaborate bioinformatics analysis and more sophisticated artificial intelligence tools.With a special focus on CM recurrence and metastasis, we applied a comprehensive data analysis approach to publicly available NGS data, integrating statistical, bioinformatics and network analytics. We analyzed the largest available miRNA-Seq data cohort of melanoma patients and identified one miRNA signature of recurrence and one miRNA signature of metastasis. Integration of RNA-Seq data from samples from the same patients allowed functional enrichment of these signatures. The targets of miRNA signatures are related to melanoma, melanogenesis, transcriptional regulation and signal transduction. In addition, miRNAs of these signatures are associated with response to several promising immunotherapies. Of note, the identified miRNAs can potentially target signature genes which, in turn, are involved in metastasis and could be used to predict patient survival.The emerged 7 miRNAs signature related to recurrence . Testing our models in an external test set, i.e., completely unseen patients’ samples for the trained models has an accuracy of 73.85% and 82.09% in predicting melanoma recurrence and metastasis, respectively. In addition, the prediction models that we produced have the potential to stratify patients, by identifying those who are at risk for tumor recurrence and metastasis. The ability of the proposed hybrid dimensionality reduction and classification technique to identify the optimal subset of clinical features and combine them in an optimal and non-linear manner allowed the improvement of the predictions. Moreover, the integration of additional clinical data in the recurrence miRNA signature raised the predictive accuracy in the external test set from 73.85% to 85.38%. The combination of gene expression profiles with clinical data significantly improves the predictive performance as has been previously highlighted by similar studies on CM ,100. In Several recent studies support the value of circulating miRNAs as potential biomarkers of CM . As suchRecent efforts to perform transcriptomics profiling on large cohorts of CM patients enable the better understanding of melanoma biology. Bioinformatics, heuristic optimization, and machine learning classification models, when combined and applied to big data provide unique opportunity to identify potentially useful miRNA signatures as biomarkers from precision diagnosis and timely prognosis of metastasis or recurrence in response to treatment. Several studies evaluating the prognostic significance of miRNAs in CM recurrence and metastasis had inconsistent results, possibly attributed to methodological variations. Different studies based on different data, or even based on the same data, but with different analytics tools come up with different inconsistent signatures. Larger cohorts of patients, multi-omics instead of single-omics data, better bioinformatics tools and better AI tools are needed to overcome these limitations. However, even in this case, experimental low-throughput validation is required before such signatures can be incorporated in clinical practice.


Arising from D. Xiong et al. Nature Communications 10.1038/s41467-020-18546-x (2020)1, the ImmuneCells.Sig was identified as a gene expression signature to predict response to immune checkpoint therapy (ICT) from two immune cell subpopulations that are highly enriched in tumors not responding to ICT. The derived signature achieved high prognostic value in the discovery dataset and three validation datasets, comparing to 12 previously reported ICT response signatures in melanoma patients. We found that the performance reported in the original paper can only be achieved by using predictive models trained individually on the same validation dataset. As the validation stage only reports training error, the results could be overoptimistic and the performance could drop if a model is trained on one dataset and applied on another dataset.In Xiong et al.1 analyzed scRNA-seq datasets from multiple studies to determine if certain types of immune cells and their subclusters are associated with ICT outcomes. They found that two immune cell subpopulations, namely, TREM2hi macrophages and γδ T cells are highly enriched in the ICT non-responding tumors. Based on this finding, they further identified a gene expression signature named ImmuneCells.Sig from the scRNA-seq datasets and a bulk gene expression dataset GSE782202 for the purpose of predicting response to immunotherapy. The authors verified the correlations of the signature with ICT outcomes and found that the signature achieved high prognostic value in the discovery dataset (GSE78220 AUC 0.98). ImmuneCells.Sig was then validated on three independent gene expression datasets of pretreatment melanoma , and achieved AUC values of 0.96, 0.86, and 0.88, respectively. PRJEB23709 is further split into two sub-datasets according to the treatment scheme, and ImmuneCells.Sig achieved AUC values of 0.88 and 0.93 for the two subsets respectively. Comparisons with AUC values of 12 previously reported ICT response signatures show that ImmuneCells.Sig predicts the ICT outcomes of melanoma patients more accurately across the above four gene expression datasets of melanoma.Xiong et al.https://github.com/donghaixiong/Immune_cells_analysis, version 2.7.4), the AUC values reported in the original paper can only be achieved by using predictive models trained individually on the same validation dataset. In such a validation setup, the reported AUC value in fact only reflects the training error rather than the test error. Hence, the result can be overoptimistic and may not truly reflect the classification accuracy if any of the trained models is applied on other external datasets.Our concerns arise from an important issue in the validation methodology of the prognostic values of ImmuneCells.Sig, which would prevent the generalization of its prediction capability on other datasets and hence limit its utilization value in medical practice. The validation datasets  were not used in developing the gene list of ImmuneCells.Sig. However, in the source codes that we retrieved from the Github repository released by the authors . The average AUC values from 50 bootstrapping in all datasets are higher than 0.75 from random gene sets Fig. . Notably6. Clearly, with only the performance on the training set, it is insufficient to conclude that the features can apply to other datasets due to possibilities of model overfit.When applying machine learning algorithms to a real-life scenario, the generalizability of the model has to be evaluated using test data which shall not appear in the model training procedure. To this end, the datasets from which the model learns its parameters shall typically be split into three sets, namely, training, validation, and test sets. The training set is used to fit the models, the validation set is used to estimate prediction error for model selection, while the test set is used for assessment of the generalization error of the chosen model. Ideally, the test set should be kept in a “vault”, and be brought out only at the end of the data analysisWe next tested the generalization capability of ImmuneCells.Sig using the four melanoma datasets from the original paper (Methods). We found that the predictive model using features from ImmuneCell.Sig did not show robust prediction results Fig. . In mostIn summary, our results do not dispute the potential immune suppressive roles of the cell subtypes identified in the original paper, or the potential predictive values of the genes in the ImmuneCells.Sig gene set. Rather, we have demonstrated that the predictive model developed in the original paper is not able to generalize across different RNA-seq datasets. Although ImmuneCells.Sig was not developed in the validation datasets, the reported predictive error on each validation dataset in the original paper is actually the training error from classification models trained on the same dataset. Therefore, the generalizability of the model was not evaluated in the original paper. We further demonstrated that ImmuneCells.Sig with the nearest-centroid classification model as described in the original paper was unable to predict ICT outcomes well on independent test sets not seen in the training set. Therefore, we conclude that there is insufficient support that ImmuneCells.Sig could be applied for clinical decision making in melanoma patients receiving immunotherapy.2, GSE910613, PRJEB237094, and MGSP5 directly from Xiong et al.1 . The preprocess procedure in the original paper included three steps. First, the raw RNA-seq reads were aligned to the hg19 human reference genome using Bowtie-TopHat (version 2.0.4)7. Then, the gene expression read counts were obtained with the htseq-count Python script from HTSeq v0.11.1 (https://htseq.readthedocs.io/en/release_0.11.1/). The read counts data were further transformed by regularized log (DESeq2 v1.28.18).We downloaded the preprocessed datasets GSE782209 and a bulk gene expression dataset GSE782202. A nearest-centroid classifier11 was then trained by cancerclass v1.32.0 R package10 to predict ICT response using the identified signature. The performance of the prediction was then evaluated by the receiver operating characteristic curves (ROC) and the area under the ROC curve (AUC).The ImmuneCells.Sig signature was discovered from the scRNA-seq datasets1. For fair comparison, the number of genes selected equals the number of genes of the ImmuneCells.Sig signature for each dataset, respectively . After 50 bootstrapping in all datasets, the highest AUC value could reach 1 for GSE78220, 0.917 for GSE91061, 0.902 for PRJEB23709, and 0.835 for MGSP (Fig. To illustrate the effect of overfitting due to training and testing on the same dataset, we evaluated random gene sets with the same implementation. We randomly drew a set of genes from all the genes present in each gene expression dataset without considering their biological functions as a signature, and trained nearest-centroid classifiers individually on each dataset using the model fitting code provided by the original paperTo test the generalization capability of the ImmuneCells.Sig signature, we changed the prediction process such that one dataset was used for training a model with the original model fitting code without modification, while the remaining independent datasets were used for testing. The predictive models using ImmuneCell.Sig did not show robust prediction results Fig. . In some


All samples were fresh-frozen from primary tumors. The estrogen receptor-positive (ER+) BCCS subtypes were: PCS1 (18%) good prognosis, stromal infiltration; PCS2 (15%) poor prognosis, highly proliferative; PCS3 (13%) poor prognosis, highly proliferative, activated IFN-gamma signaling, cytotoxic lymphocyte infiltration, high tumor mutation burden; PCS4 (18%) good prognosis, hormone response genes highly expressed. The ER− BCCS subtypes were: NCS1 (11%) basal; NCS2 (10%) elevated androgen response; NCS3 (5%) cytotoxic lymphocyte infiltration; unclassified tumors (9%). HER2+ tumors were heterogeneous with respect to BCCS.Breast cancer is heterogeneous in prognoses and drug responses. To organize breast cancers by gene expression independent of statistical methodology, we identified the Breast Cancer Consensus Subtypes (BCCS) as the consensus groupings of six different subtyping methods. Our classification software identified seven BCCS subtypes in a study cohort of publicly available data ( The intrinsic subtypes5, also known as the PAM50 subtypes, are commonly used to describe molecular features of tumors in addition to hormone receptor and HER2 status. Several groups have developed subtype systems specific to the clinically important triple-negative breast cancers (TNBC); i.e., tumors that are estrogen receptor-negative (ER−), progesterone receptor-negative (PR−), and HER2- (TNBC)8. Lehmann et al. identified six subtypes, Burstein et al. identified four subtypes, and Jézéquel et al. presented three subtypes. The clinical translation and utility of these systems could be hampered by these discordant results, likely due to diverse algorithms and patient cohorts.Gene expression-based signatures and subtyping systems are widely recognized as valuable methods for disease stratification in breast cancer. Several signatures9. The CRCSC used network cluster analysis to form subtypes that drew on information from six independent subtype systems. In a sense, two samples were classified into the same consensus molecular subtype if the six systems largely agreed that they should be classified together. This methodology had the effect of minimizing bias due to the clustering method or training cohort.The colorectal cancer community also faced the problem of multiple validated gene expression-based subtyping systems. In the absence of a gold-standard method, the Colorectal Cancer Subtyping Consortium (CRCSC), published the consensus molecular subtypes (CMS) of colorectal cancer13, evidence of the heterogeneity of HER2+ tumors. Moreover, current subtyping methods for ER+ breast cancers have not identified differential sensitivity to CDK4/6 inhibitors.Current subtyping results for breast cancer are also limited in their abilities to inform how non-TNBC tumors respond to treatments. For example, results have shown differing sensitivity to therapeutics depending on the concentration of tumor-infiltrating lymphocytes (TILs) in HER2+ tumorsIn this study, we developed a gene expression-based subtype system for breast cancer by adapting the methods leading to the colorectal cancer CMS subtypes. We chose to subtype breast cancers overall, rather than restricting attention to TNBC tumors, to avoid unnecessarily restricting the scope of the work. Following the derivation of the Breast Cancer Consensus Subtypes (BCCS) and a computer application to generate the subtypes, we identified BCCS in cohorts based on three different gene expression platforms and analyzed the cohort-independent biological and clinical features of the subtypes.n = 1992), Affymetrix (n = 2923), and BRCA (n = 1035), with gene expression based on Illumina bead array, Affymetrix microarray, and RNA-sequencing, respectively , and METABRIC-B (n = 995) using the discovery-validation partition of Curtis et al.14. BCCS subtypes were trained in METABRIC-A and their features were explored in all three cohorts . Application of the consensus subtyping method  was first applied to the METABRIC-A cohort, which includes both ER+ and ER− samples. Six gene filtering and subtyping methods (A–F) were selected to mirror those used in consensus molecular subtyping of colorectal cancershod Fig.  resultedhod Fig. . Becausen = 474, Fig. n = 193), NT2 (n = 183), NT3 (n = 73), called the ER− Consensus Training Subtypes, with unclassified samples NT0 (n = 25). (All ER− samples in METABRIC were used to provide a sufficiently large training set.) Consensus subtyping applied to the METABRIC ER+ samples , PT2 (n = 316), PT3 (n = 354), and PT4 (n = 351), called the ER+ Consensus Training Subtypes, with unclassified samples PT0 (n = 57).Consensus subtyping applied to the METABRIC ER− samples . Predictors of T1–T5 were defined on a discovery set of 75% of the METABRIC-A cohort and evaluated in the complementary test set (25% of samples) as predictors of T1–T5. Accuracy of these predictors reached a plateau of 92% in the test set using a predictor based on 500 kTSP models. This application was called BCCSclassifier(ER+/−). The sets of samples predicted to be in T1–T5 by BCCSclassifier(ER+/−) in any cohort were called Breast Consensus Subtype 1–5 (BCS1–BCS5), respectively.Positive Consensus Subtype 1–4 (PCS1–PCS4), respectively. We similarly derived the application BCCSclassifer(ER−) which computed Negative Consensus Subtype 1–3 (NCS1–NCS3) by training to NT1–NT3. Samples unclassified by these applications were labeled “tie”.This process was repeated to define a predictor of PT1–PT4 in the METABRIC ER+ samples. Using 750 predictors achieved an accuracy of 83% in a test subset of 25% of the samples. The subtypes this application [BCCSclassifer(ER+)] predicted to be in PT1–PT4 in a sample set were called estrogen-receptor The BCS, PCS, and NCS subtype assignments for all samples in METABRIC, Affymetrix, and BRCA cohorts are delineated in Supplementary Data The six methods employed in the derivation of the BCCS subtypes produced subtypes that potentially varied in the number of subtypes and may have disagreed in classifications Fig. . The conp = 3 × 10−48) than the proportions in PCS1 (0.23) and PCS4 (0.20), combined. Similarly, PR− ER+ tumors were more likely to be in PCS2 and PCS3 than PCS1 and PCS4 . The BCCS subtypes did not exhibit notable differences in proportions of LN+ tumors, tumors at least 2 cm in size, or patients over 50. The relationship between BCCS and HER2 status is discussed in depth below.As an initial exploration of the features of the BCCS subtypes, we computed the proportions of samples in each subtype having selected clinico-pathological traits for METABRIC and BRCA , and prognosis was not significantly different between PCS2 and PCS3 . Prognosis for the BCCS ER− subtypes varied slightly for METABRIC  response, IL6-JAK-STAT3 signaling, complement, apoptosis, P53 pathway, and other pathways related to immune response and hypoxia, while PCS2 was enriched in estrogen response compared to PCS3.GSEA was used to uncover subtype-specific biological activity. Specifically, enrichment for each Hallmark v7 gene set was tested between each pair of PCS1–4 and NCS1–3 in all three study cohorts  compared to PCS1.The two good prognosis ER+ subtypes, namely PCS1 and PCS4, also varied in gene set enrichment compared to other subtypes. Compared to PCS2-4, PCS1 was enriched in TGF-γ response, IL6-JAK-STAT3 signaling, complement, and other immune response gene sets compared to NCS1–2. Since all tumors in these comparisons were ER−, we examined expression levels of ESR1, PGR, AR, and estrogen response genes most significantly varying with respect to BCCS .Turning to the ER− subtypes, NCS1 was enriched in E2F targets, G2M checkpoint and MYC targets compared to NCS2 and NCS3, typical of basal-like tumors; NCS2 was enriched in androgen response and estrogen response compared to NCS1 and NCS3; and NCS3 was enriched in IFN-p < 0.01 for all comparisons). Of note, the mean expression of ESR1 was also significantly lower in each of PCS1 and PCS3 than PCS2 . Also, PCS3 samples in BRCA were more likely to be <90% positive for ER by immunohistochemistry (p = 2.1 × 10−8), PCS4 samples were more likely to be ≥ 90% positive (p = 1.5 × 10−10).Mean expression levels of hormone response genes also varied across subtypes of ER+ tumors , and PD-L1 (CD274) were all significantly higher in NCS3 compared to other samples , and significantly higher in PCS3 compared to other ER+ samples , and highest in PCS3 compared to other ER+ METABRIC tumors  depends on the levels of checkpoint inhibitor proteins in the tumor, as well as other features of immune activityγ response compared to HER2+ tumors in PCS2, and HER2+ PCS2 tumors were enriched in estrogen response compared to HER2+ PCS3 tumors. The same relationships held for HER2− PCS2 and PCS3 tumors. Thus, the BCCS subtypes articulated heterogeneity in HER2+ tumors and HER2− tumors alike.The intrinsic subtype system PAM50) defined the Her2-enriched subtype to represent all HER2+ tumors. In contrast, HER2+ tumors were found in multiple BCCS subtypes in METABRIC and BRCA  has previously been identified in TNBCp = 7.4 × 10−29) and natural killer cells (p = 3.5 × 10−29) compared to all other tumors. With respect to ER+ tumors, PCS3 exhibited significantly greater infiltration of cytotoxic lymphocytes (p = 6.3 × 10−44) and natural killer cells (p = 3.3 × 10−29) compared to other ER+ tumors. Mean infiltration of fibroblasts was greatest overall in PCS1 (p = 6 × 10−100), and greatest in NCS3 (p = 2.6 × 10−8) among ER− tumors. Endothelial cell infiltration was greatest in NCS3 or PCS1 in all three cohorts. Distributions for selected cell populations for Affymetrix and BRCA were similar , were assessed with respect to the BCCS subtype . This relationship was supported by the ASCAT tumor purity distribution in BRCA .Tumors with low or moderate cellularity in METABRIC were disproportionately found in PCS1 (Supplementary Table 14 (IntClust 1–10) grouped samples in METABRIC-A by patterns of copy number changes. The distribution of BCCS subtypes within these clusters showed several significant correlations ; IntClust 8  was enriched in PCS4 (p = 1 × 10−32); IntClust 10, a high genomic instability group typical of basal tumors (5 loss/8q gain/10p gain/12p gain), was enriched in NCS1 (p = 1 × 10−115); IntClust 5, associated with ERBB2 amplification, was enriched in NCS2, PCS2 and PCS3 (p = 1 × 10−18); IntClust 6 (8p12 loss) and IntClust 7  were enriched in PCS2 . For chromosome arms most altered in integrative clusters, we selected as representative cytobands those from the chromosome arm most significantly altered in METABRIC-A. Copy number changes for representative cytobands in BRCA were consistent with these patterns  has been assessed for METABRIC samples (see the “Methods” section). There were eight genes mutated in at least 20% of samples in some BCCS subtypes in METABRIC . PCS1 exhibited the greatest mutation rates for PIK3CA , CDH1 , and MAP3K1 . Of note, mutations in CDH1 have been associated with lobular breast cancer18, which is most frequent in PCS1.Mutation status for a selected set of genes . Of note, among ER+ tumors, TMB was significantly elevated (p = 1 × 10−16) in PCS3, and TMB was not significantly different across NCS1, NCS2, and PCS3 (p = 0.18). Given the elevated immune activity in NCS3 the low level of TMB was surprising, however, this could be an aberration due to the small number of NCS3 samples in BRCA (n = 16).The degree of genomic instability, as represented by tumor mutation burden , varied significantly by BCCS subtypes in BRCA . Beta values of methylation sites were also screened for their abilities to predict membership of individual BCCS subtypes (see the “Methods” section). The most significant predictions were for PCS1, NCS1, and NCS2; Fig. Methylation in the promoter of ESR1 has been associated with resistance to hormone therapy6 and the Burstein subtypes7 were analyzed. The Burstein subtypes were derived from GSE76275 (see the “Methods” section). BCCS subtypes were computed for these samples using BCCSclassifier (ER−). The refinement of TNBCtype subtypes to the 4 TNBCtype4 subtypes20 was computed using the TNBCtype tool (see the “Methods” section). While GSE76275 reported 198 samples as TNBC, the TNBCtype tool computed subtypes only for the 157 samples it assessed as estrogen-receptor negative by ESR1 expression level.The relationships between BCCS subtypes for TNBC tumors and prior subtyping systems for TNBC, specifically TNBCtypeTabulation of the BCCS subtypes NCS1, NCS2, and NCS3 and the Burstein subtypes  and RBSig score was significantly higher in PCS3 than PCS2 . As expected, the RBSig score was significantly higher in RB1 mutated samples than RB1 wild-type samples (p = 1.7 × 10−10). Results in BRCA were similar.The CDK4/6 inhibitor class of drugs has been found to improve survival in metastatic ER+ breast cancer patientsHerein, we developed the Breast Cancer Consensus Subtype (BCCS) system based on whole-transcriptome data from fresh-frozen primary tumor samples. The derivation method was designed to reduce bias due to the clustering method or expression assay technology. The BCCS subtypes  exhibited differences in tumor biology as assessed by gene expression, patterns of DNA alteration, and immune and stromal cell infiltration. Features specific to some subtypes have been associated with treatment outcomes in other studies, but the clinical utility of BCCS remains to be established.γ-signaling and cytotoxic lymphocyte infiltration compared to PCS2. These characteristics of PCS3 were previously identified in a set of tumors resistant to aromatase inhibitors29. Of note, methylation in the promoter of ESR1, which is being investigated as an indicator of hormone therapy resistance30, was significantly more common in PCS3 than PCS2.The most novel feature of the BCCS subtype system was the partition of poor prognosis ER+ tumors into two subtypes, PCS2, and PCS3. Both subtypes were enriched for pathways associated with a high proliferation rate, however, PCS3 (18% of ER+ tumors) was elevated in IFN-31. Analysis of clinical trials for TNBC have shown improved response for PD-L1+ tumors with high TIL counts and infiltrates by CD8+ T-cells32. We have shown that in addition to high PD-L1 expression, NCS3 and PCS3 tumors had elevated IFN-γ-signaling and cytotoxic lymphocyte infiltration. PCS3 tumors also had high TMB, which has indicated a positive response to ICI in multiple cancer types33. These multiple indicators suggest that NCS3 and PCS3 tumors may be responsive to ICI, although data on ICI response is unavailable.Staining intensity for PD-L1 protein has been used to select metastatic cancer patients for treatment by ICI, however, treatment response in PD-L1+ tumors of multiple cancer types has been uneven20. This led investigators to conclude that IM and MSL were not distinct tumor subtypes, eliminating them from TNBCtype. In the case of PCS1, however, the majority of samples had PIK3CA mutations, exhibited gain in 1q, and many had promoter methylation for ESR1. These are features of tumor cells distinct from those of other subtypes, providing evidence that PCS1 is a distinct tumor subtype.In addition to two poor prognoses ER+ tumors, BCCS presented two good prognoses ER+ subtypes, PCS1 and PCS4, that exhibited significantly different features with respect to gene expression, mutation frequency, copy number changes, tumor cellularity, and promoter methylation of ESR1. Of note, tumors with low cellularity and high stromal cell infiltration were most likely in PCS1. Single-cell analysis of TNBC samples showed that the TNBCtype subtypes with low cellularity (IM and MSL) were largely determined by gene expression from stromal and immune cell infiltrates34. The BCCS subtypes described heterogeneity in HER2+ tumors with respect to numerous traits beyond lymphocytic infiltration.We chose to analyze ER+ and ER− samples separately because subtyping of all tumor samples BCS1–5) separated them by ER status, a fortiori. This was not the case for HER2 status; HER2+ tumors were heterogeneous with respect to BCCS  tissue and a specific assay technology, which is beyond the scope of this initial study.35, leading to uncertainty in the BCCS classification. Secondly, some tumors may have a mixed subtype; i.e., features of multiple subtypes. This could be due to intra-tumoral heterogeneity, as occurs in colon cancer36. The use of continuous scores to estimate the prevalence of a BCCS subtype among the cells in a tumor would reduce the impacts of these limitations, as previously published for colorectal cancers37. The derivation of continuous scores for BCCS prediction is beyond the scope of this paper.A further limitation of the BCCSclassifier is that it is necessary to determine the ER status of a tumor prior to classification. For some patients with a low percentage of ER-positive staining, the ER status may be uncertainIn conclusion, we developed BCCS using whole-transcriptome data from fresh-frozen primary tumor samples, having biological and clinical features independent of cohort and gene expression assay platform. Among ER+ tumors, BCCS identified two poor prognosis subtypes, which differed in the degree of immune activity, and two good prognosis subtypes with one high in stromal infiltration and the second elevated in hormone response. The BCCS subtypes described significant heterogeneity in HER2+ tumors. Clinical application of BCCS will require an analytically valid assay applicable to FFPE tissue and evidence that subtypes predict responses to treatments.The BCCS were discovered and analyzed using the following three publicly available whole-genome transcription datasets.n = 1992). The METABRIC patient cohort was formed from 1992 primary breast cancer patients14. Clinico-pathological traits  and METABRIC-B (n = 995) consisting of the previously published14 discovery and validation cohorts. To denote estrogen receptor status we used the status determined by gene expression as reported by the authors.METABRIC cohort n = 992. The n = 2923). This cohort was formed by merging clinico-pathological data (Supplementary Table 39 and corrected for batch effects with ComBat using the SVA R package.Affymetrix cohort (n = 1035). Clinico-pathological data for the TCGA-BRCA cohort40 (BRCA) was obtained from NCI Genomic Data Commons  and Broad Institute GDAC Firehose (https://gdac.broadinstitute.org) and summarized in Supplementary Table 41. Transcript per million (TPM) expression measures were computed with the Rsubread feature counts algorithm42. Unless otherwise noted, log2(TPM + 1) was used as the expression value of a gene for a sample in this cohort. For GSEA, described below, gene expression measures were computed using the DESeq2 R package43, as recommended by GSEA authors. Estrogen receptor status was determined by IHC. Somatic mutation data for BRCA was obtained from LinkedOmics (http://www.linkedomics.org/data_download/TCGA-BRCA/). Tumor mutation burden was computed using the maftools R package44. DNA methylation data for BRCA samples were downloaded from GDC. Methylation data from the Illumina Human Methylation 450 arrays were restricted to the probes in common with the Illumina Human Methylation 27 array. Tumor purity (aberrant cell fraction) has been used as a measure of tumor cellularity. Tumor purity computed with ASCAT45 was obtained from COSMIC (https://cancer.sanger.ac.uk/cosmic/).BRCA cohort n = 035. Clin14. The collection of human data for TCGA-BRCA was carried out under the requirements of the TCGA Network40. All other data used in this study were obtained from the Gene Expression Omnibus and subject to the National Center for Biotechnology Information requirement that appropriate consent/permission had been obtained to submit data to a public repository. All ethical requirements from the use of the above human data were followed in this study.Data acquired in the METABRIC study were obtained under the ethical requirements of the relevant institutions and the study groupIn this study, analysis was restricted to genes, as represented by ENTREZIDs, whose expression was measured in each of the above three cohorts. In the METABRIC cohort, we associate with each such gene the Illumina probe annotated to this gene with maximal interquartile range in the METABRIC-A cohort. The expression of the gene for a sample in METABRIC is then reported as the expression level of the selected probe. In the Affymetrix cohort, the jetset R package was used to select a probe to represent a gene. We further restricted to the ENTREZIDs associated with a unique gene in the BRCA gene count data from Ensembl v90. In the end, for each of a set of 11,569 ENTREZIDs and each sample in any of the three study cohorts, there is a unique expression value of the ENTREZID for the sample.9, consists of the steps: (1) creating multiple subtyping systems using a variety of methods, (2) clustering the network of all possible subtypes, (3) for each clustered set of subtypes, identifying samples frequently assigned to the cluster’s subtypes, so-called core samples. These steps, as applied in this study, are described in the following subsections. This method was applied in the METABRIC-A cohort for subtyping all breast cancer samples, the METABRIC ER− samples for subtypes specific to ER- breast cancer, and the METABRIC ER+ samples for subtypes specific to ER+ breast cancer.The consensus subtyping method, as it was applied to create the CMS of colorectal cancerSix independent methods (A–F) were applied to the training domain to create different subtyping systems. Each of the six methods involved filtering the genes considered, applying an unsupervised clustering method to the filtered gene expression dataset, and selecting an optimal number of clusters  of the consensus matrix for k clusters is a more quantitative measure of cluster coherence. An optimal number of clusters by this metric is the value at which the area under the corresponding CDF has reached a plateau. Finally, ConsensusClusterPlus produces a “delta plot” to show the amount of change in area under the CDF between k and k + 1 clusters. In all methods we considered the range of 2–8 clusters.Methods A–C were executed using the ConsensusClusterPlus R packagev = v1,…,vm is median(|vi–median(v)|).) The clustering method selected was hierarchical clustering with average linkage and distance 1−Pearson correlation of the vectors of gene expression values. Clustering was executed with ConsensusClusterPlus, resampling 90% of the genes 1000 times, and considering 2–8 clusters.In this method, the set of genes was filtered to those with median absolute deviation (MAD) >0.5. 49 was used to select the optimal number of clusters. These steps were carried out using functions from the cluster R package50.The set of genes was filtered to those with the highest 5% of interquartile ranges. Clustering was performed by the Partition Around Medoids (PAM) method, testing 2–8 possible clusters. The gap statistic51. The NMF procedure was run 30 times, testing 2 through 8 possible clusters. The number of clusters and set of genes was selected using heatmap and cophenetic coefficient diagnostic plots.Possible sets of genes were tested by restricting to those with standard deviation >0.5, 0.8, 1, or 1.1 for the expression values in the training domain. Clustering was performed for each expression dataset with non-negative matrix factorization as implemented in the NMF R packageThe set of genes was filtered to those with an interquartile range >1.2. Otherwise, Method F agreed with Method E.9. In our study, the above six methods were applied to the training domain of samples , for each of the three subtyping projects. Application of the above six methods to the training domain resulted in a set of k total subtypes, with a different k for each of the three projects. A pair of subtypes were considered significantly connected if a hypergeometric test for over-representation of samples in the intersection of the subtypes had Benjamini–Hochberg adjusted p-value < 0.001. A graph network was defined with nodes these k subtypes, and with an edge between a pair of nodes if they were significantly connected. To cluster such a network, resulting in groups of associated subtypes, Markov cluster analysis was applied with the MCL R package52. To assess confidence in the robustness of such an association, the MCL cluster algorithm was repeated 1000 times, each iteration sampling 80% of the samples. A k × k matrix was defined with each entry the frequency that the pair of subtypes were in the same cluster over the 1000 applications of MCL. These frequencies were compared to the cluster assignments by MCL applied to the network of all samples to measure the stability of a subtype’s cluster assignment. Specifically, for a subtype assigned to a cluster, the stability score of the subtype was the average frequency for all pairs formed by this subtype and other subtypes in the cluster. Clustering performance was evaluated with weighted silhouette width using the WeightedCluster R package, using the stability scores as weights.This analysis followed the process described in Guinney et al.An application of MCL can result in varying numbers of clusters depending on the inflation factor parameter. MCL was applied for inflation factors from 1 to 10, in increments of 0.25. The MCL result with the lowest inflation factor at which the resulting weighted silhouette width reaches a maximal plateau was chosen, under the constraint that no cluster had fewer than 50 core samples (see the next subsection).x in the training domain and a cluster C consisting of subtypes s1,…,sk, a hypergeometric test for over-representation of x as a member of the si’s was performed. Then x was a core sample of C if the hypergeometric test had a p-value < 0.05. The core samples of all clusters formed the Consensus Training Subtypes for the subtyping project.The preceding process resulted in clusters of subtypes. Each cluster gives rise to a subtype of the training domain as follows. For such a sample The above consensus subtyping method is too lengthy to repeat in practical applications, and cannot be applied as a single-sample classifier. Instead, a computer application to generate subtypes on an arbitrary whole-transcriptome dataset was trained on the Consensus Training Subtypes of the training domain. The training domain was further partitioned randomly into discovery set (75%) and test set (25%). Candidate classification applications were developed on the discovery set, and the optimal performer on the test set was selected.53. A kTSP classifier contains a set of pairs of genes, and for a sample to be classified, it tests which gene in each pair has the highest expression and makes a classification decision based on the results of these tests. A single kTSP classifier decides whether a sample is more appropriately classified into subtype A or subtype B. To generalize the method to an arbitrary number of subtypes, C1,…,Cn, a kTSP classifier is defined for each pair of subtypes. Then, to classify a new sample, the kTSP classifiers for all pairs are evaluated, and the sample is classified into the subtype that is most frequently predicted, if one exists, and is labeled a “tie” if none exists. Here, kTSP predictors were derived using the ktspair R package.The core of this application is a set of prediction models derived with kTSPm sets of genes, we derived an optimal kTSP classifier from this set of genes. The Multi-kTSP classifier consisted of the resulting set of m kTSP classifiers. To classify a new sample, we evaluated each of the m predictors, and assigned the sample to the most frequently predicted subtype, or “tie” if there wasn’t a unique maximum. We created such predictors by selecting genes from among the most highly varying 1500 genes, and creating 250, 500, or 750 kTSP models in the discovery subset of training domain. We selected the minimal number of models for which accuracy in the test set as a predictor of the Consensus Training Subtypes reached a plateau.In typical applications, kTSP classifiers reach optimal performance with fewer than 10 pairs of genes. To add information from other genes we used a resampling method to derive what we have called a Multi-kTSP classifier. Specifically, we formed a family of m subsets of genes, for some predetermined number m, each consisting of randomly sampled 75% of the overall set of candidate genes. For each of 8, non-negative matrix factorization (Method II) was used by Burstein et al.7, and k-means and consensus clustering (similar to Method I) was used by Lehmann et al.6.To test the robustness of the subtyping method we selected an alternative set of six methods for re-subtyping the ER+ METABRIC dataset. The six methods varied in how the set of genes was filtered, how distance between samples was computed from expression data, and the method of clustering  was tested. For analysis of BRCA, gene expression values normalized by DESeq2 were used43, as recommended by the GSEA authors.GSEA55, implemented as an R package (http://github.com/ebecht/MCPcounter), was used to quantify the abundance of populations of immune and stromal cells in a tumor sample. For each population of cells, and each sample, MCPcounter computes the mean of expression levels of a population-specific set of genes for this sample. These population means were used as a measure of the infiltration of the cell population in this sample. To create measures on a uniform scale, population means were median centered.The MCPcounter systemDifferential methylation of specific sites with respect to BCCS subtypes was tested in BRCA. Specifically, for each subtype and array probe we applied the Kruskal–Wallis tested to the probe’s Beta values and membership in the subtype, and then ranked the probes by significance.n = 198). This dataset was formed from GSE76275 samples reported by the authors as triple negative7. Gene expression values were computed by fRMA applied to raw CEL files from Affymetrix hgu133plus2 arrays. The six subtypes of TNBC developed by Lehmann et al.6 were subsequently computed with the TNBCtype algorithm56. We applied this algorithm through the web interface (https://cbc.app.vumc.org/tnbc/). TNBCtype4 subtype of a sample was computed from the six subtype probabilities20.Subtype systems for TNBC were analyzed in the GSE76275 TNBC cohort (http://www.r-project.org) version 4.0 and Bioconductor packages (http://bioconductor.org) version 3.13. Kaplan–Meier survival models were plotted with the survminer R package. The result of a statistical test was considered significant if the p-value was < 0.05 unless an exception was explicitly stated. Hypergeometric tests were performed with the dhyper and phyper functions. To compare the mean values of a continuous variable between subsets we used the non-parametric Kruskal–Wallis test (two-sided). Accuracy and Cohen’s kappa statistic were used to assess the significance of a discrete predictor of subtypes. The midline in a boxplot indicates the median, the upper and lower edges indicate the quartiles, and the whisker lines are 1.5 times the interquartile range. The ability of a continuous score to predict membership in a subtype was assessed with the receiver operator characteristic (ROC) curve. The area under the ROC curve (AUC) gives a numerical measure of a score’s predictive significance. The quality of the prediction is better than random if AUC is >0.5 and improves as AUC increases up to 1. The plotROC R package was used for these computations.All statistical analyses were performed using R (Further information on research design is available in the Supplementary InformationSupplementary Data 1Reporting Summary