Speech emotion recognition often encounters the problems of data imbalance and redundant features in different application scenarios. Researchers usually design different recognition models for different sample conditions. In this study, a speech emotion recognition model for a small sample environment is proposed. A data imbalance processing method based on selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed to reduce the impact of sample imbalance on emotion recognition results. In addition, feature selection method based on variance analysis and gradient boosting decision tree (GBDT) is introduced, which can exclude the redundant features that possess poor emotional representation. Results of experiments of speech emotion recognition on three databases  show that our method obtains average recognition accuracy of 90.28% (CASIA), 75.00% (SAVEE) and 85.82% (Emo-DB) for speaker-dependent speech emotion recognition which is superior to some state-of-the-arts works. With the rapid development of human-computer interaction systems, the emotional intelligence has been paid much attention in recent years, by which both the emotional state and implied intentions of the human could be obtained . PrimaryIt is widely accepted that speech conveys not only the semantic meaning but also the emotional information of speakers ,7. In reAs a utility science, the data of SER obtained in the actual application environment are not standard and the acquisition and labeling of emotional speech samples requires massive cost, in which the data imbalance of each emotion category appears frequently . In respOne of the major issues of SER is acquiring an optimal emotional feature set from initial signals . Most ofIn summary, there are two main problems with SER in small sample environment. Firstly, data imbalances always exist in the emotional corpora, which impact the learning of different emotional categories in the decision space by the classifiers . SecondlTo solve these problems, a framework of SER in small sample environment is presented to reduce the influence of individual differences and enhance the practicability of SER, in which a selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed to reduce the imbalance of emotional information. Feature selection based on variance analysis and gradient boosting decision tree (GBDT) is introduced to reduce the feature redundancy. Comparative experiments are performed to verify the effectiveness of the proposed method using CASIA  , Emo-DB The main contributions of this paper are twofold. Firstly, selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed to reduce data imbalance, in which the decision space of minority class is expanded as much as possible while reducing the influence of the synthetic samples on the decision space of majority class. Secondly, feature selection based on variance analysis and gradient boosting decision tree (GBDT) is introduced, which could eliminate the redundancy between features and obtain optimal feature set with stronger characterization.The rest of paper is organized as follows. Feature extraction and data imbalance processing in SER are presented in Flowchart of the proposed SER model is shown in As shown in In data imbalance processing, a selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed for solving the problem of unbalanced data appearing in emotional classification. In feature selection, a new method based on variance analysis and gradient boosting decision tree (GBDT) is introduced to gain the lower redundancy features, in which the variance test as a feature pre-selector can quickly remove redundant features while reducing the calculation of the post-order process and GBDT can obtain the importance rank of emotional features through the fitting of the learner. Support Vector Machine is adopted to classify the emotion categories such as neutral, happy, sad, surprised, and angry.The designed samples from standard recorded database are generally used to train models for emotion recognition. However, the observation samples collected in the actual application environment are not ideal and often have sample imbalances which causes recognition problems for machines. For imbalanced data sets, samples of minority class are sparsely distributed in sample space compared with the overwhelming amount of majority class.k nearest neighbors by Euclidean distance for each sample Synthetic minority over-sampling technique (SMOTE) is a classical oversampling algorithm that constructs corresponding new samples from minority class information obtained by neighbor relations, which is a scheme based on random oversampling algorithm . The impIn spite of synthetic samples being obtained, some shortcomings of the traditional SMOTE may appear. Firstly, the new sample interpolated between sample points “a” and “b” interferes with the decision space of most types of speech samples, and even the new sample may coincide with the majority of samples. The sample point “b” is used as the neighbor of “c”, and the same problem occurs when the sample point “c” is interpolated. Then, the synthesized sample points can only be in the line between them when the sample points “a” and “e” are interpolated.When SMOTE is applied to emotional speech sample processing, it needs to be improved to overcome these shortcomings. Firstly, it uses all the minority samples in the sampling without considering whether there will be noise data in these samples . AlthougIn view of the above analysis, a selective interpolation synthetic minority over-sampling technique (SISMOTE) is proposed to solve the problem of data imbalance in SER, for which not all the minority samples need to be upsampled, but only the corresponding target points are interpolated. Difference from the traditional SMOTE is that the decision space of minority class is expanded as much as possible while reducing the influence of the synthetic samples on the decision space of majority class. Combined with the schematic diagram in k nearest neighbors of that sample are calculated based on Euclidean distance and the set of neighbors is denoted as k nearest neighbors in all speech samples for t is the regulatory factor, Q.n, and determine the sampling magnification of the algorithm N according to the imbalance ratio of the emotional sample and the number of target points, i.e., M is the number of speech samples of majority emotional class, and T is the number of speech samples of minority emotional class.As shown in l class data imbalance processing is transformed into At the same time, because SER is usually treated as a multi-classification problem, sampling rate is determined by the most size of emotion categories and the number of other emotion class with the “one-to-many” strategy. For each minority category, other emotion categories are treated as one category, i.e., Importance of each selected feature depends on the correlation between feature and emotional category. The stronger the correlation is, the better the classification ability of representative feature will be. A feature selection method based on variance analysis and gradient boosting decision tree (GBDT) is introduced for SER.The idea of variance analysis  is to usm is the number of speakers in the emotional speech samples, k is the number of emotional categories in the samples, and the n is the number of speech samples of each speaker in different emotion categories in the sample set, which is randomly selected.The formula for calculating the variance of each dimension in pre-selection isBy setting the threshold of variance, the dimension which has variance less than the threshold is considered to have little contribution in differentiating emotions, therefore such features are removed in the process of constructing the feature set. The pre-selection process of speech emotional features based on variance analysis is independent of the learner, which is equivalent to filtering the initial features and training the learner with the pre-selected feature set.Feature selection based on gradient boosting decision tree (GBDT) is introduced to eliminate the redundancy between features and obtain optimal feature set with stronger characterization. GBDT is integrated learning method based on Boosting strategy , in whicThe procedure of it can be summarized as the following steps.T and the number of leaf nodes J.K categories, and the following learning and updating process is iteratively performed T times.kth emotional class.J leaf nodes through the gradient method, calculating the gain of each leaf node, and updating the estimated values of all speech samples under the kth emotional class.C and kernel parameter The experiment was designed with the following steps. First of all, openSMILE toolkit and MATLAB R2012b were used to extract speech acoustic features, in which multidimensional features were extracted separately. And the proposed SISMOTE was carried out in data imbalance processing, in which the unbalanced emotional data reaches equilibrium. Then redundancies in emotional data were removed by the model of variance analysis and GBDT. Finally, Support Vector Machine (SVM)  was usedUsing extracted speech features, three sets of experiments were conducted. First, using the same initial feature set, the experiments by different data imbalance processing methods were performed to verify the effectiveness of SISMOTE on Emo-DB and SAVEE. Then, using the same initial emotional feature set and emotional classifier, experiments by different feature selection methods were performed to verify the validity of the proposed feature selection method on CASIA, SAVEE, and Emo-DB databases. Finally, speaker-dependent SER on CASIA, SAVEE and Emo-DB databases was carried out, in which our method was compared with some state-of-the-arts works. To ensure the rigor and fairness of the experiment, the average results are obtained after running experiments ten times.It is recorded by the Institute of Automation, Chinese Academy of Sciences. It has 9600 short mandarin utterances, in which 6 emotional states  are contained in total, and the emotional samples of this database is recorded by four speakers  in a noise-free setting with 16-bit PCM WAV at 16 kHz sampling frequency with about 35 dB .It consists of 480 short English utterances recorded by four speakers in seven basic emotions , in which the speech samples are picked from the standard TIMIT corpus and each emotion is phonetically-balanced .It is a German emotional speech database recorded by the Technical University of Berlin, by the 10 actors  of 10 statements (5 long 5 short) of seven emotions  simulation, contains a total of 800 sentence corpus, sampling rate of 48 kHz  . The speThe SISMOTE was tested using standard feature set INTERSPECH 2010  on the EAll samples of each individual were used, in which 70% samples were randomly used for training and the remaining 30% samples were used for testing. The experiment was divided into two groups based on different databases. The first group, i.e., 535 emotional speech samples in Emo-DB were randomly divided into training set and testing set in proportion (7:3), in which the emotional samples in training set were divided into 90 “anger” samples, 58 “boredom” samples, 30 “disgust” samples, 56 “anxiety” samples, 56 “happiness” samples, 36 “sadness” samples, and 36 “neutral” samples. After these unbalanced training samples were processed in different ways and training the SVM classifier, 161 test sets were used to test the classifier. The second group, i.e., 480 emotional speech samples from SAVEE were randomly divided into 336 speech samples in training set and 144 speech samples in testing set, in which the training set consists of 43 “anger” samples, 44 “disgust” samples, 38 “fear” samples, 42 “happiness” samples, 79 “neutral” samples, 48 “sadness” samples, and 42 “surprise” samples. As shown in The Emo-DB’s 374-sentence training set samples were processed in five different ways for unbalanced emotional speech. After using up-sampling method, the training set samples included 210 sentences. The training set samples were expanded to 630 sentences after using other over-sampling methods and the speech samples was balanced among emotional classes. In the same way, SAVEE’s 336 speech samples were processed by different methods. The training set samples were reduced to 266 sentences after using up-sampling method and the training set samples were expanded to 553 sentences after using other over-sampling methods.The classification model was trained using the processed data, and the test was performed using the same testing set. Comparative results using different data imbalance methods on Emo-DB and SAVEE are shown in The average imbalance rate of data between the Emo-DB and SAVEE training episodes of emotional speech sample categories exceeds 100%. As shown in At the same time, the recognition accuracy of the learning model by different data imbalance processing methods in In this section, the emotional feature selection method based on variance analysis and GBDT was tested using standard feature set INTERSPEECH 2010 on CASIA, Emo-DB, and SAVEE databases, in which the comparison among three feature selection methods  by seven kinds of classifiers , K-NearestNeighbor (KNN), Logistic Regression (LR), Decision Tree (DT), and SVM) was performed.The experiments were divided into three groups according to the source of the emotional corpus. The first group has 5400 training samples and 1800 test samples after dividing the data of CASIA. Because the CASIA data are standard and the data imbalance rate among the emotional categories of feature set is lower, the feature selection process was directly performed on the initial feature set without data imbalance processing.In the initial emotional feature set of CASIA speech sample data (1582 dimensions), the training set was processed by different feature selection methods. The average dimension of the feature set after variance analysis based feature pre-selection is 1253 dimensions, in which the threshold of variance is set to be 0.001. The average dimension of the GBDT importance assessment completion feature selection is 292 dimensions; the average dimension after RF feature selection is 230 dimensions. Different classifiers  were trained using the feature subsets generated by different feature selection methods, and the testing sets were aligned at the same time. The second group has 401 training samples and 134 testing samples after dividing the data of Emo-DB. The training set samples were expanded to 679 sentences after the data imbalance processing. Same as above, originating from the initial sentiment feature set of Emo-DB speech sample data (1582 dimension), the average number of dimensions for pre-selected feature set selected by variance analysis is 1217, and the average dimension after the GBDT feature selection is 259 dimensions. The average dimension after RF feature selection is 283 dimensions. The third group has 360 training samples and 120 test samples after dividing the data of SAVEE. After the data imbalance processing, the training set samples were expanded to 609 sentences. Same as above, originating from the initial sentiment feature set of SAVEE speech sample data (1582 dimension), the average number of dimensions for pre-selected feature set selected by the variance analysis is 1137, and the average dimension after the feature selection is 303 dimensions based on the gradient lifting tree importance evaluation, the average dimension after RF feature selection is 313 dimensions. For the speech samples under different data distributions, the emotional representation ability of the emotional feature set is improved after the feature selection. As shown in Unweighted average recall (UAR) is a great evaluation index if various types of emotions have an imbalance distribution . Table 6In this section, the proposed method was tested using standard feature set INTERSPEECH 2010 on CASIA database, in which the recognition accuracy using different numbers of samples by SVM classifier were given.As seen in In this experiment, to verify the performance of our method, comparison between our method  and some state-of-the-arts methods was carried out. The average recognition accuracy of the proposed speech emotion recognition method on emotional databases is collated, in which our method obtains average recognition accuracy of 90.28% (CASIA), 75.00% (SAVEE), 85.82% (Emo-DB) for speaker-dependent speech emotion recognition. Comparative results of recognition accuracy is shown in As shown in In this paper, a new framework of SER in small sample environment was put forward, in which the data imbalance processing method based on the selective interpolation synthetic minority over-sampling technique (SISMOTE) in small sample environment was proposed. The effectiveness of the proposal was respectively validated in multiple comparative experiments under different experimental conditions. The SISMOTE was demonstrated to be more suitable for solving data imbalance in speech emotion recognition than the traditional SMOTE. The utility of the feature selection based on variance analysis and GBDT was verified through the experimental comparison.In future work, the optimization of the model under different conditions will be carried out. For example, the sample size of the database used in this study is not large. If the data size of an unbalanced sample set is large, the distribution of samples in emotional categories may get complicated. The target domain division of a few sentimental classes may be affected, thus affecting the quality of synthesis minority emotional samples. In view of the data imbalance problem in the large sample environment, it is worthwhile to improve the existing algorithms or develop new ones.In addition, the robustness of the proposed method will be studied. More situations such as noisy environment and cross language environment will be concerned. Furthermore, data preprocessing and feature selection are two indispensable steps for both soft classification and regression. Since SISMOTE belongs to the data preprocessing and another method  belongs to the feature selection, we believe that our method can be applied to both soft classification and regression in the future. To improve the applicability of our method further, we will study on speaker-independent (SI) speech emotion recognition. Since SER is a promising work, the proposal can be applied to many occasions such as advanced driver assistant system (ADAS), remote education, human-robot interaction (HRI). The proposed method in this paper will be applied to the multi-modal emotion recognition system  and comm


Flow Cytometry is an analytical technology to simultaneously measure multiple markers per single cell. Ten thousands to millions of single cells can be measured per sample and each sample may contain a different number of cells. All samples may be bundled together, leading to a ‘multi-set’ structure. Many multivariate methods have been developed for Flow Cytometry data but none of them considers this structure in their quantitative handling of the data. The standard pre-processing used by existing multivariate methods provides models mainly influenced by the samples with more cells, while such a model should provide a balanced view of the biomedical information within all measurements. We propose an alternative ‘multi-set’ preprocessing that corrects for the difference in number of cells measured, balancing the relative importance of each multi-cell sample in the data while using all data collected from these expensive analyses. Moreover, one case example shows how multi-set pre-processing may benefit removal of undesired measurement-to-measurement variability and another where class-based multi-set pre-processing enhances the studied response upon comparison to the control reference samples. Our results show that adjusting data analysis algorithms to consider this multi-set structure may greatly benefit immunological insight and classification performance of Flow Cytometry data. Applications involve automated comparison of increasing numbers of samples, in which large numbers of cells are typically collected in every sample. Furthermore, many contemporary experiments quantitatively compare a ‘case’ (or responder) groups of samples against a ‘control’  sample group, in which expressions of identical cellular markers may be measured on the cells within all samples.Multicolor Flow Cytometry (MFC) is a powerful technique for quantitative detection of cellular marker expression at the single-cell level. MFC technology has become routine for biological studies and clinical diagnoses. In immunology, the main applications of MFC span the identification and quantification of cell subpopulations, monitoring of disease and its treatment and studying dynamic cellular processes such as cell differentiation5. Each sample might be either a control or a responder and the information of the respective group is displayed in the level 3 of the multi-set structure. In some cases, samples are paired, which means that the same person is followed over time and analysed before and during an immune response. In this case the index i is not unique (3a); while for unpaired samples, each set is indicated by a different i (3b).Figure 6. To our knowledge, none of these methods quantitatively accounts for the multi-set structure of MFC data either in the pre-processing or in the analysis step. This may lead to a suboptimal overview and interpretability of the cell variability across all samples, instead reflecting non-biologically relevant inter-group or inter-sample variability. The problem especially occurs when different number of cells are measured between samples and/or baseline shifts are present between the samples due to e.g. technical variation.Several multivariate data analysis methods have been developed to quantitatively explore the cell composition of MFC samples7 and FlowSOM8 concatenate the MFC sample measurements in a single big matrix, without retaining the information about the different sets, prior to the analysis. Both methods then comprise mean-center and scaling based on such concatenated samples. If the various samples strongly differ in the number of cells measured, the calculated mean and standard deviation would be affected and, as result, the model will mainly describe the cellular marker variability of the samples with the most cells. Instead, a model where each sample equally contributes to, is desirable to avoid misleading results. In other algorithms, such as SPADE9 and viSNE10, problems related to difference in number of cells measured may be avoided by downsampling the data to a fixed and equal number of cells per sample. Downsampling causes loss of valuable data, which may result in an unreliable model estimation and may result in losing important cells that might be essential in several high-impact application of MFC, e.g. minimal residual disease detection.Multivariate analysis methods such as Citrus11 and ECLIPSE12. Both DAMACY and ECLIPSE consider the multi-set structure during the pre-processing steps and when building the models to compare samples. However, until now we have not yet presented a detailed study to show how the multi-set pre-processing is in need when analysing MFC data, to avoid suboptimal immunological interpretation or even misinterpretation of the results.The multi-set-structure has already been an integral part of the methods DAMACYIn this paper we comprehensively review the multi-set structure and present the multiset pre-processing in order to create awareness for additional pre-processing options and how these options may be beneficial for the analysis and interpretation. The multi-set pre-processing corrects for the difference in number of cells measured per sample, by averaging the mean and the (square root) of the variance between samples and thus providing a more reliable representation of variability in the original marker expressions within the separate samples. We show how default pre-processing may be harmful by simulating a sample with fifty times more cells than other samples, by experimental quantification with an LPS benchmark study and with an obese versus lean study. The LPS study is based on neutrophils which has a high measurement-to-measurement variability and this unwanted technical variation affects all multivariate methods tested, namely PCA, SOM, t-SNE and Citrus. For this reason we show and advise to pre-process the data per sample, to correct for unwanted technical measurement-to-measurement variability and to highlight the studied underlying immunological effect. Pre-processing based on all cells of all control samples may enhance the deviation of immune response-specific marker variability from a control reference marker expression and lead to a better discrimination and diagnosis accuracy, as we show with DAMACY in an obese and lean study. Also algorithms such as viSNE and SPADE, which are distance based methods and do not explicitly use the mean, may benefit from the multi-set pre-processing. It may alter the relative differences between cells in different samples, thereby removing unwanted variation and essentially improving the information content of the models.Peripheral blood was extracted from subjects in both the LPS challenge study and obese versus lean study, all of whom gave their written informed consent before participating. All data were obtained using standardised protocols. The LPS challenge study and sample collection were approved by the medical ethics committee of Radboud University Medical Center (Radboudumc) Nijmegen, The Netherlands. The study protocol of the obese versus lean study was approved by the Medical Ethical Committee Jessa hospital, Hasselt, and Hasselt University, Belgium. Both studies were performed in accordance with the Declaration of Helsinki .www.clinicaltrials.gov), in which male healthy donors were challenged with intravenous administration of Lipopolysaccharide (LPS). The LPS dataset comprises gated neutrophils from 16 samples: 8 ‘control’ (or reference) samples who did not receive LPS, and 8 different ‘response’ samples who were administered with LPS. For the responders, whole blood was collected 180 minutes post LPS administration. Seven surface markers were measured on the neutrophils in samples of both classes: CD62L, CD11b, CD11c, CD64, CD32, CD69, and CD16. The samples size range between 6 thousand to 40 thousands cells. Further details regarding the Flow Cytometry experiments that generated the data can be found in a previous publication13.MFC data of the “Lipopolysaccharide study” were part of an endotoxin trial  in the range between 20.83 and 25.62 and 16 obese samples with BMI between 30.47 and 49.2715 to remove variability in the data that is unrelated to the problem under study, while retaining the experimentally relevant information. In Flow Cytometry, such irrelevant variability might result from instrumental artefacts due to misalignment of the laser source, baseline drift, laser power variability, or uninformative noise coming from low intensity signals. Time delays between sample collection and measurements can also bring variability that is not related to the problem under study. Especially granulocytes should either be measured fresh and as fast as possible, or measured after using advanced freezing techniques to minimize such nonspecific activation17.Data pre-processing is a crucial aspect of multivariate data analysisigth sample, with g = 0 used for the control group and g ≥ 1 for responder groups and J corresponds to the markers measured, 1…j…J. The first step of the pre-processing consists of transforming X with log  centering subtracts the column mean (or median) from every element in the column. Median centering is required when the number of cells measured is very low. This removes marker expression (or offsets) consistently present across all the cells and creates a common point of reference to quantify variability in cellular marker expression between the cells. Centering is typically applied in combination with scaling, which consists in dividing each variable by a scaling factor. Scaling equalizes the variability of each cellular marker across the cells. This allows the variability in every surface marker to contribute equally to a multivariate model of the data, regardless of the intensity of the used fluorophore or the absolute variability in abundance of every surface marker.Arcsinh and log transformation can be considered as a ‘pseudo scaling’ transformation that ameliorates magnitude differences in the fluorescence emissions 7 and flowSOM8 analyses, all the data files are bundle together in a big data matrix and, after transformation, centering is performed according to Equation m is the mean of the arcsinh (or log-) transformed cellular marker expression calculated across all the samples included in the matrix Xlog; N is the number of total cells measured; 1 is a column vector of ones with length N. The mean-centered matrix Xm is then scaled according to Equation S diagonal matrix holding the standard deviation sT; Xsc the resulting auto-scaled matrix.In the pre-processing as applied in CitrusThis ‘standard’ pre-processing ignores the multi-set structure Fig.  of multi21 are therefore available which may lead to different views and information: to centre/scale using the mean/standard deviation calculated on all the samples, on the control samples or per sample.Our ‘multi-set’ pre-processing specifically accommodates the multi-set structure of Flow Cytometry data and tackles various sub-aspects: cells measured may belong to different samples, where these can be drawn from different experimental cohorts (such as control and responder) and different numbers of cells can be measured per sample. Several strategies for centering and scalingThe different pre-processing strategies were tested on simulated data which consisted of normal distributions, representing different cell populations .In Fig. Xm based on all the samples m is calculated by dividing the sample-specific mean i of group g, over the total number of samples Ig. Centering over the whole dataset , as displayed in Fig. Centering over the whole dataset employs the data from both the responder and control groups. This centering operation requires a correction for the possibly different number of cells measured per sample, to avoid the sample with most cells dominating the calculated mean. Thus, creating a common reference point between the samples and not in the sample with the most cells, see for more infomration supplementary Figure Centering can be performed based on group (control)-level, which means the control group is used as point of reference. Centering based on controls is given by equation:i0–th control sample m0, where I0 represents the number of control samples. The resulting mresponse = 0,2) from the control reference , as shown in Fig. Also in this case, a correction for different numbers of cells per sample is performed by using the mean of each ples Eq.  will remi.e. per MFC sample, is calculated as follows:ig–th sample, ig–th sample; 1 is a column vector of ones with length per sample  , as shown in Fig. e.g. changes and/or misalignment of laser intensity, sample handling etc. that are unrelated to the biomedical information within a MFC dataset.Centering per multi-cell set, Additionally to centering, scaling is performed. As for centering, the same alternatives are available also for the scaling step. Below, we discuss and show the formulas of the different scaling options performed on the sample mean centered data matrix Xmig Eq. . Each foS is a diagonal matrix of size J × J containing the standard deviation (or median absolute deviation) of each surface marker sT, calculated on all cells of all samples. Median absolute deviation is required when the number of cells measured is very low. Scaling based on the complete data will keep the same difference in shape between control  and responder  population, but the variables are now equally important .Scaling over the whole dataset is performed with the flowing equation:S0 of dimensions (J × J) with the diagonal element containing the standard deviation scontrol = 1, 1) and emphasize the shape of the challenged population .Scaling based on the control group is given in Eq. J × J holding the standard deviation of the mean-centered surface markers of the ig–th sample. Scaling per sample makes cell population of both groups homogenous .Scaling per sample is calculated as follows:The plots in Fig. 23. A multi-set extension of PCA exists to accommodate multi-set structure present in the data and it is known as Simultaneous Component Analysis (SCA)24. When applying the SCA decomposition, the pre-processed matrix Xsc is normalized such that each sample contributes with the same amount of information. The normalization is done by blockscaling25, which consists of dividing each sample by the square root of the corresponding number of cells, see Eq. Principal Component Analysis (PCA) is a widely used method to visualize multidimensional data, including Flow Cytometry data, while retaining most of the variability expressed in the originally measured variablesT of size k = 1,…, K indicates the dimensionality of the new low-dimensional space, P* of size K × J containing the loadings and E being the residuals. The loadings represent the contribution of each cellular marker in building the low dimensional space and in describing the cell variability contained in the scores matrix. Blockscaling is essential to estimate loadings which are not highly influenced by the cellular maker variability of the samples with the most samples. The relations between cell variability and the related expression of markers in the SCA space may be combined in a single biplot26.SCA then decomposed the resulting matrix as follows:27. Previous publications have shown that upon acute inflammation, two neutrophil subsets arise in the peripheral blood, which differentially express FcγRIII (CD16) and L-selectin (CD62L). Neutrophils under homeostasis express both CD16 and CD62L, while both arising subsets are CD16+CD62L- and CD16-CD62L+, with different morphological and functional features13. The challenge aimed to characterize the expression of activation markers on the neutrophil subsets released in peripheral blood during the LPS-induced response, compared to a homeostatic reference present in the control group. Considerable shifts of the fluorescence signals are present between the samples in both groups for nearly all markers, as shown in Figure 28. Various studies have shown how different sample treatments may affect marker expression on neutrophils17. Additionally, time delay between sample collections, reagent staining and actual measurements may vary across the samples and this introduces an uncontrolled between-sample variation which can hinder an accurate data analysis and influence interpretation of the findings when the data is not accurately corrected for this.The Lipopolysaccharide (LPS) challenge study entails intravenous administration of systemic endotoxin to eight volunteers. This experiment is used to mimic acute inflammation in humans which is most prominent 180 minutes after LPS administrationAfter log-transformation, we pre-processed the LPS data with both standard pre-processing, according to Equations 29 with same parameter settings as in flowSOM8: grid size of 10 ×10, Euclidean distance to find nearest neighbor, and training length of 10 epochs. The resulting clustering performed by SOM trained on the standard pre-processed and multi-set preprocessed data can be visualized in the Supplementary Figure 9A-B, respectively. Nodes are displayed with pie chart representing the average intensities of the markers for all the cells assigned to the specific node. Based on this representation, we can observed that both trees seem to be dominated by the majority of normal mature neutrophils having CD16+CD62L+ expression. Previous work has shown how the vast number of normal-like cells presents in the responder individuals can hamper the identification of response-specific cell subsets12. However, when coloring the pie chart according to the number of cells from each sample in the nodes, a major difference is revealed between the clustering results for the diversely pre-processed data  map using the toolbox implemented in Matlabata Fig. . In factSuch between-samples variability is removed by the multi-set pre-processing and as a result the cells from the 16 samples are distributed throughout the tree Fig. . The mul10 analysis. The analysis was done using the Matlab GUI cyt, downloaded from the website https://www.c2b2.columbia.edu/danapeerlab. For each sample we randomly selected a subset of 2000 cells, so that the total number of cells analyzed was 32.000. The results of viSNE analysis are shown in the Supplementary Material II . The model identified as optimal for the analysis on the standard pre-processed data  provided the highest accuracy achievable, corresponding to 25% of misclassified samples. When Citrus analysis was trained on the multi-set pre-processed data, a perfect classification was obtained as shown in the Supplementary Material II . The phenotypes of the cell clusters associated with the four features, corresponding to a null cross-validation error rate, are shown in Fig S17B. The first three clusters are more abundant in the responder group and they may be associated to premature and mature neutrophils. The last cluster found more present in the control group compared to the responder group may be assigned to normal mature neutrophils having CD16+CD62L+ expression.In order to experimental quantify whether the pre-processing could affect the results of a discrimination model between the control and responder groups, we performed a Citrus11 to explore the different pre-processing options mentioned in the method section. DAMACY11 first describes the cellular variability in N-dimensional histograms based on Simultaneous Component Analysis (SCA) of the pre-processed data using the multi-set structure. Subsequently it uses Orthogonal Partial Least Squares Discriminant Analysis (OPLS-DA)30 on the histograms to create a regression map. The regression map shows which cell (sub)populations are more or less present in a clinical phenotype compared to control samples. The whole algorithm, including the pre-processing step, was validated using leave one out validation in the LPS results and seven-fold cross-validation with fifty iteration in the obese dataset. Both datasets were permutated 1000 times for permutation testing. This leads to well statically validated prediction accuracies that may be compared with each other and compared to maximum accuracy achieved on the permutated data to see if the models are significant and not by chance have a high accuracy. Only DAMACY was used, because the multi-set preprocessing options are not yet incorporated in other methods and thus unable to correctly validate the results. The only multiset pre-processing option possible in other methods such as Citrus (previous paragraph) is individual centering and scaling since those are not affected by the cross validation.In this section we used Discriminant Analysis of Multi-Aspect Cytometry (DAMACY)In addition to providing enhanced insight, multi-set pre-processing may also benefit the discriminative power between control and LPS responders. Supplementary table Different pre-processing strategies were tested with DAMACY on the obese vs lean dataset, using seven-fold cross validation with 50 repetitions. The prediction performance is summarized in supplementary table 15.Figure e.g. case vs control) and may be characterized by different number of cells measured. Based on classification accuracy, biomedical insight or both.The integration of the multi-set structure in the pre-processing and multivariate analysis steps of MFC data is essential to overcome the influences of non-informative aspects in the data. Here we showed that our multi-set pre-processing may be essential to extract the full biomedical information from MFC data consisting of samples (sets) which may belong to different groups , as standardly done in most analysis methods such as Citrus and flowSOM, the calculated mean shifts towards the sample(s) with most cells (see simulated data results). The effect of the shift in multivariate analysis is shown by Simultaneous Component Analysis (SCA), which was chosen as it offers a representation of cells population together with marker (co)-expression in a single plot. Blockscaling Eq.  is an inFinding biological variability in the samples which is relevant to the studied response/disease is one of major challenge in MFC data analysis. Our multi-set pre-processing strategy enables to systematically remove variability unrelated to the studied problem while retaining the informative biological information. Non-relevant biological or technical variation could provoke shifts of fluorescence signals of the same cell population among the samples, as in the case of the LPS dataset. When applying standard pre-processing to the LPS data, this variability is not removed and remained quite dominant in the multivariate analyses performed. The viSNE and SOM models thus intrinsically described mainly this unwanted variability leading to trivial conclusions. In fact, the obtained results enabled us to identify putative phenotypically different cells subsets which are not realistically representing the homogenous immune response across all the samples. We showed how this sample-to-sample variability might also lead to a suboptimal discrimination accuracy in Citrus and DAMACY analysis. By using multi-set pre-processing was beneficial for viSNE and SOM analyses of which results were better representing the phenotypical variation present in the data. It also helped in outperforming the predictive ability of the Citrus and DAMACY discriminant models.15. However, an external test set is still required to test the optimal predictive model or should be at least compared with literature and visual inspection is needed for quality control and to interpret the model31. The current pre-processing setup entails only a limited number of possible permutations, but it needs to be integrated with the other pre-processing steps for MFC, such as transformation and compensation.Diagnostic ability is another challenge for which MFC data analysis methods should be used. The multi-set structure allows for pre-processing based on only the control samples to enhance the differences between case/responder and control, which may improve the discrimination and diagnostic ability. In the case of obese versus lean data standard pre-processing already performed very well with 76.6% accuracy, when compared to 76.8% accuracy in the optimal model with control centering and individual scaling. However, the optimal model enhances the response-specific variability in the case samples, and is therefore better able to describe the relationship between the markers measured. This will allow better and more robust interpretation of the data, as shown in the comparison between Fig. Nowadays most of the widespread applications of MFC involve the measurements of cells from several patients. The same markers are measured across all the patients and this enables the arrangement of MFC data in a multi-set structure. Here we presented how the integration of multi-set structure in the pre-processing and analysis of MFC data led to better interpretation of the analysis methods the results and corrected for challenges occurring in MFC. In fact, the multi-set pre-processing proposed corrects for difference in number of cells measured across all the patients. This difference should be always taken in consideration because it may be detrimental for the interpretation of the findings of the analysis method used, as demonstrated with the simulations proposed.e.g. discrimination between control and diseased/case group. Control based centering and/or scaling may enhance the effect of the diseased/case group in the model. Individual centering and or scaling may be useful when the model shows an unequal distribution of individual samples, e.g. cells from an individual are only in one part of the t-SNE map or flowSOM tree meaning that the model only describes individual variation instead of between group variation.In addition, the versatility of the pre-processing algorithm allows several different pre-processing strategies. These include solutions to remove unwanted non-biological/technical variation between the samples and strategies to best accommodate the study research questions, The multi-set pre-processing  may be implemented in any multivariate data analysis methods. This may enable outperforming of prediction accuracy and lead to more robust results.Supporting information.


P-value = 5.51e−07). The autoencoder framework showed superior performance compared with PCA, NMF, t-SNE, and the univariable Cox-PH model based on the C-index, log-rank P-value, and Brier score. The robustness of the classification model was successfully verified in three independent validation sets. There were 1271 differentially expressed genes, 10 differentially expressed miRNAs, and 12 hypermethylated genes between the survival risk groups. Among these, miR-133b and its target genes  were used to construct a network. These genes were significantly enriched in ECM–receptor interaction, focal adhesion, PI3K–Akt signaling pathway, and glucose metabolism-related pathways. The risk subgroups obtained through a multiomics data integration pipeline using the DL algorithm had good robustness. miR-133b and its target genes could be potential diagnostic markers. The results would assist in elucidating the possible pathogenesis of COAD.The present study proposed a deep learning (DL) algorithm to predict survival in patients with colon adenocarcinoma (COAD) based on multiomics integration. The survival-sensitive model was constructed using an autoencoder for DL implementation based on The Cancer Genome Atlas (TCGA) data of patients with COAD. The autoencoder framework was compared with PCA, NMF, t-SNE, and univariable Cox-PH model for identifying survival-related features. The prognostic robustness of the inferred survival risk groups was validated using three independent confirmation cohorts. Differential expression analysis, Pearson’s correlation analysis, construction of miRNA–target gene network, and function enrichment analysis were performed. Two risk groups with significant survival differences were identified in TCGA set using the autoencoder-based model (log-rank  Colorectal cancer (CRC) is the fourth most prevalent cancer and the second primary cause of cancer-related death in the United States. . Due to Multiomics data integration provides more information on tumorigenesis and development than 1D omics data and delivers additional benefits for precision medicine . Deep leIn the present study, we applied a DL computational framework based on multiomics data  from The Cancer Genome Atlas (TCGA) and built a prognostic model based on new features transformed by an autoencoder to stratify patients with COAD. The stratification identified two survival subgroups with significantly different survival rates, which were further successfully validated in three independent datasets. Functional analysis of the two survival subgroups uncovered critical miRNAs, target genes, and signaling pathways in the biology of COAD. The robust classification of patients with COAD using this model may be beneficial for prognosis prediction and the development of precision medicine.impute package (https://www.bioconductor.org/packages/release/bioc/html/impute.html) of R. Eventually, input features with zero values across all samples were removed. E-GEOD-17538 with 232 samples , E-GEOD-39582 with 558 samples , and E-GEOD-28722 with 125 samples  were downloaded from the ArrayExpress database (https://www.ebi.ac.uk/arrayexpress/). The three datasets were used as validation sets. Clinical characteristics of TCGA set and the three validation sets are summarized in We obtained paired RNA sequencing (RNA-seq) data (RNA-seq Illumina HiSeq platform), miRNA sequencing (miRNA-seq) data (miRNA-seq IlluminaHiseq platform), and DNA methylation data (Methylation Illumina 450k platform) of 288 COAD samples with corresponding clinical information from TCGA database as the training set. For preprocessing of raw data, we first removed the probes or genes with values missing in more than 50% of samples. Methylation data were annotated using the RIlluminaHumanMethylation 450kanno.ilmn12.hg19 package . Beta vatanh as the activation function for each layer. To train the autoencoder, we employed a gradient descent algorithm with 10 epochs and 50% dropout. With two hidden layers , the autoencoder was implemented using the Python Keras library (https://github.com/fchollet/keras). By using the bottleneck layer of the autoencoder model, 275 transformed features were produced from the multiomics data.We used multiomics data from TCGA set as the input for the autoencoder, a DL framework. As shown in It should be noted that: (1) this self-coding has no output noise setting, and the loss function is described below, and (2) this self-encoder is fully connected and self-coding. The detailed principle of self-coding is as follows:n-dimensional features are input: x = , and the purpose of self-coding is to reshape X′ by output X′ through a continuous hidden layer.Suppose Wi is the weight matrix of size p × d.Given a layer, we used tanh as the activation function to connect the input X of each layer and the output y of each layer, which are shown as follows: In the k layer, X′ is defined as: kf-1 and kf(x). To train self-coding, we aimed at different weight vectors Wi._ We selected logloss as the objective function, which measures the error between input x and output x′:Wi plus L1 regularized penalty term wα, and in the active node, aα. Therefore, the objective function is defined as follows:To prevent overfitting, we used the weight vector P-value <0.05 was considered significant. We clustered the samples of TCGA set using the K-means clustering algorithm in the nbclust package (https://cran.r-project.org/web/packages/NbClust/index.html) of R. Silhouette index [k-mer was 2-6. In fact, we selected the Silhouette index and Calinski–Harabasz criterion for the evaluation index. Finally, NbClust can obtain the highest clustering number according to the calculation results. The detailed values of the Silhouette index and Calinski–Harabasz criterion are shown in Supplementary Table S5. Following obtaining labels from K-means clustering, survival of different risk subgroups was compared using Kaplan–Meier survival curves and log-rank t-test. Log-rank P-value [For each transformed feature, a univariable Cox proportional hazards (Cox-PH) model was constructed. The feature with a log-rank te index , and thete index  was used P-value , C-index P-value , and Bri P-value  were calt-distributed stochastic neighbor embedding (t-SNE) [K-means clustering of TCGA samples.The DL framework was compared with other dimensionality reduction techniques, including principal component analysis (PCA) , nonnega (t-SNE)  for perfK-means procedure -like procedure described in a previous study , we randK-means clustering analysis, we performed analysis of variance (ANOVA) [N features significantly associated with the labels of risk groups were identified based on ANOVA F-values. Default N values were set to 40 for RNA, 30 for methylation, and 30 for miRNA. The log-rank-P of this parameter was significant in both the training and validation sets, and the c-index was high.Following  (ANOVA)  with eacP-value, C-index, and Brier score. The penalize SVM package of R was employed to carry out a grid search for the optimal combination of hyperparameters of the SVM model using five-fold CV and to develop SVM models.The top 40 mRNAs and 30 methylation or 30 miRNA features identified by ANOVA were utilized to construct an SVM classifier, respectively, for predicting TCGA test data. The prediction accuracy of the SVM classification model was assessed using the log-rank Three independent confirmation sets , all of which contained RNA-seq data, were used for validation of the two survival risk subgroups. First, we selected common mRNA features between each validation set and TCGA set, respectively, which were further subjected to median scale normalization and robust scale normalization. After the two scaling steps, the corresponding top 40 mRNA features selected by ANOVA were identified to construct an SVM classifier.DESeq2 package [https://bioconductor.org/packages/release/bioc/html/DESeq2.html) of R was used to identify differentially expressed miRNAs and genes with |log2FC| >1 and FDR <0.05 as the selection cutoff. A moderate t-test using the limma package (https://bioconductor.org/packages/release/bioc/html/limma.html) in R was used to determine significant differences in methylation with |beta difference| >0.1 and FDR < 0.05 as the strict threshold.Using TCGA data, we performed differential expression analysis in each individual omics layer between two survival risk groups identified by the autoencoder. The  package  . Among the common target genes between the two databases, the differentially expressed genes (DEGs) between the two survival risk groups were selected to construct an miRNA–target gene network. Kyoto encyclopedia of genes and genomes (KEGG) pathway enrichment analysis of the identified DEGs was performed using the KOBAS tool. Pathways satisfying an FDR <0.05 were considered significant.In order to investigate whether DNA methylation affects gene expression, associations between methylation level and gene expression were evaluated by performing Pearson’s correlation analysis. A Pearson correlation coefficient < -0.5 and ng miRDB  . Each of the 275 transformed features (detailed information in Supplementary Table S6) underwent a univariable Cox-PH regression model. The 13 features (detailed information in Supplementary Table S7) significantly associated with survival  were then subjected to K-means clustering analysis. The optimal number of clusters was two. TCGA samples were classified into two survival risk groups (G1 and G2). As shown in P-value = 5.51e−7). The C-index and Brier scores were 0.766 and 0.172, respectively. These results suggested that this classification identified two different prognostic subtypes in patients with COAD.The results shown here are in part based on data generated by TCGA Research Network: K-means clustering. As depicted in P-value = 0.191, C-index = 0.633, Brier score = 6.94e−3), NMF , t-SNE , and Cox-PH  model failed to yield a significant log-rank P-value <0.05. We found that only the autoencoder could determine significant survival subgroups in patients with COAD.We compared the autoencoder to other alternative methods, including PCA, NMF, t-SNE, and univariable Cox-PH model, for prognostic classification of COAD samples. Using each of the first three approaches, we obtained 275 transformed features, which is the same as the number of transformed features from the bottleneck layer of the autoencoder, and subjected these to a univariable Cox-PH model. In the Cox-PH approach, we performed univariable Cox-PH analysis on each input feature in the three omics data types. We ranked the features based on C-index values and selected the top 13 features, followed by P-value (1.40e−4) for the three-omics training set on average. Similar results were observed for the three-omics test data . With regard to the test of each omics dataset, this model also generated significant but marginally inferior results . The numbers of common mRNAs shared by each validation set and TCGA set were 12959, 12959, and 12478, respectively. We selected the common top 40 features based on ANOVA F-value followed by SVM classification.P-value of 8.22e−4 for disease-free survival (DFS) time  time A. The cl = 0.197 D. Additi = 0.133 E.2FC| > 1 and FDR < 0.05, 2FC| > 1 and FDR < 0.05), and 12 hypermethylated genes (FDR < 0.05 and |delta methylation| > 0.1) were found  and regenerating family member 4 (REG4) were significantly down-regulated by promoter hypermethylation , protein tyrosine phosphatase receptor type Z1 (PTPR Z1), RUNX1 partner transcriptional co-repressor 1 (RUNX1T1), EPH receptor 7 (EPHA7), glycoprotein M6A (GPM6A), BicC family RNA binding protein 1 (BICC1), and ADAM metallopeptidase with thrombospondin type 1 motif 5 (ADAMTS5) were identified, and a miRNA target gene network was built  were differentially expressed between the two risk groups. All the eight genes are associated with cell proliferation, invasion, and migration enhancement and poor prognosis in various cancers [CXCR4 [GNB4, a signal transduction molecule involved in the PI3K-AKT pathway, is hypermethylated and down-regulated in both CRC cell lines and colon cancers [PTPRZ1 expression is elevated in CRC, implicating its involvement in CRC development [RUNXIT1 may play a suppressive role in CRC progression [EphA7 by hypermethylation occurs in CRC [GPM6A, suggesting that GPM6A is a novel target for epigenetic regulation during prenatal stress [BICC is a genetic determinant of osteoblastogenesis and bone mineral density [ADAMTS5 is overexpressed in CRC, promoting CRC metastasis and cancer cell invasion [ADAMTS5 is a potent biomarker for lymphatic invasion and lymph node metastasis in CRC [At present, the survival and prognosis of patients with COAD are poor. Accurate stratification of patients with COAD indicative of prognosis would help to select the optimal therapy for each patient. In the present study, RNA-Seq, miRNA-Seq, and DNA methylation data of the same patients were downloaded from TCGA to develop a survival prediction model using the DL framework. The present model was based only on TCGA database and now the GEO database. Except for TCGA database, no other databases such as GEO database contained cancer samples with ‘RNA-seq’, ‘miRNA’, and ‘methylation’ data; thus, the GEO database could not be used to verify the dimension reduction of the autoencoder model. Two risk subgroups with robust survival differences were inferred by the autoencoder framework based on multiomics data. According to the C-index value, log-rank e II CRC ,23. Furt cancers . miR-133 cancers ,26. miR-s [CXCR4 . GNB4, a cancers . PTPRZ1 elopment . RUNX1T1gression . The invgression . The dows in CRC . GPM6A il stress . The genl stress . BICC is density . ADAMTS5invasion . High exs in CRC . These r3K-Akt signaling pathway plays an essential role in the progression of colon cancer and is a promising target for cancer treatment [Functional analysis of the up-regulated DEGs revealed significant enrichment of various signaling pathways, such as ECM–receptor interaction, focal adhesion, and the PI3K-Akt signaling pathway. The down-regulated genes were significantly involved in several signaling pathways associated with glucose metabolism. A rich body of evidence has shown that the PIreatment ,40. Glucreatment . Howeverin vitro or in vivo models. We hope that the results of this study will be beneficial to future research.Here, we performed an extensive study based on published data and bioinformatic analysis. The results of the present study should be further validated using GNB4, PTPRZ1, RUNX1T1, EPHA7, GPM6A, BICC1, and ADAMTS5 may be important molecular targets for COAD.The present study robustly distinguishes survival subpopulations of patients with COAD using DL-based multiomics integration. This classification is of direct clinical importance and contributes to improved outcomes in patients with COAD. miR-133b, Click here for additional data file.


Three main conclusions are drawn from the comparisons: both methods are platform independent, they select largely overlapping gene rules associated with actual underlying tumor biology and, for large training data sets, they behave interchangeably performance-wise. While SSPs like AIMS and kTSP offer new possibilities to move gene expression signatures/predictors closer to a clinical context, they are still importantly limited by the difficultness of the classification problem at hand.The development of multigene classifiers for cancer prognosis, treatment prediction, molecular subtypes or clinicopathological groups has been a cornerstone in transcriptomic analyses of human malignancies for nearly two decades. However, many reported classifiers are critically limited by different preprocessing needs like normalization and data centering. In response, a new breed of classifiers, single sample predictors (SSPs), has emerged. SSPs classify samples in an N-of-1 fashion, relying on, e.g. gene rules comparing expression values within a sample. To date, several methods have been reported, but there is a lack of head-to-head performance comparison for typical cancer classification problems, representing an unmet methodological need in cancer bioinformatics. To resolve this need, we performed an evaluation of two SSPs [k-top-scoring pair classifier (kTSP) and absolute intrinsic molecular subtyping (AIMS)] for two case examples of different magnitude of difficulty in non-small cell lung cancer: gene expression–based classification of (i) tumor histology and (ii) molecular subtype. Through the analysis of ~2000 lung cancer samples for each case example ( Gene expression profiling by high-throughput methods like microarrays and, more recently, RNA sequencing has been used for nearly two decades in cancer research. Two of the most central aims have been to derive the following: (i) gene expression–based predictors of patient risk of disease relapse (prognostic predictors) or benefit of a specific treatment (treatment response predictors) or (ii) predictors for novel molecular disease subtypes or for established histological subgroups of a disease. A general hope has been that molecular-driven disease stratification could positively affect patient treatment and outcome for cancer patients, ultimately providing a more personalized cancer care. The most well-known example of gene expression–driven disease stratification is likely the now, in clinical routine, established molecular subtypes of breast cancer first reported in early 2000  that strOver the years, a plethora of different risk predictors as well as molecular subtype predictors have been reported for nearly all human malignancies. However, only a small number has actually been independently validated, and even fewer have made it to a stage of clinical use. Two of several reasons for this lack of clinical translation or reproducibility are likely the predictor development and implementation process itself, as well as lack of sufficiently large and unbiased training cohorts that properly mirror the relevant population-based clinicopathological demographics. In many instances, predictors of risk and molecular subtype proposed for classifying independent cases have been developed in a way that requires gene centering of input data to assure comparable relative gene expression levels across samples. Typical early steps in microarray-based gene expression data processing have included background correction, normalization (different methods for different platforms) and probe filtering (e.g. to remove cross-hybridizing probes). For RNA sequencing data, preprocessing includes alignment, quality filtering and summarization into transcript counts (e.g. Fragments per kilo base per million mapped reads (FPKM)). For both data types, gene/transcript centering across samples is often performed as a 2nd step, which creates gene expression estimates relative to the cohort processed, and no longer absolute in its nature. A classical example of this is the nearest centroid classification (NCC) method. In NCC, each class is characterized by its vector of means (centroid) derived from a training set, to which a similarity score is computed for an unknown sample to identify the class with the best (nearest) match. Many other examples exist with the common theme that classification of independent samples becomes related to both the training set from which the predictor was derived and to the composition of the cohort from which the new sample originates .Predictors based on gene rules (gene pairs) have been proposed as a possibility to circumvent the sensitivity of predictors dependent on gene centering and also to be more platform independent , 4, 6, 7In the present study, we aimed to evaluate the classification performance, robustness, platform independence and prediction agreement of two SSPs  across the five test data sets . CorrespP = 0.3 (accuracy) or P = 0.4 , Kruskal–Wallis test; The classification accuracy of the corresponding seven AIMS classifiers from the individual training data sets ranged from 0.11 to 0.99  favoring kTSP, but not in balanced accuracy , . The merged kTSP classifier ranged in classification accuracy from 0.85 to 0.97  and balanced accuracy from 0.84 to 0.97  across the five test data sets . The level of agreement (% of samples predicted similarly) was generally very high (>85%) for classifiers derived from individual training data sets, except in the Bhattacharjee et al.  test datn = 3 data sets; Affymetrix, n = 3 data sets; RNAseq, n = 1 data set  and balanced accuracy   for each trained classifier were merged (averaged) in a platform-wise manner based on the training data origin, for each classifier method separately. Distribution of the seven training data sets into three platform origins was as follows: Illumina, data set . Both kTP = 0.6) and balanced accuracy   across the six test data sets . CorrespP = 0.3) or balanced accuracy   across test data sets . Balance; et al.  data setP = 0.4) or balanced accuracy   or balanced accuracy (P = 0.7) (Mann–Whitney–Wilcoxon test). The merged kTSP classifier ranged in classification accuracy from 0.44 to 0.91  and from 0.55 to 0.93 for balanced accuracy  across the six test data sets  or balanced accuracy metrics   of 12 (83%) selected AIMS genes overlapped with the kTSP genes . A functional enrichment analysis (http://pantherdb.org) for the 10 genes identified cornification as an enriched main biological process with keratinization as subcategory  within a gene pair for the selected 7DS models. For each model and gene pair (e.g. gene A > gene B), we formed the ratio of gene A/gene B using the raw unprocessed gene expression estimates for each sample and plotted these values for each sample in the 7DS training cohort and in an independent test set .In the molecular subtype case, 243 unique genes were found across all AIMS models, whereas 140 unique genes were used to build all rules in the kTSP classifiers . The numRU cases   gene expression–based SSPs. Based on two lung cancer classification scenarios, selected to represent different levels of classification difficulty, we assess two SSP methods (kTSP and AIMS) for performance, robustness, platform independence and agreement.The 1st classification scenario, a prediction of lung cancer AC or SqCC histology, represents a hypothetically easy classification problem, due to classes with separate and distinctive biology . For this case example, we observed high accuracy (median ~0.87–0.92) in predictions across five test data sets for both SSP methods versus the endpoint variable (tumor type by histopathology) for nearly all developed models . These aKRT5) used in clinical practice today for IHC-driven classification of NSCLC histology. Moreover, our deepened analysis of the selected gene pairs for this case example reveals that the type of gene pairs selected displays expression ratios with an apparent bimodal distribution independent of technical platform. The latter greatly facilitates prediction. Taken together, for a gene expression classification problem of similar degree of difficulty, it appears that if a sufficiently large training set exists, the two tested methods perform very robust, are platform independent, identify actual biology and are interchangeable.While it appears feasible to derive accurate SSPs from small cohorts for this case example , we do note a greater stability (less variation) in accuracy for test data sets when combining cohorts into larger meta-sets for training for both SSP methods , despiteThe 2nd case example was chosen to be more challenging for predictor development, namely a prediction of (reported) gene expression–based molecular subtypes of AC , 11. In Despite the challenges connected to this case example, we do note median accuracies of the different developed models between 0.76–0.88, although with a more pronounced variation compared to the histological case example . Howevern = 383 of in total 506 original centroid genes). When comparing accuracy metrics obtained from models from the seven training data sets across the test data sets, we did not see any clear trend favoring the original or restricted (directed) gene approach  appear to be beneficial for robustness. Clearly, the overall performance of these predictors appears mostly limited to the difficultness of the classification problem at hand and likely less to the type of model . An intSupp_bbz008Click here for additional data file.


Non-parametric dimensionality reduction techniques, such as t-distributed stochastic neighbor embedding (t-SNE), are the most frequently used methods in the exploratory analysis of single-cell datasets. Current implementations scale poorly to massive datasets and often require downsampling or interpolative approximations, which can leave less-frequent populations undiscovered and much information unexploited.We implemented a fast t-SNE package, qSNE, which uses a quasi-Newton optimizer, allowing quadratic convergence rate and automatic perplexity (level of detail) optimizer. Our results show that these improvements make qSNE significantly faster than regular t-SNE packages and enables full analysis of large datasets, such as mass cytometry data, without downsampling.https://bitbucket.org/anthakki/qsne/.Source code and documentation are openly available at Bioinformatics online. Single-cell measurement technologies have become routinely used tools in medical research . While tNon-parametric dimensionality reduction techniques, such as t-distributed stochastic neighbor embedding (t-SNE)  and unifThe main issue with the standard t-SNE is that the optimization process is naive (gradient descent) and slow. To counter this, downsampling has been traditionally used . Howeverhttps://bitbucket.org/anthakki/qsne/ under an open (BSD) license.To address these issues, we implemented (i) a solver that converges methodologically faster and requires no tuning of the gradient descent parameters; (ii) an automatic parameter selection process, which removes the need for manual perplexity tuning; and (iii) a quality metric, which can be used to assess whether the projected model captures the original high-dimensional data. Our improvements are complementary and can be combined with previous efforts , and theQ to approximate the high-dimensional neighbor distribution P:m is the number of samples and ijp (ijq) are the high (low)-dimensional densities of the distribution P (Q) between the samples i and j. t-SNE uses a normal distribution for P and a t-distribution for Q  finds a lower-dimensional representation of a dataset such that the distribution of local distances between the samples is maintained . More spon for Q . The difeighbors . The oriA gradient descent scheme only allows linear convergence, which can be prohibitively slow on large datasets. Quadratic methods (such as Newton’s method) permit quadratic convergence, but evaluating the Hessian matrix directly is too expensive, so we use the limited-memory Broyden–Fletcher–Goldfarb–Shanno method (L-BFGS) , which uiσ varies from 0 to ∞. This entropy curve is used to locate the bandwidth corresponding to the specified perplexity value π. However, the curve can be also exploited to identify the bandwidths where the neighborhood structure remains insensitive. This holds also in the presence of multiple local scales, as a scale only contributes to the entropy gradient at the sensitive regions. In practice, this results in a staircase-like figure (see e.g. The neighborhood entropy see e.g.  where flH(P) represents the average number of bits needed to encode a sample of the original neighbor relationship, while the Kullback–Leibler divergence between the source and destination distributions For any mapping in the t-SNE framework, the source entropy ith sample for its optimal bandwidth and q is zero for one-to-one correspondence between the source and destination distributions  and one unpublished high-grade ovarian cancer (HGSOC) dataset. The advantage of the bone marrow and MNIST datasets is that they are manually labeled and thus it is possible to quantify whether the visualization is meaningful. For the purposes of comparison, the datasets were downsampled as it is not practical to run the original t-SNE algorithm on 100 000 s of samples, especially with various parameters. Meanwhile, the HGSOC dataset demonstrates that the improvements in qSNE enable discovering novel biomedical insights from cancer patient samples. We also analyzed a Splatter generated  measurements, which were originally used to study phenotypic heterogeneity of acute myeloid leukemia (AML) patients . We only used the training set part of the dataset, featuring 60 000 labeled samples, which are 28 × 28 pixel images of 256 gray levels each . This dataset has been frequently used to benchmark machine learning methods, and was used for evaluation e.g. by Second, we use the MNIST database  and 127 874 cells in the interval (after chemotherapy) sample.We implemented a quasi-Newton optimizer, based on the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm , on the To evaluate the performance of qSNE, we used a human bone marrow mass cytometry dataset (Levine)  divergence between distributions of the pairwise distances of the points in the input and output space—evolves as a function of the iteration count. The results in The results show also that the exact rank of the Hessian matrix approximation plays a minor role in the qualitative behavior of the convergence on these data. While a larger rank generally allows faster convergence, even constant-rank approximations feature the benefit of quadratic convergence and only incur a constant overhead per iteration, suggesting that the strategy is viable for speeding up practical large-scale problems. This is of course only possible if the data are inherently locally, but not necessarily globally, low-dimensional but embedded in a higher-dimensional space, which often is the case and can be expected, for example, in gene expression datasets due to inherent correlations.Often the cost surface features multiple local optima, which may imply convergence to a different optimum for different optimization paths. To evaluate whether the obtained visualizations are useful after the short quadratic walk, we visualized the projections for the two methods. The results are illustrated in In terms of consumed CPU time qSNE is much faster than Rtsne . This allows the analyst to focus on the relevant perplexity values. On the other hand, the perplexity parameter can be fine-tuned to an optimal value around a chosen bandwidth, provided that a sufficiently narrow range with a single optimum is selected. Moreover, a variable bandwidth across the sample space can be advantageous in case the space is not uniform, but contains clusters with various bandwidths.To exemplify the operation of the automatic perplexity selection, we generated a 10-dimensional synthetic dataset with five clusters [with standard deviation (SD) of 1] each having five subclusters (with SD of 0.25) with a total of 1000 samples (with SD of 0.01). Depending on the selected perplexity range, qSNE results in one of the possible representation of the dataset: the least perplexity optimum corresponds to the setting where each sample is separated into a separate cluster; the next one reveals each of the 25 subclusters; and the third the five high-level clusters, as shown in To demonstrate that the automated perplexity tuning is useful in practice, we performed perplexity analysis on the Levine dataset. For these data, the interesting levels of detail regarding a 2-D projection correspond to (i) 3-clustering into B-cells, T-cells and in less tissue-specific cells; or (ii) into a more detailed clustering including hematopoietic stem and progenitor cells and their differentiated forms . Small and large n represent how well the local relationships (i.e. order of the nearby samples) and global relationship (i.e. order of long-distance samples), respectively, are preserved. The Jaccard index-based metric is expensive to calculate for high-dimensional data  with varying inherent dimension of the data embedded in a 10-D space  patient ascites samples before and after chemotherapy. The proteins were selected to span HGSOC cancer markers (CA-125 and HE4)  that were dominantly enriched in either of the HGSOC markers CA-125 or HE4 in the full-resolution analysis see  and wereSingle-cell measurement technologies generate massive, high-resolution datasets. However, most of the current analysis softwares are not directly able to analyze these data and resort to downsampling, which hinders fully exploiting the high-resolution nature of the data.We report a novel implementation of the non-parametric dimensionality reduction method t-SNE, called qSNE, which utilizes a quasi-Newton t-SNE optimizer. We show that for many practical problems and parameter settings qSNE allows convergence at quadratic rate, and consequently, an order of magnitude less computation. Importantly, qSNE produces comparable visualizations, despite that it might convergence to a different optimum. In addition, we present a method optimize the input distribution bandwidth, or the perplexity parameter, automatically. This enables the data analyst to focus on only specifying the desired level of detail, and letting the optimizer to deal with parameter tuning. The perplexity tuning also opens up an avenue toward analyzing heteroscedastic data in the complex input space, where no single parameter value can produce satisfactory result. Finally, we proposed a quality metric, which can be obtained as a side product of computing the mapping. This feature is particularly important because at the moment t-SNE visualizations are used without any analysis of the model fitness, which implies that important details of a dataset may remain uncaptured by the model without any sign reported to the analyst. Herein, we propose that a quality metric should be used routinely to assess immediately whether the acquired mapping well represents the original data, which cannot be evaluated using the lower dimensional mapping alone. qSNE is best suited for datasets with 100 000 s samples with 10 to 100 features, such as large mass cytometry data, but we also demonstrated its applicability on single-cell RNA-seq data with 10 000 samples and 18 726 genes.Our improvements are general enough to be combined with future improvements, such as alternative input and output models ; tree-baWe demonstrated that such improvements are critical in analyzing large datasets containing complex, infrequent features. Specifically, we demonstrated the utility by analyzing HGSOC mass cytometry data, which was not previously feasible at the attained level of detail. Our analysis revealed cluster of cells which are only identifiable at the higher level of detail, which can aid in developing efficient interventions to overcome HGSOC chemoresistance. qSNE is freely available with documentation.btaa637_Supplementary_DataClick here for additional data file.


P < 0.01; HR = 3.84; 95% CI = 1.91–7.70) in GSE25055. Similar results were obtained in GSE25065. Moreover, in the integrated data of the two datasets with 94 responders and 21 extremely resistant samples identified from RD patients, the former had significantly longer DRFS than the latter . In summary, our signature could effectively identify patients who completely respond to PAC NAC, as well as cases of extreme resistance, which can assist decision-making on the clinical therapy for these patients.For estrogen receptor (ER)-negative breast cancer patients, paclitaxel (P), doxorubicin (A) and cyclophosphamide (C) neoadjuvant chemotherapy (NAC) is the standard therapeutic regimen. Pathologic complete response (pCR) and residual disease (RD) are common surrogate measures of chemosensitivity. After NAC, most patients still have RD; of these, some partially respond to NAC, whereas others show extreme resistance and cannot benefit from NAC but only suffer complications resulting from drug toxicity. Here we developed a qualitative transcriptional signature, based on the within-sample relative expression ordering (REO) of gene pairs, to identify extremely resistant samples to PAC NAC. Using gene expression data for ER-negative breast cancer patients including 113 pCR samples and 137 RD samples from four datasets, we selected 61 gene pairs with reversal REO patterns between the two groups as the resistance signature, denoted as NR61. Samples with more than 37 signature gene pairs that had the same REO patterns within the extremely resistant group were defined as having extreme resistance; otherwise, they were considered responders. In the GSE25055 and GSE25065 dataset, the NR61 signature could correctly identify 44 (97.8%) of the 45 pCR samples and 22 (95.7%) of the 23 pCR samples as responder samples, respectively; it also identified 13 (16.9%) of 77 RD samples and 8 (21.1%) of 38 RD samples as extremely resistant samples, respectively. Survival analysis showed that the distant relapse-free survival (DRFS) time of the 14 extremely resistant cases was significantly shorter than that of the 108 responders ( Breast cancer is a common malignancy with the highest incidence and mortality among females . A standIn clinical practice, a pathologic complete response (pCR) is defined as a non-viable invasive cancer in the breast and lymph nodes after the completion of NAC, indicating a complete response to NAC and a favorable outcome . However® , the REO pattern represents whether the expression level of i is higher or lower than that of j in the sample. Additionally, REO-based signatures are also highly robust against common factors that lead to the failure of quantitative transcriptional signatures in clinical applications, such as varied proportions of tumor epithelial cells  are found to be robust against experimental batch effects and can be directly applied to samples at the individualized level . REO is al cells , amplifial cells , and paral cells . Thus, tBased on the within-sample REOs of gene pairs, In this study, we used the gene expression data of ER-negative samples reclassified by the 112-gene-pair signature to identify a qualitative transcriptional signature consisting of 61 gene pairs to predict patients with extreme resistance to PAC chemotherapy. Our signature was well-verified in two independent datasets with survival information.1) and the MD Anderson Cancer Center2 databases. In the datasets of GSE20194 and GSE20271, we only used the expression data of patients who received paclitaxel followed by fluorouracil (F), doxorubicin [or epirubicin (E)], and cyclophosphamide. In the GSE41998 and MDA133 datasets, the treatment regimens for these patients were PFAC and PAC, respectively.We collected four expression datasets  including 250 IHC-determined ER-negative breast cancer patients in total, who accepted PAC NAC, from the Gene Expression Omnibus (GEOTwo other independent expression datasets (GSE25055 and GSE25065) were used to evaluate whether there was a difference in survival between the responsive and the resistant groups. The treatment regimen for patients in GSE25055 was PAC or PA and the treatment regimen for patients in GSE25065 was PA.Although PAC NAC is a very common chemotherapeutic regimen, doctors design individual drug delivery schemes for each patient, depending on their condition. Some patients received 6 months of NAC including PFAC , whereas others received sequential NAC starting with 4 cycles of AC administered every 3 weeks, followed by paclitaxel weekly for 12 weeks . In this study, we only considered the drug type and not the dose of each drug or the duration of chemotherapy. The clinical characteristics for each dataset are summarized in For the Affymetrix array data, the raw intensity files (.cel), downloaded from the GEO database were processed using the Robust Multichip Average algorithm (RMA) for background adjustment without quantile normalization. The probe identity documents (ID) were mapped to the Entrez gene ID according to the corresponding platform annotation files. If a probe did not map to a gene or was mapped to multiple genes, the data for this probe were deleted. If multiple probes mapped to the same gene, the arithmetic mean of the expression values for the multiple probes was taken as the final expression value for this gene.We used the 112-gene-pair signature developed by i, j) has only two possibilities, Gi > Gj or iG < Gj, where Gi and Gj denote the expression values. If the number of RD samples with a certain REO pattern (Gi > Gj or Gi < Gj) is significantly more than expected by chance, we define this gene pair as a stable gene pair of RD samples; stable gene pairs of pCR samples are defined in a similar manner. The significance of a REO in RD (or pCR) samples was determined using a binomial test (For each RD (or pCR) sample, the gene expression profile was first converted into a rank profile according to measured expression levels in ascending order . Then, pair-wise combinations of all genes were examined to determine the REO pattern of each gene pair within the sample. The within-sample REO of a gene pair  status, k denotes the number of samples that have a certain REO pattern (iG > Gj or Gi < Gj), and p0 denotes the probability of observing a gene pair with a certain REO pattern by chance . Then the P-values were adjusted using the where We then defined stable-reversal gene pairs as pairs that had a significantly stable REO pattern in the pCR samples and RD samples, respectively, but had a reversal REO pattern between the two groups.Based on the stable-reversal gene pairs between the pCR and RD, we developed an extremely resistant signature. A sample was identified as an extremely resistant sample, if the number of REOs of the signature gene pairs matching that of the extremely resistant group was significantly more than expected by chance. The threshold for identifying an extremely resistant sample was determined according to a binomial test as follows:n is the number of signature gene pairs and k is the number of gene pairs in the sample that match the REOs for the extremely resistant group. p0  is the probability of a gene pair having a certain REO pattern in a sample by chance.where The distant relapse-free survival (DRFS), defined as the time from surgery to distant recurrence or the final documented date (censored), was used as a surrogate assessment of tumor response status . A log-rThe flowchart of the process used for developing and validating the resistance signature is shown in P < 0.05) of NR61 showed the same REO patterns as observed in extreme resistance, the sample was identified as extremely resistant; otherwise, it was considered a responder.To identify an extremely resistant signature, we first extracted 169,222 gene pairs with stable  but reversed REOs between the pCR and RD group from the integrated data of the GSE20194, GSE20271, and MDA133 datasets, with 86 pCR samples and 121 RD samples in total. We then used the GSE41998 dataset to optimize the signature. Beside the pCR and RD category, the GSE41998 dataset also provided an evaluation of drug response criteria in solid tumors (RECIST), which divided the patients into four groups: complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD) . Among tResearchers  have proIn the GSE25055 and GSE25065 datasets, 122 and 61 ER-negative breast cancer samples were separately re-determined using the 112-gene-pair signature  and wereP < 0.01; HR = 3.84; 95% CI = 1.91–7.70; P < 0.01; HR = 3.07; 95% CI = 1.28–7.36; The survival analysis was then used to validate the NR61 signature, assuming that the responsive patients have a better prognosis than the extremely resistant patients. First, the survival analysis was performed in all re-determined ER-negative breast cancer patients. In the GSE25055 dataset with 122 ER-negative breast cancer patients, 108 and 14 patients were classified as responders and extremely resistant, respectively. The extremely resistant patients had a significantly shorter DRFS time than the responders , which could well predict the ER-negative breast cancer patients who were extremely resistant to PAC NAC. Based on this signature a total of 183 ER-negative patients in the two validation datasets could be divided into responder and extremely resistant patients. Our research showed that the DRFS time of the extremely resistant group was significantly shorter than that of the responders. Patients identified with extreme resistance should be recommended other treatment schemes to avoid unnecessary suffering and expenses. Additionally, this signature can correctly identify almost all patients who can completely respond to PAC NAC.Our qualitative transcriptional signature based on the within-sample REOs is robust against batch effects  and coulIn this study, PD and SD samples screened from RD samples were defined as extremely resistant to PAC NAC. The pCR-RD system is based on microscopic observation and a large number of patients are diagnosed with RD. However, in the image-based RECIST system , PD is dDue to the lack of RNA-seq data with suitable drug response information, we only tested the NR61 signature in the microarray data measured on the Affymetrix platform. In future, we will collect the breast cancer expression data from the RNA-seq platform to optimize our signature, in order to improve its cross-platform ability.In summary, the NR61 signature could be used to robustly identify patients who are extremely resistant to PAC NAC among ER-negative breast cancer patients. These patients are highly unlikely to benefit from the PAC NAC regimen and should thus be recommended other therapeutic regimens. The clinical value of the NR61 signature for extreme resistance to the PAC NAC regimen thus deserves further validation.https://bioinformatics.mdanderson.org/public-datasets/).The datasets analyzed in this study could be found in the Gene Expression Omnibus  and MD Anderson Cancer Center (ZG and YC conceived the project. YC and HC performed the computational experiments. YC and JL designed the data analyses. QG and JH interpreted the data. YC, HC, and ZG wrote the manuscript. All authors contributed to the preparation of the manuscript, and read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.