Neurofibromatosis type 1 (NF1) is a monogenic syndrome that gives rise to numerous symptoms including cognitive impairment, skeletal abnormalities, and growth of benign nerve sheath tumors. Nearly all NF1 patients develop cutaneous neurofibromas (cNFs), which occur on the skin surface, whereas 40–60% of patients develop plexiform neurofibromas (pNFs), which are deeply embedded in the peripheral nerves. Patients with pNFs have a ~10% lifetime chance of these tumors becoming malignant peripheral nerve sheath tumors (MPNSTs). These tumors have a severe prognosis and few treatment options other than surgery. Given the lack of therapeutic options available to patients with these tumors, identification of druggable pathways or other key molecular features could aid ongoing therapeutic discovery studies. In this work, we used statistical and machine learning methods to analyze 77 NF1 tumors with genomic data to characterize key signaling pathways that distinguish these tumors and identify candidates for drug development. We identified subsets of latent gene expression variables that may be important in the identification and etiology of cNFs, pNFs, other neurofibromas, and MPNSTs. Furthermore, we characterized the association between these latent variables and genetic variants, immune deconvolution predictions, and protein activity predictions. Neurofibromatosis type 1 is a rare disease and a member of the family of RASopathies (diseases caused by germline mutations in genes that encode components or regulators of the Ras/mitogen-activated protein kinase (MAPK) pathway) that occurs in approximately 1 in 3000 patients worldwide and gives rise to cognitive impairment, skeletal abnormalities, and various nerve tumors including gliomas and neurofibromas and is caused by a mutation or deletion in one NF1 allele ,3. NerveEED or SUZ12 genes, alongside CDKN2A and NF1 gene mutations as crucial co-mediators of MPNST transcriptional dysregulation, pathogenesis, and sensitivity to bromodomain (BRD4) inhibitors [MET and HGF gene amplifications in MPNSTs. Furthermore, models of MET-amplified MPNSTs were subsequently sensitive to the MET inhibitor capmatinib [The rise of high-throughput genomic and transcriptomic sequencing has enabled many advances in understanding the molecular etiology of NF1 tumor types ,9,10,11.pmatinib . Transcrpmatinib ,14. TakePrevious genomic profiling studies have demonstrated that many NF1 nerve sheath tumors (MPNST being the exception) are genetically quiet ,15,16,17However, comprehensive analysis of genomic data in NF1 tumors is limited . To enabIn this work, we reprocessed and analyzed RNA-seq data from 77 NF1 nerve sheath tumor samples to understand the biological differences that give rise to distinct tumor types in NF1 patients. Given the low sample size compared to the large feature space of RNA-seq data, we applied a transfer learning-inspired approach to meaningfully reduce the feature space with minimal decrease in information content. Transfer learning techniques can leverage large well curated datasets such as recount2  to identrenv lockfile in the GitHub repository. Key packages used include “tidyverse” [http://nfdataportal.org) with analyses stored at http://synapse.org/nf1landscape. To recapitulate the analysis from these data, all relevant code can be found at https://github.com/Sage-Bionetworks/NF_LandscapePaper_2019. All analyses were performed using the R programming language. A comprehensive list of packages used and their versions are available in an dyverse” , “PLIER”dyverse” , “synapsdyverse” , “tximpodyverse” , “immunddyverse” , and “vidyverse” . All datdyverse”   according to the workflow located at https://github.com/Sage-Bionetworks/rare-disease-workflows/tree/master/gene-variant-workflow and then uploaded to the NF Data Portal. A list of datasets and the access teams required to download them can be found in Genomic variant data were collected from exome-Seq  or wholeWe analyzed transcriptomic data from NF in the context of latent variables from MultiPLIER, a machine learning resource designed to aid in rare disease analyses . Raw trahttps://github.com/greenelab/multi-plier). To project the NF data into the MultiPLIER model, we used the GetNewDataB function. This analytical approach is described in more detail in a machine learning training module (https://github.com/AlexsLemonade/training-modules) produced by the Alex’s Lemonade Stand Foundation’s Childhood Cancer Data Lab under an open source license. MultiPLIER reuses models that were trained on large public compendia. We retrieved a model  that washttps://github.com/Sage-Bionetworks/NF_LandscapePaper_2019. In addition, to ensure orthogonality in the final set of latent variables , we calcWe used the R function prcomp to compute the principal components for To select gene expression patterns of interest, we used ensembles of random forests to sufficiently resample our modestly sized dataset. We compared random forest models built using gene expression data as well as latent variables. model set and 20% independent test set. The function createDataPartition was used to create balanced splits of the data according to the tumor type. We tuned two parameters to the random forest algorithm, mtrys, and ntrees, using an iterative approach, evaluating mtrys values of 1 to 100 and ntrees values of 250, 500, 1000, and 2000. We selected the optimal values  using fivefold cross-validation, using latent variables as input features. We then split the model training set further to generate 500 samples of training data (75%) and hold-out test data (25%) . Each of these training and test datasets were used to train separate random forests to obtain a distribution of F1 scores and feature importance scores (n = 500). Given our noisy dataset with limited sample size, the distribution of feature and F1 scores enabled estimation of confidence intervals for the feature importance as well as model performance. The main algorithm was implemented using the “caret” and “randomforest” packages in R ,45. Figuimportance score of each feature from the distribution of raw importance scores generated through 500 iterations of random forests. The top 40 features were selected according to the mean decrease of the Gini index. The union of top features from all classes was then used as a restricted feature set to train another 500 iterations of random forests as described above. However, each new forest trained with the restricted feature-set was tested using the independent test set to examine the performance of the model on a completely unseen dataset. For each class, the median F1 scores of the new ensemble of forests were compared to the previous ensemble of random forests. Improvement of median F1 scores for each class in the final ensemble of random forests compared to the earlier one suggested that the selected features from each class were sufficiently informative for their classification. This subset of features was then selected for downstream analyses.The importance of each feature was estimated using raw importance scores that measure the change in correctly classified “class” due to random permutation of the values for the feature. To select the top features for a specific class , we calculated the median https://github.com/Sage-Bionetworks/NF_LandscapePaper_2019 and results were uploaded to a Synapse table (syn21177277) that includes the tumor-specific immune cell scores for both algorithms as well as associated tumor metadata.To understand the relative immune infiltration across the nerve sheath tumors studied, we used two tumor deconvolution methods: CIBERSORT and MCP-counter, as implemented through the “immunedeconv” R package ,30,31. AWe applied the metaVIPER algorithm  to inferp-value < 0.05. Results were plotted using “ggplot” and “enrichplot” packages [A heatmap and subclusters of latent variables that had similar VIPER protein predictions were generated using the “pheatmap” R package . We obsepackages ,50. To pWe collected mRNA sequencing, exome sequencing, and whole genome sequencing data from previously published or publicly available resources as depicted in To measure the diversity of the nerve sheath tumors at the transcriptomic level, we re-processed RNA-seq data from three published datasets ,38,39 anp-value = 2.1 × 10−182, Wilcoxon test). To reduce the dimensionality of the RNA-seq data and increase the biological interpretability, we applied MultiPLIER, a transfer learning approach trained on the recount2 dataset, an independent dataset comprising thousands of gene expression experiments. This analysis quantified the expression of 962 latent variables (LVs) in all 77 samples using their gene expression data . We antiWe then sought to identify which LVs characterize the differences between individual tumors. To find these, we examined the LVs with the 5% largest standard deviation across all LVs. Many of the resulting latent variables D were foTo identify these expression features, we used random forest models, an approach that is known for its ability to characterize complex decision spaces better than basic clustering approaches . Random p-values: cNF < 2.2 × 10−16, MPNST = 3.789 × 10−11, NF = 7.297 × 10−09, pNF < 2.2 × 10−16). This suggested that the 98 selected LVs with high importance scores for each tumor type were sufficient for characterizing the specific tumor types  associated with single gene variants  due to mutation or deletion. However, NF1 patients show a great deal of phenotypic heterogeneity [in silico resources and analytical techniques to identify candidate genes or pathways unique to different tumor types in an attempt to generate testable hypotheses. By capturing complex gene expression patterns using latent variables (LVs), we identified combinations of LVs that were important to classify tumor types. An ensemble of random forests was then used to select 98 latent variables that were important and sufficient for identifying and classifying the various tumor types with reasonably high accuracy  and is thought to play a role in pathways associated with myelination [EGR2 in Schwann cell differentiation is still under active investigation [EGR2-driven pathways have been found to be significantly downregulated in Lats1/2-deficient Schwann cell-based MPNST models [RUNX2 gene (a major component of LV 624), has been shown to drive neurofibromagenesis by repression of the PMP22 gene encoded myelin protein (a Schwann cell component) [+ T cells [+ T cell markers as tumor microenvironment components in our samples   as well  T cells . As disc samples A. Alternatively, we also evaluated LVs that are uniquely associated with specific tumor types. For example, LV 24 was found to be an important feature for classifying MPNST tumor samples but not the other benign tumor samples Biii. LV Beyond looking at individual genes that comprise the latent variables, we employed orthogonal algorithms that measured tumor immune activity and regulatory protein activity to identify specific signatures represented by the latent variables. Tumor deconvolution methods  confirmed previously described observations such as the presence of mast cells in NF1 model systems and patient tumors ,72,73,74NF1-deficient or RAS-dysregulated tumors [Through the metaVIPER protein regulatory activity predictions, we were able to identify putative therapeutic candidates for further evaluation. Specifically, we found two clusters of latent variables expressed in NF, pNF, and MPNST that had regulatory proteins enriched in targets of dovitinib and drug-like small molecules that inhibit receptor tyrosine kinases, as well as cyclin-dependent kinase (CDK) inhibitors such as abemaciclib or dinaciclib. These findings are consistent with previous studies that identify CDK inhibitors as useful in models of d tumors ,78,79,80d tumors ,81. Thusin-silico study brings forth various candidate genes and pathways for further follow up, it also presents a few limitations that could be mitigated with future studies, particularly for tumor types with limited samples such as MPNSTs and neurofibromas. Most notably, analyzing genetic variants across tumor types failed to identify relevant variant signatures (Although the present gnatures . This hiIn conclusion, this work proposed a short list of testable hypotheses involving specific biological signatures for NF1-deficient nerve sheath tumors. Verification of these mechanisms in in-vitro and in-vivo models of nerve sheath tumors as well as human NF1 nerve sheath tumor tissue needs active and extensive experimental work. Although we analyzed tumor datasets from four different studies, the addition of other neurofibromatosis-driven tumor datasets will greatly aid identification of commonalities or critical differences to inform therapeutic decisions across the family of neurofibromatoses. This study, together with future work, may guide the repositioning of clinically approved drugs in the context of NF1.


To address this problem, we statistically characterize common NGS quality features and develop a novel quality control procedure involving tree-based and deep learning classification algorithms. Predictive models, validated on internal and external functional genomics datasets, are to some extent generalizable to data from unseen species. The derived statistical guidelines and predictive models represent a valuable resource for users of NGS data to better understand quality issues and perform automatic quality control. Our guidelines and software are available at The online version contains supplementary material available at 10.1186/s13059-021-02294-2. Functional genomics based on next-generation sequencing (NGS) technology is used to study regulatory elements in genomes of all types of species. It is widely used in biological and clinical applications thanks to a variety of existing complementary assays that allow the investigation of, for example, gene expression quantification (RNA-seq), epigenetic modification and transcription factor occupancy (ChIP-seq), and open chromatin regions .The analysis of NGS data requires a stepwise process handled by dedicated software tools. The first processing step is the quality control (QC) of the data. It is of crucial importance to filter out low-quality data files that would have a negative impact on downstream analyses through addition of noise or systematic bias to the analyzed dataset . When deQC tools, used at the first step of an NGS pipeline, analyze raw sequencing data. The raw data is stored in FastQ files containing a set of short strings of DNA sequences  called reads and related information such as a quality score for each base reflecting its sequencing error probability. The most popular QC tool in the NGS community is FastQC . It perfDue to the coordinated efforts of many research groups, large repositories have been created that collect NGS data files in order to make them available to the scientific community. The scope of some repositories such as GEO and ArrayExpress is to share data with a minimal amount of annotation describing experimental conditions. Annotations are created in accordance with detailed guidelines  and ontoAlthough machine learning has been used to classify the quality of reads or single-nucleotide polymorphisms , a high After extensive validation of the models, we investigated their usefulness in different scenarios such as database curation and disease diagnosis. Finally, we released to the community our statistical guidelines and a software application to use the classification models on new datasets.We have studied a large number of annotated NGS files to characterize data quality and to create machine learning models able to automatically predict quality from the raw data. Our goal is to provide an alternative to manual quality control of NGS data files, which currently requires high-level expertise and highly depends on assays and experimental conditions and is prone to human biases. Interestingly, a minimum number of uniquely mapped reads and usable fragments mentioned in the ENCODE guidelines cannot be used to categorize the ENCODE data with respect to quality . Other quality features are less interesting because of their poor performance in all subsets . Comparing species, DNase-seq showed more differences than ChIP-seq results. This is probably due to the large difference in composition between human and mouse DNase-seq samples  would not be highly biased towards specific genomic localizations or TSS relative positions different to RNA-seq, for instance, for which these two feature sets enable very high performing models . Nevertheless, we also evaluated separate models for narrow-peak and broad-peak ChIP-seq samples  often performed better than others as well as multilayer perceptron, which is a deep learning classifier based on artificial neural networks Fig. a. In genA main limitation of this study is the availability of labeled data only for human and mouse. In order to know if the models were potentially generalizable to other species, we conducted a test where models were trained with human data and tested on mouse data unseen during model training, and vice versa Fig. a. The teA model that was of importance to us was the optimal generic model trained on files from different species and different assays of the full dataset. Although its performance was high during the cross-validations Fig. b we wereWe have shown above that machine learning models trained on appropriate data from the same source (ENCODE) are powerful and unbiased predictors of NGS file quality. Testing the models on independent datasets from another source, such as the GEO  and CistThe potential of the models to filter low-quality files from diagnostic studies was evaluated on 90 samples from 6 independent gene expression studies related to the following diseases: Alzheimer’s disease (GEO Series GSE125583), Crohn’s disease (GSE66207), diabetes (GSE92724), sporadic amyotrophic lateral sclerosis (GSE76220), liver cancer (GSE25599), and thyroid papillary carcinoma (GSE63511). Assuming a large effect of the quality on gene expression data, being able to automatically identify files of low quality would potentially prevent patients from receiving inappropriate medications or treatments. We analyzed the datasets to mark samples predicted to have low quality as potential outliers  between the number of Cistrome’s bad quality flags and the low-quality probabilities derived by the optimal generic model  and only moderate with human data (0.15). Applying the specialized DNase-seq models to ATAC-seq data also performed well with mouse data (0.44) and substantially improved the results for human data (0.46). Although this is not an external or independent validation, we note that a Random Forest model directly trained on ATAC-seq data showed high performance in 10-fold cross-validations  and were applicable to other assays and species. The application of this method on labeled database content and external datasets was very promising in its ability to clearly identify problematic samples.The study of quality features derived by widely used bioinformatic tools on the ENCODE files provides statistic-based guidelines to NGS specialists who have to make a manual decision of high complexity on the quality of their files  for integration. The restricted amount of annotated low-quality data in ENCODE was therefore the main limitation of our study that could have generated biases and non-generalizable models. Our models have been trained only on two species and three assays for which we found a reasonable number of files. Interestingly, the generalization across different species of some models partly overcomes this limitation. In systematic evaluations, we could still identify biases for some specialized models when sample compositions differ widely between training data and test set. However, we found no biases towards data subsets or experiments when using the optimal generic model based on Random Forest and trained on all data types and all features. This model also generalized better to external datasets, including human and mouse RNA-seq, ChIP-seq, DNase-Seq, and even ATAC-Seq data, a type of assay that had not been used in the ENCODE training set. The inclusion of various types of assays in the training set seemed critical for generalization.In general, models may also benefit from additional features from other NGS software tools or genomic annotations , 26. StiAs it has been noted in other contexts, negative results can be valuable , and we The possible applications detailed in this study highlight the usefulness of our predictive models. Either as a researcher or as a database curator who wishes to identify low-quality files, using the models as a decision support tool can save a substantial amount of resources (up to 50% for problematic experiments). For selected external disease datasets, models trained with ENCODE data have shown their relevance by classifying potential outlier samples. Nevertheless, as explained above, we would highly recommend to directly train models on most similar clinical data to create the best possible models that will prevent the impact of quality issues to diagnosis and therefore to enable patients to receive appropriate medication or treatment. Implemented calibration methods may also be used to better compare results from different models . Yet, weFinally, the software used to create and apply the models is freely available online either in a github repository or as a docker image (see ). Using We have statistically characterized common NGS quality features of a large set of data files and optimized their complex quality control using a machine learning approach. The derived statistical guidelines and predictive models represent a valuable resource for NGS specialists. Predictive models can be unbiased, accurate and to some extent widely applicable to unseen data types. Given enough labeled data for training, this approach could work for any type of NGS assay or species. Therefore, we strongly encourage researchers to share both high- and low-quality data with the community.To analyze the potential of machine learning applied to the quality assessment of NGS data files, we implemented a workflow as shown in Fig. a. FastQ We downloaded 1321 low-quality files plus the same number of high-quality files  to define balanced training and testing sets. For the selection of high-quality files, we prioritized files that are associated with an experiment that also contains revoked files. The remaining high-quality files were chosen randomly. From this full dataset, we defined data subsets as sets of files representing a combination of species and assay such as human ChIP-seq or mouse DNAse-seq  or on data subsets containing samples for a particular combination of species and assay  as defined above. Based on the different species-assays and feature set combinations, machine learning algorithms were applied to train models that classify the quality class defined from the ENCODE status. A comprehensive grid search was applied to find the optimal algorithm and parameter setting for each subset. The algorithm set is listed within Fig. The predictive performance was evaluated by the area under receiver operating characteristic curve (auROC). Within the grid search and feature selection, a tenfold cross-validation was applied to evaluate the predictive performance. The entire grid search  was implemented using the Python package scikit-learn . The XGBPrimarily, the best performing classification model was evaluated by the best auROC. For example, the optimal generic model trained on all data and features is a Random Forest model using the entropy as splitting criterion, the default maximum depth, the squared root function to define the maximum number of features, and 1000 estimators (trees). We additionally assessed all models by the Brier loss that represents how well the predicted probabilities are calibrated (the lower the better). However, we could not find models that drastically improved after calibration for most of the cases  the accessions are given below. The data was preprocessed based on the workflow that we also applied on the data from ENCODE as explained above and in Fig. We used 610 samples from 32 datasets available in GEO and referenced in the Cistrome  databaseFrom the Cistrome database, we used 5 numerical quality metrics derived by their own pipeline  that couIn order to create a labeled training set for a human ATAC-seq specialized model with the Cistrome data, we divided the samples in two groups as follows: if a sample had more than 3 good quality flags it was labeled as high-quality, low-quality otherwise. From 90 NGS samples, our arbitrary labeling resulted in a well-balanced training set with 49% low-quality samples. On this set, we trained a Random Forest model using the settings that achieved optimal results in our grid search on the generic case . This model was evaluated using a 10-fold cross-validation.Additional file 1: Figure S1. Statistical guidelines computed on the ENCODE files selection. Figure S2. Predictive performance of tuned machine learning models. Figure S3. Within-experiment benchmarks. Figure S4. Cross-species generalization. Figure S5. Paired-end human ChIP-seq data subset. Figure S6. Counts of broad peak targets in the ChIP-seq samples. Figure S7. Counts of sample names in the DNase-seq samples. Figure S8. Comparison of predictive and calibration performance. Figure S9. Peak-type specific one-feature predictions. Figure S10. Predictive performance of peak-type specific classification models. Figure S11. Cross validated predictions of the optimal generic model across most frequent ChIP-seq protein targets. Figure S12. External validations (RNA-Seq). Figure S13. Independent validation on Cistrome’s datasets. Figure S14. ENCODE guidelines and status.Additional file 2. Table of grid search parameters. Parameters are relevant to a scikit-learn implementation.Additional file 3. Table of tuned models used by seqQscorer. Dataset: species-subset-layout, generic: all data. Feature sets: RAW (raw data), MAP (genome mapping), LOC , TSS (transcription start sites profile). Feature Selection: method-percentage (percentage of retained features), chi-square (chi2), recursive feature elimination (RFE). Algorithm Parameters: relevant to a scikit-learn implementation.Additional file 4. Table of optimal models used by seqQscorer. Optimal models are the best models for each data subset (Dataset column). Dataset: species-subset-layout, generic: all data. Feature sets: RAW (raw data), MAP (genome mapping), LOC , TSS (transcription start sites profile). Feature Selection: method-percentage (percentage of retained features), chi-square (chi2), recursive feature elimination (RFE). Algorithm Parameters: relevant to a scikit-learn implementation.Additional file 5. Table of models tuned using the grid search. Dataset: species-subset-layout, generic: all data. Feature sets: RAW (raw data), MAP (genome mapping), LOC , TSS (transcription start sites profile). Feature Selection: method-percentage (percentage of retained features), chi-square (chi2), recursive feature elimination (RFE). Algorithm Parameters: relevant to a scikit-learn implementation.Additional file 6. Table of Cistrome datasets. The table details all Cistrome datasets used for independent validation. Dataset GSE60731 is listed 2 times to represent either mouse DNase-seq samples (GSE60731) or ChIP-seq samples (GSE60731_2). GEO_Series: identifier from the GEO database.Additional file 7. Table of ENCODE datasets. The table details all ENCODE datasets used for training the model). Quality label is 0 for released files or 1 for revoked files.Additional file 8. Review history.


The experimental results show that the ISS-WME algorithm can represent the HSI structure better than other methods, and effectively improves the classification accuracy of HSIs.Due to the spectral complexity and high dimensionality of hyperspectral images (HSIs), the processing of HSIs is susceptible to the curse of dimensionality. In addition, the classification results of ground truth are not ideal. To overcome the problem of the curse of dimensionality and improve classification accuracy, an improved spatial–spectral weight manifold embedding (ISS-WME) algorithm, which is based on hyperspectral data with their own manifold structure and local neighbors, is proposed in this study. The manifold structure was constructed using the structural weight matrix and the distance weight matrix. The structural weight matrix was composed of within-class and between-class coefficient representation matrices. These matrices were obtained by using the collaborative representation method. Furthermore, the distance weight matrix integrated the spatial and spectral information of HSIs. The ISS-WME algorithm describes the whole structure of the data by the weight matrix constructed by combining the within-class and between-class matrices and the spatial–spectral information of HSIs, and the nearest neighbor samples of the data are retained without changing when embedding to the low-dimensional space. To verify the classification effect of the ISS-WME algorithm, three classical data sets, namely Indian Pines, Pavia University, and Salinas scene, were subjected to experiments for this paper. Six methods of dimensionality reduction (DR) were used for comparison experiments using different classifiers such as  With the development of science and technology, hyperspectral images (HSIs) have become the main research direction in the field of modern remote sensing technology. HSIs have a large number of spectral bands, which provide detailed spectral information about objects ,2. HowevA DR method aims at extracting important features of images, mapping high-dimensional data to low-dimensional space, and using the data in low-dimensional space to describe high-dimensional features . In rece2SLLE) algorithm, which introduces the Spearman correlation coefficient to determine the appropriate nearest neighbor points, and increases the discriminability of embedding data on the basis of supervising the LLE method. Zhang et al. [The purpose of the MDR method in HSI is to find the manifold structure in the high-dimensional space. LLE  obtains g et al.  proposedg et al.  proposedA new weight matrix is constructed to describe the structure between samples, in which the product of the spatial–spectral distance weight matrix and the structure weight matrix is taken as a new data weight matrix. Compared with the previous weight matrix, which only considers spectral distance or spatial distance, the new weight matrix integrates the spatial–spectral information and structural characteristic of the data.The model not only makes the manifold structure invariant, but also preserves the nearest neighbor relationship of the samples, when the high-dimensional data are projecting to the low-dimensional space.In this paper, an improved spatial–spectral weight manifold embedding (ISS-WME) algorithm is proposed to combine spatial–spectral information and manifold structure to extract the features of HSI. First, the spatial–spectral information of HSI is extracted with the Gaussian variant function. The product of the spatial distance matrix and the spectral distance matrix is then used to be the distance weight matrix. Then, the collaborative representation method is used to express the characteristics of the HSI structure. Samples from the same class are as much as possible in the same hyperplane after projection, and samples from the different classes are as far apart as possible. The structural weight matrix is obtained by combining the within-class and between-class weight representation matrices. The product of the distance weight matrix and the structure weight matrix is used as the new weight matrix. When the data is mapped from high-dimensional manifold space to low-dimensional space, it is easy to make abnormal points appear if only considering the structural distribution between the data points. Furthermore, it is easy to cause the problem of sparseness if only keeping the data nearest neighbor relationship unchanged during projection transformation. To overcome abnormal points and the sparseness problem, both the structure and neighbor sample relationship are taken into account in this paper. Finally, the model can be efficiently solved by solving the minimum eigenvalue to the generalized eigenvalue problem and obtaining a projection matrix. The main contributions of the proposed algorithm are as follows:This paper is arranged as follows. D-dimension features and N is the number of the samples. We assume that the D-dimensional-sample M, Given the data set  problem :(1){min∑k samples, which is closest to Xi, and D-dimensional samples into d-dimension space, it is desirable to maintain the same linear relationship:I is the identity matrix and In Equation (1), Using the method of Lagrangian multiplier, Equation (3) can be easily solved by the generalized eigenvalue decomposition approach as follows:Then, we can obtain the eigenvector corresponding to the dth smallest non-zero eigenvalues, and the low-dimensional embedding matrix can be represented as The LLE algorithm  can succk-nearest neighbors of the sample X, and Y can be solved by constructing the following optimization problem [Given the data set  problem :(5){min∑d-dimensional space are constructed by the eigenvectors corresponding to the d minimum eigenvalues.Similarly, Equation (5) constraints ensure it has a solution. And it can be solved by using the generalized eigenvalue decomposition approach as follows:The LE algorithm  introducTo solve large sparsity, inexact weight, and other problems of the LLE and LE  algorithThrough experimental study, researchers have found that classification accuracy can be improved by combining the spatial information in the analysis of HSI. Hence, the ISS-WME method is based on spatial and spectral information. It uses the variation of Gaussian function to represent the spatial and spectral distance, respectively. Given the HSI data set Therefore, the spatial–spectral distance weight matrix is as follows:In HSI, adjacent pixels in the same homogenous region usually belong to the same class, so any sample in the same class can be linearly represented by homogeneous neighbor samples. Similarly, the whole data sample centers can be represented by different classes of sample centers . Hence, In Equation (9), Likewise, the objective function of the between-class representation coefficient matrix is as follows:kth class sample, and the between-class mean coefficient matrix is In Equation (10), The within-class representation matrix Therefore, the within-class coefficient matrix is as follows:In the same way, we can set the derivative of objective function about between-class representation coefficients to be 0, so the between-class coefficient matrix is as follows:Given the HSI data set ed by Wu , both ofFurthermore, the high-dimensional data mapping to the low-dimensional space not only makes the local manifold structure unchanged, but also maintains the local neighbor relationship invariant. Introducing the weight of Equation (14) to increase the robustness of the model, the improved weight manifold embedding optimization problem is as follows:Algorithm 1 Process of the ISS-WME AlgorithmK is the nearest neighbor.1: HSI is segmented into superpixels using the SLIC segmentation method and randomly select training samples , and then use Equations (7) and (8) to calculate the spatial–spectral distance matrix between superpixels. In addition, make sure the number of superpixels and training samples is the same.2: Then, use Equations (12) and (13) to obtain the structure representation matrix between training samples. The product of the two types of matrices is taken as the new matrix Equation (14).3: According to the local manifold structure and nearest neighbor relationship of the samples, the objective function of Equation (16) is constructed. 4: By solving the generalized feature of Equation (18), the corresponding eigenvector is obtained.5: Learn a projection matrix P.Output: The data in low-dimensional space is Input: HSI data set With the method of Lagrange multiplier, the optimization problem is formed as follows:In order to verify the effectiveness of the proposed algorithm ISS-WME, we conducted experiments on three commonly used HSI data sets, namely Indian Pines, Pavia University, and Salinas scene. We considered the overall accuracy (OA) , classifThe Indian Pines, Pavia University, and Salinas scene data sets were subjected to experiments in the paper.The Indian Pines data set ,31 and SThe Pavia University data set ,32 was aFor this paper, six different DR algorithms were compared with the proposed ISS-WME method. These comparison algorithms are described as follows. PCA, Isomap, LLE, and LE are four classical DR algorithms. The SSSE algorithm combines the spectral and spatial information and WLE-LLE combines the spectral and structural information. In addition, for the LE, LLE, WLE-LLE, and SSSE algorithms, the number of nearest neighbor samples must be set in the experiment. To compare and analyze the classification results in the experiment, the nearest neighbor samples were set as 15 in all experiments. The SSSE and ISS-WME algorithms also require computational spatial and spectral information, so we set the parameters as In each experiment, each data set was divided into training samples and testing samples. We used different DR algorithms to learn a projection matrix on the training samples, and then utilized the acquired embedding matrix to project the testing samples into the low-dimensional space. Finally, we used a KNN or SVM classifier to classify the data in the low-dimensional space. Moreover, to reduce the systematic error, the results were computed 10 times to calculate the average value for each experimental result with the associated standard deviation. We used OA, CA, AA, and K to evaluate the different algorithm performances. In the Indian Pines experiment, the parameters were set to To fully attest the algorithm performance of the ISS-WME method, experiments were carried out under the conditions of different numbers of training samples, different embedding dimensions, and different DR methods. We randomly selected n%  samples from each class as the training sample set, and the rest were the testing sample set. We also set the hyperspectral dimensionality (HD) of low-dimensional embedding from 10 to 50. The results of the proposed ISS-WME method were compared with those of the other comparison DR methods.As can be seen in In order to further describe the comparison results, the quantitative comparison of classification accuracy using SVM classifiers under HD = 20 for different DR methods is summarized in n%  samples from each class as the training set, and the rest were the testing set. We also set the hyperspectral dimensionality (HD) of low dimensional embedding from 10 to 50. The results of the proposed ISS-WME method were compared with those of the other DR methods.In order to fully attest the algorithm performance of ISS-WME, experiments were carried out under the conditions of different numbers of training samples, different embedding dimensions, and different DR methods. We randomly selected As can be seen in To further describe the comparison results, the quantitative comparison of OA of different DR methods at HD = 20 is summarized in To describe the comparison results, the quantitative comparison of OA of different DR methods when HD = 20 is summarized in In this paper, a dimensionality reduction method combining the manifold structure of high-dimensional data with a linear nearest neighbor relationship was proposed. The method aimed to keep the data nearest neighbor relationship unchanged when the high-dimensional data were projecting to the low-dimensional space. Furthermore, the manifold structure of the data combined the spatial–spectral distance and structural features. To fully verify the superiority of the proposed method, the data obtained by the ISS-WME method and the six other dimensionality reduction methods were classified by two common classifiers. The results of several experiments show that the ISS-WME algorithm improves the ground object recognition ability of hyperspectral data, and the OA and kappa coefficients also support this conclusion. In the future, the dimensionality reduction labeling will be further considered to improve the classification effect through the framework of semi-supervised learning.


RNA-seq is currently considered the most powerful, robust and adaptable technique for measuring gene expression and transcription activation at genome-wide level. As the analysis of RNA-seq data is complex, it has prompted a large amount of research on algorithms and methods. This has resulted in a substantial increase in the number of options available at each step of the analysis. Consequently, there is no clear consensus about the most appropriate algorithms and pipelines that should be used to analyse RNA-seq data. In the present study, 192 pipelines using alternative methods were applied to 18 samples from two human cell lines and the performance of the results was evaluated. Raw gene expression signal was quantified by non-parametric statistics to measure precision and accuracy. Differential gene expression performance was estimated by testing 17 differential expression methods. The procedures were validated by qRT-PCR in the same samples. This study weighs up the advantages and disadvantages of the tested algorithms and pipelines providing a comprehensive guide to the different methods and procedures applied to the analysis of RNA-seq data, both for the quantification of the raw expression signal and for the differential gene expression. Compared with microarrays, RNA-seq enables the study of novel transcripts and offers higher resolution, a better range of detection and lower technical variability6. RNA-seq also offers a high degree of agreement with other techniques considered as the gold standard in transcriptomics such as qRT-PCR, both at absolute and relative gene expression measurement levels7. All these facts have led to a great expansion of RNA-seq, becoming the first choice in transcriptomic analysis for many scientists. Nevertheless, its widespread use has generated a large amount of research on algorithms and methods that has eventually produced a lack of consensus about how to analyse RNA-seq data. Thereby, in the last decade, many different algorithms and pipelines have been proposed, but there is much debate about which approaches provide the most precise and accurate results. Thus, further research to compare these methods remains necessary.In recent years, RNA-seq has emerged as an alternative method to that of classic microarrays for transcriptome analysis8 and transcriptome assembly9. The beneficial effects of this process have been evaluated in reference-based11 and de novo13 RNA-seq analyses. The alignment to a reference genome or transcriptome is normally the second step in the RNA-seq workflow and has been widely evaluated by many authors19. Once the reads have been mapped, they must be assigned to a gene or a transcript, in a process known as counting or quantification. This is followed by a normalization procedure to remove possible sequencing bias. Since counting followed by normalization is a crucial component of RNA-seq data analysis, several methods have been developed and many comparative studies evaluating their suitability have been published28. The final step in most RNA-seq studies is DE analysis. The main concern at this point is how to accurately detect differentially expressed genes (DEGs) between two or more conditions. For that purpose, a large number of tools have been developed to facilitate the analysis, and thereby to construct lists of significant DEGs38. Due to the impact of this final step, the algorithms involved in DEG detection have been compared in many publications where authors analyse the pros and cons of applying different algorithms48.RNA-seq data analysis typically involves several steps: trimming, alignment, counting and normalization of the sequenced reads, and, very often, differential expression (DE) analysis across conditions. Trimming is used to increase reads mapping rate through the elimination of the adapter sequences and the removal of poor-quality nucleotides. It must be employed non-aggressively, together with a wisely chosen read length, to avoid unpredictable changes in gene expression49 investigated several workflows to pass from RNA-seq reads to differential gene expression (DGE) in Saccharomyces cerevisiae. Seyednasrollah et al.43 compared software packages for DE using two public data sets. Teng et al.50 suggested new metrics to assess differences between competing pipelines and Williams et al.51 evaluated many workflows for DE in human monocytes sequenced as 51 bp single-end reads.In this scenario, the major challenge in RNA-seq analysis is that the different steps must be sequentially combined in a complete workflow or pipeline and users have to choose between many possible methodological approaches and options. This shows the complexity of RNA-seq analysis and it is one of the most critical points to obtain accurate results both at raw and DE expression levels. Many possible combinations of the current algorithms in RNA-seq have been comparatively analysed to help decide the best workflow, but their performance remains under discussion. For example, Nookaew et al.Facing with the problems described, the present work aims to evaluate 192 alternative methodological pipelines using Illumina HiSeq 2500 paired-end RNA-seq data measuring its accuracy and precision at raw gene expression quantification level (RGEQ). These pipelines were constructed performing all the possible combinations of 3 trimming algorithms, 5 aligners, 6 counting methods, 3 pseudoaligners and 8 normalization approaches. We also evaluated the performance of 17 DE methods using the results of the top 10 ranked RGEQ pipelines. All these algorithms and methods were selected based on their popularity as they are used in dozens of scientific publications and can be easily implemented into a pipeline  and 0.4 mM (JJN-3) for 24 h (treatment 1 [T1]), with TG003 at 50 µM (both KMS12-BM and JJN-3) for 24 h (treatment 2 [T2]), and dimethyl-sulfoxide (DMSO) (treatment 0 [T0]) was used as negative control. All the experiments were done by triplicate, making up a total of 18 samples. RNA was extracted using the RNeasy Plus Mini kit . The RNA integrity was assessed with an Agilent 2100 Bioanalyzer (Agilent Technologies). The corresponding 18 RNA libraries were constructed following the TruSeq Strand-Specific RNA sequencing library protocol from Illumina . The RNA libraries were sequenced in a HiSeq 2500 system at Lifesequencing S.L. . Paired-end reads of 101 base pairs , were generated using this platform (Supplementary Table FASTQC (v0.11.3) tool53.We used two extensively characterized MM cell lines, KMS12-BM (cell line A [CLA]) and JJN-3 (cell line B [CLB]), as previously described54. We then performed a filtering process removing all genes with < 4 gene expression units  in the six DMSO treated samples for each pipeline. This process resulted in the choice of 107 genes that satisfied this filtering criterion in all the 192 pipelines . Taqman qRT-PCR mRNA assays (Applied Biosystems) were carried out in duplicate on these 32 genes that are highlighted in the Supplementary Table To determine qRT-PCR candidate genes we first selected in our dataset 1181 genes expressed in 32 healthy tissues from the RNA-seq data published by Uhlen et al.To measure DGE by qRT-PCR we used the ΔCt method, calculated as:Endogenous control normalization, where endogenous control was calculated as the mean of GAPDH and ACTB Ct values for each sample, b) Global median normalization, in which the normalization factor was calculated using the median value for genes with Ct < 35 for each sample, and c) Most stable gene, that was determined using the 4 algorithms available in the RefFinder webtool, BestKeeper56, NormFinder57, Genorm58 and the comparative delta-Ct method59. The gene ECHS1 was ranked the top position and it was considered as the most stable in our dataset. We detected bias on GAPDH and ACTB genes expression due to the drug treatments. This bias was an under-expression of these genes in the treatment conditions as it is shown in Supplementary Fig. Global median normalization) for our downstream analysis.Three normalization approaches were tested: a) Trimmomatic60 (v.0.35), Cutadapt61 (v.1.12) and BBDuk , the last included in the BBTools suite (https://sourceforge.net/projects/bbmap/). In all cases only reads with a Phred quality score > 20 and read length > 50 bp were selected for downstream analysis. The statistical calculations for the comparisons among the trimming algorithms regarding the reads mapping rate and the surviving reads were performed using the Kruskal–Wallis test, followed by the Dunn’s post-hoc test in the dunn.test62 (v.1.3.5) package in R63 (v.3.5.1). Details about each trimming algorithm are given in Supplementary Note We decided to apply the trimming procedure to eliminate adapter sequences present in our data and to improve read quality from the FASTQ files. Adapter removal and quality trimming were carried out using Tophat216 (v.2.1.0), STAR18 (v.2.5.3a) and Hisat264 (v.2.0.0). The Human genome version GRCh37 (hg19) from Ensembl65 was used as the reference genome. Transcriptome alignment methods represented by Bowtie266 (v.2.2.6) and STAR were also tested against the Ensembl (v82) transcriptome. Finally, hybrid methods, which combine both types of alignment, were represented by RUM19 (v.2.0.5_06) that used its own hg19 reference. A pre-processing step of adding Ns letters to the incomplete reads was needed in RUM since paired reads with different lengths were not allowed. BAM files from all the methods were sorted by read name and genome position using samtools67 (v.1.3.1) and unmapped reads were discarded. The statistical calculations concerning the unmapped reads for each algorithm were made using the Kruskal–Wallis test, followed by the Dunn’s post-hoc test in the dunn.test package. Details about each aligner are given in Supplementary Note We next evaluated the genome-based alignment methods Cufflinks68 (v.2.2.1), eXpress69 (v.1.5.1), HTSeq70 (v.0.6.1p1) using both the Intersection-Strict and the Union approaches, RSEM27 (v.1.2.31) and Stringtie71 (v.1.3.3b). Gene expression values were represented using the normalization techniques provided by each algorithm: Fragments per Kilobase of Mapped reads (FPKM), Transcripts per Million (TPM), Trimmed Mean of M values (TMM from edgeR), Relative Log Expression (RLE from DESeq2), upper quartile (UQ), coverage (Cov), estimated counts (Est_Counts) and effective counts (Eff_Counts). Gene level expression values from RSEM and eXpress were obtained by summing the transcript-level estimates. Details about these counting and normalization methods are given in Supplementary Note The number of alignments mapped to each gene was counted using as reference a gene transfer format (GTF) file from Ensembl (v.82). We used six counting methods: Kallisto26 (v.0.43.1), Sailfish28 (v.0.9.2) and Salmon72 (v.0.8.2). Salmon was executed using the FMD and the quasi-mapping-based (QMB) indexing modes. Details about pseudoaligners are given in Supplementary Note These methods do not consider the classical alignment process and carry out alignment, counting and normalization in one single step. We applied three commonly used pseudoaligners: Precision was tested on the six DMSO treated control samples (CLA-T0 and CLB-T0) considering each cell line separately (three replicates by cell line). For this purpose, we considered the 107 HKg expressed in the six samples and in 32 healthy tissues previously reported  was used to assess the association between the RNA-seq data and the qRT-PCR Ct values. Then, we calculated a rank for the correlation coefficient of each gene along each pipeline. Finally, a median rank for each gene was calculated. The lower the rank, the more accurate the pipeline was. This procedure is graphically described in the bottom panel of Fig. r correlation coefficients were calculated using the corr.test function implemented in the psych74 R package (v.1.9.12.31).Accuracy was tested on the six DMSO treated samples considering separately each cell line. We calculated the accuracy of each pipeline based on the 32 gene expression values from qRT-PCR, which is considered as a gold standard method due to its accuracy and sensitivityp-values were adjusted for multiple testing by the Benjamini–Hochberg False Discovery Rate (FDR) procedure75. All these calculations were made using the dunn.test package.The best pipelines were settled considering the summation of the precision and accuracy ranks inside each cell line, thus giving the same weight for both parameters. The lower the summation, the better the performance of the pipeline was. All the comparisons involving the pipeline ranks were statistically evaluated using the Kruskal–Wallis test followed by the Dunn’s post-hoc test. baySeq76 (v.2.10.0), Cuffdiff77 (v.2.2.1), DESeq278 (v.1.16.1), EBseq79 (v.1.16.0) and the edgeR80 (v.3.18.1) generalized linear model (GLM), and exact test variants; (b) methods that assume a log-normal distribution, like Ballgown81 (v.2.8.4) and the Trend and Voom limma82 variants (v.3.32.10); and (c) non-parametric methods such as NOISeq48 (v.2.20.0) and SAMseq83 . Details about these methods are given in Supplementary Note vs. CLB-T0 > CLA-T1 vs. CLA-T0 > CLA-T2 vs. CLA-T0 > CLB-T1 vs. CLB-T0 > CLB-T2 vs. CLB-T0. All the above DEGs comparisons were carried out at 3 FDR cut-offs: FDR < 0.05, FDR < 0.01 and FDR < 0.001.Seventeen DE methods were evaluated, as the result of the combination of 11 DE algorithms and their different normalization options. Such methods can be divided into three categories: (a) methods that assume a negative binomial distribution of data: dgeR80 v..18.1 gendist function from the stats package (v.3.5.1) using the Euclidean distance as distance measure and group average as linkage method. Dendrograms were depicted using the dendextend package (v.1.13.3). The performance of the DE methods was determined through the measurement of 7 diagnostic test parameters: sensitivity or true positive rate (TPR), specificity or true negative rate (TNR), the positive predictive value (PPV), the negative predictive value (NPC), accuracy (ACC), the area under the receiver operating characteristic curve (AUC) and the Matthews correlation coefficient (MCC). These parameters were calculated using as reference the qRT-PCR Benjamini-Hockberg’s adjusted p-values in a two-sample t-test. The true positive, true negative, false positive and false negative estimates were defined based on these qRT-PCR adjusted p-values and compared with the adjusted p-values obtained by each RNA-seq DE method.The similarity among the 17 methods was calculated in R through the All the computational procedures for raw gene expression quantification were performed in a 64-bit computer with 264 GB of RAM and 64 CPUs installed with a Linux system CentOS 6.9. The DGE quantification was carried out in a 64-bit computer with 64 GB of RAM and 24 CPUs with either a Linux system Ubuntu 14.04 or a Windows 10 system.We tested the precision and accuracy of 192 RNA-seq pipelines in two independent and well-characterized MM cell lines at raw gene expression quantification level (RGEQ) Fig. . These 1Salmon in conjunction with TPM normalization. The least precise pipelines were those in which the normalization process was less strict, such as the raw reads, Numreads, effective counts and estimated counts   and the lowest alignment rate (93.4%). On the other hand, BBDuk obtained the highest percentage of aligned reads (97.5%) , closely followed by TopHat2 . On the other hand, Bowtie2 only reached a 41.5% (MAD = 1.7) of concordant alignments, showing statistically significant differences with the other four aligners (FDR < 0.05) , both producing less than 5% of unmapped reads, although HiSat2 was particularly effective with only 1.6% (MAD = 0.3) unmapped reads  between the median ranks of the pipelines involving these alignment algorithms Fig. . Howeverods Fig. b.Cufflinks and RSEM reached the highest positions in the pipeline ranking followed by HTSeq and Stringtie based pipelines. These last two methods also obtained high-ranking positions, but they showed a bimodal distribution in their rank values that was eventually translated in a greater variability, similarly to that observed in the alignment step  method performed the best on RGEQ. Other normalization methods, such as Relative Log Expression (RLE) that was second best, and Transcript Per Million (TPM) or Fragments Per Kilobase of Mapped reads (FPKM), also reached high-ranking positions for precision and accuracy in RGEQ analysis  between their cumulative ranks of precision and accuracy Fig. . When weWe tested 17 DE methods obtained from the combination of the different DE and normalization approaches. They were tested under six experimental conditions with three biological replicates per condition. Comparisons under these conditions are explained in Fig. 43, we discovered that Cuffdiff was generally the algorithm that detected the smallest number of gene expression changes, whilst SAMseq was the method that detected the largest number of changes in all compared conditions. It was also observed that SAMseq itself and Ballgown lost detection power at FDR < 0.01 and FDR < 0.001 levels. BaySeq, EBSeq and Cuffdiff also lost detection power when the contrasted conditions were more similar. On the other hand, methods derived from edgeR, limma and NOISeq were stable in all comparisons at the three FDR levels analysed  between their results and the true model. Concerning ACC and AUC , while the method with a greater AUC, or in other words, the method with the highest discrimination capacity, was baySeq at FDR < 0.001 (AUC = 0.81).We also analysed 7 diagnostic test parameters for each DE method using qRT-PCR as described in the “AUC Fig. b,c, the EBseq, DESeq2 or baySeq showed a highly variable behaviour depending on the number of DEGs of the analysed scenario   in the top-ranked positions in our pipeline list. With respect to data normalization, pipelines based on the TMM method outperformed those using RLE, FPKM, TPM and all the other normalization approaches tested. This finding is consistent with those of Wu et al.21 and Maza et al.20, but is not in agreement with those of Li et al.22, who concluded that gene normalization methods did not improve the gene expression calculations provided by the raw counts.In this work, we also compared the most commonly used methods over the stages of RNA-seq analysis: trimming, alignment, counting and normalization. Interestingly, we did not find substantial differences in RGEQ related to the trimming algorithm or the aligner used in the RNA-seq data analysis. This last finding is consistent with the recent work of Schaarschmidt et al. in We also measured the performance of the pseudoalignment on RGEQ. Recently, pseudoaligners have entered the scene as an alternative to the classic alignment algorithms. One of their main advantages is that they can carry out the processes of alignment, counting and normalization in a single step. We have seen that pipelines based on pseudoaligners have a good precision in gene expression estimation, but their accuracy is inferior to that of the classic aligners. In any case, pseudoaligners could be a good alternative as RNA-seq data exploration tools since their execution time is faster than the conventional alignment methods.limma trend obtained the best results in terms of performance, closely followed by limma voom, NOISeq FPKM, baySeq, and some derivations of edgeR. It is interesting that even though the assumptions about the underlying distribution differ, the performance of these methods is comparable, as was also described by Rapaport et al.39 with other sets of methods. Of note, in spite of their similar performance, we detected differences in behavioural patterns for these methods. Whilst limma trend, limma voom and edgeR showed a good balance among the seven diagnostic test parameters analysed, baySeq and NOISeq tended to be biased in favour of some of them. In the case of baySeq, we found that its performance was sustained by the excellent specificity of this method, however, we detected a substantial lack of sensitivity. With respect to NOISeq, its performance was supported by its good sensitivity and precision, but we found low levels of specificity. A similar pattern of unequal performance between diagnostic test parameters for NOISeq was also reported in the work of Williams et al.51, who concluded that NOISeq exhibited the lowest recall (sensitivity), but agreed with our work in the high precision of this algorithm.Regarding RNA-seq DGE quantification, we established that the performance of the methods depended on the number of DEGs present under the contrasted experimental conditions and the FDR cut-off employed to determine statistical significance. Our systematic analysis revealed that 86 and Seyednasrollah et al.43, who reported a considerable overlap among DE methods regardless of the normalization approach used. However, our results are in contrast with those of Dillies et al.25, Bullard et al.87 and Zyprych-Walczak et al.88, possibly because these authors evaluated several normalization approaches regardless of the DE method, while in our study we only assessed the normalization provided by each DE method. It is also of note that Ballgown and SAMseq performed worse than others when the compared conditions had a low number of DEGs. It must be borne in mind that this bad behaviour may lead to a high frequency of false-negative deregulated genes in the data analysis.Another important aspect of the DE analysis is the possible influence of normalization on the results. Normalization is a critical step in RNA-seq DGE analysis since it enables expression levels to be compared. According to our results, the choice of the normalization method had little effect on DEGs detection compared to RGEQ. These findings are supported by the research of Assefa et al.Supplementary InformationSupplementary Information


ATRX deletions and elevated immune marker expression in high-risk neuroblastoma. Notably, hydra analysis of all small blue round cell tumors revealed similar subtypes, characterized by changes to infiltrating immune and stromal expression signatures.Precision oncology has primarily relied on coding mutations as biomarkers of response to therapies. While transcriptome analysis can provide valuable information, incorporation into workflows has been difficult. For example, the relative rather than absolute gene expression level needs to be considered, requiring differential expression analysis across samples. However, expression programs related to the cell-of-origin and tumor microenvironment effects confound the search for cancer-specific expression changes. To address these challenges, we developed an unsupervised clustering approach for discovering differential pathway expression within cancer cohorts using gene expression measurements. The hydra approach uses a Dirichlet process mixture model to automatically detect multimodally distributed genes and expression signatures without the need for matched normal tissue. We demonstrate that the hydra approach is more sensitive than widely-used gene set enrichment approaches for detecting multimodal expression signatures. Application of the hydra analysis framework to small blue round cell tumors  identified expression signatures associated with changes in the tumor microenvironment. The hydra approach also identified an association between  MYCN non-amplified neuroblastoma, osteosarcoma, and synovial sarcoma. The low mutation rate in pediatric cancers has led some to suggest that pediatric cancers are less immunogenic. However, our analysis suggests that immune infiltration can be identified across small blue round cell tumors. Thus, further research into modulating immune cells for patient benefit may be warranted.Pediatric cancers generally have few somatic mutations. To increase the number of actionable treatment leads, precision pediatric oncology initiatives also analyze tumor gene expression patterns. However, currently available approaches for gene expression data analysis in the clinical setting often use arbitrary thresholds for assessing overexpression and assume gene expression is normally distributed. These methods also rely on reference distributions of related cancer types or normal samples for assessing expression distributions. Often adequate normal samples are not available, and comparing matched cancer cohorts without accounting for subtype expression overestimates the uncertainty in the analysis. We developed a computational framework to automatically detect multimodal expression distributions within well-defined disease populations. Our analysis of small blue round cell tumors  discovered a significant number of multimodally expressed genes. Multimodally expressed genes were associated with proliferative signaling, extracellular matrix organization, and immune signaling pathways across cancer types. Expression signatures correlated with differences in patient outcomes for  Large cancer sequencing projects, including The Cancer Genome Atlas (TCGA) and Therapeutically Applicable Research to Generate Effective Treatments (TARGET), have facilitated the development of cancer gene expression compendia –6, but tThe TME includes tumor cells, stromal fibroblasts, immune cells, and vasculature . SimilarImmunotherapies that activate the host immune system to eradicate tumors have been effective in treating several cancer types, particularly cancers with a high mutation burden , 10. PedTumor/normal differential expression analysis in which a cohort of tumor tissues is compared to corresponding normal tissue samples is an effective approach for identifying gene expression biomarkers –15, but Cancer gene expression subtypes are traditionally identified using unsupervised clustering methods such as consensus clustering analysis –21. ThesGaussian mixture models are a powerful class of unsupervised clustering algorithms that can be used to detect multimodally expressed genes –25. A GaHere, we present the hydra framework for identifying clinically relevant expression subtypes and classifying N-of-1 tumor samples using learned models. We provide an overview of the hydra framework, assess performance for detecting differential pathway expression, and apply the framework to better understand expression patterns in high-risk neuroblastoma and other small blue round cell tumors. We apply the learned models trained on publicly available cancer gene expression data to the N-of-1 setting and show that this framework can identify distinct immune and stromal expression signatures that differentiate pediatric cancer samples. Finally, we identify recurrent tumor microenvironment signatures across pediatric cancer types associated with differences in patient outcomes.Traditional parametric models, like the finite mixture model, use a fixed number of parameters (i.e. number of clusters). Over- or underfitting can occur when the parametric model does not reflect the underlying data . Unlike DP) is an infinite dimensional extension of the Dirichlet distribution  where ν is sampled from a Beta distribution, and set π1 to be the length of the first piece. We repeat this process using the remainder of the stick for each πk. The DP is truncated to the number of clusters K, which was shown to accurately approximate the infinite posterior for large K Reviewer's Responses to QuestionsComments to the Authors:Please note here if the review is uploaded as an attachment.Reviewer #1: Review: Hydra: A mixture modelling framework for subtyping pediatric cancer cohorts using multimodal gene expression by Pfeil J et al.Purpose of the research: Develop an improved computational platform to molecularly classify rare pediatric cancers with high accuracy by transcriptomic data that can be implemented in clinical settings.Problem: Current modalities are insufficient to accurately identify novel, subtle cancer specific precision oncogenic targets and networks above thresholds relating to tumour heterogeneity from the microenvironment and the cell-of-origin.What are Pfeil J et al trying to accomplish: Create a highly accurate computational analysis package that can be applied to single samples and easily utilized in clinical settings to facilitate identifying better treatment options for rare pediatric cancers.How is it done today: The authors highlight several of the numerous current approaches for analysing bulk RNAseq data from patient samples and their current limitations. Importantly, they compared their new package to two of the current standards: single-sample gene set enrichment anlaysis (ssGSEA) and gene set variation analysis (GSVA). Based on the defined metrics, their Hydra approach outperformed the other packages overall.What’s new in their approach: The authors elicit a Dirichlet process mixture model to detect multimodally distributed genes and signatures. By subsetting the data via this method, much of the background noise from the tumour microenvironment heterogeneity is reduced. The authors highlight that their approach was previously not feasible computationally for clinical setting but with new developments in Bayesian variational inference it is now possible. Their approach specifically performs better when effect sizes are smaller and fewer genes are differentially expressed which is an ideal platform for rare tumour types such as pediatric cancers.Overall impression/suggestions:- The authors were very thorough in the explanations and rationale for the methodology which is extremely helpful for individuals with limited knowledge in computational biology. The rationale for the study is sound and the approach is very reasonable.- The abstract and author summary are concise, clear and accurately reflect the data presented in the manuscript.- Would the authors elaborate as to why they chose TCGA GBM data to test hydra on initially? This a tumour predominantly found in adults and that has higher mutational frequency than pediatric brain tumours.- The authors analysed 5 pediatric tumour datasets from UCSC treehouse compendium. It is very good to see that hydra and its different components can be applied to all the tumour types. However, it was confusing when performing some analyses on only 1 type of cancer or a few and not others. For continuity, would the authors consider focusing on neuroblastoma for each of the methods presented and use the other cancer types as a more supporting role? E.g. hydra filter/enrich also on neuroblastoma data.- Have the authors considered applying hydra on the more common pediatric tumours such as lymphoma, leukemia or brain tumours? For example medulloblastoma has very well defined molecular subtypes based on transcriptomic data …would hydra better refine these currently well-defined subgroups by Paul Northcott, Michael Taylor, Stefan Pfister, Richard Gilbertson, etc…- The histological data  are underwhelming to support their models. For example, figure 3C would be much better supported if the authors had access to human patient samples to compare the findings of the xCell score based on hydra clusters. Immunohistochemistry for B-cells, CD8+ T-cells and fibroblasts are easy to do if samples are available. For the immune system, the spatial context is especially important when determining the ‘hot’ and ‘cold’ nature of the tumours.- The N-of-1 tumour analysis is very attractive. Rates of pediatric cancers are low which is a hurdle in facilitating better treatments for those affected. Having a more rigid histological analysis of the samples used (figure 4) is needed. General lymphocyte content is in not sufficient for inflammation. The pathologist could supply additional information on neutrophil, macrophages, etc… to better describe the type of inflammation occurring. This level of pathological resolution is needed to better support the findings from Hydra to be utilized on n-of-1 findings.- The hydra filter/enrich analysis of the osteosarcoma data is very interesting. I think these data would be better suited in the main figures not supplemental.- For actionable drug targets, the authors highlight JAK1 signaling in Ewing Sarcoma. The authors highlight how important interpretation of the data are (Mast cell infiltration vs. the tumour) for selecting drug targets regarding the complexities of the tumour microenvironment. Can the authors supply a more concrete cancer specific actionable target to support the filter method of Hydra…something less nuanced?- For S4B and C, these data are impressive and should be moved to main figures.- Why is S7 in the supplemental data? Did the authors assess all cluster combinations and identify only wound healing and translational regulation as important in osteosarcoma…what about cell cycle,etc..General comments:- Line 22: Though many accept that pediatric tumours have fewer mutations relative to adult cancers, this is not true for all pediatric cancers. Additionally, the spectrum of tumours in paediatrics and adults is remarkably different. Would the authors supply some citation for their statement or provide a comparison on mutational frequencies…even focusing on brain tumours comparing peidatrics to adults. Stefan Pfister’s group in Heidelberg have published some data related to this.- It may be my version, but the figures were very blurry and hard to make out details.- Line 473: these findings are not too surprisingly based since the 5 cancers assessed have links back to neural crest cells.- Line 482: should reference S7 not S6.- The method section describing the histological analysis is not sufficient. Please see above for suggestions on what to highlight.- Several graphs in the figures have no labelled axes.Reviewer #2: In the manuscript “Hydra: A mixture modeling framework for subtyping pediatric cancer cohorts using multimodal gene expression signatures“, the authors proposed a novel method for classifying tumor samples based on their expression profiles. The method focuses on multimodally expressed genes (identified through a Dirichlet process mixture model) and its effects are demonstrated in cohorts of pediatric cancers. The method is compared to a standard method in the field, gene set enrichment analysis (GSEA) and its equivalent for N-of-One type of analysis, single-sample GSEA. The method is a non-parametric mixture model and contain three tools named filter, enrich and sweep. The first result is obtained from synthetic gene expression data and demonstrate that hydra outperformed ssGSEA and GSVA . The second one identified 3 subclusters of the MYCN non-amplified neuroblastoma samples and relates these 3 subclusters to features of the tumor microenvironment. The third result illustrate the use of hydra to classify individual samples based on an annotated compendium of expression profiles, with five MYCN non-amplified neuroblastoma samples and the classification provided in the previous section. The fourth result section demonstrate the use of hydra’s functions for unsupervised pathway enrichment on the TARGET osteosarcoma dataset and on a Ewing sarcoma dataset. The final result shows how the hydra analysis on small blue round cell tumors clustered samples based on shared expression profiles and regardless of histology, while TumorMap clustering separates these different tumor types.The method is well described and the results nicely illustrate what it can achieve, on cohorts that are both accessible and of interest to the authors of the manuscript. I have a set of minor comments and recommendation to try to improve the manuscript.Comments:1) I think a discussion of runtime is warranted, especially in comparison with the other methods, and an estimation of how runtime scales with the number of samples in the analysis.2) In the first result with synthetic data, the data is modelled “as a multivariate Gaussian distribution” (l.204), which is the running assumption in Hydra (l. 94). It would be more powerful to model the synthetic data with different assumptions and measure the performance of the method.www.dockerhub.com/jpfeil/hydra failed for me, but the correct URL seems to be https://hub.docker.com/r/jpfeil/hydra. I note that the title of the manuscript indicate that hydra is a method for “subtyping pediatric cancer cohorts”, but the short description of the Docker package describes it as a “clustering pipeline to identify differentially expressed pathways”. I get how those two things are related, but I believe it would be good to clarify in the text .3) l.125: The URL 4) Some sections are very technical. For instance, the lines 111 to 116 are referring to concepts and methods that may not be common knowledge for part of the audience. I leave it to the authors to decide if they can explain it more or prefer to leave it there, assuming that the reader will consult the references cited.5) I would appreciate a guide, on the Github page or even better as part of the manuscript, describing how to use the method on a completely novel cancer type and novel datasets. How to process the data, build the model and run the different tools of Hydra? Some documentation already exists, but could be expanded a bit more if the authors want more users to adopt their method.**********Have all data underlying the figures and results presented in the manuscript been provided?PLOS Computational Biologydata availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.Large-scale datasets should be made available via a public repository as described in the Reviewer #1: YesReviewer #2: Yes**********what does this mean?). If published, this will include your full peer review and any attached files.PLOS authors have the option to publish the peer review history of their article . If published, this will include your full peer review and any attached files.PLOS authors have the option to publish the peer review history of their article  1223-442824 | ploscompbiol.org | @PLOSCompBiolPLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom


It is difficult to determine which patients with stage I and II colorectal cancer are at high risk of recurrence, qualifying them to undergo adjuvant chemotherapy. In this study, we aimed to determine a gene signature using gene expression data that could successfully identify high risk of recurrence among stage I and II colorectal cancer patients. First, a synthetic minority oversampling technique was used to address the problem of imbalanced data due to rare recurrence events. We then applied a sequential workflow of three methods  to identify genes differentially expressed between patients with and without recurrence. To stabilize the prediction algorithm, we repeated the above processes on 10 subsets by bagging the training data set and then used support vector machine methods to construct the prediction models. The final predictions were determined by majority voting. The 10 models, using 51 differentially expressed genes, successfully predicted a high risk of recurrence within 3 years in the training data set, with a sensitivity of 91.18%. For the validation data sets, the sensitivity of the prediction with samples from two other countries was 80.00% and 91.67%. These prediction models can potentially function as a tool to decide if adjuvant chemotherapy should be administered after surgery for patients with stage I and II colorectal cancer. Colorectal cancer (CRC) is one of the most commonly occurring cancers worldwide . In TaiwBased on evidence from a nationwide cohort study in the United States, adjuvant chemotherapy has been more frequently given to younger patients . HoweverAlthough adjuvant chemotherapy is not routinely recommended for stage II patients, according to the Cancer Registry Annual Report 2016) of Taiwan, the rate of surgery with adjuvant chemotherapy in stage II patients reached 53.46% 016 of Ta. Though Microarray gene expression profiling is a widely used tool to determine the prognosis of cancer, including breast cancer , non-smaAn overview of the workflow implemented in this study is shown in https://www.ncbi.nlm.nih.gov/geo/), which were obtained using the Affymetrix HG-U133 Plus2.0 Chip microarray platform. The reasons why we selected these three microarray data sets were that all these data sets were analyzed by the same microarray platform and reported the recurrence status. All raw data as CEL files were normalized with robust multichip averaging using the “affy” package of R software   is used First, to identify differentially expressed genes, three statistical methods , logistic regression, and recursive feature elimination (RFE)) were used. Each statistical method depends on different characteristics of the data, so the genes that pass the thresholds for all three methods are assumed to have a more significant influence on CRC recurrence than other genes. SAM uses a modified t-statistic to evaluate the differential expression of each gene between real data and randomly permutated data . Univarivia a random forest method is a convergence-based algorithm method; hence, the final set of significantly differentially expressed genes would be slightly different each time. Therefore, to get a more stable prediction performance, the ensemble method is used to determine a set of classifiers that make the final prediction  and minority (recurrence) class were adjusted to 102. In the 10 subsets, the mean number of differentially expressed candidate genes that passed the SAM threshold (delta≧0.6) were 13,285, of which 1,417 candidate genes also passed the univariate logistic regression threshold (coefficient≧2.4). Finally, after passing through random forest RFE, the mean number of significantly differentially expressed genes was 11. The total number of unique differentially expressed genes in the 10 subsets was 51.2 score of the voting prediction were 91.18%, 83.33%, and 89.49%, respectively  and Australia (n=290). Similar inclusion criteria and data preprocessing to that of the training data set were applied. The final sample sizes were 55 patients from the USA and 103 from Australia. The Kaplan-Meier survival plots in For each of the 24 drugs, we applied the Wilcoxon rank sum test to determine significantly different drug responses, as the sample size did not fit the normal distribution. Among the 24 drugs, no p-value passed the threshold of 0.05 . For the validation data sets, the sensitivity, specificity, and F2 score in the USA data set (GSE17536) were 77.78%, 35.71%, and 57.38%, respectively, while those in the Australia data set (GSE14333) were 70.37%, 20.51%, and 65.07%, respectively. Due to the smaller sample size of the validation data sets, the log-rank test did not attain a value indicating significance.We repeated the analysis described in this study to analyze the gene expression data from stage II patients only. The sample numbers of the three analyzed data sets dropped to 168, 37, and 66, respectively, which means that around 30% of the samples were removed. Following the same analysis procedure described in this study, 130 differentially expressed genes were identified from the ten subsets. Notably, 20 of the 130 differentially expressed genes identified from stage II patients overlapped with the 51 differentially expressed genes identified from the original analysis of stage I and II patients. This significant overlap  indicated that our algorithm had a robust performance. For the prediction performance, the sensitivity, specificity, and the F® Pathway Analysis (IPA®) software program  (www.proteinatlas.org/pathology), which provides interactive survival plots using publicly available data from The Cancer Genome Atlas   and the nalysis)  were useme Atlas . From thl cancer . GPR55 ine cells . Regardiwith CRC , 38–43. A limitation of this study is that the specificity of the prediction for the validation data sets was found to be slightly low, which seems consistent with other previous studies –49. HoweIn this study, our prediction model was developed based on gene expression features. To date, several prediction algorithms for prognosis and survival outcomes were developed in CRC patients using clinical variables and biochemical markers –54. The https://www.ncbi.nlm.nih.gov/geo/). GSE40967, GSE17536 and GSE14333.Publicly available datasets were analyzed in this study. These data can be found here: Gene Expression Omnibus  and the Center for Biotechnology, National Taiwan University, Taiwan (GTZ300).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.