Population-scale expression profiling studies can provide valuable insights into biological and disease-underlying mechanisms. The availability of phenotypic traits is essential for studying clinical effects. Therefore, missing, incomplete, or inaccurate phenotypic information can make analyses challenging and prevent RNA-seq or other omics data to be reused. A possible solution are predictors that infer clinical or behavioral phenotypic traits from molecular data. While such predictors have been developed based on different omics data types and are being applied in various studies, metabolomics-based surrogates are less commonly used than predictors based on DNA methylation profiles.In this study, we inferred 17 traits, including diabetes status and exposure to lipid medication, using previously trained metabolomic predictors. We evaluated whether these metabolomic surrogates can be used as an alternative to reported information for studying the respective phenotypes using expression profiling data of four population cohorts. For the majority of the 17 traits, the metabolomic surrogates performed similarly to the reported phenotypes in terms of effect sizes, number of significant associations, replication rates, and significantly enriched pathways.The application of metabolomics-derived surrogate outcomes opens new possibilities for reuse of multi-omics data sets. In studies where availability of clinical metadata is limited, missing or incomplete information can be complemented by these surrogates, thereby increasing the size of available data sets. Additionally, the availability of such surrogates could be used to correct for potential biological confounding. In the future, it would be interesting to further investigate the use of molecular predictors across different omics types and cohorts.The online version contains supplementary material available at (10.1186/s12864-022-08771-7). Genome-wide association studies (GWAS) have proven to be valuable in uncovering links between genes and a wide range of phenotypic traits. Such findings have led to the discovery of new disease-related biomarkers and are often the basis for gaining a better understanding of biological processes or disease-mechanisms. Since the introduction of the first GWAS powered by the availability of genome-wide single-nucleotide polymorphism (SNP) profiling, numerous studies have identified thousands of SNP-trait associations . TechnolSuch studies are susceptible to confounding by biological and technical factors that can influence omics profiles and phenotypic traits of interest. However, measured values to correct for such confounding are often not available. As a solution, differences in cell type composition are commonly accounted for using information contained in the DNA methylation profiles themselves, by either reference-based imputation  or referThe value of such predictors is not only evident when complementing missing data to account for technical or biological confounding, but also for using them as outcome variables. These molecular surrogates can be used in association studies in order to link molecular features to clinical phenotypes or exposures. Since identified associations in ome-wide association studies often have only moderate effect sizes, a common approach to detect relevant features are cross-cohort meta-analyses . HoweverAs more and more multi-omics data sets become available, it becomes possible to make use of molecular predictors to infer phenotypic traits from specific omics layers. Blood is a key specimen in clinical diagnostics reflecting on the health state of an individual. While blood metabolomics methods partially overlap with classical clinical diagnostics methods, they can measure a wide range of metabolites and have the potential to play an important role in personalized medicine approaches . RecentlWe here propose the use of metabolomics-derived surrogates in analyses of other omics layers, not only as covariates to account for confounding factors, but also as outcome variables. We investigated whether values derived from molecular predictors represent a viable alternative to measured or reported clinical or phenotypic traits to serve as outcome variables in population-scale gene expression profile association studies. To this end, we applied 17 metabolomic predictors to metabolomics data from four large population cohorts inferring values for phenotypic, exposure, and clinical traits. We performed association studies on corresponding RNA-seq data sets employing either reported/measured or inferred values as outcome variables, and systematically compared the respective results of these analyses. For five of the outcomes, where reported values were not available, we evaluated the performance of the metabolomic surrogates based on results reported in literature.In this study, we performed expression profiling studies to evaluate the performance of 17 surrogate outcomes that are based on molecular predictors. Metabolomics data used to infer surrogate outcome values and RNA-seq data used for the evaluation are part of multi-omics data sets of four large Dutch population cohorts: LifeLines (LL) , Leiden We inferred values for clinical variables by applying molecular predictors to metabolomics data. Recently reported metabolomic predictors trained on up to 22 Dutch population cohorts  were appIn the next step, the 17 metabolomic surrogates were used as outcomes of expression profile association studies. In addition to analyses employing these surrogate outcomes, analyses using measured or reported outcome values were performed. For five outcomes that had limited availability of reported values  only surrogate outcomes were used. In each linear regression model, known biological  and technical  confounding factors were included  bias and inflation were estimated as parameters (mean and standard deviation) of the empirical null distribution using a Bayesian method implemented in the R package bacon [For an initial assessment of the performance of each model, we compared the numbers of significant associations and the effect sizes between outcomes and outcome variable types. Additionally, test statistic t-statistge bacon , indicatr<0.5 were observed for the outcomes age, white blood cells, smoking, and hemoglobin, the latter having the lowest correlation values. We often observed that correlations were lower for the NTR cohort. This could be explained by a technical difference in the metabolomic profiles, with NTR missing glutamine [Since the number of significant associations and average effect sizes do not allow drawing conclusions about the similarity of the association study results employing different types of variables as outcome, we next performed pairwise comparisons of models with different types of outcome variables, i.e., reported vs. surrogate. Figure lutamine .Fig. 3PWhen comparing two alternative outcome variables, a lower or higher number of found significant associations does not necessarily imply that results are better or worse, since the values alone do not indicate if this is due to a reduction or increase of false positive (noise) or true positive findings, respectively. We observed that the expression profile association study results differ when surrogate values differ from reported values. However, we do not know which set of results is correct, as reported values could contain inaccuracies. Under the assumption that true positive findings, as opposed to false positive results, can be replicated in different cohorts , we performed replication studies to determine which outcome variable type is more consistently reflected in the RNA-seq data. We performed leave-one-cohort-out meta-analyses and replication studies for all comparisons (except for hsCRP where only two cohorts were available) using the approach described by van Rooij et al. . For eacTo arrive at a biological interpretation of the association study results, we performed gene-set enrichment analyses (GSEA) using pathways from the Reactome database. GSEA was applied to both individual cohort results and results from a meta-analysis of all four cohorts Fig. . For dirIn order to evaluate the performance of surrogates for which results could not be compared to results based on reported outcomes, we compared significantly enriched pathways data. This allows for inclusion of more cohorts in meta-analyses, even when outcomes of interest were not reported for all cohorts. This approach also increases possibilities to study clinical outcomes by allowing to infer important confounding factors. Especially investigations that rely on reuse of existing data, e.g., in the case of rare disease studies which often also suffer from low sample sizes, will benefit from this approach.In this study, we analyzed RNA-seq and metabolomics data from four large Dutch population cohorts: LifeLines (LL) , Leiden RNA-seq data of all four cohorts was generated by the BBMRI-NL Biobank-based Integrative Omics Study (BIOS) Consortium at the Human Genotyping facility (HugeF) of ErasmusMC, the Netherlands. RNA sample processing and sequencing is described in detail by Zhernakova et al. . Briefly1H-NMR) spectroscopy on the platform of the Nightingale Health Group  [Metabolomics data was generated by the BBMRI-NL Metabolomics Consortium in 2014 as described by van den Akker et al. . BrieflyFinland) .All analyses were implemented in an R v4.0.3  workflowNumeric values were used for all reported clinical variables. In case of categorical variables, they were binarized as follows. For smoking status, “current smoker” was coded as 1, and “former-smoker” and “non-smoker” were coded as 0; for sex, “male” and “female” were coded as 1 and 0, respectively; for lipid medication, “statins” were coded as 1, and “no” and “yes, but no statins” were coded as 0. In order to be able to compare effect sizes in association studies, all clinical variables were standardized to zero-mean and unit-variance .Samples with more than 10% missing values in the RNA-seq data were excluded from the analysis. Additionally, for comparisons of models employing either reported or inferred values as outcome, samples missing reported values were excluded. Subsequently, features (transcripts) missing in more than 10% of the samples were removed from the data set. Number of retained samples and features are given in Additional file The RNA-seq read counts as provided by the BBMRIomics R package were then normalized and transformed based on a previous evaluation of analysis strategies  as folloThe Nightingale Health metabolomics features inquired are the 56 variables selected by van den Akker et al. . OutlierSeventeen logistic regression models trained on up to 22 cohorts, including LL and RS, were applied to the dataset as described in Bizzarri et al. . The surp-values and effect sizes for statistical bias and inflation using the Bayesian method bacon [t-statistic). Additionally, p-values were adjusted for multiple testing using the false discovery rate (FDR) [To determine association of profiles with clinical outcomes, gene-wise linear regression models were fitted using limma . Known pod bacon , which ete (FDR) , 43. ResLeave-one-cohort-out meta-analyses and replication studies in left out cohort were performed as described in .pb)∗|βb| with pb = bacon-corrected p-value and betab = bacon-corrected effect size. The number of permutations for initial estimation of p-values was set to 1×104; the boundary for calculating p-values was set to 1×10−50.Gene-set enrichment analyses (GSEA) were performed using the R/Bioconductor package fgsea v1.16.0  and geneAdditional file 1 TWAS model parameters. This.csv file contains variable names, sample and feature numbers, and covariates per TWAS model and cohort.Additional file 2 Significantly enriched pathways from GSEA. This.html file contains plots showing significantly enriched pathways for each outcome variable. GSEAs are based on meta-analyzed TWAS results.


The availability of massive gene expression data has been challenging in terms of how to cure, process, and extract useful information. Here, we describe the use of entropic measures as discriminating criteria in cancer using the whole data set of gene expression levels. These methods were applied in classifying samples between tumor and normal type for 13 types of tumors with a high success ratio. Using gene expression, ordered by pathways, results in complexity–entropy diagrams. The map allows the clustering of the tumor and normal types samples, with a high success rate for nine of the thirteen, studied cancer types. Further analysis using information distance also shows good discriminating behavior, but, more importantly, allows for discriminating between cancer types. Together, our results allow the classification of tissues without the need to identify relevant genes or impose a particular cancer model. The used procedure can be extended to classification problems beyond the reported results. Entropic magnitudes, as defined in information theory and related areas, have been used in a wide number of areas  beyond cDNA is all about information, so there is no surprise in the use of information entropy to characterize the sequence of nucleotides ,8,9,10 dMore than two decades of experimental development have resulted in the availability of techniques to simultaneously probe thousands of gene expression levels from a single sample ,15,16,17It is known that the identification of subtle patterns in gene expression can be used as the basis for classifying tissues, differentiating cancerous from non-cancerous samples ,34,35,36Entropic measures derive from several sources, starting with information theory  and inclhttps://portal.gdc.cancer.gov/) (accessed on 10 September 2020). RNA-Seq data in the FPKM format are used. The data contain expressions for 60,483 protein-coding, RNA genes, and pseudogenes. Gene expression levels are coded into three classes with a three-value alphabet Gene expression data and their corresponding normal tissues are taken from the TCGA portal (https://reactome.org/download/current/Ensembl2Reactome.txt) (accessed on 5 February 2021) was ordered lexicographically. An ordered sequence of the gene codes was built by parsing the Reactome list and assigning to each entry the corresponding code  [E is the mutual information between the two halves of the bi-infinite string [E vs. h, which is an example of what is called a complexity–entropy map. Complexity–entropy maps show the trade-off between randomness or pattern production as measured by h and pattern recurrence or memory as measured by E; it has been used (under different names) in studies of an extensive set of systems [Consider xity (E) , also knxity (E) , can be =H(L)−HL−, the effe string . It meas=H(L)−HL−, the eff systems ,43.s is the length of the shortest program capable of reproducing the string when run in a Universal Turing Machine [s and q is defined as Finally, information distance between two sequences comes from Kolmogorov complexity theory , and the Machine . A patte mammals .All entropic measures are estimated using the Lempel–Ziv approximation.When finite data are considered, the entropy rate has to be estimated ,45 as thConsider the following factorization of a sequence s, if any factor For example, the sequence The Lempel–Ziv complexity N string is bounded by [In general, unded by (1)C(s)<N, leading to an asymptotic valueN.We used s is the infinite length output from an ergodic source with entropy rate h, thenZiv  proved t strings . For a 1und 10−2 . The Lem h, thenlim supN→und 10−2 ,50.E is estimated using a random shuffle procedure given by [s in non overlapping blocks of length M and performing a random shuffling of the blocks. The shuffling for a given block length M destroys all correlations between symbols for lengths larger than M while keeping the same symbol frequency. E, it is expected to behave in a similar manner [The effective complexity measure given by 5)E(s)=E(s)=E isr manner .s, or the length of the shortest algorithm s when run in a Universal Turing Machine.As already explained, information distance ndomness  K(s)=|s*h is also given byIt is known  that theFrom this result, it follows that :(7)d is Normal tissue exhibits a mean (median) 251) see  while, fStructuring may come from two processes: change in the symbols fractions, in this case, the symbols represent one of three classes for up- and down- and normal-regulated genes, so this process refers to the change in the symbol fraction in each class; on the other hand, structuring can be the results of the ordering of the symbols. Structuring by symbol fraction change usually dominates when one of the symbols becomes dominant in the data.E is mainly the result of symbol rearrangement, as any change coming from symbol fraction changes is canceled. The same can be said for the increase of h values.Tumor samples show a larger number of up- and downregulated genes at the other genes’ expense than the normal samples. A straightforward discrimination procedure was used based on the complexity–entropy map. The Euclidean distance to the mean (median) values of the normal and tumor points was calculated for every sample. The class to which a sample belongs was decided by taking the one with the shortest Euclidean distance from its mean (median) value to the sample point. With such criteria, the success rate of detecting normal tissue was Similar analyses were performed for the rest of the cancer types, and the complexity–entropy map for each is published as h and E values. The second group corresponds to the normal tissues at the lower-left corner with smaller h and E values. The poor discriminating performance of the PRAD, STAD, and THCA samples comes from the fact that their mean values for cancer  and normal (STAD) tissue data can not be included in their corresponding groups.If the complexity–entropy map for the mean value of all cancer types is plotted , two groThe information distance matrix for all samples within a cancer type was computed; we show in From the dendrogram of For the distance matrix, a discriminating procedure was designed based on a majority rule. The information distance to the other tissue samples was considered for each sample, and the nearest neighbor type was used as a majority discriminating criterion. The sample was assigned to the type, tumor, or normal, based on the majority of neighbor types . For eacConsider a neighborhood formed by the eight nearest samples. For the COAD type cancer, if we assigned a value of A second discriminating procedure was also used, where the distance weights each neighbor, and no significant difference was found.The success fraction for all cancer types is reported in Finally, it was studied whether using the distance matrix allows for discriminating between types of cancer. The KIRC and KIRP kidney cancer were chosen for two reasons. On the one hand, they are two types of tumors from the same organ; therefore, discrimination from the genetic data extracted from the sample tissue could make clinical sense. On the other hand, both mean value points are too close to consider discrimination from such analysis in the complexity–entropy map. Expression reference values were calculated using the set of normal tissue values for both the KIRC and KIRP datasets. The expression classes were determined as already explained using these common reference values. The distance matrix was calculated between all samples from both tumors, KIRP and KIRC, and the normal tissues. The distance matrix and dendrogram are shown in This work reports the application of entropic measures that proved useful in analyzing an extensive gene expression data set. The measures seem to allow, at least for the scope of this study, the segmentation of tumor tissue samples from normal tissue samples for various cancer types. The sorting of the data along pathways enhanced the segmentation abilities of the applied techniques. One may ask if this points to features beyond cancer discrimination. Gene expression data studies from pathway analysis have proven that, even in the case where no significant variation of a single gene expression level is involved, studying groups of genes by their functional role identifies the difference between samples of healthy subjects and subjects with some types of diseases or abnormal condition, such as diabetes and smoking epithelial tissues ,54. SuchE. The simultaneous increase of entropy density and E is a fingerprint of increased complexity. As measured by h, the given disorder can still accommodate enough pattern recurrence measured by E. In this sense, the analysis of the tumor samples shows a more complex pathway-ordered gene expression profile. This complexity does not arise mainly by changes in the fraction of samples in each symbol class, although it is clear that up- and downregulated genes increase at the expense of the non-regulated ones. The role of gene expression levels in connection with cancer’s appearance is usually discussed, but what is pointed out here is that this does not describe the whole story. There is no real physically meaningful order for the pathways, so randomization breaks any pattern at the intra-pathway ordering level, leading immediately to the disappearance of any relation between pattern production and pattern recurrence as E drops an order of magnitude.Entropic measures have been used to study the pattern gain at different levels of grammar hierarchy in written language . In our We have proven that, in several cases, and within the scope of the present study, entropy measures allow the discrimination of cancer samples from normal samples by a class analysis of gene expression levels. In both complexity–entropy maps and information distance, classification was possible with a high success rate within a cancer type, above Further experimental work is needed to evaluate if our findings are relevant in practical tumor discrimination, and current work is planned along this line. In addition, the reported results point to expanding the method to explore if it can be used to study different stages in cancer development and the occurring changes along its evolution.The developed procedure can be extended to other data types as its generalization is straightforward.


Colla corii asini , a precious traditional Chinese medicine (TCM). In all brands of CCA, Dong’e Ejiao (DEEJ) is an intangible cultural heritage resource. Seventy-eight CCA samples (including forty DEEJ samples and thirty-eight samples from other different manufacturers) were detected by laser-induced breakdown spectroscopy (LIBS) and near-infrared spectroscopy (NIR). Partial least squares discriminant analysis (PLS-DA) models were built first considering individual techniques separately, and then fusing LIBS and NIR data at low-level. The statistical parameters including classification accuracy, sensitivity, and specificity were calculated to evaluate the PLS-DA model performance. The results demonstrated that two individual techniques show good classification performance, especially the NIR. The PLS-DA model with single NIR spectra pretreated by the multiplicative scatter correction (MSC) method was preferred as excellent discrimination. Though individual spectroscopic data obtained good classification performance. A data fusion strategy was also attempted to merge atomic and molecular information of CCA. Compared to a single data block, data fusion models with SNV and MSC pretreatment exhibited good predictive power with no misclassification. This study may provide a novel perspective to employ a comprehensive analytical approach to brand discrimination of CCA. The synergetic strategy based on LIBS together with NIR offers atomic and molecular information of CCA, which could be exemplary for future research on the rapid discrimination of TCM.A synergetic strategy was proposed to address the critical issue in the brand characterization of  Colla Corii Asini, CCA) made from donkey hide has been widely used as a precious traditional Chinese medicine (TCM) in China for more than 2000 years [Ejiao  spectroscopy and near-infrared (NIR) spectroscopy ,5, liquiSpectroscopic techniques are appealing tools for fast quality control of traditional Chinese medicine (TCM) with no laborious sample preparation. For example, laser-induced breakdown spectroscopy (LIBS) and near-infrared spectroscopy (NIR) have many advantages such as minimal sample pretreatment, fast detection, less destruction, environmentally friendly and cost-effective properties of the analytical procedure, have been uesd in pharmaceuticals, food, and agriculture filed ,14,15. LAnother promising technique is NIR, which has been successful in the quality control of food and drug. As a non-destructive technique, NIR employs measurements of photon energy to obtain qualitative and quantitative information based on the interaction of the sample with NIR radiation ,21. It cThe combination of LIBS and NIR could provide complementary information on unknown species, involving not only the elemental compositions but also the molecular components. Nowadays, the two techniques have been prevailing analytical tools for quality control of TCM with the rapid development of powerful computers and data analysis tools ,25,26. IUsually, the application of LIBS and NIR spectroscopy in classification analysis needs the assistance of chemometrics approaches to construct the discriminant model. Among these approaches, partial least squares discriminant analysis (PLS-DA) is an effective algorithm used to optimize separation between different groups of samples . PLS-DA The main objective of the present study was to classify CCA samples according to their species. CCA samples mainly contain organic and inorganic components including collagen, proteins, amino acids, polysaccharides, trace elements, water, etc. . We focuThe methodology is found to be helpful for the quality control of CCA, and exhibits the promise of addressing similar needs in other precious TCM products in the future.The normalized LIBS spectra of the CCA sample is shown in −1, 6650 cm−1, 8417 cm−1, which are abundant in the molecular structures of amino acids, peptides, proteins, etc. In addition, the relatively slow absorption peak appeared at approximately 5151 cm−1, which may correspond to the frequency of the O–H bond in water molecules. The NIR spectroscopy-based fingerprint contains complex attributes of the CCA samples sufficiently, which are related to brand characterization.The PLS-DA model was built on a single LIBS data block using the selected variables  method offers perfect performance than the others. When the optimal number of LVs set as 10, this PLS-DA model provides correct classification for all samples both in the training and testing sets. The PLS-DA discriminatory information of NIR spectra pretreated by the MSC method is shown in Data fusion is the integration of data blocks from different analytical technologies into a single model ,36. ThouTaking the classification efficiency into account, SNV and MSC were the best data preprocessing methods for discriminating DEEJ from Non-DEEJ, which can eliminate redundant information and manifest differences among samples. When the optimal number of LVs set as 10, the total accuracy values of the PLS-DA model based on low-level data fusion with SNV data pretreatment reach 100% for different class samples both in the training sets and testing sets. The sensitivity values and specificity values of the PLS-DA model are 100% for different class samples both in the training sets and testing sets. The data fusion model also exhibits excellent discrimination performance with MSC data pretreatment. When the optimal number of LVs set as 11, the total accuracy values, sensitivity values, and specificity values all reach 100%, indicating no sample is misclassified.Seventy-eight CCA samples were collected from different manufacturers in China, including forty DEEJ samples and thirty-eight CCA samples from other factories called Non-DEEJ. Sample storage may influence the CCA quality. As a drug, CCA was sealed with a box. Thus, all samples were stored in a cool, dark area before analysis. The detailed information on all CCA samples is listed in The LIBS system used for CCA analysis comprises a Q-switched Nd: YAG laser operating at 1064 nm with a pulses duration of 3–5 ns at a frequency of 2 Hz. Echelle spectrometer  records the spectrum from 193 to 840 nm with an approximate resolution of 0.1 nm. The laser beam was focused on the sample surface by a microscope objective lens system with an 80 mm focal length (Avantes UV-74). The focused spot size on the target surface was about 100 μm in diameter. An optical fiber was connected to the spectrometer with an intensified gated CCD camera (LTB ARYELLE 200 and iStar from Andor Technology) to disperse and record the spectra. A laser pulse energy of 40 mJ, a gate width of 100 μs, and a delay time of 0.6 μs resulted in the maximum signal-to-noise ratio, then being selected for LIBS experiments. All LIBS experiments were performed in ambient air. “LTB-Sophi” software was used to record the emission spectrum of the species and extract analytical information. Four laser shots per location of twelve sampling locations were collected and spectra averaged aiming to limit the relative standard deviation of the peak-intensity ratios.−1 with a resolution of 4 cm−1 at ambient temperature. The spectra was recorded as Log(1/R) using air as a reference.NIR spectra were collected by Antaris Nicolet FT-NIR system . All spectra were measured in integrating sphere diffuse mode. Each spectrum was an average of 64 scans across the wavenumbers from 10,000 to 4000 cmThe original LIBS spectra are often affected by various sources of interference from light scattering, instrument noise, and background . Thus, cThe spectral preprocessing and model calculation steps were performed by the Unscrambler 9.7 software  and homemade routines in Matlab version R2009a with Statistical Toolbox. Partial least squares discriminant analysis (PLS-DA) algorithm was investigated for a single data block or fusion block respectively. To obtain robust multivariate calibration models, 78 samples were split into two sets of training  and testing  sets in a ratio of 2:1 by KennardStone (KS) algorithm. Training set is used for building the model while the testing set is used for assessing the robustness of the model . Before TP and TN indicate the number of true positive samples and true negative samples, respectively. While FP and FN indicate the number of false positive samples and false negative samples. For each class, a positive sample means itself, and a negative sample is the other class. The sensitivity corresponds to the model’s ability to correctly predict the class of the samples, while specificity reflects its ability to incorrectly prediction of samples from other classes [The three evaluation parameters were calculated as follows: classes . PerfectData fusion can produce more accurate information by integrating multiple complementary data sources. Basically, it can be categorized into three levels: low level, mid level, and high level ,38,39. LIn this study, the feasibility of individual and combining information from LIBS and NIR techniques was researched aiming to address brand discriminant of Ejiao. LIBS and NIR spectroscopies bring atomic and molecular information of species. The discrimination results of PLS-DA models showed that individual techniques had good performance at authenticating brands of CCA. Especially, the PLS-DA model based on the single NIR data pretreated with the MSC method achieved a perfect classification effect, and it obtained 100% correct classification of each class for all samples. A data fusion strategy was also investigated for complementary information from two spectroscopic techniques. Data fusion models based on LIBS and NIR with SNV and MSC pretreatment also show good predictive ability. In general, the synergetic strategy of LIBS and NIR spectra demonstrated the possibility of brand discrimination of Ejiao, and it can be used to provide a reliable method for brand protection or geographical authenticity evaluation of TCM.


The advancement of highly multiplexed spatial technologies requires scalable methods that can leverage spatial information. We present MISTy, a flexible, scalable, and explainable machine learning framework for extracting relationships from any spatial omics data, from dozens to thousands of measured markers. MISTy builds multiple views focusing on different spatial or functional contexts to dissect different effects. We evaluated MISTy on in silico and breast cancer datasets measured by imaging mass cytometry and spatial transcriptomics. We estimated structural and functional interactions coming from different spatial contexts in breast cancer and demonstrated how to relate MISTy’s results to clinical features.The online version contains supplementary material available at 10.1186/s13059-022-02663-5. Highly multiplexed, spatially resolved data is becoming available at an increasing pace thanks to recent and ongoing technical developments. In contrast to dissociated single-cell data, this data informs us on the cell-to-cell heterogeneity in tissue slices while conserving the arrangement of cells . TherefoThe proliferation of spatial technologies leads to the generation of large amounts of data. Different technologies allow for measuring different types of molecules with varying resolution, capturing different areas of tissue with diverse numbers of readouts. Immunofluorescence-based methods allow detection of the expression of tens to hundreds of proteins at subcellular resolution –4 and huCurrently, there is a limited number of computational methods available for the analysis of high-resolution spatially resolved data . One groWe introduce here a Multiview Intercellular SpaTial modeling framework (MISTy), an explainable machine learning framework for knowledge extraction and analysis of highly multiplexed, spatially resolved data. MISTy facilitates an in-depth understanding of marker interactions by profiling the intra- and intercellular relationships. MISTy is a flexible framework able to build models to describe the different spatial contexts, that is, the types of relationship among the observed expressions of the markers, such as intracellular regulation or paracrine regulation. For each of these contexts, MISTy builds a component in the model, called a view. MISTy allows for a hypothesis-driven and flexible definition and composition of views that fit the application of interest. The views can also capture functional relationships, such as pathway activities and crosstalk, cell-type-specific relationships, or focus on relations between different anatomical regions. Each MISTy view is considered as a potential source of variability in the measured marker expressions. Each view is then analyzed for its contribution to the total expression of each marker. The measured contribution points to the relevance of a potential source of interactions coming from the different spatial contexts and is estimated from the view-specific models. Our approach is modular, easily parallelizable, and thus scalable to samples with millions of cells and thousands of measured markers.While inspired by other approaches , 26–28 tTherefore, MISTy is not directly comparable to existing related methods. MISTy does not consider the expression of markers or their patterns individually. MISTy takes into account simultaneously the entire expression profile coming from different spatial or functional contexts assumed to explain the overall expression, as described by the modeled views. In principle, MISTy does not require annotation of cells or any other external information to describe the influence of the local niche (immediate neighborhood) or the broader tissue structure. Instead, it is agnostic to potential sources of bias and operates at the level of the available expression profile. MISTy does not assume linear or other fixed types of relationship between individual markers. Instead, it constructs a nonparametric and nonlinear model of the expression of each available marker as a function of the expression of all other markers at the same time  or the expression of other markers or features captured in the available views. Finally, unlike related approaches, MISTy is able to not only estimate the contribution of the available views, but also infer the importance of relations that can explain their contribution.We validated MISTy on in silico data generated by a custom algorithm. We further applied our framework on two different imaging mass cytometry (IMC) datasets consisting of 46 and 720 breast cancer biopsies respectively. On these data sets, we demonstrated how MISTy outperforms available methods by recapitulating previous results and at the same time adding interpretation and new insights. This enabled us to discover intra- and intercellular features in triple negative breast cancer that are associated with clinical outcomes. To our knowledge, this is the first method available to connect spatially resolved single-cell measurements to the clinical outcomes without the use of cell type annotation. Finally, MISTy can extract knowledge about the interactions among signaling pathways and ligands expressed in the microenvironment from different spatial views. We demonstrate this on spatial transcriptomics data of breast cancer. These case studies illustrate the flexibility of MISTy as a framework to define exploratory and hypothesis-driven workflows for the analysis of diverse types of spatial omics data in basic and translational research.MISTy is a late fusion multiview framework for the construction of a domain-specific, explainable model of the expression of markers Fig. : Fig. S1Y]u, i where each column represents a marker (i = 1. . n) and each row is a spatial location (u = 1. . L). Yi., is the vector made by all observations of the marker i. MISTy models its expression asFormally, we consider a matrix [Fv are models constructed by a machine learning algorithm for each view v. G are domain-specific functions that transform the data to generate informative variables (features) from the expression Y at the corresponding spatial localization X. Optionally, G can depend on other specific properties T, such as prior knowledge expressed as annotated functions, regions, or cell types. The G functions can be used to generate alternative views that can be inputs to the model function F. Finally, α are the late fusion parameters of the meta-model that balances the contribution of each view to the prediction.where v of additional, intercellular views and separate the effect of each view for each marker on the improvement in the predictive performance of the multiview model. We use the improvement in the predictive performance of the models as a proxy to estimate their potential as sources of interactions that can be further explored by extracting feature importances, as outlined in the following. The contribution of each view is captured by the late fusion parameters α of the meta-model. The intercept on the other hand captures (implicitly) the environmental effects on the mean expression of the targets, specific to the analyzed slide. For determining the contribution of the views, the fusion parameters (except for the intercept) are normalized such that they sum up to one MISTy always models a fixed, intraview l itself . By desiα parameters of the meta-model after training the view-specific models independently, by regularized linear regression (ridge regression), to address potential issues of multicollinearity of the view-specific model predictions. The regularization parameter is determined automatically [The above model is trained in two steps. First, the models for each view are trained independently. Second, we estimate atically . The perF with any algorithm that fulfills two requirements. First, the algorithm should construct ensemble models, with constituents trained on a bootstrap sample (bag) from the data. Second, they should be or consist of explainable models. The first criterion guarantees the unbiased use of the measurements in both steps of model training. The predictions of the constituents of an ensemble model can be made on portions of the data (out-of-bag) that were not used for their training. The second criterion means that a global explanation of the model or the feature importances can be obtained post hoc from the trained models. As proof of principle, in the implementation used in this manuscript, we consider F to be Random Forest [In terms of the choice of algorithm for training models, MISTy is a general framework and can construct models for the functions m Forest  with 100At the first level, the meta-model can be interpreted in two different ways. First, to answer the question of how much the intercellular views improve the prediction of the expression in addition to the intracellular view. This can be achieved by comparing the predictive performance of a single intracellular view vs all views combined in a meta-model. Second, by comparing the values of the fusion parameters, we can investigate how much the individual views contribute to explaining the marker expression that led to the aforementioned improvement in predictive performance Fig. A.At a second level, given this information, we can further analyze the feature importances. For each target marker, we can inspect each view-specific model and analyze how important is the contribution of each marker in that view to the prediction of the expression of the target marker Fig. B. Thus, The interpretation of the estimated relationships (interactions) is dependent on the view composition and the biological contexts of the available markers. As we show in this paper, MISTy can capture (i) a structural relationship, such as the spatial organization of cell types based on cell type identities or the expression of cells based on cell type markers, and (ii) a functional relationship between the markers, such as aspects of regulatory programs or communication-driven interactions. MISTy is designed as a method for efficient data exploration and robust hypothesis generation.We first assess the ability of MISTy to recover purely structural relationships decoupled from the influence of functional relationships. To this end, we generated three in silico tissues with specified spatial interactions between four cell types . The numFor each spatial pattern generated, we simulated 100 gene expression markers to create a synthetic dataset . Of the This model only considers cell type preferences in the immediate neighborhood. Accordingly, we created view compositions in the MISTy workflows consisting of intraview and juxtaview only. The juxtaview threshold was set to the 75th percentile of all Euclidean distances between first neighbors to simulate potential errors of determining the correct threshold when applied to real data.We considered two types of workflows: (i) The first workflow uses only information about the cell type identity and focuses on reconstructing directly the cell type composition of the tissue. (ii) The second workflow uses only information about the expression of the cell type markers and focuses on reconstructing the composition of the tissue based on the interaction of gene markers without the information about the actual cell types.For the first workflow, the identity of the cells is captured in the intraview by one-hot encoding. In particular, each cell is described by a vector of length of the total number of cell types (4), where all values are equal to zero except for the value of the variable representing the type that the cell belongs to, which is set to one. The juxtaview then captures the distribution  of the different cell types in the immediate neighborhood of each cell. For the second workflow, the intraview for each cell is represented by the expression of the 100 marker genes. The juxtaview in this workflow captures the total expression of all marker genes in the immediate neighborhood of each cell.By modeling the interactions coming from the intraview, we would capture trivial results of prediction of identity by exclusion or co-occurrence of a large number of unique cell-type markers. While the target expressions to be modeled remain in the intraview, to avoid the aforementioned issues and to allow for modeling of self-preferences, we excluded the intraview-specific model from the meta-model in these workflows. As a result, the baseline to compare the multiview model is a model with an intercept only, i.e., a model that always predicts the mean value of the target variable.When we applied the first workflow, since the structure of Tissue 1 is random, MISTy did not capture any information Fig. B. For TiWhen we applied the second workflow to Tissue 3, we obtained the estimated interactions of the markers with high importance in the immediate neighborhood as captured by the juxtaview Fig. D. For eaIn summary, both workflows converge on the same results—interaction between cell types 1 and 3—yet the second workflow provides a much more detailed view of the individual markers involved with the caveat of added complexity. With the two workflows, we demonstrated that MISTy is able to reconstruct the structural relationships based on annotated cell types and by the expression of cell type marker genes independently.Of note, in a workflow similar to the first one several related approaches, for example univariate spatial pattern identification and immediate neighborhood analysis methods, would be able to capture the self-preference structure of Tissue 2. Only the latter group of methods would be able to capture the true structure of Tissue 3. In a workflow similar to the second one, only immediate neighborhood analysis methods would be able to detect marker interactions. Some of them would require preprocessing of the data (such as clustering) before they can be applied. In the following, we introduce an in silico experiment that these groups of methods cannot be applied to.We next assessed the performance of MISTy to reconstruct functional relationships in in silico intra- and intercellular interaction networks, decoupled from the influence of structural relationships. To estimate the robustness of MISTy to infer mechanistic molecular interactions, we created a tissue simulator that can mimic the interactions of different cell types through ligand binding and subsequent signaling events Fig. ” and simWe compared two scenarios, one with no information on the cell types: in this case all the measured cells are treated equivalently; and another scenario where cell type information is considered: in this case a MISTy model is built for each cell type. The MISTy workflow consists of two views, intracellular view and broader tissue structure view (paraview). The intraview for each cell is represented by the expression of the molecular species. The paraview captures the weighted expression of all molecular species in the broader tissue structure with radius of significance of 10.Overall, the intraview alone . Further, the method also outperformed random classifier with respect to the AUPRC: the obtained AUPRC for the four cell types ranged between 0.581 and 0.737 for the intraview  and between 0.022 and 0.053 for the paraview . The low AUPRC baseline is due to the sparsity of true intercellular interactions in contrast to the total number of interactions between the cells. The sparsity of these interactions, which is also inherent in real biological systems, adds high complexity to the task of reconstruction of the direct connections. In summary, MISTy is able to reliably extract interactions in the in silico case study.MISTy derives an importance score for each pair of markers see “”. Using In particular, MISTy accurately captured the downstream intracellular signaling cascades of receptors Fig. E, left. With these workflows, we demonstrated the extent MISTy is able to reconstruct the mechanistic relationships in general and focused on a cell type of interest in a complex and fully observable. These results also outline the limitations of the approach, such as those caused by the presence of confounders and indirect interactions, whose effect becomes more prominent when reconstructing mechanistic intercellular relationships.Unfortunately, the performance cannot be directly compared to other related approaches, due to their limitation to infer interactions without additional sources of information. The comparison with the most related approach, SVCA, is limited and at best only qualitative at the level of estimated contributions of the fixed views provided by SVCA. SVCA does not provide information about the potential interactions that explain the estimated contributions. Note also that the computational resources needed to construct models even on the in silico data by SVCA are orders of magnitude larger than those needed by MISTy. For a single layout with 4000 locations, on a standard configuration using 4 processor cores, SVCA took 24 h of computation time and 2.5 GB of memory, while MISTy took 18 s of computation time (4800 fold decrease) and 900 MB of memory (2.5 fold decrease).As a first real-data case study, we applied MISTy to an Imaging Mass Cytometry dataset consisting of 46 samples of breast cancer across three tumor grades coming from 26 patients, with measurements of 26 protein markers . We procWe aggregated the MISTy results from all samples and we found that the multiview model resulted in significant improvements in the absolute value of variance explained of up to 20.1% over using the intraview alone, which accounted for an average of 31.8% of overall variance explained across all markers  of each sample. We defined the R2 signature of the MISTy results for each sample by concatenating the estimated values of the variance explained using only the intraview, the variance explained by the multiview model, and the gain in variance explained for each marker. Note that the signature relates only to the results produced by MISTy for each sample and can capture different aspects of them. In this case of the R2 signature, the performance achieved by MISTy per target for each sample. These signatures are not related to a signature composed of biological markers for the samples and thus does not provide any insights into specific marker relationships. Here, we use the R2 signature representation to group the samples by similarity of the results. The use of R2 signature allows us to compare the results of MISTy to SVCA as reported in the manuscript describing SVCA.Subsequently, we analyzed our results by the spatial variance signature (RThe maximum length of the signature vector for each sample in this dataset is 78 (26 markers × 3 measures) dimensional, when using the information for all markers. From our signature vectors and in the following analyses, we removed the results of the performance of the markers that have less than 2% of gain in variance explained. This resulted in signature vectors of length 27 (9 markers × 3 measures).2 signature, we identified a weak but visible structure in the samples driven by the tumor-grade and clinical subtypes, which is consistent with the findings of SVCA  of the RVCA Fig. E. The twsis Fig. E, we obsSince the output of SVCA only includes fractional contributions of fixed views, comparison beyond this point is not possible. As shown in the following, MISTy significantly extends the scope of possible analyses to be performed on the data.2 signature allows us to analyze the differences between the samples based on predictive performance only, more insights into the relationships between the markers can be obtained by a more detailed signature at the level of the estimated importances. The importance signature is generated by concatenating the estimated importance for each predictor-target marker pair from all views. The aggregated importances are weighted by the estimated relevance of the results , the importance of the markers that account for this separation is more uniform and different from the components of the RBy grouping the samples by tumor grade, we further analyzed the robust intercellular features of tumor samples. Since only a small number of samples came from grade 2 tumors, we considered only grade 1 and grade 3 tumor samples. In grade 1 samples, we observed the highest gain of variance explained for markers Cytokeratin 7 (4.92% ± 4.43), SMA (3.96% ± 3.09), and CREB (3.8% ± 2.31). In grade 3 samples, we observed the highest gain of variance explained for pS6 (6.95% ± 5.09), SMA (4.43% ± 3.39), and CREB (4.26% ± 3.52).We further compared the aggregated results by contrasting the important interactions between the same views intragroup and intergroup. Due to the higher overall contribution of the paraview compared to the juxtaview Fig. B, we anaThe loss of signaling during tumor progression is apparent when comparing the results view by view for the different tumor grades. The intergroup and view focused contrasts outline the interactions that were estimated as important in grade 1 samples and not important in grade 3 samples. While we observed a loss of a number of important interactions from the intraview, the loss of important interactions from the paraview is higher Fig. C, D.To highlight the ability to associate MISTy results with clinically relevant features, we analyzed a breast cancer imaging mass cytometry dataset with outcome data, based on 415 samples from 352 patients  and the distribution of clinical subtypes for grade 3 samples is more balanced , with the HR− group containing mostly triple negative subtype (79% out of 88 samples) and in the HR+ group 58,7% out of 97 samples are HR+HER2−). We next asked whether there are specific predictor-target interactions with high importance that could be linked to survival overall and for the different clinical subtypes. The performance MISTy achieved and the view contributions for the group 3 samples specifically are shown in Additional file n = 8), the estimated importance of 24 predictor-target pairs  is significantly correlated to the overall survival (p < 0.05). In the group of HR+HER2− samples (n = 20), the estimated importance of 51 predictor-target pairs  is significantly correlated to the overall survival and in the group of HR-HER2+ samples (n = 11), the estimated importance of 39 predictor-target pairs  is significantly correlated to the overall survival. Importantly, we recover many interactions, without the need of single-cell annotation, that were previously shown to be linked to poor prognosis. For example, pan-cytokeratin and ER as one of the strongest correlations.To associate MISTy results with clinical outcome, we calculated the Spearman rank correlation coefficients between the estimated importance of target-marker pairs to the overall survivability in months, selecting only pairs that contain at least 30% positive importance values. Samples from patients with multiple samples were treated as independent. Next, we performed the analysis accounting for clinical subtypes by running analysis on those samples independently. In the group of HR+HER2+ samples (n = 26), since currently no biomarkers are available that could be linked to outcome, where the estimated importance of 64 predictor-target pairs  is significantly correlated to the overall survival. We picked from the top predictor-target pairs correlated with overall survival for each view as an example for further analysis, but we provide all results for further experimental validation , with improvements of up to 24% compared to the intraview model in the case of the estrogen and hypoxia pathways . These results confirmed that MISTy models are able to extract informative spatial relationships between markers in tissue samples where spatial organization is expected.The multiview model improved significantly the variance explained of 12 of the 14 pathway activities  cellular resolution. The results show that we were not only able to recapitulate results from the literature without prior-knowledge-based cell type annotation, but to also generate new hypotheses. Our results show that the information that is available from the expression of markers in the broader tissue structure is often more important than their expression in the local cellular niche. Of note, this result, which is biologically intuitive, could not be found with previous methods that do not distinguish between para- and juxtaview. This highlights that not only cellular niches but also the tissue structure has a direct impact on cellular states and should be included in the “microenvironment” definition. Furthermore, we show how MISTy finds interactions that are associated with clinical features.Finally, we applied MISTy on a spatial transcriptomics data set measured with 10x Visium. Here, thousands of transcripts are measured in spots containing several cells. Given the richness of the data, we were able to go a step further and consider the analysis of functional features, in the form of pathway activities that were inferred from the data. In particular, we showed the crosstalk between pathways and the ligand-pathway interactions in the context of the broader tissue structure in breast cancer. Our results showed that MISTy in combination with functional transcriptomics tools and prior knowledge can be used in spatial transcriptomics to uncover coordinated functions that are maintained in niches of the tissue. Moreover, the explanatory component of the multiview model provides relevant predictors that could become the base of mechanistic models.Although the interactions extracted by MISTy cannot be considered directly as causal, they can facilitate the downstream analysis of biological systems at the tissue level in several directions: (i) to predict the behavior of systems under perturbations, by using the MISTy model to generate marker expressions based on the new conditions; (ii) to guide the reconstruction of multicellular causal signaling networks, using databases to identify mechanisms giving rise to the extracted interactions; and subsequently (iii) to construct mechanistic models of the dynamical behavior of the system constrained by the extracted explanations.The work we presented lays the foundation for further exploration of MISTy in several directions. One direction is to address the scalability of MISTy to millions of cells and thousands of markers per sample, which is beyond what the available technologies can offer, but is likely to come in the near future. To do this, we are exploring approximate but accurate methods to replace the computationally expensive step of generating views where the pairwise distances between all cells need to be calculated. Another direction is the exploration of the performance that can be achieved by MISTy with different ensemble approaches using various types of explainable constituent models. Furthermore, MISTy can be used to generate more specific views. In particular, views that capture the spatial expression of specific cell types, so that we can dissect the spatial interactions between different cell types, or views that focus on regions of the tissue, for example, healthy vs pathological, where we would model the interactions between the functionally different regions. Of special interest is also the specialization of MISTy workflows that focus on the analysis of ligand-receptor interactions while taking into account the spatial context. To this end, we look towards combining MISTy with complementary tools, such as GCGN , MESSI , cell2cehttps://saezlab.github.io/mistyR/) is fully documented and freely available from GitHub, Bioconductor, and as a Docker image.In summary, we believe that MISTy is a valuable tool to analyze spatially resolved data, adaptable to multiple data modalities and biological contexts, that will also evolve as experimental techniques improve. An implementation of MISTy as an R package named mistyR . Each of these cell types has a distinct set of receptors expressed and distinct intracellular wiring  is the concentration of species s ∈ S at the grid point  at time t. The diffusion is homogenous across the image, the diffusion coefficient of species s is ds. Only ligands are diffusing, and other intracellular molecules cannot leave the cell.This equation describes the diffusion, the production/activation, and the degradation of the species. We made the following assumptions: Pi = αi, ctXi for i ∈ {L1, L2, L3, L4, L5} and ct ∈ CT, the αi, ct coefficient defines which cell type produces which ligands and how strongly the production depends on the activity of the production node.The production term includes the generation of ligands and the activation of intracellular proteins and receptors. Production depends on the cell types and the activity of the production node: the ligand production depends linearly on the nodes above them  = γscs, where γs is a constant degradation coefficient.Degradation is proportional to the concentration of ligands, intracellular proteins, and ECM, The above model was simulated from a randomized initial condition, and the activity distribution . We then quantified the performance of MISTy for the task of reconstruction of intra- and intercellular networks from the true and the extracted interaction matrices.We aggregated the interactions from the mechanistic model for the different cell types in joint binary matrices of directed ground truth interactions for the different views. To compare the matrices to the importance matrices from the output of MISTy, we transformed the joint matrices into undirected matrices The first imaging mass cytometry dataset consists of 46 samples from 26 breast cancer patients with varying disease grades . The oriThe second imaging mass cytometry dataset consists of 720 samples from 352 breast cancer patients from two cohorts, with long-term survival data available for 281 of those patients . The sam+, PR−, HER2+, annotated with ductal carcinoma in situ, lobular carcinoma in situ, and invasive carcinoma. The mean sequencing depths were reported to be 149,800 and 137,262 reads per spot for a total of 3813 and 4015 spots per section respectively. The median UMI counts per spot were reported as 17,531 and 16,501, and the median genes per spot as 5394 and 5100 respectively. The raw data was preprocessed and count matrices were generated with spaceranger-1.0.0. Individual count matrices were normalized with sctransform implemented in Seurat 3.1.2 [The data and sample information were obtained from 10x Genomics . The datat 3.1.2 . For eacat 3.1.2  all protD = [Xn × s)( Yn × k) of spatially resolved highly multiplexed measurements of a sample, where n is the number of measured units  available in the sample, s is the number of spatial dimensions in the geometry matrix X, and kis the number of measured markers in the expression matrix Y.We consider a dataset Ncrepresents the set of neighboring cells of cell c. The neighboring cells for each cell can be determined either during image segmentation, for example by setting a threshold of membrane-to-membrane distance, or, as in the case for the application of MISTy on IMC data, by post hoc neighborhood estimation. For the application of MISTy on IMC data, the neighborhood of each cell in a sample was estimated by constructing a cell graph by 2D Delaunay triangulation followed by removal of edges with length larger than the 25th percentile of all pairwise cell distances across all samples, which corresponded to 11.236 microns from the cell centroid.The juxtaview was generated by summing the expressions of its direct neighboring cells, i.e., w is a weighing function, dcjis the Euclidean distance between cells c and j, calculated from matrix X, l is a parameter controlling the shape of the weighting function w, and z is a parameter controlling the zone of indifference. A juxtaview and a paraview with no zone of indifference will both contain the expression of the markers in the neighboring cells. Considering a zone of indifference larger than the immediate neighborhood would ensure that only the juxtaview will capture the immediate neighborhood, while the paraview will capture only the broader tissue structure excluding the immediate neighborhood.The paraview was generated by weighted aggregation of the expressions of all cells (patches) from the sample w controls the contribution of the expression coming from the broader tissue structure as a function of the distance and a parameter l that captures the radius around the cell where we consider the cells in the broader tissue structure to be significantly contributing to explaining the expression in the cell. Examples of weighting functions in MISTy are the families of radial basis w = 1(d ≤ l).We can assume the cells that are closer affect the expression within the cell more than cells that are farther away to various degrees. The weighing function l. For each IMC sample, we constructed models for each marker, with parameter l ∈ {25, 50, 100, 200, 400}. This corresponds to an effective radius of influence of 25 to 400 pixels or micrometers. The mean values of the parameter l across all samples for all markers are shown in Additional file l ∈ {2, 5, 10}, corresponding to a radius of influence of up to 10 spots. Then, for each marker, we selected the value for l such that the estimated improvement in predictive performance by using the multiview model in contrast to the intraview model is maximized. For each model (Random Forest), we estimate the predictive performance by measuring the variance explained on out-of-bag samples.For the application of MISTy to both IMC and spatial transcriptomics data, we considered the family of radial basis functions for weighting and optimized the value for the parameter αvin Eq. ( for each target marker k and the importances Ikjv) in the Ikjv)(extracted from a Random Forest model (used for the current instance of MISTy) represent the amount of variance reduction in the target expression, the MISTy interaction importances correspond to the standardized value (by mean pkv)( of the statistic under the null hypothesis of zero contribution of the fusion coefficient for view v for target kin the linear meta-model.Since the importances MISTy is conceived to be a framework applicable to any type of omics data. The performance measures and the estimated relationships are independent from the properties of the variables used to describe the data. The machine learning models (Random Forest) trained on the specific views are invariant to the scale of the predictor or the target variables. The measure of the performance of the model (variance explained) is also chosen such that issues with scale are avoided during training and interpretation.MISTy infers the interactions between the variables by the proxy task of predicting the expression, abundance, activity, or any other quantity of the target variables and estimating the importance of each of the predictor variables for this task. The estimated relationships/importances are related to the amount of reduction of variance and not to absolute values. The importance derived from variance reduction can be generalized to any measure of impurity or values extracted by other feature importance estimation methods, given the model constituents of MISTy. Since the MISTy importances are standardized, importances from multiple samples can then be aggregated by simple averaging, while their interpretation remains the same.A of estimated predictor-target interactions to an undirected graph adjacency matrix as Ap = A + AT. We then extract the community structure from the graph using the Louvain algorithm [For views that contain the same set of predictors as targets, we also identified the communities of interactions from the estimated importances. For this, we transformed the square matrix lgorithm , a commoTo evaluate the performance of MISTy models in samples with no spatial organization, we generated random samples for both the IMC and spatial transcriptomics data and ran the same pipeline as for the original data. We permuted the coordinates of each spatial unit (cell or spot) for each slide ten times for the IMC data and five times for the spatial transcriptomics data. Results were grouped and compared to the ones obtained in the slides with the original spot layout.import_omnipath_annotations function and regular expressions to filter annotations associated with the PROGENy pathway of interest. Only ligands with a paraview importance ≥ 2 were considered in this annotation  as a byproduct of pathway activation or (2) as a potential activator of a pathway. A ligand was considered as a byproduct of a PROGENy pathway if it was part of its top 1000 footprint genes. A ligand was considered as a putative activator of the pathway it predicted if at least one of its possible receptors could be assigned to it. We used OmnipathR to annotate each receptor using the  as a bypAdditional file 1. Supplementary Figures S1 to S9 and supplementary Tables S1 and S2.Additional file 2.


Untargeted metabolomics by mass spectrometry is the method of choice for unbiased analysis of molecules in complex samples of biological, clinical or environmental relevance. The exceptional versatility and sensitivity of modern high-resolution instruments allows profiling of thousands of known and unknown molecules in parallel. Inter-batch differences constitute a common and unresolved problem in untargeted metabolomics, and hinder the analysis of multi-batch studies or the intercomparison of experiments.We present a new method, Regularized Adversarial Learning Preserving Similarity (RALPS), for the normalization of multi-batch untargeted metabolomics data. RALPS builds on deep adversarial learning with a three-term loss function that mitigates batch effects while preserving biological identity, spectral properties and coefficients of variation. Using two large metabolomics datasets, we showcase the superior performance of RALPS as compared with six state-of-the-art methods for batch correction. Further, we demonstrate that RALPS scales well, is robust, deals with missing values and can handle different experimental designs.https://github.com/zamboni-lab/RALPS.Bioinformatics online. Metabolomics is the method of choice for chemical characterization of biological, clinical and environmental samples , for untargeted metabolomics that efficiently addresses all the aforementioned problems. RALPS builds on adversarial learning but implements a novel three-term loss function that suppresses batch effects while preserving biological information. Using several test sets, we show that RALPS outperforms state-of-the-art methods in terms of performance, scalability, usability and robustness.Lg) and batch discrimination loss (Ld) (rg) to the autoencoder loss function to reward tight clustering of reference samples in the embedded space. To evaluate clustering, RALPS flattens the multidimensional space by uniform manifold approximation and projection (UMAP) (Lv) to encourage a decrease in batch variation coefficients (VCs). This addition was motivated by two observations. First, many normalization methods tend to inflate the VCs of replicate measurements. Second, deep-learning models with encoder–decoder architectures are especially prone to producing outliers.RALPS was inspired by NormAE: it uses an autoencoder and a classifier to mitigate batch effects, but the loss function was extended with two additional terms beyond the reconstruction loss (oss (Ld) . First, n (UMAP) . EventuaFurthermore, we included several additional features to improve scalability, usability and robustness. First, RALPS adopts a flexible network architecture, in which the number of neurons in the model layers is automatically adjusted based on a principal component analysis. By default, RALPS sets the number of neurons equal to the number of principal components needed to describe at least 90% of the dataset variance. This modification eliminates many hyperparameters from the model and simplifies parameter optimization. Second, we introduced a randomized hyperparameter grid search and model selection logic, both of which enable finding multiple parameter sets that deliver top normalization results in an automated way. Third, we introduced input validation and a mechanism for early stopping to avoid collapsed normalization solutions that could arise because of inconsistent parametrization or increasing classifier loss. Details are provided in the Information on reference samples is provided by the user to indicate all sample groups that are supposed to be similar. These groups can include biological replicates, technical replicates, pooled study samples, spike-ins and dilution series. There is no formal limit on the number of reference groups, and although sample groups should span multiple batches, the same reference samples do not need to be present in all batches. This flexibility builds considerable freedom into the experimental design, as discussed below.13C-labeled Escherichia coli extract at different dilutions. The samples were aliquoted, frozen and analyzed seven times over the span of about 2 months with time gaps of up to 3 weeks. Each of the seven batches was acquired and processed independently. To focus on the subset of m/z peaks that possibly relate to metabolites, we selected features with m/z values that were matched to deprotonated compounds listed by the Human Metabolome Database . Upon intersecting the putative peak lists obtained from each batch and filtering low-abundance features, we obtained a data table with intensities for 170 putative deprotonated metabolites and 2856 files, divided in seven batches.We faced the problem of finding suitable multi-batch datasets for testing and comparing normalization methods. Because such datasets are rare and often associated with clinical studies that preclude full disclosure and publication . Roughlyr = 0.4, pointing to strong biases. Below, we use this dataset to benchmark RALPS against state-of-the-art approaches.To visualize the extent of batch effects, we plotted the UMAP projections for all samples . We obseAs a first demonstration, we evaluated RALPS on a multi-batch benchmarking dataset. Initially, all sample labels were included as reference groups to maximize information available for training. We ran RALPS with default parameters and a randomized grid search of size 50. The best normalization solution selected by the model selection logic was compared to the initial dataset. First, we assessed the normalization effect of our method by highlighting batch labels on the UMAP embeddings plot . We obseThe test described above builds on a rather artificial scenario in which all samples are present in every batch. In practice, however, only a small subset of samples in each batch are repetitions of samples of different batches and can be used to correct for inter-batch effects. The most common scenario is inclusion of one or two reference samples in all batches. If a single sample is available, a frequent option is to spike in standards at a relevant concentration to ensure correct recovery of significant differences, including MS measurement and data processing. To mimic this realistic scenario, we attempted to normalize the benchmarking dataset with RALPS, using only a few reference sample groups each time. Different sets of groups were tested. For instance, λv = 0 to loosen the constraints on the joint loss function optimized during training  keeping the chronological acquisition order. On each iteration, we removed the latest batch and applied RALPS to normalize the remaining data. We observed that the mean batch VC was generally constant and comparable to the full data case . We thenNext, we tested the robustness to randomly distributed missing values. A user-defined parameter (min_relevant_intensity) instructs RALPS on the lowest value to consider. The default is 1000 counts, and missing values are replaced with this minimum value. In our test, we replaced 0%, 5%, 10%, 15%, 20% and 30% randomly picked values in the data matrix with the default minimum value and then trained RALPS. We observed a decreasing trend in cross-correlation of replicates as the fraction of missing values went up . HoweverFinally, we verified how the number of neurons in model architectures affects normalization results. By default, RALPS uses as many neurons as number of principal components necessary to explain at least 90% of variance. We tested multiple values ranging from 99% to 70% variance. We did not observe any clear trends in any of four metrics used to evaluate and compare methods. The mean cross-correlation of replicates stayed above 0.98 in all cases .et al., batches instead were associated with cultivation and sampling of samples over the span of almost a year, whereas MS analysis was done sequentially with all samples. We demonstrated that RALPS outperformed other state-of-the-art methods on several key metrics. RALPS offers additional features, such as adaptive network architectures, embedded hyperparameter optimization, automated model selection and input validation. Together, these features convey flexibility, scalability, usability and robustness as confirmed by testing with different configurations of reference samples and in ablation experiments.We introduce RALPS, a novel batch normalization method based on regularized adversarial learning for untargeted metabolomics data. In this work, we demonstrated its performance on two representative datasets with thousands of samples or spectral features. The benchmarking dataset was generated to test the algorithm on MS data produced over several months. In the case of the cancer cell line data by Cherkaoui Lv mitigated but did not abolish the problem.Historically, the loss function with three terms embedded in RALPS evolved from multiple tests, we performed with several datasets, some of which are not described here. The classification term is the component that drives the removal of batch effects. However, a deeper analysis of the resulting normalized data revealed novel problems that prompted us to introduce additional terms. The observed increase in VCs for supposedly identical samples (e.g. replicates) upon normalization is common to most methods and is a particularly acute problem for NormAE. To our best knowledge, this issue has not previously been acknowledged or addressed. We offer two hypothetical explanations for this gap. The first is the use of mean absolute error for the reconstruction loss, which is sensitive to outliers. The second is that generally increasing the noise, and thus the VC, makes it more difficult for the classifier to separate batches. Hence, there is an apparent beneficial effect on batch discrimination, but detrimental side effects in downstream analyses. Introducing the variation loss A closer investigation of the RALPS results revealed that the increase in VC arose from a small set of samples . Filtering these samples by outlier detection (explained below) reversed the increase in VCs for RALPS and EigenMS, but not for NormAE, ComBat, WaveICA and PQN+POW , suggestrg was the second novel addition to the adversarial training and the key to preserving similarity of supposedly equal samples. Although RALPS relies on clustering of reference samples in the embedded space to assess grouping, alternative approaches could be considered. For example, rg could be calculated from distances in the latent space. Distance-based metrics would carry several hypothetical advantages and pitfalls. Owing to the fast computation, training would be tangibly shorter than with clustering. We expect that distance-based metrics would perform better with data characterized by subtle batch effects in which clustering fails to separate reference samples. Even in the case of all replicates falling into a single cluster , minimizing distances between all pairs of replicates would still have a regularization effect, whereas clustering would not. Among the potential drawbacks, we would expect an increased sensitivity to single outliers. Alternative paradigms for similarity preservation should be tested in the future. One additional aspect that could be optimized is the clustering algorithm. RALPS includes six methods, and HDBSCAN was chosen as the default based on performance, robustness and speed  can be encoded via groups in the batch information file. We encourage researchers from other omics fields to challenge RALPS with their own data and drive its further development by reporting experience and issues in the repository.btad096_Supplementary_DataClick here for additional data file.


In this paper, given data with high-dimensional features, we study this problem of how to calculate the similarity between two samples by considering feature interaction network, where a feature interaction network represents the relationship between features. This is different from some traditional methods, those of which learn similarities based on a sample network that represents the relationship between samples. Therefore, we propose a novel network-based similarity metric for computing the similarity between samples, which incorporates the knowledge of feature interaction network, in order to overcome the data sparseness problem. Our similarity metric uses a new Feature Alignment Similarity measure, which does not directly compute the similarities among samples, but projects each sample into a feature interaction network and measures the similarities between two samples using the similarities between the vertices of the samples in the network. As such, when two samples do not share any common features, they are likely to have higher similarity values when their features share the similar network regions. For ensuring that the metric is useful in a real-world application, we apply our metric to discover subtypes in tumor mutational data by incorporating the information of the gene interaction network. Our experimental results from using synthetic data and real-world tumor mutational data show that our approach outperforms the top competitors in cancer subtype discovery. Furthermore, our approach can identify cancer subtypes that cannot be detected by other clustering algorithms in real cancer data. I.Clustering is a key task in data mining, in which data samples are grouped into clusters. Sample within one cluster are more similar than those in different clusters. Many clustering algorithms are good at handling low dimensional data, involving only several dimensions. It is challenging to cluster data samples in a high dimensional space, especially considering that such data can be very sparse and highly skewed –3]. Hig. Hig3]. Clearly, clustering of high-dimensional sparse data requires special treatment. There are three strategies to alleviate problems caused by the sparsity and high dimensionality of the data. The first one computes semantic similarity between samples by using an external knowledge source, such as WordNet  and WordIn contrast to the above strategies, the novel similarity metric we are studying computes the similarity between samples by incorporating the knowledge of a feature interaction network, where feature interaction network consists of the features of the samples in data. In some applications, there exists such a feature interaction network. For example, in the task of subtype discovery from gene mutational data, a gene interaction network contains the relationship between genes which can help to alleviate the problem of sparseness –14]. He. He14]. Few samples and high dimensional features. Sparse and heterogeneous characteristics. From Discovering the process using the mutation data for subtype discovery is a challenging task. A good patient representation is expected to group similar patients and separate the different groups. Therefore, in this paper, we try to overcome the problems of traditional similarity-based metrics when applied to high-dimensional sparse data, e.g., gene mutational data. We propose a new network-based similarity metric to take advantage of the prior knowledge of a feature interaction network. We project each sample into the feature interaction network and measure the similarity between two samples using the similarity between features in the network in Network-based similarity metric: we propose a novel similarity metric to measure the similarity between samples using a feature interaction network.Effectiveness: When applying subtype discovery, our approach outperforms state-of-the-art algorithms in discovering cancer subtypes, and detects biologically significant cancer subtypes that cannot be identified by other top competitors using real cancer data.Thus, our main contributions are as follows:Furthermore, our network-based similarity metric can be easily incorporated into any clustering algorithm that contains data attributes with network structures.The paper is organized as follows: Section 2 discusses related work; Section 3 presents the proposed metric for computing the similarity between samples; Section 4 reports experimental results on synthetic data and uterine adenocarcinoma datasets; Section 5 concludes the paper.II.We discuss existing work on similarity-based metrics, network-based similarity metrics, and subtype discovery in cancer.A.There are some traditional similarity functions such as Euclidean distance, Cosine similarity and Pearson’s distance, which provide a way to measure how close two samples are . In probB.In practice, there are many datasets that contain explicit relations among samples, such as citation network datasets  and NELLC.Identifying cancer subtypes is essential for a wide range of applications, that of which includes a better understanding of the biological complexity of the disease and developing targeted, precision medicine therapeutic interventions , 25]. C. C25]. CGene mutational data, which can be more reliably obtained than gene expression data, help to determine how the subtypes develop, evolve and respond to therapies –29]. In. In29]. If we focus on clustering algorithms to stratify sparse and heterogeneous somatic mutational profiles, perhaps the most popular approach for subtype discovery is NMF, which does not require any prior knowledge of the expected number of subtypes or the associated mutational patterns . NMF aimIII.X to be analyzed consists of high dimensional binary features and there exists a feature interaction network G that represents the relevance between features. Excluding sparse and heterogeneous characteristics, we focus on this type of dataset with the non-overlapping characteristic.We assume that the data Definition 1 (Non-Overlapping Characteristic): Two samples who belong to the same topic may not share identical mutated genes.Defintion 2 (Sparse and Heterogeneous characteristics): The dataset can be represented as a binary matrix of feature attributes M is the number of sample data and N is the number of features. The sparse characteristic describes the scenario where most of the sample data contains few non-zero features. Additionally, the number of non-zero features is far less than the totoal number of features N. The heterogeneous characteristic describes the scenario where some of the samples contains hundreds of non-zero features, especially in comparison to the sparse characteristic.Defintion 3 (Feature Interaction Network): The feature interaction network G =  consists of N features as nodes, where V = {v1, …, vN} represents the set of vertices, E represents the set of edges, and each vertex represents one feature. Each edge connecting two vertices vi and vj has a weight which represents the relevance between two vertices, denoted as s. So if the weight of the edge that connects two vertices is high, then the two vertices are more relevant. Here, vi and vj are referred as neighbors. Edges(vi) represents all edges connecting to vertex vi and Neigh(vi) represents all neighbors of vertex vi.G, we introduce Feature Alignment Similarity (FAS) which is a new network-based similarity metric that embeds intrinsic relevance among features. FAS is designed to deal with data samples that have few overlapping features. Datasets with non-overlapping features are common in text data and biological data. For example, two cancer patients who belong to the same cancer subtype may not share identical mutated genes due to the complicated nature of cancer diseases  represents how much the weight of feature pi allocates to to feature qj. The similarity of two samples to the maximum cumulative cost required to align all features of one sample to the other sample, namely, s represents the feature relevance that is discussed in subsection 3.2. The maximal similarity between two samples can be calculated using the following objective function,P with Q, the entire outgoing weight from feature pi equals w(pi), namely qj must equal w(qj), namely, F is found by solving this linear optimization problem, and the best average time complexity of solving the FAS problem is O(N3logN), where N is the number of all features in the feature interaction network . T. Ti and k = 0k above, the only known paths correspond to the single edges in the graph. At k = 1, paths that go through vertex 1 are found. At k = 2, paths going through vertices 1 and 2 are found, and so on. Given two vertices 1 and 4, the path of greatest similarity is . T. T42]. TThe above optimization is a special case of the Earth Mover’s Distance , 45], a, a45], aIV.Evaluation using Synthetic Data. How accurately does NetAP detect cancer subtypes with respect to various gene network structures? Does NetAP outperform the stateof-the-art algorithms used for cancer subtype discovery?Performance using Real-World Data. Can NetAP detect cancer subtypes that are clinically meaningful? What is the impact of different gene networks on performance? Can NetAP identify cancer subtypes that cannot be detected by other clustering algorithms?The task of clustering cancer patients using tumor mutation information is difficult. A real-world cancer dataset typically has hundreds of samples, but the number of gene mutations can be well above 15,000 as shown in In our empirical study, we observe that AP is the strongest baseline clustering algorithm even though it does not use a gene network. A possible explanation is that the power of belief-propagation can better tune the center of each cluster. Hence, in our experiments, we chose AP to integrate with gene interaction networks for optimal performance.1 All experiments were conducted on a Windows machine with an Intel 437 2.9 GHz CPU and 8GB memory. We implemented NetAP in Matlab.A.2 and the PathwayCommons gene interaction network . C. C48]. CNormalized Mutual Information (NMI) is a clustering validation metric that effectively measures the amount of statistical information shared by the predicted cluster assignments and ground truth, independent of absolute cluster labels. Two patients are assigned to the same cluster if and only if they are similar; thus, clustering can be viewed as a series of pair-wise decisions.Rand Index (RI) measures the percentage of clustering decisions that are correct. Rand Index can be adjusted for the chance clustering of elements, which will result in one of its variants called Adjusted Rand Index (AR). AR has a value between 0 and 1, and RI can have negative values.Chi-Square is used to determine whether there is a significant difference between expected clusters and observed clusters.P-Value can determine how significant clustering results are by performing a hypothesis test commonly used in statistics.3 with Nonnegative Matrix Factorization (NMF) [K as the real number of clusters of each dataset.We compared our NetAP algorithmon (NMF) , Latent on (NMF) , Affiniton (NMF) , and Neton (NMF) . For eacA. Here, W vector of coefficients can be interpreted as the k topic membership weights for the corresponding document. We use the open-source MATLAB implementation4 for NMF based on Euclidean distance.an unsupervised learning technique originally employed to decompose high-dimensional data 5α and β of LDA are set as 0.1 and 0.1.a directed graphical model that models a document as a mixture of topics and a topic as a mixture of words. When LDA is used for text clustering, we choose the maximum value from a mixture of topics as its cluster label for each document. LDA based on Gibbs sampling is chosen as comparison .5α and 6 Based on empirical observation, the Pearson correlation coefficient is chosen as the distance metric. We set parameter λ = 0.9 for AP.a clustering algorithm that takes as input measures of similarity between pairs of texts and simultaneously considers all data points as potential examples. For AP, we use the “apcluster” package in R.et al. [a clustering algorithm that incorporates the information from gene networks into network-based non-negative matrix factorization . The souet al. .more clustering algorithms are used for subtype discovery. Here, we choose these algorithms  cannot deal with extremely sparse data such as somatic mutation data that has lots of 0s and few 1s, even though incorporating network information can help to alleviate the sparseness problem to a certain degree.In summary, we can conclude that NetAP is the most appropriate clustering algorithm for clustering gene mutations, which can produce a robust division of patients into subtypes from somatic mutation profiles combining gene interaction network. Because NBS and NetAP are the only two algorithms using gene network, we will compare them in more detail that use two more gene networks.1)We chose four existing methods  in the previous experiment. To fully assess the effectiveness of our method, we conducted more experiments to compare with other clustering algorithms K = 3, 4, …, 12) using five metrics . Clearly, NetAP works better than NBS on these two gene interaction networks in general, except on AR and RI metrics using STRING network. Especially, when increasing the number of subtypes K, NetAP can achieve better results than NBS. Similar to the results that use the PathwayCommons network, the experimental results give further evidence that our method NetAP is more robust for subtype identification than NBS.As NetAP is naturally dependent on gene interaction networks, we examine how different gene networks affect the quality of NetAP with NMI metric. We chose the following three gene networks: PathwayCommons, STRING and HumanNet. 3)To assess the biological significance of the identified subtypes, we examine whether they correlate with observed clinical data. V.In this paper, our goal is to propose a novel similarity measure that computes the similarity between samples by incorporating the knowledge of interaction networks to overcome data sparseness problem. Our metric does not directly compute the similarity between two samples but measures the similarity by the similarity from the embedded features of one sample to another after the two samples are projected into feature interaction network. In this way, although two samples do not share one common feature, they are likely to belong to the same clustering when their mutations share the similar network regions. When applied to the discovery of cancer subtypes, our approach demonstrates effectiveness and efficiency on synthetic and uterine adenocarcinoma datasets along with three popular gene networks using five different metrics. In the future, we plan to integrate multiple layers of information beyond somatic mutations  into our method for better subtype identification.