Interpretable machine learning models for gene expression datasets are important for understanding the decision-making process of a classifier and gaining insights on the underlying molecular processes of genetic conditions. Interpretable models can potentially support early diagnosis before full disease manifestation. This is particularly important yet, challenging for mental health. We hypothesise this is due to extreme heterogeneity issues which may be overcome and explained by personalised modelling techniques. Thus far, most machine learning methods applied to gene expression datasets, including deep neural networks, lack personalised interpretability. This paper proposes a new methodology named personalised constrained neuro fuzzy inference (PCNFI) for learning personalised rules from high dimensional datasets which are structurally and semantically interpretable. Case studies on two mental health related datasets (schizophrenia and bipolar disorders) have shown that the relatively short and simple personalised fuzzy rules provided enhanced interpretability as well as better classification performance compared to other commonly used machine learning methods. Performance test on a cancer dataset also showed that PCNFI matches previous benchmarks. Insights from our approach also indicated the importance of two genes (ATRX and TSPAN2) as possible biomarkers for early differentiation of ultra-high risk, bipolar and healthy individuals. These genes are linked to cognitive ability and impulsive behaviour. Our findings suggest a significant starting point for further research into the biological role of cognitive and impulsivity-related differences. With potential applications across bio-medical research, the proposed PCNFI method is promising for diagnosis, prognosis, and the design of personalised treatment plans for better outcomes in the future. Over the past decade, there has been numerous attempts in combining machine learning methods with high-throughput gene expression data for diagnosis with promising results such as in the detection of cancer2, mental illnesses3, and genetic disorders5. A further objective would be predictive medicine where attempts are made to predict disease progression and response to intervention6.In molecular genetics, high-throughput gene expression profiling methods, such as microarray and RNA-sequencing are often used to examine the transcriptomic changes between case (patient) and control groups in comparative studies of complex diseases. It is also widely understood that complex, polygenic diseases often do not have clear signals characterised by only a few genes7. Challenges ensue, as some machine learning models tend to overfit, and the ‘black-box’ nature of some models make it difficult to reveal the underlying biological mechanisms represented in the data. The inherent heterogeneity of some genetic conditions represented in the data can also exacerbate the problem of confounding and lack of transparency, resulting in the model making correct decisions via incorrect reasoning. To trust the results of a machine learning model, interpretable methods are needed, in which the models explain why certain diagnostic/prognostic outputs are produced8.Appropriate modelling of gene expression data can help detect differential gene activations, leading to the identification of critical biological pathways even before the disease is fully developed (at the prodrome phase) which can be used for better informed interventions. However, high-throughput gene expression datasets are high dimensional and usually have a comparatively small sample size10. Neuro-fuzzy Inference systems (NFIS) facilitate automated rule learning from data, which can generate rules characterised by descriptive language involving fuzzy predicates. Knowledge in a NFIS is expressed as IF–THEN rules, with forward chaining propositions involving fuzzy sets. The presence of ‘fuzziness’ allows the NFIS to capture the ambiguity of descriptive language for closer resemblance of human decision making. Therefore, the IF–THEN rule based reasoning, facilitates linguistic representations of the decision-making process of the classifier via natural language. However, fuzzy rules acquired from a NFIS, usually require further adaptation to obtain concise and interpretable rules that can be used in clinical settings to advance biological knowledge11.Previous research has shown that fuzzy inference systems are a convenient class of methods that can explain the decision-making process of the model in a systematic manner13. Another approach is to build “personalised” machine learning models for each individual using data from other related individuals, selected with respect to clustering criteria12. This can generate individual profiles that further enhance the interpretability and allow the development of precision medicine and treatments tailored for every individual. Methodologies for integrating personalised modelling with NFIS have been developed, such as TWNFI and TWRBF15. The TWNFI and TWRBF models perform well in terms of prediction accuracy, however, the extracted fuzzy rules are not optimized in terms of providing interpretability.Under the assumption that the learned rules are applicable to all new data samples, most NFIS approaches aim to learn global rules using the entire training set. However, due to the heterogenous nature of some conditions , global rules may not be appropriate for all genetic subgroups. In some NFIS, local cluster-based IF–THEN rules have been extracted, for example in relation to gene expression patternsConsidering these limitations, this paper proposes a new methodology outlined in Fig. 18. The methodology proposed in this paper focuses on the implementation of these conditions and extracting rules based on personalised profiles of the patients.In fuzzy inference systems, interpretability refers to the readability and comprehensibility of the extracted rules. The general interpretability criterion is defined by two facets, the structural aspects, and semantic aspects of fuzzy rules. For a thorough treatise regarding interpretability conditions of FIS, we refer the reader to20. However, this is not applicable for NFIS with gradient based optimisation. Rule-base simplification or reducing the number of rules is also essential for complexity reduction. The simplest approaches utilise incremental procedures which start with a small rule base and incrementally add more rules until the model accuracy converges or a threshold is reached. Conversely, it is also possible to start with a high number of rules and incrementally shorten the rule base21. Genetic or evolutionary approaches have also been utilised to tackle multiple, conflicting objectives to find the optimal trade-offs between minimising the number of rules and maximising the accuracy22. Other methods which may include merging similar rules using similarity metrics, possibility measures or disregarding redundant rules have also been applied23. Structural interpretability improves the rule readability and are commonly applied to NFIS models. However, semantic interpretability is often neglected.The structural interpretability aspect focuses on the readability of the rules, this mainly refers to the components that describe the model complexity, including number of rules, number of rule antecedents (features), and number of membership functions. Fuzzy rules in a conventional system include all features in its premise as antecedents. Of course, the rules will be easier to understand if they are defined by only the most important features in the data. A common method is to perform feature selection  before defining the fuzzy partitions. Alternatively, some studies considered a more localised ‘rule-by-rule' approach by having a ‘don’t-care’ clause, indicating a uniform membership over all elements in the universal set17. Common conditions for semantic interpretability include distinguishability, coverage, and relational preservation18 . Some of these conditions require the use of inequality constraints, which can make optimisation difficult due to the non-convex and non-linear nature of NFIS loss functions. Previous studies attempt to clip these variables via IF-ELSE conditions26. However, this may result in unstable training and non-optimal solutions. To overcome these issues, the PCNFI method utilises the log barrier method used in inequality constrained optimisation problems. The log barrier belongs to family of interior-point methods, which approximate Lagrangian optimization as a sequence of unconstrained problems27. This helps facilitate easy training with standard gradient-based methods.For a fuzzy model to be interpretable, it is required that the semantics of the learned rules and those known naturally to the users are coherent. Simply put, reading the linguistic/symbolic representation of the rules should make logical sense and relate to some meaning28. A primary advantage of personalised modelling is that it can identify a cluster of input vectors to build unique profiles for each individual. Developing a system to offer personalised models and profiles for each patient can be quite useful in clinical applications, especially when dealing with high dimensional gene expression datasets that have a heterogeneous nature with molecular subgroups which may exhibit different disease progression across individuals who may require different treatments. Previous predictive personalised models include WWKNN12, TWRBF and TWNFI15. While the previous personalised NFIS based methods showed good performance, the extracted personalised fuzzy rules were not optimised in terms of better interpretability. The rule learning process was not enforced to follow pre-defined semantic and structural interpretability conditions, so that interpretability becomes part of the learning process. In the current paper, the proposed PCNFI method improves the model performance by incorporating interpretability conditions in the methodology.Personalised modelling is based on transduction, to build a model on data samples that are in some way relevant to the input vectorThe PCNFI methodology is applied to two sets of gene expression data for diagnosis of mental illnesses and another Liver cancer gene expression dataset for classifying between tumours and non-tumours tissue.29. This is a two-class problem comparing healthy controls against UHR. The dataset comprises of 84 total samples with 55 UHR participants and 28 healthy controls. For more information on the acquired gene expression dataset, please refer to30.This is a cohort study characterizing differences in youth at ultra-high risk (UHR) for psychosis compared to healthy controls. Participants were drawn from mental health care, community-based services, and various educational institutions in Singapore. Healthy controls in this study are participants who did not fulfil the UHR criteria and had no psychiatric history. Gene expression measurements were taken from Peripheral blood; with 34,694 genes (features)31.Bipolar disorder (BD) is a psychiatric disorder characterized by instability in mood, resulting in manic and depressive episodes. This cohort was collected from the Netherlands with 240 controls and 240 cases. Peripheral whole blood was drawn and processed for genotyping and RNA sequencing from 240 controls and 240 cases, of whom 227 and 13 were diagnosed with bipolar disorder type 1 and type 2 respectively. The dataset has 20,583 genes (features) after pre-processing. This is also a two-class classification problem, controls vs Bipolar cases with a total of 467 samples. For more information on the acquired gene expression dataset, please refer to32 This is also a two-class classification task with the aim to classify between tumours and adjacent non-tumorous tissues of HCC patients. Although, this is not a diagnosis problem, we have used this example to show that the classification performance of PCNFI can be generalized to other  diseases.This dataset is based on the expression profiling of tumor and adjacent non-tumorous tissues of Hepatocellular Carcinoma (HCC) patients, a common type of liver cancer. The National Cancer Centre of Singapore (NCCS)/SingHealth Tissue Repository provided tissues of the HCC patients. Samples included 59 tumours and 59 adjacent non-tumorous samples. The dataset has 47,325 genes (included non-annotated genes). For more information, please refer toTo test the classification performance and structural interpretability of PCNFI for diagnosis of UHR and bipolar, we try different combinations of the number of features, neighbours and rules. The generalisation of our approach is tested by leave one out cross validation (LOOCV). The unlabelled test vector from each LOOCV split is considered as an input vector (excluded from training), for which a personalised model is built using the labelled training data. For the proposed PCNFI model, the main hyperparameters include, the neighbourhood size (selected from each class), the number of rules and the strength of penalties and log barrier during constrained optimisation. The optimal strengths for the penalties and the log barrier are manually chosen by considering the optimal level of trade-off between the accuracy and interpretability.In Fig. 33 on the training dataset and using the same features on the unlabelled test set. The Hyperparameters are tuned via a Bayesian optimisation approach, called tree-structured parzen estimator34. This method approaches hyperparameter optimisation problem from a probability perspective and uses past trails to choose the next best set of hyperparameters for evaluation.The PCNFI model is also compared to other popular classification techniques including Support Vector Machine (SVM), Naïve Bayes, Gradient Boosted Trees and Multi-layered perceptron. Feature selection is performed separately in each LOOCV split by ranking the features using signal to noise ratio method (SNR)35. Comparisons using other metrics  are reported in Supplementary Fig. Figure The previous section has shown that PCNFI can achieve good accuracy with structural interpretability by having a small number of rules and rule antecedents. Next, we demonstrate the semantic interpretability improvements given by PNCFI through three examples in Fig. Figure One of the main advantages of fuzzy inference systems is the possibility for symbolic representation of the rules. The symbolic representation can help describe the general behaviour of the system in natural language. The fuzzy sets utilised for rule learning in this paper are not fixed and are fine-tuned by minimising an error. This makes linguistic labels such as ‘high’, ‘low’ unsuitable as the support (range) for these variables is not properly defined. Instead, we describe the system in other imprecise terms such as ‘about’ or ‘approximately’. A fuzzy set in an antecedent condition is described as ‘gene 1’ is about entclass1pt{minima36. MRI studies on ATR-X patients (syndrome caused by mutations on the ATRX gene) showed severe glial defects and white matter disruption37.Table . Contrarily, the degree of memberships for rule representing the control class are high. For the RANBP2 and TSR1 genes the inputs have higher memberships in the sets representing the case class. However, since other sets representing the control class have membership close to zero, the effect of these genes (RANBP2 and TSR1) on the overall outcome gets mitigated. As the propositions are linked with a product operator the resulting firing strength for this rule is very low and the individual is correctly classified as control.For another perspective, the personalised rule profiles can be visualised to show how rules are activated for an individual (a randomly selected sample from the bipolar data). The example in Fig. 38.The personalised model for each input sample is created by selecting genes with respect to the nearest neighbouring samples to the input. This may result in unique sets of genes being selected for each sample. It is expected that there will be some overlap between selected gene sets across different personalised models, signifying possible common origins or functional overlaps. Indeed, deepened insights into those genes which are frequently selected amongst the top sets of discriminatory genes may provide information regarding their potential to serve as diagnostic biomarkers for mental health status progression. The size of the neighbourhoods for the LYRIKS data was 30 samples from each class and for the bipolar dataset, 35 samples from each class. We created a total of 84 personalised models corresponding to 84 samples in LYRIKS dataset and 467 Personalised models corresponding to 467 samples in bipolar dataset. Table The reasoning behind the output of a classification system (diagnosis) is essential for clinical and biological application of gene expression datasets. Henceforth, this paper has proposed a new method called PCNFI, to better understand the decision-making process for gene expression classification via interpretable personalized rules. The contribution of this study is twofold, first is the novel PCNFI methodology itself and second is the biological findings obtained using this method on two gene expression datasets (LYRIKS and bipolar). The main component of the PCNFI method is the implementation of semantic constraints using constrained optimisation. Some semantic conditions like the boundedness of UOD require the use of inequality constraints. In general, optimising with such constraints is difficult due to the non-convex and non-linear nature of NFIS losses. A common work around used in previous studies is to clip the variable if it violates the UOD range. However, this may result in unstable training and non-optimal solutions. In this work we have implemented the inequality constraints using the log barrier method which approximates constrained optimization as an unconstrained problem. This leads to all constraints being fully handled by gradient descent optimisation as in standard unconstrained losses. Overall, PCNFI implements two other constraints as penalties to maintain distinguishability and relational preservation between fuzzy sets. Consequently, the resulting rules have been shown to be more interpretable while maintaining good accuracy. Another essential component of our methodology is personalised modelling, which means that rules are personalised for every individual based on their genetic profile. Based on the profiles, the rule antecedents vary for individuals in terms of which genes are included and the conditions which causes the rule to fire.38. The ATRX gene is involved in transcriptional regulation via the chromatin remodelling process39. According to prior research, the ATRX gene appears to regulate the activity of two genes: HBA1 and HBA2, which are required for the production of haemoglobin protein, which transports oxygen throughout the body40. Therefore, ATRX gene mutations are associated to X-linked syndromes with cognitive disabilities as well as alpha-thalassemia (ATRX) syndrome. Impaired oxygen transport may result in chronic low-level ischemia, to which, regions implicated in a mechanistic cascade to psychosis e.g., hippocampus43, are particularly vulnerable44. Reduced expression of ATRX is associated with hippocampal dysfunction46 and a neurodevelopmental syndrome47. However, to our knowledge we are the first to associate ATRX with risk for psychosis.The biological findings of this paper include the ATRX and TSPAN2 genes as possible biomarkers for differentiating the UHR, bipolar and healthy individuals48. This may reflect involvement in oligodendrocyte development and neuroinflammation49, which is a risk factor for impulsivity, emotional dysregulation and psychopathology50. Impulsivity is high in several psychological conditions . Our current findings are novel in highlighting the importance of Tspan2 for Bipolar disorder in humans and represents a critical starting point for further research. For example, together with mania, hyperlocomotion and excitement, impulsivity contributes to an ‘excitement’ symptom cluster associated with specific neurophysiological functions52, also seen in the very early stages of psychosis53, that should be investigated in relation to Tspan2 in future studies.Tetraspanin 2 (Tspan2) is typically localised to compact myelin. Its gene expression and exon usage are increased in the mesocorticolimbic pathway (nucleus accumbens) of highly impulsive Wistar rats54 and RFE-SVM55 could also be tested and compared against SNR in terms of stability and reproducibility however, such benchmarks would require proper gold-standard evaluation data, which falls outside the scope of this work. Another limitation may be our neighbourhood selection approach which relies on Euclidean distance with uniform weights. Different types of distance functions and varying feature (gene) weights should be explored to define more accurate patient profiles. Gene expression is only one of many biological modalities. Although high-dimensional, within themselves, gene expression alone may not capture fully the richness and completeness of information needed to make confident assertions regarding diagnosis and individualized disease progression. This presents limitations on the quality of the explanations provided by our model. Thus, future work should try to incorporate other OMICS variables (inclusive of proteins and metabolites), other behavioural and clinical variables or demographic variables to help define a more accurate and representative neighbourhood. Another future direction of our research is to extend and apply the proposed PCNFI method to advanced cohort studies that monitor patients’ performance over longitudinal data measurements for early prediction of mental illnesses. This includes gene expression time series data, where the personalised rules in PCNFI will include a time component.It should be noted that there are limitations to our approach. The PCNFI methodology has utilised SNR to select the most important genes. While the advantage of SNR is that it is fast, it may not be the best way to select top genes. SNR is a univariate feature selection method which does not account for the interactions between the genes. For future work, other gene selection methods like DeSEQ2This paper has presented a new constrained personalised neuro fuzzy inference (PCNFI) methodology for extracting interpretable personalised rules from high-dimensional gene expression datasets. The proposed method allows for structural and semantic interpretability of the fuzzy rules by improving the readability and comprehensibility of the rules. To improve the structural interpretability, this paper has utilised methods for feature selection, clustering and merging of the rules. Semantic interpretability is improved by constrained optimisation during the fine-tuning phase. The effectiveness of the proposed method is demonstrated on two case studies of gene expression datasets. Results indicate that the proposed method not only offers superior interpretability, but also achieves better classification accuracy compared to traditional machine learning algorithms . The results showed 82% classification accuracy on the bipolar dataset which shows on-average up to 8% of improvement compared to other classification methods when using only 4 to 8 genes. While showing up to 88% classification accuracy on the LYRIKS dataset when using only 6–20 genes. Performance comparison on the liver cancer gene expression dataset showed that PCNFI matched previously reported benchmarks. In addition to increasing the accuracy, the proposed method extracted personalised rules that are interpretable and can be used to explain certain decisions (diagnosis) given by the model. The rules are also helpful to describe the general behaviour of the system in natural language and through visualisations, hence providing explainability for end-users. Certain genes that regulate mechanisms implicated in risk for psychosis but have never been discovered as associated factors of psychosis, have also been identified in this research. Further research is required to evaluate biological mechanisms through which these genes are associated with risk for psychosis. The proposed method can be potentially used for personalised early diagnosis and prognosis in many other diseases, based on relevant gene expression data.10. Hence, they are universal approximators that can perform non-linear mappings between input and output pairs, along with having an “IF X is A, THEN Y is B” rule structure, where A and B are the fuzzy sets associated with the respective antecedents and consequents. Fuzzy logic enables class definitions to be interlaced and can be used for creating meaningful linguistic representations. Unlike classical Boolean logic, which only allows true or false propositions, fuzzy logic allows propositions to have degrees of truth. As a result, the system is more transparent to humans, while also providing additional validation tools for experts.Neuro-fuzzy inference systems (NFIS) are types of artificial neural networks based on the principles of fuzzy logicNeuro fuzzy inference methods are supervised learning methods that enable data-driven rule generation. Given data of the form Fuzzification determines the degree to which an input belongs to each of the fuzzy sets via membership functions, therefore it is converting crisp values into fuzzy values. In this method, a gaussian membership function (MF) was used, as they are continuously differentiable, this facilitates easy optimisation with gradient based methods. As part of updating the knowledge base (learning procedure) the shape of the gaussian curves can be adapted by tuning parameters mentclass2pt{minimThe rule layer computes the product of the previous layer’s inputs. A node at the rule layer represents the propositions part of a rule that takes d inputs; hence, the product operator allows for modelling interactions between the propositions in the antecedent.The normalisation layer computes the firing strength of each rule, reflecting to what extent the input The defuzzification layer defuzzifies the input from the previous layer into a single crisp value.The output from the defuzzification layer is fed into the sigmoid function to output a probability Feature selection using SNR to filter out genes with low variance.Selecting nearest neighbours from the data using features selected from step 1.Structural interpretability with a second layer of feature selection and initial fuzzy partitioning.Training a NFI model with constrained optimisation to achieve semantic interpretability.Classification using the trained model and testing the input vector.Merging similar fuzzy sets to reduce redundancy.The proposed methodology consists of the following steps:Further details of the proposed methodology are explained in the following subsections.In personalised modelling, a separate model is built on the nearest neighbouring samples to each input vector. Neighbours are selected by considering a distance  between the input vector and labelled samples . In high dimensional spaces such as gene expression data, distance metrics can become ineffective and may converge to become equidistant. To resolve this problem, we perform a first layer of feature selection to filter out genes with low variance. Then, the distances between an input vector sing Eq. . The calj is calculated as shown in Eq. , however this measure is computationally expensive due to the integration operation. Therefore, it is not suitable for efficient implementation of gradient-based training. Instead, an analytical measure based on distance . The objective function (loss function) is minimised with respect to some constraints on To ensure that relational preservation is maintained, the loss ity, Eq.  can be ut to solve the problem, and uses the solution as the starting point for the next iteration with increased value of t. As such, by following this method, the constrained optimisation problem can be solved with gradient based methods. The initial values are also required to be strictly feasible. The constrained optimisation problem can be solved by the following algorithm:The equations 1,c2 Eq.  are usedDepending on strength of the Supplementary Information 1.


There is a growing number of multi-domain genomic datasets for human tumors. Multi-domain data are usually interpreted after separately analyzing single-domain data and integrating the results post hoc. Data fusion techniques allow for the real integration of multi-domain data to ideally improve the tumor classification results for the prognosis and prediction of response to therapy. We have previously described the joint singular value decomposition (jSVD) technique as a means of data fusion. Here, we report on the development of these methods in open source code based on R and Python and on the application of these data fusion methods. The Cancer Genome Atlas (TCGA) Skin Cutaneous Melanoma (SKCM) dataset was used as a benchmark to evaluate the potential of the data fusion approaches to improve molecular classification of cancers in a clinically relevant manner. Our data show that the data fusion approach does not generate classification results superior to those obtained using single-domain data. Data from different domains are not entirely independent from each other, and molecular classes are characterized by features that penetrate different domains. Data fusion techniques might be better suited for response prediction, where they could contribute to the identification of predictive features in a domain-independent manner to be used as biomarkers. Next-generation sequencing (NGS) technologies have opened the way for the production of huge amounts of genomic data, which can potentially shed light on the complex biological mechanisms that connect genotypes to phenotypes . In the Specific clinically relevant phenotypes are regulated by multiple genomic domains. Focusing an analysis on a single domain might reduce the potential to correctly predict the development of a disease and the response to therapy. Mutations of the breast cancer genes 1 and 2 , for example, determine the development of homologous recombination deficiency (HRD) in ovarian cancer and make tumors more sensitive to PARP inhibitors ,4. The sData fusion (DF) techniques have emerged as a way to merge the information contained in different domains, with the aim of performing feature extraction or patient or sample cluster analyses while considering multiple sources of genomic information at the same time . Data fuSimilarity network fusion (SNF) is a commonly used, heuristic DF clustering method that simply involves computing a sample correlation network for each domain that is then merged into a meta network based on the similarity of the single-domain networks. SNF allows for sample clustering, which can yield clusters with enriched, clinically relevant features ,9. Data After preprocessing, data fusion methods are applied and class discovery approaches such as various clustering methods can be performed in order to reveal multi-domain molecular classes. Clustering can be performed on a matrix representing the fused domains, such as for joint singular value decomposition (jSVD) , or can In our previous work, we developed and adapted data fusion approaches for the prognostic classification of uveal melanoma (UM) . We perfSkin melanoma is a neoplasm of the skin, originating from melanocytic nevi, which is able to evolve into aggressive metastasizing tumors, some of which are resistant to therapy . Early mIn 2015, 333 primary and metastatic cutaneous melanoma samples from 331 patients were analyzed by Akbani et al. , who conhttps://gdac.broadinstitute.org, accessed on 1 July 2022), while the RNA-seq data were retrieved using the R package RTCGAToolbox (version 2.22.1) ), with N1 and N2 as the number of features of the RNA-seq methylation matrix). The minimization stopped when the norm of the projected gradient was lower than −12).The joint singular value decomposition function, as previously described in ,23, was ARI) and silhouette score.We used two indices to evaluate the DF methods’ performances, the adjusted Rand index (RI) is a measure of the agreement between two partitions of one group of elements; if we think of a set of patients classified into a set of groups by two methods, it is possible to define the RI as:The Rand index  represent elements not in agreement, such as elements in the same or different groups in only one classification method.With The adjusted Rand Index takes into account random labeling with a permutation model and is computed as:ARI score was computed with the adj.rand.index of the pdfCluster R package (version 1.0-3) [Hence, a value of 0 is defined as random labeling and 1 total agreement . The ARIn 1.0-3) .SI) is a measure of the fit of the clustering. If we define the average distance of a point a(i) and b(i) as the minimum of the distances between SI for each observation can be computed as:The silhouette score (SI) was computed with the silhouette function of the cluster R package (version 2.1.2) [The average value of the silhouette score for each cluster  .The significance analysis of microarray bootstrapping method as implemented in R (Samr)  was usedARI. The survival curves were computed for patients using the survival data (n = 282) from Supplementary Table S1D of [The molecular and clinical features of the samples used for the performance evaluation were extracted from Supplementary Table S1D, previously published in . The mute S1D of .All data fusion methods were able to integrate the information based on the RNA-seq and methylation skin melanoma dataset, while maintaining some degree of similarity between single domain classes, as reported in previously published Supplementary Table S1D  Table 1Table 1. To observe the degree of agreement between the jSVD clusters in the single domains, we computed the differentially expressed genes and methylated probes based on the domains used for data fusion using the SAMr package . The heaThe overexpressed genes in cluster 2 are enriched for GO terms associated with immune genes terms . If we cTo verify whether the predicted clusters were associated with different survival curves, we plotted their relative Kaplan–Meier curves ,34,35. TThe application of DF approaches based on the SKCM dataset reported here defined 3 clusters of patients with most of the methods presented. Based on the jSVD classification, we extracted the discriminative features of the RNA-seq and methylation domain data.In our previous work , we applRecent work shed light on the role of different genomic domains in data integration and cancer subtyping. Different combinations of miRNA, CNA, mRNA, and DNA methylation effectively depend on each other in the TCGA datasets , since DWhen evaluating the reliability of data fusion methods, the overlap between defined clusters and clinically relevant features has been assessed using scores such as ARI and log-rank test values based on survival rates and measures of cluster quality such as the silhouette score ,11,12. AAnother general limitation of DF development is the absence of a test set, where a gold standard for machine learning is used to tune the model parameters based on a set of samples (training set) and evaluate the performance based on a different set  . This isThe application of data fusion in the approach shown here produced a classification result with one cluster that was very similar to the single-domain RNA-seq immuno subtype. This subtype has been associated with better survival in multiple datasets and gene expression classification systems . In partPrevious studies reported a potential improvement in prognostic tumor classification with data fusion compared to state-of-the-art single-domain-based analysis techniques . The SNFThe integration of multiple data sources can result in more robust feature selection and might improve the detection of gene pathways that are targeted by events in different genomic domains  . The pot


DNA methylation is a process that can affect gene accessibility and therefore gene expression. In this study, a machine learning pipeline is proposed for the prediction of breast cancer and the identification of significant genes that contribute to the prediction. The current study utilized breast cancer methylation data from The Cancer Genome Atlas (TCGA), specifically the TCGA-BRCA dataset. Feature engineering techniques have been utilized to reduce data volume and make deep learning scalable. A comparative analysis of the proposed approach on Illumina 27K and 450K methylation data reveals that deep learning methodologies for cancer prediction can be coupled with feature selection models to enhance prediction accuracy. Prediction using 450K methylation markers can be accomplished in less than 13 s with an accuracy of 98.75%. Of the list of 685 genes in the feature selected 27K dataset, 578 were mapped to Ensemble Gene IDs. This reduced set was significantly (FDR < 0.05) enriched in five biological processes and one molecular function. Of the list of 1572 genes in the feature selected 450K data set, 1290 were mapped to Ensemble Gene IDs. This reduced set was significantly (FDR < 0.05) enriched in 95 biological processes and 17 molecular functions. Seven oncogene/tumor suppressor genes were common between the 27K and 450K feature selected gene sets. These genes were RTN4IP1, MYO18B, ANP32A, BRF1, SETBP1, NTRK1, and IGF2R. Our bioinformatics deep learning workflow, incorporating imputation and data balancing methods, is able to identify important methylation markers related to functionally important genes in breast cancer with high accuracy compared to deep learning or statistical models alone. DNA methylation is important in cancer development and progression due to its role in silencing tumor suppressor genes or enhancing oncogene expression . It invoDeep learning ,6 . As machine learning algorithms do not tend to work well with ‘no data’, these markers were either removed or imputed before proceeding further. We selected either removal or imputation based on research in Lena et al.  where auAnalysis of the 27K dataset revealed 2597 CpG markers with null values across all 337 samples. Hence, a cut-off of 80% for a missing CpG marker across all samples was used to remove a specific CpG marker. The remaining 24,981 CpG markers across the 337 samples had a total of 4911 missing values. Four different imputation techniques were used to fill in these missing values namely zero, k-nearest neighbor (KNN), mean, and iterative imputation. During zero imputation, the missing values are replaced with zero. For KNN-imputation, the missing CpG markers are imputed using the average values or weighted by Euclidean distance from CpG markers distributed across ‘k’ neighboring samples . Mean imA similar path was followed for imputation for the 450K dataset. Initially, there were 485,577 CpG markers with almost 89,671 CpG markers having null values across all 851 samples. However, unlike in the 27K dataset, there were several samples with a significant percentage of null values. Hence, a cut-off of 30% for a missing CpG marker across samples was used to remove them. After removal, the dataset had 395,722 CpG markers across the 851 samples with a total of 332,330 missing values that required imputation. A similar process of mean, zero, and KNN imputation techniques were followed with the exclusion of iterative imputation due to memory constraints. Our developed workflow is visualized in ANOVA is a technique that can compare the means of different groups. ANOVA uses F-tests (ratio of variances) to statistically test the equality of means. Larger values represent greater dispersion. Therefore if a specific feature, results in more separation in the means or classes, it will have a higher score. ANOVA offers an advantage over the common T-test which is known to use a repeating set of comparisons among two attributes at a time . ANOVA FRandom Forests ,26 are aThe class imbalance of tumor and normal samples is very high in the datasets as observed in The proposed deep learning sequential model is built using TensorFlow . Two varDataset is separated into positive and negative tumor outcomes.The limiting outcome is randomly separated into two sets containing 70% (for training) and 30% (for testing) of the data.A subset of the non-limiting outcome, equal to 70% of the limiting outcome, is randomly chosen.The two subsets of the two outcomes, equivalent in number, is combined to form the training data.All remaining samples are combined to form the testing data set.Both data sets are randomly shuffled internally.Due to the imbalance of outcomes in the datasets, preprocessing is aimed toward generating a random, balanced training dataset where there is an equal number of tumor and normal samples. The following steps were used to ensure this selection:Gene Set Enrichment Analysis (GSEA) was performed on the genes associated with the reduced set of CpG markers. GSEA identifies sets of genes that are enriched in a particular dataset when compared to a control. GSEA considers all genes in the dataset instead of considering only the subsets of genes with significant changes in gene expression. TCGAbiolinks package and ShinyGo  were usep-value < 0.05 to indicate a significant difference likely exists between score categories. We feel that further combining categories would obscure any ability to investigate a potential trend or pattern in the data. In this survival analysis, the categories created were combined already to alleviate any sample size or power issues. We propose that presenting the data with more categories will be more informative and helpful to the reader when considering this data and considering similar analyses with their own datasets.Cox Proportional Hazards modeling was used to determine significant survival differences based on the 7-gene set expression score. These scores were then summed across each of the seven genes split into tertiles for each sample, so each sample would have one total score. This total expression score was categorized into five groups  because sample sizes were small for more extreme scores. A survival analysis was performed using this total expression score. We used a log-rank p-value greater than a threshold value  were removed from the total features. The number of features is reduced to 3704 for 27K and 125,949 for 450K datasets. Similar analysis of datasets after application of SMOTE, creating a balanced dataset, saw 15,483 features being selected for 27K and 260,159 features for the 450K. Results from Random Forests saw a lesser number of markers being selected from both datasets compared to ANOVA. Finally, to incorporate results from both these algorithms, the reduced features obtained from the ANOVA F-test are applied to the random forest model. This approach reduced the number of important markers from 24,981 to 336 for the 27K imbalanced dataset, while the final list of markers for the balanced dataset was 475. For the 450K dataset, it was observed that the number of features reduced from 395,722 to 1044 for imbalanced data and 1445 for the balanced dataset.To verify the efficacy of the proposed feature selection technique, the baseline deep learning model was applied to three variants of the 27K dataset as highlighted in Two different models were tested on the original 450K datasets. As mentioned earlier, the difference between the two models is the number of filters per layer of the model. The base model is denoted with filters  while the larger model has filters . A total of 851 samples were used for both models where 750 samples were tumor and 101 normal as shown in p-values (those associated with breast cancer), were annotated to identify which genes were associated with those CpGs. This gene list was then used to perform enrichment analysis and understand how these genes interact, and to infer the functional impact of the CpGs.It is important to know the metastatic potential of primary malignant tissue as it is related to the choice of therapy. Previous studies indicate that sets of gene expression profiles can successfully predict survival . After fGene Set Enrichment Analysis (GSEA) was performed on six different gene sets associated with the markers identified in Additionally, GSEA analysis was performed on the 27K and 450K with SMOTE gene lists using ShinyGo . Of listTo evaluate the enriched gene sets identified above and understand their association with cancer, a comparative analysis was performed between these genes with cancer-related genes from other investigated references, one being the combined gene sets from COSMIC  and TSGeRTN4IP1, MYO18B, ANP32A, BRF1, SETBP1, NTRK1, IGF2R identified as overlapping between the 27K with SMOTE, 450K with SMOTE, and the tumor suppressor/oncogene list, an expression score was calculated. For each of the seven genes, expression values were broken into tertiles for all samples. The lowest tertile received a score of 0, middle tertile score of 0.5, and highest tertile score of 1. A survival analysis was performed using this total expression score. The overall log-rank p-value (=0.0027) from the survival analysis indicates a significant difference in survival is observed based on the total expression score . This list of seven genes highlights those genes that are known to be important in cancer development and happen to be on both Illumina chips. There is one study by Savci-Heijink et al.  observinThe strengths of this study include a large sample size, established, and standardized pre-processing of the methylation data, using simulation to balance an unbalanced dataset, and imputing missing data. There is great overlap between the 27K and 450K datasets in terms of overlapping site-specific markers (94% of 27K loci appear in the 450K set) and correlation of methylation values R2=0.95) . This meWe proposed a deep learning framework that can capture the most significant biomarkers responsible for breast cancer. Our model is capable of handling a high volume of data with missing values and class imbalance. We observed that reduced features with a balanced class performed better in predicting outcomes than features with an imbalanced dataset. We also performed Gene Set Enrichment Analysis on the sets of genes reduced by our model. To evaluate the efficacy of our model, we compared the reduced sets of genes with several cancer resources. The results seem to support the notion that deep learning methodologies for cancer prediction can be extended for use in the prediction of different types of cancer which will form the basis of our future work. Incorporating methylation data into this story is important from a public health standpoint providing a potential point of prevention and from a treatment standpoint for a potential point to target the P53 or cellular senescence pathways. Deep learning models can be computationally expensive as shown in this research, but to provide accurate results there is a need to handle more diverse datasets as well as take less time to train. Further research will focus on expanding the training dataset to incorporate other clinical variables in the decision-making process. We will also incorporate tumor sub-types and grade information followed by a web-based API to enhance the efficacy of the proposed approach. The feasibility of label-specific weights while training a deep learning model as an alternative to the application of SMOTE will also be explored to verify if that is a more robust technique compared to generating synthetic data for addressing data imbalance. Our focus on methylation markers rather than gene expression has discovered some novelty in breast cancer-specific markers. In addition, these methylation markers have been annotated to be associated with specific genes based on distance, however, these methylation markers may in fact alter or affect other more distant genes in the genome. To supplement this approach, we will explore modifications in the feature selection so that it can accept microarray gene expression data alongside methylation values. This enables the evaluation of specific genes based on their differentially expressed values and their previous association with a cancer type. Identification of these core genes will further reduce methylation markers that are being analyzed by the deep learning model thereby establishing a more robust and targeted approach. By fulfilling these five limitations, we will continue to develop functionality and test its use to enhance the utility of this workflow for the cancer research community.


Tumor stratification plays an important role in cancer diagnosis and individualized treatment. Recent developments in high-throughput sequencing technologies have produced huge amounts of multi-omics data, making it possible to stratify cancer types using multiple molecular datasets. We introduce a Network Embedding method for tumor Stratification by integrating Multi-omics data. Network Embedding method for tumor Stratification by integrating Multi-omics pregroup the samples, integrate the gene features and somatic mutation corresponding to cancer types within each group to construct patient features, and then integrate all groups to obtain comprehensive patient information. The gene features contain network topology information, because it is extracted by integrating deoxyribonucleic acid methylation, messenger ribonucleic acid expression data, and protein–protein interactions through network embedding method. On the one hand, a supervised learning method Light Gradient Boosting Machine is used to classify cancer types based on patient features. When compared with other 3 methods, Network Embedding method for tumor Stratification by integrating Multi-omics has the highest AUC in most cancer types. The average AUC for stratifying cancer types is 0.91, indicating that the patient features extracted by Network Embedding method for tumor Stratification by integrating Multi-omics are effective for tumor stratification. On the other hand, an unsupervised clustering algorithm Density-Based Spatial Clustering of Applications with Noise is utilized to divide single cancer subtypes. The vast majority of the subtypes identified by Network Embedding method for tumor Stratification by integrating Multi-omics are significantly associated with patient survival. Cancer is generally due to a variety of factors under the occurrence of somatic variation, which can lead to the cell growth of abnormal regulation and the formation of abnormal lesions , provideThe stratification of tumors into clinical and biological subtypes benefits precision oncology. For example, an entropy-based consensus clustering (ECC) method , neural Integrating multiplatform molecular data, such as gene expression data, miRNA expression data, and DNA methylation data, can effectively identify cancer subtypes . Howeveret al. posited a network-based stratification (NBS) method, which applies network propagation to discover cancer subtypes by gathering patients with comparable network mutations  approach for identifying clinically relevant patient categories from the somatic mutation spectrum of a large number of patients  expression data and protein–protein interactions (PPIs) through network embedding method. First, we cluster the samples with DNA methylation and mRNA expression data and calculate the Pearson correlation between genes in each cluster. Then, the gene pairs with strong correlation are preserved in PPI. Next, patient features are constructed by integrating corresponding gene features and somatic mutation profiles of cancer types. Finally, a supervised learning method Light Gradient Boosting Machine (lightGBM) is used to classify cancer types based on patient features, while an unsupervised clustering algorithm Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is utilized to divide single cancer subtypes.The DNA methylation data, mRNA expression data, somatic mutation data, and patient clinical data employed in the study are all downloaded from the TCGA database . We collThe overview of NESM is illustrated in The interactions between genes are reflected in the PPI network. To better mine the features of genes in the PPI network, we adopt the struc2vec model , patients in COAD cancer are considered as positive samples, while patients from other cancers are negative samples. By using 5-fold cross validation, the positive and negative samples are separated into training and testing sets. The 5-fold cross validations perform 100 times and the average of the results is the final AUC value. Moreover, we compare with 3 latest methods: NES, NBS, and ECC methods. As shown in Another aim in this work is to classify patients with the same cancer type into the corresponding subtypes. We assemble clinical information on patients with 14 cancer types from the TCGA database and obtain staging information to assess the stratification of tumor mutations. The tumor stage refers to the primary tumor and the extent of intraindividual spread. In general, the more extensive the spread, the worse the prognosis. The tumor-node-metastasis (TNM) staging system is the most widely utilized around the world. After T, N, and M are determined in TNM staging, corresponding general staging can be obtained, namely stage I, II, III, and IV. The stage I COAD tumor is taken as an example, and the stage I COAD tumor is labeled as a positive sample. Nonstage I COAD tumor markers are negative samples. We find that the average AUC of most tumor stages is higher than 0.75, while the average AUC of kidney renal clear cell carcinoma (KIRC) tumor stages is 0.66. To test the algorithm's robustness, we evaluate the parameters involved in the algorithm. In our method, 2 main parameters affect the algorithm: initial clustering and weight screening. We find that the time cost of the algorithm increases with the increase of parameter value. Therefore, we take Based on the above studies, we believe that patients with similar clinical information may be more inclined to cluster together. This means that we can obtain the optimal cluster of patients through an unsupervised learning algorithm, that is, patients are subdivided. We use the DBSCAN clustering algorithm to cluster the same cancer patients. The number of clusters matches the number of subtypes reported in the literature when it comes to identifying medical tumor subtypes. Taking CESC as an example, it is divided into 2 subtypes according to clinical and endocrine features (divided into—type I and type II) or histopathological features . We generate 2 groups of patients with endometrial carcinoma (CESC) in the data. We also evaluate the clustering results by calculating the Silhouette Coefficient and Calinski–Harabasz Index . For theCancer is a multifaceted illness caused by both hereditary and nongenetic components. With the development of technology, multi-omics data has recently been widely used for various cancer types. In this work, NESM constructs patient features by integrating corresponding gene features and somatic mutation profiles of cancer types. Since network topology information is extracted by integrating DNA methylation, mRNA expression data and PPIs through network embedding method, it is contained though the gene features. We apply supervised classification algorithms to classify pan-cancer and individual cancer stages. The experimental results show that the patient features extracted by the NESM method are effective for tumor stratification. When cancer subtypes are subdivided, the vast majority of subtypes identified by the NESM method are significantly associated with patient survival. NESM extracts features mainly from network topology, which is not considered by most methods.It allows better classification and subdivision of cancers into subtypes than other methods, but it still has some limitations. For example, the choice or construction of a PPI network may have an impact on the NESM model. In addition, the rate of somatic mutation varies greatly among different tumor types. Some tumor types  have a high mutation rate, while others have a low mutation rate [such as rectum adenocarcinoma (READ) and BRCA]. In the current NESM framework, we only integrate normal tumor samples that match somatic mutation profiles, DNA methylation, and mRNA expression data. Integration of other types of omics data, including RNA sequencing, individual patient proteomics, and whole-genome sequencing, may further improve the NESM model. Second, the framework of the method is to cluster patients based on patient features extract from specific data sets. This framework can be used to address the tumor stratification problem using a variety of additional algorithms. For example, we can use a graph convolution neural network to improve the prediction accuracy and use other clustering algorithms, including hierarchical clustering and Gaussian mixture model clustering. In future work, it will provide some clues for precision oncology and clinical applications.jkac243_Supplemental_MaterialClick here for additional data file.jkac243_Figure_Supplemental_S1Click here for additional data file.jkac243_Figure_Supplemental_S2Click here for additional data file.jkac243_Figure_Supplemental_S3Click here for additional data file.


Recently, the field of polymer nanocomposites has been an area of high scientific and industrial attention due to noteworthy improvements attained in these materials, arising from the synergetic combination of properties of a polymeric matrix and an organic or inorganic nanomaterial. The enhanced performance of those materials typically involves superior mechanical strength, toughness and stiffness, electrical and thermal conductivity, better flame retardancy and a higher barrier to moisture and gases. Nanocomposites can also display unique design possibilities, which provide exceptional advantages in developing multifunctional materials with desired properties for specific applications. On the other hand, machine learning (ML) has been recognized as a powerful predictive tool for data-driven multi-physical modelling, leading to unprecedented insights and an exploration of the system’s properties beyond the capability of traditional computational and experimental analyses. This article aims to provide a brief overview of the most important findings related to the application of ML for the rational design of polymeric nanocomposites. Prediction, optimization, feature identification and uncertainty quantification are presented along with different ML algorithms used in the field of polymeric nanocomposites for property prediction, and selected examples are discussed. Finally, conclusions and future perspectives are highlighted. Th. Th16]. Polymer nanocomposites have synergistic properties that can easily be tailored for attaining a desirable specific set of properties by selecting the appropriate combination of continuous and dispersed phases. For optimization and material design, all the processing parameters should be taken into account simultaneously. Modelling the complex relationships between the governing parameters (both input and output) is very arduous. Despite the availability of large experimental setups and computational tools, it is tedious and time-consuming to explore the significance of each of the governing parameters experimentally. Over the last two decades, material science has experienced a progressive shift from developing raw computational techniques for the design of novel materials to developing coupled methods that improve the results’ reliability via computational predictions and experimental validation. Finite element and molecular dynamics simulations have been applied to model the material behaviour in numerous arenas; however, the complexity and computational intensiveness of the approaches have prompted researchers to look for additional alternatives ,21,22. TThe application of ML to polymeric nanocomposites enables us to predict numerous multifunctional properties based on both the components and their proportions. Many ML algorithms have been developed for polymer composites depending on the property types and the datasets available. However, most of the studies are restricted in scope by the constraints of multiple variables which result in increased dimensionality and uncertainty caused by the randomness of the data.This paper aims to provide a brief overview of the most important findings related to the application of ML for the rational design of polymeric nanocomposites. First, different types of nanomaterials used in polymer nanocomposites are described. Then, prediction, optimization, feature identification and uncertainty quantification are presented along with different ML algorithms used in the field of polymeric nanocomposites for property prediction, and selected examples are discussed. Finally, conclusions and future perspectives are highlighted.Different allotropes from carbon have been recently used as nanofillers in polymeric nanocomposites, including fullerenes, quantum dots (QDs), carbon nanotubes (CNTs), graphene (G) and its derivatives graphene oxide (GO) and reduced graphene oxide (rGO) . The syn2 and sp3 carbon atoms. The mass spectrum of the residue showed peaks corresponding to ball-like polyhedral molecules which they called “buckyballs”. The most known is C60, named Buckminster fullerene . Th. Th38]. The heat-up approach B is a si2 hybridization (3) and outstanding mechanical, thermal and electrical properties which depend on their diameter, length and chirality  of the same length as the chromosome to be mutated is generated and a probability threshold is defined. Mask positions matching a value in the vector greater than the threshold write a “1” in the mask, and “0” the vector value is less than or equal to the threshold.ip in generation i, always variable within the range [Minp − Maxp], where Minp is its minimum value and Maxp indicates the maximum mutation probability. MaxN corresponds to the maximum number of generations the algorithm can run.For real, integer or finite coding, we have non-uniform mutation and modified non-uniform mutation. The non-uniform qualifier refers to the fact that the probability of mutation is variable, depending on the number of generations elapsed. The probability of mutation decreases as generations progress. This aspect is similar to the simulated annealing process, in which the system freezes as the algorithm progresses. The following equation allows us to calculate the mutation probability ip, increasing with the number of generations i.The modified non-uniform mutation is the opposite of the non-uniform mutation in the sense that in this case the probability of mutation increases as the algorithm progresses. This method is more effective against the problem of premature convergence. As the algorithm progresses, diversity tends to decrease and the algorithm focuses on smaller solution areas. The modified non-uniform mutation allows jumps to new, unexplored spaces, avoiding convergence to some local optimum. The equation expresses the calculation of the probability of mutation Swap mutation involves exchanging the values of randomly selected genes.Adjacent genes swap mutation selects two consecutive genes on the chromosome and reverses their order.In inversion mutation, two random positions on the chromosome are chosen and the gene values are swapped between them. Suppose the positions are i and j. The in-terchange operation exchanges the value of gene i with j and vice versa, exchanging genes (i + 1) and (j − 1), (i + 2) with (j − 2) and so on until the interval [i–j] is covered, the last genes exchanged being (i + n) and (j − n), with n = (j − i + 1)/2.In shift mutation, a set of consecutive genes to be altered and a direction of displacement (to the right or to the left) is determined. The process consists of each gene in the interval moving to occupy the immediate position until the last gene affected is made to correspond to the first position of the same group of genes. The effect is that the values of all affected genes are shifted one position on the chromosome.For order-based coding, there are mainly four types of mutation, depending on which genes are altered and how they change: swap mutation of any two genes, swap of two adjacent genes, random swap of a set of consecutive genes, or swap of a set of consecutive genes by applying a defined shift to all of them:0, its neighbourhood is explored by searching for a solution s1 close to and better than s0. The neighbourhood of a solution is formed by the set of solutions that are reached with a small movement or modification in the solution. Solution s1 is better than s0 if its fitness is closer to the optimum than fitness of the other. The main problem with this technique is the great ease with which the algorithm gets trapped in local optima. There are several possibilities to avoid this outcome:Extending the size of the neighbourhood.Limiting the number of search iterations.Repeating the search algorithm with different starting solutions.Local search is a frequently used method in solving combinatorial optimisation problems. Given a solution sNone of them on their own has produced satisfactory results. What generates the best results is to combine the local search technique with other types of mechanisms in what are called hybrid algorithms.GA is a common choice for global optimization and has been used to search polymer space . It comp2 or cellulose nanocrystals. Mechanical properties such as Young’s modulus, tensile strength and fracture strength were optimized with minimum errors and regression values above 95%. This study shows promising results for optimizing the amount and type of nanofillers to be added to polymeric matrices in order to improve mechanical durability.Rabothata et al.  used GA xi), and the covariance, c , where i, j vary from 1 to n. The parameter n is the number of data points and x is the input vector. The GP model can be expressed as:Y(x) is the output of interest and err(x) is the error related with the dataset noise. The GP does not need setting a hypothesis and finding suitable values for the weights in the framework. It generates a distribution of all the potential functions that are someway consistent with the training data.Gaussian process (GP) is a non-parametric stochastic algorithm used for solving non-linear problems . It is oWang et al.  investigHansoge et al.  applied Qin et al.  studied The outstanding multi-functional properties of polymeric nanocomposites have made them perfect candidates for a wide number of applications including aerospace, automobile, marine, civil, and many other technologically demanding industries. The increasing request for these nanocomposites demands a comprehensive investigation of their physical, chemical and mechanical behaviour under different environmental conditions. The physical properties, size and shape of the nanofiller and the microstructure of the nanocomposites are important factors that condition their final properties. It requires a lot of time and energy to find high-performance nanocomposites in thousands of combinations, and this process is very hard and long. The ML approach, trained on enormous amounts of data, has been demonstrated to be a very powerful predictive tool for data-driven multi-physical modelling, leading to unique insights and the exploration of their properties beyond the skill of conventional computational and experimental analyses. Recent studies have demonstrated its usefulness for structure–property linkage analysis and for speeding up the design of polymeric nanocomposites. Different ML algorithms including ANN, ANFIS, MLP, CNN, GA, GP, etc., have been successfully applied to create a mapping between the fingerprinted input and the target property. The results demonstrate a very good correlation between the predicted properties and the experimental values, with correlation factors higher than 0.9. Therefore, studies prove that ML algorithms have many advantages versus conventional computing in terms of resolution and cost-effectiveness; they can achieve greater accuracy in the predicted properties, require less expert analysis and fine-tuning and provide superior flexibility since they can be re-trained using a custom dataset for any use case.Over the past years, most ML-driven approaches were applied to neat polymers and fibre-reinforced composites. Until 2020, less than 100 papers on ML applied to polymer nanocomposites were reported. However, in the past two years, over 200 studies dealing with polymer nanocomposites have been published, many of them related to the prediction of mechanical properties. The prediction accuracy and generalization of ML models are strongly correlated with the quantity and quality of samples in the dataset, and these data are still limited for polymer nanocomposites. Thus, only a few online specific databases such as NanoMine have been built . This prOther options are currently been explored. In particular, hybrid machine learning can be applied for the property prediction of polymer nanocomposites. It is based on the idea of combining multiple ML algorithms to increase the overall prediction capability by tuning mutually and generalizing or adapting to unseen data . Ensembl


Most existing TWAS tools require individual-level eQTL reference data and thus are not applicable to summary-level reference eQTL datasets. The development of TWAS methods that can harness summary-level reference data is valuable to enable TWAS in broader settings and enhance power due to increased reference sample size. Thus, we develop a TWAS framework called OTTERS (Omnibus Transcriptome Test using Expression Reference Summary data) that adapts multiple polygenic risk score (PRS) methods to estimate eQTL weights from summary-level eQTL reference data and conducts an omnibus TWAS. We show that OTTERS is a practical and powerful TWAS tool by both simulations and application studies. Here, the authors present a TWAS framework OTTERS that adapts multiple polygenic risk score methods to estimate eQTL weights from summary-level eQTL data. Both simulation and real studies show OTTERS is powerful across a wide range of genetic architectures. Researchers have successfully deployed TWAS analyses to identify risk genes for complex human diseases, including Alzheimer’s disease8, breast cancer11, ovarian cancer13, and cardiovascular disease15. A typical TWAS consists of two separate stages. In Stage I, TWAS acquires individual-level genetic and expression data from relevant tissues available in a reference dataset like the Genotype-Tissue Expression (GTEx) project17 or the North American Brain Expression Consortium18, and trains multivariable regression models on the reference data treating gene expression as outcome and SNP genotype data  as predictors to determine genetically regulated expression (GReX). After Stage I that uses the GReX regression models to estimate effect sizes of SNP predictors that, in the broad sense, are effect sizes of expression quantitative trait loci (eQTLs), Stage II of TWAS proceeds by using these trained eQTL effect sizes to impute GReX within an independent GWAS of a complex human disease or trait. One can then test for association between the imputed GReX and phenotype, which is equivalent to a gene-based association test taking these eQTL effect sizes as corresponding test SNP weights21.Transcriptome-wide association study (TWAS) is a valuable analysis strategy for identifying genes that influence complex traits and diseases through genetic regulation of gene expression19, FUSION20, and TIGAR22. While these methods all employ different techniques for model fitting, they all require individual-level reference expression and genetic data to estimate eQTL effect sizes for TWAS. Therefore, these methods cannot be applied to emerging reference summary-level eQTL results such as those generated by the eQTLGen23 and CommonMind24 consortia, which provide eQTL effect sizes and p values relating individual SNPs to gene expression. The development of TWAS methods that can utilize such summary-level reference data is valuable to permit the applicability of the technique to broader analysis settings. Moreover, as TWAS power increases with increasing reference sample size25, TWAS using summary-level reference datasets can lead to enhanced performance compared to using individual-level reference datasets since the sample sizes of the former often are considerably larger than the latter. For example, the sample size of the summary-based eQTLGen reference sample is 31,684 for blood, whereas the sample size of the individual-level GTEx V6 reference is only 338 for the same tissue. Consequently, TWAS analysis leveraging the summary-based eQTLGen dataset as a reference can likely provide insights into the genetic regulation of complex human traits.For Stage I of TWAS, a variety of training tools exist for fitting GReX regression models using reference expression and genetic data, including PrediXcan31 that can predict phenotype in a test dataset using summary-level SNP effect size estimates and p values based on single SNP tests from an independent reference GWAS. We can adapt these PRS methods for TWAS since eQTL effect sizes are essentially SNP effect sizes resulting from a reference “GWAS” of gene expression. Thus, our predicted GReX in Stage II of TWAS is analogous to the PRS constructed based on training GWAS summary statistics of single SNP-trait association. Here, we adapt four representative summary-data-based PRS methods—p value thresholding with linkage disequilibrium (LD) clumping (P+T)26, frequentist LASSO32 regression-based method lassosum27, nonparametric Bayesian Dirichlet Process Regression (DPR) model-based33 method SDPR29, and Bayesian multivariable regression model-based method with continuous shrinkage (CS) priors PRS-CS28 for TWAS analysis. We apply each of these PRS methods to first train eQTL effect sizes based on a multivariable regression model from summary-level reference eQTL data (Stage I), and subsequently use these eQTL effect sizes  to impute GReX and then test GReX-trait association in an independent test GWAS (Stage II).In this work, we propose a framework that can use summary-level reference data to train GReX regression models required for Stage I of TWAS analysis. Our method is motivated by a variety of published polygenic risk score (PRS) methodsp values using eQTL weights trained by each of the PRS techniques outlined above, and then calculates an omnibus test p value using the aggregated Cauchy association test34 (ACAT-O) with all individual TWAS p values . OTTERS first constructs individual TWAS tests and ues Fig. . OTTERS 20 approach across all simulation settings considered. Many of the individual PRS methods also outperform FUSION in these settings. We then apply OTTERS to blood eQTL summary-level data  from the eQTLGen consortium23 and GWAS summary data of cardiovascular disease from the UK Biobank (UKBB)35. By comparing OTTERS results to those of FUSION20 using individual-level GTEx reference data of whole blood tissue, we demonstrate that OTTERS using large summary-level reference datasets and multiple gene expression imputation models can successfully reveal potential risk genes missed by FUSION based on smaller individual-level reference datasets and only one model. Finally, we conclude with a discussion.In subsequent sections, we first describe how to use the PRS methods on summary-level reference eQTL data in Stage I TWAS, and then describe how we can use the resulting eQTL weights to perform Stage II TWAS using OTTERS. We then evaluate the performance of individual PRS methods and OTTERS using simulated expression and real genetic data based on patterns observed in real datasets. Interestingly, when we assume individual-level reference data are available, we observe that OTTERS outperforms the popular FUSIONZ-score statistics proposed by FUSION20 and S-PrediXcan36 (see details in Methods).For the standard two-stage TWAS approach, Stage I estimates a GReX imputation model using individual-level expression and genotype data available from a reference dataset, and then Stage II uses the eQTL effect sizes from Stage I to impute gene expression (GReX) in an independent GWAS and test for association between GReX and phenotype. GReX for test samples can be imputed from individual-level genotype data and eQTL effect size estimates. When individual-level GWAS data are not available, one can instead use summary-level GWAS data for TWAS by applying the TWAS p values (based on a single variant test) from the eQTL summary data as well as a reference LD panel from samples of the same ancestry29. Although all PRS methods are applicable to TWAS Stage I, we only consider four representative methods—P+T26, Frequentist lassosum27, Nonparametric Bayesian SDPR29, Bayesian PRS-CS28 (see details in Methods).Since eQTL summary data are analogous to GWAS summary data where gene expression represents the phenotype, we can follow the idea from PRS methods to estimate the eQTL effect sizes based on a multivariable regression model using only marginal least squared effect estimates and g using P+T, lassosum, SDPR, and PRS-CS methods that each infers cis-eQTL weights using cis-eQTL summary data and an external LD reference panel of the same ancestry (Stage I). Once we derive cis-eQTL weights for each training method, we can impute the respective GReX using that method and perform the respective gene-based association analysis in the test GWAS dataset. We thus derive a set of TWAS p values for gene g, one per training method. We then use these TWAS p values to create an omnibus test using the ACAT-O34 approach that employs a Cauchy distribution for inference (see details in Supplementary Methods). We refer to the p value derived from ACAT-O test as the OTTERS p value. The ACAT-O34 approach has been widely used in hypothesis testing to combine multiple testing methods for the same hypothesis39, which has been shown as an effective approach to leverage different test methods to increase the power while still managing to control for type I error. Adding TWAS p values based on additional PRS methods to the ACAT-O test can possibly improve the power further at the cost of additional computation.As shown in Fig. 41 and Mount Sinai Brain Bank (MSBB) study42 for simulation. We divided 14,772 genes into five groups according to gene length, and randomly selected 100 genes from each group . We randomly split samples into 568 training (30%) and 1326 testing samples (70%) to mimic a relatively small sample size in the real reference panel for training gene expression imputation models. From the real genotype data, we simulated six scenarios with two different proportions of causal cis-eQTL, We used real genotype data from 1894 whole-genome sequencing (WGS) samples from the Religious Orders Study and Rush Memory and Aging Project (ROS/MAP) cohortZ-scores to perform a total of 50,000 TWAS simulations. We generated the GWAS Z-scores from a multivariate normal distribution with 38, where We generated gene expression of gene Z-scores to conduct a gene-based association test. In addition to gene-based association tests based on eQTL weights per training method, we further constructed the corresponding OTTERS p values. We evaluated the performance of the training methods with test samples, comparing test p value In Stage I of our TWAS analysis, we applied P+T (0.001), P+T (0.05), lassosum, SDPR, and PRS-CS methods to estimate eQTL weights using eQTL summary data and the reference LD of training samples. In Stage II of the TWAS, we used the estimated eQTL weights and the simulated GWAS p values based on the four PRS training methods together for analysis in our OTTERS framework obtained the highest power across all scenarios.As shown in Fig. p values in Supplementary Fig. p values when the correlation among tests is strong34. This suggests that modest OTTERS p values may be interpreted with caution.To evaluate the type I error of the individual PRS methods along with OTTERS, we picked one simulated replicate per gene from the scenario with We also compared the performance of our individual PRS training methods to those of FUSION, assuming individual-level reference data were available for the latter method to train GReX models. As shown in Fig. ons Fig. . These s23 with n = 31,684 blood samples, to train GReX imputation models for 16,699 genes. For test data, we downloaded the transcriptomic data of 315 blood tissue samples that are in GTEx V8 but were not part of GTEx V6 (as GTEx V6 samples contributed to the reference eQTLGen consortium summary data). For these 315 samples, we compared imputed GReX to observed expression levels. We considered trained imputation models with test 43. We also compared the imputation accuracy of these five training models to those using FUSION based on a smaller individual-level training dataset . For such models, we compared the test To evaluate the imputation accuracy of P+T (0.001), P+T (0.05), lassosum, SDPR, and PRS-CS methods in real data, we applied these training methods to summary-level eQTL reference data from the eQTLGen consortiumWe observed that PRS-CS obtained the most “valid” GReX imputation models with test mentclass2pt{minimmentclass2pt{minimPR Table .Table 1TBy comparing test ods Fig. , we obse23 reference data and reference LD from GTEx V8 WGS samples44, we applied our OTTERS framework to the summary-level GWAS data of Cardiovascular Disease from UKBB 35 (see Methods). We performed TWAS of cardiovascular disease for 16,678 genes. First, for each gene, we obtained TWAS p values per individual training method (P+T (0.001), P+T (0.05), lassosum, SDPR, and PRS-CS). Second, we performed genomic control45 for TWAS test statistics generated under each specific training model, by scaling all test statistics to ensure that the median test p value equals to 0.5. Last, we only considered genes with test GReX p values across all five training models using ACAT-O to obtain our OTTERS test statistics and p values. Genes with OTTERS p values < Using the eQTL weights trained by P+T (0.001), P+T (0.05), lassosum, SDPR, and PRS-CS methods with the eQTLGenp value as the independently significant gene. OTTERS obtained 38 independently significant TWAS genes . Of the 38 independent genes that OTTERS identified from TWAS with eQTLGen reference data of whole blood, FUSION only identified 8 of these genes  when using the GTEx V6 reference data of the same tissue. FUSION did identify 13 additional OTTERS genes , when considering all available tissue types in GTEx V6 reference data. These genes were identified by FUSION when considering the GTEx V6 reference data of artery, thyroid, adipose visceral, nerve tibial tissues, etc. For example, the most significant gene FES  obtained by FUSION using the same UKBB GWAS summary data of cardiovascular disease but using a smaller individual-level reference expression dataset from GTEx V6  that were not detected by FUSION using GTEx V6 data, where EDN3  was shown to be active in the cardiovascular system and play an important role in the maintenance of blood pressure or generation of hypertension47.Our OTTERS method also identified 17 genes , P+T (0.05), lassosum, SDPR, and PRS-CS for three example genes that were only detected by one or two individual methods  from the UKBB GWAS summary statistics and shape coded with respect to the direction of UKBB GWAS Z-score statistics. Generally, significant TWAS p values would be obtained by methods that obtained eQTL weights with relatively large magnitudes for SNPs with relatively more significant GWAS p values.By comparing OTTERS results with the ones obtained by individual methods Table , we founSIDT2, which was a significant risk gene identified by both PRS-CS and SDPR, and had p values < EDN3, which was only identified by P+T methods , even with a more stringent rule  Xeon(R) processor (2.10 GHz). The evaluation was based on 1000 MCMC iterations for SDPR and PRS-CS (default) without parallel computation , the frequentist method lassosum27, as well as the Bayesian methods PRS-CS28 and SDPR29 within our omnibus test for optimal inference. We note that additional PRS methods such as MegaPRS30 or PUMAS31 could also be implemented as additional OTTERS Stage I training methods. Higher TWAS power might be obtained by adding more PRS methods in OTTERS Stage I, with additional computation cost. We also note that the existing SMR-HEIDI53 method, which uses summary-level data from GWAS and eQTL studies to test for possible causal genetic effects of a trait of interest that were mediated through gene expression, could also be used as an alternative method besides TWAS. However, the SMR method generally restricts eQTL for consideration, excluding those where the eQTL p values larger than a certain threshold, e.g., 0.05.Our OTTERS framework represents an omnibus TWAS tool that can leverage summary-level expression and genotype results from a reference sample, thereby robustly expanding the use of TWAS into more settings. To this end, we adapted and evaluated five different PRS methods assuming different underlying genetic models, including the relatively simple method P+TIn simulation studies, we demonstrated that the performance of each of these five PRS methods depended substantially on the underlying genetic architecture for gene expression, with P+T methods generally performing better for sparse architecture, whereas the Bayesian methods performing better for denser architecture. Consequently, since the genetic architecture of gene expression is unknown apriori, we believe this justifies the use of the omnibus TWAS test implemented in OTTERS for practical use, as this test had near-optimal performance across all simulation scenarios considered. While we developed our methods with summary-level reference data in mind, we note that our prediction methods and OTTERS perform well (in terms of imputation accuracy and power) relative to existing TWAS methods like FUSION when individual-level reference data are available.In our real data application using UKBB GWAS summary-level data, we compared OTTERS TWAS results using reference eQTL summary data from eQTLGen consortium to FUSION TWAS results using a substantially smaller individual-level reference dataset from GTEx V6. OTTERS identified 13 significant TWAS risk genes that were missed by FUSION using individual-level GTEx V6 reference data of blood tissue, suggesting that the use of larger reference datasets like eQTLGen in TWAS can provide additional findings. Interestingly, the genes missed by FUSION were instead detected using individual-level GTEx reference data of other tissue types that are more directly related to cardiovascular disease. By comparing OTTERS to FUSION when the same individual-level GTEx V8 reference data of whole blood samples were used, we still observed that OTTERS identified more risk genes than FUSION, which we believe is due to the former method accounting for the unknown genetic architecture of gene expression by using multiple regression methods to train GReX imputation models. These applied results were consistent with our simulation results.Among all individual methods, P+T is the most computationally efficient method. The Bayesian methods SDPR and PRS-CS require more computation time than the frequentist method lassosum as the former set of methods require a large number of MCMC iterations for model fit. By comparing the performance of these five methods in terms of the imputation accuracy and TWAS power in simulations and real applications, we conclude that none of these methods was optimal across different genetic architectures. We found that all methods provided distinct and considerable contributions to the final OTTERS TWAS results. These results demonstrate the benefits of OTTERS in practice, since OTTERS can combine the strength of these individual methods to achieve optimal performance.p values from individual training methods. Since the existing tools for P+T, lassosum, SDPR, and PRS-CS were originally developed for PRS calculations, we adapted and optimized them for training GReX imputation models in our OTTERS tool. For example, we integrate TABIX54 and PLINK55 tools in OTTERS to extract input data per target gene more efficiently. We also enable parallel computation in OTTERS for training GReX imputation models and testing gene-trait association of multiple genes.To enable the use of OTTERS by the public, we provide an integrated tool (see Code Availability) to (1) Train GReX imputation models  using eQTL summary data by P+T, lassosum, SDPR, and PRS-CS; (2) Conduct TWAS  using both individual-level and summary-level GWAS data with the estimated eQTL weights; and (3) Apply ACAT-O to aggregate the TWAS 56. This might limit the transferability of our TWAS results across populations. Third, our OTTERS cannot provide the direction of the identified gene-phenotype associations, which should be referred to as the sign of the TWAS Z-score statistic per training method. Last, even though the method applies to integrate both cis- and trans- eQTL with GWAS data, the computation time and availability of summary-level trans-eQTL reference data are still the main obstacles. Our current OTTERS tool only considers cis-eQTL effects. Extension of OTTERS to enable cross-population TWAS and incorporation of trans-eQTL effects is part of our ongoing research but is out of the scope of this work.The OTTERS framework does have its limitations. First, training GReX imputation models by all individual methods on average cost ~20 min for all five training models per gene, which might be computationally challenging for studying eQTL summary data of multiple tissue types and for ~20K genome-wide genes. Users might consider prioritizing P+T (0.001), lassosum, and SDPR training methods, as these three provide complementary results in our studies. Second, the currently available eQTL summary statistics are mainly derived from individuals of European descent. Our OTTERS trained GReX imputations model based on these eQTL summary statistics, and the resulting imputed GReX could consequently have attenuated cross-population predictive performance57, and epigenome-wide association studies using summary-level reference data of methylation-phenotype relationships reported by Genetics of DNA Methylation Consortium (GoDMC) (see Data availability). OTTERS would be most useful for broad researchers who only have access to summary-level QTL reference data and summary-level GWAS data. The feasibility of integrating summary-level molecular QTL data and GWAS data makes our OTTERS tool valuable for wide application in current multi-omics studies of complex traits and diseases.Our OTTERS framework using large-scale eQTL summary data has the potential to identify more significant TWAS risk genes than standard TWAS tools that use smaller individual-level reference transcriptomic data and use only a single regression method for training GReX imputation models. This tool provides the opportunity to leverage not only available public eQTL summary data of various tissues for conducting TWAS of complex traits and diseases, but also the emerging summary-level data of other types of molecular QTL such as splicing QTLs, methylation QTLs, metabolomics QTLs, and protein QTLs. For example, OTTERS could be applied to perform proteome-wide association studies using summary-level reference data of genetic-protein relationships such as those reported by the SCALLOP consortiumStage I of TWAS estimates a GReX imputation model using individual-level expression and genotype data available from a reference dataset. Consider the following GReX imputation model from 22. Once we configure the model in Eq. . For the remaining selected test SNPs, we use marginal standardized eQTL effect sizes from eQTL summary data as eQTL weights for TWAS in Stage II. We considered 55 (see Code availability). We denote the P+T method with P+T=Frequentist lassosum: with standardized del (Eq. ) is \doc58, and an identity matrix 32 penalized loss function, the frequentist lassosum method27 can tune the LASSO penalty parameter and By approximating 33 as implemented in TIGAR22 estimates 5929 assumes the same DPR model but can be applied to estimate the eQTL effect sizes Bayesian SDPR: Bayesian DPR methodmentclass2pt{minim28 assumes the following normal prior for p values) and reference LD correlation matrix Bayesian PRS-CS: the PRS-CS methodmentclass2pt{minimg using P+T, lassosum, SDPR, and PRS-CS methods that each infers cis-eQTLs weights using cis-eQTL summary data and an external LD reference panel of similar ancestry (Stage I). Once we derive cis-eQTLs weights for each training method, we can impute the respective GReX using that method and perform the respective gene-based association analysis in the test GWAS dataset using the formulas given in Eq. . We refer to the p value derived from ACAT-O test as the OTTERS p value.As shown in Fig. n in Eq.  . Default settings were used to train GReX imputation models by FUSION in our simulation and real studies. LASSO regression was performed only for genes with positively estimated expression heritability. The eQTL weights of the best-trained GReX imputation model will be used to conduct TWAS by FUSION.To train GReX imputation models by FUSION with individual-level reference data, we trained Best Linear Unbiased Predictor modeln = 315) were used to test GReX imputation accuracy in our studies. GTEx V8 RNA-seq data of all whole blood samples (n = 574) were also used as reference data for comparing the performance of OTTERS and FUSION.GTEx V8 dataset (dbGaP phs000424.v8.p2) contains comprehensive profiling of WGS genotype data and RNA-sequencing (RNA-seq) transcriptomic data across 54 human tissue types of 838 donors. The GTEx V8 WGS genotype data of all samples were used to construct reference LD in our studies. The GTEx V6 RNA-seq data of whole blood samples were used to train GReX imputation models by FUSION, and the GTEx V8 RNA-seq data of additional whole blood samples  including GTEx V6 as a sub-cohort. eQTLGen samples consist of 25,482 blood (80.4%) and 6202 peripheral blood mononuclear cell (19.6%) samples. We considered SNPs with minor allele frequency > 0.01, Hardy–Weinberg p value > 0.0001, call rate > 0.95, genotype imputation 23. We only considered cis-eQTL (within ±1 MB around gene transcription start sites) with a test sample size > 3000. As a result, we used cis-eQTL summary data of 16,699 genes from eQTLGen to train GReX imputation models for use in OTTERS in this study.The eQTLGen consortiumn = 459,324, case fraction = 0.319)35 were generated by BOLT-LMM based on the Bayesian linear mixed model per SNP63 with assessment centered, sex, age, and squared age as covariates. Although BOLT-LMM was derived based on a quantitative trait model, it can be applied to analyze case–control traits and has a well-controlled false-positive rate when the trait is sufficiently balanced with a case fraction ≥10% and samples are of the same ancestry. The tested dichotomous cardiovascular disease phenotype includes a list of sub-phenotypes: hypertension, heart/cardiac problem, peripheral vascular disease, venous thromboembolic disease, stroke, transient ischemic attack (tia), subdural hemorrhage/hematoma, cerebral aneurysm, high cholesterol, and other venous/lymphatic diseases.Summary-level GWAS data of Cardiovascular Disease from UKBB (Further information on research design is available in the Supplementary InformationPeer Review FileReporting Summary