Disease classification based on machine learning has become a crucial research topic in the fields of genetics and molecular biology. Generally, disease classification involves a supervised learning style; i.e., it requires a large number of labelled samples to achieve good classification performance. However, in the majority of the cases, labelled samples are hard to obtain, so the amount of training data are limited. However, many unclassified (unlabelled) sequences have been deposited in public databases, which may help the training procedure. This method is called semi-supervised learning and is very useful in many applications. Self-training can be implemented using high- to low-confidence samples to prevent noisy samples from affecting the robustness of semi-supervised learning in the training process. The deep forest method with the hyperparameter settings used in this paper can achieve excellent performance. Therefore, in this work, we propose a novel combined deep learning model and semi-supervised learning with self-training approach to improve the performance in disease classification, which utilizes unlabelled samples to update a mechanism designed to increase the number of high-confidence pseudo-labelled samples. The experimental results show that our proposed model can achieve good performance in disease classification and disease-causing gene identification. Recently, bioinformatics technologies have provided efficient ways to diagnose diseases, and machine learning methods applied in bioinformatics have achieved remarkable breakthroughs in the field of disease diagnosis . DiseaseHowever, disease classification using gene expression data also faces challenges because of the characteristic high dimensions and small sample sizes . GeneralMany regularization methods have been proposed to identify significant genes to achieve high-performance disease diagnosis. Regularization methods have recently attracted increased attention in gene selection and have become a key technique to prevent over-fitting . For exanal data ,23. Resenal data  and infenal data . Stable nal data . TherefoThe rest of this paper is structured as follows. Machine learning methods for disease diagnosis can be traced back to the 1990s . Since tSemi-supervised learning, an active research topic in machine learning in recent years, aims to label an amount of unlabelled data to improve the performance of a model. Many recent successful examples of semi-supervised learning in bioinformatics have been presented. For example, a semi-supervised network to solve the high-dimensional problem of identifying known essential disease-causing genes has been proposed . Chai etp < 1) [p < 1) penalties and revealed the extreme importance and special role of the p < 1) penalty can provide a different sparsity evaluation and that the q < 2) penalty can provide a grouping effect with different q values.From a biological point of view, most likely only a few genes can strongly indicate targeted diseases, and most genes are irrelevant to cancer classification. The irrelevant genes may introduce noise and reduce the classification accuracy. Given the importance of these problems, effective gene selection methods can help classify different types of cancer and improve the prediction accuracy . Stabili< p < 1) . A multiunctions . Zeng etunctions  investigd Hastie  indicatey value is closer to 0 or 1 for the logistics regression, the smaller the loss value. These values represent high-confidence samples.The deep forest approach provides an alternative to deep neural networks (DNNs) to learn super-hierarchical representations at low cost. n samples, where w is a model parameter. w and latent sample weights v, the minimization over w is a weighted loss minimization problem, independent of regularizer Consider a pseudo-labelled training dataset https://www.ncbi.nlm.nih.gov/geo)are utilized. A brief description of these datasets is shown in In this study, three public cancer datasets from the National Center for Biotechnology Information, U.S. National Library of Medicine (https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE4115) is from Boston University Medical Center. The numbers of lung cancer and healthy samples are 97 and 90, respectively, and each sample contains 22,215 genes.The lung cancer dataset (GSE4115) (https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE21050) from the French Institut Bergonie contains 310 samples, which consist of 183 lung cancer and 127 normal lung samples, with 54,677 genes as the model input.The breast cancer dataset (GSE21050) (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1524991/) is from the MIT Whitehead Institute. After preprocessing, the prostate dataset contains 102 samples and 12,600 genes in two classes, tumour and normal, which account for 52 and 50 samples, respectively.The prostate cancer dataset , logistic regression (LR), support vector machine (SVM) and random forest (RF). In the experiments, a portion of the three disease datasets is treated as unlabelled samples to assess the classification accuracy of the proposed method. The labelled and unlabelled samples are randomly selected in every run of the program. The classification performance achieved by the various methods for the three datasets is shown in p-values indicate that the results are significant. Many studies consider function analysis for gene expression. For example, USP6NL in To further illustrate the performance of our method in computer-aided diagnosis, stable LASSO is used in this work . The topMeanwhile, the heat map correlation between the genes is illustrated in In this paper, we proposed deep forest and semi-supervised with self-training  to solve disease classification and gene selection problem based on different types of diseases. The deep forest method is consistently superior to other conventional classification methods, possibly because the deep forest approach learns more significant advanced features in the learning process. Semi-supervised learning provides an effective alternative to alleviate the challenges of over-fitting and improves the robustness of the model in the experimental process. Improved experimental results can be obtained by combining semi-supervised learning and the deep forest model. By simultaneously considering all classes during the gene selection stages, our proposed extensions identify genes leading to more accurate computer-aided diagnosis by doctors.In the experiments, we used datasets for three types of diseases to assess and investigate the performance of our method using trained from 10-fold cross-validation and different sizes datasets. The results show that our proposed disease classification approach has achieved higher prediction accuracy than other methods published in the literature. However, the relevance threshold is different in the context of classification performance when the number of training instances is small. Therefore, how to determine the relevance threshold in the adaptive problem will be a focus of our work in the future. Additionally, we believe that our mechanism can also be applied to other types of disease diagnosis problems and can be expanded to various classifications of disease states.


Large-scale transcriptomic data is used by biologists for the discovery of new molecular patterns or cell subpopulations. Clustering is one of the most popular methods for dimensionality reduction and data analysis for large scale datasets. The major problem while clustering the data is the selection of the optimal number of clusters (k) for each dataset and to discover new insights from it. We have developed Recursive Consensus Clustering (RCC), an unsupervised clustering algorithm for novel subtype discovery from both bulk and single-cell datasets. RCC is available as an R package and facilitates the generation of new biological insights through intuitive visualization of clustering results. Clustering of datasets with an unknown number of clusters: Algorithmically identifying the optimal number of clusters in a dataset is a difficult mathematical problem especially for big datasets with a large number of clusters1. For example, algorithms like tSNE + k-means2 or hierarchical clustering require the user to input k for a given dataset which might not always be optimal. This also restricts the discovery of novel subtypes in a given dataset.Finding novel subtypes with user-friendly tools: Subtypes that are not known beforehand are likely to be missed when applying popular clustering methods like k-means, hierarchical clustering, pam, mclust3, etc.4 As an example, consider large-scale TCGA pan-cancer dataset5 which includes samples from multiple cancer types  with each cancer type having distinct molecular subtypes. Two major publications that analyzed this transcriptome data were able to largely find clusters with tissue-specific cancers and not the subtypes within each cancer type with clinical relevance6. The novel subtypes were discovered only after using sophisticated integrated analysis of multi-omic data.Recent advances in the field of RNA sequencing has resulted in a wealth of data which allows us to classify and study the transcriptomic subtypes/cell types in different biological systems. The clustering of transcriptomic data reduces the dimensionality of the data and allows a researcher to better analyze, visualize and interpret the data for biological insight. Clustering is an unsupervised technique that allows the grouping of similar objects and enables division of data. Though a widely used technique, researchers face the following challenges in performing clustering on transcriptomic data:https://github.com/MSCTR/RecursiveConsensusClustering). The recursive clustering of the dataset reveals finer structures in the data leading to the identification of novel subtypes. RCC uses ConsensusClusterPlus7 (CCP) as the base algorithm for dividing the data. Most of the clustering algorithms are heuristic in nature and the clusters obtained using them depend on the initial seed value8. As a result, different runs of the same algorithm on the same dataset yield different clusterings. To deal with this issue the concept of consensus clustering was developed where you run the algorithm multiple times and take the consensus of clusters from those runs as your final result. CCP, an R package for consensus clustering uses n iterations of any one of the clustering algorithms  to divide the data into various subgroups. For each of the n iterations, CCP repetitively takes a subset of samples/cells and a subset of genes/features to give a consensus of these repetitions, which results in a more robust clustering output relative to the clusters obtained through a single iteration.We have developed Recursive Consensus Clustering (RCC), a user-friendly R package that allows a researcher to find novel and biologically meaningful subtypes in transcriptomic datasets without requiring computational expertise . Annotation matrix file for samples/cells can be provided optionally.9, the brain tissue samples are further divided into 13 subtypes which include different sections of brain like cortex, basal ganglia, cerebellum, amygdala, cerebellar hemisphere, substantia nigra, hippocampus, hypothalamus and spinal cord. When we take genes which are variable across all the tissue types, the clustering algorithms are not able to divide the data at these sub-tissue levels  in R for k-means clustering of the datasets. CCP uses three input parameters; seed, pItem (proportion of samples), and pFeature (proportion of genes/transcripts), that determines the final clustering outcome. Depending on the type of dataset, a user may need to change the pItem and pFeature. To reduce user input and increase robustness, we use eight combinations of pItem/pFeature  plots indicative of cluster stability, produced by CCP, RCC finds the best ks10 for each parallel run, in turn, giving multiple k values that satisfy the best k selection criteria (Methods). The k with maximum frequency is selected as the optimal k for the clustering .The RCC algorithm is described in five steps as shown in Fig. 11, we run CCP eight times using a different random seed and different parameters for each run to get the most stable clusters (Methods).We have used k-means as the underlying clustering algorithm for RCC as we found it works well for bulk transcriptomic datasets of different sizes. In principle, any clustering algorithm can be substituted for k-means. As the result obtained from the k-means algorithm is highly dependent on initial seed value12, Biase13, and Pollen14 datasets, 100 times on the Darmanis16 dataset and 10 times on Human tissue9, TCGA pan-cancer5, and Neftel17 datasets. We calculated the Adjusted Random Index (ARI) using the mclust3 package in R of each run with that of one randomly selected run. ARI calculates the concordance between two cluster assignments. The cluster assignment can be based on the annotation/labels from the annotated datasets or the clusters obtained through RCC and other algorithms. Higher ARI indicates better concordance between two cluster assignments with value 1 being the perfect concordance between two cluster assignments. We see highly consistent clustering results between RCC runs with ARI ranging from 0.5 to 1  and SC318 (for single-cell dataset) packages which also have the feature of selecting the optimal k. We have also compared our algorithm against tSNE + k-means2 and hierarchical clustering  by giving it the k based on known annotation from the original publications  and then log2 transformed before running RCC. To benchmark RCC, we used the algorithms given in  for each dataset across all the algorithms is shown in Table We used the datasets mentioned in Table in Table  on the bi is the marker panel which shows the number of marker genes identified for each cluster at a given level. The first row is markers for clusters found at level one, the second one for markers found at level two, and so on. The color-scale is from white to red with white indicating zero markers found and bright red indicating the highest number of markers found. Grey color indicates that the particular cluster is not further divided and hence no new markers are found which are distinct from the previous level. The second-panel j is the RCC clustering level panel. It shows the cluster information for samples at each level starting with level one. The third panel k is the annotation panel. The annotation panel shows the distribution of samples across all the clusters. Due to the two-dimensional nature of the plot, it becomes easier to view the cluster assignments along with its annotation. The cluster annotation plot allows the user to view the cluster assignment of each sample based on a particular attribute. It shows the proportions of samples based on attributes across all the clusters allowing the user to view the most dominant attribute or label in a given cluster.Along with clustering, RCC package also provides the user with visualization tools for better interpretation. In case of large-scale datasets, it becomes difficult to view the cluster assignments along with the attributes of each sample/cell in the data. To make it easier to view the clustered data, RCC package provides the users with tracking and cluster annotation plots. The tracking plot makes it easy for the user to view the clustered data and interpret it compared to other visualization tools Figure . ColumnsThe main objective of developing RCC was to find novel subtypes automatically with minimal user input that can help in generating novel biological insights. We demonstrate this utility of RCC by applying it to seven different datasets of varied sizes and different complexities Table .Ivy GAP dataset: The Ivy GAP12 dataset contains the RNA-Seq profiles of anatomic structures in glioblastoma, grade IV brain cancer. These anatomic structures include Leading Edge (LE), Cellular Tumor (CT), Microvascular Proliferation (MVP), and Pseudopalisading Cells around Necrosis (PAN) which are described in detail in the original paper. RCC clusters not only represent all the known classes but also find two subsets of CT and PAN samples each  and trophectoderm (TE). Both tSNE + k-means and SC3 were not able to find distinct clusters of these subgroups whereas RCC was consistently able to divide these cells into two clusters at level 2, again highlighting the power of recursive clustering  originally derived from neonatal male human foreskin fibroblasts are consistently getting subdivided into three clusters across multiple runs  patient samples. We combined both the datasets which include 4055 cells and 15 known cell types in total. RCC found 43 clusters across these cell types . The dataset we used has 7930 cells and 31 different known  cell types. Neftel et al. showed that the malignant GBM cells are majorly found in four cellular states which are determined by six meta modules based on the following gene signatures; Hypoxia independent (MES1like) and hypoxia dependent (MES2like) mesenchymal related gene sets, astrocytic (AClike) marker gene set, oligodendroglial (OPClike) lineage marker gene set, and stem and progenitor cell signatures (NPC1like and NPC2like) as well as two cell cycling modules namely G1S and G2M. They concluded after extensive analysis that there are largely four cellular states of glioblastoma cells  and each patient has varying proportions of these states, the proportions which are influenced by genetics and microenvironment. At the gene expression level, the cells are more similar to each other based on their cellular state rather than based on their parent tumor. Results of RCC recapitulates these findings as can be seen in the pie charts demonstrating cells from each tumor clustering based on their meta module score   Fig. .Figure 9Large-scale transcriptomic data and especially single-cell sequencing have now become the mainstay tools for understanding molecular mechanisms in disease and developmental biology. With datasets ranging from a few hundred samples to thousands (millions in case of single-cell data), there is a significant challenge in analyzing these datasets to gain biological insights. Clustering is always used for such analyses but automatically identifying the underlying number of clusters remains a challenge. A typical pipeline that is used involves taking known annotation of samples and then selection of clusters using manual identification through visualization. The problem with this approach is 1. reliance on known information when the objective is to unearth new information 2. Partial exploration of data and 3. An extensive amount of work requiring computational expertise. To address these limitations of current clustering methods, we introduce the concept of recursive clustering with an approach to automatically identify the optimal number of clusters in transcriptome datasets. We show that using our algorithm, RCC, available as an R package, with just a few commands we can recapitulate findings obtained through multi-step complex analysis and also identify additional subtypes not described in the original studies. We can fully explore the datasets in just a few steps without the need for domain knowledge. We have compared our method to the most popular methods and shown that we are uniquely able to identify novel subtypes and interpret results conveniently by visualization of intuitive output plots. Also, it works equally well for both bulk and single-cell data.23 and geometric sketching for subsampling of data that preserves heterogeneity24, will be an essential approach to maximize the gain of biological insights from large-scale transcriptomic datasets.One of the limitations of our algorithm is computational time. This time is dependent on the underlying base algorithm which currently is consensus clustering with k-means. As new efficient clustering algorithms become available, k-means can be replaced by those for improved performance. To truly harness the power of single-cell transcriptomics for a better understanding of biology, the next step is the analysis of large integrated datasets with millions of cells. Our method, when combined with methods like scanorama for integration of multiple datasetsIn conclusion, RCC is a user-friendly data clustering algorithm that can be used on both bulk and single-cell transcriptomic datasets. RCC analysis facilitates novel subtype discovery in the transcriptomic data allowing the user to tease out the unknown finer structure in large datasets through intuitive visualization of clustering results.k selection 5) Recursive clustering and 6) Output. Each step is described in detail below.Data input: RCC takes an expression matrix (X) and configuration file as input. The expression matrix is a csv file where the columns represent samples/cells and the rows represent expression values for genes/transcripts. The input matrix should be normalized and log-transformed after adding a pseudo count of 1. Along with the expression matrix, the user also has the option of uploading a sample information file which contains clinical information or attributes for the samples/cells to be clustered. The configurable parameters with default values are discussed at the end of this section.Feature selection and data scaling: The current clustering algorithms either use all the available genes or top variant genes or top principal components (PCA) or Single-cell Hierarchical Poisson Factorization (ScHPF)25 factors which are low dimensional representations of the datasets. These methods give us only a global view of a dataset and may not reveal the finer subgroups within the dataset. In RCC, initially all samples and top n% variant genes are selected, then normalized and clustered as described in steps 3–4. Based on the previous level of clustering, the feature set changes for subsequent recursive runs. We tested multiple datasets with different top n% variant genes and found that the top 1% of variant genes work well for the single-cell transcriptomic data, whereas the top 3–5% genes work well for bulk tissue data. This is a user-defined parameter in RCC where the user can select top n% of variant genes/features to be selected for further clustering. For a bulk-transcriptome dataset, a minimum of 500 genes/features are required for further clustering. If the top n% variant genes/features are <500, RCC by default takes the top 500 variant genes for further analysis. For single-cell transcriptomic data the number of genes detected per cell can be as low as 200 hence no minimum criteria is applied. After subsetting the top n% variant genes/features, data scaling is done using the following formula:where X is the expression measure of a gene in a cell/sample, μ is the mean expression measure of a gene across all the cells/samples and σ is the standard deviation of expression measure of a gene across all the cells/samples.Parallel processing of ConsensusClusterPlus: The base algorithm used for RCC is ConsensusClusterPlus14 (CCP) which performs k-means clustering on the dataset. CCP is run in parallel eight times with a different seed each time for increased robustness. We chose eight parallel runs of RCC, with each run implementing n iterations of CCP as it reduced the processing time and improved the robustness of algorithm. The output of CCP includes stability evidence for a given number of clusters (k) and their assignments. CCP is run each time with the following parameters:maxK) for each run is calculated by taking the minimum between 10 and 1/10th of the number of samples. maxK changes for each recursive run of RCC. We limit the maxK to 10 because based on our testing, the best k selection is not reliable for larger k.Maximum number of clusters: The maximum number of clusters : As discussed earlier, RCC runs CCP in parallel eight times to get stable clusters. To enable selection of the most robust clusters, the pItem and pFeature values are changed for every run. The selected pItem values range from 0.6–0.9 whereas the pFeature values used are 0.8 and 1. Different combinations of the mentioned values are as shown in Table Clustering algorithm: RCC uses k-means algorithm with Euclidean distance. K-means is applied on the z-scored data matrix with the Hartigan and Wong algorithm. By default, the number of centers is set to 10 and the maximum number of iterations is set to 10^9 in the k-means function.Seed: The k-means clustering algorithm requires an initial seed number to generate clusters. The initial seed value is very crucial for the clustering of data as it effects the repeatability and reproducibility of the results. To find robust clusters RCC generates random seed values for all of the eight parallel runs of CCP.Optimalkselection: Based on the cumulative distribution function (CDF) plots produced by CCP, RCC finds the best k10 (Simulated data in methods) for each parallel process as described below in turn giving eight k values. The k with maximum frequency is selected as the optimal k for that run of RCC. An example CDF plot of consensus matrices for k = 2:6 is shown in Figure N samples for 100 runs of clustering is an N x N matrix with each cell containing the number of times the row and column samples cluster together. For a given consensus matrix M, CDF is defined over the range of 0 to 1.RCC algorithm has six major steps: 1) Data input 2) Feature selection and data scaling 3) Parallel processing of ConsensusClusterPlus 4) Optimal c varies from 0 to 100. The perfect CDF plot for a given k (number of clusters), where for every run we get the same result, will have a CDF line with slope 0° starting at c = 0 and ending at c = 99. The perfect CDF value will be:Consensus index k = 2,3,4,5,6. Top panel shows the consensus matrices for each k and the bottom panel shows the CDF plot with slope and line length calculations. The dot- dash line shows the maximum CDF value for each k.Figure ±0.5. We take all CDF values in that range and fit a line through it. If the slope of the line is > minimum slope threshold, we trim the line till we get the slope below threshold or the line length is smaller than the minimum line length.The clustering of real data typically does not result in perfect CDF value so we create an allowance of 22. This parameter helps in selecting only biologically meaningful clusters for a dataset. To find the clusters with biological significance, RCC calculates differentially expressed genes across all the clusters for a given run. The clustering is selected only if it has n% of genes up regulated with FDR <0.01 in at least one of the clusters.K-means clustering algorithm tends to find stable clusters even when the sample distribution is random but asymmetric resulting in sub-classification without biological relevancek based on the following criteria:Line length is > minimum line length (default: 30)Line slope is < minimum slope threshold (default: 10°)Intra cluster stability > 0.8Inter cluster overlap < 0.2Differentially expressed genes > minimum % of genes in at least one of the clusters k values are selected then we break the tie assigning weights to each k. We assign weights to each k if the slope of k is ≤ 5° and if line length ≥ 40. The k with maximum weight is selected as the best k .If multiple We select the best k = 2,3,4,5 satisfy the criteria a:d. All the ks also satisfied criteria e. Based on f, k = 3,4,5 is selected as the best k. We get similar values from the remaining eight runs which in turn gives us an array of best ks. The optimal k is selected out of these best ks based on frequency distribution. The best k with the highest frequency is selected as the optimal k.5.Recursive clustering: Once the optimal k for a particular level is selected, each of the subdivided datasets go on for further clustering. The samples/cells get recursively clustered until one of the following criteria is met:Number of samples/cells in the dataset is lesser than the minimum number of samples required for clusteringk is zero; I.e no best k is selected in all the parallel runs of RCC.Optimal 6.Output: Upon submission of an expression matrix and their respective annotations, RCC gives the following output:Cluster information file (ClusterInfo.csv): The basic output of RCC is the cluster information file which has the cluster assignment of all the samples in the submitted datasets at every level along with the final cluster assignment.Cluster annotation plot : The cluster annotation plot allows the user to view the cluster assignment of each sample based on a particular attribute using the clusterAnnotation function. The Cluster annotation plot shows the distribution of all the attributes (e.g. cell types or tissue types) across all the clusters. The columns represent the attributes present in the dataset and the rows represent the clusters : The trackingPlot function is a visualization tool that allows the user to view the clustered data in an easy and interpretable manner : The geneMarkers function in RCC calculates the specific genes which are significantly up regulated in the clusters at each level with FDR <0.01 and log2 fold change values> 1. These genes are the specific markers for their representative clusters. RCC calculates and plots the markers across the levels making it easier for the user to visualize the data clustering. In the marker plots, the columns represent the samples/cells and the rows represent the genes.SSGSEA analysis (ssgsea.pdf): To find the biological significance of the clusters, RCC allows the user to perform a single sample gene set enrichment analysis (SSGSEA)26 using the function ssgsea. This function allows the user to find the enrichment of particular gene sets in all the samples of the clusters. The user can input the gene sets of their importance in CSV format to perform the SSGSEA analysis. The SSGSEA heatmap shows the enrichment of all the gene sets across all the samples. In the plot, the columns represent the samples/cells and rows represent the gene sets. For example, Figure Cluster attribute enrichment analysis : When a sample information file that contains clinical information or attributes is provided the user can perform cluster attribute enrichment analysis using the function clusterAttr. This function implements Fisher’s exact test to find a significant correlation between the attributes  and clusters. This helps in understanding the biological/clinical significance of the cluster.Kaplan–Meier analysis : RCC allows the user to perform survival analysis for all the clusters using the function clusterSurvival.In the example Figure 27.Output of RCC analysis described in this manuscript are available as Supplementary Data. All the gene enrichment analysis was done using Toppfun.csv format where the first column indicates the parameter name and second column indicates the parameter value. The configurable parameters are:Input expression matrix.Input annotation file: This is an optional parameter. If one does not have any annotation for the sample dataset this field can be left as NA.Minimum slope threshold: The default value is 10°. Lower the threshold tighter the clustering. Recommended values are between 5–15°.Minimum number of samples required for clustering: This option allows the user to decide the minimum sample size that is sufficient for further sub grouping. The default value is 20. A smaller number will likely result in a larger number of clusters.Minimum line length: The default value is 30. Perfect clustering will have a line length of 99. Higher the value tighter the clusters. Recommended values are between 30–60.Percentage of genes/features to be used for clustering: We use the top 3–5% of variant genes in bulk data and the top 1% of variant protein-coding genes in single-cell data for clustering. For bulk data, the recommended values are between 2–5% and for single-cell data, it is 1–2%.Minimum percentage of genes that are differentially expressed: The default value for large datasets (i.e. datasets having more than 1000 cells/samples) is 20% and for small datasets, it is 10%.Type of dataset: RCC accepts both bulk as well as single-cell transcriptomic datasets. This parameter is used by RCC for determining feature selection criteria. If the data is bulk tissue, then a minimum of 500 or top n% genes/features (whichever is higher) are taken for clustering.Output directory: Absolute path to the folder where the user wants to output the resultsAlong with the input matrices, RCC also requires an input configuration file. This file allows the user to adjust the clustering parameters or use the default ones based on their requirements. The configuration file is in the28 package in R to check if RCC was able to select the best k. Two simulated matrices were generated where mat1 had five sub- groups and each subgroup had an exact number of samples in them, whereas mat2 had five subgroups with varying proportions of samples in them. The best k found by RCC is the same as true k in the simulated dataset  CPU E5–2630 v4 processor running at 2.20 GHz × 40 using 251.8 GiB of RAM, running Ubuntu version 16.04.Project name: RecursiveConsensusClusteringhttps://github.com/MSCTR/RecursiveConsensusClusteringProject homepage: Operating system: Platform independentProgramming language: RLicense: GPLAny restrictions to use by non-academics: Nonehttps://www.msctr.org/2019/05/30/recursive-consensus-clustering/Datasets used: Datasets:Supplementary information.


LogSum + L2 penalized logistic regression model, and furthermore used a coordinate decent algorithm to solve it. The results of simulations and real experiments indicate that the proposed method is highly competitive among several state-of-the-art methods. Our proposed model achieves the excellent performance in group feature selection and classification problems.Biomarker selection and cancer classification play an important role in knowledge discovery using genomic data. Successful identification of gene biomarkers and biological pathways can significantly improve the accuracy of diagnosis and help machine learning models have better performance on classification of different types of cancer. In this paper, we proposed a  Many studies have shown that microarray data can be used to classify the different types of cancer, which includes how long the incubation period is, and what drugs are effective in the diagnosis and treatment processes.With the development of DNA microarray technology3, only a small number of genes (biomarkers) strongly indicate the target cancer, while other genes are not related to disease. Therefore, the data with unrelated genes may bring noise, and make the machine learning approaches less easy to find pathogenic genes that cause the disease. Moreover, from a machine learning perspective, the large number of genes (features) with few samples in the datasets may cause overfitting4, and have negative impact on classification performance. Due to the importance of these issues, effective gene (biomarker) selection methods are needed to help classify different cancer types and improve prediction accuracy.From a biological point of view8 evaluate genes based on discriminative power without considering their regulation correlations with other genes. The main disadvantage of the filtering methods is that it examines each gene separately, and makes each gene independent, thereby ignores the possibility that the genes have combined and grouping effects. This is a common problem with statistical methods, such as t-test, which can also examine each gene individually.In recent years, many methods for gene selection in microarray datasets have been developed and generally can be divided into three categories: filters, wrappers, and embedded methods. Filter methods11 utilize feature assessment measures based on the learning performance to select subsets of genes. Generally, they can acquire a small number of related genes to notable promote the learning ability. In some cases, the results of the wrapper methods are better than those of the filter methods. However, the main fault of wrapper methods is their computational cost is high.Wrapper methods26 that perform feature selection as part of the learning procedure of a single process. Under similar learning performance, the computational efficiency of embedded methods is more efficient than wrapper approaches. Hence, embedded methods have recently attracted a lot of attention in the literature. The regularization methods are important embedded technologies, which can perform feature selection and model training simultaneously. Many regularization methods have been proposed, such as Lasso12, SCAD13, adaptive Lasso14, MCP15, Lq (0 < q < 1)16, L1/218, LogSum19, etc. These methods perform well with the independent feature selection. When the features are highly correlated, some regularization methods which pay attention to the grouping effect can be used to select the groups of the relevant features, such as group Lasso20, Elastic net21, Fused Lasso22, OSCAR23, adaptive Elastic net24, SCAD-L225, L1/2 + L226.A third set of feature selection approaches is the embedded methods27. Medjahed et al. applied the K-nearest neighbors (K-NN) to the diagnosis and classification of breast cancer28. Meanwhile, some researchers used the logistic regressions with optimization methods for binary cancer classification33. However, the traditional logistic regression model has two obvious shortcomings, mainly in the following two aspects:Feature selection problem.All or most of the feature coefficients obtained by fitting the logistic regression model are not zero, i.e. all most of the features are related to the classification target and not sparse. However, the key factors affecting the model are often only a few in many practical problems. This non-sparseness of the logistic models increases the computational complexity on the one hand and is not conducive to the actual interpretation of the practical problems.Overfitting problem.The logistic regression models can often obtain good precision for the training data, but for the test data outside the training set, the classification accuracy rate is not ideal. In fact, not only logistic regression, many other data analysis models will also be affected by overfitting. It has become one of the hot research topics in statistics, machine learning and other fields.On the other hand, many machine learning models have been used to analyze microarray gene expression data for cancer classification. For example, Furey et al. used support vector machines (SVMs) to classify cell and tissue types35 proposed the sparse logistic regression based on the Lasso regularization and the coordinate descent methods. Algamal et al.37 proposed the adaptive Lasso and the adjusted adaptive elastic net for gene selection in high dimensional cancer classification. Like sparse logistic regression with the L1 regularization method, Cawley and Talbot30 investigated sparse logistic regression with Bayesian regularization. Liang et al.38 investigated the sparse logistic regression model with the L1/2 penalty for gene selection in cancer classification.In recent years, there is growing interesting to apply the regularization techniques in the logistic regression models to solve the above mentioned two shortcomings. For example, Tibshirani and FriedmanLogSum + L2 penalized logistic regression model. The main contributions of this paper include.LogSum + L2 penalized logistic regression model.Our proposed method can not only select sparse features (biomakers), but also identify the groups of the relevant features (gene pathways). The coordinate decent algorithm is used to solve the We also evaluate the capability of our proposed method and compare its performance with other regularization methods. The results of simulations and real experiments indicate that the proposed method is highly competitive among several state-of-the-art methods.Inspired by above mentioned methods, in this paper, we proposed a LogSum + L2 penalized logistic regression model and its optimization algorithm. “The rest of this paper is organized as follows. In “LR), which is a statistical method for modeling a binary classification problem. Suppose we have n samples and p genes. Datasets X and y are the genes matrix and the dependent variable, respectively. So, the n samples mean the set D, yi is a corresponding variable that takes a value of 0 or 1, x with class label y correctly. The We focused on binary classification using logistic regression And the MCP thresholding operator is given as follows:In Eq. , \docume17, and its penalty function can be written:Xu et al. have proposed Then the univariate half thresholding operator for a 26. Equation . We use the tenfold cross-validation (CV) approach to optimize the regularization parameters or tuning parameters  of the In this experiment, we initialize the coefficient 39 were defined as the follows:At the beginning, we divided the datasets at random into the training sets and the test sets. In our experiment, the approximate 70% of samples are proposed as training sets, and the rest are used as test sets. We repeated the simulations 30 times for each penalty method and computed the mean classification accuracy, mean classification sensitivity, and mean classification specificity on the training and test datasets respectively. To evaluate the quality of the selected features for the regularization approaches, the sensitivity and specificity of the feature selection performanceLogSum + L2 penalized logistic regression model is always the best.The training results of different methods on simulate datasets are reported in Table LogSum + L2 penalized logistic regression model is still the best one among the four methods. And in Scenario 1, whatever LogSum + L2 model is far apart, and its accuracy and specificity are not much different compared with other three methods.Table β-Sensitivity and β-Specificity. The approximate results are similar to the previous two Tables. In the same LogSum + L2 penalized logistic regression model contains the greatest number of features and highest sensitivity and specificity. And in different Table https://www.ncbi.nlm.nih.gov/geo/). Some detail information and introduction will be shown below:GSE10072: Series GSE10072 is a gene expression signature of cigarette smoking and its role in lung adenocarcinoma development and survival. Tobacco smoking can cause 90% of lung cancer cases, but the changes in the level of the molecules that lead to cancer development and affect survival are still unclear.GSE19188: Series GSE19188 is a dataset about gene expression for early stage Non-small-cell lung carcinoma (NSCLC). 156 tumors and normal samples are aggregated into the expected group. The prognostic characteristics of 17 genes showed the best correlation with the survival time after surgery.GSE19804: Series GSE19804 is a dataset about Genome-wide screening of transcriptional modulation in non-smoking female lung cancer in Taiwan. Although smoking is a major risk factor for lung cancer, only 7% of women with lung cancer in Taiwan have a history of smoking, which is much lower than that of white women. Researchers extracted RNA from paired tumors and normal tissues for gene expression analysis to explain this phenomenon. This dataset and its reports comprehensively analyze the molecular characteristics of lung cancer in non-smoking women in Taiwan.We use three publicly available lung cancer microarray datasets, which download from GEO . According to the experimental results, the LogSum + L2 penalized logistic regression model can find some unique genes, which cannot be identified by other regularization models but are significantly related to the disease. Therefore, we believe that the LogSum + L2 penalized logistic regression model can accurately and efficiently identify cancer-related genes.Table LogSum + L2 penalized logistic regression model. We also evaluate the capability of our proposed method and compare its performance with other regularization methods. The results of simulations and real experiments indicate that the proposed method is highly competitive among several state-of-the-art methods. The disadvantage of the proposed method is its three regularization parameters need to be tuned by the k-fold cross-validation approach.Successful identification of gene biomarkers and biological pathways can significantly improve the accuracy of diagnosis and help machine learning models have better performance on classification of different types of cancer. Many researchers used the logistic regressions with optimization methods for binary cancer classification. However, the traditional logistic regression model has two obvious shortcomings: feature selection and overfitting problems. In this paper, we proposed the 46. We will apply the proposed LogSum + L2 penalized logistic regression model to identify the non-coding RNA biomarker of human complex diseases as the future direction of our research.In recent years, increasing associations between of microRNAs (miRNAs) and human diseases have been identified. Based on accumulating biological data, many computational models for potential miRNA-disease associations inference have been developed


With the high prevalence of breast cancer, it is urgent to find out the intrinsic difference between various subtypes, so as to infer the underlying mechanisms. Given the available multi-omics data, their proper integration can improve the accuracy of breast cancer subtype recognition. In this study, DeepMO, a model using deep neural networks based on multi-omics data, was employed for classifying breast cancer subtypes. Three types of omics data including mRNA data, DNA methylation data, and copy number variation (CNV) data were collected from The Cancer Genome Atlas (TCGA). After data preprocessing and feature selection, each type of omics data was input into the deep neural network, which consists of an encoding subnetwork and a classification subnetwork. The results of DeepMO based on multi-omics on binary classification are better than other methods in terms of accuracy and area under the curve (AUC). Moreover, compared with other methods using single omics data and multi-omics data, DeepMO also had a higher prediction accuracy on multi-classification. We also validated the effect of feature selection on DeepMO. Finally, we analyzed the enrichment gene ontology (GO) terms and biological pathways of these significant genes, which were discovered during the feature selection process. We believe that the proposed model is useful for multi-omics data analysis. Breast cancer is the most common cancer and the main cause of cancer deaths besides lung cancer in women . The numThe current research on breast cancer subtypes focuses mainly on the molecular typing. In 1999, molecular typing of cancer was first proposed by the National Cancer Institute (NCI) . In 2000In this article, breast cancer was divided into five subtypes, namely luminal A, luminal B, HER2(+), TNBC, and unclear subtype, the same classification as in a published article . The detWith the explosive growth of massive biological data, the transformation of traditional biological statistical methods to computer-aided methods makes machine learning become an important part of predicting cancer prognosis . If all At present, the commonly used feature selection methods are mainly divided into the following three types: filter, wrapper, and embedded . These cWith the continuous development and improvement of high-throughput technology, there are increasingly more types of omics data obtained through high-throughput technology. Based on these omics data, there have been many studies on the classification of breast cancer subtypes. Brian D. Lehmann et al. used gene expression data to perform cluster analysis to determine the subtypes of triple-negative breast cancer . Sorlie In this study, DeepMO, a model using deep neural networks based on multi-omics data, was employed to classify breast cancer subtypes. DeepMO contains a type-specific encoding subnetwork to learn the features of each omics type and combines features of each omics type, and a classification subnetwork is used to classify different breast cancer subtypes. In this study, the input of DeepMO contains mRNA data, DNA methylation data, and copy number variation (CNV) data, and the output of DeepMO is the predicted molecular subtypes of breast cancer. The workflow of DeepMO is illustrated in In this study, the data on breast cancer were collected from The Cancer Genome Atlas (TCGA) , which iThere are 606 samples containing the three types of omics data simultaneously. Each type of omics data contains different levels of complementary information, so different aspects of breast cancer subtypes can be obtained from different perspectives. These breast cancer samples were divided into 5 subtypes, as shown in In addition, before performing feature selection, we normalized these data. For transcriptome data, we used the expression value from TCGA and excluded genes with missing values exceeding 200 samples. For DNA methylation data, only the regulatory relationship between gene transcription and related promoter hypermethylation or hypermethylation was considered. For CNV data, we used CNV annotation in PennCNV  to convep small n” paradigm [p is the number of features and n is the number of samples, it is not too fruitful to use the deep learning model to train the network weights on the omics data directly. Therefore, we assumed that the feature selection algorithm of omics data can further improve the deep neural network models [In general, a deep learning model does not need to select features separately because it can be done through the weight of the neural network. However, due to the “large paradigm  in the ok models . This isp-value in hypothesis testing using corresponding samples of each classification task. Then, for each omics data, we selected the top-k feature as the input of the deep neural network. In our experiments, the value of k was set to 5000, which means that 5000 features of each omics data were input into the deep neural network.In this study, we used the chi-squared test   loss. For multi-classification, we utilized Softmax regression and CrossEntropy loss instead.In this study, we verified the performance of DeepMO on both binary classification and multi-classification. For binary classification, we used accuracy (Acc) and area under the curve (AUC) as performance measurements. The data of binary classification have two types of samples, positive and negative, which are represented by https://github.com/linyq2117/DeepMO.In our experiments, we first measured the performance of binary classification of any two breast cancer subtypes. Then, we compared the performances of multi-classification based on multi-omics data and single omics data. Additionally, the performance of DeepMO on multi-classification was compared with some state-of-the-art data integration methods. Furthermore, we analyzed the effect of feature selection, and validated its role in classification using deep neural networks based on multi-omics data. Finally, we also analyzed the enrichment gene ontology (GO) terms and biological pathways of these significant genes discovered during the feature selection process. The code is available at To investigate the performance of DeepMO on binary classification, we compared the results by using single omics data and combining three omics data based on DeepMO to classify any two subtypes of breast cancer, including (1) luminal A versus luminal B, (2) luminal A versus HER2(+), (3) luminal A versus TNBC, (4) luminal A versus unclear, (5) luminal B versus HER2(+), (6) luminal B versus TNBC, (7) luminal B versus unclear, (8) HER2(+) versus TNBC, (9) HER2(+) versus unclear, and (10) TNBC versus unclear. Specifically, the mean accuracy and AUC of 5-fold cross-validation were used as the measurements. These accuracy and AUC values on any two subtypes of breast cancer by using different types of omics data are shown in In addition, to exclude the possibility of overfitting, we selected some newly annotated samples from TCGA as independent test data. The numbers of distinct breast cancer subtypes in test dataset are shown in In this study, we compared the performance of binary classification by using MKL and DeepMO. As above, accuracy and AUC were used as the classification performance measures. These accuracy and AUC values on any two subtypes of breast cancer by using MKL and DeepMO are shown in To better evaluate the performance of the proposed model, we used the model to predict breast cancer subtypes based on multi-classification. First, we compared the accuracy of multi-classification between DeepMO using single omics data and multiple omics data. The mean accuracy of 5-fold cross-validation on all subtypes of breast cancer by using single omics data and combining three omics data is shown in Due to the imbalance of class, which had an effect on the neural network, we took some measures to cope with it. In this study, we used both the undersampling and oversampling methods to reduce the effect of imbalanced data. We selected samples by the weight of each subtype and the weight was reciprocal to the number of each subtype. Therefore, the subtype with smaller samples had more probability to be selected. This may make our model applicable to imbalanced data. To evaluate this method, we removed all samples of one subtype each time, and performed multi-classification again. The results are shown in To further evaluate the performance of DeepMO on multi-classification, we selected some state-of-the-art methods of omics data integration, including the logistic regression model/multinational model with Elastic Net (EN) regularization  and RandWe utilized the chi-squared test to select the top 5000 important features before integrating multi-omics data using deep neural networks for reducing features and increasing training speed. The top 5000 features of three omics data types for multi-classification are shown in the Additionally, our experiments indicated that feature selection can improve accuracy on multi-classification. The accuracy without and with feature selection on multi-classification is 0.771 and 0.782, respectively.p-values of the chi-squared test in increasing order. These heatmaps in CDK12, GRB7, ORMDL3, PSMD3, STAC2, and STARD3. These genes may be significant in classifying breast cancer subtypes.To better analyze the important genes selected by the feature selection method, we displayed a heatmap depicted by the top 30 genes selected by feature selection on different omics data. For each omics data, the top 30 genes were ranked by Some significant genes have been reported in related literature. ER coactivator MED1 was a novel crosstalk point for the HER2 and ERα pathways. Its expression was positively correlated to HER2 status of tumors from tissue microarray analysis of human breast cancers . XBP1 waTo further understand the differences between breast cancer subtypes, we conducted an enrichment analysis of the differential genes screened from the omics data. We selected the top 1000 genes by chi-squared test on each omics data. Then, the whole set of human genes was employed as the background among the union of selected genes from three omics data using Metascape  against p-value from each of the 20 clusters. Each cluster did not exceed 15 terms, and the total did not exceed 250 terms. The network was visualized using Metascape, where each node represents an enriched term and is colored first by its cluster ID , we selected a rich set of terms and presented them as a network graph, where terms with similarity >0.3 were connected by edges. We chose the term with the best uster ID a and the(p-value b. From FIn this study, we utilized DeepMO, a method using deep neural networks and multi-omics late integration to classify breast cancer subtypes. First, we used the chi-squared test to select features that are closely related to labels. Afterward, we validated its performance both on binary classification and multi-classification. For binary classification, we compared accuracy and AUC obtained by DeepMO using single omics data and multiple omics data. We also compared DeepMO with MKL. The results showed that our proposed model based on multi-omics data performed best in most cases. For multi-classification, we first compared DeepMO using multi-omics data and single omics data and found that using multi-omics data can obtain better accuracy. We also compared DeepMO with other data integration methods, including ConcateEN, ConcateRF, EnsembleEN, EnsembleRF, and MKL. The compared results indicated that DeepMO was superior to all these methods. Furthermore, we compared the performance of DeepMO with feature selection and without feature selection. The results indicated that feature selection can improve DeepMO. Subsequently, we plotted the heatmap of some important genes, which visually showed that DNA methylation and mRNA data can distinguish breast cancer subtypes more clearly. Finally, we analyzed pathways and gene ontology, including KEGG, GO_BP, GO_CC, and GO_MF, and found most significant terms and similarity among them.Feature selection is an effective approach to removing irrelevant and redundant gene features and reducing the dimension. In our model, although encoding networks can learn important features from the original gene features and achieve the reduction of features, there are still too many features inputting into encoding networks, which makes it difficult and slow to train the network weights. In addition to considering the complexity of the algorithm, we assumed that the deep neural network models may be further improved by feature selection algorithms. The results in As we all know, the hyperparameters in deep neural networks, such as the number of nodes in the hidden layers, mini-batch size, learning rates, weight decay, the number of epochs, and dropout rate, have an important impact on the performance of the neural network. However, in this study, we only used the default or moderate values and did not adjust the hyperparameters. Therefore, the result may be better after tuning hyperparameters and selecting better values. Additionally, the imbalance of data had an effect on the neural network. In this study, we used both the undersampling and oversampling methods to reduce the effect of imbalanced data. We selected samples by the weight of each subtype and the weight was reciprocal to the number of each subtype. Therefore, the subtype with smaller samples had more probability to be selected. This can solve data imbalance to some extent.The purpose of this study was to classify breast cancer subtypes by using deep neural networks based on multi-omics data from TCGA. The classification results show that, by using the proposed model, integrating multi-omics datasets can improve the performance as compared to using single omics data for classifying breast cancer subtypes. Moreover, the proposed model is superior to other state-of-the-art methods on binary classification and multi-classification. At the same time, through the analysis of important genes and pathways, we tried to find some biological explanations for the differences between breast cancer subtypes, and provide guidance for exploring the biological models of breast cancer subtypes. We believe that the proposed model is useful for multi-omics data analysis.


The source code is licensed under the MIT License. seekCRIT is developed and tested on Linux CentOS-7.Over the past two decades, researchers have discovered a special form of alternative splicing that produces a circular form of RNA. Although these circular RNAs (circRNAs) have garnered considerable attention in the scientific community for their biogenesis and functions, the focus of current studies has been on the tissue-specific circRNAs that exist only in one tissue but not in other tissues or on the disease-specific circRNAs that exist in certain disease conditions, such as cancer, but not under normal conditions. This approach was conducted in the relative absence of methods that analyze a group of common circRNAs that exist in both conditions, but are more abundant in one condition relative to another . Studies of differentially expressed circRNAs (DECs) between two conditions would serve as a significant first step in filling this void. Here, we introduce a novel computational tool, seekCRIT , that identifies the DECs between two conditions from high-throughput sequencing data. Using rat retina RNA-seq data from ischemic and normal conditions, we show that over 74% of identifiable circRNAs are expressed in both conditions and over 40 circRNAs are differentially expressed between two conditions. We also obtain a high qPCR validation rate of 90% for DECs with a FDR of < 5%. Our results demonstrate that seekCRIT is a novel and efficient approach to detect DECs using rRNA depleted RNA-seq data. seekCRIT is freely downloadable at  seek for differentially expressed CircRNAs In Transcriptome), that identifies the differentially expressed circRNAs (DECs) between two conditions from high-throughput sequencing data. seekCRIT provides a straightforward normalized quantification of circRNAs and statistical measures by adapting a junction-count-based estimation approach. Using publicly available ribosomal RNA depleted RNA-seq data and our own rat retina RNA-seq data, we show that seekCRIT can efficiently detect circRNAs and identify DECs. We also obtain a high qPCR validation rate of 90% for DECs with a FDR of < 5%. Our results demonstrate that seekCRIT is a novel and efficient software to detect DECs using rRNA depleted RNA-seq data.The focus of circRNA studies has been on condition-specific circRNAs, however, there are situations in which circRNAs exist in both conditions with different abundance. Here, we introduce a new and robust analytic software, seekCRIT ( PLOS Computational Biology Software paper.This is a Alternative splicing (AS) refers to the production of multiple mRNA isoforms from a single gene due to alternative selection of exons or splice sites during pre-mRNA splicing. It is a primary mechanism of gene regulation in higher eukaryotes and significantly expands the functional complexity of eukaryotic organisms . While cAlthough these circRNAs have garnered considerable attention in the scientific community due to their major role as a microRNA (miRNA) activity modulator ,4 and thSeveral computational tools for identification of circRNAs from sequencing data have been developed, including MapSplice, DCC 1212, NCLscseek for differentially expressed CircRNAs In Transcriptome), a novel computational tool for the identification of DECs between two biological conditions using ribosomal RNA (rRNA) depleted RNA-seq data. seekCRIT has many distinct features over existing computational approaches. First, seekCRIT offers a simple automated platform for the detection of DECs between two biological conditions. seekCRIT can identify a set of DECs using RNA-seq data (fastq files) from each condition, genome (fasta), and transcriptome (gtf or refSeq). This simple input requirement allows researchers to identify DECs with no complicated steps. Second, seekCRIT can work with replicates. This feature is important because it has become customary to generate hundreds of millions of RNA-seq data with replicates due to a dramatic drop in high-throughput sequencing cost. Finally, and most importantly, seekCRIT provides a straightforward normalized quantification of circRNAs and statistical measures by adapting a junction-count-based estimation approach [Here, we introduce seekCRIT : seekCRIT pipeline begins with raw RNA-seq data from the rRNA depleted library preparation.Genome sequence (fasta): To facilitate the use of seekCRIT, genomic sequences for various species are provided.Transcriptome annotation: This annotation file has all information for genes, transcripts, exons, and introns for the given species. Back-splicing and canonical linear splicing are determined based on this annotation. The annotation file should be in the commonly used GTF (Gene Transfer Format) format or refSeq format. Both GTF files and refSeq files for various species are provided.Optional parameters to specify minimum number of circular junction count (CJC) for high-confidence circRNAs and to specify the number of threads to use for faster running time: To increase the sensitivity of circRNA detection, we allow users to require a minimum number of CJC. Users can also specify the number of threads they want to use for an improved running time.Given proper inputs, seekCRIT performs a series of automated processes to uncover the DECs. In short, it enables users to check the quality of raw sequences, map reads to genome and transcriptome, find chimeric and linear alignments, detect circRNAs, estimate expression levels per circRNA per sample, compute statistical measures  per circRNA, and identify DECs at a user defined FDR cut-off, typically 5%.CJC and LJC (Linear Junction Count) represent the counts of circRNA and linear RNA isoforms respectively. Assuming that the read counts follow a binomial distribution, the maximum likelihood estimate (MLE) of the expression level (PBI) of a circRNA in a given sample can be calculated as Similar to the exon inclusion level (PSI or ψ) estimate for alternatively spliced cassette exons –41, we dIn the circRNA detection process, seekCRIT first maps reads to genome and transcriptome using the STAR aligner . Since tUJC and DJC are required to compute PBI. seekCRIT examines mapped reads in the bam files to detect upstream junction count and downstream junction count associated with the detected circular RNAs. For each circular RNA, seekCRIT examines the bam file and compiles number of reads that span upstream junction (for UJC) and downstream junction (for DJC) of the given circular RNA.seekCRIT uses Fisher’s exact test to calculate the p-value. For each circRNA, the CJC and LJC for each sample are computed from chimeric alignments and linear alignments, respectively. From these values, seekCRIT constructs the 2x2 table  to compu1 and PBI2 as its expression levels in sample 1 and sample 2, respectively. We then define the circRNA expression level difference, deltaPBI (or ΔPBI), as PBI1—PBI2. seekCRIT then examines all circRNAs and identifies DECs that meet |ΔPBI| ≥ 5% and FDR < 5%.To detect DECs between two samples, for each circRNA, we define PBITo demonstrate the applicability of seekCRIT, we analyzed three rRNA depleted RNA-seq datasets : two pubTo assess the overall accuracy of our PBI estimates and DECs detection, we randomly selected 15 circRNAs covering three categories; five from upregulated: ΔPBI < -5%, five from downregulated: ΔPBI > 5%, and five from unchanged: |ΔPBI| < 5%. 10 circRNAs in upregulated and downregulated categories meet FDR < 5% as well.To systematically assess the accuracy of seekCRIT and its statistical measure, we generated simulated data and used gold-standard DEC set obtained from the simulated data to calculate the true positive rate  of seekCRIT under various read lengths and read depths. Our gold-standard DEC set has 1055 DECs and it allowed us to investigate factors that could influence the sensitivity of seekCRIT. To assess the importance of sequencing depth and read length on detection of DEC, we randomly generated RNA-Seq data from human (hg38) and estimated the sensitivity under various levels of sequencing depth and read length. Each simulated RNA-seq consists of 5% of reads from circular RNA, 91.5% of reads from linear mRNA, 2% of reads from intron, 1% of reads from intergenic region, and 0.5% of reads from sequencing error. We performed 10 independent random samplings at each sequencing depth (from 10 million to 120 million reads) and at each read length (from 50 bp to 150 bp). As expected, we observed low sensitivity at lower levels of sequencing depth, especially when the number of reads per sample was below 40M, and at lower levels of read length Fig F in . Our simLong Evan’s female rats (weighing 250–300 g) were used in this study. The animals were housed on a 12-hour light-dark cycle with food and water ad libitum. The treatment and care of all animals were approved by the University of Louisville Institutional Animal Care and Use Committee (IACUC 12062) and were performed in accordance with the Association for Research in Vision and Ophthalmology Statement for the Use and Care of Animals in Ophthalmic and Vision Research.We examined three RNA-seq datasets and found that majority (at least 74% in our datasets) of the detected circRNAs are commonly expressed in both biological samples . Thus, iTable inCpd is shown in To show that seekCRIT can work with/without replicates, two publicly available datasets from the tissues that were known to have high abundance of circRNAs were obtained from the NCBI GEO. The first data set contains somata and neuropil samples from rat brain. Both samples have about 40 million 151bp single-end reads which are deep enough to conduct splicing studies for highly or moderately expressed genes in eukaryotes –52. For Table inA second publicly available dataset contains human umbilical vein samples under a normal oxygen level  and a low oxygen level . Each sample has two replicates with ~42 million paired-end reads  per replicate. seekCRIT detected over 3,000 circRNAs and for each circRNA, seekCRIT computed PBI per sample by combining the junction counts from each replicate. At FDR 5% and |ΔPBI|>5%, seekCRIT identified 48 DECs  and an ischemic injury sample collected 12 hours after the injury (IR12). Each sample has ~100 million 150 bp single-end reads and we truncated the last 50 bps due to their low read quality. seekCRIT detected 3,278 circRNAs and 40 of these were DECs at FDR 5% and |ΔPBI|>5%   showed more than 5% changes in the RT-qPCR results. We believe false negatives arise for circRNAs with low expression due to a systematic bias in RNA-seq. This bias has been addressed in other high-throughput sequencing studies as well .Fig E. inSome previous research has found that one gene could generate multiple circular RNAs . To verihttps://github.com/UofLBioinformatics/seekCRIT. seekCRIT is freely downloadable at S1 TextThe details of the sequencing, RT-qPCR assay, validation results, seekCRIT output format, supplementary tables A, B, and supplementary figures A through F can be found here.(DOCX)Click here for additional data file.S1 TableDifferentially expressed circular RNAs (DECs) detected from seekCRIT analyses on three datasets: Somata vs. Neuropil, Normaxia vs. Hypoxia, and CTRL vs. IR12.(XLSX)Click here for additional data file.S1 FileSource code, license, usage, and test data can be found here.(ZIP)Click here for additional data file.


It is currently difficult for pathologists to diagnose pancreatic cancer (PC) using biopsy specimens because samples may have been from an incorrect site or contain an insufficient amount of tissue. Thus, there is a need to develop a platform-independent molecular classifier that accurately distinguishes benign pancreatic lesions from PC. Here, we developed a robust qualitative messenger RNA signature based on within-sample relative expression orderings (REOs) of genes to discriminate both PC tissues and cancer-adjacent normal tissues from non-PC pancreatitis and healthy pancreatic tissues. A signature comprising 12 gene pairs and 17 genes was built in the training datasets and validated in microarray and RNA-sequencing datasets from biopsy samples and surgically resected samples. Analysis of 1,007 PC tissues and 257 non-tumor samples from nine databases indicated that the geometric mean of sensitivity and specificity was 96.7%, and the area under receiver operating characteristic curve was 0.978 . For 20 specimens obtained from endoscopic biopsy, the signature had a diagnostic accuracy of 100%. The REO-based signature described here can aid in the molecular diagnosis of PC and may facilitate objective differentiation between benign and malignant pancreatic lesions. Pancreatic cancer (PC) is the fourth leading cause of cancer deaths in the United States and the sixth leading cause in China. Patients with PC have a 5-year survival rate of 8.5% in the United States and 7.2% in China . The diaTherefore, it is sometimes necessary to perform a tissue biopsy for the pathological diagnosis of PC. In clinical practice, more accurate and definitive pathological diagnoses can be made using biopsies from EUS-guided fine needle aspiration (EUS-FNA), and this method has an overall diagnostic accuracy of 91% . HoweverIn this study, based on the REOs of 12 gene pairs, we identified a qualitative transcriptional signature for the early diagnosis of PC. The signature can accurately discriminate both PC tissues and adjacent-normal tissues from normal pancreatic tissues and non−PC pancreatitis in both biopsy and surgical resection samples.1 and were measured by the Illumina, Affymetrix, or Agilent platforms. The training cohort consisted of 5 microarray datasets , with 74 normal pancreatic tissues, 72 pancreatitis tissues, and 269 PC samples , the Genotype-Tissue Expression (GTEx) project, and the Cancer Genome Atlas (TCGA) Data Portal samples . The tunTCGAbiolink,” including 183 non-formalin-fixed and paraffin-embedded (FFPE) samples of primary pancreatic tumors. Ensembl ID for protein-coding messenger RNAs (mRNAs) was annotated to symbol name using GENCODE27. The number of fragments per kilobase of non-overlapped exons per million fragments mapped (FPKM) was calculated first and was then transformed into transcripts per kilobase million (TPM) values. All mRNAs with TPM values < 1 in more than 90% of the samples were considered to be noise and removed prior to downstream analysis. Data of the GTEx project were retrieved from the UCSC Xena browser2. For microarray datasets measured by Affymetrix platforms, the robust multiarray average (RMA) procedure was performed, with raw CEL files for background, using the R package “affy” for adjustment (Transcriptome HTSeq-counts data of the TCGA-Pancreatic Adenocarcinoma (PAAD) project were downloaded from the Genomic Data Commons using the R package “justment . For othFor the purpose of this study, tumor samples were identified as “cancer” or “cancer-adjacent normal” because the transcriptional characteristics of apparently normal tissue that is adjacent to a tumor differs from healthy normal tissues , whereasi and j was assigned a value of “1” if gene “i” had higher expression and “0” if gene j had greater expression. A “reversal gene pair” (RGP) was defined by the presence of the same REO pattern in more than 85% of the tumor samples  and a reversed pattern in more than 85% of the non-tumor samples  in the training dataset. Then, RGPs were filtered using the tuning dataset to establish a candidate REO signature of PC. The rank difference for each RGP was computed for each sample as:Data analysis consisted of several sequential steps . First, Ri and Rj represent the rank of gene i and j within a sample, and Rij represents the rank difference.where Rij, the geometric mean of the mean value of Rij in tumor samples was used to assess the extent of reversal of the gene pair between tumor and non-tumor samples:Then, X{X ∈ }. However, in most cases, a sample in a specific X may have an where The selected candidate REO signatures were then sorted in a descending order according to their pROC”. The diagnostic performance was further evaluated in each independent dataset by calculation of sensitivity and specificity. In this procedure, sensitivity refers to the proportion correctly identified true positives and specificity to the proportion of correctly identified true negatives. Sensitivity and specificity were recorded at the median cutoff of the voting threshold.All samples in the training, tuning, and validation datasets were first pooled together to assess the general predictive performance in different samples  using receiver operating characteristic (ROC) analysis and calculation of the area under the curve (AUC). The accuracy was defined as the portion of correctly identified samples in the entire cohort, and the AUC, sensitivity, and specificity were calculated using the R package “3). The data analysis source code is provided in the github: https://github.com/xlucpu/PCsig.All statistical analyses were performed using R software  to identify common gene pairs with stable REOs in 74 normal pancreatic tissues  and 72 pancreatitis samples . Then, we identified 14,633,175 gene pairs with identical REO patterns in at least 85% of normal pancreas and pancreatitis samples as stable gene pairs of non-tumor samples. We also identified 18,300,104 gene pairs with stable REO patterns in at least 85% of the 269 PC samples in the training cohort. Next, we used these data to identify 269 RGPs between the non-tumor and tumor tissues . After ak gene pairs for sample classification using majority vote rule. The results indicated that for k ranging from 1 to 20, the largest geometric mean of sensitivity and specificity (93.79%) occurred for k = 12  from non-tumor samples. For the 1,007 PC tissues and 257 non-tumor samples from the 9 external validation databases, the geometric mean of sensitivity and specificity was 96.7% and the area under receiver operating characteristic curve was 0.978  had a diagnostic accuracy of 100%. For normal pancreatic tissues obtained by autopsy (GTEx), our method correctly classified all 248 samples as normal pancreatic tissue. The other validation sets consisted of samples from surgically resected tissues. For data measured by microarray, our signature correctly identified 96.07% of 842 tumor tissues as tumor samples. The details of diagnostic performance for the transcriptional signature are shown in PC is expected to become the leading cause of cancer-specific mortality in Western countries by 2030, and yet early diagnosis remains difficult . It is tIn contrast to previous studies that used a transcriptomic diagnostic signature based on complicated procedures of data normalization and parameter fitting , we analSeveral genes in the transcriptional signature proposed here have established roles in the carcinogenesis of PC. For instance, laminin γ2-chain (LAMC2) is a well-known PC-related gene whose level is elevated in the circulation of PC patients . This geThere were some limitations in our study. First, due to the limited number of biopsy tissues, we only used samples collected from surgery (not from biopsy) in the training set, and this may have led to selection bias. Second, our study design was retrospective, with genomic data derived from publicly available databases. Prospective clinical studies are needed to validate our findings.In summary, we constructed and validated an REO-based signature consisting of 12 gene pairs and 17 genes that can aid in the early diagnosis of PC.All datasets generated for this study are included in the article/Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.Y-JZ and C-JY obtained data from TCGA, GEO, and GTEx database, designed the study, and wrote the manuscript. Y-JZ, X-FL, J-LM, X-YW, Q-WW, Y-JG, and H-MC analyzed and interpreted the data. X-FL was responsible for the statistical analyses. X-BL and F-RY contributed to conception, design, and funding. All authors have been involved in revising and proofreading of the manuscript, and approved the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Identifying biomarkers that are associated with different types of cancer is an important goal in the field of bioinformatics. Different researcher groups have analyzed the expression profiles of many genes and found some certain genetic patterns that can promote the improvement of targeted therapies, but the significance of some genes is still ambiguous. More reliable and effective biomarkers identification methods are then needed to detect candidate cancer-related genes. In this paper, we proposed a novel method that combines the infinite latent feature selection (ILFS) method with the functional interaction (FIs) network to rank the biomarkers. We applied the proposed method to the expression data of five cancer types. The experiments indicated that our network-constrained ILFS (NCILFS) provides an improved prediction of the diagnosis of the samples and locates many more known oncogenes than the original ILFS and some other existing methods. We also performed functional enrichment analysis by inspecting the over-represented gene ontology (GO) biological process (BP) terms and applying the gene set enrichment analysis (GSEA) method on selected biomarkers for each feature selection method. The enrichments analysis reports show that our network-constraint ILFS can produce more biologically significant gene sets than other methods. The results suggest that network-constrained ILFS can identify cancer-related genes with a higher discriminative power and biological significance. The recent development of high-throughput gene expression profiling provided an opportunity for researchers to better understand the molecular characteristics of the cancer disease, leading to advances in its diagnosis and treatment. Accurate identification of the cancer diagnostic biomarkers is very critical for the provision of appropriate therapies and drug development. Some gene mutations, such as BRCA1, BRCA2, VHL, PBMR1 and others were identified to be correlated with an increased tumor aggressiveness in cancer . A few tl_1 penalty with graph regularization procedure to simultaneously obtain sparsity and smoothness for the linear model analysis i, where e denotes a 1D array of ones. Ranking the š(i) scores in a descending order, we can get the most discriminative features at the top of the sorted list.Matrix Fig 1.In order to measure the prediction performance of our proposed method, the training data reduced to selected genes was used to train the classifiers which were subsequently employed to be tested on the rest of samples. We performed the model training and validation process on the samples in part two. As can be seen in Fig 2. Generally, we observed that the network-constrained ILFS shows better prediction performance, except in the case of KIRC. For KIRC, mRMR and ReliefF had the greatest predictive power. Moreover, network-constrained ILFS showed much better performance for LIHC than other cancers. To explore the distribution of AUCs, we also plotted the accuracies obtained by each feature selection method on the PRAD data set as shown in Fig 3. Another measurement to assess these feature selection methods is the F1 score. The F1 score is a weighted average of precision and recall, which can be calculated by formula (11). The results are listed in To evaluate the prediction performance, several popular feature selection methods, including mRMR, LASSO, PLS-DA and ReliefF and the original ILFS were compared with the proposed method. We provided the top 100 biomarkers obtained by the above-mentioned feature selection methods for five cancer types in the Fig 4 shows the significance values of the selected oncogenes ratio for each method on five cancer types using hyper geometric tests. It is obvious that the proposed method outperforms other methods in this task. The selected oncogenes by the network-constrained ILFS are listed in The efforts of many scientists resulted in revealing some genetic mutations that might be involved in cancer development. The IntOGen-mutations platform summarizes the somatic mutations, genes and pathways that are involved in tumor genesis . We collFig 5 shows the number of enriched GO BP terms for five cancer types. The detailed information of the GO functional enrichment analysis is listed in the Finding gene groups that show predictive power is no longer a very hard job. However, mining biomarkers that provide insights into the biological mechanisms remains a challenge. To assess the interpretability significance of selected biomarkers, we adopted two ways: GO functional enrichment analysis and GSEA . The topFrom GO BP enrichment analysis results, we found that the selected genes by our method for five cancer types are significantly involved in GO:0000398~mRNA splicing, via spliceosome, GO:0010467~gene expression, GO:0008543~fibroblast growth factor receptor signaling pathway, and GO:0006370~7-methylguanosine mRNA capping. The core spliceosome machinery has been demonstrated to be overexpressed in multiple cancers and affect autophagy and cell proliferation, becoming a potential therapeutic target for malignant solid tumors treating , 35. TheIn GSEA analysis, the most significantly enriched gene sets from our method include MORF_SOD1 (Neighborhood of superoxide dismutase 1 in the MORF expression compendium), MORF_CSNK2B  for prostate cancer, GCM_CSNK2B  and MORF_EIF3S2  for breast cancer. Casein kinase 2 (CK2) is a ubiquitous serine/threonine protein kinase. A previous study has demonstrated that CK2 is to be overexpressed in a number of human cancers, including prostate and breast cancer , 40. SODIn the field of genomics, it is very common to have a high dimensional data with low sample size; thus, feature selection plays a very critical role in scientific discoveries. Most existing feature selection methods rank the features only from the statistical perspective. Such methods tend to filter out those genes that show the best discriminative power in model training. However, a lot of those genes are meaningless when it comes to the biological process and interpretability. This can be perceived from our experiments result. It is easy to find in In this study, we proposed a novel feature selection method which combined the biological network with the statistical method of ILFS. We applied this method to identify biomarkers in the gene expression data of BRCA, COAD, KIRC, LIHC, and PRAD. First, we compared it with the methods of ILFS, mRMR, LASSO, VIP score using PLS-DA and ReliefF on estimation precision and selection ratio of known oncogenes. Then, we performed functional enrichment and gene set enrichment analysis on selected features and perceived that the selected features are meaningful from a biological perspective. The results indicate that the network-constrained ILFS is helpful in cancer biomarkers identification.S1 Fig(TIF)Click here for additional data file.S1 Table(XLSX)Click here for additional data file.S2 Table(XLSX)Click here for additional data file.S3 Table(XLSX)Click here for additional data file.S4 Table(XLSX)Click here for additional data file.S5 Table(XLSX)Click here for additional data file.S6 Table(XLSX)Click here for additional data file.S7 Table(XLSX)Click here for additional data file.S8 Table(XLSX)Click here for additional data file.S9 Table(XLSX)Click here for additional data file.