Cancer in all its forms of expression is a major cause of death. The bladder cancer is also causes the same. finding the biomarkers responsible for the cancer is a challenging task and in certain cases, such as disease progression and therapy response, it become more challenging. The advancements in technology provides latest machine learning methods that help to identify the genomic biomarkers computationally. In this paper, the genomic biomarkers are tracked for bladder cancer from Univeristy of Calgary cohort and different bioinformatics methods, such as differential gene expression, survival rate estimation, consensus gene selection approaches were optimally used. The elastic-net based regression method has been utilized as a machine learning method which shows satisfactory results.Cancer in all its forms of expression is a major cause of death. To identify the genomic reason behind cancer, discovery of biomarkers is needed. In this paper, genomic data of bladder cancer are examined for the purpose of biomarker discovery. Genomic biomarkers are indicators stemming from the study of the genome, either at a very low level based on the genome sequence itself, or more abstractly such as measuring the level of gene expression for different disease groups. The latter method is pivotal for this work, since the available datasets consist of RNA sequencing data, transformed to gene expression levels, as well as data on a multitude of clinical indicators. Based on this, various methods are utilized such as statistical modeling via logistic regression and regularization techniques (elastic-net), clustering, survival analysis through Kaplan–Meier curves, and heatmaps for the experiments leading to biomarker discovery. The experiments have led to the discovery of two gene signatures capable of predicting therapy response and disease progression with considerable accuracy for bladder cancer patients which correlates well with clinical indicators such as Therapy Response and T-Stage at surgery with Disease Progression in a time-to-event manner. Cancer is a severe disease, often grievously impacting individuals around the world. It is an unexpected and potentially deadly mutation in the normal human cell life cycle, which causes defective or damaged cells to grow uncontrollably either in the organ from which the cells originated or in other parts of the body where they do not belong organically. Such growths, if they turn out to be malignant instead of benign, form tumors that can in turn export cancerous cells to other parts of the body where they form metastases. According to the International Agency for Research on Cancer, a total of almost 19.3 M new cases of cancer were recorded in 2020, and almost 10 M deaths [Being able to harness computational power for bioinformatics in general and specifically for cancer research is both an opportunity and a privilege, considering the scale of this disease and the potential impact a breakthrough in managing the disease might have on the affected population.Cancer of the bladder  is the fourth most common type of cancer among men and the ninth most common cancer among women in the United States , whereasIn its most common form, bladder cancer is identified as urothelial carcinoma (UC), with more than 90% being classified as such ,5. This During the last fifteen to twenty years, there have been major advancements in genomic profiling technologies, huge leaps in genomic sequencing, and consequently an emergence of selective molecular targeted therapies. These factors have lead to a dramatic increase in the importance of biomarkers where biomarkers are defined as follows: “A biomarker is any substance, structure or process that can be measured in the body or its products and influence or predict the incidence of outcome or disease” . When foThis paper aims to find the most relevant qualified genes for therapy response and disease progression among the cohort of bladder cancer patients. With the motivation to identify those genes which may impact highly in these two different scenarios, the research contributions of this paper are indicated below.A novel method to identify the qualified genes in bladder cancer among the therapy response and disease progression patients.A novel algorithm to separate the histology-based samples grouping into neuroendocrine, utothelial, sqaumous, micropapillary, and other secondary histologies.A pipeline has been designed including differential gene expression (DGE) analysis, Wilcoxon rank sum test, Kaplan–Meier estimation, PCA-based dimensionality reduction and clustering, consensus gene selection approaches.Elastic-net-based regularized regression method has been utilized as a machine learning method with k-fold cross-validation and ROC curve;, time-to-event prediction is used to showcase the prediction result.The research area of biomarker discovery has made dramatic progress since the beginning of the current century fueled mostly by major technological advancements such as digital computers. These technological advancements have enabled researchers to incorporate computer-implemented computational methods in the discovery process, since they allow for a virtually unlimited number of experiments/simulations, ease of hypothesis testing, and the capability to manipulate vast amounts of data.Specifically, from a brief review of the available literature for cancer biomarker discovery, one can identify a variety of computational methods employed for that purpose, spanning from traditional statistical approaches to state-of-the-art machine learning techniques and tools tailored to the exact needs of the problem .Before going into the various works that have been conducted and the methods that have been explored, it is interesting to mention a discussion paper from Steyerberg et al.  that looDespite machine learning methods having matured to their current state, one can find many research works that are mostly, if not exclusively, reliant on more traditional statistical approaches. Specifically, in 2020 Grivas et al.  used CoxIn the same year, Font et al.  used KapKim et al.  used R tIn many cases, there is a synergy between machine learning models and statistical tests. Kamoun et al.  has madeTibshirani et al.  first apUse of ML models, namely, Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (K-NNs), and Extreme Gradient Boosting (XGB) are commonly found in cancer predictions. S. Prusty et al.  proposedThe main focus of this work is the discovery of genomic biomarkers, for various clinical endpoints, among bladder carcinoma (BLCA) patients. The endpoints of interest include response to neoadjuvant chemotherapy (NAC), disease progression, survival outcome (either as time-to-progression or time-to-last followup since treatment), and T-Stage at the time of surgery. In this section, we have presented the materials  and methods, such as differential gene expression analysis, Kaplan–Meier estimator, dimensionality reduction, clustering, signature score generation, sample selection, and application of machine learning algorithms. All the steps used in this work are explained in detail in this section.The discovery dataset used was provided by the University of Calgary’s Cumming School of Medicine. It consists of a retrospective cohort of 102 BLCA patients who underwent chemotherapy. The clinical file, provided in the Excel workbook format, included a wide variety of indicators which are described on a high level, ranging from smoking habits and hazard exposure, to blood work and tumor stage and to survival status and various outcome dates.Initially, the cohort contained 209 samples. However, it had to be limited to only patients coming from the University of Calgary hospital, since both the clinical and the gene expression data for patients with IDs outside the University of Calgary cohort were not available. Those patients were identified by their Diagnostic TURBT Path which was a dedicated field in the patient record that would start with an “SR-” prefix..rds format. The genomic data sequencing was carried out by Decipher Biosciences [There are many ways to discover genomic biomarkers for disease groups. In the context of this research, the chosen approach was to analyze gene expression levels which were found in a file consisting of 46,050 unique gene expression values for 124 patients provided in sciences . Gene exIn order to correctly match the gene expression information to the clinical data for each patient, an additional utility metadata file was needed. This was needed as well since the samples had to be limited to the intersection of those present in both the clinical and the gene expression files. By taking advantage of 2 columns in the file, the Diagnostic TURBT Path #  and the CELfile name, the desired joint dataset was created.Response patients (PR + CR) and the Disease patients (SD + PD) were grouped together, respectively. Based on that, a new column was created in the clinical file mentioned above, taking the value 0 for samples in the former group, indicating the lack of the event of disease, and 1 for the latter.For some clinical endpoints of interest, it was deemed beneficial to lessen the degree of resolution. Specifically, the Therapy Response column consisted of 4 distinct values: 0 = Partial Response (PR), 1 = Complete Response (CR), 2 = Stable Disease (SD), and 3 = Progressive Disease (PD). Instead of looking at each category individually, the A similar approach was followed in the case of the histology indicator. Information about the samples’ histology was split into 2 columns, predominant and secondary. Out of 102 samples, 98 were predominantly urothelial, with only 4 being neuroendocrine. However, information about the secondary histology was a lot more variable, taking any (or a combination) of the following values of secondary histologies: 1 = Urothelial; 2 = Squamous; 3 = Neuroendocrine/Small Cell; 4 = Plasmacytoid; 5 = Adeno; 6 = Micropapillary; 7 = Papillary; and 8 = Other.In addition to the above, 41 samples had a missing value in this field, which signified that only the predominant histology was significant or present. Consequently, with secondary histology being the main discriminator, these 8 aforementioned categories were reduced to 5 groups, based on the process described in Algorithm 1.Since this process ignores insignificant histological differences with respect to the problem in focus, it provides a greater number of samples for each cancer subtype group. This grouping may therefore result in more insightful analyses  with respect to the clinical image of each patient, as well as potentially reveal valuable discriminating factors, such as differentially expressed genes, among the different groups.Algorithm 1: Histology-based sample groupingSimilarly, another clinical endpoint that underwent the same pre-processing was the T-Stage at surgery. According to the American Joint Committee on Cancer (AJCC) Staging Manual  there arDifferential gene expression (DGE) analysis is a very important step and one of the most common applications of RNA sequencing data. The main purpose of this process is to highlight potential systematic differences between the expression levels of certain genes among different patient groups. These groups may be defined by the straightforward distinction between control patients and patients who represent actual disease cases. At the same time, they can consist of sets of patients separated based on population demographics such as age or sex, health risk factors such as smoking or dietary habits, or different clinical characteristics such as cancerous tumor stage or therapy response, etc. The set of genes that might emerge as differentially expressed can then be utilized as biomarkers for either diagnostic or prognostic purposes in the clinical environment. In the scope of this work, a rather simplistic approach was followed in the direction of DGE analysis. This was mainly driven by the following two facts: the available RNA-Seq data were pre-processed  and the DE genes were not directly reported/relied upon.DEv2. This short pipeline (DEv2) was initiated by filtering out the majority of the genes around the mean level of expression, as explained in p-value cutoff, also known as A decision was made to not incorporate any of the popular packages already found in the literature (DESeq2) , edgeR ,28, EBSep is the cutoff, and the result is a structure that holds the p-values for each gene, which are given by the row names. Columns 1 and 2 pertain to the over or underexpression of the genes.Here, t-test ”, it requires that the groups under comparison be independent. This requirement is always satisfied during the experimental processes followed in this work.As noted above, the Wilcoxon Rank Sum Test, also known as Mann-Whitney U test, is a very popular nonparametric method employed for the comparison of outcomes between two groups. Often described as the “nonparametric version of a two-sample p-values). Contrary to the t-test’s null hypothesis of equal means, the Wilcoxon Rank Sum Test’s null hypothesis assumes equal medians, indicating a focus of the latter on the whole distribution of values in each group.The purpose of such tests is to be able to reject a null hypothesis, in other words the assumption that there is no difference between two populations, with a level of certainty higher than a specific threshold , as opposed to other patients, member of a different group B . This amount of time is known as survival time [As mentioned in the background section, one of the most frequently used indicators in medical research is the time elapsed from the beginning of surveillance until an event  takes place for patients belonging to a group val time , whereasThe most popular approach to the survival analysis problem is the Kaplan–Meier (KM) estimator . It is aN observed lifetimes (time - to - event), assuming an ascending ordering with respect to their length, we have r indicates these values for which Also known as the Product–Limit (PL) estimate, the KM estimate can be defined in the following manner: For y-axis and time on the x-axis. The entire aforementioned process is part of the background of many popular packages of statistical software, such as the survival R package [For the vast majority of use-cases, the results are presented and examined in a visual manner through the Kaplan–Meier curves. These curves are easy to interpret in that they are intuitive plots that take the shape of declining horizontal steps, depicting the percentage of survival on the  package ,38, whic package , which cAs mentioned in the dataset section, the discovery cohort used include gene expression levels for 46,050 genes. This is a very large set of variables, which greatly outnumbers the number of available samples and contains a lot of uninformative data. Not all genes that are profiled are overexpressed or inhibited in a population of cancer patients, hence seeking potential biomarkers among the entire group adds unnecessary delays due to the complexity of the discovery process which increases the run-time without providing useful insights.A simple yet effective additional step for commencing the differential expression analysis on a limited but more suitable candidate genes subset is to select a mean expression level cutoff. A cutoff value of 0.7 was selected in order to discard a significant part of genes irrelevant to a cancerous condition. This value also allows for greatly inhibited genes to be included.It is important to note that the starting set of genes had been initially reduced to the intersection of the ones present in the UofC cohort by the ones found in data generated by the TCGA Research Network , accesseAs mentioned in the “Logistic Regression” subsection, a crucial part of the dimensionality reduction pipeline is the usage of the elastic-net regularization method for the logistic regression model. This model behaves particularly well in sparse modeling problems, such as the one studied here.Another technique employed for the same purpose of selecting a few suitable features out of a deluge of candidate variables is the popular Principal Component Analysis (PCA) ,46,47. T1.z is the scaled data point, and it is implemented in Python’s sklearn library as the sklearn.preprocessing.StandardScaler class [Data standardization: This is a very important first step to ensure that each continuous variable included in the analysis process has an equal contribution . Variables with significantly larger ranges (variances) than other ones might dominate the resulting Principal Components, since they will seemingly be carrying more information. The most simple standardization formula iser class .2.The next step is the calculation of the covariance matrix. This information will allow for the discovery of potentially correlated variables and, therefore, the elimination of redundancies.3.Finally, in order to identify the Principal Components, the eigenvectors and eigenvalues of the covariance matrix have to be calculated. Sorting the eigenvalues in a descending order essentially reveals the ordering of the most important vectors, meaning the ones that explain the greatest amount of variance of the data.In brief, the PCA consists of the following steps:K-Means is available through the sklearn package as the sklearn.cluster.KMeans class, and it was the clustering method of choice for this research. In order to come up with the optimal number of consistent clusters, the silhouette method [matplotlib module’s functions, the clusters formed could be clearly visualized. An example of this can be seen in An early effort to cluster the patients incorporated the above process through a Python pipeline. Specifically, sets of genes (signatures) that would be found as potentially correlated to a certain clinical endpoint were used for variable selection. Then, following the data scaling, the clustering process would take place. Arguably the most popular clustering method, e method  and the e method  were useUnfortunately, the resulting clusters, although they usually managed to distinguish well most of the therapy responders from the non-responders, did not provide adequate results for robust conclusions in the clinical setting, according to a medical expert. For that reason, the rest of the clustering analysis took place in the R language pipeline, based on different methods as well as different packages.In the field of bioinformatics, specifically in the genomic data analysis area, clustering is heavily used, not always as a standalone method but often in conjunction with heatmaps ,52,53.ConsensusClusterPlus R package [For the experimental processes in this research, clustering was implemented through the  package . This mo package . The ide package .PAM clustering is more robust compared to the more popular K-means approach, since it tries to minimize a number of dissimilarities instead of squared Euclidean distances of each data point to a central cluster point. PAM, however, is noted to be susceptible to the issue of handling large or high-dimensional datasets, therefore making it less desirable for clustering gene expression data. However, this does not pose an issue in this work since the high dimensionality issue is being addressed prior to the clustering step, and the number of genes has been greatly reduced.Aside from the algorithm, another important aspect of the clustering process is the distance metric. Unlike the aforementioned Euclidean distances or any geometrically defined metric, Spearman correlation  values wVisualization of clustering results was achieved through the utilization of the heatmap3 R package .The heatmap, initially appearing as a shading matrix to aid in statistical analysis ,60, has This data visualization technique provides a compact description of the quantitative variables of interest in a 2D matrix format containing area marks colored with a diverging colormap. In its cluster form, the heatmap includes trees drawn on the borders showing how the matrix is ordered with respect to the derived clusters which, in turn, are the result of the hierarchical clustering applied on the data. Specifically, in the field of computational biology, the depicted data involve the level of expression of a set of genes along a group of different samples (patients). For example, in In order to be able to use a potentially meaningful signature in conjunction with time-to-event data under an estimator, such as the Kaplan–Meier, the nature of the predictor variables had to change. To be specific, a predictive model based on a certain gene signature would utilize the level of gene expression for each one of the genes that comprise the signature itself. These levels are, by definition, continuous variables. A discretization step is therefore necessary at this point.glmnet models, is rather suitable for achieving the discretization goal, since it nicely quantifies the chances that a sample exhibits an event or not.A first step towards a straightforward solution to this problem is to create a signature score. Adopted by other research work in the field such as Hsiao et al. , this apThe essential discretization process happens as a second step, where the aforementioned signature scores are grouped into different risk categories. Since the score itself reflect the probability of the event of interest taking place or not, the splitting does not have to be very sophisticated. For that purpose, the quartile base R function was employed. The output of said function is the a set of threshold values that define the quartiles of the data at hand. Quartiles divide the data points in four parts in an orderly manner, such that the first quartile is the middle value between the minimum value available and the median, the second quartile is the median itself, and the third quartile is the value between the median and the data maximum ,64. OnceOne of the main difficulties in this work was obtaining two separate patient groups, one for discovery purposes and the other for validation. The solution employed here was to split the data into training and test sets, as is traditionally done in machine learning model training. A common practice in many research works that involve biomarker discovery or the analysis of different risk groups in general also includes using an “in-house” patient cohort for discovery and external groups for validating potential findings. Given the peculiarities of the University of Calgary cohort, meaning that it consists of urothelial bladder cancer patients that underwent neoadjuvant chemotherapy, it is particularly demanding to find a suitable validation set.EGAS#00001002556. There were mainly two issues with utilizing this patient cohort. First, the bureaucratic process required in order to get access to the data was significantly lengthier than initially expected, with the first inquiry taking place on 31 May 2021, and the data only becoming available on 1 August 2021, allowing for little to no time to work with it. In addition to that, as opposed to the format of data utilized in this work, which is gene expression levels as continuous variables, the RNA sequencing information provided through this dataset was in unprocessed FASTQ [One such dataset available online was deposited to the European Genome-Phenome Archive ,66, undeed FASTQ  format, After extensive research of candidate validation sets and experimenting with some of them, such as a collection of samples curated by multiple TCGA platforms , a decisOne of the major issues that had to be mitigated was the lack of fixed training and test sets during model development. Each time a logistic regression model was created, new patients would be picked for training and validation. As mentioned previously, stratified sampling was ensured; however, there was no guarantee that the composition of the sets would remain identical. Furthermore, a consequent implication of this is the fact that based on the patients the model would be trained on, the genes that would be picked as model predictor variables would change as well. To tackle these problems initially, internal validation was approached through a model “consensus”.The four main components of the pipeline are, in order, the DEv2 custom function for the simple differential expression analysis step, the training of an elastic-net regularized logistic regression model, the counter of the selected gene occurrences, and the final logistic regression model .alpha taking a value of 0.01, which can be considered more strict, significantly reducing the number of genes qualifying as differentially expressed.During the first step, the execution of DEv2, a simple filtering was applied in order to eliminate candidate genes where the level of expression was very close to 0. The threshold value was set to 0.7. Following that, a Mann–Whitney U test was conducted, with the significance threshold Having reduced the potential predictor genes from tens of thousands to usually at most a few hundred, the second step was to create a logistic regression model. The important point here, however, is not the model, but the selection of predictor variables. The elastic-net regularization process eliminated most genes and was limited to just a few that were optimal for said model.These genes were counted in the occurrence counter and which concluded one gene picking cycle. This process was repeated 100 times. After its completion, the most frequently picked genes were gathered in a vector, in descending order of frequency.The 100 most frequent genes were, in their turn, fed as the input for the final logistic regression model creation, where the elastic-net once again picked the best predictors for the model. This method focused on “standardizing” the genes, and not the patients, for the training set. An abstract illustration of the pipeline is provided in A second, more complex but also more complete approach was developed, having as a main objective the standardization of the patient sets. The key idea was picking a training-testing split that achieved the maximum overlap  with a gThe first stage for this approach was an iterative process, highly similar to the consensus approach. There was, however, a significant difference: the gene selection is better informed since the entire dataset is taken into consideration for the model creation. The frequency of the selected genes is not as important as the size of the overlap; therefore, any gene that is selected even once is kept in a gene collection. Once the collection is complete, the process goes into a second, very similar iterative phase, where the differentially expressed genes are more scrutinized  as the best, only if the gene overlap is bigger than the previously found and the accuracy of the new model is not worse than the one already acquired. The output of this function led to fixing the training set for the final model creation, and in doing so it greatly impacted the gene selection for the resulting signature.One of the main objectives for discovering a biomarker related to a certain clinical endpoint is, ultimately, to be able to build a model that will ideally have prognostic capabilities. Since many such endpoints indicate either the occurrence or the absence of an event gression .logit and it is defined as follows:P is the result of the logistic function, also known as the sigmoid functionx can be defined as a linear regression equationa is the constant, also called intercept of the equation, and i, with Binary logistic regression utilizes the logistic function by log-transforming the odds ratio. The result of this step is called a x, we need a decision boundary. For this kind of applications a boundary of 0.5 is usually picked, such thatIn order to be able to classify a sample elastic-net regularized regression [LASSO [ridge [P exceeds the number of samples N, a scenario that is rather commonplace in genomic data analysis.The exact method that was chosen was the gression . The elan [LASSO  and ridgO [ridge ,72 methoO [ridge . This prglmnet [partition function of the R package splitTools [stratified splitting, ensuring that an adequate number of samples from all classes will be present in each set. Following that step, cv.glmnet was employed in order to complete a k-fold cross-validation (k set to 10) and come up with the optimal regularization parameter lambda.min) that minimizes the mean cross-validation error. Finally, the glmnet model training function was called, with lambda.min  are perfectly separated. In order to potentially further improve the predictor’s performance, R’s Finding the optimal class separation threshold (cutpoint) was done for both endpoints. For Therapy Response, that value was found to be equal to 0.9535. Note that this value refers to each model’s response, which is the probability of each sample for a certain class membership. Considering the newly found threshold, the accuracy improves by approximately 3%, reaching 80.7%. For Disease Progression, however, the improvements are even greater. The reason for such an improvement can be somewhat implied by the rather high 95% Confidence Intervals reported above. With a revised threshold drawn at a probability of 0.21%, the model is a perfect predictor, correctly classifying all samples.This is a particularly interesting result, since after a meticulous effort to eliminate any type of selection bias, two gene signatures emerged, capable of predicting either eight out of 10 times or perfectly the outcome of very important indicators for cancer patients.Unrelated to genomic biomarkers per se, additional insights were gained with regards to the correlation of various clinical indicators and the potential for some of them to act in a predictive manner.p-value = 0.0023, indicating a very significant correlation, said grouping can be a trustworthy estimator of the amount of days it will take for certain percentages of patients to exhibit disease progression after NAC . See Following a similar rationale, the indicator of T-stage at surgery was found to be significantly correlated with estimating the time-to-disease progression  for the same endpoint of time-to-disease progression  The generated sets of patients could act as a significantly correlated estimator (gression .The power of such findings lies in the fact that clinicians can presumably improve or ascertain their decision making, may that be for a treatment plan or for the duration of watchful waiting , withoutThe last reported result in this section is the outcome of a heatmap provided with the PAM algorithm’s clustering results of the patients as input. All patients and the most frequent genes as they occurred from the Consensus method see  were conIn this paper, various recent works were explored related to bladder cancer biomarkers. Different researchers have used a plethora of approaches to this topic from classic statistical approaches to very elaborate, state-of-the-art machine learning methods including Deep Networks, to different ones stemming from the perspective of systems biology, such as pathway analysis. We have focused on grouping based on therapy response and disease progression on the patients. Using the histology-based sample grouping, gene expression analysis, Wilcoxon Rank Sum Test, and clustering, we have reported some of the biomarkers which have exceptionally high correlation with the bladder cancer patient cohort. The Kaplan–Meier curves are presented for the survival analysis of the patients with explanation. One of the challenges we have faced which can be considered for future development is external validation. With the support of external datasets, we may validate the hypothesis more accurately.


Telemetry data are the most important basis for ground operators to assess the status of satellites in orbit, and telemetry data-based anomaly detection has become a key tool to improve the reliability and safety of spacecrafts. Recent research on anomaly detection focuses on constructing a normal profile of telemetry data using deep learning methods. However, these methods cannot effectively capture the complex correlations between the various dimensions of telemetry data, and thus cannot accurately model the normal profile of telemetry data, resulting in poor anomaly detection performance. This paper presents CLPNM-AD, contrastive learning with prototype-based negative mixing for correlation anomaly detection. The CLPNM-AD framework first employs an augmentation process with random feature corruption to generate augmented samples. Following that, a consistency strategy is employed to capture the prototype of samples, and then prototype-based negative mixing contrastive learning is used to build a normal profile. Finally, a prototype-based anomaly score function is proposed for anomaly decision-making. Experimental results on public datasets and datasets from the actual scientific satellite mission show that CLPNM-AD outperforms the baseline methods, achieves up to 11.5% improvement based on the standard  Satellites are currently one of the most sophisticated technological systems, performing their mission in harsh conditions . It is uAnomaly detection is a technique for identifying unexpected patterns of deviation from normal behavior and has attracted a significant amount of research attention in recent years ,6,7. SatSemi-supervised anomaly detection methods use a significant amount of normal data to create a normal profile of correlation between variables. Samples with behaviors deviate from the normal pattern are labeled as anomalies. Recently, deep anomaly detection methods have received a significant amount of attention, as they primarily employ well-designed encoders to capture complex correlations between variables, such as the autoencoder-based method ,11,12,13In this paper, we propose an anomaly detection method called contrastive learning with prototype-based negative mixing for anomaly detection (CLPNM-AD). Contrastive learning aims to learn a transformation-invariant feature representation and evenly distributes the sample on the hypersphere in representation space ,20. CombGoing beyond previous anomaly detection methods, CLPNM-AD can capture complex correlations between telemetry data variables, maintaining a local semantic structure while pushing samples with different semantics far apart, making normal and anomaly samples as separable as possible. Specifically, we first employ an augmentation process with random feature corruption to obtain augmented samples. Following that, a prototype consistency strategy is used to capture the semantic category information of the samples. After that, a prototype-based negative mixing contrast loss is used to obtain a normal profile. Unlike the vanilla contrast loss, which maps all samples evenly onto the hypersphere, the proposed method will make the true negative samples from all directions around the anchor point  have an effective gradient such that these samples are separable from the anchor, making normal and anomaly samples distinguishable. Finally, anomaly decisions are made using a prototype-based anomaly score function. Extensive experiments are conducted to verify the effectiveness of the proposed method.We propose CLPNM-AD, a correlation anomaly detection method, which combines prototype consistency and prototype-based contrastive loss to guide the encoder to construct an accurate normal profile, making normal and anomaly samples more distinguishable and thus facilitating the detection of anomalies.We apply a sample augmentation process with random feature corruption to generate positive samples for contrast learning and to help capture the complex correlations between variables.We combine prototype consistency and prototype-based contrast loss to learn features that facilitate the distinguishing of normal and anomalous samples. First, prototype consistency is proposed to capture the semantic categories of samples. Then, a prototype-based hard negative mixing strategy is applied to preserve local semantic information and push samples of different prototypes farther apart.We propose an anomaly score function that indicates the degree of the anomaly of a sample by calculating the Mahalanobis distance between the sample and the prototype conditional Gaussian distribution.We conduct extensive experiments to evaluate the performance of CLPNM-AD on three public datasets and one satellite telemetry dataset from our actual mission. The experiment results show the excellent performance of the proposed method.The contributions of this paper are summarized below:The remainder of the paper is organized as follows: Anomaly detection of satellite telemetry data has recently become a popular research topic, with a large body of research literature emerging. Traditional anomaly detection methods have mainly used statistical or machine learning methods to detect anomalies. In , a kerneSelf-supervised representation learning methods use the data themselves to construct supervised information to achieve data representation learning. In recent years, self-supervised deep anomaly detection methods have made significant progress, improving anomaly detection performance and gaining much attention. In , an encoContrastive learning as a type of self-supervised method aims to learn a transformation-invariant feature representation such that multiple views  of a sample keep attracting while repelling to other samples . In 29]29], the Contrastive learning learns features by comparing similar or dissimilar pairs of samples, so the quality of positive and negative samples determines the quality of the learned features. Contrastive learning is a self-supervised representation learning method that does not have access to sample labels. When anchor samples and negative samples form negative pairs, these samples may belong to the same semantic class, and the hardness-aware property of contrastive learning will cause these samples to be repelled, destroying the local semantic structure and affecting representation learning. These false negative samples continue to be a major issue in contrastive learning, but relatively little research work has been conducted in this area.In , the autAnomalies in satellite telemetry data are classified as point anomalies, contextual anomalies, collective anomalies and correlation anomalies, as illustrated schematically in Point anomalies refer to a single data point or several consecutive data points that deviate significantly from other data points. Contextual anomalies mean that the data points deviate a local threshold determined by the temporal context, although not deviating the global threshold. Collective anomalies are consecutive points that do not exceed a threshold, but the subsequence formed by these points violates the pattern of the original telemetry time series. As shown in Correlation anomalies occur when the stable correlation between multiple telemetry parameters is broken and the correlation between parameters exhibit different patterns. Correlations between satellite telemetry parameters include physical correlations, logical correlations and operational mode correlations. As shown in The correlations between satellite telemetry parameters in real-world scenarios can be very complex, necessitating anomaly detection methods capable of modeling such complex correlations. The objective of this paper is primarily to propose a correlation anomaly detection method that is sensitive to abnormal changes in the correlation between satellite telemetry parameters.N is randomly selected, and the contrastive loss is defined on augmented views generated from the mini-batch. Let Contrastive learning learns representations by grouping differently augmented views of the same data sample together . During ollowing ,27, we dWe first present the problem definition of satellite telemetry data correlation anomaly detection. Then, we give an overview of CLPNM-AD and briefly introduce its modules. Finally, we describe the proposed method in detail.This paper is based on the assumption that, in routine operation, correlations between satellite telemetry variables exhibit some common patterns, and that ground operators should focus on the few outliers that deviate from these common patterns.t is represented by a vector, as in the formj-th dimension of telemetry data at time t and M is the dimension of sample Each sample of satellite telemetry data from a subsystem or device at time b is the anomaly score that predicts how anomalous the sample x is, and then to create an anomaly decision function x is normal or abnormal:x is determined to be anomalous if its anomaly score, The semi-supervised scenario is addressed in this paper, in which the training dataset The CLPNM-AD framework is shown in x. We then propose a contrastive learning with the prototype-based negative mixing (CLPNM) method to learn the representation of the training data First, we use the augmentation functions As a vital part of contrastive learning, techniques to generate views are domain-specific U={Q∈Then, the optimization objective in Equation  can thusrn–Knopp  algorithn–Knopp [minQ∈U〈Q,Q is then used to guide the learning of the neural network parameters. To encourage the two augmented views N denotes the mini-batch size and K denotes the number of prototypes.The pseudo-label matrix Q, on one hand, is used to learn clustering consistency, and on the other hand, is used to mitigate the sampling bias and obtain hard negatives to guide the contrastive loss. The soft label is represented by Q in the preceding section, while we require the hard label in this part to reflect the semantic clusters of the data. To acquire the hard labels, we simply apply the The pseudo-label matrix C in We obtain hard negatives by mixing the anchor feature vector with sample vectors from different prototypes in the mini-batch. The weight of sample mixing is determined by the distance between prototypes. We denote the distance between two prototypes asFor the anchor feature vector The prototype-based contrastive loss is defined asThe final training objective couples Q. This is accomplished by alternately performing the following two steps:The core of the proposed method is to learn a model Step 1: representation learning. Given the pseudo-label assignments matrix Q, the parameters of model Equation .Step 2: self-labeling. Given the model P by using Equation ∀y:ΓTo determine whether a sample is normal or anomalous, we use representations and prototypes to assess the degree of anomaly of the sample. Refs. ,42 show Q in K prototypes, and all K conditional Gaussian distributions share a common covariance N data samples from the entire training set for the calculation (not the samples in a batch):N denotes the number of samples contained in the training set, c and Let x by the following equation:Based on the empirical class mean and the covariance of prototype conditional Gaussian distribution obtained above, we define the anomaly score The anomaly score indicates the degree of abnormality of each sample. The higher the anomaly score, the more abnormal the sample is. To achieve an anomaly decision for the samples, the protocol in the literature ,17 is ushttps://github.com/guoguohang/CLPNM_AD, (accessed on 15 March 2023).In this section, we describe the selected datasets and the baseline methods, and perform a series of experiments to validate the proposed method. Firstly, the proposed method is compared with the state-of-the-art methods and validated by statistical hypothesis testing. Then, the robustness of the method is verified on data with different contamination ratios. Furthermore, the effectiveness of each module of the method is verified by ablation experiments. Finally, sensitivity analyses are also implemented to explore the response of the model to different hyperparameters. The code is available at We adopt three public datasets: Thyroid, Satellite, Satimage and one actual telemetry data source from the Quantum Science Experiment Satellite, also known as Micius. The statistical information of the datasets is shown in http://odds.cs.stonybrook.edu/thyroid-disease-dataset/, (accessed on 29 March 2023)), which contains six continuous attributes. The original dataset contains three classes. As hyperfunction is a minority, researchers in the field of anomaly detection treat hyperfunction as anomaly class.https://doi.org/10.57760/sciencedb.o00009.00042, (accessed on 29 March 2023)), which contains 19 attributes, and the time span is from January 2017 to February 2019. Micius has four operation patterns. As pattern 4 is rare, we treat pattern 4 as anomaly class and the rest as normal class.http://odds.cs.stonybrook.edu/satellite-dataset/, (accessed on 29 March 2023)). Data from three minority categories, 2, 4 and 5, are combined to form the anomaly class, while the remaining classes are combined to form the normal class.http://odds.cs.stonybrook.edu/satimage-2-dataset/, (accessed on 29 March 2023)) and is also integrated from Landsat Satellite. Combining the training and test data in the Landsat Satellite dataset, there are 71 outliers in class 2 and all other classes are merged into a normal class.OC-SVM . One-ClaLOF . Local ODAGMM . Deep AuDeep SVDD . Deep SuGOAD . GOAD isNeuTraL AD . Neural The following six methods are selected to compare with CLPNM-AD:G denotes the set of real anomalous samples and R denotes the set of anomalous samples identified by our method.Following the settings in ,17, the k in the stochastic augmentation process This section describes the network structure as well as the details of model training. In our experiments, all the CLPNM-AD instances are implemented based on the Pytorch framework. For each dataset, the network structure of the encoder, projection head and prototype in CLPNM-AD are shown in Equation  is set tOn four datasets, we examine the effectiveness of CLPNM-AD compared with that of six baseline methods. We use bold font to indicate the best OC-SVM maps samples to one side of the hyperplane using a kernel function and does not take into account the semantic information shared by different samples. Simply mapping these samples to one side of the hyperplane tends to result in the overlapping of samples and thus makes it difficult to distinguish between normal and abnormal samples. Therefore, the results in LOF uses the distance between a sample and its surrounding points for anomaly detection, which takes into account the local similarity of the samples and to some extent the semantic similarity information of the samples. As a result, LOF outperforms OC-SVM, particularly on the Micius dataset, where OC-SVM performs suboptimally. However, because LOF is based on the original samples and lacks feature extraction capability, it cannot effectively model the data’s characteristics.DAGMM uses the reconstruction error and intermediate layer vectors as feature vectors, followed by a Gaussian mixture model for anomaly detection, achieving good performance on most datasets because the mixture component of the Gaussian mixture model models the semantic category information of the samples. However, DAGMM performs poorly on the Micius dataset because the distinction between normal and abnormal samples was not very clear, and it cannot map different semantic category samples apart, resulting in poor performance.DSVDD achieves anomaly detection by mapping samples into a hypersphere, but it lacks the ability to capture the semantic information of the samples and maps all samples into the same hypersphere, which results in samples with different semantic information overlapping in the feature space and does not facilitate the distinguishing of normal and abnormal samples. As can be seen from GOAD augments the samples with a random affine transformation and then maps the augmented samples to the feature space for anomaly detection by predicting the transform used. The affine transform is used as the semantic class in this approach, and the augmented samples obtained using the same affine transform are clustered into one class in the feature space. Such an approach lacks sample semantic information modeling and achieves good results when the anomalous samples are of a single semantic category and the anomaly rate is low, e.g., as shown in In comparison to GOAD, NeuTraL AD employs a trainable multilayer neural network as the data augmentation function and a loss function similar to contrast loss to separate samples with different transformations for anomaly detection. The method suffers from the same limitation of a lack of sample semantic information modeling, and thus its performance is inadequate, as shown in Overall, CLPNM-AD outperforms all baselines on four datasets. The one-dimensional convolutional network in CLPNM-AD helps to capture the features of the samples, which can be adapted to anomaly detection tasks on different datasets. In addition, the consistency loss in CLPNM-AD helps the network to capture the semantic information of the samples, while the hard negative mixing strategy provides stronger guidance to help the contrast loss to separate and disperse the semantically different samples onto the hypersphere as much as possible, which allows the model to accurately profile the normal samples and thus make the normal and abnormal samples more distinguishable.p-value, and the hypothesis is tested at a significance level of To quantitatively assess the effectiveness of CLPNM-AD, we statistically evaluate the results of 20 runs of CLPNM-AD with six baseline methods on four datasets using the Wilcoxon rank sum test. We test which of the null hypothesis H0 or the alternative hypothesis H1 holds for each pair of methods: H0: A ≈ B, H1: A > B, where A represents the As shown in In general, on all datasets, the CLPNM-AD method outperforms all baseline methods, and the advantage is statistically significant.In satellite telemetry anomaly detection applications, the training data are frequently mixed with noisy data, or the anomalous samples are incorrectly labeled as normal during the training dataset construction, so we need to investigate the model’s sensitivity to contamination. In this experiment, we investigate how CLPNM-AD and all baselines respond to contaminated training data. We adopt the experiment setup in the literature  accordinCLPNM-AD has a certain tolerance for contaminated data, which may be because the consistency loss allocates the samples in a batch equally into each semantic category, preventing contaminated samples from being assigned to the same cluster. As a result, the contaminated samples have less impact on the probability distribution calculation of each semantic category, making the model somewhat robust to the anomalous samples mixed in the training set.In general, CLPNM-AD is robust against contaminated data, maintaining good performance as the contamination ratio In this experiment, we investigate the impact of various modules in the model. We repeat our experiment with/without a specific key module. These key modules are primarily vanilla InfoNCE loss As shown in K and batch size N on the performance of CLPNM-AD on the Thyroid, Micius, Satellite and Satimage datasets.In this section, we discuss the effect of the number of prototypes K increases from 4 to 24. We conjecture that the model degeneration on Thyroid is due to the fact that the dataset itself has specific semantic clusters, the number of which is close to four, and increasing the number of prototypes too much will break this semantic structure.N is varied. On Thyroid, however, CLPNM-AD was more clearly affected by the batch size, with the method achieving the best performance at batch size In this paper, we propose a correlation anomaly detection method for telemetry data, namely CLPNM-AD. The CLPNM-AD framework first employs an augmentation process with random feature corruption to obtain augmented samples. Following that, a prototype consistency strategy is used to capture the semantic category information of the samples. After that, prototype-based negative mixing contrastive loss is used to pull positive samples from the same prototype closer together and to push samples from different prototypes farther apart to obtain a profile that makes normal and anomaly distinguishable. Finally, anomaly decisions are made using a prototype-based anomaly score function. Extensive experiments are carried out to demonstrate superior performance to baselines on a variety of datasets, with up to 11.5% improvements on the standard The method proposed in this paper combines contrastive learning and prototype learning to train an encoder to build a profile of normal telemetry data and then use that profile to distinguish between normal and anomalous samples, allowing for better anomaly detection. Our contribution is to use the distance between prototypes as a weight to synthesize hard negative samples and guide encoder learning, resulting in a more discriminative normal profile and a new framework for detecting correlation anomalies in telemetry parameters. In our real-world satellite operation tasks, the method complements existing limit checking and expert system approaches.The results of the ablation experiments in


The progression of complex diseases sometimes undergoes a drastic critical transition, at which the biological system abruptly shifts from a relatively healthy state (before-transition stage) to a disease state (after-transition stage). Searching for such a critical transition or critical state is crucial to provide timely and effective scientific treatment to patients. However, in most conditions where only a small sample size of clinical data is available, resulting in failure when detecting the critical states of complex diseases, particularly only single-sample data.In this study, different from traditional methods that require multiple samples at each time, a model-free computational method, single-sample Markov flow entropy (sMFE), provides a solution to the identification problem of critical states/pre-disease states of complex diseases, solely based on a single-sample. Our proposed method was employed to characterize the dynamic changes of complex diseases from the perspective of network entropy.The proposed approach was verified by unmistakably identifying the critical state just before the occurrence of disease deterioration for four tumor datasets from The Cancer Genome Atlas (TCGA) database. In addition, two new prognostic biomarkers, optimistic sMFE (O-sMFE) and pessimistic sMFE (P-sMFE) biomarkers, were identified by our method and enable the prognosis evaluation of tumors.The proposed method has shown its capability to accurately detect pre-disease states of four cancers and provide two novel prognostic biomarkers, O-sMFE and P-sMFE biomarkers, to facilitate the personalized prognosis of patients. This is a remarkable achievement that could have a major impact on the diagnosis and treatment of complex diseases. TherefoRecently, the dynamic network biomarker (DNB) has been proposed to qualitatively describe the critical state of a biological system . SpecifiSpecifically, inspired by the previous study , 2021b, i.e., a significant alteration in sMFE score indicated an early warning of the critical transition to disease deterioration. By searching the literature, we found that the following clinical characteristics during the deterioration of diseases  occur, and distant metastasis occurs in the later stage of cancer development, which is difficult to recover by external treatment and can be regarded as the basis for disease deterioration. Specifically, the critical state for ESCA was identified in stage IIIA disease before distant metastasis, and stage IIB was identified as the critical state for COAD before lymph node metastasis; stage II was identified critical state for KIRC before the tumor migrates to form distant metastasis, and that of LUAD was identified in stage IIIB disease before the tumor cells invaded distant tissues or organs. In addition, we further analyze the signaling gene and propose two types of biomarkers, i.e., optimistic sMFE (O-sMFE) and pessimistic sMFE (P-sMFE) biomarkers , colon adenocarcinoma (COAD), kidney renal clear cell carcinoma (KIRC), lung adenocarcinoma (LUAD), were used to validate our algorithm. In all datasets, the critical states identified by sMFE preceded severe disease deterioration, in) between any pair of members in the DNB group rapidly increases;The Pearson correlation coefficient (PCCout) between one member of the DNB group and any other non-DNB member rapidly decreases;The Pearson correlation coefficient (PCCin) or coefficient of variation for any member in the DNB group drastically increases.The standard deviation   was consSpecifically, given an interaction network G with py (MFE) . For cenReference samples  and only one disease sample were required in the sMFE method to identify the tipping point with the following algorithm .http://string-db.org) ((Step 1) Constructing a global directed network -db.org) , with a -db.org)  method. (Step 2) Mapping data into the global network L, and (Step 3) Calculating the local Markov flow entropy (MFE) for each gene. Specifically, we extract each gene ument}NG , the numM represents the count of neighbors in the local network n reference samples at time point t. Specifically, n reference samples at time point t.n + 1 mixed samples, where a single sample from an individual was added to n reference samples at time point t, the same with the prior step, i.e.,(Step 4) Calculating the differential MFE called Then, the n reference samples and n + 1 mixed samples at time point t, respectively.i.e.,(Step 5) The fifth step is to calculate the global single-sample Markov flow entropy sMFE. Furthermore, t-test was applied to test if the identified critical state significantly different with the other states. The alternative state was viewed as critical state if P < 0.05 ) was used to download four unrelated clinical tumor datasets: esophageal carcinoma (ESCA), colon adenocarcinoma (COAD), ), and lung adenocarcinoma (LUAD). RNA-seq data from tumor and tumor-adjacent samples, as well as clinical information, were included in these datasets. Then we classified the tumor samples into different stages based on clinical (stage) information downloaded from TCGA, samples lacking stage information were discarded, and The Cancer Genome Atlas (TCGA) database . This data was then used to generate a global template network, which was composed of nodes representing proteins and edges representing interactions between proteins. All interaction used for discussion was picked with high confidence (higher than 0.8). Nodes that are not connected to any other nodes are removed from the PPI network.The first step in constructing the molecular global template network was to download the protein–protein interaction networks for Second, the direct interaction network was constructed based on rewiring the protein–protein interaction (PPI) network by GNIPLR (gene networks inference using projection and lagged regression)  method. In the end, the directed network, created in prior step, was adapted to map the gene expression of each RNA-seq dataset to form a molecular network for further analysis.ncifcrf.gov)) were used to conduct the enrichment analysis. These statistical analysis carried in the article was done using R Software v 4.0.3 ), and the R Survival package was used to make Cox survival analysis. The visualization results and network analysis are implemented using Cytoscape software (www.cytoscape.org).The Kyoto Encyclopedia of Genes and Genomes (KEGG) database Mapper tool (KEGG Mapper Color) and the Database for Annotation, Visualization and Integrated Discovery (DAVID) Functional Annotation Tool  from TCGA datasets. These datasets contain much clinical information, in which doctors divide cancer patients into different stages according to the size of the tumor diameter, such as KIRC samples were divided into four stages , while LUAD samples were divided into seven stages . The tumor-adjacent (TA) samples were deemed relatively normal and used as a reference for each cancer when calculating sMFE according to the proposed algorithm. The average sMFE score curve for each cancer stage was calculated to reflect the dynamic change and was shown in P = 0.0077) at stage IIIA, suggesting an upcoming critical transition after stage IIIA.; that is, the tumor migrates and invades to form distant metastasis at stage IIIB and ultimately cause distant metastasis (stage IV)  . While atage IV) . Of noteP = 0.0045; P = 0.16) in the mean gene expression of DEGs were detected in stage IIIB and therefore failed to provide timely early-warning signals of the critical transition. Moreover, it is worth noting that survival analysis showed that the survival time of samples taken before the transition point (Stage IIB) was significantly different than that taken after the transition point (Stage IIB)  in the sMFE score was observed in stage II  between stage I~II and stage III~IV KIRC samples according to clinical information , indicating an upcoming critical transition after stage IIIB; that is, stage IV was characterized by a distant metastasis process, in which the tumor cells invaded distant tissues or organs  consist of common signaling genes in samples in the identified critical state. In DNB theory, these DNB molecules exhibit statistical properties when the system reaches a critical point. Biologically, they may be involved in important biological reaction processes during disease progression. These DNB molecules were mapped to the STRING database to form a connected PPI network, and their dynamic changes in the process of disease evolution were studied from the network level.i.e., stage IV was characterized by a distant metastasis process, in which the tumor cells invaded distant tissues or organs  biomarkers; and those with good prognosis, termed optimistic sMFE (O-sMFE) biomarkers. Furthermore, those two types of biomarkers are not only the signaling genes, pessimistic/optimistic biomarkers, but also the non-differentially expressed genes usually screened out by most researchers in the first step. More details about the identification method are provided in SCYL3 and TRPS1 (P = 0.016 and 0.036 respectively) were longer than those without, as shown in EPCAM and POLK  had shorter survival times than those without.The meaning of these two biomarkers can also be understood in this way, samples of optimistic sMFE biomarkers that appeared in their signaling gene had better-predicted prognoses than those without; in other words, these samples were expected to survive longer than others. Conversely, samples with pessimistic sMFE biomarkers in their signaling genes had lower predicted prognoses than those without; that is, these samples were expected to survive for a shorter time than other samples. A total of four genes were identified as optimistic sMFE biomarkers for ESCA , while sAGR2  had better prognoses, with longer survival times, compared to other samples. Conversely, samples with P-sMFE biomarkers BST1 and VPS39  had worse prognoses, with shorter median survival times of almost 5 years. As shown in ACOX2  promotes abnormal proliferation of quiescent cells by enhanced JUNB expression via the MAL-SRF pathway (PIK3CA-mutation was reported related to metastatic breast cancer (Protein tyrosine kinase 2 (PTK2) was a novel therapeutic target to overcome acquired EGFR-TKI resistance in non-small cell lung cancer and as a driver of radioresistance in HPV-negative head and neck cancerPTK2/FAK and radioresistance  method was proposed in this study. Its effectiveness and reliability have been tested across multiple diseases.i.e., pessimistic sMFE (P-sMFE) and optimistic sMFE (O-sMfE) biomarkers. Furthermore, those two types of biomarkers not only the signaling genes, pessimistic/optimistic biomarkers, but also the non-differentially expressed genes and are therefore usually screened out by most researchers in the first step, so they can provide novel insight for further study into the molecular mechanisms of how tumor onset and disease deterioration work.In this study, we developed a computational method, single-sample Markov flow entropy (sMFE), to explore the dynamic changes in combined effects on molecular associations and thus characterize the perturbation of a single sample close to the tipping point. In addition, functional enrichment analysis revealed that the common signaling genes for four cancer datasets are involved in significant biological processes or pathways, for instance, the cell cycle , PI3K-Aka priori knowledge-based PPI network by GNIPLR, which is different from the past studies that mainly focused on undirected networks. Third, the sMFE method provided two distinct types of biomarkers applicable for identifying novel biomarkers unlike any previous one, drug targets in drug development and prognostic indicators for prognostic analysis.In a word, the sMFE approach has several important advantages. First, this method is a model-free data-driven and single-sample-based algorithm, which is conducive to the development of personalized medicine. Second, our method was based on the direct interaction network, constructed from an The sMFE method provides a solution to the identification problem of critical states/pre-disease states of complex diseases solely based on a single sample, making it suitable for real-world clinical data. In addition to effectively identifying the critical states or transition points of the four cancers, our method provided two new prognostic biomarkers, optimistic sMFE (O-sMFE) and pessimistic sMFE (P-sMFE) biomarkers. Therefore, this method has great practical application potential in the fields of personalized medicine, identification of molecular mechanisms of disease progression, and preventive medicine.10.7717/peerj.15695/supp-1Supplemental Information 1i.e., the signaling genes of identified samples included the given candidate gene i, while unidentified samples didn’t. The given candidate gene i is regarded as the candidate biomarker when it satisfies the following formula. n/(n+m) > A or m/(n+m) > A, where n denotes the number of identified samples from the long-survival group, and m denotes the number of identified samples from the short-survival group. The A is an empirical parameter and set as 0.6. [Step 3] The third step is to identify two types of biomarkers (optimistic sMFE biomarkers and pessimistic sMFE biomarkers) from candidate biomarkers. Specifically, for the candidate biomarker (gene j), the prognosis results respectively based on identified samples and unidentified samples were exhibited and compared through Kaplan–Meier (log-rank) survival analysis. If the survival curve of identified samples were significantly longer than that of unidentified samples , gene j was perceived as an optimistic sMFE biomarker (O-sMFE). Similarly, if the survival curve of identified samples was statistically shorter than that of unidentified samples , gene j was perceived as a pessimistic sMFE biomarker (P-sMFE).[Step 1] The first step is to categorize sample s into a long-survival group and a short-survival group based on their clinical information, that is, the samples with a survival time greater than the median survival time were deemed as the long-survival group, and vice the samples with a survival time less than the median survival time were considered as the short-survival group. [Step 2] The second step is to find candidate biomarkers. For each sample, the signaling genes  that non-differently expressed were selected as a set of candidate genes. The samples were divided into two groups (identified samples and unidentified samples) according to whether their signaling genes contained the given candidate gene i, Click here for additional data file.10.7717/peerj.15695/supp-2Supplemental Information 2Results of the dynamic network changes for the four cancers.Click here for additional data file.10.7717/peerj.15695/supp-3Supplemental Information 3Click here for additional data file.10.7717/peerj.15695/supp-4Supplemental Information 4Based on sMFE score, there is a critical transition around stage III.Click here for additional data file.10.7717/peerj.15695/supp-5Supplemental Information 5TA samples, tumor-adjacent samples.Click here for additional data file.10.7717/peerj.15695/supp-6Supplemental Information 6Click here for additional data file.10.7717/peerj.15695/supp-7Supplemental Information 7Click here for additional data file.10.7717/peerj.15695/supp-8Supplemental Information 8Click here for additional data file.


Modeling of single cell RNA-sequencing (scRNA-seq) data remains challenging due to a high percentage of zeros and data heterogeneity, so improved modeling has strong potential to benefit many downstream data analyses. The existing zero-inflated or over-dispersed models are based on aggregations at either the gene or the cell level. However, they typically lose accuracy due to a too crude aggregation at those two levels.We avoid the crude approximations entailed by such aggregation through proposing an independent Poisson distribution (IPD) particularly at each individual entry in the scRNA-seq data matrix. This approach naturally and intuitively models the large number of zeros as matrix entries with a very small Poisson parameter. The critical challenge of cell clustering is approached via a novel data representation as Departures from a simple homogeneous IPD (DIPD) to capture the per-gene-per-cell intrinsic heterogeneity generated by cell clusters. Our experiments using real data and crafted experiments show that using DIPD as a data representation for scRNA-seq data can uncover novel cell subtypes that are missed or can only be found by careful parameter tuning using conventional methods.scpoisson.This new method has multiple advantages, including (1) no need for prior feature selection or manual optimization of hyperparameters; (2) flexibility to combine with and improve upon other methods, such as Seurat. Another novel contribution is the use of crafted experiments as part of the validation of our newly developed DIPD-based clustering pipeline. This new clustering pipeline is implemented in the R (CRAN) package The online version contains supplementary material available at 10.1186/s12859-023-05349-2. Single cell RNA-sequencing (scRNA-seq) estimates the transcriptome at the individual cell level. ScRNA-seq can directly measure cell-to-cell heterogeneity, which is more challenging using bulk RNA sequencing. First applied in 2009 , scRNA-sExisting methods typically model the scRNA-seq data at the gene level for differential expression analysis to find biomarkers, and at the sample level for clustering of cells to find cell subtypes; however, they may lose accuracy due to a too crude aggregation at those two levels. This aggregation has led to attempts to explicitly model the apparent resulting zero-inflation or over-dispersion. We propose more precisely addressing these issues by modeling the distribution of each individual entry of the data matrix.A major challenge is that scRNA-seq data typically contain a large number of zero counts for gene/cell combinations (often exceeding 90%) . This isVarious approaches have been proposed to address barriers that limit the interpretation of scRNA-seq data –23. On tPoissoneity to sets of similar matrix entries is studied using Quantile-to-Quantile plots (Q-Q plots), together with simulated envelopes indicating natural variation, in addition to over-dispersion and zero-inflation hypothesis tests.To improve the accuracy of statistical modeling and gain more precise inference, we propose a novel and principled approach to studying individual entries of the gene-by-cell matrix. This approach is based on the independent Poisson distribution (IPD) statistical framework, where every gene in each cell follows its own Poisson distribution. Working with such a model is challenging because the maximum likelihood estimate of each Poisson parameter is simply the corresponding count, which is too noisy to be useful. To solve this problem which presents for the validation of the IPD model we first start with several biologically homogeneous data sets derived from single clonal cell lines . Next, wBased on this newly proposed IPD framework which focuses on individual entries of the scRNA-seq data matrix, we further develop procedures based on the computation of Departure from the IPD (DIPD) as a data representation to replace the scRNA-seq count data by the logistic transformation of probabilities of Departure to ensure modeling accuracy and to effectively deal with zeros. The output will be a data matrix of the same dimension of scRNA-seq with continuous values.model departure).This enables our development of other new computational approaches including clustering and other downstream tasks through the novel concept of DIPD. The DIPD is initialized by a rough two-way parameter approximation of the data. Next, different cell types are captured by departures from the naïve two-way approximation. Then the data is bisected using Poisson departure as the distance measure. The clustering algorithm terminates when there is no significant deviation from Poissoneity for any cell group. For some data this approach gives results similar to those using other pipelines . For othcrafted experiments for the comparison of the DIPD with other methods in a principled way. While we demonstrate the value of our proposed model departure data representation for clustering, we anticipate it will be useful for additional downstream tasks, such as differential expression analysis, gene set tests and trajectory analysis, because it provides a useful replacement for the conventional data matrix.In sum, the IPD statistical framework has the potential to capture meaningful biological properties at a higher resolution than prior normalization methods, without the need for more complicated probability distributions. We demonstrate the usefulness of model departure DIPD as a novel data representation by conducting downstream analysis, such as clustering of cells. Our newly developed DIPD-based clustering pipeline is validated in multiple experimental data. Another important contribution of this paper is the use of the novel method called Poissoneity postulates that each matrix entry (gene by cell) comes from an independent Poisson distribution. As stated in Methods, the Poisson parameter for each matrix entry can be estimated using GLM-PCA . The sucTo study the Poissoneity of scRNA-seq data, we first explore the simplest case: cells picked at random from a clonal cell line processed as a single batch  with \dmentclass2pt{minimFigure The Kolmogorov-Smirnov (KS) test is employed to assess the deviation of the observed data from the theoretical distribution. It calculates the maximum vertical distance between the empirical cumulative distribution function (CDF) of the observed data and the theoretical Poisson CDF, is used to quantitatively measure the deviation on Q-Q envelope plots. A larger KS statistic indicates a greater discrepancy between the two distributions. The results from gene-level entries have p-values less than 0.005 in all three Poisson parameters, indicating poor fits. The p-values based on aggregated matrix entries are all larger than 0.2, indicating a good fit. Considering the KS test can be conservative, we further use the over-dispersion test and the zero-inflation test to measure the goodness-of-fit. The results from zero-inflation tests are all non-significant. The over-dispersion tests have non-significant p-values except for mentclass2pt{minimNext, we use these goodness-of-fit tools (for matrix entries with similar Poisson parameters) to study batch variation Fig. . Each pl GLM-PCA  with \doOne of the experiments deliberately violated the single cell assumption. Plate 8 (green) had two cells per well, i.e. per library. It shows over-dispersion at entclass1pt{minimaAnother experiment has an equal mixture of two different cell lines (Plates 5A and 6A). In Fig. mentclass2pt{minimHere, we introduce a novel data representation (DIPD) based on a departure from the IPD. The initial step is based on a crude two-way parameter approximation, where variation across cells is modeled by a cell-level parameter, and variation across genes is modeled by a gene-level parameter (as defined in equation ). This iIn the departure matrix, each entry is quantified by the relative location of that original count with respect to the tentative Poisson distribution, whose parameter comes from the initial two-way approximation. The departure measure is captured by a Poisson Cumulative Distribution Function (CDF), which leaves the unexpectedly small counts nearly 0 and unusually large counts close to 1. Next, the departure measure is put on a more statistically amenable scale using the logit function. As a result, unexpectedly large counts give large positive values and unexpectedly small counts give large negative values.Figure Hclust-Departure, operates as follows: Starting with the UMI count matrix (UMI), a very crude two-way parameter approximation (more details in Methods) is used to estimate Poisson parameters (D) measure from the naïve model. This DIPD-based matrix serves as the input for the clustering step. Clustering with D) are similar, and therefore Sigclust2 should not find significant clusters.A major application of this data representation is cell clustering using DIPD. This can be used directly as input into other algorithms. It also opens the possibility for a novel clustering algorithm, as illustrated in Fig. igclust2 . For a hHclust-Departure, we compared it with a commonly used package, Seurat (version 3.1.1) [To investigate the performance of n 3.1.1) .Hclust-Departure to the DIPD-based matrix resulted in no significant splits  is testentclass1pt{minimaentclass1pt{minimaentclass1pt{minimaHclust-Departure resulted in two clusters, consistent with the known cell lines. Seurat clustering also identified two clusters under the default setting (resolution parameter 0.8). Additional file Combining the data from two clonal cell lines (Plates 5A and 6A) in an equal mixture provided a positive control, as the two cancer cell lines were from independent patients, but of the same lineage . Figure Hclust-Departure, to data comprised of a mixture of three cell lines, at a ratio of 1:3:6 [Hclust-Departure identified three clusters. Using the default setting, Seurat identified 9 clusters  is used without dimensionality reduction or feature selection. Seurat (panels c and d) is applied using the top 2000 most variable features as defined by default. The cell type labels are manually assigned to each cluster using known lineage markers. The clusters discovered by Hclust-Departure are consistent with those identified by Seurat. In addition, Hclust-Departure identifies several significant subclusters within common Seurat labels (namely B-cells (light/dark green clusters), NK cells (light/dark gold clusters) and erythrocytes (light gray/black clusters)).The results are visualized using t-distributed Stochastic Neighbor Embedding (t-SNE)  and UMAPHclust-Departure, we identified differentially transcribed genes using the t-test  or the Wilcoxon rank-sum test (cluster size less than 30)  Fig. . The genHLA) class II molecules and Cystatin C (CST3), LYZ, TMSB4X; the other does not. Thus, the distribution of biologically defined lineage markers validated this unsupervised clustering result.Figure MZB1 and FKBP11 and  LTB (an early B cell differentiating factor) differentiate the two clusters confirming that two clusters, rather than one, was consistent with the known biology.Figure CD56 and CD16 [ZNF90, UBA52 and FAU suggests that those cells were in an active transcriptional state. The absence of TUBB indicates that these cells were in a mature state.Figure and CD16 . The preHBB, HBA2 and HBA1 which are expressed during terminal erythroid differentiation [YBX1, a transcriptional factor and SERBP1, an anti-apoptotic gene, further support the notion that the two clusters depict different stages of erythroid development.Figure ntiation . The expHclust-Departure identifies biologically plausible populations from this complex mixture of cells, establishing equivalent performance to existing scRNA-seq algorithms. It also identifies additional subtypes. Obviously, other algorithms can be tuned to fit previously known subpopulations. However, the choice of correct tuning parameters for those methods is necessarily heuristic, specific to each data set, and not necessarily reproducible or robust. By comparison, Hclust-Departure has no tunable parameters, other than the significance level (and neither has Sigclust2).In sum, Hclust-Departure and other pipelines is the actual clustering algorithm. We therefore combine the DIPD data representation with the Louvain algorithm as implemented in Seurat.A key difference between Hclust-Departure without dimension reduction performs slightly better  data sets used by Duò et al. . The Zheter Fig.  panel c.ter Fig.  panel d.D) or log-normalized data as input. These were (a) the number of principal components  and (b) the resolution parameter in the clustering step  [D as input outperforms Seurat using normalized counts as input; however, there are parameter constellations that lead to dramatic performance degradation independent of the data representation. In sum, DIPD-based data representation D combined with Louvain clustering outperforms other normalization steps for UMI data.To further define the performance of the hybrid approach, different parameters were explored using either DIPD-based representation (mix4uneq ). These ex (ARI)  and the ex (ARI)  Fig. 8)D) or logD and Louvain clustering as the optimal combination, a direct comparison between algorithms that use different data representations and have multiple tunable parameters is difficult using experimental data sets with possibly unknown subpopulations: overfitting cannot be decided on experimental data. An alternate approach is a simulation based on theoretical distributions alone. This also is challenging because many aspects of the deep biological variation in scRNA-seq data are unknown and beyond current in silico modeling capabilities. These limitations motivate the use of crafted experiments. Here, carefully chosen perturbations are overlaid onto real data. Crafted experiments maintain the complexity of the real data, but control the signal versus noise by considering a range of perturbations from weak to strong. We performed two different types of crafted experiments.Even though the experiments above point to DIPD-based data representation Variation in library size  is a driver of non-relevant variation in scRNA-seq. To explore this issue we artificially magnified the library size and compared different data representations  can be incorporated into existing scRNA-seq analysis pipelines and gives improved overall performance independent of the particular clustering algorithm.Working on the scale of probabilities rather than counts offers numerous advantages. First, due to the characteristics of scRNA-seq data (many zeros and low counts in most matrix entries), working in probability space is a more appropriate way to represent the underlying data structures. The DIPD-based data matrix provides a useful tool to uncover cell heterogeneity from observed counts into a model departure from the hypothesized Poisson parameter matrix, as input to any subsequent analyses. The large number of zeros in scRNA-seq data, which have been considered in row or column based analyses to be zero-inflation, is more precisely viewed as a large number of very small Poisson probabilities. Similarly, the previously reported over-dispersion is explained by variation in the set of individual Poisson parameters within the framework Fig. .Implementing Sigclust2 in clustering provides explicit hypothesis testing for each cluster, which avoids parameter tuning. A direct comparison of different data representations demonstrated that DIPD has an improved performance over conventional log-normalized data Figs. , 8. A hyA limitation of this pipeline is computational speed because it uses the full feature set  single cell line data, (ii) three cell line mixture data, (iii) normal human PBMC data, (iv) data from a mouse tissue infected with the human immunodeficiency virus (HIV). The data represented a variety of technical platforms.To study a scRNA-seq data set which is as homogeneous (and thus Poisson) as possible, single cell line experiments were considered. The first data set is on the experiments of Landis et al. . This daThe overall experimental design is nested, generating different levels of batch variation. Batch category one represents technical replicates called plates. Cells within a plate are from the same cell line, collected at the same time and hence are homogeneous in that sense. Batch category two represents data from experiments or biological replicates. The full data set contains 10 plates, . The data were pre-processed as described in Landis et al. . SpecifiPoissoneity under different scenarios, i.e. different degrees of batch variation. The data are summarized in Table This carefully constructed data enabled us to validate the This data set was generated from a mixture of three cell lines by 10X Genomics . There aThis scRNA-seq data was generated using 10X Genomics originally from Zheng et al. . Cells cThis data set was based on a study by Cheng et al. . This stWe first discuss the GLM-PCA algorithm, which is applied in parameter estimation for our assessment of the IPD framework. Then we give a brief review of the Seurat pipeline, for data pre-processing steps and cell clustering as an example for the state-of-the-art in RNA-seq data analysis.GLM-PCA is an algorithm for computing an analog of PCA in the context of generalized linear models (GLM) . A typics et al. .Seurat  is an Rquality control: Genes with less than three positive counts overall were excluded; cells, where the unique gene counts (the number of detected genes) were above 2500 or below 200, were excluded; cells with total mitochondrial gene counts greater than 5% of the overall total were excluded.normalization by cell: The gene expression for each cell  was calscaling: The expression of each gene was scaled to have a mean of 0 and variance of 1 across cells. A variation of standard scaling includes regularized negative binomial regression, which is called SCTransform [ransform .linear dimension reduction: The data was represented by the first 15 principal components obtained by Euclidian PCA.clustering: Cell clustering was done with a graph-based clustering approach using the Louvain algorithm and visualized using t-SNE or UMAP methods.In the following section, we describe the approach to the assessment of the validity of the IPD statistical framework. We propose DIPD as a novel data representation, which is a measurement of the relative location of UMI counts with respect to the independent Poisson distribution at the individual entry level. The cell heterogeneity can be better reflected at the scale of continuous possibilities than in the original scale with excess zeros. Therefore, we further develop a departure-based cell clustering algorithms to identify cell subpopulations.We work with scRNA-seq data with individual matrix entries through an IPD statistical framework, where each matrix entry  for mean et al. )Zero-inflation testzero-inflation, where the number of observed zeros was larger than the expected number of zeros. To compare the proportion of zeros among the matrix entries with that expected from the mixture distribution:A much different departure from the Poisson that can arise in certain applications was Again, from our IPD framework, each gene expression measurement for each cell (i.e. each matrix entry) comes from an independent Poisson distribution with parameter Fitting of a simple two-way approximation The model  values if the UMI count is much lower (or higher) than expected from the simple two-way approximation. The collection of cells with such novel data representation can be plugged into a standard clustering algorithm .The proposed clustering starts with the DIPD-based matrix computed for the complete data set. Hierarchical clustering using Euclidean distance and Ward’s linkage is recommended from a top-down viewpoint. At each step, we re-calculated the two-way approximation again within each subcluster, and the potential for further splitting is calculated using Sigclust2 , a methoWe do not need to set the number of clusters beforehand. Thinking of the number of clusters in a multi-scale way as in Liu et al. , a coarsFor each matrix entry UMI count equation . The valAdditional file 1. The summary of goodness-of-fit measures, including the Kolmogorov-Smirnovtest, the over-dispersion test, and the zero-inflation test.Additional file 2. Supplementary figures including heatmaps and UMAP visualizations.Additional file 3. The Q-Q envelope plot for small discrete counts.Additional file 4. The summary of the computational time required for implementing DIPD on the datasets used in this manuscript.