However, analysis methods that integrate both types of data are lacking. We describe SURF, Statistical Utility for RBP Functions, for integrative analysis of large collections of CLIP-seq and RNA-seq data. We demonstrate SURF’s ability to accurately detect differential alternative transcriptional regulation events and associate them to local protein-RNA interactions. We apply SURF to ENCODE RBP compendium and carry out downstream analysis with additional reference datasets. The results of this application are browsable at  TheyPA class   project [The ENCODE (ENCyclopedia of DNA elements) project , 10 rece project  and the  project , to asseA number of computational tools exist for detecting differential alternative splicing from RNA-seq data across two conditions  , 30 and lab/surf . The resvariable exons (or sub-exons) [event body.SURF is designed for large-scale analysis of RBP CLIP-seq and RNA-seq data and includes analyses and discovery modules Fig. . A key ib-exons)  defined differential regulation using RNA-seq. Specifically, DrSeq administers the relative exon usage (REU) to characterize any type of AS, ATI, and APA events by detecting differences between the experimental conditions, i.e., wild-type versus knockdown RNA-seq data  or the read count of all the other exonic regions of the same gene (l=0). Similar to the DEXSeq formulation [Kijl with a Negative Binomial (NB) distribution assj is the size factor of the jth sample and adjusts for the sequencing depth, and αi is the dispersion parameter for event i. The mean parameter μijl is modeled by a log-link function asWe extended the DEXSeq framework to detect differential ATR events of all the eight types by “DrSeq” for ata Fig. b. The RE/K0 Fig. b. For thmulation  for exonρ(j) is the experimental condition of sample j  and g(i) is the gene (group) from which the ith event originates. Here, l=1 due to identifiability constraints; hence, it quantifies the logarithm of REU, Kij1/Kij0, for the ith event across all samples regardless of experimental conditions. Finally, βEC) in Eq.  versus LAML (TCGA). The module first lifts over the SURF-inferred location features to transcript (or gene) level and forms the sets of transcript (or gene) targets of the RBP, while keeping them specific to ATR event types. For example, the transcript targets of SRSF1 through SE regulation (SRSF1-SE) are the set of transcripts with at least one differential ATR event of type SE regulated by SRSF1. We apply an area under the curve (AUC) approach similar to that in  to evaluOverall, SURF (i) leverages established appealing characteristics of DEXSeq  and geneWe evaluated the differential ATR event detection module (DrSeq) of SURF by comparing it to rMATS . The choThe overall results, summarized in Fig. Since a single ATR event typically spans multiple exonic bins, DrSeq combines data over exonic bins belonging to the same event and leads to better overall performance. In fact, the two “ad hoc” approaches we designed either have satisfactory FDR control at the expense of power loss or sustain a fairly large power while trading off FDR control. This analysis further supports that our extension of DEXSeq to ATR event level provides it with the competitive functionality as rMATS  in addition to its unique feature of encompassing differential analysis of ATI and APA event classes.We applied SURF to a collection of 104 paired RNA-seq and eCLIP-seq datasets from the v78 release of the ENCODE project. We extracted AS, ATI, and APA events from a filtered version of the current human genome annotation file . We observed that ATI and APA are the major sources of differential ATR  associated with differential SE significantly  to enable association analysis. In addition, we estimated the average numbers of false positives in the association analysis by shuffling the grouping labels of differential ATR events and rerunning the analysis multiple times. This resulted in an average of 3.88 (with an s.d. of 2.38) false positive associations at target FDR level of 0.05 . We further assessed the overall significance of each feature by aggregating the SURF-identified global positioning associations across all the 76 RBPs for each individual event type  exhibited marked enrichment compared to others with a p value of 4.88×10−4 according to Pearson’s χ2 test. The same observation held for A3SS events  and highlighted the notable contribution from regions beyond proximal alternative splice sites. Overall, SURF identified 31 (40.7%) RBPs that showed evidence for interaction with features distal to the event body, suggesting a broader genomic scope of functional protein-RNA interactions than previously revealed.While most global positioning tendencies of RBPs that associate with AS, ATI, and APA events   Fig. c. The evp value 3.00×10−4 from an exact Poisson test with a fold change of 1.50, Additional file p value 5.138×10−5 from an exact Poisson test with a fold change of 1.27). A similar result continued to hold with TCGA mutation data and is available from Additional file Recent studies highlighted central roles of RBPs in neurodegeneration, auto-immune defects, and cancer –64. We hWe next performed a comparative analysis of transcriptional activity of genes and transcripts harboring specific RBP-ATR event type combinations in normal versus tumor samples by leveraging GTEx and TCGA resources. We specifically considered TCGA LAML sample for which the K562 erythroleukemic cell line has the most relevance for. Harnessing of gene and transcript targets based on SURF-inferred location features capitalizes our construction of location features—each location feature is designated to a single ATR event that originates from a unique transcript. Among the 222 available RBP-ATR event type combinations, 132 (or 131) of them had more than 100 genes (or transcripts) with at least one SURF-inferred location feature  from GTEx whole blood samples to TCGA LAML samples. Isoform percentage of a transcript depicts the percentage of its abundance over its parent gene’s abundance, and its comparison across conditions elucidates how the contribution of a transcript to overall gene abundance is changing across conditions. Comparison of isoform percentages revealed a diverse pattern across ATR event types Fig. d, suggesgression , 66. In To test the differential activity for individual transcript sets, we applied the discovery module of SURF, with their paired controls by including the transcripts originating from the same corresponding target genes. Of all the RBP-ATR event type combinations, 18 of them (among the 131 with more than 100 transcripts) exhibited significantly different transcriptional activity between the whole blood (GTEx) samples and LAML (TCGA) samples, at FDR cut-off of 0.05 Fig. f. The deIn this study, we presented SURF, a new integrative framework for the analysis of large-scale CLIP-seq and coupled RNA-seq data and applied it to ENCODE consortium data Table . A key aAssociation of DrSeq-identified differential ATR results to global positioning preferences of RBPs with eCLIP-seq data revealed large diversity among RBPs and suggested dual inclusion and exclusion roles for RBPs based on the positioning of protein-RNA interactions. Furthermore, aggregation of the positioning preferences across RBPs within each event type revealed that locations near the upstream and sometimes downstream constitutive exons of the ATR events also play a significant role in ATR. Sequence analysis of SURF-inferred location features  revealed known and novel interaction modalities for RBPs in ATR. To further assess the overall relevance and importance of SURF-inferred location features, we compared the somatic mutation densities of these regions with carefully designed controls. This analysis revealed that location features inferred as associated with differential ATR events harbored significantly more somatic mutations both in ICGC and TCGA consortia data. Further integration with reference transcriptomes from GTEx and TCGA highlighted contribution of APA and AS regulation in diverse transcriptome activity in cancer and revealed RBPs with ATR event-specific differential transcriptional activities in LAML tumors compared to GTEx normals.SURF presents the first framework for systematic integrative analysis of large collections of CLIP-seq and RNA-seq data and includes a discovery module for probing public TCGA and GTEx transcriptome data. This comprehensive approach coupled with the ability to identify differential ATR events from settings with multiple biological replicates underlines SURF’s potential as an integrative tool. There are a number of directions that naturally build on the results of SURF. Since we have observed significant enrichment of somatic mutations from ICGC and TCGA in SURF-identified location features, evaluation of impact of individual somatic mutations on RBP binding with in silico calculations , 70 coulc% across all samples or (ii) TPM estimate less than 1 across all samples were removed. We varied the threshold c for filtering relative isoform abundance as 5%, 10%, and 25% and considered the following strategies: (1) IP ≥5%, (2) IP ≥10%, (3) IP ≥25%, (4) TPM ≥1, and (5) IP ≥5% and TPM ≥1. Additional file In order to improve FDR control, we pre-filtered transcripts in the human genome annotation file (GENCODE version 24) before extracting ATR events, as suggested by . We perfvariable bins of exonic regions for each transcript as the set of absent exonic regions of the transcript compared to the full gene model  events including exon skipping (SE), alternative 3 ′ (A3SS) or 5 ′ (A5SS) splicing, intron retention (RI), alternative first exon (AFE), alternative 5 ′UTR (A5U), and intronic (IAP) or tandem (TAP) alternative polyadenylation, according to its relative position within the gene  in transcriptome quantification for RNA-seq. For a given event, we first counted the number of reads residing on the event body (in million), where we utilized only the reads with mapping quality score greater than 10 . To account for sequencing depth differences, the counts were then normalized by the size factor of each sample [base mean, which is also reported by DrSeq. Next, we adjusted the base mean by dividing it with the pseudo length of each event (in kilobase) and used this adjusted mean as the coverage of ATR events. Here, the pseudo length of an ATR event is defined as its actual length (in kilobase) shifted by the read length minus one, that is, 99 bp . The adjustment with pseudo length  regularized those extremely short (in length) ATR events and reduced the effects of overlapping counting. This had negligible effect on long ATR events. Finally, we excluded any ATR events with read coverage smaller than 0.05, and among the overlapping ATR events, we only kept the event with the highest read coverage for the downstream analysis.In the analysis module 2 of SURF, we filtered out (i) low coverage events and (ii) overlapping ATR events of the same type to avoid near-duplication observations in the association analysis. Specifically, we employed a h sample  and averfeature signal.Each ATR event has a set of location features, each of which is a genomic region of potential protein-RNA interaction. This step of SURF takes as input the aligned reads from the IP and accompanying control input samples of eCLIP-seq and summarizes the eCLIP-seq signals at these features. To this end, for each feature, it first counts the number of overlapping reads (in million) in each sample by requiring an overlap of at least 12 bp between the read and the location feature. The 12 bp corresponds to the 25% of the read length in the ENCODE eCLIP-seq experiments. Then, the read counts are normalized by the sequencing depth of each sample. Next, the normalized read counts are weighted by the genomic length (in kilobase) of the feature, which is then averaged separately across the IP and control input samples. Finally, the difference between normalized IP and input samples is reported as i with event type τ(i)=t, let xik denote the feature signal at the kth location feature. The random variable Yi∼Bernoulli(pt) corresponds to whether an event is differentially regulated (Yi=1) or not (Yi=0). We link its mean parameter pt to the eCLIP-signals of the location features marginally with a logit function asWe used a logistic regression framework to derive global positioning preferences, e.g., position-specific rules for RBPs in regulating ATR. Specifically, for each event t and quantifies a baseline for the probability of observing a differential ATR event. This parameter also accounts for the position independent effects of RBP in regulating this event type. Position-specific main effect k.Here, parameter t, we also considered increased and decreased regulation separately: model for Yi=1 corresponds to increased regulation  and model for Yi=1 represents decreased regulation . Both models incorporated the same set of equal regulation  events for Yi=0 . The reason for separating increased and decreased regulation is primarily due to the possibility that interactions of a single RBP at a specific pre-mRNA location feature might result in both inclusion and exclusion of the alternative site, depending on the presence/absent of other collaborative/competitive RBPs. Differential ATR analysis of the ENCODE compendium data of 104 RBPs with module 1 of SURF resulted in, on average and after filtering out overlaps, 2182.02  events with increased REU and 1914.98  events with decreased REU. This enabled fitting of model  at multiple location features. Then, for every event type, these values were averaged across RBPs and interpreted as feature abundance  whether or not a location feature is significantly associated with the differential ATR and (ii) whether or not the location feature is x, where x∈{α,β,...,θ} is the location feature for that ATR event type. All the tests employed Yates’s correction for continuity.We quantified the abundance of a location feature by the normalized number of RBPs that significantly associated with it. Specifically, we aggregated the results of global positioning associations across all the 76 RBPs for each individual event type. For each RBP and each event type, we first applied minus logarithm (base 10) transformation on nce Fig. c. Next, p value (adjusted for multiplicity when appropriate). SURF then generates a list of inferred location features by filtering as follows. First, it selects statistically significant location features at FDR of 0.05. Note that each selected feature has a corresponding training dataset, which consists of differential ATR events and control events. SURF then extracts the list of genomic regions for each selected location feature from this corresponding set of differential ATR events. Finally, leveraging the quantified feature signals from the analysis module 2, it filters out the genomic regions with feature signals, i.e., normalized background adjusted eCLIP-seq read counts, below 20 and defines the set of remaining genomic regions as the inferred features. Imposing the eCLIP-seq signal condition ensures that the set reflects the consequences of direct protein-RNA interaction. Note that the inferred features are event-specific and are associated with specific regulation status. As a result, each inferred feature has its associated ATR event and inferred RBP regulation function .As a result of association testing in the analysis module 2 of SURF, each location feature is assigned a α, β,...). The collections of interest corresponded to (i) 52 individual RBPs, (ii) 8 types of ATR events, (iii) 44 event × location feature combinations , and (iv) 222 RBP-ATR event type combinations . Finally, we searched for the top 3 motifs with the minimum width of 5 bp and the maximum of 12 bp in each collection of sequences, where we filtered the resulting motifs by minimum occurrences of 40 times. The SURF-inferred location features and the output of 326 individual MEME runs (version 5.0.5) are deposited (see Availability of supporting data). For clustering of identified motifs, we used a metric based on Kullback-Leibler divergence of position weight matrices (PWMs) as adapted in [For SURF-inferred features, we performed de novo motif discovery using the MEME-suite , with a apted in  and Waldapted in .activity of a target set in a sample by the AUC. The AUC evaluates the ranking of its elements compared to other quantified units in the sample in terms of expression .The discovery module of SURF takes in sets of RBP targets  and perform differential analysis on these sets across conditions, in terms of their activity. This module adapts a rank-based approach, inspired by the single-cell RNA-seq application of , to scorAdditional file 1 Additional file 1 contains supplementary Figs. S1-S44 and Tables S1-S7.Additional file 2 Review history


Uterine corpus endometrial carcinoma (UCEC) is the second most common type of gynecological tumor. Several research studies have recently shown the potential of different ncRNAs as biomarkers for prognostics and diagnosis in different types of cancers, including UCEC. Thus, we hypothesized that long noncoding RNAs (lncRNAs) could serve as efficient factors to discriminate solid primary (TP) and normal adjacent (NT) tissues in UCEC with high accuracy. We performed an in silico differential expression analysis comparing TP and NT from a set of samples downloaded from the Cancer Genome Atlas (TCGA) database, targeting highly differentially expressed lncRNAs that could potentially serve as gene expression markers. All analyses were performed in R software. The receiver operator characteristics (ROC) analyses and both supervised and unsupervised machine learning indicated a set of 14 lncRNAs that may serve as biomarkers for UCEC. Functions and putative pathways were assessed through a coexpression network and target enrichment analysis. Uterine corpus endometrial carcinoma (UCEC) is the second most common type of gynecological tumor, in either developed or underdeveloped countries . Accordihttps://pubchem.ncbi.nlm.nih.gov/compound/2733526) therapy [Obesity, aging, early menarche, late menopause, nulliparity, breast cancer, and diabetes mellitus history are some of the endogenous risk factors for developing the disease . Other rSeveral researches have recently shown the potential of different noncoding RNAs (ncRNAs) as biomarkers for prognostics and diagnosis in different types of cancers , 8–10, iHOTAIR was found to be overexpressed [HOTAIR suppressed significantly the endometrial tumorigenesis, leading to smaller tumors [MALAT1, is known to be overexpressed during endometrial hyperplasia and also during early carcinoma stages; however, its expression is significantly lower in advanced stages, as well as during metastasis [In UCEC, the lncRNA xpressed  and appaxpressed . The silr tumors , 22. Anotastasis .Given their role in modulating UCEC progression, we hypothesized that lncRNAs could efficiently discriminate solid primary tumor (TP) and adjacent normal tissue (NT) with high level of confidence. To test our hypothesis, we performed an in silico differential expression analysis comparing TP and NT from a set of patients, targeting highly differentially expressed lncRNAs that could potentially serve as biomarkers for UCEC. These potential biomarkers were tested using both supervised and unsupervised machine learning. Also, their relationship with other genes was verified by gene coexpression networks and enrichment analysis. From our knowledge, previous works considered only the mRNAs , 25, focFrom the 46 selected samples (TP and NT from 23 patients) of the total RNA data, we obtained a total of 8,700 transcripts, after normalization and filtering, that went through the differential expression analysis. In order to assess the relationships with other regulatory RNAs and pathways, we also analyzed miRNA data, from which we selected 42 samples , and 1,881 miRNAs were attained.The total RNA normalization and filtering of the data guaranteed that the outliers had been removed, and the samples were well normalized , meaningLINC01376, was upregulated, indicating that its expression is essential for maintaining the tumor environment. In order to predict whether the candidates were good for classification models, we plotted AUC-ROC curves for each of them. Five candidates had an AUC greater than 0.7, thus demonstrating high confidence results: LINC01376, BRWD1-AS1, LINC00244, LINC02475, and ZNF667-AS1 , or may be a lincRNA, acting as a trans regulatory element. Also, lncRNAs can connect to DNA, regulating the transcription process, or bind to proteins affecting their stability . It is possible that BRWD1-AS1 could be essential for the normal regulation of the corresponding mRNA, thus its downregulation may lead to cell survival considering that BRWD1 has an important role in apoptosis. It is important noticing, however, that BRWD1 was not differentially expressed in our analysis. Another lncRNA-AS that stood out in our analysis was ZNF667-AS1, with a logFC of approximately −5.16 and an AUC of 0.717. Other studies have demonstrated that ZNF667-AS1 is commonly downregulated in several cancer types, including UCEC [ZNF667-AS1 inhibited the proliferation of cancer cells and its downregulation was negatively correlated with the overall survival of patients, tumor size, and FIGO stage [Members of the poptosis . BRWD1-Aing UCEC , 37. Vrbing UCEC  showed ting UCEC . In cervGO stage .PLCG1-AS1, LINC01411, and LINC02249) presented elevated expression variation and low AUC and could not be considered good models. Thus, 14 remained during the SVM analysis. The results for all tested sets are shown in To evaluate if our combined set of differentially expressed lncRNAs is able to correctly separate TP from NT samples, we performed the SVM approach. From the seventeen lncRNAs, three of them . A gene coexpression network was plotted from the result of the correlation analysis  and was TCF4-AS1, LINC02249, and LINC02475, which may indicate that they possibly play a role on the expression of CHD5. To our knowledge, all three lncRNAs have not been biologically associated with cancer to date, nor to any other disease. CHD5 was also positively correlated to hsa-mir-767, which is downregulated in UCEC. Downregulation of this miRNA has also been detected in lung adenocarcinoma cells [Many studies first identified CHD5 as a tumor suppressor gene in neuroblastomas –60. Subsma cells . No miRNGPR161 (upregulated) was associated with the negative regulation of the hedgehog signaling pathway. However, previous studies revealed that abnormal activation of this pathway is related to cell proliferation in endometrial cancer [GPR161 is associated with low survival probability in UCEC  but it is not prognostic for UCEC. Additionally, GPR161 has been found to be upregulated in breast cancer as well, acting as a promoter of cell proliferation and invasion [GPR161 and MIR545 are probably promoting cell proliferation on UCEC as well, but other elements may also be involved. Again, no miRNA prediction binding site tool   showed tl cancer , 71. Accl cancer , high exinvasion . The coeinvasion  and hepainvasion . We alsotool see  identifiHDAC7, GTF2A1L, and MAPKAPK2 were involved in the viral carcinogenesis pathway. Previous studies have demonstrated that inhibition of histone deacetylases, such as HDAC7, induces apoptosis, cell cycle arrest, and growth inhibition in endometrial cancer cells [LINC01376, the only lncRNA that is upregulated in UCEC from our set, positively and strongly correlates with HDAC7 (r = 0.91), which reinforces LINC01376 as a marker. Again, based on the Human Protein Atlas [HDAC7 is associated with low survival probability in UCEC (p=0.045), but it is not considered a prognostic for UCEC. Based on KEGG (https://www.genome.jp/kegg/), MAPKAPK2 also participates in cellular senescence.KEGG analysis also performed on pathfindR (v. 1.2.1)  indicateer cells –79. Morein Atlas , high exin silico prediction of lncRNA function. As stated before, characterization of most lncRNAs remains to be done; therefore, we were able to predict the functions of seven lncRNAs from our set: BRWD1-AS1, LINC00504, SHANK2-AS1, MACC1-AS1, WRD86-AS1, LINC00632, and ZNF667-AS1. The first four were strongly associated with male reproductive system pathways, such as spermatogenesis, male gamete generation, and sperm motility. The alteration of this pathway is not a novelty in endometrial cancer [LINC00632 and ZNF667-AS1 were potently related with neural functions such as neuron projection, synapse organization, and neurotransmitter secretion and transport, whereas WRD86-AS1 was linked to the spliceosome complex and RNA splicing, as well as to breast, thyroid, and colorectal cancers.In order to look for more putative pathways and biological functions of the lncRNAs from our set, we performed an l cancer , 81. LINEven though there have been lncRNAs reported as oncogenic drivers , survivaLINC01376 and BRWD1-AS1) with a minimal reduction in accuracy and specificity. This data must be validated in clinical samples, but it is predicted to contribute in the diagnosing UCEC. While there have been similar works on lncRNAs [In summary, our data suggest a set of 14 lncRNAs as highly effective biomarkers of UCEC, albeit none can be reckoned as drivers and coexpression associated genes are not prognostic. Moreover, this set can be reduced to the two top lncRNAs ( lncRNAs  that pavhttps://www.R-project.org/). Data download, preprocessing, and differential expression analysis were performed using TCGABiolinks package (v 2.7.1) [https://www.bioconductor.org/). Total RNA and miRNA data must be separately downloaded from TCGA. For the total RNA data, 587 samples were downloaded: 35 NTs and 551 TPs. We analyzed an initial set of 23 patients that had both NT and TP samples, thus a total of 46 samples. The results were then visualized on volcano plots, and the 50 most differentially expressed transcripts (top 50) were featured on a heatmap, plotted using the package gplots (v. 3.0.1). For the miRNA data, a total of 579 samples were downloaded, 33 NTs and 545 TPs. We analyzed a set of 21 patients  that had both TP and NT samples, thus a total of 42 samples.All data analyzed in this study are available at the Cancer Genome Atlas (TCGA), and all analyses were performed using the R software (v. 3.4.0) (v 2.7.1) , availabp values (au) of approximately 95% were considered as statistically significant groups. Hierarchical clusters were plotted using the pvclust package (v. 2.0–0) [For the putative lncRNA biomarkers, we first performed a supervised prediction model using the area under the curve (AUC) of the receiver operating characteristic (ROC) with the pROC package (v. 1.13.0) , then a . 2.0–0) . We also. 2.0–0) , an onliGDCquery with the following options: project = “TCGA-UCEC,” data.category = “Transcriptome Profiling,” data.type = “Gene Expression Quantification,” workflow.type = “HTSeq—Counts” and legacy = FALSE. The query for the microRNA data uses the following options: project = “TCGA-UCEC,” data.category = “Transcriptome Profiling,” data.type = “miRNA Expression Quantification” and legacy = FALSE. Then, we downloaded both data using the function GDCdownload with the previously set queries and the option method = “api.” The function GDCprepare was then used to transform the downloaded total RNA data into summarized experiment data, making it suitable for the analyses.The UCEC harmonized total RNA expression data (hg38) and miRNA expression data downloaded from TCGA were obtained through RNA-seq using Illumina HiSeq platform. Total RNA and miRNA are separately download due to differences in the data composition. First, we set the query for the download of the total RNA data, using the function TCGAanalyze_Normalization with the options geneInfo = geneInfoHT and method = “gcContent,” and then by gene length, changing the method option to “geneLength.” Then, a quantile filter was applied using the function TCGAanalyze_Filtering with the option method = “quantile” and qnt.cut = 0.25. Finally, the Spearman correlation among samples was checked using the R function cor and both NT and TP sample groups were separated with the function TCGAquery_SampleTypes.The data normalization was performed first by GC content, using the function TCGAanalyze_DEA with the options fdr.cut = 0.01, logFC.cut = 5, and method = “glmLRT.” This high cutoff was chosen in sense to detect the genes with the highest potential for group differentiation.We considered as differentially expressed those genes with a false discovery rate (FDR) < 0.01 and log fold change (logFC) ≥ |±5|, using the function TCGA_SurvivalKM and the following options: Survresult = TRUE, p.cut = 0.5, ThreshTop = 0.67, and ThreshDown = 0.33. Survival curves for each lncRNA were generated and analyzed separately.Considering the expression on TP samples of all differentially expressed lncRNAs, we performed a Kaplan–Meier survival analysis with the function For the ROC analysis, we focused in the differentially expressed lncRNAs. The individualized results were visualized on a AUC-ROC curve plot, which is a graph showing the performance of a classification model at all classification thresholds. The evaluation metric used was the AUC, which represents the degree or measure of separability, that is, how much the model is capable of distinguishing between two groups .set.seed (3033). Then, we used the function trainControl that controls the computational nuances of the train method. At this point, we applied a 1000 bootstrap replication, with the option method = “boot” and number = 1000.For the SVM, we first set a randomly chosen seed of 3033, making our work replicable, with the function train with the previously set trainControl as a parameter, along with the options preProcess = c , tuneLength = 10, and method = “svmLinear” or method = “svmRadial”. Next, we tested our classifier at specific cost (C) values for linear method, using the function expand.grid with cost values ranging from 0 to 5 and adding the tuneGrid parameter with the set grid to the train function.Next, we set our train and prediction data. The train data contained the selected differentially expressed lncRNAs from the 23 initial patients. Here, three lncRNAs were cut out of the analysis for presenting elevated expression variation. Our prediction model data contained the same lncRNAs, however, from 24 samples, different from the 46 samples we have been working on: 12 TPs and the remaining 12 NTs from TCGA database that did not have TP/NT correspondence in the same patient. The expression data from the 24 new patients were analyzed apart from the set of 23 patients. To train our data, we tested both land radial kernel methods, using the function This process was repeated for three different sets of lncRNAs. The first set contained all selected lncRNAs, and the best performance was obtained using linear kernel method, with cost = 0.25. The second set had the ones that attained an AUC greater than 0.7. The best performance was obtained using the linear kernel method, with cost = 1. The third set had the two lncRNAs that attained the best AUC. The best performance was obtained using a radial basis function kernel method, with standard values of sigma = 0.07 and cost = 5.r > 0.7 and r < −0.7 and p value <0.05. The network was visualized using Cytoscape (v. 3.5.1) [A Spearman correlation analysis among the expression data of all differentially expressed transcripts and miRNAs was performed in order to plot a gene coexpression network. The cutoff for the statistically significant correlations was . 3.5.1) . For bet. 3.5.1)  and path. 3.5.1) .The miRNAs and their putative targets were used to predict their interaction using the online software TargetScan (release 7.1)  and miRD


There are situations though, in which we have multiple outcomes or multiple treatments, in which the multivariate meta-analysis framework which performs a joint modeling of the different quantities of interest may offer important advantages, such as increasing statistical power and allowing performing global tests. In this work we adapted the multivariate meta-analysis method and applied it in gene expression data. With this method we can test for pleiotropic effects, that is, for genes that influence both outcomes or discover genes that have a change in expression not detectable in the univariate method. We tested this method on data regarding inflammatory bowel disease (IBD), with its two main forms, Crohn’s disease (CD) and Ulcerative colitis (UC), sharing many clinical manifestations, but differing in the location and extent of inflammation and in complications. The Stata code is given in the Appendix and it is available at:   Concerning meta-analysis of microarray and RNAseq studies there are three broad categories of statistical methods that make use of effect sizes, p-values or ranks. For several reasons, the effect size approach which is based on common practices in meta-analysis, is preferable (in particular for the problem at hand) and thus it was advocated early in the literature In gene expression experiments, the mRNA levels of thousands of genes are measured simultaneously in tissue samples. After the initial pre-processing steps that may differ among methods , the identification of differentially expressed genes  are administered or different exposures (risk factors) are recorded. In another potential example several (related) diseases are investigated against a common healthy group (multiple outcomes). Such examples include diseases of similar etiology like autoimmune diseases , or diseases with similar manifestations, like Crohn's disease CD and UlceWe used StataX), measured in study i = 1,2,…k. The effect sizes for the comparison of the two different disorders, are the standardized mean differences, which is referred to as Cohen's dXi1, Xi2 and Xi0 are the means of the expression of the particular gene in CD, UC and control groups respectively. pooled,iS is the pooled standard deviation, in study i, given by:n1i, n2i and n0i being the sample size of each group and S1i, S2i and S0i the gene expression standard deviation for each group.The model we use is based on the standard model of multivariate meta-analysis d1i and d2i will be estimated by:d1i and d2i will bein = n0i + n1i + n2i being the total sample size of the study The variance of the effect size estimates p-values and confidence intervals. These methods include resampling methods such as bootstrap and permutation, Bayesian methods and penalized or regularized t-tests. The bootstrap and permutation are easily applied but would require enormous computational time. Regularized t-tests on the other hand, require additional modifications for the multivariate setting. Here, we resort to an old but reliable alternative that has been shown to perform well, at least for moderate sample sizes (i.e. ~ 5–10 per condition). The idea is based on the fact that the sample estimates of the standardized mean difference d has the tendency to overestimate the absolute value in small samples. This bias introduced by d can be corrected using the so-called Hedges’ g, which generates an unbiased estimate. A correction factor, called J, is employed to convert from d to Hedges’ g.J is given by υι = n0i+ n1i+ n2i − 3. When g is used, the variances will also change to:gi =  is distributed following a multivariate normal distribution around the true means, according to the marginal model:Σi we denote the within-studies covariance matrix:Several statistical methods have been proposed to handle the issues of small sample sizes efficiently and calculate accurate Σi are the study-specific estimates of the variance  = ρwsi1si2. It is important to note that the elements of Σi are considered known quantities. In contrast, by C we denote the between-studies covariance matrix, which is to be estimated during the fitting process:τ = τ1 = τ2, C reduces to:The diagonal elements of variance , whereasvariance , for insThis model can be fitted in any statistical package capable of fitting random-effects weighted regression models with an arbitrary covariance matrix, such as SASg1 and g2 are equal to zero . The test statistic, W, follows a chi-square (χ2) distribution on 2 degrees of freedom and it can be obtained using the delta method:As we already mentioned, a major advantage of this model is that it can accommodate, under the MAR assumption, studies reporting only one of the parameters of interest, resulting in borrowing strength from external studies H0) would be: D = g1 − g2 = 0 and it can be tested using the Wald test :D, would be obtained easily by the delta method:Another useful test, would be that of the equality of the estimates. In this case the null hypothesis  and various correction methods have been proposed for this task. These methods are grouped into two categories, the ones that control the family-wise error rate FWER and the W) we identified 260 statistically significant DEGs that rejected the null hypothesis that both g1, and g2 are equal to zero. From the 260 statistically significantly DEGs, 53 were found to be statistically significant only by the global test and not in any of the diseases. These genes could not have been identified using univariate methods. The global test (D) was used as well, but only two genes (TRIP6 and IGHG1) were found to have equal change in the expression levels. Afterwards, we used these DEGs for further investigation with the help of bioinformatics overrepresentation analysis . In this case, even though the correction method we used is reported to perform well, other more advanced options may be preferable (for instance to use a bootstrap correction).The method was applied to microarray data; however, it can easily be applied also to RNAseq data. As we already mentioned, a major difference of Microarrays and RNAseq lies in the pre-processing steps, but for the purposes of a meta-analysis the analyst will not have to deal with that; one will assume that the data are processed and normalized. In general, data from microarrays and RNAseq are analyzed using the same meta-analysis methods ,23, and www.compgen.org/tools/multivariate-microarrays. Contrary to common microarrays data formats, the code requires that samples are listed one per line, whereas the expression for the various genes (or probes) are given in columns. Moreover, numerical variables indicating the type of the sample  and the actual study, are needed. It is the user's responsibility to ensure that the variables are coded appropriately. We need to mention that the method does not need all outcomes to be measured in every study. As we noted earlier, under the MAR assumption the method can incorporate studies that consider only g1, onlyg2, or both and in this case the increase in power is larger. However, for the proper estimation of between studies covariance, it is advised that a reasonable number of studies should report both outcomes. The code also works if a gene is present only in one study, but in this case the results are based only on the data provided by the particular study. Nevertheless, we also coded To conclude, we have presented an application of the multivariate meta-analysis to gene expression data. The Stata code is given in the Appendix and it is available at This work is part of Konstantina's Vennou thesis, which is co-financed by Greece and the European Union  through the Operational Programme «Human Resources Development, Education and Lifelong Learning» in the context of the project “Strengthening Human Resources Research Potential via Doctorate Research” (MIS-5000432), implemented by the State Scholarships Foundation (IKY).


Cell-type specific gene expression profiles are needed for many computational methods operating on bulk RNA-Seq samples, such as deconvolution of cell-type fractions and digital cytometry. However, the gene expression profile of a cell type can vary substantially due to both technical factors and biological differences in cell state and surroundings, reducing the efficacy of such methods. Here, we investigated which factors contribute most to this variation. We evaluated different normalization methods, quantified the variance explained by different factors, evaluated the effect on deconvolution of cell type fractions, and examined the differences between UMI-based single-cell RNA-Seq and bulk RNA-Seq. We investigated a collection of publicly available bulk and single-cell RNA-Seq datasets containing B and T cells, and found that the technical variation across laboratories is substantial, even for genes specifically selected for deconvolution, and this variation has a confounding effect on deconvolution. Tissue of origin is also a substantial factor, highlighting the challenge of using cell type profiles derived from blood with mixtures from other tissues. We also show that much of the differences between UMI-based single-cell and bulk RNA-Seq methods can be explained by the number of read duplicates per mRNA molecule in the single-cell sample. Our work shows the importance of either matching or correcting for technical factors when creating cell-type specific gene expression profiles that are to be used together with bulk samples. RNA Sequencing is a well-established method for comparing the transcriptome between different cell types, conditions and cell states . Cell tySingle-cell RNA-Seq facilitates the study of distinct cell types. However, the number of patients involved in such experiments is usually small compared to datasets containing bulk data from biopsies, such as The Cancer Genome Atlas (TCGA). It is therefore desirable to be able to conduct studies on bulk data with mixed cell types, with the help of mathematical tools that can help extract similar information as is available in single-cell data. One example of such a tool is cell type deconvolution, which estimates the fractions of different cell types in a mixed sample from the RNA-Seq data. This is implemented in for example CIBERSORTx , EPIC 66, CPM 77, MuSiC Representative RNA-Seq gene expression profiles for cell types can be created from either FACS/MACS-sorted bulk samples or single cell datasets where the average expression of cell populations can be used. However, the RNA-seq profiles for a cell type can vary substantially, both due to biological differences between samples and technical biases. It is therefore challenging to construct universal gene expression profiles for cell types that perform well in all conditions. The variation in RNA-Seq originating from sequencing at different laboratories has previously been examined , 13. In There are a number of methods for measuring the impact of different factors, such as cell type, tissue of origin, or dataset on gene expression. ANOVA is commonly used to estimate such differences, for example. Another example is variancePartition , a methoNormalization is a crucial step when analyzing RNA-Seq data. In the beginning of the RNA-Seq era, library size normalization, for example FPKM  and TPM in silico by mixing the cell type expression profiles at different fractions. The batch correction applied on the synthetic mixed samples is then projected back to the cell types. A drawback of this approach is that it introduces a bias depending on the cell fractions selected in the synthetic mixture. Acquiring representative cell type profiles for a given dataset of mixed samples remains a challenge. In this study, we sought to quantify the relative importance of the most important factors that cause undesired variation between gene expression profiles of individual cell types for typical cases encountered in deconvolution and other methods that use cell type profiles. We assembled a collection of public datasets and evaluated normalization and batch correction methods, quantified different sources of variation between samples for different sets of genes, and investigated technical biases between single-cell and bulk RNA-Seq. In addition, we evaluated the impact of using different cell type profiles in deconvolution.A common technique used to overcome technical biases is to computationally remove batch effects. The batch effect removal tool ComBat  implemenTo limit the complexity of the study, only B and T cell profiles were used. We gathered 74 publicly available RNA-Seq bulk samples of B and T cells from 5 different sources, and pooled 31 cell populations from single-cell data from 7 public datasets, for a total of 105 samples . The datFirst, we investigated data normalization approaches, which is challenging since our dataset contains both bulk data and pooled single-cell samples from UMI-based methods. This prevents the typical use of normalization methods that operate on counts, since the counts are not directly comparable between these sequencing technologies. In short, the bulk samples need to be corrected for transcript length, whereas the drop-seq based samples should not. We therefore decided to test three well-established methods that can operate on non-count data: library size normalization (TPM/CPM) , TrimmedFig 1 Comparing the relative log expression across all genes and samples for three normalization methods highlights the inadequacy of library size normalization for these samples (We used PCA to investigate the batch effects between different labs and between single-cell and bulk data. To quantify sources of variation in gene expression across samples we investigated to what extent different factors could explain the variance in gene expression. Factors examined were laboratory (which here includes different laboratory procedures and sequencing methods), UMI-based single-cell vs bulk, cell type, cell subtype (distinguishing between different types of B and T cells), tissue of origin (distinguishing between samples containing the same cell types but being collected from different tissues), individual, and whether samples are technical replicates. We normalized the 105 samples using TMM as described above and used the R package variancePartition  to estim2 fold change > 1, henceforth named LM22S). When including all genes, lab is the dominating factor, highlighting the importance of working with a reduced gene set for methods such as deconvolution. For housekeeping genes, defined as genes having a stable expression across tissues and cell types [ll types , lab expTo investigate the influence of gene expression on explained variance, we estimated explained variance as a function of gene expression for all genes . As expeTo evaluate the effect of batch correction using ComBat, we performed the same analysis as in We also investigated the variance reduction from using samples from the same individual (taken at different time points) and technical replicates (the same sample sequenced multiple times) . Our resTo investigate the factors explaining the variance between profiles from pooled single-cell data, we used variancePartition on the single-cell samples to analyze the factors lab, cell type, and tissue. The SC melanoma dataset was excluded from the analysis because it was the only dataset using the Smart-Seq2 technology, and consequently had a high impact on the variance explained by lab . We evalTo unravel the sources of variation between pooled single-cell data and bulk, we investigated the EVAL dataset which contains bulk and single-cell (10x) data generated from the same samples . We usedWe defined difference in gene expression per gene as the log2 fold change between the 10x pool and bulk data  and investigated to what extent that difference could be explained by different technical covariates across genes. First, we calculated the number of discarded UMI duplicates per UMI for each gene. We defined the UMI copy fraction asSecond, we investigated if transcript length introduces a bias due to differences in the sequencing protocol between 10x and bulk. Bulk reads can come from the entire transcript whereas the 10x reads originate from the sequences close to the polyA tail and is thereby less affected by transcript length. Third, we investigated the effect of the GC content of genes since this is a known source of bias in RNA-Seq . We inveTo evaluate how much of the differences between 10x data and bulk can be explained by the technical covariates, we measured the improvement in correlation between 10x and bulk data after regressing out the covariates. To exclude the possibility that the differences originate from stochasticity, i.e. lack of reproducibility of data, we first checked the log space correlation between cortex 1 and 2. Although the samples originate from slightly different parts of the brain, the different samples had a Pearson correlation of 0.989 for bulk and 0.981 for 10x, showing low stochasticity. We also investigated the role of sampling effects when regressing out the UMICF variable  Figs, coTo assess the impact of technical sources of variation on deconvolution of cell type fractions, we conducted a qualitative comparison of deconvolution performance across profiles generated from different datasets. We generated in total 40 synthetic mixed bulk samples containing 50% from a B cell sample and 50% from a T cell sample, both produced at the same lab. We further generated in total 12 pairs of cell type profiles (B cells and T cells from the same lab), originating from both single-cell and bulk data. Cell-type specific gene expression profiles are useful for analyzing bulk RNA-Seq samples containing mixed cell types, since it enables the use of advanced computational methods such as deconvolution of cell type fractions and digital cytometry. In this study, we investigated the impact of different sources of variation on cell-type specific gene expression profiles. We evaluated normalization methods and the effect of batch correction and used variancePartition to quantify the contribution of variance originating from differences in lab, cell-type, cell subtype and tissue of origin for different gene sets. Furthermore, we investigated the biases between UMI-based single-cell and bulk RNA-Seq. Finally, we examined the effect of cell type profile features on the performance of deconvoluting cell type fractions.We found that the variance explained by different factors is heavily dependent on the gene set, where lab explains the most variance when considering all genes. For deconvolution of cell type fractions and similar methods, the genes that separate cell types are of greater interest. Although cell type/cell subtype is the factor that explains most variance here, other factors such as lab and tissue of origin also have a substantial effect on gene expression for these genes. Consistent with our findings, large variation across experiments together with small differences between some cell subtypes has previously been shown . Our decNormalization and batch correction are important steps in the analysis of RNA-Seq data. In this study, we have shown that while library size normalization is inadequate, TMM  applied Our strategy for TMM normalization is very similar to that of GeTMM , where cWe also sought to examine in detail the technical biases present for cell-type specific gene expression profiles derived from UMI-based single-cell RNA-Seq when compared to bulk RNA-Seq data. We found that the number of duplicate reads per mRNA molecule in the single-cell data (UMICF) can explain a substantial fraction of the differences between single-cell and bulk, whereas transcript length biases are generally small in comparison to other effects. GC content has previously been reported to introduce bias , and it This study is limited in that we only investigated two cell types. We can thus not claim that the technical variation is of the same magnitude as differences across cell types in general, but only between B and T cells. Furthermore, the study was not fully balanced; some sources of variation are represented more strongly than others, which can have a slight effect on the results. For example, most samples originate from blood, increasing the influence of that specific environment when estimating the importance of the tissue of origin variation factor. In addition to these limitations, we only investigated one deconvolution software since the focus of this study was cell type profiles and not deconvolution algorithms; it could be that other methods are less sensitive to batch effects between profiles and mixtures.Our work suggests that estimating the number of duplicate reads per mRNA molecule can help in predicting and correcting for technical bias and thereby yield more comparable samples, both across bulk samples, single-cell samples and between bulk and single-cell. These results need to be further validated in more datasets, and the factors introducing this bias need to be investigated in more detail. Although such factors may differ across experiments, it is possible that a library of factor patterns aggregated from many single-cell experiments could be used for a more generalized prediction and correction of bias in bulk data. Such a method would be useful for a broad range of applications extending beyond the generation of gene expression profiles for deconvolution or digital cytometry.In this study, we investigated the sources of variation in cell-type specific gene expression profiles. We demonstrated that technical effects resulting from different laboratory procedures and data types explains much of the variance across samples and confounds analyses such as deconvolution, but also that biological traits such as cell subtype and tissue of origin are important to consider when generating cell-type specific gene expression profiles. These results provide valuable insight to users of computational methods such as deconvolution of cell type profiles and digital cytometry, highlighting the importance of matching both technical protocols and biological traits between cell type profiles and bulk data samples.We downloaded the publicly available RNA-Seq datasets listed inWe downloaded fastq files for BULK 1–4 to reduce the technical variability across datasets induced by the computational pipeline, and processed them using Kallisto   even though they had not, which was motivated by their use of similar techniques and too low sample numbers to be treated as separate labs.For single-cell datasets, cell type classifications were retrieved from the authors of the study in cases where it was not publicly available. For Smart-Seq2 data (the MEL dataset), we used the TPM values produced by the authors, and pooled the cells within a cell population by calculating the average expression per gene. For 10x data, we pooled the cells by first summing the counts from all cells for each gene, and then scaled the expression to a total sum of 10All samples are described in more detail in TPMi represents the normalized expression for gene i and Ex is the expression of gene x before normalization.TPM normalization was performed according toPCi), where the sum of all gene expression values equals the original library size, according tox is the original counts for gene x. The pseudo-counts differ from the original counts in that they are corrected for transcript length, but with identical library size. The purpose of this scaling was to make the number of counts in the TMM normalization as similar as possible to the original counts, since this number is used in the normalization calculations. The same procedure should ideally be applied to Smart-Seq2 data, but since we didn’t have access to raw counts, TMM normalization was applied directly to TPM values for these two samples.TMM normalization was performed using the calcNormFactors  functionFor quantile normalization we used the function normalize.quantiles in the preprocessCore package  . As batch, we used dataset id with one modification; the datasets PBMC68k, B10k and CD4TMEM were treated as the same dataset since they had to few samples to be batch corrected separately. We deemed that this was reasonable, since the data is produced in a similar way and published in the same publication.i = logL2(i+bE), where Li is the log transformed expression of gene i, Ei is the expression of gene i in pseudo-TPM, and b is a constant set to 1, which is added to avoid taking the logarithm of zero values.We applied log transformation for many analyses to make the expression data more normally distributed. The log transform was applied according to 2 fold change” (LFC) throughout the results to compare the expression of a gene between two samples. This was calculated asEi,1 and Ei,2 represent the expression of gene i in the two samples which are to be compared.We use the term “logWe used the functions “fitExtractVarPartModel” and “plotVarPart” from the R package variancePartition  to calcuFor generation of cell type profiles used for deconvolution with CIBERSORTx, the bulk profiles were generated by inputting data sample-wise, TPM normalized, in a data matrix to CIBERSORTx. The reason for using TPM normalization was that this is recommended by the CIBERSORTx team and it was of interest to see the difference in performance between TPM and quantile normalization . The pooled single-cell profiles were handled in a similar way, where each pooled sample from the design matrix was treated as a single bulk sample. Profiles were created using only cells from datasets within the same lab. The final calculation of profiles was done by CIBERSORTx, using the “Create Signature Matrix” function (specifying RNA-Seq). For single-cell data, the cells of each type were sorted into 100 bins for disk quota reasons in CIBERSORTx. Each bin was given the gene expression of the mean of all the cells in the bin, TPM normalized. The final profiles were then generated using the CIBERSORTx “Create Signature Matrix” function (specifying sc RNA-Seq).For generating mixtures used for deconvolution, the expression values of one T-cell sample multiplied by 0.5 was added to a similarly treated B-cell sample, both TMM normalized. The mixture samples were then TPM-normalized before being processed by CIBERSORTx. The reason for this procedure is that TPM-normalization before mixing the data could confound the mixture, creating a mixture where one cell type is given more weight, since library size normalization is sometimes inaccurate.For the LM22 results, the fraction of B cells is the sum of the fractions for “B cells naive”, “B cells memory”, and “Plasma cells”. For the single-cell profile sets labeled “pooled” (7 and 8) cells were pooled and input to CIBERSORTx as bulk profiles. For the remaining single-cell profile sets (9–13) cells were pooled into 100 B cells and 100 T cells for each profile pair due to disk quota reasons and input as single-cell profiles. For evaluation of profiles generated from bulk data, the mixtures generated from the same lab as the profiles were removed to avoid using mixtures and profiles generated from the same data.CIBERSORTx deconvolution was run using the function “Impute Cell Fractions”, specifying batch correction (B or S) and quantile normalization to produce such results. For quantile normalization, separate cell type profiles were produced, specifying that the cell type profiles should be quantile normalized.https://cibersortx.stanford.edu/), accessed 2020-06-05.CIBERSORTx was used from the CIBERSORTx web page L, where Lcorr,i is the corrected gene expression for gene i, Lorig,i is the original gene expression, pi is the predicted value of gene expression from the fit and mean(p) is the mean of all predicted values from the fit. The UMICF covariate was set to NA unless more than 5 unique UMIs were available for the gene, to avoid the noise induced by too few measurement points.To regress out one or more covariates, a linear or loess  curve was first fitted to the logFor all analyses, we removed a few outliers with extreme values for transcript length and GC content, in total 232 genes. UMICF was set to NA for all genes for which we had five or fewer UMIs , since we deemed the amplification measure to be too noisy otherwise. All excluded genes were left untouched by the regression. In total (including genes for which UMICF was set to NA) 26,563 genes were used in the calculations for both Cortex 1 and Cortex 2. The genes where UMICF was set to NA were still used in the correlation calculation.For estimating the confidence intervals in We used the function R function wilcox.test for the Wilcoxon signed rank tests for investigating if correlation differences between covariates were significant, with the parameters alternative = "greater", paired = TRUE. The same bootstraps were used for all covariates, making it possible to compare them with a paired test.https://github.com/SysBioChalmers/SingleCellToolbox). The processed data and source code are available at: http://doi.org/10.5281/zenodo.4011593. To ensure the quality of our analyses, we verified and validated the code using a combination of test cases, reasoning around expected outcome of a function and code review. The details of this activity are available in the verification matrix available with the code.The data was analyzed using R version 3.6.1 and MATLAB R2018b. MATLAB was used for assembling the single-cell data and exporting the pooled samples to a text file; the rest of the analysis was done in R. The MATLAB code uses the component SingleCellToolbox for importing public single-cell datasets Click here for additional data file.S2 FigThe plot shows how the explained variance by the different factors change with gene expression .(PDF)Click here for additional data file.S3 FigA. All genes (12072 genes). B. Housekeeping genes (3393 genes). C. LM22 (395 genes). D. LM22S genes. E. Identical to D, with the difference that cell subtype is replaced with cell type (B/T). F. Explained variance per gene expression. The plot shows how the explained variance by the different factors change with gene expression .(PDF)Click here for additional data file.S4 FigA, B: Comparison between TMM and Quantile Normalization. C-D: The effect of including Smart-Seq2 samples. E. Variance explained by having samples from different individuals compared to samples from the same individual but taken at different time points. F. Variance explained by having different samples compared to technical replicates, where the same sample has been sequenced several times. G, H. Same data as E, but separated on cell type into two groups to make individual factor more comparable to the technical replicates shown in F.(PDF)Click here for additional data file.S5 FigThe figure presents data from the EVAL dataset, Cortex 1, 10x single-cell data, normalized using TMM. Only genes with 5 molecules or more is shown.(PDF)Click here for additional data file.S6 FigA. Gene expression for cortex 1 from the EVAL dataset plotted as 10x vs bulk. The red line represents a perfect correlation. B. Gene expression for cortex 1 from the EVAL dataset after regressing out the differences in UMICF and GC content between 10x and bulk using a loess fit, which improves the correlation. C. Average Pearson correlation coefficient between 10x data and bulk in log scale after regressing out technical covariates , using linear or loess regression. The correlation shown is the average of the correlations from cortex 1 and 2 of the EVAL dataset, using quantile normalization.(PDF)Click here for additional data file.S1 Table(XLSX)Click here for additional data file.S2 Table(PDF)Click here for additional data file.S1 Note(PDF)Click here for additional data file.


Single-cell RNA-seq makes possible the investigation of variability in gene expression among cells, and dependence of variation on cell type. Statistical inference methods for such analyses must be scalable, and ideally interpretable.We present an approach based on a modification of a recently published highly scalable variational autoencoder framework that provides interpretability without sacrificing much accuracy. We demonstrate that our approach enables identification of gene programs in massive datasets. Our strategy, namely the learning of factor models with the auto-encoding variational Bayes framework, is not domain specific and may be useful for other applications.https://github.com/YosefLab/scVI/.The factor model is available in the scVI package hosted at v@nxn.seBioinformatics online. The study of the regulatory architecture of cells has revealed numerous examples of co-regulation of transcription of large numbers of genes  is a popular linear method for dimensionality reduction in single-cell RNA-seq . In the While PCA is easy to use and is often applied to single-cell RNA-seq data, the method has some drawbacks. PCA models data as arising from a continuous multivariate Gaussian distribution, and thus optimizes a Gaussian likelihood . This moAutoencoders consist of a pair of functions: a representation function and a reconstruction function, which are typically parameterized as neural networks . The twoInference using VAEs scales to arbitrarily large data since mini-batches of data can be used to train the parameters for both the inference model and the decoder function . HoweverThe generative model of scVI, when data are from a single batch and zero-inflation is deactivated, isIn this model, This way the expression level We also considered a variation of this model where the latent variables rization . Such a rization .To explore the potential for interpretability in the VAE framework, we implemented a linearly decoded variational autoencoder (LDVAE) in scVI. The model was applied to two datasets of single-cell RNA-sequencing from a large number of developing mouse embryos in different stages of development . Using a consumer-grade GPU (NVIDIA GeForce RTX 2070), inference only requires 2 s per 1000 cells to reach 10 epochs, with a total time of less than an hour for the full dataset. The inference times did not depend on the sparsity of the data, as the neural architecture is fixed, and operations are invariant to observed values . Investihttps://github.com/pachterlab/SGYP_2019 as well as Figshare at https://doi.org/10.6084/m9.figshare.11725920.v1. For convenience, the embryo data from https://github.com/YosefLab/scVI/blob/master/tests/notebooks/linear_decoder.ipynb.Jupyter notebooks to produce the results are available at Our results show that interpretable non-Gaussian factor models can be linked to variational autoencoders to enable interpretable, efficient and multivariate analysis of large datasets. This is useful for the investigation of gene co-expression in large scRNA-seq datasets, and the approach we have outlined should be applicable in other settings where interpretability is paramount.btaa169_Supplementary_DataClick here for additional data file.


LINC01697, LINC02487, LOC105376575, AC005083.1, SLC8A1-AS1, and U62317.1) from a list of 29 differentially expressed lncRNAs using the least absolute shrinkage and selection operator (LASSO) method in the discovery dataset of 167 OSCC samples and 45 normal oral tissues. Using the logistic regression method, we constructed a six lncRNAs-based diagnostic risk model (6lncRNAScore) which was able to differentiate between OSCC samples and control samples with high performance with AUC of 0.995 and high diagnostic specificity of 88.9% and sensitivity of 98.2% in the discovery dataset. The diagnostic performance of the 6lncRNAScore was further validated in another two independent OSCC dataset with AUC of 0.968 and 1.0. Functional enrichment analysis for lncRNA biomarkers-related mRNAs suggested that lncRNAs biomarkers tended to be involved in the lipid metabolic process. Together, our study highlighted the importance of lncRNAs in OSCC and demonstrated the utility of lncRNA expression as a promising biomarker for early diagnosis of OSCC.Long non-coding RNAs (lncRNAs) are recently emerging as a novel promising biomarker for cancer diagnosis and prognosis. Despite these previous investigations, the expression pattern and diagnostic role of lncRNAs in oral squamous cell carcinoma (OSCC) remain unclear. In this study, we identified six novel lncRNA biomarkers ( Oral cavity and pharynx cancer are one of ten leading cancer types with approximately 53,000 estimated new cases and 10,860 estimated deaths in the United States according to cancer statistics, 2019 . Oral sqRecent advances in high-throughput sequencing theologies have found that the approximately 2% human genome encodes only ∼20,000 protein-coding genes and the vast majority of the human genome were actively transcribed as non-coding RNAs (ncRNAs) . Long ncIn this study, to evaluate the potential of lncRNAs biomarkers in the diagnosis of OSCC, we compared differential expression profiles of lncRNAs between OSCC samples and controls in a larger OSCC dataset. Then we used multiple statistical methods to identify novel lncRNA biomarkers and developed an lncRNA-based diagnostic prediction model, which was validated in several independent OSCC datasets.1 including 167 OSCC samples and 45 normal oral tissues from GSE30784 dataset2  database,dataset2 , 26 OSCCaset2  of three OSCC sample datasets on Affymetrix Human Genome U133 Plus 2.0 (Affymetrix HG-U133 Plus 2.0) were downloaded directly from the GEO database and were processed and normalized using robust multichip average method by R “affy” package. LncRNA expression data of all samples in three datasets were retrieved by repurposing the probes based on the NetAffx annotation files of the probe sets and the annotation files of RefSeq and GENCODE according to previous studies , 2019. Fp-value < 0.05 was considered as differentially expressed lncRNAs.Differential expression analysis of lncRNAs between OSCC samples and control samples was performed using the R package “limma” (version 3.42.0), those lncRNAs with | log2(fold change)| > 1 and false discovery rate (FDR) adjusted The least absolute shrinkage and selection operator (LASSO) method was used to select the most useful predictive features from the list of differentially expressed lncRNAs as diagnostic lncRNA biomarkers in the discovery dataset. Then all diagnostic lncRNA biomarkers were fitted a logistic regression model as the covariates, and an lncRNA-based diagnostic risk model was constructed by using the sum of expression value of lncRNA biomarkers weighted by the unbiased coefficients estimates from the logistic regression model  as follox is expression levels of each lncRNA biomarker, w is the coefficients estimates from the logistic regression model, and b is 18.35146902. The risk score range from 0 to 1, and >0.5 was used as the cutoff for the diagnosis of OSCC.Where The diagnostic performance of the lncRNA-based diagnostic risk model was evaluated by a receiver operating characteristic (ROC) curve and the area under the ROC (AUC).Functional enrichment analysis of GO and Kyoto encyclopedia of genes and genomes (KEGG) pathway was performed to identify significantly enriched GO terms and KEGG pathways of mRNAs correlated with diagnostic lncRNA biomarkers using the R package “clusterprofiler” .p-value < 0.05]  | > 1 and FDR-adjusted  < 0.05] . Of them < 0.05] . Resultsed test) .LINC01697, LINC02487, LOC105376575, AC005083.1, SLC8A1-AS1, and U62317.1) from the list of differentially expressed lncRNAs as optimal diagnostic biomarkers considering a balance between classification accuracy and the number of lncRNAs  seem to be tumor suppressors and downregulated in OSCC, while lncRNA U62317.1 tended to be an oncogene and up-regulated in OSCC  were significantly downregulated and one lncRNA U62317.1 was significantly upregulated in OSCC samples compared with control samples, which is consistent with observed in the discovery dataset. Then each sample of GSE9844 was assigned a risk score according to 6lncRNAScore, and was classified as OSCC-like or normal-like samples. Finally, the 6lncRNAScore correctly classified all 26 OSCC samples and eight of 12 control samples, achieving an AUC of 0.968 with the sensitivity of 100% and the specificity of 66.7%  of 38 samples. We first investigated the expression patterns of six lncRNA biomarkers between OSCC samples and control samples in the GSE9844 dataset. As shown in of 66.7% .LINC01697, LINC02487, LOC105376575, AC005083.1, and U62317.1) of six lncRNA biomarkers revealed consistent expression patterns as observed in the discovery dataset and GSE9844 dataset except for SLC8A1-AS1. When the 6lncRNAScore was tested in the GSE74530 dataset, the 6lncRNAScore correctly classified all six OSCC samples and three of six control samples, achieving an AUC of 1.0 with the sensitivity of 100% and the specificity of 50%  of 12 samples. The expression pattern of six lncRNA biomarkers between OSCC samples and control samples in the GSE74530 dataset was examined. As shown in y of 50% .r > 0.8 and p < 0.05). Then we performed GO and KEGG enrichment analysis for 95 mRNAs correlated with six lncRNA biomarkers using the R package “clusterprofiler.” Results of functional enrichment analysis showed that mRNAs correlated with six lncRNA biomarkers might be involved in several lipid metabolic processes, such as membrane lipid metabolic process, cellular lipid catabolic process, sphingolipid metabolic process, fatty acid derivative metabolic process, and so on  between expression levels of mRNAs and six lncRNA biomarkers and identified 95 mRNAs correlated with six lncRNA biomarkers  and one oncogenic lncRNAs (U62317.1). Although the huge number of lncRNAs have been identified using high-throughput experimental technologies, only a very small fraction of lncRNAs were well functionally characterized. After the literature search, we found that several lncRNA biomarkers identified in our study have been reported to be involved in human diseases. LINC02487 has been reported as a tumor suppressor to inhibit migration and invasion of oral cancer cells through directly binding protein USP17 .Publicly available datasets were analyzed in this study. These data can be found here: Three publicly available OSCC datasets were obtained from the Gene Expression Omnibus (GEO) database (YL conceived and designed the experiments. YL, XC, and HL performed the experiments and analyzed the data. YL wrote the manuscript. All authors read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Metastatic cervical carcinoma from unknown primary (MCCUP) accounts for 1–4% of all head and neck tumors, and identifying the primary site in MCCUP is challenging. The most common histopathological type of MCCUP is squamous cell carcinoma (SCC), and it remains difficult to identify the primary site pathologically. Therefore, it seems necessary and urgent to develop novel and effective methods to determine the primary site in MCCUP. In the present study, the RNA sequencing data of four types of SCC and Pan-Cancer from the cancer genome atlas (TCGA) were obtained. And after data pre-processing, their differentially expressed genes (DEGs) were identified, respectively. Gene ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway analysis indicated that these significantly changed genes of four types of SCC share lots of similar molecular functions and histological features. Then three machine learning models,  which consisted of ten genes to distinguish these four types of SCC were developed. Among the three models with prediction tests, the RF model worked best in the external validation set, with an overall predictive accuracy of 88.2%, sensitivity of 88.71%, and specificity of 95.42%. The NN model is the second in efficacy, with an overall accuracy of 82.02%, sensitivity of 81.23%, and specificity of 93.04%. The SVM model is the last, with an overall accuracy of 76.69%, sensitivity of 74.81%, and specificity of 90.84%. The present analysis of similarities and differences among the four types of SCC, and novel models developments for distinguishing four types of SCC with informatics methods shed lights on precision MCCUP diagnosis in the future. Metastatic cervical carcinoma from unknown primary (MCCUP) is defined as metastatic disease in the lymph nodes of the neck without any evidences of a primary tumor after appropriate investigations. It is a type of cancer which originates from unknown primary sites, and squamous cell histology is prominent, accounting for 75–90% of cases . The speThe reasons for the failure of primary tumor diagnosis is not fully elaborated; however, the small size of the primary tumor may increase the difficulties in identifying the primary site and the evolutions of the cancer cell itself may be one of the reasons too . The devResearch discoveries derived through cancer genome and transcriptome studies have potential clinical impact as biomarkers . And macIn the present study, a dataset from The Cancer Genome Atlas (TCGA) RNA-Seq data of squamous cancer and TCGA Pan-Cancer (PANCAN) data were employed to conduct a series of bioinformatics analyses, and three machine learning models  were developed to explore the potantial effective tool to distinguish these squamous cancers.1. The GDC sample sheet of all squamous cell carcinomas of TCGA database2 were downloaded from TCGA database by using the searching strategy .The Cancer Genome Atlas RNA-Seq data of four types of cancer , and the phenotype data and TCGA PANCAN data were downloaded from University of California Santa Cruz (UCSC) Xena databaseBased on sample ID in the GDC sample sheet, the samples of SCC in the data PANCAN were extracted. Using the same methods we extracted the SCC data of the four types of cancer data . Then we renamed above data as PANCANsqu, CESC, ESCC, HNSC, and LUSC spectively. Using the function Rtsne provided by R Rtsne to visualize PANCANsqu based on t-distributed stochastic neighbor embedding (t-SNE) algorithm .The DESeq2 R package was used to identify differentially expressed genes (DEGs) of each squamous cancer data  , 2. PadjGene ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysisP < 0.01 was set as the threshold value.GO and KEGG pathway enrichment analysis was performed using clusterProfiler R package . The enr3 was used for analyzing the protein-protein interaction (PPI) of the DEGs of each cancer, and Cytoscape software4 was used to visualize the PPI network of the DEGs  values were determined to evaluate the performance of the established classifier in the validation set.The modeling process is briefly described below:Fristly, the function trainControl was used to define the parameters of sampling and cross-validation.method = “repeatedcv”, number = 10, repeats = 3, returnResamp = “all”, classProbs = TSecondly, The function train was used to build three training models.1.method = “rf”, mtry = 22.method = “svmRadial”, sigma = 0.3469467 and C = 13.method = “nnet”, size = 5 and decay = 0.1Lastly, The function predict was used to predict the sample type base on the training model, function extractPrediction and extractProb to acquire the model prediction results and their probabilities.To evaluate the performance of the model that have been established, the function confusionMatrix was used to obtain the confusion matrix and the ROCR R package to draw the ROC curve.Volcano plots were generated to visualize the distribution of expressed genes between cancer and normal controls from the four RNA-Seq data . Red or green dots in the plots represent significantly upregulated or downregulated genesrespectively . Venn diGO and KEGG pathway enrichment analyses of DEGs were performed using clusterProfiler R package, and the results were shown in the For “BP”, CESC and LUSC both showed enrichment in cornification, epidermis development, skin development, epidermal cell differentiation, keratinocyte differentiation, and keratinization. The differential DEGs expressed in ESCC mainly showed enrichment in digestion, extracellular structure organization, extracellular matrix organization, regionalization and hormone transport. The DEGs expressed in HNSCC mainly showed enrichment in response to muscle contraction, muscle system process, muscle filament sliding, actin-myosin filament sliding, myofibril assembly and striated muscle cell development.For the “cellular component (CC)” ontology, ESCC and LUSC both showed enrichment in transmembrane transporter complex, transporter complex, ion channel complex, anchored components of membrane. The differential genes expressed in HNSC mainly showed enrichment in myofibril contractile fiber part, sarcomere, I band and Z disc. The differential genes expressed in CESC showed enrichment in cornified envelope, collagen-containing extracellular matrix, condensed chromosome, contractile fiber and cell-cell junction.Regarding “MF”, the DEGs of ESCC and HNSCC both showed enrichment in extracellular matrix structural constituent, receptor ligand activity and passive transmembrane transporter activity. The differential genes expressed in LUSC mainly showed enrichment in substrate-specific channel activity, ion channel activity, ion gated channel activity and gated channel activity. The differential genes expressed only in CESC mainly showed enrichment in DNA-binding transcription activator activity, RNA polymerase II-specific glycosaminoglycan binding, actin binding and extracellular matrix structural constituent.For “KEGG pathway enrichment analysis”, LUSC mainly showed enrichment in Retinol metabolism, ascorbate and aldarate metabolism, Metabolism of xenobiotics by cytochrome P450 and Bile secretion. The differential DEGs expressed in ESCC mainly showed enrichment in Neuroactive ligand-receptor interaction, Protein digestion and absorption, Maturity onset diabetes of the young, Pancreatic secretion, Viral protein interaction with cytokine and cytokine receptor and Drug metabolism – cytochrome P450. The DEGs expressed in HNSCC mainly showed enrichment in Hypertrophic cardiomyopathy (HCM), ECM-receptor interaction, Salivary secretion, Calcium signaling pathway, and Dilated cardiomyopathy (DCM).The PPI network of the four types of squamous cancer was constructed using the STRING online database and Cytoscape . Then thThe t-SNE algorithm was used to visualize PANCANsqu data prior to model construction, as shown in Several statistical methodologies  were used to construct the prediction model with data from the training set, and for each statistical methodology, using subsets of DEGs (the top 10 genes generated by feature selection) instead of all DEGs as variables. Receiver operating characteristic curves were used to evaluate the predictive value . Among tIn this study, we investigated methods for the accurate diagnosis of the primary site of  MCCUP using microarrays of four potential primary tissues . We identified significant DEGs from four RNA-Seq data. The similarities and differences among the four types of squamous cancer were then analyzed using bioinformatics methods based on these significant DEG sets. Based on the data of TCGA PANCAN, a predictive RF model consisting of a ten-gene signature was established that could effectively discriminate between the four types of carcinoma.The four potential primary sites for MCCUP, CESC, ESCC, HNSC, and LUSC shared some common features. GO and KEGG pathway enrichment analyses showed clear similarities and differences among these four types of carcinoma. The KEGG pathway in which the DEGs of these four cancers are enriched, respectively, is rarely the same. The PPI network analysis provided detailed interactions/connections among the common DEGs. In the PPI network of the four types of squamous cancer, the top ten highly connected genes were selected. It is clear that in the top ten genes in PPI network of CESC, ESCC, and LUSC, nine of them  are the same.BUB1 mitotic checkpoint serine/threonine kinase (BUB1), BUB1 mitotic checkpoint serine/threonine kinase B (BUB1B), both of them play a central role in mitosis which are reported associated with aneuploidy and several forms of cancer . Cyclin Research discoveries derived from cancer genome and transcriptome studies have potential clinical impacts on biomarkers . MachineHowever, there are no studies using machine learning approaches for the diagnosis of MCCUP. In the present study, three statistical methodologies were used to construct a prediction model using data from the training set. For each statistical methodology, the use of subsets of DEGs instead of all DEGs improved the predictive performance. Among the three statistical methodologies  used to construct the prediction model, the ten gene RF model including C11orf85, LA16c-431H6.6, MYBPH, MAP9, FMO2, SCGB3A1, BPIFA1, TBX1, SRRM2, and AC016549.1 showed the best performance both in the training set and in the external validation set.The ten-gene signature capability of effectively differentiating the four types of squamous carcinoma has potential diagnostic value in MCCUP. The training set and validation cohorts were retrospective, therefore these findings must be validated prospectively in future studies. In addition, we just analyzed four potential primary sites of MCCUP, future studies should include additional potential primary sites of  MCCUP and more extensive data, as well as more complex machine learning methods.In conclusion, the present study analyzed the similarities and differences among CESC, ESCC, HNSC, and LUSC, which are four potential primary sites of MCCUP. A ten-gene predictive RF model was established based on the RNA-Seq data of the four types of carcinoma, which might have clinical utility for the accurate diagnosis of MCCUP and provide useful guidance for personalized and precision therapy.https://portal.gdc.cancer.gov/repository and https://xenabrowser.net/datapages/.The datasets for this study can be found in The Cancer Genome Atlas (TCGA) and the University of California Xena (UCSC Xena), Datasets link: DL and KC designed the study. DL, JJ, and XL were primarily responsible for conceptualization, methodology, and writing – reviewing and editing and contributed to this study equally. JJ and XL were responsible for data curation, software, and writing – original draft preparation. HW, SF, XS, and ZW were responsible for data revision. ZC, XY, HW, and KC revised the manuscript. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Thirty to forty percent of patients with Diffuse Large B-cell Lymphoma (DLBCL) have an adverse clinical evolution. The increased understanding of DLBCL biology has shed light on the clinical evolution of this pathology, leading to the discovery of prognostic factors based on gene expression data, genomic rearrangements and mutational subgroups. Nevertheless, additional efforts are needed in order to enable survival predictions at the patient level. In this study we investigated new machine learning-based models of survival using transcriptomic and clinical data.Gene expression profiling (GEP) of in 2 different publicly available retrospective DLBCL cohorts were analyzed. Cox regression and unsupervised clustering were performed in order to identify probes associated with overall survival on the largest cohort. Random forests were created to model survival using combinations of GEP data, COO classification and clinical information. Cross-validation was used to compare model results in the training set, and Harrel’s concordance index (c-index) was used to assess model’s predictability. Results were validated in an independent test set.TNFRSF9, BIRC3, BCL2L1 and G3BP2. Thereafter, we applied machine-learning models to predict survival. A set of 102 genes was highly predictive of disease outcome, outperforming available clinical information and COO classification. The final best model integrated clinical information, COO classification, 4-gene-based clusterization and the expression levels of 50 individual genes .Two hundred thirty-three and sixty-four patients were included in the training and test set, respectively. Initially we derived and validated a 4-gene expression clusterization that was independently associated with lower survival in 20% of patients. This pattern included the following genes:  Our results indicate that DLBCL survival models based on the application of machine learning algorithms to gene expression and clinical data can largely outperform other important prognostic variables such as disease stage and COO. Head-to-head comparisons with other risk stratification models are needed to compare its usefulness. DLBCL has an estimated incidence in the United States of 6.9 new cases per 100,000 people/year [dnisone) .MYC and BCL2 and/or BCL6, frequently named “double-hit” lymphomas, are associated with significantly shorter survival and have been reclassified as a new group of lymphomas by the World Health Organization [Improved biological characterization of DLBCL has led to the identification of new disease subtypes with prognostic implications. DLBCL cases with dual rearrangement of nization , 5. Siminization , 8, pavitime to event and censoring of the data.In the meantime, the emergence of artificial intelligence has brought new expectations to the field of medicine, particularly for disease diagnosis and prognostication. Classical models such as cox proportional hazard model and the log-rank test assume that patient outcome consists of a linear combination of covariates, and do not provide decision rules for prediction in the real-world . On the ML has been applied to predict survival in different clinical scenarios with encouraging results. The implementation of ML-based survival models is increasingly popular in order to provide patient-centered risk information that can assist both the clinician and the patient. Kim et al.  recentlyIn this study, we used gene expression data from DLBCL cases in order to create new models of survival based on retrospective data. Initially, we sought to identify transcripts and gene expression patterns associated with prognosis. Afterwards, we used this information in order to fit a random forest model capable of predicting overall survival with high-concordance. Comparisons with clinical data and COO classification are provided. We believe that our results will facilitate the establishment of individualized survival predictions in DLBCL.Affymetrix HG U133 plus 2.0 arrays for gene expression quantification. As the data from GSE23501 depends from British Columbia biobanks and part of the data from GSE10846 also originated from the same location, we used Spearman correlation to rule out duplicate samples. Indeed we detected 4 samples with almost perfect correlation (> 0.99) which we treated as duplicates and were removed from downstream analysis. A case treated with rituximab, doxorubicin, bleomycin, vinblastine and dacarbazine was also discarded, making a final validation set of 64 cases. No other pairs of samples were strongly correlated at the gene expression level (> 0.9). COO classification was originally deposited with gene expression data, and in both cases this classification was inferred exclusively from gene expression data. Log2-transformed expression data for both cohorts were obtained from the Gene Expression Omnibus (GEO) database [The gene expression database GSE10846 was used for training and the gene expression database GSE23501 was used as an independent test set . Briefly, the Mclust algorithm determines the most likely set of clusters according to geometric properties . An expectation-maximization algorithm is used for maximum likelihood estimation, and the best model is selected according to Bayes information criteria. The association of each of these probe-level clusters with overall survival was calculated using cox regression. Thereafter, those probes whose clusterization was significantly associated with survival  were selected for multivariate clusterization using the same Mclust algorithm. Cluster prediction was performed on the test set using parameters estimated in the training cohort, and cox regression was used to verify the association of this clusterization with overall survival. The Shoenfeld’s test was used to assess the proportional hazards assumption.The lgorithm  was usedp-value < 0.05) were discarded from further analysis.We initially tested the association of each probe with overall survival in the training set using multivariate cox regression. The Schoenfeld’s method was used to assess the proportional hazards assumption. Those probes which violated this assumption  was calculated as the integrated Brier score divided by time, and represents the average squared distances between the observed survival status and the predicted survival probability at each time point. CRPS is always a number between 0 and 1, being 0 the best possible result. Survival prediction on the test cohort was performed using the predict.rfsrc function with default parameters. Harrel’s concordance index (c-index) was used to assess model discriminative power on the bootstrapped training set and on the test set. C-index reflects to what extent a model predicts the order of events  in a cohort [Random forest survival models were created with the age in R . We decia cohort . C-indexvimp function, and we iteratively removed those samples with negative or low weight (importance < 1 × 10− 4). The number of random splits to consider for each candidate splitting variable (“nsplit”) was optimized by testing the performance of the algorithm in the training set with values in the range of 1 to 50 splits. Finally, we chose the best model in terms of c-index for replication in the validation set.Variable reduction was performed by iteratively removing those variables with low importance. Variable importance was calculated with the p-value < 0.05). These probes corresponded to the following genes: TNFRSF9, BIRC3, BCL2L1 and G3BP2. Two of these genes were significantly associated with survival in the test set, namely TNFRSF9  and BCL2L1 .Single probe clusterization revealed the existence of four probes strongly associated with overall survival  3.53, 95% confidence interval (CI) HR 2.01–5.93; Figs. p-value 2.06 × 10–9, HR 6.93, 95% CI HR 3.68–13.06). Cluster prediction on the independent test set classified a group of 20.31% of the patients in this cluster, and multivariate cox regression confirmed a significant and independent association with adverse outcome , rendering C-indexes of 0.6340 and 0.6202 in the training and test cohorts, respectively  and tuning the “nsplit” parameter in the training cohort, an improved model was constructed based on 54 items (Supplementary Table MS4A4A expression (probe id: 1555728_s_at) was the highest of all variables, followed by that of 4-gene expression clusterization. Furthermore, the expression of SLIT2 (probe id: 230130_at), NEAT1 (probe id: 220983_s_at), CPT1A (probe id: 203633_at), IGSF9 (probe id: 229276_at) and CD302 (probe id: 205668_at) were superior to that of COO classification.Finally, we tested several combinations of GEP-based variables and clinical information Table . The besIn this study we present a new random forest model to predict survival in DLBCL based on clinical and gene expression data. Using cox regression and unsupervised clustering we identified a set of transcripts and a 4-gene expression cluster associated with overall survival. This information was used to fit predictive models of survival using random forests. The best model outperformed some of the most important prognostic factors known in the field of DLBCL. Moreover, its combination with clinical information and COO classification rendered survival predictions that show high concordance with reality.MYC and BCL2 or BCL6 but no accompained by their genomic rearrangement) was described several years ago [. [TNFRSF9 [BIRC3 [BCL2L1 [The importance of gene expression biomarkers in DLBCL has been known for a long time. The COO classification was described almost two decades ago, linking DLBCL cellular ontogeny with clinical outcome . Similarears ago . Recent ears ago  presenteears ago . Similarears ago  used genears ago , whereass ago [.  identifi[TNFRSF9 , BIRC3 [9 [BIRC3  and BCL2 [BCL2L1 .Other interesting studies have reported notable advances in DLBCL risk stratification. Reddy et al  used exoThis is the first approach to our knowledge that combines GEP with artificial intelligence for survival prediction of DLBCL patients. Machine learning models come along with substantial benefits in the area of survival prediction. Firstly, there is no prior assumption about data distribution, and complex interactions between the variables can be modelled. Secondly, they do not simply rely on pre-defined assumptions about the pathology . Finally, gathered information is used to directly predict patient outcome, and individualized survival curves are obtained. These personalized approaches overcome the imperfect patient subgrouping derived from classical studies, and thus they are more useful in clinical practice. Our results might be particularly useful in order to select high-risk patients for inclusion in clinical trials.International Prognostic Index (IPI), NCCN-IPI and “double-hit” status. Although the IPI has proven to improve prognostic stratification of gene expression arrays [This study, like many others in the field of disease prognostication, has some limitations. Firstly, some important prognostic features were not available for this study, such as fragility scores, n arrays , there in arrays ). FurtheThis study presents a machine learning-based model for survival prediction of DLBCL patients based on GEP data and clinical information. The results of our model are superior to those described with current risk stratification scores , and head-to-head comparisons with other published machine learning approaches in the field of DLBCL are needed in order to compare their predictive utility. We believe that our results will pave the way towards the establishment of individualized survival predictions that will be useful in clinical practice and might prompt the development of novel first-line therapeutic interventions for selected patients.Additional file 1 : Supplementary Figure 1. Representation of out-of-bag CRPS over time.The red line represents CRPS for the whole population (see main text). Additionally, stratified CRPS by quartiles of out-of-bag ensemble (predicted) mortality are provided. Vertical lines above the x axis represent death events.Additional file 2 : Supplementary Table 1. List of the probes associated with overall survival using univariate cox regression. Only those probes with FDR < 0.1 are shown. Supplementary Table 2. Microarray probes included in the GEP_0.1 gene expression pattern. Supplementary Table 3. Importance of the different variables in the best random forest model after variable pruning.