Methods for depth normalization have been assessed primarily with simulated data or cell-line–mixture data. There is a pressing need for benchmark data enabling a more realistic and objective assessment, especially in the context of small RNA sequencing.We collected a unique pair of microRNA sequencing data sets for the same set of tumor samples; one data set was collected with and the other without uniform handling and balanced design. The former provided a benchmark for evaluating evidence of differential expression and the latter served as a test bed for normalization. Next, we developed a data perturbation algorithm to simulate additional data set pairs. Last, we assembled a set of computational tools to visualize and quantify the assessment.We validated the quality of the benchmark data and showed the need for normalization of the test data. For illustration, we applied the data and tools to assess the performance of 9 existing normalization methods. Among them, trimmed mean of M-values was a better scaling method, whereas the median and the upper quartiles were consistently the worst performers; one variation of remove unwanted variation had the best chance of capturing true positives but at the cost of increased false positives. In general, these methods were, at best, moderately helpful when the level of differential expression was extensive and asymmetric.Our study (1) provides the much-needed benchmark data and computational tools for assessing depth normalization, (2) shows the dependence of normalization performance on the underlying pattern of differential expression, and (3) calls for continued research efforts to develop more effective normalization methods. Earlier methods were based mostly on the scaling strategy, which calculates a scaling factor  for each sample to adjust the data.Key ObjectiveThere is a pressing need for objective assessment of depth normalization methods using realistic and robust benchmark data in the context of small RNA sequencing.Knowledge GeneratedWe collected such benchmark data and accompanying test data and developed computational tools. We demonstrated their use with the assessment of 9 normalization methods. These methods were, at best, moderately helpful, especially when differential expression was extensive and asymmetric.RelevanceOur study provides evidence on the subpar performance of existing normalization methods and calls for continued research efforts to develop more effective methods.8 They play an important regulatory role in gene expression in the cell and are closely linked to cell apoptosis and carcinogenesis.10We set out to develop the data and analytics to enable a more realistic and objective assessment of depth normalization methods, focusing on a class of small RNAs called microRNAs (miRNAs). MiRNAs are 18 to 22 nucleotides long, which minimizes the potential bias in abundance estimation due to gene length variation, as seen in RNAs.Toward this end, we designed and collected a unique pair of data sets for the same set of tumor samples at Memorial Sloan Kettering Cancer Center (MSKCC). The first data set was collected using (1) uniform handling to minimize data artifacts and (2) balanced sample-to-library assignment (via blocking and randomization) to avoid confounding for any residual artifacts with the sample groups under comparison. For the same set of samples, a second data set was collected without using such careful study design, resulting in obscuring depth variations. Evidence of differential expression was assessed in the first data set, serving as the benchmark; normalization methods were tested in the second, followed by differential expression assessment and benchmark comparison. Throughout the rest of this article, we refer to the first data set as the benchmark data set and the second as the test data set. For extra quality assurance, we added for the benchmark data set two pooled samples shared across all libraries and 10 prevalidated calibrators added at fixed concentrations.2In addition to the empirical data, we devised a data perturbation strategy to generate additional data set pairs under various differential expression scenarios. Furthermore, we put together a set of graphical tools and numeric metrics for visualizing and quantifying the impact of depth normalization on differential expression detection. Here, we report the data and tools and illustrate their use on nine normalization methods that are relatively commonly used in the literature.13 Their tumor samples are typically large enough for RNA extraction with sufficient quantity and quality for sequencing. In this study, we used 27 MXF samples and 27 PMFH samples, which were all from newly diagnosed, previously untreated, primary tumors collected at MSKCC between 2000 and 2012. Sample preparation and extraction were performed in the Singer laboratory at MSKCC. Library preparation and sequencing were done in the Tuschl laboratory at Rockefeller University. Detailed description of these steps was previously reported.15Myxofibrosarcoma (MXF) and pleomorphic malignant fibrous histiocytoma (PMFH) are the two most common and aggressive subtypes of genetically complex soft tissue sarcoma.14With barcode multiplexing, we used three libraries to sequence the 54 individual tumor samples and two pooled samples (one from pooling the 27 MXF samples and the other from pooling the 27 PMFH samples); each library included nine MXF samples, nine PMFH samples, the pooled MXF sample, and the pooled PMFH sample. We carefully planned our study for generating this benchmark data set so the library preparation and read capture were each processed by a single experienced technician in one run. In addition, samples of each tumor type were randomly assigned to the three libraries and the 20 barcodes. To further ensure data quality, we included 10 calibrators with fixed input concentrations for each sample to use as negative controls for differential expression.16We assessed the agreement of the three sequencing runs for each pooled sample, as well as the agreement between group means based on individual samples and those based on pooled samples, using scatter plots and concordance correlation coefficients.https://www.r-project.org/about.html) as the primary method for differential expression analysis, with the results reported in this article; and edgeR as a secondary method with the results reported in the Data Supplement.19 Additional description of the choice of these methods is provided in the Appendix.We assessed the evidence against the null hypothesis of equivalent expression in MXF versus PMFH using the benchmark data for the 54 tumor samples. We used voom ,1 median,2 trimmed mean of M values (TMM),17 DESeq,3 and PoissonSeq.20 The latter included quantile normalization (QN) with and without removing low-count miRNAs,21 surrogate variable analysis (SVA),6 and remove unwanted variation (RUV), with three variations: RUVg, RUVr, and RUVs.5 Detailed description of these methods is provided in the Appendix.We examined nine normalization methods, including six scaling-based methods and three regression-based methods . The former included total count (TC),The test data were assessed for differential expression, before and after normalization, using voom as the primary method and edgeR as the secondary method in the same manner as the benchmark data.We assembled several useful graphical tools and numeric measures to evaluate the impact of a normalization method on the overall distribution of the test data and on the differential expression status and significance ranking of miRNAs, as well as to explore the relationship among different normalization methods.22 More specifically, it adjusts the log2 count data for each sample by subtracting its median and displays the deviations as a box plot. In our implementation, we sorted the box plots for the samples within each group by the upper quartiles of their deviations determined in the benchmark data.The relative log expression plot examines the impact of normalization on the overall data distribution.P values with the benchmark.6 More specifically, it uses indices 1 to K as the x-axis, and the percentage of agreement in the miRNAs ranked among the top K between the two data sets as the y-axis. The plot can be done for multiple normalization methods at a time, with each method plotted in a different color.The concordance at the top (CAT) plot compares the ranking of significant miRNAs on the basis of the test-data P values from differential expression analysis were used to derive a marker set at a given significance level: markers with P values smaller than the significance level were declared differentially expressed, and those having larger P values were declared nondifferentially expressed. Differential expression statuses were compared between the two data sets graphically via the Venn diagram.The The comparison of differential expression statuses between the two data sets was summarized numerically using a false-negative rate (FNR) and a false-discovery rate (FDR), at the risk of abusing the terminology. They were compared between different normalization methods, using the scatter plot.P values  were subjected to hierarchical clustering using the Euclidean distance and the Ward linkage. Clustering results were displayed using the dendrogram.To assess similarity between normalization methods, their test-data We examined the normalization methods in additional data sets simulated under a range of differential expression scenarios. More specifically, the sample-type labels were permuted to reach a specific proportion of differential expression and a specific magnitude of median group-mean difference (for log2 count) in the benchmark data set, and the same permutation of group labels was then applied to the test data. The goal was to generate data sets with various proportions of differentially expressed markers  and magnitudes of mean difference among differentially expressed markers. This strategy allowed us to maintain the between-miRNA correlation structure in each data set and not make any parametric assumptions regarding the distributions of sequencing count and depth variation.pam function in the cluster library) and labeled as groups 1 and 2; (2) nine seed samples were randomly selected from each group, with three samples from each sequencing library; (3) the remaining 36 samples were then randomly and equally allocated to the two groups. For each permutation of the 54 samples, differential expression between the two groups was assessed with regard to the proportion of differential expression and the magnitude in terms of the median of marker-specific group-mean differences.Data were simulated with a combination of hierarchical clustering and random shuffling, with stratification by library , in the following three steps: (1) the benchmark data of the 54 samples were clustered using K-means clustering to two clusters  differential expression with group mean difference approximately 0 (ranging from −0.5 to 0.5);Limited and asymmetric: 2% differential expression with group mean difference approximately 3 (2-4);Moderate and asymmetric: 10% (8%-12%) differential expression with group mean difference approximately 3; andExtensive and asymmetric: 20% (15%-25%) differential expression with group mean difference approximately 3.Each pair of the simulated data sets was analyzed and compared in the same manner as the empirical data. The results, in terms of FNR and FDR, for each simulation scenario were displayed aggregately as box plots.P-value range, 0.11 to 0.96; There was excellent agreement between the replicates for each pooled sample ; median and UQ were consistently the worst performers . Among rP values. The comparison showed separation of scaling methods from regression-based methods and separation of median and UQ from the other scaling methods  and asymmetric, TMM, TC, and PoissonSeq occasionally decreased the FDR, yet frequently increased FNR; again, QN with filtering tended to decrease the FDR and increase the FNR, whereas RUVr tended to decrease the FNR and increase the FDR; the other methods decreased neither the FNR nor the FDR.When differential expression in the data was moderate  and asymmetric, test data with no normalization were often associated with a reasonable FDR and FNR, due to the strong level of biologic signal; TMM occasionally decreased the FDR yet frequently increased the FNR, whereas the other scaling normalization methods consistently increased both the FDR and the FNR; RUVr tended to decrease the FNR but not the FDR.Taken together, the performance of normalization methods depended on the specific pattern of differential expression and, in general, only brought limited benefits; TMM tended to outperform the other scaling methods, and RUVr tended to outperform the other regression-based methods; the median and UQ methods were consistently the worst performers among the methods examined in our study.In absence of a predetermined standard for comparison, authors of a new normalization method have the freedom to select evaluation procedures that favor their method and subsequently claim superiority over other methods. This self-assessment trap results in contradictory information for users deciding on a method. In this study, we addressed this problem for miRNA sequencing by developing the carefully designed pair of data sets, the resampling-based simulation algorithm, and the relevant graphical and numeric analytics. We make these data and tools publicly available to the research community so interested researchers can reproduce our study and study additional methods reported in the literature and new methods as they emerge.25 We expect this general strategy to be useful in assessing normalization for other types of high-throughput data, as well. In this article, we used this strategy to examine the unique challenge of depth normalization for miRNA sequencing. Our findings can provide a useful stepping stone for understanding the same issue of depth normalization for the sequencing of RNAs and metagenomics.27 Compared with miRNA sequencing, RNA sequencing is subject to an additional source of depth variation from gene length, whereas metagenomic sequencing faces the complication due to data compositionality and sparsity to a greater extent.28 They present even more challenging scenarios to have effective normalization methods.We previously successfully applied the paired data set strategy to assess normalization methods for miRNA microarrays.In conclusion, caution should be exercised when applying depth normalization, and more effective methods should be developed with robust, realistic, and objective assessment.


A major racial disparity in prostate cancer (PCa) is that African American (AA) patients have a higher mortality rate than European American (EA) patients. We filtered the SEER 2009–2011 records and divided them into four groups regarding patient races and cancer grades. On such a partition, we performed a series of statistical analyses to further clarify the aforementioned disparity. Molecular evidence for a primary result of the epidemiological analysis was obtained from gene expression data. The results include: (1) Based on the registry-specific measures, a significant linear regression of total mortality rate  on the percentage of (Gleason pattern-based) high-grade cancers (PHG) is demonstrated in EAs (p < 0.01) but not in AAs; (2) PHG and its racial disparity are differentiated across ages and the groups defined by patient outcomes; (3) For patients with cancers in the same grade category, i.e. the high or low grade, the survival stratification between races is not significant in most geographical areas; and (4) The genes differentially expressed between AAs’ and EAs’ tumors of the same grade category are relatively rare. The perception that prostate tumors are more lethal in AAs than in EAs is reasonable regarding AAs’ higher PHG, while high grade alone could not imply aggressiveness. However, this perception is questionable when the comparison is focused on cases within the same grade category. Supporting observations for this conclusion hold a remarkable implication for erasing racial disparity in PCa. That is, “Equal grade, equal outcomes” is not only a verifiable hypothesis but also an achievable public health goal. Adenocarcinomas amount to 95% of PCa cases2. Racial disparities in this cancer type have been revealed by numerous epidemiological studies5. One of those disparities is that African American (AA) patients have a higher mortality rate than their European American (EA) counterparts. A popular explanation for this disparity is that prostate tumors may be more aggressive in AAs than in EAs8. Molecular evidence for such an explanation has been reported by several studies in the past years11.Prostate cancer (PCa) is the most commonly diagnosed non-skin cancer and the second leading cause of cancer mortality in American men14. For example, Fowler and Terrell (12) reviewed the outcomes of 148 black and 209 white men with localized prostate cancer treated with surgery or radiation therapy over an 11-year period at a Veterans Affairs medical center. Their results showed that, after a median follow-up of 96 months, there were no significant differences in all-cause, cause specific, metastasis-free, clinical disease-free, or prostate-specific antigen recurrence-free survival rates in 109 black and 167 white men with low-stage cancer treated with surgery or radiation therapy, or in 39 black and 42 white men with high-stage cancer treated with radiotherapy.Nevertheless, the validity of the notion that prostate tumors may be more aggressive in AAs than in EAs is severely compromised by the “Equal care, equal outcomes” phenomenon or conception, as demonstrated inOn the other hand, the conception of “Equal care, equal outcomes” has its own problem. That is, it is often a conclusion drawn from the results of limited-size cohorts. To achieve a more comprehensive and accurate understanding of the racial disparities in the outcomes of PCa patients, we performed an integrative analysis of large-scale clinical data collected by the Surveillance, Epidemiology, and End Results Program (SEER) and the gene expression data deposited in the Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases. Most importantly, in addition to other research objectives, we primarily aim to test the “Equal grade, equal outcomes” hypothesis, namely that no disparity in mortality exists for patients whose cancer is in the same grade category (low or high).In this study, we used three sources of data, i.e., the SEER data, the TCGA digital gene expression data (Ex-1), and an aggregation of three GEO microarray gene expression datasets (Ex-2). The samples in all of these datasets compose patients of multiple races, facilitating disparity research, especially the test of the proposed hypothesis. Our statistical analysis focused on the samples from non-Latin European American (EA) and African Americans (AA) populations. The integration of the SEER data with the gene expression data was achieved by an approximate alignment of the cancer grades via Gleason patterns (GP). That is, the high-grade or low-grade cancers in the SEER data correspond to the GP-4 or GP-3 specimens in the gene expression data. Specifically, we partitioned the SEER samples into four groups, i.e., AA-LG, AA-HG, EA-LG and EA-HG, where HG and LG denote the high-grade and low-grade cancer (patient), respectively. Our primary results or conclusions for a specific research question were derived by performing pairwise comparisons of those four groups. The gene expression data were further analyzed to test if a biological basis, inferred from the stratification of gene expression levels across populations and Gleason patterns, exists for the SEER data-based results about mortality disparity patterns.We retrieved prostate cancer SEER data from the 2009–2011 database, and then refined the data to generate a working dataset (SEER-WD), containing 86996 AA and non-Hispanic EA cancer cases collected by 48 registries . Each registry represents a county or a parish in California, Louisiana and other five states. These registries are selected as each of them has at least 100 AA or EA patients documented during the studied time period  ranking from 2 to 4, 5 to 6 and 7 to 10, respectively . In this study, we combined I and II into the low-grade category (LG), and combined III and IV into the high-grade category (HG). The main difference between a HG patient and a LG patient is that a representative tumor specimen from the former but not from the latter contains Gleason patterns 4 or 5 (GP-4 or -5) as the primary or second prevalent ones. It is well known that tumorous cells in GP-4 and GP-5 are more aggressive than those in GP-318. Therefore, HG and LG could be considered as two prostate cancer “subtypes” with inequivalent lethality.The SEER determines the Total mortality rate (TMR) and prostate cancer specific mortality rate (PSMR) are calculated using the formula TMR = M/T and PSMR = M1/T, respectively. Here, T is the total number of the patients; M is the number of patients with the values in the “Vital status recode (study cutoff used)” column of the SEER data being “Dead”; and M1 is the number of patients with the values in the “SEER cause-specific death classification” column of the SEER data being “Dead (attributable to this cancer dx)”.19. A refined version of this dataset, which includes 65 GS-6(3 + 3), 102 GS-7(3 + 4), 78 GS-7(4 + 3) and 44 GS-8(4 + 4) tumors, is focused on in our study. TCGA quantified and normalized the gene expression levels using the RSEM (RNASeq by Expectation Maximization) method21.Ex-1 contains the log2 transformed level-3 digital (RNA-seq) gene expression profiling of 333 TCGA prostate adenocarcinomas samples whose Gleason patterns (GPs) were reviewed/corrected by two pathologists22, GSE6266723, and GSE7229124. These three cohorts are respectively consisted of 131, 182 and 139 samples. Together, the numbers of GS-5, -6, -7, -8, -9, and -10 tumors are 1, 86, 265, 54, 42 and 2, respectively. There are two tumors whose GSs are missing in the clinical data. In order to reduce unnecessary complexities in statistical analysis and succinctly present the results, we further ignored the difference between GS-5 and GS-6 tumors (i.e. treat both of them as “GS-6”) and the difference among GS-8, GS-9 and GS-10 ones . The gene expression levels of all these tumors were measured by the same platform, i.e. Affymetrix Human Exon 1.0 ST Arrays. We first downloaded the raw data from the GEO database and then used the frozen Robust Multi-array Analysis algorithm25 to summarize and normalize the transcriptomic quantities to generate gene-level expression profiling of each individual sample. Finally, the quantile normalization was repeatedly applied to the expression matrix of the 452 samples.Ex-2 is a composite dataset containing the clinical and gene expression information of primary prostate cancers in three cohorts: a section of GSE2103426. As a result, before performing the advanced analysis, we used a machine learning method  to partition the GS-7 tumor samples into the GP-3 and GP-4 specimens. In this way, we substantially augmented the clinical information of the gene expression datasets, which facilitates the comparison of gene expression programs between high-grade and low-grade cancers.While Ex-1 and Ex-2 are the largest datasets available to us for studying racial disparities in gene expression of PCa tissues, our research is still subject to a challenge arising from data limitations. That is, the numbers of samples from the interested minority population, i.e. AA, is not sufficiently large. In both datasets, only 12–13% of tumor samples are from AA patients. This situation is further complicated as these tumors are distributed among four to five Gleason Grade-based categories. In particular, it is inappropriate to use the gene expression information of GS-7 tumors, which account for approximately a half of the samples, in a disparity analysis. The reason is that, for a GS-7 tumor, the Gleason pattern (GP-3 or GP-4) of the specimen used in the RNA-seq or microarray experiment is uncertain. This is different from a GS-6 or GS-8 tumor whose experimental specimen can be heuristically assigned to GP-3 or GP-4 category, respectively. On the other hand, we have identified a strong expression signature of 288 genes for distinguishing GP-4 specimens from GP-3 specimens in another work27, and an on-line available R function heatmap.3 (http://www.biostars.org/p/18211/).We used linear regression to model the relationship between the proportion of high-grade cancers and mortality rate. The differences of survival time between two patient groups were evaluated by Cox-PH regression. T-test was used to identify the differentially expressed genes between two sample categories. The hierarchical clustering analysis was performed using Ward’s method and the Manhattan distance. Fisher’s exact test was adopted to test the independence of two different sample partitions and to evaluate the functional enrichment of significant genes. The employed software includes the relevant functions in R packages “stats”, “gplot” and “survival”, the David toolsvm function in the R package “e1071” was implemented with the default parameters except for the class weights and kernel type. We specified the class weights as the reciprocals of the fractions of the GP-4 and GP-3 samples in the training set. The pattern of each GS-7 tumor was predicted twice using the linear and sigmoid kernels, respectively. Only the tumors with consistent predictions were kept for further analysis. Our preliminary study showed that such a double-kernel prediction and filtering can warrant the sensitivity and specificity being over 0.9.We used the Support Vector Machine (SVM) (9) to predict the actual Gleason pattern (GP-3 or GP-4) of the specimen from a GS-7 tumor. A SVM model was trained on a dataset consisting of GS-6 and GS-8 samples. The All the data used in this study have been previously published or can be freely obtained from public resources. All analyses were performed using standard statistical methods in accordance with relevant guidelines and regulations. All protocols of experiments and information collection were approved by the National Institute of Health (NIH) when the original owners of the data carried out their studies. The gene expression data meet the minimum information standards. The informed consent was obtained from all prostate cancer patients, who were over 18.Our results show that the nation-wide TMR and PHG for the EA population are 0.126 and 55.9%, respectively. The corresponding quantities for the AA population are 0.157 and 60.4%, respectively. Registry-specific TMR and PHG, called R-TMR and R-PHG hereafter, are calculated with respect to each individual registry as the “experimental” unit. For EAs, the R-TMRs range from 0.08 to 0.181 and R-PHGs range from 42.7% to 71.0%. For AAs, the R-TMRs range from 0.065 to 0.23 and R-PHGs range from 43.6% to 74.7%. According to a paired t-test, the difference between AAs and EAs is extremely significant in both R-TMR (p = 0.003) and R-PHG (p < 0.000001). As expected, a significant linear regression of R-TMR on R-PHG is observed in EAs (p < 0.01) and the Pearson correlation (r) between these two metrics is up to 0.38. However, such a relationship is not observed in AAs  for EAs and AAs range from 0.004 to 0.067 and 0.012 to 0.08, respectively. According to a paired t-test, the difference between AAs and EAs is significant (p < 0.01). Similar to the situation for R-TMR, a significant linear regression of R-PSMR on R-PHG is observed in EAs  but not in AAs , i.e. <45, 45–49, 50–54, 55–59, 60–64, 65–69, 70–74, 75–79, 80–84, and > = 85 years  contains 10077 patients who died during the follow-up periods regardless of the cause of death. The second (WD-2) contains 2825 patients whose deaths were attributable to prostate cancer. Using the same method employed in the previous section, we graphically depict the association patterns between PHG and patient age. The patterns obtained from WD-1 and WD-2 are demonstrated in Fig. I, II, III and IV, respectively. I: Racial survival disparity is observed in patients of both HG and LG (cancer) groups, which is the pattern present in Georgia::Fulton County, Louisiana::East Baton Rouge Parish and other three registries. II: The disparity is observed only in patients of HG group, which is the pattern present in California::Alameda County, New Jersey::Essex County and other three registries. III: The disparity is observed only in patients of LG group, which is the pattern in Louisiana::Caddo Parish. IV: The disparity is not observed in patients of both groups, which is the pattern present in California::Riverside County, Georgia::DeKalb County and other two registries.Among the registries included in SEER-WD, sixteen have at least 280 patients in both EA and AA populations. Geographical area-specific survival analysis is performed on the data of these registries. The statistical significance of the survival stratification of the AA-HG and EA-HG groups, as well as the stratification in AA-LG and EA-LG groups, is evaluated. The results for all-causes mortality are shown in Fig. The results of survival analysis for prostate cancer specific mortality are shown in Supplementary Fig. This set of analyses is separately performed on the TCGA digital gene expression dataset (Ex-1) and the composited microarray gene expression dataset (Ex-2). The objectives include: (1) evaluating the racial effects on individual gene expression levels of the tumor specimens with the same Gleason pattern (or grade category); and (2) evaluating the effects of Gleason patterns on individual gene expression levels of the tumor specimens of the patients of the same race. Based on the patient races (i.e. AA and EA) and the Gleason patterns (i.e. GP-3 and GP-4) of tumor specimens, we establish four sample groups, i.e. AA&GP-3, AA&GP-4, EA&GP-3 and EA&GP-4. In Ex-1, the sizes of the four specimen groups are 21, 17, 106 and 136, respectively. In Ex-2, the corresponding numbers are 30, 19, 159 and 164, respectively. For the objective (1), we conducted two comparisons, i.e. AA&GP-3 versus EA&GP-3 and AA&GP-4 versus EA&GP-4. For the objective (2), we performed another two comparisons, i.e. AA&GP-3 versus AA&GP-4 and EA&GP-3 versus EA&GP-4.The results obtained from the analysis of dataset Ex-1 are shown in Fig. Apparently, the genes with different expression levels between AA and EA specimens of the same Gleason pattern are rare, and the contribution of the Gleason patterns to the variability of gene expression activity is much more significant than patient races. This conclusion is verified by the same comparison analysis performed on the composite microarray gene expression data , the nation-wide PHG in AAs is ~1.2 times of that in EAs; Second, for the patients with tumors in the same grade category (HG or LG), the survival stratification between races is not significant in most counties (or parishes) located in different states; and third, the genes with different expression levels between AA and EA patients with cancers in the same grade category are very limited in number. These results, especially the second one, hold a remarkable implication for erasing racial disparities in prostate cancer. That is, “Equal grade, equal outcomes” is not only a verifiable hypothesis but also an achievable public health goal. In some areas, such as DeKalb County in Georgia and Jefferson Parish in Louisiana, AA and EA patients have the similar outcomes. The experiences of these regions in patient surveillance and therapy could be useful to some other areas, such as Fulton County in Georgia and Wayne County in Michigan, where the mortality disparity is substantial. The reason is that, while the between-race survival differences of same-grade patients observed in the latter counties or parishes reflect the findings of several publications30, which suggests that racial inequality in prostate cancer outcome is mainly due to socioeconomic imbalances rather than biological factors. Based on a comprehensive analysis of the data of three cohorts, i.e. SEER, a pool of four randomized clinical trials and an equal access health care system, Dess et al. found that prostate cancer–specific mortality by race did not appear to differ in the equal-access health care system, and the outcomes were even better for black than for white patients in well-designed and well-conducted clinical trials. Regarding the analysis performed on the SEER data, Dess et al. used a propensity model to adjust the statistics of patient outcomes for socioeconomic imbalances and clinical factors such as biopsy Gleason score (corresponding to “grade” in our study). As a result, they did not identify substantial racial disparity in prostate cancer specific mortality.Our opinion about the achievability of the “Equal grade, equal outcomes” goal is largely supported by the results of a recent publicationOn the other hand, our study and results are subject to the limitations of using the SEER data and some conclusions could be weakened. Especially, the causes of ~20% of death cases are missing in the data but the relevance of prostate cancers in those deaths cannot be excluded. This could lead to an underestimate of PSMR and temper the benefit of a PSMR-based survival analysis. Moreover, SEER does not have a mechanism to guarantee the consistence of the protocols in measuring the Gleason scores (GS) of tumors across hospitals and, therefore, miscoding for GS-based grades of tumors is unavoidable. This potentially compromises the accuracy of the disparity evaluation performed in this study.The first is the biological functions of the genes that show “significant” differences between the AAs’ and EAs’ tumors. Using the expression dataset Ex-1 (or Ex-2), we identify 49 and 51 (or 50 and 74) differentially-expressed genes in the contrasts, AA&GP-3 versus EA&GP-3 and AA&GP-4 versus EA&GP-4. The functional enrichment analysis of those genes does not show any relevance to tumor mortality. Those genes also hardly overlap with the 46 significant genes identified in a previous study on racial disparity of gene expression10, which are enriched with a few gene ontology terms related to immune response. The second is whether there is a disparity in the mutation spectrum of tumor cells. Because of the lack of a predominant cancer gene which is widely mutated over prostate tumors31 and the limited sample size of AA patients, our preliminary analysis of the TCGA somatic mutation data does not identify any other genes with significantly different mutation frequencies between AAs and EAs. Although ETS fusions, predominated by TMPRSS2-ERG fusions, are two-times popular in EAs’ tumors compared to AAs’ tumors35, their effect on patient outcomes is controversial38. The third is the possibility of attributing the observed disparity in mortality rate to the potential disparity in cancer progression. While, to our knowledge, this point has been barely addressed by previous studies, the possibility is small in our opinion. The reason is that, in general, a GP-3 tumor cannot directly progress into a GP-4 tumor39.There are other points relevant to the aforementioned conclusion worthy of further discussion. Due to the complex demographical and socio-economic factors as well as the measurement errors arising from the limited sizes of AA patients in some registries, it is no wonder that there is a substantial variability in R-PHGs and R–TMRs, i.e. registry-specific PHG and TMR measures. However, a positive correlation between the R-PHGs and T–TMRs is expected if there is no demographical disparity in patient surveillance and therapy. Here, we do observe such a relationship in EAs but not in AAs. This represents a unique inequality. The underlying reason may be that, among AA communities, there is substantial variability in access to effective surveillance and therapy that masks the effect of cancer grade on patient survival. Therefore, this disparity could be attributed to AAs’ poor socio-economic situation in some communities.42.It is well known that PCa patients with tumors diagnosed before the age of 50 years are rare and that such early-onset patients are more frequent in AAs than in EAs. According to the records of the working dataset SEER-WD, 2030 EA men and 1105 AA men are within this age category (i.e. <50 years). They account for 3.0% and 6.1% of the EA patients and AA patients, respectively. Importantly, we find a relatively lower PHG in those early-onset AA patients whose death is attributable to prostate tumors. As shown in Fig. Supplementary Figure S1-S5.


Alternative splicing (AS) is an important mechanism of regulating eukaryotic gene expression. Understanding the most common AS events in colorectal cancer (CRC) will help developing diagnostic, prognostic or therapeutic tools in CRC.Publicly available RNA-seq data of 28 pairs of CRC and normal tissues and 18 pairs of metastatic and normal tissues were used to identify AS events using PSI and DEXSeq methods.The highly significant splicing events were used to search a database of The Cancer Genome Atlas (TCGA). We identified AS events in 9 genes in CRC  or metastasis . Except for CHEK1, all other 8 splicing events were confirmed by TCGA data with 382 CRC tumors and 51 normal controls. The combination of three splicing events was used to build a logistic regression model that can predict sample type  with near perfect performance (AUC = 1). Two splicing events (COL6A3 and HKDC1) were found to be significantly associated with patient overall survival. The AS features of the 9 genes are highly consistent with previous reports and/or relevant to cancer biology.The significant association of higher expression of the COL6A3 E5-E6 junction and HKDC1 E1-E2 with better overall survival was firstly reported. This study might be of significant value in the future biomarker, prognosis marker and therapeutics development of CRC. Colorectal cancer (CRC) is the third and second most common cancer in men and women worldwide  and it’sAS is an important mechanism of regulating eukaryotic gene expression. Ninety-two to 94 % of human genes undergo AS. Different protein isoforms produced by the same gene through AS may have related, distinct or even opposing functions . RegulatAberrant splicing events in CRC and/or other cancer types were identified using either microarray or RNA-seq techniques in the past –17. The In this study, we sought to identify recurrent AS event as potential biomarkers for diagnosis, prognosis and potential new therapeutic targets in CRC. With the publicly available RNA-seq data, we studied 28 pairs of CRC and normal tissues and 18 pairs of metastatic and normal tissues. As results, we identified AS events of 9 genes in CRC or metastasis. Eight splicing events were confirmed by independent TCGA data with 382 CRC tumors and 52 normal controls, which could be used as biomarkers of CRC. One AS event in CLK1 gene might be a novel therapeutic target. Finally, two AS events were found to be significantly associated with patient overall survival.We collected two public RNA-seq data related to CRC from GEO database (Supplementary Table https://www.ncbi.nlm.nih.gov/sra/) using SRA Toolkit. Reads were mapped to the human hg19 genome using STAR (2.5.3a) [Raw reads in fastq format of the two GEO datasets (Supplementary Table 1) were downloaded from SRA ((2.5.3a) . Only thhttps://bedtools.readthedocs.io/en/latest/index.html). Instead of visualizing read number coverage, all read coverage data in the bedgraph file were normalized to reads per million (RPM) value by dividing by the total number uniquely mapped reads in a sample. Then the bedgraph files were converted to bigwig file using the bedGraphToBigWig program downloaded from UCSC genome browser (http://hgdownload.soe.ucsc.edu/admin/exe/) and visualized in UCSC genome browser (https://genome.ucsc.edu). The height of each coordinate in the genome represents the RPM value of the read coverage at that coordinate.The Sashimi Plot showing the junction read number was generated using Integrative Genomics Viewer (IGV) . The unihttps://github.com/tianjiagene/GneneBlockReadCount). The exon read counts table were used as input for the R library DEXSeq [P-value< 0.001 and fold change> 2 were used to select significantly regulated exons.For splicing analysis of exons, we used Refseq defined exons and counted the reads mapped to them using custom perl and R scripts  were compared using Wilcoxon rank-sum test. For ΔPSI (the PSI change between two sample groups), the PSI values of each group were averaged and then the difference was calculated. https://cran.r-project.org/web/packages/survminer/). P-value is calculated based on the log-rank test. The data for percent of stromal cells in tumor tissues were downloaded from the GDC website . The correlation of junction usage and percent of stromal cells was tested using Spearman correlation.All TCGA patient overall survival (OS) data and junction usage data were downloaded from TSVdb . We seleThe junction usage data for the eight genes in 382 CRC tumors samples  and 51 normal colon or rectal tissue samples were used to train and predict whether a sample is a CRC or normal. Each time, up to 3 junctions/genes were selected as predictors. A logistic regression model was built based on randomly selected 50% of the samples. Then the model was used to predict the rest 50% of the samples. The Area Under Curve (AUC) of Receiver Operating Characteristic (ROC) curve was used as indicator of how good the model is. For the same set of predictors, this training and testing process was performed 5 times. The average of the 5 AUC scores was used to evaluate the overall performance of the set of predictors. Among the 5 models in the best performed predictor set, the model with the top AUC score was picked as the final model.P < 0.05. Overall the ∆PSI values are correlated between CRC18P and CRC10P cohorts  of about 50% in NC samples and higher inclusion level (median 80–90%) in CRC samples in CRC18P and CRC10P  of about 10% in NC samples and higher inclusion level (median 40–50%) in CRC samples in CRC18P and CRC10P Fig. . In CRC1Exon v10 of the CD44 gene was found to be upregulated in CRC compared to NCs Fig. . Exon v1P-value< 0.001 and fold change> 2 , and MC vs. NC and CRC vs. NC in CRC18P and CRC10P data showing a consistent trend (same sign). 6, 3 and 25 events were identified from PSI_exon, PSI_junc5’ and PSI_junc3’ data, respectively  and MC (vs. NC) in CRC18P cohort , all other 8 genes showed significant splicing changes in tumors samples compared to normal tissues , in which high stroma content is associated with lower expression of the isoform that is more highly expressed in tumors  data for patients with different splicing profile for the 10 genes. For each junction usage data, we separated all of patients with OS data into two equal sized groups (high and low junction usage groups). We found that two junctions (COL6A3 E5-E6 junction and HKDC1 E1-E2 junction) showed a significant difference in OS  of the transcript  with alternative first exon regulation in CRC. Although some studies indicated that many of these genes are linked to tumorigenesis or metastasis, the detailed functions of these splicing events in CRC have not been reported. ARHGEF9 has been shown to play a role in tumor cell migration, invasion and metastasis by linking oncogene CHD1L and Cdc42 pathway in hepatocellular carcinoma (HCC) . HKDC1 eIn this study, AS of two genes (SERPINA1 and CALD1) were found to be associated with metastasis. For SERPINA1, although the two splice isoforms have the same start codon in exon 4, their different 5’UTR sequences may contribute to different RNA decay or protein translation of the gene. Elevated expression of SERPINA1 has been associated with the advanced stage, lymph node metastasis, poor prognosis and shorter overall survival in CRC  and gastThe logistic regression analysis generated 17 models using 2 or 3 junctions as predictors with an average AUC above 0.99. This indicated that the splicing events identified here can be used as potential diagnosis biomarkers in CRC. It remains unclear whether the splicing events can be detected in circulating tumor cells. Although doing RNA-seq using patient samples is not feasible in the clinical setting, easier experimental approaches, such as RT-qPCR, NanoString, or AmpliSeq can be designed using as few as two or three genes of the 8 genes identified in this study.We have identified two genes (COL6A3 and HKDC1) with AS associated with overall survival. The high gene expression of COL6A3 in stroma has been linked to poor overall survival in CRC , while hZong et al. have developed a prognostic predictors model using AS pattern of 13 genes identified from TCGA data . HoweverIn summary, we have identified significant AS of 7 genes in CRC and its metastasis tissue, and 3 genes with the stronger effect in metastatic tissue compared to normal tissue. Among them, the more inclusion of COL6A3-E6, CD44v8–10, and the more exclusion/decrease of HNF4A P1-driven isoform, CALD-E5b and E6 have been reported in CRC previously. Other 5 splicing events were newly identified in this study. Although the splicing events of these genes were not reported previously, their gene expression level was linked to tumorigenesis or prognosis of CRC (SERPINA1) or other cancer types . Targeting the kinase activity of CLK1 was suggested to be a therapeutic approach in cancer therapy. Considering the high consistence of AS events of the 9 genes identified in this study and previously studies, as well as their high relevance to cancer, it might suggest that the splicing signature of the 9 genes could serve as diagnosis, prognosis markers and facilitate drug development of CRC as well.Additional file 1: Figure S1. RNA-seq read coverage of CLK1 exon 4 and 5  for all samples in CRC18P and CRC10P datasets. E4 showed more inclusion in CRC and MC samples (labeled in red).Additional file 2: Figure S2. RNA-seq read coverage of COL6A3 exon 6 and 7  for all samples in CRC18P and CRC10P datasets. E6 showed more inclusion in CRC and MC samples (labeled in red).Additional file 3: Figure S3. RNA-seq read coverage of CD44 exon 5 to exon 16 for all samples in CRC18P and CRC10P datasets. Exons v8–10 showed more inclusion in CRC and MC samples (labeled in red).Additional file 4: Figure S4. RNA-seq read coverage of ARHGEF9 alternative first exons in CRC18P and CRC10P datasets. Exon E1a showed more exclusion/downregulation in CRC and MC samples (labeled in blue).Additional file 5: Figure S5. RNA-seq read coverage of HKDC1 alternative first exons in CRC18P and CRC10P datasets. Exons E1a, E2a and E3a showed more inclusion/upregulation in CRC and MC samples (labeled in red).Additional file 6: Figure S6. RNA-seq read coverage of CHEK1 alternative first exons in CRC18P and CRC10P datasets. Exon E1a showed more exclusion/downregulation in CRC and MC samples (labeled in blue).Additional file 7: Figure S7. RNA-seq read coverage of HNF4A alternative first exons in CRC18P and CRC10P datasets. Exon 1A showed more exclusion/downregulation in CRC and MC samples (labeled in blue).Additional file 8: Figure S8. RNA-seq read coverage of SERPINA1 exon 1 to exon 4 in CRC18P dataset. Exon E1a showed more exclusion/downregulation in MC samples (labeled in blue).Additional file 9: Figure S9. RNA-seq read coverage of CALD1 exon 5 to exon 7 in CRC18P dataset. Exons E5b and E6 showed more exclusion/downregulation in MC samples (labeled in blue).Additional file 10: Figure S10. Scatter plot of ΔPSI_exon values. Each dot is an exon. Only exons with significant regulation  in either x or y axis were shown. Black dots represent events that are significant in both x and y axis. A linear regression line is based on all dots in the plot. The total number of dots (n), Spearman’s correlation coefficient value (ρ) and P-value  are shown.Additional file 11: Figure S11. Boxplot of PSI values for genes in Figs. Additional file 12: Figure S12. Splicing events identified in this study confirmed by TCGA junction expression data. Boxplots of junction usage of 16 junctions of 9 genes in 51 normal tissue and 382 CRC or metastatic tissue (tumor). P-values are based on Wilcoxon Rank-Sum Test. Nominal P-values are shown and adjusted P-values using Bonferroni correction can be found in the supplementary Table Additional file 13: Figure S13. Logistic regression predicts sample type using junction usage data. (A) Spearman’s correlation coefficient of the junction usages of the TCGA data for the 8 genes. The 8 genes were separated to 3 groups (as indicated) based on the correlation values. (B) AUC scores of the ROC curves for the logistic regression using different number and combinations of the predictors. Average value of the five AUC scores for each formula were shown and the error bar is the standard deviation. Each AUC score was generated using randomly selected half of the data to train a logistic regression model and tested on the rest of the data.Additional file 14.


By using a flexible file format and distributed indexing, iMOKA can easily integrate data from multiple experiments and also reduces disk space requirements and identifies changes in transcript levels and single nucleotide variants. iMOKA is available at https://github.com/RitchieLabIGH/iMOKA and Zenodo 10.5281/zenodo.4008947.iMOKA (interactive multi-objective  Exploring such large matrices to find biologically relevant k-mers is intractable unless the analysis focuses only on a very small subset of the sequencing data [Studies of variation in gene expression have considerably advanced knowledge of disease etiology and classification –3. To ca samples . These krganisms , 8. Howeing data  or by using data .k-mer analysis), a novel approach and software that allows non-specialists to make use of k-mers to explore large amounts of mammalian sequencing data. This approach is agnostic of the type of sequencing data used, is not biased towards annotated genetic elements, and can analyze transcript levels and single nucleotide variations in one pass. Importantly, iMOKA is interactive; it allows the user to import and merge samples from different studies and tailor their exploration of k-mers to specific genomic elements of interest such as splicing events, mutations, or global gene expression. We tested iMOKA on four clinical datasets: the classification of breast cancer subtypes and response to chemotherapy of breast, ovarian cancer, and diffuse large B cell lymphoma (DLBCL). We find that iMOKA found features that are more accurate than classical bioinformatics approaches, takes up less space, uses less memory, has faster runtimes, and can be run on a computer cluster or on a laptop.Here we present iMOKA (interactive multi-objective k (default 31) [k-mer counts in a dedicated binary file using a custom prefix-suffix structure that drastically reduces the disk space requirements  or very short branches in the graph structure. Bifurcations or bubbles in these graphs generally arise from the existence of multiple sequence isoforms that differ by point mutations or alternative splicing events [k-mers in a manner that is fast and accurate.iMOKA imports sequencing files in FASTQ, FASTA, BAM format, or SRR identifiers via its user interface. It then counts the occurrences of all sequences of given length ault 31)  using thault 31)  in each rements “” sectiony break “” sectiong events . By combk-mer graphs to a reference genome to annotate them with known genomic features such as known RNA transcripts, point mutations, or mRNA splicing events. iMOKA provides a random forest classifier that uses filtered k-mer graphs as features (Supplementary methods) and provides the user with a classification model and a sorted list of k-mer graphs that were most used in the tree models and that are thus of higher interest , luminal B (LumB), HER2-enriched (HER2), and basal-like. The second was a cohort of 240 ovarian cancer patients where the objective was to predict response to chemotherapy. The third was a smaller cohort of 118 breast cancer patients where the objective was also to predict response to chemotherapy. The last was an even smaller cohort of 17 DLBCL patients divided according to their responsiveness to the chemotherapy.iMOKA uses a k-mer counts, percentage-spliced-in (PSI), transcripts per kilobase million (TPM), and sequencing counts. The two latter were measured and tested across annotated genes and transcripts separately. The algorithms we benchmarked were DESeq2 [k-mer counts; and Whippet [k-mer based methods HAWK [k-mers that could classify the input samples or, for the last two methods, were killed after 2 weeks of runtime on our computer cluster.In our benchmark, we included methods based on four different types of features which were e DESeq2 , edgeR [e DESeq2 , and lime DESeq2  for TPM  Whippet  for alteods HAWK , KOVER [ods HAWK , Kisspliods HAWK , and GECods HAWK  because In our benchmark, we compared the list of features output by each algorithm by using them in a random forest classifier and determining their out of bag scores (OOB score). The out of bag score tests how well each classifier performs without having to set aside a portion of the data specifically as a test set. It is as reliable as using a test set , 19 withFinally, for the largest dataset, the molecular classification of breast cancer, we performed a 5-fold cross validation of the entire iMOKA procedure and all other benchmarked algorithms, using 4/5 of the dataset for data reduction and creation of a random forest model and 1/5 of the dataset as the test set.k-mers output by iMOKA , luminal B (LumB), HER2-enriched (HER2), and basal-like . We bencWe additionally performed a 5-fold cross validation of the entire iMOKA procedure and all other benchmarked algorithms including feature reduction and model generation. The accuracies of the final models Fig. S show a ck-mers overlapping different types of events  [k-mer sequence in the iMOKA interface, we can visualize the representation of each k-mer in the 4 classes  , Forkhea1 (ESR1) , Forkhea1 (ESR1) , xenopus1 (ESR1) , and Mel1 (ESR1) . By clicIt is worth noting that iMOKA picked up 120 potential alternative splicing events. Amongst these were 4 extensively studied splicing isoforms   identifik-mer selected by iMOKA. The gene list is strongly enriched for breast cancer-associated genes and of genes associated with the function commonly dysregulated in cancer cells, such as cell cycle, cell division, and motility  gene. IDUA is responsible for the degradation of the mucopolysaccharides, heparan sulfate, and dermatan sulfate which modulate angiogenesis, cell invasion, metastasis, and inflammation [Our second benchmark was performed on a dataset of high-grade serous ovarian cancers taken from the TCGA_OV cohort . We incl% Table S. Again, apy Fig. a. The gaapy Fig. b. Via thammation  and impoammation . In agrek-mers with an individual accuracy between 70 and 83.8%  study  and consapy Fig. a.Fig. 4k-mers overlapping the gene TBC1D9, iMOKA discovers that the gene as a whole is differentially expressed between conditions but also discovers alternatively expressed introns . Finally, most methods that calculate differential expression are designed for relatively small cohorts and do not scale well in memory with large cohorts: DESeq2 and edgeR for example required additional RAM in order to analyze the differential expressed transcripts in the TCGA BRCA (TCGA_BC) analysis  , read counts, or splice site usage.Using k-mer based approaches such as iMOKA have the advantage of being portable; k-mer sequences will not change with new versions of the genome. This is crucial for the integration of omics data with other clinical data such as imaging or patient file records.By creating a reliable, cross-platform user interface, iMOKA allows non-specialists to leverage the predictive power of our approach in a manner that is fast and accurate. In addition, iMOKA uses a flexible data structure that allows the easy integration of new samples and uses only a fraction of the disk space required for stocking compressed sequencing files. In addition, The input data can be given as SRR identifier, BAM, FASTA, or FASTQ files. In the first and second cases, the corresponding FASTQ files are automatically generated using sra-tools’ fastq-dump  and SAMtk-mers of the length chosen by the user (default k = 31). Its output is converted into a sorted binary file optimized for the following steps of iMOKA and a JSON file containing the metadata information.For each sample, KMC3  is used The binary file is divided into two parts: a suffix portion, containing the nucleotidic sequence and the relative count, and a prefix portion, which contains the prefixes and the positions of the respective suffixes.The length of the prefix is defined using the following formula, an adaptation from :\documep is the prefix size and t is the total number of different k-mers for the current sample.where k-mer count file of each sample in the analysis. The JSON file also stores the sum of all the k-mer counts that will be used as a normalization factor:The input to the feature reduction step is a JSON file containing the name, group, and localization of the sorted binary whereNij is the normalized count of the ith k-mer of the sample jCij is the raw count of the ith k-mer of the sample jTj is the sum of the counts of all the k-mers of the sample jRF is a rescaling factor, used to increase the value of all the normalized values and avoid computational problems related to precision. By default, RF = 1e9Each thread starts the creation of the matrix and the reduction step in parallel, using an OpenMP  implemenwhereT is the total number of threads availableKt is the first k-mer analyzed by the thread t (from 0 to T excluded) considering all the possible ordered combination from 0 to 4kk is the length of the k-mers (default 31)k-mer analyzed by each thread is Kt + 1 – 1. For example, with 2 threads (T = 2) and k = 31, the first k-mers for each threads will be:The last Finally, the buffer size reserved for each sample is dependent on the number of parallel processes, the number of total samples, and the available memory reserved:whereBuff is the length of the bufferavail is the available RAM in GiB, defined by the user using the environmental variable “IMOKA_MAX_MEM_GB”RAMN is the number of samples in the matrixT is the total number of threads availableα is a factor representing the GiB occupied by 1000 k-mers, approximated to 0.011k-mer is calculated using the NaiveBayesClassifier method implemented in the library mlpack [k-mer, the samples are randomly divided into test and training sets, with an equal number of samples for each group scaled to the smallest one:The accuracy of each y mlpack . For eacwhere:nmin is the dimension of the smallest groupntest and ntrain are respectively the dimension of the test and training setsptest is the test fraction, 0.25 by defaultk-mer count) xk at a time, the NaiveBayesClassifier class computes for each label yj:Using one feature  in at least one of the pairwise comparisons are saved in a text file, along with the accuracy values.The k-mer that runs in parallel to the Bayesian filter (BF).In order to speed up the process of accuracy estimation, we introduced an additional filter based on the Shannon entropy  of the ck-mer k and its counts in the different samples Ck = , we compute its entropy value Hk as follows:For a given Hthr, tuned according to the lowest entropy detected in the previous batch of k-mers that passed the accuracy filter (Hmin).The filter uses an adaptive threshold, Hthr = 0, so all the k-mers in the first batch are evaluated by the BF and the lowest entropy is saved as Hmin. During the analysis, Hthr is updated when more than Eup  passes the BF. The first assignment is always:Initially Subsequently:a1» a2 ensure that the new threshold is not set too close to the minimum Hmin.The adjustment parameters k-mers required to update the threshold (Eup) increases by 30 at each update in order to reduce the number of computations and reduce the fluctuations of the threshold. Figure Sk-mers from the previously defined datasets showing that the number of k-mer would have been rejected by the entropy filter but would have had an accuracy higher than 60% are rare and that the adaptive threshold is able to find a mild cutoff that can save more than 50% of the computation, like in TCGA BC, or can let the BF evaluate most of the k-mers in case of difficult datasets, like in BEAUTY.The number of k-mers that successfully passed the reduction are used as nodes in a graph. A link between two nodes is created if they overlap by a minimum number of nucleotides defined by parameter w (default = 1). This parameter can be increased if the user notices multiple small sequences in the final result, caused usually by k-mers with accuracy close to the given threshold arguments -T and -t, respectively the minimum accuracy required to consider a k-mer in the graph construction and the minimum accuracy required to generate a sequence from a graph.The iMOKA then prunes short bifurcations in the graph where there is only one node following the bifurcation. If there are multiple sequential bifurcations, then the branch with the lowest accuracy is removed.The accuracy values are then rescaled from 0 to 100 for each pairwise comparison in order to normalize the accuracy values and favor the features that are able to classify pairs of classes that are more difficult to separate.k-mers preceding the bifurcation are added to each of those new sequences.Since each bifurcation could correspond to a biological event such as a point mutation or splicing isoform, each separate path that results from a bifurcation will be kept as a separate sequence for downstream analysis using a depth-first graph traversal approach. When the traversal meets a bifurcation, the branch having the most similar accuracies values to the bifurcating node is kept in the current sequence and others will generate new sequences. Furthermore, to maintain the context of the bifurcations, three The sequences generated from the graphs can be aligned to a reference genome. Currently, iMOKA supports any aligner that provides an output in SAM or pslx format and uses the information given in the JSON configuration file “mapper-config” (-m argument) to align and to retrieve the annotation file, in GTF format. In this manuscript, we used gmap v. 2019-05-12 with the human genome GRCh38 and the GENCODE annotation v29, excluding from the file the entries with the transcript type “retained_intron”.k-mer graphs are aligned, iMOKA identifies the following “alignment derived features” (ADF):Mutations, insertions, deletions, and clipping are identified by the letters “M”, “I”, “D” and “S,” respectively, in the alignment’s CIGAR string.k-mer graph is split across exons.Alternative splice sites are identified when a k-mer graphs. Since regions with sequence variations not associated with the classes generate holes in the graphs reducing the portion of the transcripts that generate useful k-mers, a higher threshold might result in classifying DE event as general “gene” event, that is, the best k-mer in a gene.Differential expression (DE) is identified if 50% (set by parameter d) of an annotated transcript is covered by the k-mer graphs.Alternative intronic events are identified if 50% (set by parameter d) of an annotated intron is covered by the k-mer graph maps to the genome but not to any annotated transcript.Intergenic events are identified if the k-mer graphs that have no mapping or map to multiple sites.Unmapped or multimapped events are created for those Once the k-mer per event, the one with the highest accuracy score. Table SiMOKA will preserve one The feature reduction component of iMOKA is implemented in C++ using the following libraries: MLpack , armadilTranscript abundance was computed using Salmon  version In order to compare the same number of features extracted by each pipeline, we used the sklearn method SelectFromModel to select 20 features using a decision tree classifier (DTC) trained with all the samples and all the features in order to identify twenty features that, in combination, can be good classifiers. Using an increasing number of features, from 2 to 20, we trained multiple RandomForestClassifier to retrieve the out of the box scores.We also performed a 5-fold cross validation of the largest and better characterized dataset, TCGA BRCA, to evaluate the accuracy of a model on unseen data. For each fold, we performed the feature reduction using only the training in each method. The final list of features is reduced similarly as for the oob score determination and the balanced accuracy score is estimated for the test set.Additional file 1. TCGA_BC_aggregated.json - iMOKA results for the dataset TCGA_BC.Additional file 2. TCGA_OV_aggregated.json - iMOKA results for the dataset TCGA_OV.Additional file 3. BEAUTY_aggregated.json - iMOKA results for the dataset BEAUTY.Additional file 4. DLBCL_aggregated.json - iMOKA results for the dataset DLBCL.Additional file 5. GO - folder containing the DAVID gene ontology result for each dataset.Additional file 6. iMOKA_supplementary.docx - Supplementary materials.Additional file 7. Supplementary Figures S1-S7.Additional file 8. Review history.


Can immunologic heterogeneity be identified in histologically stable kidney allografts?This prognostic study used 28 public kidney transplant data sets with 2273 kidney tissue to develop and validate an unbiased, 6-gene and 5-cell-type transcriptional Instability Score to provide histology-independent reclassification of human transplant samples. Using this score, 46% of histologically stable samples were found to have molecular evidence of rejection, which was validated by an independent cohort that showed undiagnosed graft rejection and poor projected graft function survival.These findings suggest that the Instability Score could provide an important adjunct for comprehensive and highly quantitative phenotyping of protocol kidney transplant biopsy samples and could be integrated into clinical care for accurate estimation of subsequent patient clinical outcomes. This prognostic study uses microarray gene expression data from the National Center for Biotechnology Information Gene Expression Omnibus to assess whether immunologic heterogeneity can be identified in histologically stable kidney allografts. Clinical decision and immunosuppression dosing in kidney transplantation rely on transplant biopsy tissue histology even though histology has low specificity, sensitivity, and reproducibility for rejection diagnosis. The inclusion of stable allografts in mechanistic and clinical studies is vital to provide a normal, noninjured comparative group for all interrogative studies on understanding allograft injury.To refine the definition of a stable allograft as one that is clinically, histologically, and molecularly quiescent using publicly available transcriptomics data.In this prognostic study, the National Center for Biotechnology Information Gene Expression Omnibus was used to search for microarray gene expression data from kidney transplant tissues, resulting in 38 studies from January 1, 2017, to December 31, 2018. The diagnostic annotations included 510 acute rejection (AR) samples, 1154 histologically stable (hSTA) samples, and 609 normal samples. Raw fluorescence intensity data were downloaded and preprocessed followed by data set merging and batch correction.The primary measure was area under the receiver operating characteristics curve from a set of feature selected genes and cell types for distinguishing AR from normal kidney tissue.KLF4, CENPJ, KLF2, PPP1R15A, FOSB, TNFAIP3)  and 5 immune cell types   that were combined into 1 composite Instability Score (InstaScore) . The InstaScore was applied to the hSTA samples: 626 of 1154 (54%) were found to be immune quiescent and redefined as histologically and molecularly stable (hSTA/mSTA); 528 of 1154 (46%) were found to have molecular evidence of rejection (hSTA/mAR) and should not have been classified as stable allografts. The validation on an independent cohort of 6 months of protocol biopsy samples in December 2019 showed that hSTA/mAR samples had a significant change in graft function  and graft loss at 5-year follow-up (r = 0.17). A drop by 10 mL/min/1.73m2 in estimated glomerular filtration rate was estimated as a threshold in allograft transitioning from hSTA/mSTA to hSTA/mAR.Within the 28 data sets, the feature selection procedure identified a set of 6 genes (The results of this prognostic study suggest that the InstaScore could provide an important adjunct for comprehensive and highly quantitative phenotyping of protocol kidney transplant biopsy samples and could be integrated into clinical care for accurate estimation of subsequent patient clinical outcomes. Furthermore, there is a failure to uncover the molecular biologic diversity in the histologic definition of a stable allograft. This bias is further amplified during interrogation of kidney transplant biopsy samples across different pathologists and investigators in public data sets.Breakthroughs in surgical approaches and development of newer generations of immunosuppressive drugs have resulted in reduction of clinical allograft acute rejection (AR) and improvements in life expectancy and quality of life for kidney transplant recipients.14 in Gene Expression Omnibus,15 a public genomics data repository, to investigate the molecular diversity of stable allografts.19 We proposed that for accurate definition of a stable allograft, the sample must be associated with (1) stable clinical function, (2) normal kidney histology with AR , and (3) absence of a transcriptional signature of AR (molecularly stable [mSTA]). Recognizing the previously discussed variabilities in allograft histology interpretation, we expected that some of the labeled stable samples in these data sets (that only use the first 2 criteria listed above) would have inherent molecular variability. Our analysis has resulted in the generation of a histology-independent composite gene and cell-specific computational Instability Score (the InstaScore) to discern molecular rejection in hSTA allografts, classifying clinically stable (truth) samples as histologically and molecularly stable (hSTA/mSTA) or clinically and histologically stable (untruth) samples with molecular rejection (hSTA/mAR). Thus, our prognostic study proposes an approach to recognize immunologic heterogeneity in hSTA kidney allografts.In this study, we have aggregated, to our knowledge, the largest public data set for human kidney transplantation to date: 2273 kidney tissue microarray samples from 28 publicly available normal and transplant kidney tissue data sets15 for biopsy kidney transplant samples from January 1, 2017, to December 31, 2018. Any public, deidentified data available as open access were not subject to local institutional review board requirements or patient consent as allowed under the Common Rule. For any private data used, we obtained the approval of the institutional review board of the University of California, San Francisco, and written informed consent from all patients. After stringent data quality control procedures (eMethods in the PRISMA), Standards for Reporting of Diagnostic Accuracy (STARD), and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines.For this prognostic study, we carried out a comprehensive search for publicly available microarray data at the National Center for Biotechnology Information Gene Expression Omnibus database20 (R Foundation) .To identify differentially expressed genes, we used the Significance Analysis of Microarrays,26 to perform functional annotations for the significantly upregulated and downregulated genes with a false discovery rate less than 0.05. For the gene network analysis, we used the STRING protein-protein association networks database.27We leveraged the Gene Ontology database using the gene set enrichment analysis with clusterProfiler28 xCell leverages gene expression data from microarray or RNA-sequence experiments to estimate the presence of up to 64 immune and stromal cell types in a mixture. We focused on 34 immune-related and 11 nonimmune cell types .To estimate the presence of certain cell types in biopsy samples, we used the cell type enrichment tool xCell.r > 0.75 × max[r]). After, we applied the recursive feature elimination technique with the random forest (RF) model using caret.29 We used a 5-fold cross-validation technique with 100 repeats and benchmarked a model by computing the area under the receiver operating characteristic (AUROC) curve, and the results were reported with both AUROC and precision-recall area under the curve (AUCPR). To minimize possible bias of the data random split and to avoid the model overfitting, the tolerance of 1% to the feature selection mechanism was introduced, ie, the algorithm chose a model with a smaller number of features that performed no worse than 99% from the best model. A final set of selected features was benchmarked by applying the RF model to the testing set. The R package feseR30 was adopted and modified for the implementation of the parallel computations.In order to select the most important features in distinguishing AR vs normal samples, first, the data were split into training and testing sets in the ratio 80:20. All feature selection steps were performed on the training set with benchmarking on the testing set. Among the significant features, we searched for features correlated with the outcome   and 5 immune cell types   that were combined into 1 composite InstaScore . We leveraged all currently publicly available kidney biopsy microarray data  between classes  enriched in AR and 12 cell types  that were enriched in normal kidneys (P < .001): one was mostly enriched in lymphocytes, NK cells, and macrophages, and the other had minimal lymphocyte activation and may have represented temporal differences in rejection evolution or recovery. We observed that B cells, dendritic cells, macrophages, and T cells formed cell type–specific subclusters that suggested the coordinated activation of immune cells in the kidney tissues. These results are in agreement with previous observations33 that have shown AR subphenotypical splits by gene expression and cell type. Unsupervised clustering of hSTA along with AR and normal samples exposed their heterogeneity, hinting that some hSTA samples have molecular signal closer to AR samples   to 5 cell types from the original set of 37 significantly enriched cell types: CD4+ Tcm, CD4+ Tem, CD8+ Tem, NK cells, and TH1 cells, with CD4+ Tcm having the largest effect size in this model   meta-analysis study of transplant rejectionA generated RF classification model for these 6 genes and 5 cell types, internally validated using 5-fold cross-validation with 100 repeats, obtained an AUROC of 0.98  for the genes alone and an AUROC of 0.92  for the cell types for identification of a tissue sample with histologically confirmed AR A. We fur10We then applied the InstaScore to the 1154 transplant samples that were identified by pathologists in each of the data sets as hSTA, classifying samples as more similar to normal kidneys or as more similar to the rejected kidney allograft group (mAR); hSTA/mSTA identified samples with molecular and histologic evidence of no rejection, and hSTA/mAR identified histologically stable allografts with transcriptional evidence of ongoing molecular rejection. The InstaScore identified 528 hSTA grafts (46%) in this study as having mAR A, a miscP < .001; P < .001; We represented the scores for each sample as a scatterplot in 34 with transcriptional data on serial protocol kidney transplant biopsy samples at 0, 3, 6, 12, and 24 months36 and with longitudinal functional outcomes up to 5 years after initial engraftment. We tested the correlation association of the locked InstaScore with the change in estimated glomerular filtration rate (eGFR) and graft loss events over this time period and found high correlation values for cell type infiltration and activation model with delta eGFR  and graft loss events .In order to assess the functional relevance of the InstaScore by gene expression and cell types, we explored its clinical use in an independent microarray data set from 67 unique patients with hSTA grafts  from a randomized clinical triallta eGFR A (r = 0.P < .001). Given these results, it appears that the InstaScore on the 6-month protocol biopsy samples could differentiate patients more likely to have progressive graft injury and decline in graft function over time, even though the 6-month biopsy histology findings, serum creatinine, or donor-specific antibodies cannot provide the same discriminatory information.Using the predicted hSTA subphenotypes, we estimated a delta eGFR separating threshold of −10 at 5-year follow-up  and those that are most likely incorrectly annotated as stable but have molecular injury similar to AR (hSTA/mAR).Tissue histology is indispensable for the diagnosis of allograft pathology, but its recognized limitations have resulted in the incorporation of data inputs from transcriptional and proteomic studies. Here, we present, to our knowledge, the first unsupervised transcriptional and cell-state framework to map and rephenotype human kidney allografts with undiagnosed graft dysfunction. Unlike other published studies, by others35 as it identified hSTA/mAR 6-month protocol biopsy samples that had a higher risk of progressive graft injury and failure at 5 years posttransplant.Approximately half (46%) of the biopsy samples were wrongly annotated as stable and reclassified as hSTA/mAR by the InstaScore; these samples were found to be scattered across multiple data sets, supporting that their presence is not due to failure of histologic characterization at any particular transplant program, and highlighting the failure of histology to detect relevant molecular inflammation. The InstaScore was independently validated for functional relevance,KLF4, CENPJ, KLF2, PPP1R15A, FOSB, and TNFAIP3, in the InstaScore are biologically relevant in the immune response and activation and innate immunity. KLF2, KLF4, and TNFAIP3 regulate kidney injury.44KLF2 is vasoprotective, and KLF4 is renoprotective; both genes are highly expressed in the endothelium47 and are associated with endothelial ischemia reperfusion injury in AR.48TNFAIP3 has antiapoptotic and anti-inflammatory functions and expression in endothelial, myeloid, and infiltrating T cells, which results in adverse clinical outcomes in AR.51CENPJ functions as a transcriptional coactivator in STAT 5 signaling and tumor necrosis factor–induced NF-κB–mediated transcription,53 both of which are central regulators of inflammation. The phosphatase PPP1R15A is only expressed in stressed cells and negatively regulates acute kidney injury via type 1 interferon;54 clonal expansion; and memory T-cell, plasma cell differentiation, and enhanced B-cell responses.55FOSB expression is associated with the progression of kidney disease.56 Thus, all InstaScore genes are crucial for endothelial cell integrity, and T-cell activation, and have functional relevance to the kidney and rejection.58The 6 feature selected genes, + Tcm, CD4+ Tem, CD8+ Tem, NK, and TH1, also relate to rejection biology,59 together with macrophages, NK cells, and B cells.64 In the immunologic response to the allograft, T cells terminally differentiate and divide into Tcm cells and CD8+ and CD4+ Tem cells,65 which produce interferon-γ, IL-4, and IL-5 and cytotoxic molecules like granzyme, granulysin, and perforin.67 CD4+ Tcm had the largest effect size in the InstaScore, likely because Tcm cells are characterized by slow effector function and reactive memory and increased response to repeat antigenic stimulation.68 In the hSTA/mAR grafts, these cells are primed to differentiate into Tem with low levels of antigen recognition, such as with varying exposure to baseline immunosuppression.69 Hence, identification of the hSTA/mAR phenotype in an otherwise clinically and histologically stable allograft may be of critical importance to triage allografts at greater risk of accelerated temporal immune injury.The 5 feature selected cell types, CD4Given the design of the study, there are a few inherent limitations. First, the publicly available data had limited access to clinical and demographic reports, which could potentially be valuable in incorporation with InstaScore. Second, batch effects had to be controlled for, for which we conducted comparisons of multiple normalizing methods. We chose the RF model to capture possible nonlinear feature interactions to identify the best feature set, although other more complex  or less complex  methods could also be considered as optional methods. Although the study is based on bulk microarray data, more precise measurement techniques, eg, single-cell RNA sequence, might be used to better capture finer changes in gene expression and cell composition, provide additional validation to the results and, in a future study, refine InstaScore. This may give InstaScore the ability to recognize different types of rejection, which can be identified at the molecular level, long before they can be detected at the histology.This prognostic study leverages supervised machine learning on the largest bulk transcriptional human kidney and kidney transplant data set to improve kidney allograft sample phenotyping beyond the capabilities of tissue histology alone. In this study, the InstaScore revealed a level of biologic diversity within the classification of a stable graft not shown by histology alone; based on these findings, the InstaScore may provide an immune map to help refine our understanding of diverse graft functional states. The InstaScore provides a new tool to apply polymerase chain reaction–based analysis of the minimal gene set to kidney allograft biopsy samples embedded in formalin frozen paraffin to identify hSTA/mAR grafts at greater risk for subsequent overt rejection and allograft damage. These patients may benefit from proactive immunosuppression adjustments to reduce molecular inflammation, preserve allograft function, and improve allograft survival.