Sample size calculation and power estimation are essential components of experimental designs in biomedical research. It is very challenging to estimate power for RNA-Seq differential expression under complex experimental designs. Moreover, the dependency among genes should be taken into account in order to obtain accurate results.In this paper, we propose a simulation based procedure for power estimation using the negative binomial distribution and assuming a generalized linear model (at the gene level) that considers the dependence between gene expression level and its variance (dispersion) and also allows equal or unequal dispersion across conditions. We compared the performance of both Wald test and likelihood ratio test under different scenarios. The null distribution of the test statistics was simulated for the desired false positive control to avoid excess false positives with the usage of an asymptotic chi-square distribution. We applied this method to the TCGA breast cancer data set.We provide a framework for power estimation of RNA-Seq data. The proposed procedure is able to properly control the false positive error rate at the nominal level.The online version of this article (doi:10.1186/s12859-017-1648-2) contains supplementary material, which is available to authorized users. Discovering differential expression has been the main focus of many biological experiments and many patient cohort studies for several decades. Since the invention of microarray chips twenty years ago, a huge amount of data has been generated by profiling thousands of genes in various cell lines, model organisms, and human samples. Recently, RNA-Seq technology became the replacement of array technology because of its ability to not only quantify the transcriptome but also detect gene isoforms, novel transcripts, and gene fusion –3. SimiledgeR, which provides an exact test for two group comparisons initially and then was expanded to allow multifactor designs by a generalized linear model. Additionally, Love et al. [DESeq2 for differential expression analysis, which provides shrinkage estimators for both log fold change and dispersion by imposing a hierarchical model on them.RNA-Seq studies generate count based data. Several earlier published papers used Poisson distribution to model the count data –6. Due te et al.  developeRNAseqPS [RnaSeqSampleSize. Other studies have been published for sample size calculation and power estimation at a data set level by evaluating the proportion of true discoveries. Shyr and Li [PROPER method by Wu et al. [For testing differential expression with RNA-Seq experiments, several studies have attempted to provide sample size calculation and power estimation at a single gene level in the recent literature. Fang and Cui  introducRNAseqPS  and an Rr and Li  proposedr and Li  simulateu et al.  is a proThe above-mentioned literature on RNA-Seq sample size calculation and power estimation employed common analysis approaches, such as edgeR or DESeq2, that assume the negative binomial distribution. However, all these NB-based approaches have resulted in an inflated type I error rate as reported in several papers –17. AccuX with mean μ and dispersion ϕ is denoted as NB. It has variance μ+μ2ϕ and probability mass function as follows: A negative binomial random variable x=0,1,2,⋯ ;μ>0;ϕ>0.ϕ follows a log-Normal prior distribution with mean as a function of μ. The dispersion’s functional trend is modeled as Love et al.  assume tTo estimate this functional form, gene-wise dispersion estimators were regressed against the means of the normalized counts. This approach provides gene-wise shrinkage estimators of the dispersion parameter by assuming the mean-dispersion dependence for all genes and shows adequate power for detecting differential expression especially in small sample size experiments.γ to denote the fold ratio of a gene between two biological conditions. We are interested in testing the hypothesis H0:γ=1 vs. hypothesis H1:γ≠1. Let Without loss of generality, we use μ under Θ0 is According to Rettiganti and Nagaraja , the maxΘ, the MLE of μ is γ is ϕ is estimated by numerically maximizing the likelihood.While under γ with H0:log(γ)=0 vs. H1:log(γ)≠0 is given by A Wald test for testing log transformed α will be the PCER for a single gene or at the marginal level.With thousands of genes tested in an RNA-Seq study, multiple comparison adjustment is necessary. While the Bonferroni method for controlling the family-wise error rate  is very conservative, a less conservative procedure, named the extended interpretation of the Bonferroni method, for controlling the mean number of false positives can be used for multiplicity adjustment . In otheα)th percentile from the null test statistics will be used as a significance cutoff for testing under the alternative hypothesis by comparing the test statistics with this percentile cutoff value.Inferences on the Wald test and the LRT typically rely on the chi-square distribution by asymptotic theory for large sample sizes. But this may lead to liberal results for small sample sizes since asymptotic theory may not work as expected. To address this issue for small sample sizes, the simulation based test by Rettiganti and Nagaraja  is used n; mean expression μ; fold ratio between conditions γ, nominal false positive error rate α, number of simulations T.Specify all input parameters: sample size per condition Estimate the mean-dispersion functional form using pilot data set or specify an assumed functional form.ϕ using Eq. μ.Calculate dispersion NBT times under both null and alternative hypotheses using the input parameters.Simulate count data from Fit NB model and obtain the test statistics  under the null hypothesis.α)th percentile as the significance cutoff.Calculate 100 under the alternative hypothesis.Calculate power for the specified input parameters.n, mean expression μ of control group, treatment-to-control fold ratio γ, and nominal false positive error rate α. Simulation settings were ϕ was calculated for each μ from the mean-dispersion functional form ϕ=0.26+3.65/μ estimated from an unpublished canine thyroid RNA-Seq data set  and a high-dependency (ϕ=0.032+3.65/μ) were also considered in the simulations as shown in Fig. α)th percentile from the null Wald and LRT statistics.Count data were simulated from a negative binomial distribution under two experimental conditions (e.g. control vs. treatment) with equal dispersion parameters or unequal dispersion parameters ratio of 1.5) between conditions. The parameters needed to calculate power of a single gene or marginal level are sample size per condition .5 between=5,10,20,40 with mean expression μ=20. When sample size increases, the distribution of either Wald or LRT statistics converges toward the chi-square distribution with 1 degree of freedom with a faster convergence for the LRT. The discrepancy is quite large when the sample size is small. Figure Figures μ=100 for the Wald test and the LRT at α=0.001. In both plots, power increases with larger sample sizes and larger absolute fold changes. Figure γ=2 under the alternative hypothesis and α=0.001. In both plots, power increases with larger sample sizes and larger mean expression levels. The Wald test and the LRT have similar power at different parameter values. Compared to the results for the medium-dependency functional form, both low-dependency and high-dependency functional forms have similar critical values and false positive error rates, but power estimation is lower(higher) for low(high)-dependency.  vs. III-IV (238 samples) when fitting the Many published methods on identifying differentially expressed genes are based on the negative binomial distribution, and the inference mainly relies on asymptotic theory which is biased for small sample sizes. Several studies by Leng et al. , Lund etIn all current published methods on differential expression detection and power estimation, the dispersion parameter is assumed equal across conditions. Under this assumption, the power will be misestimated if dispersion values are very different across conditions. Therefore in the simulations we allowed the dispersion parameter to be equal or unequal across conditions to achieve accurate power. When the dispersion parameter is assumed equal, the exact test method by Li et al.  can be uThe proposed work not only can be applied to multiple groups and multiple factor designs through generalized linear models, it can also be extended to the data set level. In this case, the null distribution of the test statistics could be simulated for each gene or for a group of genes with similar expression profile for a proper control of the false positive error rate.With the emergence of RNA-Seq technology in recent years, RNA-Seq experiments have been widely used as an alternative to microarrays in biomedical research. Due to different data types, data analysis and power estimation are also different. New methods on sample size calculations and power estimation using the negative binomial distribution have already been proposed for this new technology. To overcome some of the limitations in current methods, we provide a framework for power estimation of RNA-Seq experiments by proposing a simulation based procedure, which provides a proper false positive control and can be applied in generalized linear model settings.


Driven by the recent advances of next generation sequencing (NGS) technologies and an urgent need to decode complex human diseases, a multitude of large-scale studies were conducted recently that have resulted in an unprecedented volume of whole transcriptome sequencing (RNA-seq) data, such as the Genotype Tissue Expression project (GTEx) and The Cancer Genome Atlas (TCGA). While these data offer new opportunities to identify the mechanisms underlying disease, the comparison of data from different sources remains challenging, due to differences in sample and data processing. Here, we developed a pipeline that processes and unifies RNA-seq data from different studies, which includes uniform realignment, gene expression quantification, and batch effect removal. We find that uniform alignment and quantification is not sufficient when combining RNA-seq data from different sources and that the removal of other batch effects is essential to facilitate data comparison. We have processed data from GTEx and TCGA and successfully corrected for study-specific biases, enabling comparative analysis between TCGA and GTEx. The normalized datasets are available for download on figshare. Large-scale sequencing studies have recently generated a great wealth of RNA-seq data. For example, The Cancer Genome Atlas (TCGA) has quantified gene expression levels in >8000 samples from >30 cancer types. On a similar scale, the Genotype Tissue Expression (GTEx) projectThese resources offer a unique opportunity to gain better insight into complex human diseases. However, the integrative analysis of these data across studies poses great challenges, due to differences in sample handling and processing, such as sequencing platform and chemistry, personnel, details in the analysis pipeline, etc. For example, the RNA-seq expression levels of the majority of genes quantified are in the range of 4-10  for TCGA, and 0-4 (log2 of RPKM) for GTEx , a conse3–10. However, these databases or pipelines either directly incorporated expression data from the literature, retaining unwanted batch effects in the data8, or only combined and reanalyzed samples from smaller studies, hence, not taking advantage of the power provided by the recent large data sets10. A recently published pipeline, the Toil RNA-seq Pipeline11, attempts to unify RNA-seq data from different sources by uniformly processing raw sequencing reads. However, Toil does not remove batch effects that are introduced by sources other than the differences in read alignment and quantification. To take full advantage of the large volume of available RNA-seq data, an integrative RNA-seq resource is necessary.To facilitate research on abnormal gene expression in human diseases, a variety of databases and pipelines have been developed to combine RNA-seq from different studiesHere, we developed a pipeline for processing and unifying RNA-seq data from different studies. By unifying data from GTEx and TCGA, we provide reference expression levels across the human body for comparison with the expression levels found in human cancer. Our method removes batch effects by uniformly reprocessing RNA-seq data. Specifically, we used raw sequencing reads of the RNA-seq samples downloaded from GTEx and TCGA, realigned them, re-quantified gene expression, and then removed biases specific to each study.https://cghub.ucsc.edu). When FASTQ files were not available, e.g., for stomach adenocarcinoma, we downloaded aligned sequence reads (in BAM format) and extracted reads from BAM files with the Java program ubu.jar (https://github.com/mozack/ubu) before processing samples using our pipeline. GTEx samples were downloaded from the Database of Genotypes and Phenotypes , which hosts >9,000 RNA-seq samples (in SRA format) for the GTEx study.Raw paired-end reads of the RNA-seq samples for the TCGA project were retrieved from the Cancer Genomics Hub , using recommended parameters, e.g., ‘—outFilterType BySJout’ and ‘—outFilterMultimapNmax 20’, etc., which are also standard options of the ENCODE project for long RNA-seq pipeline. Samples with alignment rates less than 40% were excluded from further analysis.We employed STAR alignerhttp://picard.sourceforge.net/index.shtml), RseQC13, and mRIN14 were used to evaluate sample quality. RNA degradation, as detected by mRNA, was present in some GTEx and TCGA samples. Since degradation can bias expression level measurements and cause data misinterpretation, we decided to exclude samples with evidence for degradation. To determine an appropriate degradation cutoff for mRIN, we used prostate cancer samples from the TCGA project, which had undergone extensive pathological, analytical, and quality control review and which had been shown to include a significant portion of degraded samples15. We used -0.11 as the degradation threshold for mRIN: samples with mRIN<−0.11 were regarded as degraded and, thus, excluded from further analysis.The software tools FastQC, Picard . To verify mRIN’s performance on other tissues, we manually examined coverage uniformity over gene bodies for other tissues using the tool RseQC16 to quantify gene expression. The program ‘rsem-calculate-expression’ in the RSEM package requires strand specificity of the RNA-seq sample, which is estimated using RseQC13.When running STAR, we specified an option ‘—quantMode TranscriptomeSAM’ to make STAR output a file, Aligned.toTranscriptome.out.bam, which contains alignments translated into transcript coordinates. This file was then used with RSEM17 to generate integer-based read counts. Overall, the output of FeatureCounts was highly consistent with that of RSEM (Spearman correlation > 0.95). However, for genes with multi-mapping reads , FeatureCounts differs from RSEM and tends to underestimate expression levels in comparison with RSEM (because it discards multi-mapping reads). For example, the transcript of the PGA3 gene, which encodes the human pepsinogen A enzyme, which is highly abundant in the stomach, is identical to the transcripts of two other genes, PGA4 and PGA5. Its measurement in stomach by FeatureCounts (in default settings) is generally lower than that by RSEM . As indicated in We ran ComBat in the R package SVAseq2-transformed the sample-gene matrix. Next, we utilized the R function ‘prcomp’ (with the ‘center’ option set to TRUE) to perform principal component analysis. The two-dimensional PCA plot was created using the R function ‘autoplot’.To perform principal component analysis, we first remove genes with invariant expression levels and then logFor hierarchical clustering of expression data, we used the R function Heatmap.3 using default parameters  as well as the 1000 most variable genes in the data matrix.https://github.com/mskcc/RNAseqDB). The versions of the tools, e.g., STAR and RSEM, are described in a README file at https://github.com/mskcc/RNAseqDB/blob/master/README.md.The detailed parameters we used to run STAR, RSEM and other tools and the codes of our pipeline are available at GitHub .The maximum likelihood gene expression levels computed using RSEM, i.e., the expected_count in RSEM’s output, are in Data Citation 1. This dataset includes 52 data files, each being a sample-gene matrix of a certain tissue type  in RSEM’s output are in Data Citation 2. This dataset (of data files) was quantile normalized, but not corrected for batch effects.The normalized gene expression levels (FPKM) are in Data Citation 3. This dataset was not only quantile normalized, but was corrected for batch effects (using ComBat).To allow proper batch bias correction, we processed only samples from tissues that were studied by both GTEx and TCGA . Tissues15) and were excluded from further analysis. We also discarded samples with low alignment rates and samples not used in the final GTEx study, resulting in a total of 9109 (89%) high-quality samples for further analysis.We downloaded and processed raw paired-end RNA-seq data from 10,366 samples, including 2,790 from GTEx and 7,576 from the TCGA project . 831 sam18 to correct for non-biological variation accounting for unwanted differences between GTEx and TCGA samples of a particular tissue type (see Methods).To correct for batch biases, we first created a sample-gene matrix for each tissue-tumor pair by merging gene expression levels of the corresponding GTEx and TCGA samples. Regardless of the actual batch that a sample belonged to in an RNA-seq experiment, we treated all GTEx samples as one batch and TCGA samples as another. Then, we ran ComBatTo examine how well our pipeline was able to correct study-specific batch effects, we systematically compared the effects of uniform realignment, expression quantification, and batch effect correction for three tissues: bladder, prostate and thyroid. When using expression levels reported by the TCGA and GTEx projects, even after applying upper-quartile normalization to bring expression levels into comparable ranges , samplesHowever, uniform realignment and expression quantification using our pipeline did not fully resolve these differences; while the first principal component was now the tissue, the second principal component was still defined by the source , indicat18 (see Methods), which successfully corrected our example data and resulted in clustering by tissue type . We found that batch effect removal by itself is not sufficient, and that the combination of uniform processing of sequencing reads followed by additional batch effect removal is required to make data from the TCGA and GTEx projects comparable . We valiOur results demonstrate that uniform realignment and expression quantification, together with explicit correction for study-specific biases, are not only effective, but also necessary for removing batch effects and making samples from different studies comparable.The bladder is more proximal  to prostate than to thyroid, and this tissue proximity is reflected in Here, we corrected batch biases for each tissue separately. We also evaluated a different strategy to remove batch biases between TCGA and GTEx as a whole. For the three tissue types processed through our pipeline, we used all TCGA normals as one batch and GTEx normals as another batch to run ComBat. Our preliminary analysis showed this strategy was not effective in making RNA-seq samples from the two studies comparable .ERBB2, IGF2, and TP53, in our batch-effect corrected data .Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


BCL11A, SELL, IRF1, SMAD1, CASP1, BIRC5, and MAX and CTCL-downregulated MDM4, SERPINB3, and THBS4 genes. With respect to sample biases, no matter if we performed subgroup analyses or full data set analysis, fresh samples tightly clustered together. While principal component analysis revealed that fresh samples were spatially closer together, indicating some preprocessing batch effect, they remained in the proximity to other normal/benign and FFPE CTCL samples and were not clustering as outliers by themselves. Notably, this did not affect the determination of DEGs when analyzing ≥2009 samples (fresh and FFPE biopsies) vs. ≥2009 FFPE samples alone.Cutaneous T-cell lymphomas (CTCLs) are a heterogeneous group of malignancies with courses ranging from indolent to potentially lethal. We recently studied in a 157 patient cohort gene expression profiles generated by the TruSeq targeted RNA gene expression sequencing. We observed that the sequencing library quality and depth from formalin-fixed paraffin-embedded (FFPE) skin samples were significantly lower when biopsies were obtained prior to 2009. We also observed that the fresh CTCL samples clustered together, even though they included stage I–IV disease. In this study, we compared TruSeq gene expression patterns in older (≤2008) vs. more recent (≥2009) FFPE samples to determine whether these clustering analyses and earlier described differentially expressed gene findings are robust when analyzed based on the year of biopsy. We also explored biases found in FFPE samples when subjected to the TruSeq analysis of gene expression. Our results showed that ≤2008 and ≥2009 samples clustered equally well to the full data set and, importantly, both analyses produced nearly identical trends and findings. Specifically, both analyses enriched nearly identical DEGs when comparing benign vs. (1) stage I–IV and (2) stage IV  CTCL samples. Results obtained using either ≤2008 or ≥2009 samples were strongly correlated. Furthermore, by using subgroup analyses, we were able to identify additional novel differentially expressed genes (DEGs), which did not reach statistical significance in the prior full data set analysis. Those included CTCL-upregulated  Classically, MF may present with centrally distributed erythematous patches and plaques that are not specific to CTCL and are commonly misdiagnosed as chronic eczema, psoriasis, pityriasis rubra pilaris, drug eruptions, and dermatophyte infections. Finally, histopathological analysis of skin biopsies and PCR evaluation of T-cell receptor clonality lacks sensitivity in early MF patients and in erythrodermic disease. Unfortunately, current time to CTCL diagnosis from its initial presentation averages ~6 years (Cutaneous T-cell lymphomas (CTCLs) represent ~4–8% of all non-Hodgkin’s lymphomas and are characterized by infiltration of malignant T lymphocytes into the skin . Most paJAK/STAT/NF-κB signaling, cytokine production, chromatin remodeling, cell cycle checkpoint regulation, DNA repair, as well as cancer testis and embryonic stem cell signaling and function  vs. 2009–2012 (more recent) years. Indeed, recent samples that were freshly obtained and snap frozen had comparable total number of sequencing reads , while older FFPE samples had often <300 K sequencing reads . In addiNotably, these variables  were not formally evaluated in the prior publication but may contribute to the observed heterogeneity. These variations contribute toward a larger problem, known as the batch effect, in the field of gene expression-based analyses that utilize TruSeq, RNA-Seq, gene expression microarrays, and other approaches to identify DEGs. Differences in preprocessing, sequencing runs, technicians/centers, date of experiments, populations, and experimental design can account for heterogeneity that will remain despite normalization and use of control samples. Potential consequences of batch effect include reduction of statistical accuracy, introduction of spurious DEGs, and discrepancies between observed and true correlations . SeveralIn this study, we aimed to characterize TruSeq gene expression patterns separately in older (≤2008) vs. more recent (≥2009) FFPE samples to determine whether clustering analyses results display robustness when compared to the full data set. We also explored sample processing biases (old vs. new and FFPE vs. freshly obtained snap frozen).As described before , all patProcessed TruSeq data from Litvinov et al  were re-Unsupervised hierarchal clustering was performed in R, using packages stats, cluster, and gplots. Pairwise dissimilarity (distance) matrix was calculated using Gower’s method, which performs well in the case of incomplete/missing data when compared to other methods . ClusterPrincipal component analysis was performed on scaled, centered TPM data using package pcaMethods . Probabit-test. Power analysis showed an 86% power to detect a twofold expression change at a significance level of 0.05 for the comparison between the smallest subgroups, with complete data points. Correlations were computed using Spearman’s rho, on log-2 ratios. Mean RINs were compared using a Bayesian analysis with Markov Chain Monte Carlo (MCMC) simulations, using R package rjags; at least 100,000 iterations were performed to estimate p values.Differences in mean TPMs were determined using two-tailed Ward’s n = 9) CTCL skin biopsies, one early and two late-stage samples, while the other two subgroups (B and C) were very heterogeneous with respect to their composition. For the second large cluster on the center right panel (Cluster 3), a similar admixture was observed with three subgroups, one subgroup being comprised primarily benign skin samples (C) and the other two containing predominantly early (A) and advanced (B) stage CTCL disease samples.We previously noted that the ≤2008 FFPE samples had significantly decreased number of sequence reads per sample when compared to the ≥2009 samples . Therefore, we repeated unsupervised hierarchical clustering for benign samples (skin tags and benign inflammatory dermatoses), fresh liquid nitrogen snap-frozen CTCL samples, and either ≤2008 or ≥2009 FFPE CTCL samples. For ≤2008 FFPE sample analysis Figure , one of n = 28) and 2 stage IV CTCL samples.We then performed unsupervised hierarchical clustering for benign samples  vs. stage IV CTCL disease. Similarly, two analyses were performed for ≤2008 and ≥2009 FFPE biopsies. In the case of ≤2008 samples Figure , there wn = 7) with advanced CTCL disease tightly clustered together, while 2B (n = 20) and 2C (n = 11) samples included primarily benign dermatoses and skin tags .Surprisingly, for ≥2009 samples Figure , greatert-test to compare either benign dermatoses vs. (1) all CTCL samples or (2) stage IV CTCL. In our initial report , when ≤2008 stage I–IV CTCL or ≤2008 stage IV CTCL samples were compared to benign skin samples. This list included 47/75 DEGs that were enriched in the initially reported full data set  were found when freshly obtained and for ≥2009 samples, FFPE CTCL biopsies were analyzed together in a similar way  were revealed as being statistically significant (Table As presented in Table data set . New higp < 10−16), while this indicator was ρ = 0.55  when comparison was made between benign dermatoses and stage IV FFPE biopsies.Based on these combined results, 42/75 DEGs were confirmed in both analyses, which highlights significant robustness of these tests. Of course, many of the initially identified DEGs did not achieve statistical significance since the number of samples analyzed in each of these subanalyses  was significantly smaller than when all the data were analyzed as one set. Moreover, based on the original TruSeq data, subgroup analysis showed consistency in log-2 ratios between ≤2008 and ≥2009 CTCL samples. Indeed, rank–rank correlation when comparing benign dermatoses vs. all FFPE CTCL samples was ρ = 0.71  vs. mid (IIB and III) vs. late (IV) stage CTCL for ≤2008 vs. ≥2009 samples. Similarly, we noted a significant molecular heterogeneity that was seen in our original report . HoweverTOX, EED, and LCP2 were upregulated in late-stage CTCL, while ATXN7, CHD1, HUNK, TP63, KIT, JUNB, LTBP4, HDAC2, and OTUB2 were expressed preferentially in early-stage CTCL samples  early-stage (≤IIA) to mid and late CTCL stages (≥IIB) samples and (2) stage I vs. stage IV CTCL samples. Similarly, in this case, we analyzed ≤2008 and ≥2009 CTCL samples separately to test the robustness of the TruSeq results based on the year of skin biopsy. For ≤2008 samples, 12 genes were highlighted as being statistically significant: p < 10−4). However, there was no correlation when comparing stage I vs. stage IV FFPE tissues .This subgroup analysis showed moderate consistency in log-2 ratios ≤2008 and ≥2009 samples when comparing early vs. mid and late FFPE CTCL samples . RNA libraries were also more concentrated in the freshly obtained samples . There was no difference in RINs between ≤2008 and ≥2009 FFPE samples . RNA libraries were less concentrated in ≤2008 FFPE samples vs. ≥2009 samples, possibly explaining in part the lower TruSeq sequencing depth in ≤2008 samples .With respect to the RINs, a measure of sample quality prior to conducting the TruSeq analysis, we observed lower RINs for FFPE samples than freshly obtained snap-frozen samples. However, these RINs were within the expected range for FFPE samples , 57. RINTo detect possible batch effects, we performed PCA on TPM data. We aimed to determine whether (1) freshly obtained flash snap-frozen samples cluster together and (2) ≤2008 and ≥2009 FFPE samples cluster in different areas. We observed a tight cluster of fresh CTCL samples (gray), whether using ≤2008 Figures A,B or ≥2TOX, FYB, LEF, and STAT signaling genes, inflammatory interleukins, NF-κB pathway signaling members, cancer testis genes, etc. We had previously reviewed in detail how these genes relate to the biology of CTCL tumorigenesis  of the samples. Clustering analysis showed that ≤2008 and ≥2009 samples clustered equally well to the full data set and, furthermore, in a number of instances they demonstrated even better defined clusters. In particular, for ≤2008 samples, clusters were more reminiscent of the three clusters found in the landmark Boston CTCL cohort , 32, 48 igenesis .Furthermore, this subgroup analysis enabled us to discover additional genes, which did not reach statistical significance in the full data set analysis. One may find it to be counterintuitive. However, indeed, despite the inherently decreased power, potential reasons why additional DEGs can be identified through subgroup analysis may include reduced variability on a per-sample basis due to increased in-group similarity and removal of outliers in some groups.BCL11A (regulation of RNA transcription), SELL (cell adhesion molecule in the selectin family), IRF1 (Interferon transcription factor), SMAD1 , CASP1 (caspase involved in proteolysis), BIRC5 , MAX (Myc-associated transcription factor), and CTCL-downregulated MDM4 (negative regulator of p53), SERPINB3 (serine protease involved in inflammatory response), and THBS4 (cell-cell and cell-matrix interactions) genes. Of note, THBS4 promoter was previously found to be frequently hypermethylated in 52% of CTCL samples, which leads to the downregulation in expression of this tumor suppressor gene , FYB (T-cell adaptor protein), and GTSF1 (germ cell maintenance) were upregulated, and LTBP4 (latent TGF-beta binding protein) was downregulated in later CTCL stages. By merging subgroup analysis of ≤2008 and ≥2009 FFPE samples, our targets included TOX, GTSF1, and LTBP4 as well. In particular, TOX overexpression is a hallmark of poor prognosis in CTCL, although low level of TOX expression has been previously reported in benign dermatoses , SKAP1 (T-cell adhesion), and LCP2  were upregulated in advanced CTCL stages. Surprisingly, we also found multiple genes with higher expression in early-stage tumors. These included BCL11A (see above), ATXN7 , HUNK (AMPK-related kinase), CHD1 (chromatin remodeling), TP63 (transcription factor), KIT (receptor tyrosine kinase), JUNB (transcription factor), HDAC2 (histone deacetylase), and OTUB2 . Based on these combined results, transcription factors, chromatin remodelers, and global cell signaling processes are upregulated early in the disease, while in the advanced stages of CTCL, T-cell-specific genes, inflammatory mediators, and stem cell/germ cell maintenance genes appear to be driving cancer progression. These results further argue that subgroup analysis can often yield additional clues into the biology of cancers.Subsequently, when we studied the DEGs enriched in both (1) early vs. mid and late CTCL and (2) stage I vs. stage IV disease, four genes were differentially expressed: rmatoses , 60. TOX disease , 31, 34.via chromatin immunoprecipitation (R2 > 0.70)  . Interes > 0.70) , which i > 0.70) , which i > 0.70) . In summAll patients were enrolled in the study in accordance with the IRB-approved protocols: PA12-0267, PA12-0497, and Lab97-256 at the MD Anderson Cancer Center (MDACC) and A09-M106-13A and 13-201-GEN at McGill University/McGill University Health Centre (MUHC). This study was carried out in accordance with the recommendations of the Research Ethics Board of the McGill University/McGill University Health Centre with written informed consent from all subjects in accordance with the Declaration of Helsinki. This study was carried out in accordance with the recommendations of the MD Anderson Cancer Center (MDACC) Research Ethics Board, which exempted us from obtaining written informed consent from patients, who earlier signed a hospital consent allowing their stored biopsy samples to be used for research.PL, EN, MT, LM, AW, DS, XN, NP, MG, MD, and IL procured and analyzed patient sample data presented in this paper. PL, EN, and IL performed bioinformatic and statistical analyses. MT and AW performed pathological analysis of the original skin samples. PL, EN, MT, LM, AW, DS, XN, NP, MG, MD, and IL wrote the paper. NP, LM, MG, DS, MD, and IL supervised the study.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer YG and handling editor declared their shared affiliation.


Precision medicine is a rapidly growing area of modern medical science and open source machine-learning codes promise to be a critical component for the successful development of standardized and automated analysis of patient data. One important goal of precision cancer medicine is the accurate prediction of optimal drug therapies from the genomic profiles of individual patient tumors. We introduce here an open source software platform that employs a highly versatile support vector machine (SVM) algorithm combined with a standard recursive feature elimination (RFE) approach to predict personalized drug responses from gene expression profiles. Drug specific models were built using gene expression and drug response data from the National Cancer Institute panel of 60 human cancer cell lines (NCI-60). The models are highly accurate in predicting the drug responsiveness of a variety of cancer cell lines including those comprising the recent NCI-DREAM Challenge. We demonstrate that predictive accuracy is optimized when the learning dataset utilizes all probe-set expression values from a diversity of cancer cell types without pre-filtering for genes generally considered to be “drivers” of cancer onset/progression. Application of our models to publically available ovarian cancer (OC) patient gene expression datasets generated predictions consistent with observed responses previously reported in the literature. By making our algorithm “open source”, we hope to facilitate its testing in a variety of cancer types and contexts leading to community-driven improvements and refinements in subsequent applications. These efforts have significantly benefited from the liberal sharing of data and open-source scripts utilized for these efforts. Over the last few years, there has been a number of alternative machine-learning (ML) approaches employed in personalized cancer drug prediction, each associated with variable degrees of success –3. For ee.g., ,2), suche.g., ,7.https://github.com/chuang95/KEA_DrugResponse). Also available at this GitHub site are general procedures for open application of our software to additional datasets (https://github.com/chuang95/KEA_DrugResponseLearning). We have employed the algorithms to explore the effect of a variety of alternative learning datasets on predictive accuracy leading to several unanticipated findings. First, predictive accuracy was significantly improved when microarray probe level expression data rather than average gene expression values were employed in the model building process. Second, predictive accuracy was improved when models were built upon a diversity of cancer types. Third, the pre-filtering of learning datasets based upon preconceived biological models significantly reduces predictive accuracy. Application of our optimized models to publically available ovarian cancer (OC) patient gene expression datasets generated predictions highly consistent with observed responses to a variety of drugs. By providing true open access to our software, we seek to encourage additional improvements in current methods, as well as, constructive comparisons with alternative approaches leading to the development of optimal ML-based strategies for personalized cancer medicine.We present here an open source software platform using a highly versatile support vector machine (SVM) algorithm that utilizes standard recursive feature elimination (RFE) methods to predict cancer drug response. In pilot applications, we utilized publicly available datasets from NCBI Gene Expression Omnibus (GEO)  that we https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE32474) ) . SVM has been successfully applied in a variety of biological applications in recent years . Our SVSVM models built upon large datasets typically contain uninformative features, and a number of feature selection methods have been developed to identify subsets of features with optimal predictive accuracy –12. We eThe minimal number of informative features associated with optimally predicted responsiveness to the seven drugs modeled in this study ranged from 10 to 32  derived from 18 of the NCI-60 cell lines. In one case, the 18 cell lines were representative of only two types of cancers (lung and melanoma) while in the other case, the 18 cell lines were randomly selected to be representative of all 9 types of cancer comprising the NCI-60 dataset . As shown in To explore this issue, we compared the relative accuracies of two SVM-derived models designed to predict response to the commonly prescribed cancer drug carboplatin. The respective models were built using gene expression profiles and drug response profiles . In the other case, we downloaded and employed the original probe data as our learning set . The results presented in To test this possibility, we compared the relative predictive accuracies of two SVM-based algorithms developed to predict the sensitivity of the set of breast cancer cell lines recently employed in the NCI-DREAM Challenge to the drug doxorubicin . In one Some methods to assess risk of cancer progression and/or severity focus almost exclusively on genes previously identified as drivers of cancer onset/progression . The advhttp://foundationone.com/docs/FoundationOne_tech-info-and-overview.pdf). In the second model, the learning dataset included probes of all significantly expressed genes . The expression values for each individual array were normalized back to the NCI-60 gene expression data matrix.While our predictive models were established using gene expression and drug sensitivity data from human cell lines, we were interested in conducting preliminary evaluations of the models' ability to predict the response of human cancer patients to chemotherapeutic treatments. Toward this end, we downloaded three independently derived (Affymetrix) gene expression datasets of 273 ovarian cancer patient tumors from the Gene Expression Omnibus (GEO) repository  repository competitions where alternative computational solutions are openly deposited . We beliToward that end, we present here an open access support vector machine (SVM)-based algorithm for the predictive response of cancers to seven widely employed chemotherapeutic drugs. The algorithm combines a standard SVM approach with a "one-by one" data normalization pipeline. We have employed the algorithm to explore the effect of a variety of alternative learning datasets on predictive accuracy leading to several unanticipated findings. For example, although it may seem intuitive that drug predictive models for a specific type of cancer should optimally be built upon data from the same cancer type, our results suggest that this may not always be the case. The predictive accuracy of the drug response of a particular cancer type was significantly increased when the model was built using data from a variety of cancer types. This finding is consistent with growing evidence that molecular signatures of optimal cancer drug response are not necessarily defined by the cancer's tissue of origin .Microarray platforms typically monitor gene expression levels using multiple probe sets. This allows discrimination between the expression patterns of alternative splice variants and/or other gene transcript isoforms. Most often, the input expression data for the building of ML predictive models utilizes average expression values across all gene probes. We found that higher accuracy is attained when all probes are incorporated in the learning dataset presumably because some isoforms are more informative than others with respect to drug response and this information is lost or diluted when individual probe data are combined in an average value.i.e., genes that have been previously identified as playing key roles in cancer onset and progression. For this reason, the molecular profiles of putative cancer driver genes or other pre-defined subsets of genes are often considered sufficient for the accurate prediction of optimal drug therapies. We found that predictive accuracy can, in fact, be significantly reduced when expression profile datasets are pre-filtered prior to ML-based model building. This result suggests that genes involved in cancer drug response are not necessarily limited to those involved in cancer onset even when the drug in question is designed to target a specific group of driver genes.Personal cancer drug therapy, as currently envisioned, involves the targeted inhibition of one or more "cancer driver" genes, Although our models were built using the publically available NCI-60 cancer cell line datasets, we are encouraged that predictions using publically available human patient datasets are generally consistent with clinical observations. By making our predictive models open source, we hope to encourage the testing of predictions in additional human datasets representative of a diversity of tumor types.https://github.com/chuang95/KEA_DrugResponse). In addition, a user-friendly web server is currently under construction that will further enhance public access to our methods.In summary, our findings demonstrate that significant improvements can be made in the predictive accuracy of ML-based algorithms by modulating the format and/or type of learning datasets employed in the model building process. This finding is likely to be relevant regardless of the type of ML approach employed. While our results illustrate several paths by which the predictive accuracy of our ML-based cancer drug prediction algorithm was improved, these and additional possibilities need to be tested with larger and more extensive datasets. We believe that such goals are most effectively attained by communal efforts where the research community is provided open access to the underlying code and pipelines employed so that meaningful improvements and comparisons with alternative methods can be made . Toward It is our hope that through community sharing of this and other open source cancer drug prediction algorithms and associated data formatting/normalization procedures that the attainment of a major goal of personalized cancer medicine will be facilitated.Standard gene expression data analyses were conducted as previously described . Individmean+0.50 SD while resistant label indicates GI50 values below mean+0.50 SD. GI50 values that lie within ±0.50 SD of the mean are marked as indeterminate.We define the negative log transformed GI50 value for each cell line. In this approach, the higher the transformed GI50 value, the more sensitive the cell line is to the drug. Three labels were collected: sensitive (marked 1), resistant (marked -1) and indeterminate (marked 0). The sensitive label indicates GI50 values above svmtrain (Matlab R2013b version 8.2.0.701), and the kernel function is linear. The samples are represented as a vector x, and the two classes are divided in the dataspace by a hyperplane wx’ + b = 0 that maximizes the margins between the learning samples of the two classes. This margin is defined such that:w and b are the weight vector and bias parameters from the SVM model. The input x is the normalized test sample gene expression data with RFE selected i number of features. The classification of the patient drug response is based on this score. We call a sample a responder to the drug if this score is higher than 0, and a non-responder to the drug if the score is lower than 0.The microarray gene expression values of the NCI-60 cell lines are formatted as a matrix, and sub-divided into training (75%) and validation (25%) datasets. Each probe of a gene is analyzed as a separate feature for each sample. We applied SVM on training data to get weights for each feature, and sort the features based on the weights (Table C in Recursive feature elimination (RFE) was performed to find the minimum set of features that maximized accuracy in the classification on the test dataset . The apparea under the curve, tpr = true positive, fpr = false positive, tnr = true negative and fnr = false negative. We optimize AUC by maximizing tpr—fpr or minimizing a sum of (absolute) normalized error fpr + fnr. Optimal models are associated with higher AUC values.ROC curves were generated using the standard function as outlined below:S1 FileFig A. Ranked display of -log transformed GI50 values for the other six chemotherapeutic drugs for each of the NCI-60 cell lines.Fig B. Evolution of accuracy of predicted drug responses for the other six chemotherapeutic drugs using SVM-RFE selection for gene probe classifiers.Fig C. Visualization of the optimal separation between drug sensitive and resistant NCI-60 cell lines for the other six chemotherapeutic drugs.Fig D. The models built using data from the 9 cancer types vs. 2 cancer types.Fig E. Example of expression levels for probes from the same gene, NEAT1.Fig F. Comparison of LOO-cross validation of predicted response to carboplatin using our SVM-RFE method vs. two other commonly employed methods.Fig G. Comparison of the average gene expression for the learning datasets derived from 2 cancer types  vs. 9 cancer types .(PDF)Click here for additional data file.S2 FileTable A. Expression values of gene probes (Affymetrix_U133_2.0_plus) for the NCI-60 cell lines.Table B. Sensitivity (GI50) of NCI-60 cell lines to seven chemotherapeutic drugs.Table C. Ranking of probe (feature) weights for employed in the recursive feature elimination (RFE) process for the prediction of carboplatin sensitivity.Table D. Probes associated with optimal predictive accuracy for 7 chemotherapeutic drugs.Table E. Several of the most informative genes predictive of carboplatin sensitivity have been directly or indirectly implicated with apoptosis.Table F. Comparison between predicted response rates of ovarian cancer patients to 7 chemotherapeutic drugs and response rates as reported in the literature.(XLSX)Click here for additional data file.


FALF), in which we implement the idea of selecting optimal training samples to enhance generalization performance for two different kinds of classifiers, discriminative and generative (e.g. SVM and KNN). The optimal samples are selected by first estimating the boundary of each class and then calculating the fuzziness-based distance between each sample and the estimated class boundaries. Those samples that are at smaller distances from the boundaries and have higher fuzziness are chosen as target candidates for the training set. Through detailed experimentation on three publically available datasets, we showed that when trained with the proposed sample selection framework, both classifiers achieved higher classification accuracy and lower processing time with the small amount of training data as opposed to the case where the training samples were selected randomly. Our experiments demonstrate the effectiveness of our proposed method, which equates favorably with the state-of-the-art methods.Hyperspectral image classification with a limited number of training samples without loss of accuracy is desirable, as collecting such data is often expensive and time-consuming. However, classifiers trained with limited samples usually end up with a large generalization error. To overcome the said problem, we propose a fuzziness-based active learning framework ( Remote sensing is a mature field of science and extensively studied to extract the meaningful information from earth surface or objects of interest based on their radiance acquired by the given sensors at short or medium distance  2]. One. One2]. Nowadays, researchers are broadly studying hyperspectral image classification techniques for the case of a limited number of training samples, both with and without reducing the dimensionality of hyperspectral data. In this regard, the recent works  5] 6]  [6] 5] [  [6] 5] Supervised learning techniques, which require class label information, have been widely studied for hyperspectral image classification ; howeverTo cope with the issues that are discussed above, several techniques have been developed. These include discriminant analysis algorithms with different discriminant functions   13], fe, fe13], To some extent, semi-supervised approaches have addressed the problem of a limited number of labeled training samples by generating the labels though machine-machine interaction. The primary assumption of semi-supervised classification methods is that the newly labeled samples for learning can be generated with a certain degree of confidence from a set of limited available labeled training samples without considerable cost and efforts  21]. Se. Se21]. All the techniques discussed above assume that the labeled training samples are limited in number, and these methods enlarge the initial training set by efficiently exploiting the unlabeled samples to address the “ill-posed problem”. However, to achieve the desired results, several vital requirements need to be met. For example, the quantity of the generated data should not be too large such that it may increase the computational complexity, and the samples should be properly selected to avoid any confusion in correctly classifying the unseen samples. Above all, the obtained samples and their class labels must be obtained without substantial cost and time.Active learning techniques can be used to overcome the above-mentioned issues. In general, active learning techniques are referred to as a special subcategory of semi-supervised learning techniques  34]. Wi. Wi34]. In general, the additional labeled samples are selected randomly or by using some information criteria or source of information to query the samples and their class information. Random selection of the training samples is more-often subjective and tends to bring redundancy into the classifiers. Furthermore, it reduces the generalization performance of the classifiers. Moreover, the number of samples required to learn a model can be much lower than the number of used samples. In such scenarios, there is a risk that the learning model may get overwhelmed because of the uninformative samples queried by the learning model.FALF). In FALF, the classifier comes with an integrated data acquisition module that ranks unlabeled samples based on their confidence for the future query that has the maximum learning utility. Thus, the proposed framework aims to achieve the maximum potential of the learning model using both labeled and unlabeled data, whereas the amount of training data can be kept to a minimum by focusing only on the most informative training samples. This process leads to a better utilization of information in the data, while considerably minimizing the cost of labeled data collection and improving the generalization performance of the classifiers.To this end, in this work, an active learning framework using a single sample view critical-class-oriented query is proposed for hyperspectral image classification. We call this scheme fuzziness-based active learning framework .Investigating the potential of the proposed framework to reduce the classification time while maintaining a good accuracy under high dimensionality.The main idea of this work is to employ and retain the relationship between misclassification rate of boundary samples and fuzziness for each class to select samples for the training set. The important steps of our proposed algorithm are summarized below:Randomly select 5% of labeled training samples from each class.Train Support Vector Machine (SVM) and Fuzzy K-Nearest Neighbor (FKNN) on randomly selected samples and test them for the rest of the samples.Record the fuzzy membership matrix.Calculate the fuzziness from fuzzy membership matrix for each sample and estimate the distance between the sample and the boundary.Based on the threshold of fuzziness magnitude, divide the samples into two subgroups as lower and higher fuzziness magnitude groups.i.e., TP and FP) for each class in both groups individually.Determine the correct rate of classification and misclassification (Step 5), i.e. the samples with high/lower fuzziness and lower/higher distance to the boundary.(A): Pick 5% of the hardest correctly predicted samples, i.e. the samples with higher fuzziness and lower distance to the boundary. OR (B): Randomly select 5% of the correctly predicted samples, without taking into account their fuzziness and distance from the class boundary.  or (B), individually, and predict the rest of the samples and determine the accuracy respectively.It is to be noted that step 7 (A) and (B) are two alternative ways to select the samples. In general active learning approaches, the samples are selected through step 7 (B), but we propose to select the samples using step 7 (A) and compare the accuracies obtained by both ways in experimental and results section.The intuition behind selecting the hardest correctly predicted samples using step 7 (A) is that such samples contain the most information about boundaries rather than the samples with lower fuzziness in magnitude. The threshold value between lower and higher fuzziness is set by trial and error. The proposed methodology significantly boosts the performance of the classifier for hyperspectral image classification not only in terms of accuracies but also reduce the classification time.Here we will theoretically explain the procedure for estimating the boundary of each class, and then how to build a relation between the samples and the estimated boundaries to select the target samples.Generally, there are two kinds of classifiers: those that use some specific formula to estimate class boundaries (discriminative), and others that use some distribution for the same task (generative). For example,  40] [41 [4140]  used an Before we discuss the boundary extraction process for both classifiers, it is helpful to understand the concept of a fuzzy membership function, because we seek the output of each classifier in the form of fuzzy membership grades.N sample vectors {r1, r2, r3, …, rN}, and a fuzzy partition of these N sample vectors represents each sample vector’s degree of membership to each of the C classes. The fuzzy C partitions have certain characteristics as defined below:μij ∈ , and μij = μi(rj) is a function that represents the membership  for the jth sample rj, to the ith partition, i ∈ 1, 2, 3, …, C, and j ∈ 1, 2, 3, …, N.Let us assume a set of ri and rj denotes the ith and jth training samples respectively. The mathematical hypothesis of SVM is given by:SVM aims to find the optimal hyperplane according to the maximization of the margin on the training data. In SVM, data is mapped from the input space into a high dimensional feature space using an implicit function; such mapping is directly associated with a kernel function ci is the ith class label, b and αi are unknown parameters which are determined by quadratic programming. Furthermore, αi is a vector of non-negative Lagrange multipliers; therefore, the solution vector αi is sparse and the samples ri which correspond to nonzero αi are called support vectors. Thus, the samples ri corresponding to αi = 0 have no contribution to the construction of the optimal hyperplane. From the literature, one can find several extensions of SVM . If the component of a sample vector is equal to 0 or 1, then the algorithm behaves like a common KNN. FKNN search is similar to the traditional KNN search. In traditional KNN, each sample can only belong to one class, which is the majority class in KNN search, whereas in FKNN, a sample can belong to multiple classes with different membership degrees associated with these classes. FKNN can be summarized in the following steps:K nearest neighbors rj, j ∈ 1, 2, 3, …., K, of the given sample r using Euclidean distance function from the set of the samples.First find r − rj‖ is the Euclidean distance and μi(rj) is the membership value of the point rj for the ith class. The parameter m controls the effective magnitude of the distance of the prototype neighbors from the sample under process  or generally it can be expressed as E: R → R+, which satisfies the following axioms as defined in [Consider a mapping fined in  45] ..48].κ) coefficient and overall accuracies are considered as the evaluation metrics since these are widely used in existing works. The kappa coefficient is obtained by using the expressions given below [The Kappa , true negative (TN), false positive (FP), and false negative (FN) are usually compared. In this regard, we have made several judgments that are presented in Tables We have abbreviated the test names in Tables FOR and FDR is a statistical method used in multiple hypothesis testing to correct for multiple comparisons. It measures the proportion of false negatives that are incorrectly rejected. FOR is computed by using FN and TP; it can also be computed by taking the complement of negative predictive values (NPVs). FDR measures the proportion of actual positives that are incorrectly identified and is computed by using FP and TP.For class-based classification judgments, we have done two statistical analysis, which is presented in Figs Alfalfa”, “Corn Notill”, “Corn Mintel”, “Corn”, “Grass Pasture”, “Grass Trees”, “Grass Pasture Mowed”, “Hay Windrowed”, “Oats”, “Soybean Notill”, “Soybean Mintel”, “Soybean Clean”, “Wheat”, “Woods”, “Buildings Grass Trees Drives” and “Stone Steel Towers””.The summary classes for classification are as follows. Indian Pines: classes 1 to 16 are ““Asphalt”, “Meadows”, “Gravel”, “Trees”, “Painted Metal Sheets”, “Bare Soil”, “Bitumen”, “Self-Blocking Bricks” and “Shadows””.Pavia University: classes 1 to 9 are, ““Water”, “Trees”, “Asphalt”, “Self-Blocking Bricks”, “Bitumen”, “Tiles”, “Shadows”, “Meadows”, and “Bare Soil””.Pavia Centre: classes 1 to 9 are, ““To evaluate the performance of our proposed framework, the following state-of-the-art methods are compared. All competing methods are evaluated on two publicly available real hyperspectral datasets and the average performance of 5-fold cross validation is presented. The detailed performance comparison of the proposed algorithm with state-of-the-art methods defined below is presented in Tables Multinomial Logistic Regression with Active Learning Active Learning framework using hierarchical segmentation Spatial Coherence Batch Mode Active Learning Spectral-Spatial Classification using Loopy Belief Propagation and Active Learning We can find many classical active learning frameworks in the literature that are similar to the proposed framework. For example, the work proposed by Lughofer in  focused In contrast to , Nie et The samples between the ranges of 0.7–1.0 in fuzziness magnitude). We then fuse a specific percentage of these hardly predicted samples back into the original training set to retrain the classifier from scratch for better generalization and classification performance on those samples which were initially misclassified.In our work, we start evaluating our hypotheses from 5% of randomly selected training samples and we demonstrate that randomly adding more samples (step 7(B)) back into the training set slightly increases accuracy but the classifiers become computationally complex. Therefore, we decided to separate the set of samples that were most difficult to predict in our first phase of classification  test results are significantly improved as we can see from Figs A second most important factor involved in the training and testing phase is computational time, which is significantly improved for both classifiers. Therefore, to make the model efficient and quick, we fuse the most difficult and informative samples back into the training set to retrain the classifier in each experiment. The classification accuracy and Kappa Click here for additional data file.S2 FileAlfalfa-46”, “Corn Notill-1428”, “Corn-Mintel-830”, “Corn-237”, “Grass Pasture-483”, “Grass Trees-730”, “Grass Pasture Mowed-28”, “Hay Windrowed-478”, “Oats-20”, “Soybean Notill-972”, “Soybean Mintel-2455”, “Soybean Clean-593”, “Wheat-205”, “Woods-1265”, “Buildings Grass Trees Drives-386” and “Stone Steel Towers-93””. The ground truths are freely available at [The ground truth classes and the number of samples per class (class name-number of samples) are as follows: ““(TXT)Click here for additional data file.S3 File(M)Click here for additional data file.


Here we describe a new method, standardized fold change (SFC), for differential analysis of microarray data. We estimated the performance of SFC, the t test and Limma by generating simulated microarray data 100 times. SFC performed better than the t test and showed a higher sensitivity than Limma where there is a larger value for fold change of expression. SFC gave better reproducibility than Limma and the t test with real experimental data from the MicroArray Quality Control platform and expression data from a mouse cardiac allograft. Eventually, a group of significant overlapping genes was detected by SFC in the expression data of mouse cardiac and hepatic allografts and further validated with the quantitative RT‐PCR assay. The group included genes for important reactions of transplantation rejection and revealed functional changes of the immune system in both heart and liver of the mouse model. We suggest that SFC can be utilized to stably and effectively detect differential gene expression and to explore microarray data in further studies.Murine transplantation models are used extensively to research immunological rejection and tolerance. Here we studied both murine heart and liver allograft models using microarray technology. We had difficulty in identifying genes related to acute rejections expressed in both heart and liver transplantation models using two standard methodologies: Student's  FNRfalse negative rateFPRfalse positive rateLimmalinear models for microarray dataMAQCMicroArray Quality ControlPODpost‐operative dayqRT‐PCRquantitative RT‐PCRSFCstandardized fold changeAt the stage of organ failure, organ transplantation is a life‐saving medical procedure, though it still has some problems, e.g. transplant rejection and the requirement for life‐long immunosuppressive drugs. Transplantation models without immunosuppression are important and the mechanisms of rejection and tolerance in these models need to be revealed.t test and linear models for microarray data (Limma) are two popular choices t test utilizes information for all the samples (or standard deviations) in one microarray probe and is conducted independently among different probes Microarray technology is well established and widely used, providing a picture of gene expression or RNA profiling in different tissues t test and Limma, no significant intersecting genes were obtained in these models. Therefore, we developed a new method, named standardized fold change (SFC), to detect differential expression by taking information from the neighbors of one probe with an adjustable bin size. To compare SFC with the t test and Limma, we generated a simulated data set to estimate the performance and used the real experimental datasets from the MicroArray Quality Control (MAQC) platform and the transplantation model to estimate the reproducibility. We concluded that SFC can be applied as a new and effective approach to detect differential expression and contribute more reliable results in microarray studies. Then, SFC reported a set of significant genes from expression data from the murine heart and liver allograft, and we further validated them by qRT‐PCR. Gene expression changes revealed functional reactions and pathway activities in the early stage of allograft in both heart and liver.In the present study, we established murine heart and liver allograft models and used microarray technology to reveal the significant genes that related to transplant rejection. By using the Male B10.BR , B10.D2 , C57BL/10  and CBA (H‐2k) mice (weighing 25–30 g) were purchased from the Shizuoka Laboratory Animal Center  and housed and cared for in agreement with the guidelines of the Institutional Animal Care and Use Committee and the National Research Institute (Japan) for Child Health and Development guidelines on laboratory animal welfare. The Committee on the Care and Use of Laboratory Animals at the National Research Institute accepted the experimental protocol for Child Health and Development (Permission no.: 2002‐003). All surgical procedures were conducted under anesthesia with isoflurane/oxygen, and all attempts were made to minimize suffering.Cardiac transplantation was performed from a sex‐matched B10 donor to a CBA recipient by microsurgical techniques. Intra‐abdominal vascularized heterotopic mouse cardiac transplantation was performed The probe signals from microarray data were firstly natural log‐transformed and then manipulated with quantitative normalization. To assess the differential expressions among cases and controls, the statistic SFC is defined as:b nearest neighbors as the variance, where the default bin size of b here is 1000. The SFC software now implements this algorithm in the Linux system and can be found at https://github.com/WeichenZhou/SFC.For the variance of each probe, we ranked all probes by the mean values of signals from all samples and then took the median value of its We generated the simulated data from simple formulas with the Gaussian noise  as a default distribution for gene expression data 0 as 0% and k is 1. The control and case samples in the alternative hypothesis are shown as follows:where θ represented the differential expression underlying cases versus controls and we defined θ1 as 10%, 25% and 50%, respectively. The size of real positive calls consists of 1%, 5% and 10% of the whole simulated data, respectively. Following these, a 100‐time simulation was conducted to assess the false positive rate (FPR) and the false negative rate (FNR).We defined θThe MAQC project was developed by the US Food and Drug Administration (FDA) to provide standards and quality control metrics and involved six centers  that are major providers of microarray platforms and RNA samples P‐values from expression data were adjusted by the Bonferroni correction. After getting all significant probes from SFC, we converted the probe level significance to gene level using an annotation file. Venn diagrams showed the significant genes with differential expression. Pathway and gene ontology (GO) enrichment analyses were performed by using the Database for Annotation, Visualization and Integrated Discovery  with the Bonferroni correction‐adjusted P‐values < 0.05 GSE89340. All data were conducted by quantile normalization before processing by different methods. Limma can be found as the R package limmagplots. All R packages can be downloaded from Bioconductor (www.bioconductor.org).We detected differential expression of genes between cases and controls in three phases: POD5 of cardiac transplantation, POD5 of hepatic transplantation and POD8 of hepatic transplantation. All m of each primer in a final reaction volume of 20 μL of KAPA SYBR® FAST qPCR kit . The PCR cycling conditions were as follows: 95 °C for 30 s, and 50 cycles of 95 °C for 5 s, 60 °C for 1 min. The normalized threshold cycle (Ct) value of each gene was obtained by subtracting the Ct value obtained for 18S rRNA. The cardiac mRNA levels were analyzed on POD5. Figure 4 indicates the number of copies of each of the three representative mRNAs measured in the syngeneic grafts or allografts obtained from three individuals. The relative amount of each mRNA was normalized to that of 18S rRNA. All experiments were analyzed in three mice per time point and expressed as the mean ± SEM. The significance level was set as P < 0.05 compared with syngeneic grafts on day 5.The RNA was reverse‐transcribed to cDNA using a PrimeScript® RT Reagent Kit  as described previously b. As we set up the default value of b as 1000, the variance of cases and controls in one probe can be obtained by calculating the median for those probes separately. Eventually, by following Eqn P‐value can be further estimated based from these.We observed that the distribution of the mean value and variance of one probe signal is non‐linear Fig. . The infy = x, with the Gaussian noise added. The basic formulas are adjustable with the parameters k. The signals of the alternative hypothesis were described by Eqn We investigated the FPR and the FNR of the three methods under the null hypothesis and alternative hypothesis. As indicated in Eqn t test showed a positive bias with these parameters .Under the null hypothesis, the rates of the three methods are all near the significance threshold between 5% and 6% Fig. A. Under rs Table . For thet test.Reproducibility is an indispensable estimator for the experiments and algorithms t test among six platforms when detecting both the top 100 and the top 1000, while for intra‐platform reproducibility, all three methods did not perform well in detecting either the top 100 or the top 1000 significant genes  of the null hypothesis and alternative hypothesis also can be adjusted (Eqns P = 0.01 and 0.001) for different values of θ and k. With a more stringent significance level , the FPRs were decreased while the FNRs were increased, which was observed by all three methods  and Limma performed with a high FNR (sometimes more than 90%). This suggests the t test will give more positive hits with a high FPR, while Limma will report fewer hits to reduce the FPR but miss some true positive ones. Importantly, SFC can give a good balance of FPR and FNR, and perform well for both FPR and FNR with a stringent significance level.Microarray is widely used and accepted as a stable, well established and less costly technology to investigate gene expression data ted Eqns  7. Moreoods Figs  and S2. P‐value and control the false discovery rate t test) with different significance levels . Limma and the t test had a large number of positive hits when the P‐value was 0.05 in three phases  is often introduced for multiple comparisons to adjust the ses Figs  and S6. ion Figs  and S4, in vivo and functional changes of the immune system both at the cardiac and at the hepatic level after 5 days of allografts We therefore investigated the functions of these 52 genes Table , revealit test by using the nearby information of one probe in pooled probes. We utilized SFC for the real data of mouse transplantation models, and it reported a more stable and convincing set with 52 significant genes, revealing insights into pathway and gene expression changes after both cardiac and hepatic allografts. Nineteen genes were further randomly picked up and validated by qRT‐PCR. We suggest SFC is a new and effective approach that can detect differential expression and help to obtain more reliable information in microarray studies.In conclusion, based on the quality control experimental data and simulated data, SFC performed better than Limma and much better than the WZ, YW, XL and JW designed the project. LS supported the MAQC data, and MF and XL supported the mouse model and validations. WZ carried out the analysis and simulations. WZ, XL and JW wrote the manuscript. LJ, XL and JW contributed to the final revision the paper. All authors read and approved the final manuscript.Fig. S1. Distribution of mean and variance of sample microarray signals in each probe derived from the MAQC data.Click here for additional data file.Fig. S2. Bar graphs of FPR and FNR from the three methods under the null hypothesis (H0) and the alternative hypothesis (H1) with the level of significance set at P < 0.01.Click here for additional data file.Fig. S3. Bar graphs of FPR and FNR from the three methods under the null hypothesis (H0) and the alternative hypothesis (H1) with the level of significance set at P < 0.001.Click here for additional data file.Fig. S4. Venn diagrams of significant gene numbers analyzed by the t test and Limma with the level of significance set at P < 0.05 after the Bonferroni correction.Click here for additional data file.Fig. S5. Venn diagrams of significant gene numbers analyzed by the t test, Limma and SFC with the level of significance set at P < 0.05.Click here for additional data file.Fig. S6. Venn diagrams of significant gene numbers analyzed by the t test, Limma and SFC with the level of significance set at P < 0.001.Click here for additional data file.Table S1. Evaluation of three methods with the level of significance set at P < 0.01.Table S2. Evaluation of three methods with the level of significance set at P < 0.001.Table S3. Primer sequences for qRT‐PCR.Click here for additional data file.Table S4. GO term and pathway enrichment analysis based on the 52 significant genes.Click here for additional data file.