The Connectivity Map (CMAP) database, an important public data source for drug repositioning, archives gene expression profiles from cancer cell lines treated with and without bioactive small molecules. However, there are only one or two technical replicates for each cell line under one treatment condition. For such small-scale data, current fold-changes-based methods lack statistical control in identifying differentially expressed genes (DEGs) in treated cells. Especially, one-to-one comparison may result in too many drug-irrelevant DEGs due to random experimental factors. To tackle this problem, CMAP adopts a pattern-matching strategy to build “connection” between disease signatures and gene expression changes associated with drug treatments. However, many drug-irrelevant genes may blur the “connection” if all the genes are used instead of pre-selected DEGs induced by drug treatments.We applied OneComp, a customized version of RankComp, to identify DEGs in such small-scale cell line datasets. For a cell line, a list of gene pairs with stable relative expression orderings (REOs) were identified in a large collection of control cell samples measured in different experiments and they formed the background stable REOs. When applying OneComp to a small-scale cell line dataset, the background stable REOs were customized by filtering out the gene pairs with reversal REOs in the control samples of the analyzed dataset.https://github.com/pathint/reoa.In simulated data, the consistency scores of overlapping genes between DEGs identified by OneComp and SAM were all higher than 99%, while the consistency score of the DEGs solely identified by OneComp was 96.85% according to the observed expression difference method. The usefulness of OneComp was exemplified in drug repositioning by identifying phenformin and metformin related genes using small-scale cell line datasets which helped to support them as a potential anti-tumor drug for non-small-cell lung carcinoma, while the pattern-matching strategy adopted by CMAP missed the two connections. The implementation of OneComp is available at OneComp performed well in both the simulated and real data. It is useful in drug repositioning studies by helping to find hidden “connections” between drugs and diseases.The online version of this article (doi:10.1186/s12967-017-1302-9) contains supplementary material, which is available to authorized users. The strt test  lack powt test –16 lackst test  to buildIn contrast to the small number of treated samples, there are in total 492, 277 and 229 gene expression profiles of control samples, respectively, for MCF7, PC3 and HL60, which scatter in different experimental batches of CMAP and there are more available in other data sources. Thus, it would be desirable to exploit these valuable control data to aid the differential expression analyses of small-scale treated samples. However, gene expression profiles from different experimental batches or laboratories cannot be directly compared with each other due to experimental batch effects , 18. In In this study, by analyzing the gene expression profiles of three types of commonly used cancer cell lines, HepG2, HCT116 and MCF7, we show that the REOs of genes pairs are highly reproducible in the control samples of a particular cell type, even though these samples were collected from different laboratories with different cultivation conditions, but widely disrupted after certain treatments. Based on this observation, we adapted RankComp to datasets with only one technical replicate. The modified algorithm, named OneComp, was evaluated based on data of three cell lines. To demonstrate the usefulness of the OneComp algorithm, we applied it to an application case study on repositioning two drugs, phenformin and metformin, for NSCLC based on the CMAP database.http://www.ncbi.nlm.nih.gov/geo/) and ArrayExpress [http://www.ebi.ac.uk/arrayexpress/) databases. For the drug-repositioning study, two non-small-cell lung carcinoma (NSCLC) sample datasets [Control samples of human cell lines HepG2 for liver cancer, HCT116 for colorectal cancer and MCF7 for breast cancer from different laboratories were collected to build a background gene pairs  and used RMA (Robust Multichip Average)  for backA and B, is denoted as A > B (or A < B) if gene A has a higher (or lower) expression level than gene B . In a large collection of control samples for a cancer cell line, gene pairs with significantly stable REOs are determined by a binomial test [k denotes the total number of control samples, s denotes the number of samples that have a certain REO pattern  in k normal samples, and pe (pe = 0.5 here) is the probability of observing one of two REO patterns in a normal sample by chance. For the multiple binomial tests, the P values are adjusted by the Benjamin and Hochberg method to control the false discovery rate (FDR) [In a sample, the REO of two genes, ial test  as follote (FDR) .k overlapping gene pairs among which s pairs show the same REO patterns, the consistency score is the ratio, s/k. The probability of observing a consistency score of s/k by chance is evaluated by the binomial distribution model [We defined a consistency score to quantify the consistency between two lists of stable gene pairs separately identified from two independent collections of control samples measured by different laboratories for a cancer cell line. For two lists of stable gene pairs, if there are on model .The OneComp method is a customized version of RankComp  adaptingA, if its expression level is lower (or higher) than the expression level of gene B in the customized background gene pairs but opposite in the treated sample, then this reversal gene pair is supposed to support the up-regulation (or down-regulation) of A in the treated sample. Fisher’s exact test is used to test the null hypothesis that the frequency of reversal gene pairs which support the up-regulation of gene A in the treated sample is not different from the frequency of reversal pairs which support the down-regulation of gene A in the treated sample. When the null hypothesis is rejected, gene A is judged to be up- or down-regulated in the treated sample compared with the control sample if the frequency of reversal pairs which support the up-regulation of gene A in the treated sample is higher or lower than the frequency of reversal pairs which support the down-regulation of gene A in the treated sample.The following steps are the same as RankComp . Brieflyhttps://github.com/pathint/reoa [The C-language implementation of OneComp and the original RankComp method is available at int/reoa .The performance of OneComp was evaluated based on three large datasets of HepG2, HCT116 and MCF7 cell lines. For each cell line, we collected a large dataset with several technical replicates of both the control and the drug-treated groups and identified DEGs with SAM. OneComp was applied to identify DEGs from the subsets of the large dataset, each with only a pair of control and treated samples, which were evaluated through comparing with the DEGs identified by SAM in the full large dataset. Those DEGs exclusively identified by OneComp were further evaluated according to the observed genes expression dysregulation directions (up- or down-regulations) between the treated and control samples in the subsets.k overlapping DEGs among which s genes have the same dysregulation directions (either up or down), the consistency score is the ratio, s/k. The probability of observing a consistency score of s/k by chance is evaluated by the binomial distribution model [The consistency score was also used to quantify the consistency between two lists of DEGs. For two lists of DEGs, if there are on model .k is the number of disease signature genes overlapping with the drug signature genes, among which s genes could be consistent with drug treatment, then the drug-disease reversal score is defined as (1 − s/k). The probability of observing a drug-disease reversal score by chance is evaluated by the binomial distribution model [We defined a drug-disease reversal score to reflect the therapeutic effectiveness of a drug to NSCLC based on the hypothesis that the dysregulation directions of the DEGs in the disease signature tend to be reversed after drug treatment if the drug is therapeutically effective to the disease . If k ison model .https://portals.broadinstitute.org/cmap/) for the definition of the score.In CMAP, without the preselection of DEGs induced by a drug treatment, a complex drug-disease scoring scheme is used to evaluate the connection between a drug and a disease, basically based on the same hypothesis as for the reversal score. Readers are advised to see the CMAP website   from 102 control samples available in GEO or ArrayExpress, defined as the background gene pairs   of the analyzed datasets see “”. This c6) Table . There aSimilar results were observed for the comparison studies on GSE7161 and GSE37820, the datasets for the HCT116 and MCF7 cell lines, respectively . The results showed that none of the mean drug-disease scores between NSCLC and phenformin were significant  value larger than 2 in both the datasets as the “query signature” of NSCLC. We used this “query signature” to search drugs which have links with NSCLC in the CMAP database through the CMAP portal website   Table . Similary) Table . The resP < 5%). For the stage IIIB to IV patients, four of the six scores support the therapeutic potential of the drugs  samples since LUAD is the main subtype of NSCLC. To evaluate the influence of disease stages on the drug-disease reversal score, we analyzed two groups of the NSCLC samples according to their therapeutic methods , 40, staP values, we explained the possible anti-NSCLC mechanisms of the two drugs, respectively, through gene ontology (GO) enrichment analysis and protein–protein interaction (PPI) network analysis. The 761 drug-target disease genes for metformin were enriched in six functional categories of GO , including cell cycle, DNA replication, chromosome condensation and other key cellular processes as described in Additional file P < 5% instead of FDR < 5%, 16 significant GO functional categories were found which include cell proliferation, cell cycle and other cellular processes as described in Additional file If the DEGs up- or down-regulated in the disease signature could be reversed by the drug treatment, they were defined as the drug-target disease genes for the drug. Based on the two list of drug-target disease genes for phenformin and metformin identified from MCF7, which showed the most significant We demonstrated that the REOs of gene pairs are highly stable in control samples measured in different laboratories for a particular cell line. Thus, it is feasible to build a stable REO background landscape for a particular cell line using a large collection of control samples previously measured in different laboratories. For applications to small-scale cell datasets, the stable REO background is customized by filtering out the REOs which are not kept in the control sample(s) of this dataset. Then, OneComp can be used to identify DEGs effectively with statistical control with the customized stable REOs background. Through the analysis on phenformin and metformin treated samples, we showed that the detected DEGs in small-scale cell data with only one or two technical replicates are valuable in the studies for drug repositioning.Nevertheless, there are also several inherent limitations to use the proposed method for the analysis of small datasets. Although the average consistency scores of the overlapping DEGs between every two technical replicates within GSE41326 are up to 98.20% which indicated that these DEGs with the same dysregulation directions should be true dysregulated genes, there are 42.54% of DEGs on average that were detected in only one technical replicate. The average consistency score of non-common DEGs is only 43.45% if we compare these DEGs identified in one subset with the observed expression difference directions in another subset. This suggests that approximately 32.46% of the overall dysregulated genes detected in one subset of GSE41326 may be unrelated to the biological state of our interest. Similar results were also observed for the GSE7161 and GSE37820 datasets , PC3 (for prostate cancer) and HL60 (for promyelocytic leukemia cancer) cell lines for each of the two biguanide drugs (phenformin and metformin) to infer the “connection” between the drugs and NSCLC. The underlying hypothesis behind the CMAP project is that there are similar (or significantly overlapped) changes in the gene expression profiles of cell lines obtained from different cancer types exposed to the same drug. Here, we observed that, based on the data for three types of cancer cell lines, two of the three reversal scores between phenformin and NSCLC and all of the three reversal scores between metformin and NSCLC were significant (P < 1%), supporting the therapeutic potential of both metformin and phenformin to NSCLC. This also indicates a high consistency between the drug-signatures from drug-treated MCF7, PC3 and HL60, partially supporting the above-mentioned hypothesis. Nevertheless, this hypothesis needs to be fully addressed by evaluating whether the drug-signatures identified from multiple cell lines for different diseases or the same disease are significantly overlapped if treated by the same drug.In this study, we revealed that REOs of genes pairs are highly reproducible in the control samples of a particular cell type. Based on this finding, we customized the RankComp method to the application scenario of identifying DEGs in small-scale cell line experiments with only one or two technical replicates. The method performed well in simulated small-scale cell datasets. Using the DEGs identified by OneComp between antidiabetic biguanide drugs (phenformin and metformin) treated samples and corresponding control samples, we built “connections” between biguanides and NSCLC. The “connections” are statistically significant and support biguanides as potential anti-tumor drugs for NSCLC, while the pattern-matching strategy adopted by CMAP missed these “connections”. Therefore, OneComp method is useful in drug repositioning studies by helping to find more hidden “connections” between drugs and diseases.Additional file 1: Table S1. Overlap of DEGs detected by SAM (FDR < 20%) and OneComp.Additional file 2: Table S2. The consistency of remained DEGs solely identified by OneComp.Additional file 3: Table S3. Drug-disease reversal scores of the drug repositioning for phenformin and metformin for LUAD in different stages.Additional file 4: Table S4. The result of GO gene ontology enrichment of the 761 reversed associated with metformin (FDR < 5%).Additional file 5: Table S5. The result of GO gene ontology enrichment of the 298 reversed associated with phenformin (P < 5%).Additional file 6: Figure S1. DEGs overlapping between every two paired technical replicates within GSE41326, GSE37820 and GSE7161. Blue pie represents the DEGs of one paired technical replicates, red pie represents the DEGs of another paired technical replicates. The overlap in the pie represent common DEGs detected in both the two paired technical replicates and the number in the brackets in the overlapping region represent the consistency score, which denotes the percentage of DEGs that display the same dysregulated direction between common DEGs detected in both the two paired technical replicates. The number in the brackets in the blue region represent the consistency scores by using the observed expression differences (up- or down-regulations) between paired treated and control technical replicates in red region as the benchmark to evaluate the DEGs of the blue region paired samples. Similarly, The number in the brackets in the red region represent the consistency scores by using the observed expression differences (up- or down-regulations) between paired treated and control technical replicates in blue region as the benchmark to evaluate the DEGs of the red region paired samples.


Gene expression is orchestrated by distinct regulatory regions to ensure a wide variety of cell types and functions. A challenge is to identify which regulatory regions are active, what are their associated features and how they work together in each cell type. Several approaches have tackled this problem by modeling gene expression based on epigenetic marks, with the ultimate goal of identifying driving regions and associated genomic variations that are clinically relevant in particular in precision medicine. However, these models rely on experimental data, which are limited to specific samples (even often to cell lines) and cannot be generated for all regulators and all patients. In addition, we show here that, although these approaches are accurate in predicting gene expression, inference of TF combinations from this type of models is not straightforward. Furthermore these methods are not designed to capture regulation instructions present at the sequence level, before the binding of regulators or the opening of the chromatin. Here, we probe sequence-level instructions for gene expression and develop a method to explain mRNA levels based solely on nucleotide features. Our method positions nucleotide composition as a critical component of gene expression. Moreover, our approach, able to rank regulatory regions according to their contribution, unveils a strong influence of the gene body sequence, in particular introns. We further provide evidence that the contribution of nucleotide content can be linked to co-regulations associated with genome 3D architecture and to associations of genes within topologically associated domains. Identifying a maximum of DNA determinants implicated in gene regulation will accelerate genetic analyses and precision medicine approaches by identifying key gene features. In that context decoding the sequence-level instructions for gene regulation is of prime importance. Among global efforts to achieve this objective, we propose a novel approach able to explain gene expression in each patient sample using only DNA features. Our approach, which is as accurate as methods based on epigenetics data, reveals a strong influence of the nucleotide content of gene body sequences, in particular introns. In contrast to canonical regulations mediated by specific DNA motifs, our model unveils a contribution of global nucleotide content notably in co-regulations associated with genome 3D architecture and to associations of genes within topologically associated domains. Overall our study confirms and takes advantage of the existence of sequence-level instructions for gene expression, which lie in genomic regions largely underestimated in regulatory genomics but which appear to be linked to chromatin architecture. The regulation of gene expression involves a plethora of DNA/RNA-binding proteins that bind specific motifs present in various DNA/RNA regulatory regions. At the DNA level, transcription factors (TFs) typically bind 6-8bp-long motifs present in promoter regions, which are close to transcription start site (TSS). TFs can also bind enhancer regions, which are distal to TSSs and often interspersed along considerable physical distance through the genome . The curpression –25. Sevepression –23. Specpression . Quante pression . In thatrevisiae .Here, we built a global regression model per sample to explain the expression of the different genes using their nucleotide compositions as predictive variables. The idea beyond our approach is that the selected variables (defining the model) are specific to each sample. Hence the expression of a given gene may be predicted by different variables in different samples. This approach was tested on several independent datasets: 2,053 samples from The Cancer Genome Atlas  and 3 ENCODE cell lines (RNA sequencing). When restricted to DNA features of promoter regions our model showed accuracy similar to that of two independent methods based on experimental data , 19. We http://www.univ-montp3.fr/miap/~lebre/IBCRegulatoryGenomics.RNA-seq V2 level 3 processed data were downloaded from the TCGA Data Portal. Our training data set contained 241 samples randomly chosen from 12 different cancers (20 cancerous samples for each cancer except 21 for LAML). Our model was further evaluated on an additional set of 1,270 tumors from 14 cancer types. We also tested our model on 582 TCGA microarray data. The TCGA barcodes of the samples used in our study have been made available at http://gdac.broadinstitute.org) using firehose_get. We collected data for 73599 isoforms in 225 samples of the 241 initially considered. All the genes and isoforms not detected (no read) in any of the considered samples were removed from the analyses. Expression data were log transformed.Isoform expression data  were downloaded from the Broad TCGA GDAC , we used GENCODE v24 annotations.TCGA isoform TSSs were retrieved from N in the regulatory sequence s, with N in {A, C, G, T} and l the length of sequence s, andNpM dinucleotide percentage in the regulatory sequence s, with N and M in {A, C, G, T} and l the length of sequence s.The nucleotide (n = 4) and dinucleotide (n = 16) percentages were computed from the different regulatory sequences where:w be a motif and s a nucleic acid sequence. For all nucleotide N in {A, C, G, T}, we denoted by P(N|wj) the probability of nucleotide N in position j of motif w obtained from the PWM, and by P(N) the prior probability of nucleotide N in all sequences.Motif scores in core promoters were computed using the method explained in  and Posiw at position i of sequence s is computed as follows:w| the length of motif w, si+j the nucleotide at position i + j in sequence s, The score of motif w for sequence s is computed as the maximal score that can be achieved at any position of s, i.e.:l the length of sequence s.The score of motif Models were also built on sum scores as:n scores . Taking DNA shape scores were computed using DNAshapeR . BrieflyR) by dividing the sum, for all genes, of the intersection length of enhancers with gene The coordinates of the enhancers mapped by FANTOM on the hg19 assembly  were conhttps://gdac.broadinstitute.org/). We used the genome-wide SNP array data and the segment mean scores. In order to assign a CNV score to each gene, the coordinates (hg19) of the probes were intersected with that of GENCODE v19 genes using Bedtools intersect [Processed data were downloaded from the firehose Broad GDAC . The hg19 cis-eQTL coordinates were converted into hg38 using UCSC liftover and further intersected with the different regulatory regions. We restricted our analyses to cis-eQTLs impacting their own host gene. We computed the density of cis-eQTL per regulatory region (R) by dividing the sum, for all genes, of the number of cis-eQTLs of gene i (eQTLsi) located in the considered region for gene i (Ri) by the sum of the lengths of this regulatory region for all genes:ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b150_GRCh37p13/BED/):The v6p GTex http://gdac.broadinstitute.org) using firehose_get. The coordinates of the methylation sites (hg18) were converted into hg38 using the UCSC liftover and further intersected with that of the core promoters (hg38). For each gene, we computed the median of the beta values of the methylation sites present in the core promoter and further calculated the median of these values in 21 LAML and 17 READ samples with both RNA-seq and methylation data. We compared the overall methylation status of the core promoters in LAML and READ using a wilcoxon test.Illumina Infinium Human DNA Methylation 450 level 3 data were downloaded from the Broad TCGA GDAC  to compute the Gini coefficient for 16,134 genes on the 16,294 considered in our model. Gini coefficient measures statistical dispersion and can be used to measure gene ubiquity: value 0 represents genes expressed in all sam- ples while value 1 represents genes expressed in only one sample. To compute Gini coefficient we used R package ineq. We then computed, for the 241 samples, Spearman correlation between Gini coefficients and model gene absolute errors. Similar analyses were performed with 1,897 FANTOM 5 CAGE libraries to compute the Gini coefficients for 15,904 genes.We used 8,556 GTEx RNA-seq libraries  .ℓ1-penalized regression problem of finding the vector coefficient β = {βi} in order to minimizeyc(g) is the centered gene expression for all gene g, i for gene g and ∑i |βi| is the ℓ1-norm of the vector coefficient β. Parameter λ is the tuning parameter chosen by 10 fold cross validation. The higher the value of λ, the fewer the variables. This is equivalent to minimizing the sum of squares with a constraint of the form ∑i |βi| ≤ s. Gene expression predictions are computed using coefficient β estimated with the value of λ that minimizes the mean square error. Lasso inference was performed using the function cv.glmnet from the R package glmnet [e−16), while linear model and random forest yielded similar results . Moreover, computing time for linear model was much lower than that of random forest. These results emphasize the merits of linear model such as LASSO in their interpretability and efficiency.We performed estimation of the linear regression model e glmnet . The LASe glmnet  and Rande glmnet . S1 Tablet al. [stabpath and stabsel from the R package C060 for glmnet models [We used the stability selection method developed by Meinshausen et al. , which iet al.  to set tt models .rpart package in R [Regression trees were implemented with the age in R . In ordeR function p.adjust [R hypergeometric test function phyper [We considered TADs mapped in IMR90 cells  containip.adjust . Second,n phyper . Correctn phyper .http://www.univ-montp3.fr/miap/~lebre/IBCRegulatoryGenomics.The matrices of predicted variables (log transformed RNA seq data) and predictive variables  as well as the TCGA barcodes of the 241 samples used in our study have been made available at y(g) is the expression of gene g, xi,g is feature i for gene g, e(g) is the residual error associated with gene g, a is the intercept and bi is the regression coefficient associated with feature i.We built a global linear regression model to explain the expression of genes using DNA/RNA features associated with their regulatory regions :http://www.univ-montp3.fr/miap/~lebre/IBCRegulatoryGenomics for the barcode list). For each dataset (i.e sample), a regression model was learned and evaluated. See The advantage of this approach is that it allows to unveil, into a single model, the most important regulatory features responsible for the observed gene expression. The relative contribution of each feature can thus be easily assessed. It is important to note that the model is specific to each sample. Hence the expression of a given gene may be predicted by different variables depending on the sample. Our computational approach was based on two steps. First, a linear regression model We first evaluated the contribution of promoters, which are one of the most important regulatory sequences implicated in gene regulation . We extret al. [Promoters are often centered around the 5’ most upstream TSS (i.e. gene start). However genes can have multiple transcriptional start sites. The median number of alternative TSSs for the 19,393 genes listed in the TCGA RNA-seq V2 data is 5 and only 2,753 genes harbor a single TSS . We theret al.  in the cWe noticed that incorporating the number of TSSs associated with each gene drastically increased the performance of our model . Multiplet al., who showed that CpG content is insufficient to encode promoter activity and that other features might be involved [Provided the importance of CpGs in promoter activity , we firsinvolved .We integrated TF motifs considering Position Weight Matrix scores computed in the core promoter and observed a slight but significant increase of the regression performance  and TFs tend to form clusters on regulatory regions . The binWe indeed observed a similar bias in the results obtained by TEPIC , a regreAdditional genomic regions were integrated into our model. We first thought to consider enhancer sequences implicated in transcriptional regulation. We used the enhancer mapping made by the FANTOM5 project, which identified 38,554 human enhancers across 808 samples . This maSecond we analyzed the contribution of regions defined at the RNA level, namely 5’UTR, CDS, 3’UTR and introns, which can be responsible for post-transcriptional regulations , 42–50 . For allWe used a forward selection procedure by adding one region at a time: (i) all regions were tested separately and the region leading to the highest Spearman correlation between observed and predicted expression was selected as the ‘first’ seed region, (ii) each region not already in the model was added separately and the region yielding the best correlation was selected (‘second region’), (iii) the procedure was repeated till all regions were included in the model. The correlations computed in a cross-validation procedure at each steps are indicated in We compared models built on ssDNA and dsDNA, and ssDNA-based models yielded better accuracy Because RNA-associated regions  had greater contribution to the prediction accuracy compared to DNA regions , we compared the accuracy of our model in predicting gene vs. transcript expression. We retrieved the normalized results for gene expression  and the matched normalized expression signal of individual isoforms  for 225 TCGA samples. Accordingly, we generated a set a predictive variables specific to each isoform see . We founIn the above sections, our complete model, built on 160 variables corresponding to 4 nucleotide and 16 dinucleotide rates in 8 distinct regions , was traet al. [We sought the main DNA features related to gene expression. The complete model built on all 8 regions (160 variables) selected ∼ 129 predictive variables per sample. We used the stability selection algorithm developed by Meinshausen et al.  to identWe next determined whether stable variables exert a positive (activating) or a negative (inhibiting) effect on gene expression. For each sample, we fitted a linear regression model predicting gene expression using only the standardized variables that are stable for this sample. The activating/inhibiting effect of a variable is then indicated by the sign of its regression coefficient: < 0 for a negative effect and > 0 for a positive effect. The outcome of these analyses for all variables and all samples is shown We observed that the accuracy of our model varied between cancer types . In ordee−7. Likewise analyses were performed with 1,897 FANTOM CAGE libraries [We further computed Gini coefficient for 16,134 genes using 8,556 GTEx libraries [cis expression quantitative trait loci (cis-eQTLs). Zhou et al. indeed showed that the effect of eQTL SNPs can be predicted from a regulatory sequence code learned from genomic sequences [cis-eQTLs preferentially affect DNA sequences at precise locations (e.g. TF binding sites) rather than global nucleotide composition (i.e. nucleotide/dinucleotide percentages used as variables in our model). We used the v6p GTEx release to compute the average frequencies of cis-eQTLs present in the considered genomic regions and directly linked to their host genes (cis-eQTLs (10 times less than any other regions), while containing comparable amount of SNPs  using 60 nucleotide/dinucleotide percentages computed in the 3 promoter segments (CORE+DU+DD) and 471 JASPAR2016 PWM maximum scores computed in the CORE segment (pink) or (ii) using 60 nucleotide/dinucleotide percentages computed in the 3 promoter segments (CORE+DU+DD) and 471 JASPAR2016 PWM sum scores computed in the CORE segment (green). All sequences were centered around the 2nd TSS and the 2 models were fitted on 16,294 genes for each of the 241 samples.(PDF)Click here for additional data file.S2 FigDinucleotide percentages (y-axis) along 140,604 DNA regions centered around GENCODE v24 TSSs ±2000 bp (the distance to TSS is shown in the x-axis). Dinucleotide combinations are represented as first nucleotide on left and second nucleotide on top. The promoter segmentation used in this study  is indic(PDF)Click here for additional data file.S3 FigWe considered 19,393 TCGA genes listed in TCGA and the TSSs annotated by GENCODE v24.(PDF)Click here for additional data file.S4 FigThe model is built using 20 variables corresponding to the nucleotide (4) and dinucleotide (16) percentages computed in the CORE promoter (red), DU (green) or DD (yellow) centered around the second TSS as predictive variables (green). Linear models are also built on the number of isoforms (dark pink) and the number of TSSs (dark blue). Finally models are built using the combinations of variables indicated. All different models were fitted on 19,393 genes for each of the 241 samples considered. The prediction accuracy was evaluated in each sample by evaluating the Spearman correlation coefficients between observed and predicted gene expressions. The correlations obtained in all samples are shown as violin plots. These two last plots underscored the importance of these two variables in predicting gene expression.(PDF)Click here for additional data file.S5 FigThe 19,393 genes listed in one LAML sample (TCGA.AB.2939.03A.01T.0740.13_LAML) (pink) and a subset of 11,359 genes with assigned FANTOM enhancers (green) were considered. The median expression of genes with assigned enhancers is greater than that of all genes (PDF)Click here for additional data file.S6 FigA: Models were built using nucleotide and dinucleotide percentages computed on dsDNA  or on ssDNA  in all the regulatory regions . The 2 models were fitted on 16,294 genes for each of the 241 samples. The prediction accuracy was evaluated in each sample by evaluating the Spearman correlation coefficients. B: Same analyses focusing on each of the indicated regions.(PDF)Click here for additional data file.S7 FigA: Models were built using different set of variables including nucleotide (4 x 8 regions), dinucleotide (16 x 8 regions) and/or trinucleotide (64 x 8 regions) percentages computed in all the regulatory regions . All different models were fitted on 16,280 genes for each of the 241 samples considered. The prediction accuracy was evaluated in each sample by evaluating the Spearman correlation coefficients. B: Models were built using nucleotide (4 x 8 regions) and dinucleotide (16 x 8 regions) percentages computed in all the regulatory regions and trinucleotide (64) percentages computed in each of the indicated region separately.(PDF)Click here for additional data file.S8 FigThe procedure is identical to that described in (PDF)Click here for additional data file.S9 FigA) and in an additional set of 1,270 samples corresponding to 14 different cancer types (B). The prediction accuracy was evaluated in each sample by evaluating the Spearman correlation coefficients between observed and predicted gene expressions. The correlations obtained in all samples of each data sets are shown as violin plots in A (training set) and B . The color code indicates the cancer types. The horizontal dashed lines indicates the median correlation .The model with 160 variables (20 (di)nucleotide rates in 8 regions) was built on 16,294 genes in 241 samples corresponding to the initial training set corresponding to 12 cancer types ((PDF)Click here for additional data file.S10 FigThe model with 160 variables (20 (di)nucleotide rates in 8 regions) was built on 9,791 genes in 582 samples with matched RNA-seq and microarray data. The prediction accuracy was evaluated in each sample by evaluating the Spearman correlation coefficients between observed and predicted gene expressions. The correlations obtained in all samples with RNA-seq- or microarray-built models are shown as violin plots.(PDF)Click here for additional data file.S11 FigCNV absolute segment mean scores were computed for each as explained in Materials and Methods section. Model prediction absolute error for each gene are given by our predictive model using nucleotide and dinuclotide percentages computed in all the regulatory regions. Models were fitted on 16,294 genes for each of the 234 on 241 samples having CNV TCGA data available. The median correlation for the 234 samples is -0.014.(PDF)Click here for additional data file.S12 FigA linear regression model was built, for each sample, on standardized stable variables only. The boxplots show absolute values of the corresponding coefficients in all samples for each variable considered. Color code as in (PDF)Click here for additional data file.S13 FigA: Regression tree leading to a group of genes well predicted in all samples. This tree has been learned on the sample TCGA.FC.A5OB.01A.11R.A29R.07_PRAD using all nucleotide composition in all regions. The red path defines a group of 996 genes which has low Lasso error in all samples and cancer types. This group was used for functional annotation Click here for additional data file.S1 TableEach model is fitted for each tumor, using all the variables over all regions (160 variables among 8 regulatory regions). First and second columns are median correlation and mean square error over all the tumors. The third column represents mean computing time per tumor (in minutes) on a standard laptop.(PDF)Click here for additional data file.S2 TableGenomic regions were ranked according to their contribution in predicting gene expression. First, all regions were tested separately. Introns yielded the highest Spearman correlation between observed and predicted expressions and was selected as the ‘first’ seed region. Second, each region not already in the model was added separately. 5UTR in association with introns yielded the best correlation and was therefore selected as the ‘second’ region. Third, the procedure was repeated till all regions were included in the model. The contribution of each region is then visualized starting from the most important (left) to the less important (right). The correlations computed at each steps are indicated.(PDF)Click here for additional data file.S3 TableThe procedure is identical to that described in (PDF)Click here for additional data file.S4 TableThe group of 996 genes is obtained by fitting a regression tree on the sample TCGA.FC.A5OB.01A.11R.A29R.07_PRAD using all the nucleotide composition in all regions. These genes are well predicted (mean error < 1st quartile) for all samples of different type cancers. This group of genes was further annotated using the DAVID functional annotation tool. Only the top 5 biological processes indicated by DAVID is shown. The GO term yielded by this analysis corresponded to general and widespread biological processes indicating that these genes likely corresponded to housekeeping genes.(PDF)Click here for additional data file.S5 TableThe group of 1,531 genes is obtained by fitting a regression tree on the sample TCGA.IB.7646.01A.11R.2156.07_PAAD using all the nucleotide composition in all regions. These genes are well predicted (mean error < 1st quartile) for all LGG and PAAD samples but not that of LAML, DBLC and LIHC. This group of genes was further annotated using the DAVID functional annotation tool. Only the top 5 biological processes indicated by DAVID is shown. The GO term “Nervous system development” indicates that these genes can be involved in specific biological processes.(PDF)Click here for additional data file.S6 Tablecis-eQTL per regulatory region by dividing the sum of cis-eQTLs intersecting with the region considered for all genes by the sum of the lengths of the same regulatory region of all genes. see We computed the density of (PDF)Click here for additional data file.S7 TableWe computed the density of SNPs per regulatory region by dividing the sum of SNPs intersecting with the region considered for all genes by the sum of the lengths of the same regulatory region of all genes. We only considered SNPs detected on chromosomes 1, 2 and 19. see (PDF)Click here for additional data file.S8 TableWe computed the density of enhancers per regulatory region by dividing the total length of the intersection between the enhancers and the region considered for all genes by the sum of the lengths of the same regulatory region of all genes. see (PDF)Click here for additional data file.


As interactions between the immune system and tumour cells are governed by a complex network of cell–cell interactions, knowing the specific immune cell composition of a solid tumour may be essential to predict a patient’s response to immunotherapy. Here, we analyse in depth how to derive the cellular composition of a solid tumour from bulk gene expression data by mathematical deconvolution, using indication-specific and cell type-specific reference gene expression profiles (RGEPs) from tumour-derived single-cell RNA sequencing data. We demonstrate that tumour-derived RGEPs are essential for the successful deconvolution and that RGEPs from peripheral blood are insufficient. We distinguish nine major cell types, as well as three T cell subtypes. Using the tumour-derived RGEPs, we can estimate the content of many tumours associated immune and stromal cell types, their therapeutically relevant ratios, as well as an improved gene expression profile of the malignant cells. Mathematical approaches can be used to assess immune cell composition from the tumour's bulk expression data. Here the authors optimise the CYBERSORT-based deconvolution algorithm by including cell type-specific reference gene expression profiles generated from tumour-derived single-cell RNA sequencing data. Unfortunately, only a subset of patients  show long-lasting responses post checkpoint blockade3. Combining prospective patient selection based on predictive response biomarkers (=precision medicine) and immunotherapy has the potential to further transform patient care. To date, it has been shown that location and abundance of immune cells are prognostic for predicting patient outcome on standard therapy5. In addition, for checkpoint inhibitors-like anti-PD1, anti-PDL1, and anti-CTLA4 agents, the presence of relevant T cell populations correlates with treatment efficacy6. Thus, it is likely that the key to predicting response to immunotherapy lies in the patient-specific immune cell composition at the site of the tumour lesion.Enhancing a patient’s immune response to cancer using immune checkpoint inhibitors is arguably the most exciting advance in the treatment of cancer in the past decadedeconvolution7. To date, deconvolution of bulk gene expression has been described and validated for haematological malignancies9, where RGEPs can be established from peripheral blood mononuclear cells (PBMCs). This approach has been applied theoretically to solid tumours10, but until recently it has been impossible to validate this extrapolation experimentally. It has been difficult to obtain RGEPs for cell types that are not available in the peripheral blood, such as endothelial cells (ECs) and cancer-associated fibroblasts (CAFs) and it remains unclear to which extent the gene expression profile of an immune cell changes upon tumour infiltration. With the advent of the single-cell RNA sequencing (scRNA-seq) technology, however, it is now possible to determine gene expression profiles for tumour-infiltrating immune cells, tumour-associated non-malignant cells, and individual tumour cells from the same solid tumour biopsy.In theory, it is possible to infer the immune, tumour, and stroma cell content of a solid tumour from its bulk gene expression profile if reference gene expression profiles (RGEPs) can be established for each tumour-associated cell type. Mathematically, this class of inverse problems is known as 11, to characterise the baseline immune cell gene expression we accessed data from PBMCs originating from four healthy subjects12 and last, we generated immune and tumour cell gene expression profiles from four ovarian cancer ascites samples in-house. In the following, we show that gene expression profiles from tumour-associated immune cells and from PBMCs differ substantially. Therefore, reference profiles obtained from PBMCs are insufficient to deconvolve the bulk profile of a melanoma tumour sample. We find that indication-specific immune cell RNA-seq profiles from different patients are sufficiently similar to each other to define a consensus profile for each cell type, and that these consensus profiles enable accurate deconvolution of bulk tumour profiles. Our results show that the generation of specific RGEPs is both necessary and sufficient to enable reliable estimation of tumour composition from bulk gene expression data. Our approach resolves tumour-associated cell types that cannot be estimated by RGEPs derived from PBMCs. We can identify nine different cell types including immune cells, CAFs, ECs, ovarian carcinoma cells and melanoma cells. In addition, RGEPs for immune cells can be used to estimate the unknown gene expression profiles of tumour cells from bulk gene expression data patient specifically. Our work emphasises the importance of generating RGEPs specific to each indication of interest.We collected and investigated RNA-seq gene expression profiles of more than 11,000 single cells from three distinct primary human tissue sources: To characterise cells associated with the tumour microenvironment we accessed data from 19 melanoma patients12; (2) data-set of 4645 tumour-derived single cells from 19 melanoma patient samples11 and an unpublished data-set of 3114 single cells from four ovarian cancer ascites samples. Single-cell RNA-seq data requires careful data processing and normalisation particularly when comparing data originating from different sources and sequencing technologies. To characterise the single cells and to illustrate genome wide similarities and differences in their gene expression profiles, we applied the dimensionality reduction technique t-distributed stochastic neighbour embedding (t-SNE)13. This is an unsupervised machine learning algorithm that places each single cell into a two-dimensional plane. Cells with gene expression profiles that are similar are placed close to each other and farther apart if they are more different. Figure 6. Although our methods can easily be extended to include additional cells and further subdivisions, we limited ourselves to the nine major cell types to benchmark our classification algorithm. As previously reported11 and shown in Supplementary Fig. 14. However, the microenvironment has a clear and quantifiable impact on gene expression. In the following, we will address the question if gene expression profiles based on PBMCs are good approximations for what is observed in the tumour microenvironment and how the PBMC-derived gene expression profiles impact the quality of deconvolution of bulk expression data.First, to investigate the extent to which gene expression profiles change as immune cells move from peripheral blood to the tumour microenvironment, we compared immune cell scRNA-seq profiles across three human data-sets: 1) data-set of 4000 single cells derived from peripheral blood of four healthy subjects data-set11. Also, we compared the predicted cellular composition for all ascites samples experimentally with fluorescence-activated cell sorting (FACS). As depicted in Fig. First, we observed that the frequency of each cell type appears to be distinct for each sample as depicted in Fig. The microenvironment-specific gene expression profiles of immune cells, as well as the true composition of a given sample can be obtained by scRNA-seq and can serve as ground truth to benchmark deconvolution approaches. We studied how the deconvolution results of bulk gene expression data, for instance of a melanoma sample, are affected by microenvironment-specific changes and by patient-to-patient variation. As benchmark for the deconvolution, we constructed artificial “bulk” gene expression data by aggregating all single-cell gene expression data for each of the 27 samples, as well as different sets of REGPs by different strategies for averaging over tissue sources and patients. We compare the inferred, a priori known cellular composition of a given sample using five different RGEPs  than for RGEP2 and RGEP3 or for CNTR1 and CNTR2 (Pearson correlation ρ ≥ 0.98). For RGEP1, due to the unavailable reference profiles for tumour-associated cell types the true proportion of unknown cells is larger than for the other RGEPs and the estimation quality is mediocre (Pearson correlation ρ = 0.65). For RGEP2 and RGEP3, as well as for CNTR1 and CNTR2, the true proportion of unknown cells is negligibly small. Correlation is not a good measure of accuracy in case the true proportion of cells is small. For RGEP1 the estimation performs well for T cells , B cells (Pearson correlation ρ = 0.99) and macrophages/monocytes (Pearson correlation ρ = 0.99). However, the accuracy improves further for all other settings (Pearson correlation ρ ≥ 0.99). For RGEP1 the estimation for DCs (Pearson correlation ρ = −0.04) is poor and mediocre for NK cells (Pearson correlation ρ = 0.78). The estimation for DCs improves considerably for RGEP2 (Pearson correlation ρ = 0.82) and RGEP3 (Pearson correlation ρ = 0.95). The estimation for DCs still improves slightly for CNTR1 (Pearson correlation ρ = 0.97) but reaches its maximum only for CNTR2 (Pearson correlation ρ = 1.00), indicating that gene expression of DCs is heavily dependent on the source of isolation, which is in agreement with the evidence that distinct subsets of DCs are highly specialised in the generation of immunity15. The estimation for NK cells improves slightly for RGEP2 (Pearson correlation ρ = 0.82) and reaches close to optimal in RGEP3 (Pearson correlation ρ = 0.95) compared to CNTR1 (Pearson correlation ρ = 0.96) and CNTR2 (Pearson correlation ρ = 1.00). For RGEP2 to CNTR2, estimates for the tumour-associated cell types  become available and are estimated accurately (Pearson correlation ρ ≥ 0.95). Interestingly, the estimation for the malignant cells does not improve much upon inclusion of patient-specific information, suggesting that deconvolution using consensus profiles is feasible. This is possible because the tumour cells are in general very different from the non-malignant cells which make their deconvolution easier  compared to the other settings (Pearson correlation ρ ~ 0.95), indicating that gene expression of those cell types is influenced by patient-specific microenvironment. Interestingly, when considering the distance from the bisection  for all RGEPs. For CD4+ and regulatory T cells, the estimation results using RGEP1 are only mediocre (Pearson correlation ρ = 0.63 and ρ = 0.43) but improve significantly for RGEP2 (Pearson correlation ρ = 0.87 and ρ = 0.94). This is also reflected in the ratios of Treg/CD4+, CD8+/Treg, and CD4+/CD8+ T cells that reach accurate estimations for RGEP2 . The estimation for all T cell subsets and ratios does not significantly improve for CNTR1 but does improve for CNTR2 (Pearson correlation ρ = 1.00), indicating that gene expression of T cells is influenced by the patient-specific microenvironment. In summary, deconvolution using consensus gene expression profiles based on indication-specific gene expression profiles (RGEP3) were sufficient to obtain reliable estimates of the cellular composition of the samples without requiring patient-specific data on the individual cell types. Deconvolution using gene expression profiles based on data from peripheral blood (RGEP1) or based on averages across all three data-sets/indications (RGEP2) was considerably less accurate. When considering the distance from the bisection  bulk data, we profiled three of the four ovarian cancer ascites samples additionally with bulk RNA sequencing and applied the deconvolution approach using RGEP3 to obtain estimates of the cellular composition of the samples. Further, using the same three samples, we quantified the cellular composition experimentally by FACS and by single-cell RNA sequencing followed by algorithmic cell-type classification. Figure 16. Using scRNA-seq data, we calculated the estimated tumour cell expression profiles for each patient sample and compared them to the true tumour cell profile . For example, neutrophils are not represented in scRNA-seq data, as they are difficult to isolate, highly labile ex vivo and therefore are difficult to preserve with current single-cell isolation methodsile Fig. . As someent Fig. . NotablyCellular heterogeneity is present in any biological sample. Single cell RNA-seq allows us to understand how cellular heterogeneity contributes to function or patient outcome. However, it is still much easier to obtain bulk gene expression data. The work presented here shows how deconvolution approaches can be applied to bulk gene expression data to infer cellular composition and to provide a tool to link cellular heterogeneity to biological function or drug response from bulk gene expression data. We show that with indication-type and cell type-specific RGEPs deconvolution methods like CIBERSORT can accurately estimate the cellular composition of a given biopsy sample and in addition give us more accurate information about the tumour cell gene expression profiles by eliminating contamination from non-malignant cells. This is most relevant if the tumour cell content ranges between 20 and 70%. Our work showcases the feasibility of deriving cleaner tumour cell gene expression profiles. The number of tumour samples in our study, however, is too limited to allow for representative biological conclusion on tumour cell biology in these patient samples.17. By combining well-established deconvolution algorithms with state-of-the-art single-cell RNA-seq data of tumour biopsies, we showed that indication-specific consensus profiles of immune, stromal, and tumour cells, obtained directly from the tumour microenvironment, can be used to obtain accurate estimates of the cellular composition of a given sample. Patient to patient variability, however, will continue to be a confounding factor for deconvolution methods. One strategy to address patient to patient variability is to analyse a large set of matched tumour and blood samples from different tumour types18 and to quantify the impact of patient to patient variability on the proposed deconvolution method. Such data set of matched tumour and blood samples is necessary to enable more extensive cross-validation studies to prove that indication-specific consensus profiles provide accurate results across a larger population of patients. Overall, we found that the origin and quality of the reference profiles play a more dominant role than the deconvolution algorithm or gene set that is used, although gene sets designed to address as many cellular subsets as possible are clearly needed for accurate estimations of cellular heterogeneity.Benchmarking different gene expression reference profiles and different deconvolution algorithms, we showed that the estimation accuracy is ultimately limited by the origin and quality of the RGEPs. RGEPs derived from PBMCs are insufficient to enable accurate deconvolution of tumour bulk gene expression dataWith the availability of public scRNA-seq data from PBMCs and melanoma samples, as well as the ability to generate scRNA-seq data ourselves, we found that the gene expression profiles of tumour-associated immune cells differ considerably from those of blood-derived immune cells. Despite this systematic modulation, we found that patient-to-patient differences do not confound the deconvolution of bulk expression data and that consensus reference profiles can be established for each cell type, including tumour cells, for each specific microenvironment and indication.We restricted our analyses to nine major cell types and three T cell subsets. Additional subdivisions can be added by defining these cell types in the scRNA-seq data-set and by choosing an appropriate gene set to enable these subdivisions. In practice, it is best to limit the number of subdivisions as much as possible, as more uncertainty is introduced when attempting to distinguish cell types with similar profiles. In our analysis, we see limitations of the approach for distinguishing between CD4+ (non-regulatory) T cell and regulatory T cells. As can be seen in Fig. 19. It is not yet clear, however, how well these new protocols will work for tumour samples and the established protocols require the sample to be acquired and analysed within hours. Reference scRNA-seq data-sets from tumour samples can be obtained, but only in very controlled settings. With the appropriate data sets in place, however, deconvolution approaches enable routine clinical samples to be analysed both for cell content and patient-specific tumour cell gene expression profiles. Bulk RNA-seq data can easily be obtained from either flash-frozen or formalin-fixed, paraffin-embedded (FFPE) tissue samples, including both surgically resected material and core needle biopsies. The deconvolution approach presented here enables the estimation of immune cell content and improved tumour gene expression profiles in a clinical trial setting. The immune cell content of a tumour sample can also be determined by using more established multiplexed methods like immunohistochemistry (IHC) or immunofluorescence (IF)20 or newer methods like imaging mass cytometry using FFPE tissue samples21. The advantage of these techniques is that a larger number of cells can be analysed and that these techniques also provide information about the spatial distribution of the different cell types.Given that we obtained the best results using indication-specific and cell-specific RGEPs, it is likely that consensus reference profiles for immune and stromal cells will need to be established from scRNA-seq data for every solid tumour indication. Why, then, are these deconvolution approaches necessary? At this time, scRNA-seq experiments are difficult to perform in a routine clinical setting. Cryopreservation protocols are currently being developed that will help to make scRNA-seq data more easily accessibleHowever, these methods are limited to the number of proteins that can be analysed simultaneously currently (ranging from ~10 to 100), advantage of the deconvolution approach is that it is unbiased . It allows one to link both the cellular characteristics and the cellular content with treatment response. We anticipate that this approach will aid in the discovery of novel predictive response biomarkers for both conventional and immune-directed therapy by taking cellular composition into account.https://www.umassmed.edu/ccoe/core-services/tissue-and-tumour-bank/). Samples were obtained from patients that provided their informed consented to the general procedural that remnant material can be used for research purposes . Samples were shipped on ice on the same day and processed upon receiving. Each sample was filtered through a 70 μm filter and the cells were centrifuged down at ~300×g in a swing bucket centrifuge. Cells were re-suspended in PBS–5% FBS and counted. 1e7 viable cells were frozen down in 90% FBS + 10% DMSO and stored at −80 °C until use. Cryopreserved cells were thawed at 37 °C water bath. Cells were spun down and re-suspended in PBS-0.1% BSA and stored on ice. Viable cell number was measured, and cells were diluted to 1.6–2e5 cells per ml in PBS–0.1% BSA. About 3000 cells were encapsulated using the InDrops procedure24 at the Single Cell Core at Harvard Medical School. The libraries from about 1000 cells per sample were sequenced with the Illumina Nextseq 500 method at the Molecular Biology Core Facility at Dana-Farber Cancer Institute. Total RNA was isolated from the same samples using RNeasy kit (Qiagen) and sent to Beckman Genomics for sequencing library construction and Illumina RNA sequencing.Ovarian cancer ascites samples of four deidentified patients were obtained from The University of Massachusetts Cancer Center Tissue and Tumour Bank in Worcester, MA  scale as provided by current quantification methods28 byyi represents the gene expression profile of the ith sample, HKi, denotes the average gene expression over all housekeeping genes of the ith sample and 29.To ensure cross-sample comparability, all single-cell melanoma, PBMC, and ascites samples were normalised to the average expression of 3559 housekeeping genesAscites were stained with either anti-human CD45 , CD3 , CD4 , CD8 , CD25 , CD56 , CD127  CD16  Abs or with anti-human CD45 , CD1c , HLA-DR , CD14 , CD15  CD16  Abs in PBS 2% FBS for 20 min at 4 °C. The antibodies were purchased from BD Pharmingen or Biolegend. The samples were acquired on an LSRFortessa flow cytometer (Becton Dickinson) and analysed using FlowJo software.11, malignant and non-malignant cells were not treated separately, and only the scRNA-seq gene expression data were used for generating a training set. A workflow chart of our classification approach is depicted in Supplementary Fig. 30 was used to identify clusters based on the t-SNE map as shown in Fig. For classification of scRNA-seq data, a multi-step approach was developed. In contrast to the classification approach presented by Tirosh et al.The artificial “bulk” gene expression data that was used for testing the deconvolution approach was generated from single-cell RNA sequencing data by aggregating reads from all cell barcodes for each patient sample. As single-cell and conventional bulk sequencing differ in their quantification biases, we cannot assume that single-cell-based RGEPs are applicable for deconvolution of conventional bulk sequencing data. Therefore, to apply the deconvolution based on single-cell RGEPs, conventional sequencing must be adapted to closely mimic the quantification process in single-cell sequencing, however, without the cell barcoding that would be problematic in a clinical trial setting.32 and have been validated on blood samples9 or in vitro cell mixtures32. Detailed reviews on this topic are available33. A method called CIBERSORT was proposed and its performance was compared to previously existing methods8. Using blood samples of a total of 41 patients, the authors could show that CIBERSORT outperforms the other methods when comparing the deconvolution results to flow cytometry data. The authors of the study also released a set of 547 genes , which was used for their deconvolution approach. The CIBERSORT method (and most of the other methods mentioned above) assumes that the gene expression profile of an unknown bulk sample can be explained by the weighted sum of the cell type-specific profiles of which it is composed. The weights vector leading to the linear combination can be obtained by solving a linear equation system computationally. As biological data can be obscured by technical and biological variabilities, methods for deconvolution need to be robust against noise. The contamination of the sample with unknown cell types can be a further source of noise. A method called v-Support vector regression (v-SVR) combines feature-selection with a linear loss function and L2-regularisation34 and is therefore robust against noise33. The performance of deconvolution approaches has been widely demonstrated on in vitro mixtures and setting where the cellular gene expression profiles were directly measured, such as for peripheral blood mono-nuclear cell (PBMC) content in blood. Although there have been attempts to use RNA-seq data for deconvolution of cell mixtures35, so far, the accuracy of the approach has not been evaluated in a realistic setting and in a systematic manner. The potential of having absolute expression values from RNA-seq data rather than relative data from microarray has not been fully exploited.Computational approaches to decipher the relative immune cell content in the tumour environment from microarray or RNA-seq gene expression data have been proposedsignature gene set. The gene expression levels of these genes need to be informative enough to distinguish between cell types contained in the mixture/bulk sample. For our comparison, we chose five different signature gene sets. The LM22 gene set8 consists of 547 genes of which 496 are contained in the scRNA-seq PBMC, melanoma, and ascites data-sets. The Table S12 gene set11 contains 244 genes that are preferentially expressed in regulatory T cells of which 239 are present in all three data-sets. The Table S3 gene set11, a list of 391 genes that have been identified as differentially expressed among the cell types in scRNA-seq data. Therefore, 374 genes are contained across all three scRNA-seq data-sets. A Merged gene set, generated by merging all genes from the LM22, the Table S3 and the Table S12 gene sets and adding the 45 marker genes used for classification training. It consists of 1076 unique genes, with 1015 genes in common with the scRNA-seq data. An All genes gene set, consisting of 17,936 genes that are contained in the all three scRNA-seq data-sets with 17,933 non-zero genes for at least one single-cell profile.The basis for an accurate deconvolution is the choice of the v-SVR we used the implementation of libSVM36 for MATLAB . The parameters were set to “-s 4 –t0 –n0.50 –h0 –c 1 -q”. The mldivide function from MATLAB uses the pseudo-inverse of the matrix B for solving for w = pinv(B)*m. This is equivalent to using w = mldivide. The fitlm function from MATLAB fits a linear model to the data based on a least-squares fit. The main difference to the mldivide function is that for fitlm an intercept is considered. For the CVX package for MATLAB37 use used the setting lambda = 1 and solved using the SDPT3 algorithm38 for semidefinite-quadratic-linear programming problems  of our deconvolution results and calculated the median and lower and upper quartiles.To assess the quality of our deconvolution results, we compared the true cellular fractions, as calculated from the number of single-cell measurements for each cell type and patient, with the estimation result by calculating Pearson’s correlation coefficient 13. For removing noise and improving the performance, a principle component analysis can be used to reduce the initial dimensionality before running t-SNE. We used the Barnes–Hut implementation of t-SNE39 with the default settings to analyse our data. The result is a map that reflects the similarities between the high-dimensional input data as depicted in Fig. To obtain a low-dimensional representation of high-dimensional data, dimensionality reduction methods can be applied. t-SNE enables the reduction from many to two dimensions while keeping local neighbourhoodsTo calculate the gene expression profile of an average tumour cell for each individual patient, we need to subtract the explained portion of gene expression from the patient’s bulk sample gene expression profile and rescale the expression with the estimated tumour proportion, i.e.https://figshare.com/s/711d3fb2bd3288c8483a. The corresponding data-sets can be obtained using the download scripts provided in the source code repository.Source code for running the benchmark study and producing the figures is available at https://www.ncbi.nlm.nih.gov/gds) under accession number GSE72056 in a pre-processed format. Single-cell RNA-seq data of PBMCs from patient blood samples were downloaded from the 10x Genomics website ”12) and 1000 random cells were selected randomly for each donor to ensure similar size as for the melanoma and ascites data-sets. The ovarian cancer ascites data has been deposited together with the source code and is available at https://figshare.com/s/711d3fb2bd3288c8483a. The raw Ovarian Cancer ascites RNA-seq read data has not been submitted to a public repository due to privacy concerns, but can be made available upon reasonable request to the corresponding author pending appropriate approval from study participants. The authors declare that all the other data supporting the findings of this study are available within the article and its supplementary information files and from the corresponding author upon reasonable request.Single-cell melanoma data were obtained from Gene Expression Omnibus (GEO; Supplementary InformationPeer Review File


As single-cell RNA sequencing (scRNA-seq) technologies have rapidly developed, so have analysis methods. Many methods have been tested, developed, and validated using simulated datasets. Unfortunately, current simulations are often poorly documented, their similarity to real data is not demonstrated, or reproducible code is not available. Here, we present the Splatter Bioconductor package for simple, reproducible, and well-documented simulation of scRNA-seq data. Splatter provides an interface to multiple simulation methods including Splat, our own simulation, based on a gamma-Poisson distribution. Splat can simulate single populations of cells, populations with multiple cell types, or differentiation paths.The online version of this article (doi:10.1186/s13059-017-1305-0) contains supplementary material, which is available to authorized users. The first decade of next-generation sequencing has seen an explosion in our understanding of the genome . In partAs scRNA-seq data have become available there has been a rapid development of new bioinformatics tools attempting to unlock its potential. Currently there are at least 120 software packages that have been designed specifically for the analysis of scRNA-seq data, the majority of which have been published in peer-reviewed journals or as preprints . The focMuch of the existing software focuses on assigning cells to groups based on their expression profiles (clustering) before applying more traditional DE testing. This approach is taken by tools such as SC3 , CIDR 66, and SeAnother defining characteristic of scRNA-seq data is its sparsity; typically expression is only observed for relatively few genes in each cell. The observed zero counts have both biological (different cell types express different genes) and technical (an expressed RNA molecule might not be captured) causes, with technical zeros often referred to as “dropout”. Some analysis methods  and produces a series of diagnostic plots comparing aspects of scRNA-seq data. The combined datasets are also returned, making it easy to produce additional comparison plots or statistics. Alternatively, one SCESet can be designated as a reference, such as the real data used to estimate parameters, and the difference between the reference and the other datasets can be assessed. This approach is particularly useful for comparing how well simulations recapitulate real datasets. Examples of these comparison plots are shown in the following sections.Splatter provides implementations of our own simulation model, Splat, as well as several previously published simulations. The previous simulations have either been published as R code associated with a paper or as functions in existing packages. By including them in Splatter, we have made them available in a single place in a more accessible way. If only a script was originally published, such as the Lun  and Lun The negative binomial is the most common distribution used to model RNA-seq count data, as in the edgeR  and DESePublished in “Pooling across cells to normalize single-cell RNA sequencing data with many zero counts” , the LunIn “Overcoming confounding plate effects in differential expression analyses of single-cell RNA-seq data”  Lun and The scDD package aims to test for differential expression between two groups of cells but also more complex changes such as differential distributions or differential proportions . This isThe BASiCS package introduced a model for separating variation in scRNA-seq data into biological and technical components based on the expression of external spike-in controls . This moWe have developed the Splat simulation to capture many features observed in real scRNA-Seq data, including high expression outlier genes, differing sequencing depths (library sizes) between cells, trended gene-wise dispersion, and zero-inflation. Our model uses parametric distributions with hyper-parameters estimated from real data Fig. . The corMore specifically, the Splat simulation initially samples gene means from a Gamma distribution with shape The library size  varies within an scRNA-seq experiment and can be very different between experiments depending on the sequencing depth. We model library size using a log-normal distribution . We can also consider the relationship between the expression level of a gene, calculated including cells with zero counts, and the percentage of zero counts in that gene. The Lun and scDD simulations produce too few zeros at low expression levels, while the Simple and Lun 2 simulations produce too many zeros at high expression levels. It is important to note that as the scDD simulation removes genes with more than 75% zeros prior to simulation this model can never produce genes with high numbers of zeros as shown in Fig. Although the analysis presented in Figs. For each dataset, we estimated parameters and produced a synthetic dataset as described previously. We then compared simulations across metrics and datasets by calculating a median absolute deviation (MAD) for each metric. For example, to get a MAD for the gene expression means, the mean expression values for both the real data and the simulations were sorted and the real values were subtracted from the simulated values. The median of these absolute differences was taken as the final statistic. To compare between simulations, we ranked the MADs for each metric with a rank of one being most similar to the real data. Figure Looking across the metrics and datasets we see that the Splat simulations are consistently highly ranked. In general, it seems that the datasets are not zero-inflated and thus the zero-inflated simulations do not perform as well as their regular counterparts. The Splat simulations were least successful on the Camp cerebral organoid and Engel T-cell datasets. The complex nature of the Camp data (many cell types) and the full-length protocols used by both may have contributed to Splat’s poorer performance. In this situation the semi-parametric, sampling-based models may have an advantage and the Lun 2 simulation was the best performer on most aspects of the Camp data. Interestingly, the Simple simulation was the best performer on the Engel dataset. This result suggests that the additional features of the more complex simulations may be unnecessary in this case or that other models may be more appropriate. The Lun simulation is consistently among the worst performing. However, given that this model is largely similar to the others, it is likely due to the lack of an estimation procedure for most parameters rather than significant problems with the model itself. The scDD simulation also often differed significantly from the real data, which is unsurprising as this simulation is designed to produce a filtered dataset, not the raw datasets used here. A comparison based on a filtered version of the Tung dataset, showing scDD to be a better match, is provided in Additional file Most importantly we see that simulations perform differently on different datasets. This emphasizes the importance of evaluating different models and demonstrating their similarity to real datasets. Other comparisons may also be of interest for evaluation, such as testing each simulated gene to see if it matches known distributions, an example of which is shown in Additional file: 1 Figure S14. The Splatter framework makes these comparisons between simulation models straightforward, making it easier for researchers to choose simulations that best reflect the data they are trying to model.The simulation models described above are sufficient for simulating a single, homogeneous population but not to reproduce the more complex situations seen in some real biological samples. For example, we might wish to simulate a population of cells from a complex tissue containing multiple mature cell types or a developmental scenario where cells are transitioning between cell types. In this section, we present how the Splat simulation can be extended to reproduce these complex sample types Fig. .Fig. 5ExSplat can model samples with multiple cell types by creating distinct groups of cells where several genes are differentially expressed between the different groups. Previously published simulations can reproduce this situation to some degree but are often limited to fixed fold changes between only two groups. In the Splat simulation, however, differential expression is modeled using a process similar to that for creating expression outliers and can be used to simulate complex cell mixtures. Specifically a multiplicative differential expression factor is assigned to each gene and applied to the underlying mean. For DE genes, these factors are generated from a log-normal distribution while for other genes they are equal to one. Setting the number of groups and the probability that a cell comes from each group allows flexibility in how different groups are defined. Additionally, parameters controlling the probability that genes are differentially expressed as well as the magnitude and direction of DE factors can be set individually for each group. The resulting SCESet object contains information about which group each cell comes from as well as the factors applied to each gene in each group Fig. .A common technical problem in all sequencing experiments is batch effects, where technical variation is created during sample collection and preparation. The Splat simulation can model these effects using multiplicative factors that are applied to all genes for groups of cells. Adding this extra layer of variation allows researchers to evaluate how methods perform in the presence of unwanted variation Fig. .A common use of scRNA-seq is to study cellular development and differentiation. Instead of having groups of mature cells, individual cells are somewhere on a continuous differentiation path or lineage from one cell type to another. To model this, the Splat simulation uses the differential expression process described above to define the expression levels of a start and end cell for each path. A series of steps is then defined between the two cells types and the simulated cells are randomly assigned to one of these steps, receiving the mean expression levels at that point. Therefore, the simulation of lineages using Splat is defined by the differential expression parameters used to create the differences between the start and end of each path. It also incorporates the parameters that define the path itself, such as the length (number of steps) and skew (whether cells are more likely to come from the start or end of the path).In real data it has been observed that expression of genes can change in more complex, non-linear ways across a differentiation trajectory. For example, a gene may be lowly expressed at the beginning of a process, highly expressed in the middle and lowly expressed at the end. Splat models these kinds of changes by generating a Brownian bridge  between the two end cells of a path, which is then smoothed and interpolated using an Akima spline , 32. ThiFurther complexity in simulating differentiation paths can be achieved by modeling lineages with multiple steps or branches. For example, a stem cell that differentiates into an intermediate cell type that then changes into one of two mature cell types. These possibilities are enabled by allowing the user to set a starting point for each path Fig. .k-means-based approach available from Bioconductor [To demonstrate how the simulations available in Splatter could be used to evaluate an analysis method we present an example of evaluating a clustering method. SC3  is a cononductor . As wellngs Fig. . We alsongs Fig. . This prFigure While it is beyond the scope of this paper, clearly this evaluation could be extended, for example, by including more clustering methods, more variations in simulation parameters, and investigating why particular results are seen. However, these data, and the code used to produce them, are an example of how such an evaluation could be conducted using the simulations available in Splatter.The recent development of single-cell RNA sequencing has spawned a plethora of analysis methods, and simulations can be a powerful tool for developing and evaluating them. Unfortunately, many current simulations of scRNA-seq data are poorly documented, not reproducible, or fail to demonstrate similarity to real datasets. In addition, simulations created to evaluate a specific method can sometimes fall into the trap of having the same underlying assumptions as the method that they are trying to test. An independent, reproducible, and flexible simulation framework is required in order for the scientific community to evaluate and develop sophisticated analysis methodologies.Here we have developed Splatter, an independent framework for the reproducible simulation of scRNA-seq data. Splatter is available as an R package from Bioconductor, under a GPL-3 license, and implements a series of simulation models. Splatter can easily estimate parameters for each model from real data, generate synthetic datasets and quickly create a series of diagnostic plots comparing different simulations and datasets.As part of Splatter we introduce our own simulation called Splat. Splat builds on the gamma-Poisson  distribution commonly used to represent RNA-seq data, and adds high-expression outlier genes, library size distributions, a mean-variance trend, and the option of expression-based dropout. Extensions to Splat include the simulation of more complex scenarios, such as multiple groups of cells with differing sizes and levels of differential expression, experiments with several batches, or differentiation trajectories with multiple paths and branches, with genes that change in non-linear ways.We performed an evaluation of the six simulation models currently available in Splatter by comparing synthetic data generated using estimated parameters to five published datasets. Overall Splat performed well, ranking highly on most metrics. However, other simulations performed better for some metrics or better reproduced specific datasets. We found the Camp cerebral organoid dataset the most challenging to simulate, perhaps because of the complex nature of this sample, which is comprised of many different cell types. In addition, this dataset  used a full-length protocol, which may contain additional noise compared to the UMI datasets .One of the key features of scRNA-seq data is the high number of zero counts where no expression is observed for a particular gene in a particular cell. This can be especially challenging to simulate as not only must there be the correct number of zeros but they must be correctly distributed across genes and cells. We found that introducing dropout (in Splat) or zero-inflation (in Lun 2) often failed to improve the match to real datasets, suggesting that they are not truly zero-inflated. Together, the results demonstrate that no simulation can accurately reproduce all scRNA-seq datasets. They also emphasize the variability in scRNA-seq data, which arises from a complex set of biological  and technical  factors. Non-parametric simulations that permute real data could potentially produce more realistic synthetic datasets but at the cost of flexibility in what can be simulated and knowledge of the underlying parameters.Finally, we demonstrated how Splatter could be used for the development and evaluation of analysis methods, using the SC3 clustering method as an example. Splatter’s flexible framework allowed us to quickly generate multiple test datasets, based on parameters from real data. The information returned about the simulations gave us a truth to test against when evaluating the method. We found that SC3 accurately clustered cells and was precise in identifying DE and marker genes.The simulations available in Splatter are well documented, reproducible, and independent of any particular analysis method. Splatter’s comparison functions also make it easy to demonstrate how similar simulations are to real datasets. Splatter provides a framework for simulation models, makes existing scRNA-seq simulations accessible to researchers and introduces Splat, a new scRNA-seq simulation model. As more simulation models become available, such as those replicating newer technologies including k-cell sequencing, they can be adapted to Splatter’s framework. The Splat model will continue to be developed and may, in the future, include additional modules such as the ability to add gene lengths to differentiate between UMI and full-length data. We hope that Splatter empowers researchers to rapidly and rigorously develop new scRNA-seq analysis methods, ultimately leading to new discoveries in cell biology.To easily generate a simulation that is similar to a given dataset, Splatter includes functions to estimate the parameters for each simulation from real datasets. Just as with the simulation models themselves, the estimation procedures are based on what has been published and there is variation in how many parameters can be estimated for each model. We have given significant attention to estimating the parameters for the Splat simulation. The parameters that control the mean expression of each gene . The Salmon [http://imlspenticton.uzh.ch:3838/conquer/) and converted to a gene by cell matrix using the tximport [Each of the real datasets used in the comparison of simulations is publicly available. Raw FASTQ files for the Camp dataset were downloaded from SRA (accession SRP066834) and processed using a Bpipe (v0.9.9.3)  pipelinee Salmon  quantifitximport  package.For each dataset the data file was read into R v3.4.0)  and conv.0  and cp values less than 0.05. True DE genes were defined as genes where the simulated DE factor was not equal to 1 in one or more groups. Marker genes were defined as genes where the DE factor was not equal to 1 in a single group . Clustering metrics were calculated using the clues R package [Parameters for Splat simulations used in the example evaluation of SC3 were estimated from the Tung dataset. Twenty synthetic datasets were generated using these parameters with different random seeds. Each simulation had three groups of different cells, with probabilities of 0.6, 0.25 and 0.1, and a probability of a gene being differentially expressed of 0.1. Factors for differentially expressed genes were generated from a log-normal distribution with location parameter equal to –0.1 and scale parameter equal to 0.3. For each simulation the SC3 package was used to cluster cells with  package . To eval package .https://github.com/Oshlack/splatter-paper under an MIT license.Session information describing the packages used in all analysis steps is included as Additional file Additional file 1: Figures S1–S17Diagrams of other simulation models, Splatter comparison output for all datasets, example non-linear gene, dispersion estimate correction, mean-zeros fit, benchmarking, and processing times (PDF 17991 kb)Additional file 2:Table of the median absolute deviations used to produce Fig. Additional file 3:Session information. Details of the R environment and packages used for analysis. (PDF 118 kb)


Cq). When RT-qPCR does not reach the limit of detection, the Cq is labeled as “undetermined”. Current state of the art qPCR data analysis pipelines acknowledge the importance of normalization for removing non-biological sample to sample variation in the Cq values. However, their strategies for handling undetermined Cq values are very ad hoc. We show that popular methods for handling undetermined values can have a severe impact on the downstream differential expression analysis. They introduce a considerable bias and suffer from a lower precision. We propose a novel method that unites preprocessing and differential expression analysis in a single statistical model that provides a rigorous way for handling undetermined Cq values. We compare our method with existing approaches in a simulation study and on published microRNA and mRNA gene expression datasets. We show that our method outperforms traditional RT-qPCR differential expression analysis pipelines in the presence of undetermined values, both in terms of accuracy and precision.Reverse transcription quantitative polymerase chain reaction (RT-qPCR) is considered as the gold standard for accurate, sensitive, and fast measurement of gene expression. Prior to downstream statistical analysis, RT-qPCR fluorescence amplification curves are summarized into one single value, the quantification cycle ( Cq. Common procedures for calculating Cq-values are based on the number of cycles needed to cross a certain threshold, or on second derivatives of the amplification curve . Therefve [e.g. ]. If a tα, which can lead to an increased false positive rate or a reduced power. Within a microarray context, it has been shown that error propagation can improve the accuracy of the differential expression analysis substantially. But, even when error propagation is provided, the analysis can still be suboptimal: each step in a modular approach is optimized individually without taking the previous and the future analysis steps into account  (qbasePLUS version 2.4). This gives a biological justification for the choice of 39. With a LOD of 39, 2.5% of the observations are censored. Differential expression analysis between HR and LR was performed by the UCNR method.The LOD is set at 39. This choice is based on the application of a data-driven LOD selection criterion to the COG data see . Since t10 transformed p-values from the UCNR p10 values . Since UCNR correctly controls for the type I error, the method thus guarantees a higher sensitivity. The UCNR detects 43 out of the 58 genes as differentially expressed  between the HR and LR group. A full listing of the analysis results is available as supplementary material . The anaCq-values. It performs normalization and differential expression analysis simultaneously while providing a statistical rigorous way for handling undetermined Cq values (UV). Generalized Wald tests are used for assessing differential expression and the model parameters of interest have an interpretation in terms of log2 fold changes.We present a unified censored normal regression (UCNR) model for assessing differential expression in qPCR experiments. The method acts on the raw Ignoring censoring will generally lead to inconsistent estimators . Figs 1 δi| = 2) is illustrated in Figs p-values and estimates of the log2 fold change remain more stable with increasing degree of censoring with the UCNR method than with the other analysis methods. Figs 2 fold change estimates and low power to detect differential expression when censoring reaches levels so that few samples with concentrations above the LOD remain. For microRNAs containing less censored observations, or when the microRNA is not differentially expressed, UCNR still outperforms the other methods, but the differences between the methods are smaller  or type I error  occurred, is used to measure the robustness of the test. For individual microRNAs, the proportions resulting from our model are never smaller and mostly larger than for the classical analyses, indicating its robustness .p-values. Classical sequential analyses are thus found to be more conservative than the unified method with reference genes. Note that the unified method also correctly controls the type I error rate. Hence, the power gain does not come at the expense of false positives.The UCNR method extends the common normalization strategies, such as modified global mean normalization and the usage of reference genes or genes resembling the mean. Classical approaches first normalize the data to remove technical variability and statistical analyses are conducted on the normalized data. As a result, the standard errors of the fold change estimates are incorrectly estimated. Our method accounts for estimating the normalization factor. When normalizing using reference genes, the unified model uses the reference genes throughout the full analysis, resulting in a larger sample size and more degrees of freedom. This also affects the p-values can be obtained from a sequential procedure by applying an adjustment factor and corrected degrees of freedom. Note, however, that after these adjustments, the sequential approaches will still suffer from lower accuracy and precision in the presence of UV.Large scale experiments with many targets being measured are less vulnerable for incorrect standard errors estimates, even when a sequential approach is applied. From a theoretical perspective, correct standard error estimates and MYCN amplified (MNA) and MYCN single copy (MNSC) tumor samples. We compared the results for the MNA upregulated miR-17-92 cluster with the results obtained by the sequential analyses using multiple t-tests after MOD normalization and imputation of UV according to LOD and MNV+1 . If the UV results from technical error such as failure of amplification rather than a concentration below the LOD, an additional assumption of this technical failure being random  is needed.Since the UCNR model relies on a normal distributed process, the method is thus only applicable when the assumption of normality is not violated. The same holds for analyses with multiple k-group designs) or can be extended by including one or more confounder variables. The method guarantees to correctly account for the normalization which is simultaneously performed with the estimation.Since the UCNR model is basically a linear regression model, it can also be adapted to more complex study designs  without requiring ad hoc imputation algorithms. The model integrates the normalization procedure within the statistical analysis. We showed that the estimator and hypothesis tests are robust in the presence of UV and that our method outperforms popular imputation methods in terms of accuracy and precision.We proposed a unified censored normal regression (UCNR) model for analyzing differential expression in qPCR experiments. The model acts on the raw S1 Figt-tests with MOD normalization and LOD imputation of the UV (red dashed line), multiple t-tests with MOD normalization and MNV+1 imputation (blue dotted line) and multiple t-tests with MOD normalization and KNN imputation (black dotted-dashed line). A bias closer to zero suggest more accurate estimates. A small RMSE indicate a high precision of the estimator. The sharp jumps in the curves happen when a complete miRNA gets censored, which heavily affects the normalisation constants.At the bottom the grey circles indicate the removal of a complete miRNA (as a consequence of censoring). The numbers on top of some of the grey circles represent the number of remaining miRNA in the study. Estimators are obtained by UCNR (green solid line), multiple (PDF)Click here for additional data file.S2 Figt-tests with MOD normalization and LOD imputation of the UV (red dashed line), multiple t-tests with MOD normalization and MNV+1 imputation (blue dotted line) and multiple t-tests with MOD normalization and KNN imputation (black dotted-dashed line). A bias closer to zero suggest more accurate estimates. A small RMSE indicate a high precision of the estimator. The sharp jumps in the curves happen when a complete miRNA gets censored, which heavily affects the normalisation constants.At the bottom the grey circles indicate the removal of a complete miRNA (as a consequence of censoring). The numbers on top of some of the grey circles represent the number of remaining miRNA in the study. Estimators are obtained by UCNR (green solid line), multiple (PDF)Click here for additional data file.S3 Fig10(p)-values for the hypothesis tests: UCNR (green solid line) and Wilcoxon rank sum after MNV+1(blue dotted line), LOD (red dashed line) and KNN Click here for additional data file.S4 Figδi| = 2) are tracked during the simulation study. (a) Estimates of differential expression by UCNR (green solid line), multiple t-tests with MOD normalization and LOD imputation (red dashed line), MNV+1 imputation (blue dotted line) and KNN imputation (black dotted-dashed line). Censoring an observation at some point for this particular microRNA is marked by a black circle (MNA group) or a grey square (MNSC group) on the horizontal axis. (b) Plot of −log10p-values for the hypothesis test . (c) Box plot of differential expression estimates.Two differentially expressed microRNAs (true |(PDF)Click here for additional data file.S5 Figt-tests with MOD normalization and LOD imputation (red dashed line), MNV+1 imputation (blue dotted line) and KNN imputation (black dotted-dashed line). Censoring an observation at some point for this particular microRNA is marked by a black circle (MNA group) or a grey square (MNSC group) on the horizontal axis. (b) Plot of −log10p-values for the hypothesis test . (c) Box plot of differential expression estimates.Two non-differentially expressed microRNAs are tracked during the simulation study. (a) Estimates of differential expression by UCNR (green solid line), multiple (PDF)Click here for additional data file.S1 Tablep-values and adjusted p-values (using the Benjamini and Hochberg procedure) for testing for no differential expression. 43 out of the 58 microRNAs are differentially expressed at the 5% false discovery rate.The table shows parameter estimates and their standard errors (SE) for each of the 58 microRNAs, as well as the two-sided (PDF)Click here for additional data file.S2 Table2 fold change .The table shows the estimated log(PDF)Click here for additional data file.S3 Table2 fold change .The table shows the estimated log(PDF)Click here for additional data file.S1 Appendix(PDF)Click here for additional data file.