In addition to detecting novel transcripts and higher dynamic range, a principal claim for RNA-sequencing has been greater replicability, typically measured in sample-sample correlations of gene expression levels. Through a re-analysis of ENCODE data, we show that replicability of transcript abundances will provide misleading estimates of the replicability of conditional variation in transcript abundances . Heuristics which implicitly address this problem have emerged in quality control measures to obtain ‘good’ differential expression results. However, these methods involve strict filters such as discarding low expressing genes or using technical replicates to remove discordant transcripts, and are costly or simply ad hoc. As an alternative, we model gene-level replicability of differential activity using co-expressing genes. We find that sets of housekeeping interactions provide a sensitive means of estimating the replicability of expression changes, where the co-expressing pair can be regarded as pseudo-replicates of one another. We model the effects of noise that perturbs a gene’s expression within its usual distribution of values and show that perturbing expression by only 5% within that range is readily detectable (AUROC~0.73). We have made our method available as a set of easily implemented R scripts. RNA-sequencing has become a popular means to detect the expression levels of genes. However, quality control is still challenging, requiring both extreme measures and rules which are set in stone from extensive previous analysis. Instead of relying on these rules, we show that co-expression can be used to measure biological replicability with extremely high precision. Co-expression is a well-studied phenomenon in which two genes that are known to form a functional unit are also expressed at similar levels, and change in similar ways across conditions. Using this concept, we can detect how well an experiment replicates by measuring how well it has retained the co-expression pattern across defined gene-pairs. We do this by measuring how easy it is to detect a sample to which some noise has been added. We show this method is a useful tool for quality control. Recent analyses of RNA-seq have emphasized the value added by experimental designs with more samples, permitting better estimation of biological and technical variability –4. UnforSystematic and large scale studies assess new technologies for appropriate guidelines and best practices at great cost, time and effort. When microarrays began to be used for drug development, the Food and Drug Agency (FDA) set up the MicroArray Quality Control (MAQC) for general standards of practice in order to assure high quality and replicability . In a siExtensive work and funding were put into these efforts: evaluating terabytes of data, across numerous preparation conditions, platforms, sample types, experiment sizes, read depths, computational quantification methods, statistical measurements and other diagnostics. The projects successfully identified the strengths and weaknesses of various aspects of RNA-seq but, alarmingly, could not generalize these findings to a consensus gold-standard method or technique. Different platforms gave different biases, depending on the chemistry of the reactions, length of the reads, the amplification of the sample and method of degradation of ribosomal RNA, to name a few. These are reflected in underlying sequence-based biases from GC content, genome size and gene size.Many of these properties are addressed directly in quality control practices which have a strong history in microarray analyses and can be applied to RNA-seq without too much loss of generality. A notable example is batch detection and correction, long identified as critical in microarrays , and fo2 FC ~1–2). But these are general rules whose relevance must vary from case to case. How well do these heuristics generalize to carefully designed experiments conducted in an entirely different biological system?Similarly, spike-ins of known concentrations of RNA, such as those of the External RNA Control Consortium (ERCC) , can be In most individual labs, there will be little to no data to assess this question directly. Few labs can afford experimental designs which use samples purely as replicates for quality control. This is unfortunate, since often the experimental design is specifically intended to address noise in the biological system. There is no easy and systematic way of seeing whether this worked, outside of the experiment giving expected results, but it is unexpected results that are particularly valuable. Ideally, the results that are used for validation can provide a finely calibrated sense of performance without impinging on the capacity for novel discovery, but this will be challenging since the same data will likely be used in both cases. As a consequence of this, assessment of RNA-seq efficacy in any targeted experiment is typically difficult, relying on experts in the particular experimental system finding the results plausible (but not too obvious). The few truly generally applicable quality controls are extremely coarse, and focus more on technical properties, such as read depth, coverage, and related measures.Ideally, each lab could conduct their own RNA-seq quality control, relying either on costly replicates, or some sub-class of ‘expected’ result which does not diminish the freedom to obtain other unexpected results. Our approach to this problem is to focus on modelling the behavior of tightly co-expressed genes. Gene-pairs which are very tightly co-expressed can be thought of as a form of a pseudo-replicate and have the advantage of reflecting a broadly applicable biological property orthogonal to the tested variation in most studies, which are usually based on differential expression. Of course, to the extent we rely on ‘knowing’ pairs of genes are co-expressed to determine experimental success, we diminish our ability to find we were mistaken about this property. To deal with this issue, our analysis relies on weak relationships across many genes but no strong knowledge of any single gene and only a tiny fraction of gene-gene-pairs or ‘housekeeping interactions’.In this paper, we demonstrate a straightforward but surprisingly powerful method, which we have named AuPairWise, to measure replicability by modelling the effects of noise within observed co-expression. Because of the breadth and extent of co-expression, it provides a sensitive, yet general means of performing a quality control. We argue this in roughly three steps. We begin our analysis by performing some sample and replicate-based analyses of RNA-seq quality to establish general properties. Second, we show how common heuristics for quality control appear in a co-expression based framework. Finally we provide a means of directly quantifying this by disruption of known co-expression pairs using a model based on empirical distributions of expression data.https://github.com/sarbal/AuPairWise).By providing a direct quality control measure, we hope to make it straightforward to customize experiments to do better than a general heuristic—such as discarding all low-expressing genes–would normally permit. We make our software available in a convenient form for bioinformatic use   and the other on variation in gene expression (gene-level replicability). The earliest method is to assess whether a given pair of samples is well correlated across its transcripts or genes. This measures whether genes have similar expression in the two samples and by this standard, all RNA-seq has high enough replicability that we never need worry. Unfortunately, this doesn’t map to a very experimentally useful measure since most experiments are concerned with a gene’s variation in expression under changing conditions . To see why these two forms of replicability do not align, consider an early reference experiment  which rermation) . But thirmation) , and indrmation) .Rather than this sense of replicability, we are primarily concerned with the degree to which changes in expression for a given gene are replicable . While the importance of replicability in terms of differential expression has been appreciated from the first MAQC forward , it is d“Measuring replicability: correlations and sample estimates”). First, we show the gene-gene correlations for each sample , in which we consider whether a gene’s variation from condition to condition is replicable when the same conditions are re-measured. This is similar to the idea of checking if genes are reproducibly being detected as differentially expressed, i.e., if genes are DE in both replicates, but we are measuring correlations and not fold changes.T, or 78.2%—Fisher’s transformation).To assess this for each gene, we would like a range of conditions which we expect that gene to vary under, i.e., different tissues, varying time points or treatments. We would also like replicates for each of those conditions. With these, we can measure the degree to which, for each gene, its variation is distinguishable. One experiment that fits this general design was ENCODE’s RNA-seq profiling experiment (GSE35584), which looked at various biological cell lines under multiple treatments. We take ENCODE’s own labelling of the data into two batches to define the grouping of the data into sets of samples. ENCODE performed a number of perturbations likely to be genome-wide in effect. In this case, if a gene’s changed expression is meaningful, it should be correlated with its replicate across the range of conditions. However, not all genes are expected to show differential activity, and particularly if they are only modestly perturbed. For each gene, we can measure whether it does, in fact, show variation that is consistent when the same condition is measured . In the ENCODE data , we find after multiple test correction that over 16,000 genes were significantly correlated  with their replicates across the samples , giving “Measuring replicability: correlations and sample estimates” for more details). Similarly, we can assess the potential coverages of prior experiments based on their sample count, and, as a proxy, assess how replicable their results are for particular genes. Evaluating 1,451 publicly available experiments, we estimate that many experiments only have enough samples to produce significantly replicable results for a small fraction of the genome, with only 34% of experiments assessed having coverage above 50%  cover 90% of the transcriptome we would need to increase the number of samples to over 50, and for 95% coverage, over a hundred samples . This can be controlled for since such artifacts would generically improve the correlation between a gene and all others, rather than specifically itself in the replicate data.We calculated the correlation of each gene with all others in the replicate data to provide such reference information. We then calculated the rank of a gene’s correlation with itself versus all other genes within the experiment. If the variation in expression that a gene shows is meaningful, we would expect it to have a higher correlation to the replicate of itself than other genes. We do observe from the distribution of self-correlation  versus correlations to other genes  that genes are much better correlated to themselves in replicate data than to other genes  with almr = 0.97, SD 0.04). If we relax our definition slightly so that the gene has to rank its replicate within the top 10%, we see that 90% of the genes pass that threshold and that they have an average correlation of 0.9. We have used this mean value as a cutoff (Pearson's r = 0.9) to characterize ‘reasonable’ replicability for our further analysis of this study. This is more stringent than simple significant correlation; we are requiring it to be above the mean of genes with a non-parametric ‘preference’ to be correlated with themselves and close to the point nearest the top right corner (best performance). Of course, because actual biological co-expression exists (a fact we will exploit), pristine results in self-correlation are not to be expected and all downstream results are robust to any reasonable variation in this threshold. We discuss the potential role of normalization creating generic correlation across all genes in the “Robustness analysis of AuPairWise” section.The average correlation of the genes in the experiment is high  should be discarded from analyses , filtering on human samples and those in SRA as of January 2015. Of the resulting 39K samples, we then filtered on samples only in a single experiments , and then calculated the sample size for each of these. For a final follow up, we then obtained 83 RNA-seq experiments from the Gemma database [To illustrate our analyses, we used data from publically accessible RNA-seq experiments where we define an experiment as a set of samples. The majority of the work and analyses were performed using the following datasets: ENCODE GSE35584) , Marioni584 , Mardatabase  that werdatabase . These ers) or Pearson (r) correlation coefficient for each sample to its replicate  is equal to the correlation (Spearman’s rssi or Pearson’s ri) between the expression values of the gene in experiment X  and the replicate experiment Y , across n samples.Gene-gene correlations as a gene-level replication metricWe define an experiment as an ordered set of expression values for genes across multiple samples, with each sample defined as belonging to some class of biology from which RNA can be extracted. A replicate experiment is defined as an experiment across samples with identical class memberships, but varying on one of three other levels. This could be varying at a technological level , technical level , or a biological level . The ENCODE dataset were purely technical replicates. To measure the replicability of an experiment in sample-sample space, we took the expression levels of an experiment, and calculated the Spearman . After adjusting for multiple tests with p.adjust in R, we then calculated the number of genes replicated as the number of genes with a significant correlation value (q<0.05). To estimate the number of samples required to get 90% of the genes replicated, we assumed that the range of correlation values would remain the same, but varied the number of samples n (within the T-statistic estimate equation), and recalculated the adjusted p-values. This gives us, for each n, the number of genes that will be significantly correlated, assuming the distribution of correlation coefficients is the same.We used the R functions To compare our measures of replicability to the MAQC filters, we calculate the fold change  and the Fold change calculationSelecting co-expressed pairs”) that can be used as the biological indicator, and a noise model (section “Perturbation: the noise model”) to determine the strength of these pairs. As a general overview, the nature of the test is a way to calibrate the noise within the system, as it attempts to detect outliers where we have purposely generated them. The more detectable noise is, the lower the original noise in the data (without the perturbation). As there are outliers in the real data, those samples are precisely the ones where we can add a substantial amount of noise without detection. Thus, when we perturb out system with noise, we are making a statement about the data prior to the noise being included. The degree to which perturbations can go undetected is the degree to which the experiment is noisy.The overall idea behind the AuPairWise method is to leverage the biological signal of co-expressing pairs as a measure of replicability of an experiment. Therefore, the method requires a set of co-expressed pairs , correcting for multiple hypothesis testing using the Benjamini-Hochberg adjustment (p.adjust) in R [In order to better characterize what GO terms were enriched in our human co-expressed gene-pairs set, we ran a simple gene set enrichment analysis using the hypergeometric test (st) in R .We define an experiment as the expression levels of all genes across all samples for a range of conditions. Samples are not constrained to biological replicates, but should contain at least two conditions. Using the expression levels of each gene-pair, we calculated a score based on the residuals of the linear regression model fitting the points. Each point is a representative of a sample and its “expression replicate” (co-expression partner). Because we expect the expression levels of the genes to be “replicated”, there should be high correlation (the gene correlations across samples we defined earlier), and thus the residuals should be small, and “outliers” few and random. We rank these residuals, thus giving each sample a score. If a sample is perturbed, it should on average score the worst across all genes. We illustrate this in P is the rank of the held-out positive, TP the number of true positives, TN the number of true negatives, and N the total number of samples. If ranked correctly, we get an AUROC of 1. If ranked randomly, the AUROC value ranges between 0 and 1 and on average will be 0.5.AUROC calculationIn order to measure how robust the experiment was to noise, we then perturbed the system by randomly selecting a sample and added a relative amount of noise to all the genes in that chosen sample. We used a perturbation model that was a function of the dynamic range of a gene and the sample size of the experiment, such that a noise factor of 10 meant for an experiment with 100 samples the gene would vary within 10% of that gene’s expression values (as measured by the current experiment). First, we rank the expression profile of that gene  across the samples. We then select a sample to perturb at random . As we aWe used the AUROC as a measure of the performance of the experiment.As a proof of principle method for experiments where there were replicates available in the experimental design, we used the replicates and looked at every gene-paired with itself. In this case, we took the residuals of the gene replicates from the linear model, aggregating across all the genes (paired with their own replicate), and generating a ROC and calculating the AUROC. As described previously, we varied the amount of noise, repeating and permuting across the samples.For the two-fold validation, where we perturbed more than one sample, we performed the same experiment, selecting at random half the samples and shuffling the expression values of the genes according to the noise factor. In this case, we shuffled each sample independently, so that the shuffled samples did not affect the values of the remaining samples to be shuffled. The AUROC was calculated as above, and the experiment was repeated across the different noise factors, and across different random set of samples.https://github.com/sarbal/AuPairWise). We have briefly summarized the code below.For further information on how to the use the AuPairWise method and obtain the metrics, see the github repository Pseudocode for the AuPairWise methodS1 Text(DOCX)Click here for additional data file.S1 Fig(A) For the fold change values, we see that for the recommended FC cutoffs (the blue dotted lines), the mean correlations are low and increase the further we move (black line). Removing low expressing genes shows even higher mean correlations (grey line). (B) For the average expression values, once again, we see that for the recommended cut-off, we have higher mean correlations (black line), that are improved even more with the inclusion of the fold change filter (grey line).(TIF)Click here for additional data file.S2 FigHere we took 83 RNA-seq experiments and ran the AuPairWise method for 100 repeats across a range of noise factors. (A) The density plot (solid line) shows that most experiments fall within the 19% noise factor range as assessed to cause an AUROC shift to 0.8. There is a dependence on sample size of course, with larger experiments requiring smaller noise perturbations (data points in figure). (B) Significance of difference in performance AUROCs for biologically related gene-pairs compared to random pairs for all the experiments. The geometric mean of these p-values is 4.67e-06. For comparison, the ENCODE reference dataset with 18 samples sits with a p-value of 6.52-e14.(TIF)Click here for additional data file.S3 Fig(A) The co-expressed pairs (shown in red) outperform the protein-protein pairs (brown), the conserved yeast-mouse co-expressed pairs (pink) and random pairs (blue). These are the results of 1000 repeats. (B) Using different normalization methods on the expression data has a small effect on the performance AUROCs of the stoichiometric pairs. Here we compared the effects of RPKM (brown), TPM (blue), TMM (orange), VST (red), and simply ranking the expression data (green). These are the results of 1000 repeats.(TIF)Click here for additional data file.S4 FigThe distributions are for 1000 runs of the method on the same dataset, varying the sample being perturbed. The colored distribution is the co-expressed pair performances, while the lighter distribution is the performance of an equal number of random gene-pairs. The solid line is the average AUROC for the co-expressed pairs, the dashed for the random pairs. The p-value is the significance of the difference between these two distributions . Panels are for different noise factors (A) No noise added (p~0.85), (B) 10% noise (p~2.78e-5), (C) 25% (p~2.71e-112) and (D) 50% (p~2.76e-153). We see that although the average AUROCs may be similar, the distributions are significantly different.(TIF)Click here for additional data file.S5 FigHere we show the standard error (SE) of the performance AUROCs of the stoichiometric pairs as a function of the parameter choices on the ENCODE dataset, holding the sample size constant. (A) First we calculate the effect of the number of runs on the analysis, by varying the number of repeats from 10 to 1000. The SE decreases as the number of runs increases, as expected, with the noise factor having a small effect on the SE. (B) Varying gene-pair numbers (between 10 and 2500) shows a consistent SE only when 500 or more gene-pairs are used. To note, all variance is below 0.01 for almost all these parameter choices, indicating the results are quite robust. (C) We show the performances (average AUROCS) as a function of to the average number of genes within the sampled gene-pairs (and mapping between average number of genes and gene-pairs in inset).(TIF)Click here for additional data file.S6 FigUsing the ENCODE dataset once again and running the method 1000 times, we varied the number of samples perturbed in the experiment. The n-fold cross-validation  outperform 2-fold cross-validation . The 2-fold validation does not have an AUROC performance of 1 at 100% (~0.98), likely due to the fact that a great deal of co-expression is lost, hence preventing the method from detecting any structured co-variation.(TIF)Click here for additional data file.S7 FigThe co-expressed pairs (darker lines) outperform the random pairs (lighter lines), for both the RNA-seq (green) and Microarray (blue) datasets. But the microarray dataset has more similar performances between the co-expressed pairs and the random pairs, compared to the RNA-seq version. The microarray dataset also performs worse/close to the RNA-seq random pairs.(TIF)Click here for additional data file.S8 Fig(A) Expression of gene X from experiment. (B) The first step is to order the expression levels, and interpolate between the samples to get a continuous distribution of expression. (C) Then with this distribution, we define the boundaries for the noise included expression levels for that gene. For instance, with 10% noise, the new rank is limited to the yellow rectangle bounds, which corresponds to an expression change, constrained between expression values of ~1 to ~2 (the orange rectangle). (D) We then pick the new expression level and standardize it. (E) This is repeated until all the genes in the selected sample have “noise” added to them.(TIF)Click here for additional data file.S1 Table(XLSX)Click here for additional data file.S2 Table(XLSX)Click here for additional data file.S3 Table(XLSX)Click here for additional data file.S4 Table(XLSX)Click here for additional data file.S5 Table(XLSX)Click here for additional data file.


CDH1 and in the PI3K pathway are the most frequent molecular alterations in ILC. We identified two main subtypes of ILCs: (i) an immune related subtype with mRNA up-regulation of PD-L1, PD-1 and CTLA-4 and greater sensitivity to DNA-damaging agents in representative cell line models; (ii) a hormone related subtype, associated with Epithelial to Mesenchymal Transition (EMT), and gain of chromosomes 1q and 8q and loss of chromosome 11q. Using the somatic mutation rate and eIF4B protein level, we identified three groups with different clinical outcomes, including a group with extremely good prognosis. We provide a comprehensive overview of the molecular alterations driving ILC and have explored links with therapy response. This molecular characterization may help to tailor treatment of ILC through the application of specific targeted, chemo- and/or immune-therapies.Invasive lobular carcinoma (ILC) is the second most frequently occurring histological breast cancer subtype after invasive ductal carcinoma (IDC), accounting for around 10% of all breast cancers. The molecular processes that drive the development of ILC are still largely unknown. We have performed a comprehensive genomic, transcriptomic and proteomic analysis of a large ILC patient cohort and present here an integrated molecular portrait of ILC. Mutations in  Breast cancer is a heterogeneous disease and has traditionally been subdivided into distinct histological subtypes based on cell morphology. About 60–75% of breast cancers are invasive ductal carcinomas (IDC)1Gene expression-based molecular subtypes have been used as a reference to describe breast cancers511Treatment decisions made by oncologists for breast cancer are mainly based on results obtained in large trials, in which ILCs are only a minor subgroup. It is, therefore, not always the case that the conclusions from “breast cancer” trials also apply to ILC.www.ratherproject.com), we set out to improve the molecular characterization of ILCs by searching for potential molecular subtypes and oncogenic driver events. In addition, we aimed to understand the molecular events leading to different clinical outcomes. We collected a large cohort of 144 ILC patients with complete clinical data and long follow-up, and performed comprehensive molecular profiling of their primary tumour. The integration of multiple molecular data reveals two distinct molecular subtypes of ILC and provides new insights into the molecular factors associated with this disease.As part of the Rational Therapy for Breast Cancer (RATHER) consortium (To explore the biology of invasive lobular carcinomas (ILCs), we performed comprehensive molecular profiling of 144 untreated tissue samples from primary ILC tumours with 6.8 years median clinical follow-up  using (i) targeted DNA sequencing to study somatic variants on a set of 613 genes ; (ii) SNP6 arrays to study somatic copy number alteration (CNA) profiles; (iii) DNA microarrays to study gene expression and (iv) reverse-phase protein arrays (RPPA) to measure the expression of 168 selected proteins and phospho-proteins . For 131 samples (91% of samples profiled), we obtained DNA sequencing, CNA and gene expression data , 112 of Extensive stability analysis of clustering of the gene expression data using a variety of clustering approaches identified two robust expression subtypes . Based oPDCD1 (PD-1), CD274 (PD-L1) and CTLA4 (CTLA-4) are expressed at higher mRNA levels in the IR subtype . The first, referred to as Immune Related (IR), shows up-regulation of genes characteristic of cytokine/chemokine signalling. Representing the enriched pathways as nodes in an Enrichment Map subtype . The IR  subtype . In addi subtype , Fig. 1Fxpressed , suggestxpressed .ESR1) and progesterone (PGR) receptors  subtype, shows higher levels of oestrogen  on the TCGA and METABRIC samples. On the TCGA samples, the Reactive-like subtype is associated with the IR subtype, while Immune-related and Proliferative subtypes are associated with the HR subtype . On the METABRIC samples, the subtypes do not show this association  . The difCDH1 (42.8%), PIK3CA (34.8%), GATA3 (5.1%), MAP3K1 (5.1%), and AKT1 (5.1%) (CDH1 mutants show lower expression at both the mRNA (Wilcoxon p = 2.4e-5) and protein (Wilcoxon p = 8.9e-4) levels , NF1 (4.3%), ERBB2 (4.3%) and TP53 (3.6%) (AKT1 (5.1%), PIK3R3 (2.9%), PTEN (1.4%), PIK3CB (1.4%), PIK3CG (1.4%), PIK3CD (1.4%) and PIK3CA that tended to be mutually exclusive  levels . However3 (3.6%) . The PI3xclusive . This co subtype . GATA3 mantly so  and are We performed high-density SNP genotyping and identified recurrent copy number alterations (CNAs) using ADMIREPGR, GATA3 and FN1, which are up-regulated at the mRNA and RPPA level, corroborating our previous results . Comparing mutations, CNAs, gene expression and RPPA between both subtypes, we found four potential drivers of the HR subtype :  results , but alsBoth IR and HR subtypes show similar clinical outcomes . To idenCOL11A and THBS2) in the EMT signature defined by Anastassiou et al.Performing an integrative analysis of the RPPA and gene expression data using iClusterTo assess patient outcome, we performed Cox proportional hazards regression, stratified by biobank and fitted with commonly used clinical variables . TumoursTo combine the various predictive signals, we trained decision trees based on RPPA epitopes, the EMT factor, mutation rates and chromosomal instability . We founet al. recently showed that eIF4B phosphorylation is responsible for the increase in CIP2A translation, a key factor in estradiol-enhanced proliferationeIF4B is required for the binding of mRNA to ribosomes and functions in association with the translation initiation complexCDH1 and that activate the PI3K pathway. Thus, PI3K pathway inhibitors may represent a plausible treatment option for ILC patients. We also found low frequency mutations in a number of signalling molecules including ERBB2, MAP3K1, and MAP2K4 and low frequency inactivating mutations in TP53 and the transcription factor GATA3, which is critically important for ER signalling.In this study we provide a detailed analysis of biological processes in ILC. We integrated genomic, proteomic, transcriptomic and clinical data for this specific subgroup of breast cancers. Unsupervised clustering of the genome-wide gene expression revealed two subtypes: an immune related subtype, characterized by lymphocytic infiltration and up-regulation of “checkpoint proteins”, and a hormone related subtype, characterized by active ER/PR signalling and EMT . SimilarILCs often show a characteristically diffuse pattern of growth and clinical samples often have a relatively low tumour content. To rule out the possibility that the two identified subgroups were consequence of different tumour cellularity, we repeated the clustering using samples with high tumour cellularity (>50%). This analysis robustly identified the two subtypes . In addiCD4 and CD8A. In both ER-negative and ER positive/HER2 positive breast cancer, the presence of CD8+ T cells was shown to be associated with a significant reduction in the relative risk of death from diseasePDCD1 (PD-1), CD274 (PD-L1) and CTLA4 (CTLA-4). We observed some diffuse staining in both tumour and immune cells in some of the samples  material were extracted from the hospital database. We also sourced FF tissue from adjacent matched normal tissues when available (n = 55). Subsequently, we collected matched formalin fixed paraffin embedded (FFPE) tissue blocks for TMA construction. The NKI-AVL and Cambridge medical ethical committees approved the study and the use of anonymized archival tissue in this study. All experiments were performed in accordance with relevant guidelines and regulations. For the Cambridge samples, the study was approved by ‘NRES, Cambridgeshire 2 Research Ethics Committee’ (project number 07/H0308/161). Following the Dutch national guidelines on the use of left-over tissue and exceptions/clarifications in Dutch national law, the NKI-AVL ethical committee allowed performing the study without collecting informed consent, provided that patient data would be anonymized (decision letter July 29th 2010). This decision was reviewed by ethical expert Prof. Elaine Kay. Following the WHO definition of ILC, we used a morphological diagnosis to select the samples. Tumour samples were centrally revised for tumour percentage of the FF material, histological grade on FFPE and ILC sub-classificationAs presented earlierDNA sequencing was performed on an Illumina HiSeq 2000 platform. For each sample, Illumina TruSeq index libraries were constructed according to manufacturer’s instructions before being enriched by capture with a biotinylated RNA probe set targeting the human kinome and a range of cancer related genes . We sequenced 10 to 12 samples on a single Illumina HiSeq 2000 lane to generate 50 bp paired-end reads. On average, we obtained 26,985,771 unique reads on each run. The average kinome coverage (mean bait coverage) for the whole sequencing dataset is 133X, ranging from 36 to 258. On average, 91% of the target positions are covered by 20x. We aligned the raw sequencing data with the Burrows-Wheeler Aligner (BWA) version 5.10, backtrack algorithm, to the human genome (Ensembl 37) removing duplicate reads and reads with mapping quality <60. Single nucleotide variants and indels were called using SAMtools on unique paired aligned data. We used dbSNP and variant data from the Exome Variant Server together with a pool of normal 80 DNAs taken from various tissues to remove potential germline variants. A set of candidate somatic variants was selected for validation by sequencing matched tumour and normal material. Variants found back in the tumour sample and not in the normal are validated mutations (VALIDATED); variants found in both the tumour and the normal samples are rare germline variants (SNP); variants not found back in the tumour samples are false positive calls (ABSENT); finally some variants were tested but the experiment failed (FAILED).The RNA for microarray analysis and sequencing was purified using the Qiagen RNeasy micro kit according to manufacturer’s protocols. RNA was amplified, labelled and hybridized to the Agendia custom-designed whole genome microarrays (Agilent Technologies) and raw fluorescence intensities were quantified using Feature Extraction software according to the manufacturer’s protocols. We performed background subtraction using an offset of 10. All probe intensities <1 were set as missing values. The log2 transformed probe intensities were quantile normalizedWe applied several different clustering algorithms on the top 1000 genes with highest median absolute deviance: hierarchical clustering with Pearson distance and ward D1, single, average and complete linkage, as well as non-negative matrix factorization (NMF). The ward D1, average and NMF methods returned stable clustering results as assessed by consensus clustering. All three methods found largely the same two clusters . To defith, 2014.We used the ILC samples of METABRIC3VO4 and 10 mM NaF) and boiled for 10 min at 100 °C. Samples were sonicated for 1–2 min to break the DNA and spun for 10 min at 13,000 rpm. Supernatant was snapfrozen and protein concentration was measured . Samples with sufficient protein concentration (>0.5 mg/ml) were printed onto nitrocellulose-covered slides  using a dedicated arrayer  in five serial dilutions (0.5 to 0.03125 mg/ml) and two technical replicates. Arrays were labelled as described in Rondeau et al.Three sections of fresh frozen tissue were lysed in hot Laemmli buffer , Protease inhibitor cocktail complete MINI EDTA-free , 2 mM Na0, ProteaWe profiled a panel of 15 ILC-like cell lines . Drug seet al.We applied a factorization integrating RPPA and gene expression data, and then did a gene set enrichment analysis (GSEA)et al.To contrast the IR and HR subtypes we also used GSEA. Genes were ranked by differential expression  between the two clusters. To specifically investigate oestrogen signalling, we used the list of up and down regulated genes upon oestrogen stimulation of MCF-7 cells as identified by Zwart Decision trees were built using conditional inference treeshttp://dx.doi.org/10.6084/m9.figshare.1301848Clinical data: http://dx.doi.org/10.6084/m9.figshare.1360201Immunohistochemistry data: http://dx.doi.org/10.6084/m9.figshare.1373846Variants data:http://dx.doi.org/10.6084/m9.figshare.1314577Copy number data: Gene Expression Omnibus (GEO) accession GSE68057http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE68057Gene expression data:Gene Expression Omnibus (GEO) accession GSE66647http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE66647RPPA data:Public data access to the raw data is available with the following DOIs:How to cite this article: Michaut, M. et al. Integration of genomic, transcriptomic and proteomic data identifies two biologically distinct subtypes of invasive lobular breast cancer. Sci. Rep.6, 18517; doi: 10.1038/srep18517 (2016).


Recent comprehensive assessments of RNA-seq technology support its utility in quantifying gene expression in various samples. The next step of rigorously quantifying differences between sample groups, however, still lacks well-defined best practices. Although a number of advanced statistical methods have been developed, several studies demonstrate that their performance depends strongly on the data under analysis, which compromises practical utility in real biomedical studies. As a solution, we propose to use a data-adaptive procedure that selects an optimal statistic capable of maximizing reproducibility of detections. After demonstrating its improved sensitivity and specificity in a controlled spike-in study, the utility of the procedure is confirmed in a real biomedical study by identifying prognostic markers for clear cell renal cell carcinoma (ccRCC). In addition to identifying several genes previously associated with ccRCC prognosis, several potential new biomarkers among genes regulating cell growth, metabolism and solute transport were detected. The recent comprehensive assessments of the RNA-seq technology provide important guidelines to produce high-quality RNA-seq data sets –4. The oIn addition to producing accurate estimates of gene expression levels, the utility of the RNA-seq technology depends on the availability of rigorous tools for downstream analysis of these data such as quantifying differences between sample groups. This part still lacks well-defined best practices. Although a number of advanced statistical methods have been developed . It determines an optimal test statistic directly from the data by maximizing the reproducibility of the detections across bootstrap samples (refer to MATERIALS AND METHODS for details). The utility of reproducibility optimization in microarray studies of gene expression has been demonstrated ,14. In tA and B with five technical replicates per group. All the replicates were enriched with 92 synthetic polyadenylated oligonucleotides introduced by the External RNA Control Consortium (ERCC) (A and B. All the samples were sequenced using Illumina HiSeq2000 platform and produced paired-end reads of length 100 bp. The reads were mapped and assembled using TopHat (v.2.0.3)   to validv.2.0.3)  and UCSCv.2.0.3) .https://tcga-data.nci.nih.gov/tcga/). The demographics data and clinical features were available in the supplementary tables of the original study (The ccRCC data set was published by The Cancer Genome Atlas (TCGA) . All theal study . In totaThe ccRCC prognostic findings were validated in an independent RNA-seq data set published by Sato et al. . The mRNThe necessity of normalization has been proved in RNA-seq studies ,21. Accoa priori assumptions about data set distributions contradicts the observed biological variation in real RNA-seq experiments. To eliminate biases, we propose to learn an appropriate test statistic directly from the data, building on our data-adaptive reproducibility optimization procedure ROTS (Reproducibility Optimized Test Statistic) (Making atistic) . The inpj. The gene rankings are estimated using a family of modified t-statistics:t-statistic. The other special cases include signal log-ratio  estimate to assess the significance of differential expression. In RNA-seq studies, small numbers of replicate samples and large biological variation remain the main challenges which call the efficiency of statistical methods under question. With ROTS, the optimized Z-score and reproducibility are the main indicators to decide the success of differential expression detection. As a rule of thumb, reproducibility Z-scores below 2 indicate that the data or the statistics are not sufficient for reliable detection.The output of ROTS is the optimized All the statistical analyses in this manuscript were performed using R version 3.0.2. For comparing ROTS with other available methods, we used edgeR (version 3.2.4), DESeq (version 1.12.1), DESeq2 (version 1.0.19), Cuffdiff 2.0.2, Limma (version 3.16.7), baySeq (version 1.12.0), NOISeq (version 2.12.0) and PoissonSeq (version 1.1.2).hclust function using the Ward method and Manhattan distances. The patient-specific risk scores were calculated similarly as by Shaughnessy et al.  and human brain samples (n = 5) spiked with 92 synthetic oligonucleotides provided by the External RNA Controls Consortium (ERCC) (refer to MATERIALS AND METHODS for details). The pre-determined fold-changes of the synthetic RNAs  enable measurement of true positive (sensitivity) and true negative detection rates (specificity) of different gene ranking methods.The improved sensitivity and specificity of ROTS over state-of-the-art methods was demonstrated in a controlled spike-in study. Spike-in data sets are benchmarks to investigate the strengths and weaknesses of new computational methods. The spiked data set presented by Rapaport et al. , previouP < 0.01 compared to all other methods except for baySeq for which P = 0.077). The second best method was baySeq (AUC = 0.891), whereas NOISeq showed the lowest AUC value (AUC = 0.704). This comparison was repeated for smaller sample sizes  with selected methods . Again ROTS performed significantly better than the other methods with sample sizes larger than 2 . To further investigate the ability of the methods to control type I error rate  the approach of the original paper (The Receiver Operating Characteristic (ROC) analysis showed how ROTS outperforms the other methods in terms of sensitivity and specificity , which is an important clinical problem. The ccRCC accounts for the majority of cases of kidney cancer . The cliR = 0.57, Z = 5.27). To focus on the most promising candidate markers for ccRCC prognosis, those 152 genes that showed log2 fold-change above 1.6 (∼3-fold change) and average expression above the lowest 30% were retained for subsequent analysis, similarly as in the RNA-seq assessment studies  with the longest survival time (>60 months) and 40 patients with the shortest survival time (<12 months); see Supplementary Table S1 for the characteristics of the groups. The ROTS reproducibility values indicated appropriate reproducibility of the results (0% with tP < 10−15). While the 5-year survival in the best group (blue) was ∼80%, it decreased below 20% in the worst survival group (red); in the two intermediate groups the 5-year survival was 50–60%. This supports the prognostic value of the detected genes beyond the 80 samples used for detecting the markers.To assess the utility of the 152 ROTS detections as prognostic markers, a risk score was defined for each of the 442 patients in the complete TCGA data in terms of signal log-ratio of the up-regulated versus the down-regulated genes ,33 refe. AccordiP < 10−15). Over 90% of the detected 152 genes showed a concordant change to the same direction in both data sets . Importantly, a highly significant association between the risk scores and survival verified the risk score model developed using the TCGA data . These results confirm the capability of ROTS to robustly identify reproducible candidate markers well as the potential of the risk score model to identify especially poor prognosis ccRCC patients.To avoid over-fitting to a single study, further validation of the markers and risk score model was carried out in a completely independent data set of 100 ccRCC patients . The sigP < 0.05). Categorization of the genes into five main biological function groups and four additional biochemical function groups implies major involvement of metabolism (∼18% of the genes), particularly glucose metabolism, as found also in the original TCGA study  tool suggests three major functional groups: molecular transport, small molecule biochemistry, and amino acid and lipid metabolism . Some of them, such as members of solute carrier family 16, were detected by the TCGA study. Additionally, ROTS detected 19 previously undetected solute carrier genes, including for example SLC38A5, which transports glutamate, an essential nitrogen donor for cancer cells to build amino acids and to maintain mTOR activity  of the prognostic genes were from the cellular transporter and solute carrier groups Figure . The soly Figure .The better prognosis genes included most of the detected glucose metabolism and organic anion/cation transporters. Increased expression of six glycolytic genes were categorized into good prognosis group . The glycolytic genes are direct transcriptional targets of HIF-1α, which is known to function as a tumor suppressor in kidney cancer in contrast to several other cancer types ,35. The P < 0.05). For instance, the overlapping genes included the ACE2 and NPR3 that regulate blood pressure by the renin-angiotensin system. Additionally, new genes within this pathway were detected, including REN, the primary regulator of the pathway secreted by kidney cells. To further validate the method we studied the association of several known genes mutated in RCC with our prognostic groups . In line with previous reports ROTS predicted PBRM1 mutations to correlate well with good prognosis (P < 0.01) (P < 0.01). Several other mutated genes were also tested for prognostic correlation  but their mutation rate was too low to reveal any significant correlation. Moreover, our algorithm predicted a strong trend toward better prognosis in the VHL mutated group (P < 0.05). Although studies attempting to correlate VHL mutation with patient prognosis have been somewhat conflicting, correlation of wild-type VHL with poor prognosis has been reported earlier  ,37. Also earlier ,39.Taken together, our results demonstrate the utility of the RNA-seq technology to detect reproducible markers when an appropriate test statistic is applied. Systematic tools are needed for unbiased and effective analysis of RNA-seq data to fulfill the high promise posed by the technology. Instead of developing new variants of different statistical tests, users require practical tools to choose an optimal method for their own data. The reproducibility optimization procedure enables this. Additionally, it provides information about the quality of the detections; low reproducibility values indicate that the data or the test statistics are not sufficient for reliable detection.The high validation percentage of the detected ccRCC prognostic markers in independent data supports the general potential of ROTS in clinical RNA-seq studies. Although the main focus was on reliable marker detection, the developed ccRCC risk score model illustrates the use of the detections for disease signatures. The identified novel candidate genes serve as good starting points for further validation studies to confirm their utility in the clinic as support tools to predict ccRCC prognosis or revealing novel potential targets for ccRCC treatment. Our results suggest that ROTS enables stratification of patients in prognostic groups that can help to select patients for future RCC adjuvant trials and closer post-nephrectomy follow-up to timely reveal metastatic disease. The identified markers imply high potential of genes regulating cell growth and metabolism but also ion transport apart from glucose transport. It is also noteworthy that four genes regulating blood pressure through the renin-angiotensin system were identified as good prognosis markers, as blood pressure is commonly elevated in RCC patients and functions as a surrogate marker for tyrosine kinase inhibitor treatment efficacy.http://www.btk.fi/research/research-groups/elo/software/rots/. In order to make the study fully reproducible, we provide all the codes to run the analyses in the Supplementary Code, which provides detailed information about the methods and parameters used in this study.ROTS package is implemented with R and the source code together with Windows and Mac OS binaries are freely available at


Feature selection techniques have become an apparent need in biomarker discoveries with the development of microarray. However, the high dimensional nature of microarray made feature selection become time-consuming. To overcome such difficulties, filter data according to the background knowledge before applying feature selection techniques has become a hot topic in microarray analysis. Different methods may affect final results greatly, thus it is important to evaluate these pre-filter methods in a system way.In this paper, we compared the performance of statistical-based, biological-based pre-filter methods and the combination of them on microRNA-mRNA parallel expression profiles using L1 logistic regression as feature selection techniques. Four types of data were built for both microRNA and mRNA expression profiles.Results showed that pre-filter methods could reduce the number of features greatly for both mRNA and microRNA expression datasets. The features selected after pre-filter procedures were shown to be significant in biological levels such as biology process and microRNA functions. Analyses of classification performance based on precision showed the pre-filter methods were necessary when the number of raw features was much bigger than that of samples. All the computing time was greatly shortened after pre-filter procedures.With similar or better classification improvements, less but biological significant features, pre-filter-based feature selection should be taken into consideration if researchers need fast results when facing complex computing problems in bioinformatics.The online version of this article (doi:10.1186/2047-2501-2-7) contains supplementary material, which is available to authorized users. During the last decade, feature selection techniques have become an apparent need in many biological and medical analyses fields , 2. WithHowever, the expression data sets generated by microarray technology are often composed of a large number of molecular as potential features compared with a limited number of samples. The expression profile data are often described as a matrix in bioinformatics with rows representing features and columns representing samples. Due to the limitation of many factors such as the cost and ethics of acquiring large number of samples from patients, it is difficult to make the data suitable for existing feature selection algorithms. Feature selection techniques are used in microarray data analyses through selecting a small subset of molecular by removing relatively redundant, noisy, and irrelevant part of the data. However, the high dimensional nature of microarray made feature selection become time-consuming processes.To overcome such difficulties, filter some features according to the characteristics of data before applying feature selection techniques which we named ‘pre-filter’ procedures in this paper is a good choice. Considering the characters of bioinformatics, researches started to reduce features based on background knowledge in the fields of biology, medicine, and statistics, etc. Thus, many pre-filter methods had been proposed based on statistical or biological considerations as follows: (1) statistical-based pre-filter methods: using statistical methods to find out the differential expressed molecular among different conditions. These procedures are usually simple and fast. Take differential expression molecular identifications as an example, researches used statistical methods to find molecular with expression values fluctuated among different conditions. It is often the first step of microarray analyses and is also one of the most commonly used pre-filter methods. In such kind of procedure, statistical test such as t-test and ANOVA are usually chosen (according to the number of different conditions) due to their stability and easy operability. However, these procedures may often identify features that are isolation from the others. (2) biological-based pre-filter methods: using enrichment analysis based on biological function and/or pathway information to find out potential disease-related molecular. One of the most important goals of microarray analyses is finding the biomarkers with significant biological meanings. Gene Ontology (GO)  is compoHowever, it is unclear to us that how much improvement these pre-filter methods could bring on the feature selection results. In this paper, we compared the performance of these pre-filter methods on 4 microRNA and 10 mRNA microarray datasets. L1 logistic regression was used as the representation of feature selection methods to perform the analyses after pre-filter procedures. All the samples’ class labels were known and used to evaluate the results by using 5-fold cross validation. Our results showed that both of the two kinds of filter methods could increase classification precision slightly while the combination of them could increase the AUC (Area Under Curve) of ROC (Receiver Operating Characteristic) curve slightly. The features were found to be significant on biological levels. All the computing times were shortened greatly.microRNA and mRNA expression profiles of human hypertrophic cardiomyopathy (HCM) were downloaded from NCBI GEO   were selected based on t-test, with threshold 0.05. Genes with  Type 3: Expression of all genes on microarray with disease related functions. 372 validated HCM related genes were collected from GeneCards  and GAD  Type 4: Expression of differential expressed genes with disease related functions. Similar to the construction processes of type3, these 4 datasets were built by picking out DEGs annotated to the same GO terms  or biological pathways of validated HCM related genes. These 4 datasets were named as type4-BP, type4-MF, type4-CC, and type4-Pathway, correspondingly. Type 1: Expression of all microRNAs on microarray. This dataset was built by mapping all the 1145 probes on microarray to 819 mature human microRNAs. Their corresponding expression values in all the samples were extracted to construct the type1 microRNA dataset. Type 2: Expression of differential expressed microRNAs. Differential expression microRNAs (DEM) were selected based on t-test, with threshold 0.05. The expression values of the microRNAs with p-value not over 0.05 were extracted from all the samples to build the type2 microRNA dataset. Type 3: Expression of all microRNAs on microarray with validated disease related genes as targets. 19550 validated microRNA-mRNA relationships were downloaded from mirTarBase . MicroRN Type 4: Expression of differential expressed microRNAs with validated disease related genes as targets. Similar to the construction processes of type3, the expression values in all the samples of DEMs with at least one validated HCM related gene were chosen to build the type4 microRNA dataset.4 types of microRNA datasets were built as follows  was irrelevant to the gene lists. For each functional term and gene list, the a was the number of genes annotated to a certain functional term, b was the total number of genes, d was the number of genes in the list, and n was the number of genes in the list annotated to this functional term. All the functional terms with p value not above 0.05 were chosen as enriched terms.Of which, L1 logistic regression to perform the feature selection procedures due to its ability to dispose the high dimensional data [We used nal data . The modxn ∈ RN was the n-th feature and yn ∈ RN was the label of the n-th sample. We used  as the coefficients and intercept of L1 logistic regression. The L1 logistic regression model was listed as follows:Let L(.) was the loss function and λ was a regularization parameter which had the ability to dispose high dimensional data.where L1 logistic regression on all the 14 datasets as illustrated above (10 mRNA datasets and 4 microRNA datasets). Three measures including AUC value, precision, and computing time were computed and compared for these test datasets.5-fold cross validation was used to analyze the classification results of A receiver operating characteristic (ROC) was a graphical illustrates the performance of a classifier with the discrimination threshold varied. The area under the curve (AUC) was equal to the probability that a classifier would rank a randomly chosen positive instance higher than a randomly chosen negative one . A biggeLet TP and FP stand for true positives and false positives, the precision was calculated as:4 types including 10 datasets were built for the mRNA expression profiles (See ‘Methods’ for details). The detailed information for the number of raw variables in each set could be found in Table For the 4 microRNA datasets, both the statistical-based and biological-based pre-filter methods could reduce the number of features greatly while the combination of them could extract 8 microRNAs from all the 819 mature microRNAs . These indicated that pre-filter methods may improve the performance of feature selection techniques on samples’ positive prediction levels. From Figure The performance of pre-filter methods on microRNA expression profile did not show similar results with mRNA Additional file 1:


A number of statistical models has been proposed for studying the association between gene expression and copy number data in integrated analysis. The next step is to compare association patterns between different groups of samples.We propose a method, named dSIM, to find differences in association between copy number and gene expression, when comparing two groups of samples. Firstly, we use ridge regression to correct for the baseline associations between copy number and gene expression. Secondly, the global test is applied to the corrected data in order to find differences in association patterns between two groups of samples. We show that dSIM detects differences even in small genomic regions in a simulation study. We also apply dSIM to two publicly available breast cancer datasets and identify chromosome arms where copy number led gene expression regulation differs between positive and negative estrogen receptor samples. In spite of differing genomic coverage, some selected arms are identified in both datasets.We developed a flexible and robust method for studying association differences between two groups of samples while integrating genomic data from different platforms. dSIM can be used with most types of microarray/sequencing data, including methylation and microRNA expression. The method is implemented in R and will be made part of the BioConductor package SIM. Many experimental studies produce multiple types of molecular profiles per sample to help understand if, and how, different molecular levels influence each other. This has led to a growing interest in methods for analyzing multiple high-dimensional datasets together. As a result, a number of methods have been developed in recent years for studying the association patterns between high-dimensional datasets, i.e. performing integrated genomic analyses -6. In a Studying these association patterns using integration analysis evokes another very important but yet overlooked question: Do these association patterns differ between groups of samples? Consider for example the case where gene expression regulation is studied on the basis of genomic copy number changes. In such studies, there is often interest in comparing gene-dosage led gene expression variation between groups of samples. Furthermore, if there are clinical variables defining the groups of samples, there is interest in finding group-specific associations between copy number and gene expression.In this paper, we propose a method, named dSIM, to find these association differences between two groups of samples, while using an extension of the model described in . It firsy with n observations is y = ⊤, the regression model can then be written as We first give a brief overview of the model and method of SIM  before dXjk is the gene expression measured for jth sample and kth probe, and α is the intercept. Here, βk is the coefficient value for kth gene expression probe; it explains the association patterns between copy number probe y and gene expression data {Xjk,j=1,…,n;k=1,…,p}. For testing the copy number and gene expression associations, the global test [H0:βk = 0,k = 1,…,p} in high-dimensional data by using a random effects model context, where it is assumed that the coefficients {βk} come from a normal distribution with mean 0 and variance ω2. Instead of testing H0:βk = 0, the global test [H0:ω2 = 0 against the alternative H1:ω2>0.bal test ,10 is usbal test ,10 consiThe main objective of this work is to test for differences in association between copy number and gene expression data, when the samples are grouped according to some binary grouping variable (for example estrogen receptor status in the case of breast cancer). In other words, we would like to find regions where copy number variation regulates gene expression differentially between two groups. One possible way of doing this would be to divide the datasets into two groups and to fit the model given in (1) for these two groups separately. However, this idea has three serious drawbacks. First, analyzing the two subsets separately results into two distinct sets of p-values, that should be combined and interpreted. Second, it is inefficient as the power of finding associations is reduced by considering a subset of samples at a time. The third issue ignored by analyzing the datasets separately like this is the baseline association between copy number and gene expression. Association patterns common to both groups should be removed before we start looking at the differences between them.Our new approach dSIM extends the model given in (1) as follows. It starts by correcting for the baseline associations and then works with the corrected data to directly test for differential association patterns. For the sake of simplicity, we describe the model in the following subsections by using one copy number probe at a time. In practice and in the examples shown in this paper, it is applied to all copy number probes in a similar way.p≫n) is dealt with by fitting (1) using ridge regression [λ. We do this by leave-one-out cross-validation. For each λ value, the dataset is divided into a test set containing a single sample i from the original sample set, and a training set containing all remaining samples with j = 1,…,n,j≠i. The ridge regression model is fitted and the coefficients are estimated on the training dataset. Note that the coefficients estimated here are obtained without introducing the group effect. These estimated coefficients are then used to predict the test sample as For removing copy number and gene expression associations common to both groups, we start by fitting the model given in (1) on all samples together. The issue of  are now used for testing the association differences between the two groups of samples. For performing the test, first we write a linear model with the residuals as dependent data. This model holds for all samples together and is given as The cross validated residuals {δ acts as the intercept and δcj represents the baseline shift in association between copy number and gene expression for group 2 when compared to group 1. Here, cj∈c, where c, the factor defining the groups, can be written as c = ⊤. Now, since {Rj} represents average residual effects over all samples, it is important to parameterize c in such a way that 1⊤c = 0. Here 1 is a column vector of ones with length n. Therefore, for all examples in the paper, we define jth sample belongs to group 1 and jth sample belongs to group 2, where cj are values in c and nG1, nG2 are the number of samples for group 1 and group 2 respectively. The value Mjk is the gene expression data for the jth sample and kth probe with grouping effect in it. It can be defined, in words, as an gene × group interaction term or Mjk = Xjkcj, where Xjk is the gene expression measured for jth sample and kth probe and cj is the corresponding group variable. The error term in the model is denoted by εj.In this model, γk,k = 1,…,p}, which account for the difference in association between the copy number data and gene expression data between the two groups. Similar to the {βk} coefficients, we here assume that {γk} come from a normal distribution with mean 0 and variance θ2. Our approach is to then test the variance for the distribution of these {γk} values with, H0:θ2 = 0 against the alternative H1:θ2>0. This is done by using the global test [H0:θ2 = 0 is not known because of the ridge regression step. Hence, we propose to use permutation of the independent data sample labels to yield p-values.The effect of interest in model (4) is represented by the parameters {bal test ,10. MoreX, where X is the n×p gene expression data matrix with samples as rows and genes as columns. For this, we first permute the sample labels B times. The baseline association is corrected, residual values are obtained, and the global test [Permutation resampling can be used to estimate p-values while avoiding parametric assumptions about the joint distribution of the test statistic. In the microarray setting, the joint distribution under the complete null hypothesis of the test statistic can be estimated by permuting the rows of the matrix bal test ,10 is doλ. This ridge regression step given in (2) does not involve the grouping factor c. Moreover, permuting the sample label of X retains the correlation structure, distributional characteristics exhibited by the genomic data matrix X and the ill-conditionedness of X⊤X. Since λ depends on X⊤X, we can use the same optimized λ value and hence, the same residuals for each permutation. Permuting the sample labels avoids the optimization of λ parameter for every permutation, making the method computationally less intensive.As described in subsection ‘Correcting for the baseline association’, while fitting the regression model and calculating the residuals using ridge regression, one needs to optimize the tuning parameter m copy number probes which gives us a matrix of global test p-values, P∗, with m rows and (B+1) columns. The first column of this matrix consist of the observed global test p-values for all probes, obtained for the observed data (subsection ‘Testing for the group effect’), whilst remaining columns represent permuted global test p-values obtained after permuting the sample labels.The steps involved in the method described in the previous subsections as well as this one are done for a single copy number probe at a time. These are then repeated for all R, Meinshausen’s permutation based method makes false discovery proportion (FDP) confidence statements of the form o(R) is the proportion of type I errors for the set R in hypothesis testing and P∗, which gives us a single vector of length m, containing the multiple testing corrected dSIM p-values.To exploit the dependence structure of the permuted dSIM p-values, we use Meinshausen’s procedure for multiple testing correction . For genoportion ,12. We aWe perform various simulations to study the properties of dSIM. For this, we start with generating the copy number data and gene expression data, both of them with same dimensions. The copy number probes are assumed to have similar distribution across the two groups and the gene expression data is generated per group, as a function of copy number. In our case, we consider copy number as the dependent data and use Meinshausen’s approach for multiple testing correction. More details on the simulation setup are given in the Additional file Y and the two different groups of gene expression data X as follows: Scenario 1 For this scenario, copy number and gene expression show similar association patterns for both groups of samples. Hence, no significant association differences between the two groups are expected. Under this scenario, we test the specificity of dSIM. Scenario 2 In this scenario, copy number and gene expression show different association patterns between groups of samples. Hence, significant association differences between two groups should be detected. Scenario 3 This scenario has association patterns between copy number and gene expression for one group of samples only, so here again differences should be detected. This scenario is similar to the second scenario in the sense that there are association differences present between groups of samples.For scenario 1, no probes are selected, as expected  is changed. Each study is then run 50 times, producing 50 simulated datasets from the same setup. The data points for the matrix X change between these 50 runs, but the number of probes and the regions showing differential association between the the two groups remain exactly the same. For every run, the λ value is estimated using the method described in section ‘Correcting for the baseline association’. This makes sure that while the value of λ might vary a lot across the four simulations, intra-study variations would be minimal. The ROC curves for these simulation studies are given in Additional file λ displays some variations.Firstly, we study the variation of λ values over the dSIM p-values. For this we perform a simulation study where only some simulated copy number probes show differential association with gene expression data. For each probe, the optimum lambda value (λopt) is obtained and a range of 50 λ values around λopt is generated. For each of these λ values, Meinshausen-selected dSIM p-value is calculated, resulting in a vector of 50 p-values for each probe. These p-values are then plotted against the corresponding λ value for some chosen probes. From Additional file λ values. The probes that exhibit differential association between groups of samples are selected as significant irrespective of the λ values. The probes with no association, on the other hand, are never selected by dSIM and Meinshausen’s method, irrespective of the changes in λopt. This shows that dSIM is capable of identifying the probes with differential association and maintaining the true positive rate, even with small variations in the value of λopt.Secondly, we study the effect of changing To demonstrate that separate analysis suffers from differences in sample sizes, we performed a small simulation study. In this study we simulated 50 datasets with 45 samples and 100 probes in each. Out of these 45 samples, 30 were assigned to group 1 and 15 to group 2, so that to have one larger and one smaller group of samples. All the other parameters between the two groups of samples are kept exactly the same, including the region of copy number aberrations. The samples in both groups show similar type of associations with between copy number and gene expression data. Hence, there is no differential association between the two groups of samples.We started with performing a separate analysis of these 50 datasets, using SIM. The multiple tested corrected p-values obtained from separate analysis, for one of the 50 simulated datasets, are shown in Additional file We apply dSIM to two publicly available breast cancer datasets, namely, the TCGA dataset and the NKI dataset . Both daWe considered estrogen receptor (ER) status of the samples as the grouping variable. The TCGA dataset consists of 134 ER-positive samples and 32 ER-negative, while the NKI data consists of 43 ER-positive and 25 ER-negative samples. ER status is considered an important prognostic and a predictor factor for endocrine response in breast cancer. It has been shown to have strong association with the prognosis of the disease and exhibits strong associations with distinct gene expression patterns . This maWe start by simply analyzing the NKI and TCGA datasets by fitting model (1) per group, ER-positive and ER-negative. We report the results through whole genome plots for both groups, while controlling the FDR at 5%. The R package SIM was used to perform these analyses. The results from the separate analyses are then compared with the results from the application of dSIM to the NKI and TCGA datasets.The selected probes for the two groups of samples, obtained from separate analyses of the NKI dataset, are shown in Additional file Similar results can be seen from the separate analysis of TCGA dataset also  so as to focus just on 11q13 and re-analyzed the datasets using SIM. The raw p-value distributions are shown in Additional file From the Additional file Firstly, we fitted a ridge regression between each copy number probe and all gene expression probes in 11q13. We then found that coefficients with the largest ridge estimates tended to be the same for ER-positive and ER-negative groups of samples, for those copy number probes with significant association with all gene expression probes, as given by SIM. Secondly, we looked at the SIM results for these copy number probes in more detail, by studying the individual (standardized) influences of each gene expression probe on the global test. The scatter plot of global test z-scores for ER-positive and ER-negative groups of samples are shown in Additional file From the biological viewpoint, it is important to check if the genes detected by dSIM are known to be associated with breast cancer and more, importantly, estrogen receptor status. Literature review brings up important breast cancer-associated genes located in regions detected by dSIM. Here we focus on chromosome arm 12p of the NKI dataset, where tumors in the ER-negative group showed more frequent gains compared with the ER-positive group, and several regions of differential association were detected. We selected the five most differentially associated copy number probes on 12p , was positively associated with copy number gain in the ER-negative samples -selective coregulator that potentiates the inhibitory activities of anti-estrogens and represses the activity of estrogens [PHB2 might inhibit tumor growth in ER-positive tumors, the biological consequences of upregulation in ER-negative breast cancer are unknown.Overexpression of  outcome . Our dats Figure C and 4D.strogens . While uIt should be noted that in examples where gene expression probe and copy number probe do not overlap with each other, it is important to check for specious associations. It is possible that the dSIM detections are due to co-amplification of copy number probes, for example.Together, these results show that by testing differential copy number-expression associations between relevant groups of tumors, new hypotheses for tumor biology based on underlying genetic aberrations can be generated.In this subsection we show the approximate computation time of dSIM. To demonstrate how the computation time is effected by size of the gene sets, we ran dSIM on datasets with different number of probes. For all datasets, number of samples in Group 1 and Group 2 were 67 and 16, respectively. Calculation time of dSIM is more affected by the size of the gene set  than by the sample size itself. The reason for this is that dSIM is fitted per outcome gene. Hence, larger the number of genes, higher will be the calculation time. Sample sizes (for dependent and independent datasets) have an effect on the calculation time but it is almost negligible. We ran all analyses using R i386 2.15.2 on an Intel(R) Core2Duo CPU with 3.00 GB of RAM. The run times are given in Table We proposed a method to compare associations found between two high-dimensional data sets, for two groups of samples. Our method models associations between features in one data set and sets of covariates in the other, thus facilitating the search for markers that are related to a set of features.Our method uses all data at the same time in the model, thus being less susceptible to small sample sizes in (at least) one of the groups, compared with separate analysis per group. In addition, we consider associations between probes and gene sets. These characteristics make for a powerful method that finds robust differences in associations.To the best of our knowledge, we are the first to suggest an integrated analysis method for comparing association patterns between two groups of samples. The most closely related method seems to be the one proposed by Artmann et al. , as theyOur method does not require that features are differentially valued between the two groups of samples. Indeed, we argue that this is not necessary and, in our view, not even desirable. We have in fact made sure that dSIM results cannot be driven merely by differences on the data distribution of the dependent data between the two sample groups. In our view, such differences in a single data set would not necessarily mean different associations between sample groups, and should not be seen by the method as such either. Furthermore, the lack of differential values does not rule out differential associations, so by considering differential values one actually restricts the results too much.This makes biological sense. Let us take for example the setup in the TCGA data example. It is entirely possible that DNA copy number behaves in the same way, and displays the same distribution, in both ER-positive and ER-negative groups, and yet are associated differently with gene expression sets in those groups. This could happen due to another molecular mechanism coming into play, say. We feel that an open-minded analysis should be able to pick up those effects.A unique feature of dSIM is that it corrects for the baseline association before looking for differential associations between groups of samples. Thus, the association effects common to both groups are eliminated, and only those association patterns left over in the residuals are analyzed. In subsection ‘Correcting for the baseline association’, we describe the steps taken by dSIM to correct the baseline association while using ridge regression. The rational behind the usage of ridge penalty is that the metric for the global test statistic is the same as that of ridge. Therefore, the entire method makes use of the same metric.λ for ridge penalization using leave-one-out cross-validation. Then the leave-one-out cross-validated predictions, corresponding to the selected λ value, are used to get the residuals. Unlike the traditional way of using the fitted predictions, we use the cross-validated predictions, as leaving out a sample during cross-validation makes the estimated coefficients unbiased towards that sample. This, in turn, avoids overfitting of the model when predicting the outcome for the left out sample, hence minimizing the loss while obtaining the residuals.The residual effects studied by dSIM are relatively small as they are obtained after removing the large association effects during baseline association correction. These effects may weaken further due to overfitting of the ridge penalized model, making them harder to detect. Hence, for detecting these weak effects, it is important to minimize the loss of information during baseline association correction. We ensure this by optimizing the tuning parameter The fact that we test for a large number of copy number probes simultaneously requires choosing an appropriate multiple testing correction method. As the dSIM p-values are generated using permutation testing, the traditional methods for controlling FWER like Bonferroni, Holm or Hommel are not very useful. Firstly, they are too conservative and secondly, they do not take into account the dependence structure of the data ,29. The In this paper we focus on analyzing copy number regulated gene expression, where the gene sets are defined on the basis of their genomic locations. The method can be easily extended to analyze other types of genomic data and gene sets as well. For example, there could be interest in finding group-specific interaction effects between microRNA and mRNA expression, while looking at pathway specific genes. It is also possible to invert the model given in (1) to have copy number as independent data and gene expression as dependent data. However, we should indicate that ridge penalty does not exploit the inherit spatial correlation structure found in natural ordering of the copy number probes. In the case where copy number data is independent data, one possible option would be to use fused lasso penalty instead of ridge.nG - 1, where nG is the number of groups. As the association patterns are then compared between more that two groups, the number of interaction terms (γ) in the model increases. One possible way to compare multiple groups is to use the method proposed in this paper for performing pairwise comparison. This problem is beyond the scope of this paper and will be dealt with elsewhere.Another interesting extension of this model is to consider more than two groups of samples. The extension involves complex steps as the degrees of freedom goes up from one to H0:θ2 = 0, where θ2 is the variance of the distribution for {γk}. The permutation test not only tests for H0:θ2 = 0 but also for δ = 0, where δ is the intercept. The correct size of the test is therefore not guaranteed if θ2 = 0 but δ≠0. However, because we use a global test for the null hypothesis H0:θ2 = 0 internally in the permutation, it is unlikely that the permutation procedure has any serious power for H0:δ = 0. In practice, dSIM is not sensitive to those copy number variations that do not affect gene expression levels.Our method, dSIM, is based on a linear model where we assume that there is a linear relationship between the two genomic datasets. However, we should point out that the non-linear associations present between the datasets may not be detected by dSIM. We also assume the the distribution of the errors to be normal. In cases, with non-normal random errors, one can consider transforming the data before using dSIM. Another assumption made by dSIM is in using the global test for testing the null hypothesis In summary, we developed a method to test for the copy number and gene expression associations differing between two groups of samples. Through several simulation studies, we showed the robustness of dSIM under various conditions. Application of dSIM to the TCGA and NKI breast cancer datasets highlights the importance of having all samples together in the model.We propose a novel method for identifying genes that show different expression regulation between two groups. By using all samples together, it can more objectively and effectively find such differences, compared to separate analyses. It can help elucidate differences in gene expression regulation between two groups of samples due to copy number alterations or other (epi) genetic changes.https://mynotebook.labarchives.com/share/Supporting\%2520data/My45fDI1MDQyLzMvVHJlZU5vZGUvNDcyNjkzMTc0fDkuOQ==, DOI : 10.6070/H48K771S).The data sets supporting the results of this article are available in the LabArchives repository (The R scripts for simulation studies and dSIM can be made available upon request.The authors declare that they have no competing interests.NC implemented the method in R, wrote the manuscript and performed the data analysis. This work uses an original idea by RXM, further developed by RXM, JJG, WvW and NC. RXM and JJG supervised the work. JMB helped with biological interpretation of the results. All authors read and approved of the final manuscript.Supplementary material. Supplementary document containing details about the simulation study setup, additional figures and tables.Click here for file