The vast majority of human genes (>70%) are alternatively spliced. Although alternative pre-mRNA processing is modified in multiple tumors, alternative hyper-splicing signatures specific to particular tumor types are still lacking. Here, we report the use of Affymetrix Human Exon Arrays to spot hyper-splicing events characteristic of myasthenia gravis (MG)-thymoma, thymic tumors which develop in patients with MG and discriminate them from colon cancer changes.We combined GO term to parent threshold-based and threshold-independent ad-hoc functional statistics with in-depth analysis of key modified transcripts to highlight various exon-specific changes. These denote alternative splicing in MG-thymoma tumors compared to healthy human thymus and to in-house and Affymetrix datasets from colon cancer and healthy tissues. By using both global and specific, term-to-parent Gene Ontology (GO) statistical comparisons, our functional integrative ad-hoc method allowed the detection of disease-relevant splicing events.ACHE gene. Corresponding changes in spliceosome composition were indicated by co-decreases in the splicing factors ASF/SF2 and SC35. Parallel tumor-associated changes occurred in colon cancer as well, but the majority of the apparent hyper-splicing events were particular to MG-thymoma and could be validated by Fluorescent In-Situ Hybridization (FISH), Reverse Transcription–Polymerase Chain Reaction (RT-PCR) and mass spectrometry (MS) followed by peptide sequencing. Our findings demonstrate a particular alternative hyper-splicing signature for transcripts over-expressed in MG-thymoma, supporting the hypothesis that alternative hyper-splicing contributes to shaping the biological functions of these and other specialized tumors and opening new venues for the development of diagnosis and treatment approaches.Hyper-spliced transcripts spanned several categories, including the tumorogenic ERBB4 tyrosine kinase receptor and the connective tissue growth factor CTGF, as well as the immune function-related histocompatability gene HLA-DRB1 and interleukin (IL)19, two muscle-specific collagens and one myosin heavy chain gene; intriguingly, a putative new exon was discovered in the MG-involved acetylcholinesterase  Changes in gene expression, and particularly in alternative splicing patterns are often disease-associated In search for tumor-specific alternative hyper-splicing signatures, we selected MG-thymoma, an epithelial tumor of the thymus gland Linkage between regulated genes and the corresponding transcript quantities highlights tumor over-expressed transcripts as preferred targets for identifying hyper-splicing signatures post-hoc functional analyses such as T-test, ANOVA ad-hoc detection of changed functional GO categories have been evolving. These largely use threshold-based statistical approaches  that detect relative enrichments of gene categories given a population of genes At the functional level, alternative hyper-splicing can modify tumor properties, since gene products may play roles in multiple, often seemingly unrelated, routes. To approach these processes, tools for functional analyses of microarray data have been developed To increase the range of solutions and enable detection of subtle and non-trivial changes, several threshold-free methods were developed , with detailed term-to-parent statistical analysis on the GO categories represented on the arrays. First, we analyzed our in-house MG-thymoma study and the Affymetrix colon cancer ACHE gene, which is ubiquitously expressed, undergoes alternative splicing at both termini (3′ and 5′) and contributes to many different biological processes We applied an post-hoc statistics with in-depth analysis of key transcripts and FISH, RT-PCR and mass spectrometry followed by peptide sequencing validation revealed pronounced alternative hyper-splicing in several gene categories modified in MG-thymoma tumors. These spanned tumorogenic, immune function and muscle-specific transcripts, involved reduced expression of the splicing factors ASF/SF2 and SC35 and extended into the discovery of a putative new exon in the MG-involved ACHE gene. Together, these findings support the notion of a major contribution of alternative hyper-splicing to MG-thymoma features, opening new venues for diagnosis and treatment of specific tumor types, and revealed specific tumor-type alternative splicing signatures reflecting MG-thymoma and colon cancer properties.To study the contribution of alternative splicing to tumor biology, we developed a novel approach for identifying alternative hyper-splicing signatures. Using human exon microarrays, we compared MG-thymoma tumors from patients with MG to Affymetrix datasets from healthy tissues and colon cancer. Combined ad-hoc and ad-hoc analysis method to gene-level summary signals  Iter-PLIER gene-level data of Affymetrix colon cancer data set ad-hoc functional analysis and compared each GO term to the global array population. We observed ample changes in both BP and MF categories  and compared to complement receptor activity . Cell cycle terms showed both decreases and increases compared to its direct parent, cellular physiological process . More GO categories changed compared to their direct parent terms  than compared to the whole array population , identifying specific GO levels that changed and pointing at specific disease-related genes.Each term represented by the array transcripts was compared to its direct parent terms in the GO tree, as well as to its tree root (MF/BP), using both continuous and discrete data analyses, as well as tests for location and dispersion . Several changed  comparedACHE, which expresses at low levels but is highly relevant to MG COL11A1  and more moderately in colon cancer compared to healthy colon , is notably involved in chronic inflammation COL1A1 was up-regulated in MG-thymoma compared to healthy thymuses and in colon cancer compared to healthy colons. The tumor-related genes included the tyrosine kinase receptor ERBB4  which was up-regulated in MG-thymoma (DPT), which belongs to cell adhesion and protein binding, was down-regulated in both cancer types  was up-regulated in MG-thymoma compared to control thymus, control colon and colon cancer samples  is CNTNAP2, which exhibited over 4-fold increase in 4 core probe-sets. CNTNAP2 was highly up-regulated in MG-thymoma, exhibited low expression in colon compared to thymus, and was down–regulated in colon cancer , encodes a receptor activity member of the latrophilin G-protein coupled receptors (GPCR). LEC3 was up-regulated in MG-thymoma, but down-regulated in colon cancer , and the iroquise homeobox gene IRX2, (transcription factor activity). In proteolysis, we identified the subtilisin PCSK2, with 4-6 fold increases in 8 of its core probe-sets in MG-thymoma and increases in colon cancer (PHEX and ADAMTS20 (proteolysis), both up-regulated in MG-thymoma but not in colon cancer (immune response we found the histocompatibility MHC-II complex gene HLA-DRB1 (IL) 19 expressed higher in thymus than colon (SEMA3D (cell differentiation) was highly up regulated in MG-thymoma but not colon cancer , an inflammatory and apoptosis modulator Connective tissue growth factor  increased in MG-thymoma compared to healthy thymuses, with higher thymus than colon expression and increases in colon cancer as well  exhibited higher expression levels in the thymus than other examined tissues , which interacts with different proteins, including ASF/SF2  SC35 and ASF/SF2 decreases in the MG-thymoma tissues -rich protein  tissues , compatiAChE-R protein variant indicated a significant increase in the number of stained cells in the MG-thymomic section compared to healthy thymus  decreased according to the discrete approach compared to both response to external stimulus and detection of bacteria and compared to response to external stimulus (Muscle contraction (BP) globally increased according to both the discrete approach (P = 0.02) and to the KS test (P = 0.01), compatible with our recent report organismal physiological process, taxis  and establishment of localization (increases and decreased according to the discrete approach).FISH for y thymus . The larus  in MG-thymoma yielded two potentially new exons. These ACHE exons also appeared in exon array data sets from colon cancer and multiple healthy tissues ACHE exons  showed values close to those of the constitutive exon 2 , and overlapping changes were observed in binding  increased, both in the MS and according to the exon array analysis .ad-hoc and post-hoc statistics, in-depth analysis of key modified transcripts and multi-leveled validation tests to identify a hyper-alternative splicing signature for MG-thymomas. Our findings highlighted hyper-splicing in tumor-related, immune function and muscle-specific transcripts with distinct patterns from those of colon cancer or healthy thymus and included parent-child relationships in the GO hierarchy ASF/SF2 and SC35, previously shown as proto-oncogens HLA-DRB1, associated with a haplotype predictive of MG susceptibility in female We combined exon Arrays with MYH14 and in MYH10. A possible splice shift observed in MYH14 gene, as compared to decrease of it mRNA in colon cancer. Indeed, antibodies against structural muscle proteins, among them myosin, were detected by others in the sera of MG patients We found general increases in muscle-related GO terms and increased myosin levels (both mRNA and protein). The mRNA increase was significant both in COL1A1 gene, we detected up-regulation in both MG-thymoma and colon cancer. Another collagen important for cartilage collagen formation and for organization of the extracellular matrix COL11A1, was up-regulated in MG-thymoma, perhaps reflecting a compensation mechanism(s) over the degradation of its protein product. The diverse expression levels of core exon probe sets in COL11A1 between colon cancer and MG-thymoma tumors may unveil many unknown smaller transcripts, relevant to different types of cancer. Currently, most of cancer research is conducted using standard 3′ arrays, which are limited in their discovery scope. Our study highlights potential benefits from using high-resolution arrays in the study of cancer. Importantly, type I collagen reduces DPT mRNA levels DPT was down-regulated both in MG-thymoma and in colon cancer, suggesting inter-relationship between the observed changes.For cancer to metastasize from a primary tumor, the extracellular matrix-physiological barrier whose primary structural constituent is collagen-must be degraded to allow the passage of tumor cells. Collagen is further involved in the immune response to metastasis LPHN3, which may function in both cell adhesion and signal transduction, processes that are both essential in tumorigenesis IRX2, involved in the regulation of developmental processes via the WNT pathway and which is notably amplified in soft tissue sarcomas CNTNAP2, an exceptionally large gene which encompasses almost 1.5% of chromosome 7, and which functions in the vertebrate nervous system as both a cell adhesion and a receptor molecule Other MG-thyoma increased transcripts included the latrophilin ACHE gene for challenging the resolution power of the Exon array technology. In both MG-thymoma and control thymuses, we detected both the predictable 3′ alternative splicing and a previously unknown exon between introns 3 and 4, validated using RT-PCR, real-time RT-PCR and sequencing on MG-thymoma samples. In-depth analysis of Affymetrix exon array data sets suggests ubiquitous expression of this novel exon, which likely originated by ALu exonization . All MG patients included in the study were treated with anticholinesterases but not with corticosteroids or immunosuppressors. Thymoma tissues were removed for therapeutic purpose and fragments of normal thymus tissues were removed to make the large vessels more accessible during cardiac surgery. Pathologists at the Centre Chirurgical Marie Lannelongue  determined the classification of thymoma. The Institutional Review Board: Comité de Protection des Personnes de Kremlin-Bicêtre approved the use of human tissues based on verbal consents alone. Our IRB explained that written informed consents were not mandatory, since the tissues were initially harvested for therapeutic purposes and because all donors remained anonymous. Samples are described under RNA was extracted from frozen thymus samples using the RNeasy lipid tissue kit  as per manufacturer's instructions from healthy and pathological thymus samples. DNase treatment was applied to avoid DNA contamination. RNA integrity was confirmed by gel electrophoresis, and RNA concentration and purity were assessed spectrophotometrically.1 µg of total RNA from MG-thymoma , control  thymuses was labeled with the Affymetrix exon array whole transcripts sense targeting labeling assay and reagents, including r-RNA reduction and labeling with Streptavidin-phycoerithrin. Each sample was hybridized to a GeneChip® Exon 1.0 ST Array  according to manufacturer's instructions, and results scanned to create four .CEL files using Affymetrix GCS 3000 7G scanner and GeneChip Operating Software v. 1.3 to produce .CEL intensity files.http://www.affymetrix.com/support/technical/sample_data/exon_array_data.affx).Human colon cancer and all other available tissues sample data sets of Affymetrix exon array in .CEL files were downloaded from Affymetrix web site  were summarized using ExACT Mapping of transcript clusters to UniGene identifiersTranscript cluster IDs were assigned to their corresponding UniGene clusters using transcript_annot file Mapping of UniGene identifiers to corresponding GO termswww.geneontology.org) enabled defining of all the UniGenes (UG) associated with each represented GO term. Statistical tests found increases or decreases in expression using the discrete Fisher exact test with a 2-fold threshold, and continuous KS statistics, analyzing each tail separately.EASE Colon cancer exon array data set  functional analysis20 samples of colon cancer and healthy colons Colon cancer arrays comparison using GO analysesFrom colon cancer exon array data Enriched BP GO categories in the list of changed genes in colon cancer, using microarray analyses, obtained from Bush and coworkers MG-thymoma exon array data set functional analysisUG identifier lists were extracted and then GO MG-thymoma exon array permutated data setTo perform permutations, the median fold changes of the thymus samples were mixed twice. Then, log2 ratio between the mixed datasets was calculated. Continuous and discrete analyses were performed on the MG-thymoma permutated data . KS, variances and Kruskal-Wallis tests also served to examine specific changes in GO categories compared to their direct and indirect parent GO terms.Database constructionA MySQL database was constructed and used to store exon array expression signals at the exon probe-set level, as well as annotation data from the analyzed samples, in corresponding tables, for efficient transcript-specific queries.Other applicationsAn upper layer Java program was used to access specific transcripts information. To examine global exonic and gene-level signals at different gene fragments, a Visual Basic application was written. Matlab 7.1.0.246–R14 2) were photographed from each section with SC35 and ASF/SF2 labeling and the mean density per area was quantified with the software package ImagePro4 . Myosin immunohistochemistry was performed on 106μ2 sections and the same software was used to analyze the number of stained cells per area. Student's t-test was used to determine the statistical significance between healthy and MG-thymoma sections.Polyclonal antibodies for SC35, Myosin  and AS/SF2  were used at 1∶100, 1∶100, 1∶20 dilutions, respectively. Sections were deparaffinized, permeabilized and incubated with 100 mM Glycine in PBS for 20 min at room temp, preblocked with buffer containing 5% donkey serum, 0.5% Tween20 in PBS  and incubated with primary antibodies . Biotinylated antibodies were incubated with Cy3-conjugated streptavidin . Sections were coverslipped and analyzed by confocal microscopy using a Bio-Rad MRC-1024 scanhead  equipped with a digital camera and Olympus FV-1000 confocal microscopes, using excitation and emission parameters suitable for Cy3. Four areas ). Microscopy and data analysis involved four 106μ2 areas from each section and stained cell numbers per area were determined.Deparaffinization and rehydration of the tissue, permeabilization by proteinase K and prehybridization were as detailed elsewhere For each sample, 0.4 µg RNA was used for 20 µl cDNA synthesis using Promega RT-PCR kit . Real-time RT-PCR was performed in triplicate for each sample using ABI prism 7900HT and SYBR green master mix . ROX, a passive reference dye, was used for signal normalization across the plate. Primer sequences are described under * Additional Figure S1The number of GO categories presenting discrete and continuous changes in MG-thymoma by Venn diagrams. BP and MF categories that presented discrete (denoted as D) 2-fold change (dark gray) or continuous (denoted as C) change of median transcript exonic expression level using KS statistics (light gray) and both methods (intersection areas) as compared with the total population of UniGene clusters represented on the array. Note that in both MF and BP, more categories decreased than increased, and more categories showed change in the continuous approach than the discrete.(0.14 MB TIF)Click here for additional data file.Figure S2Tumor-specific Gene and Exon Level Expression Changes. Exon level probe sets showed a decrease fold change trend in MG-thymoma compared to healthy thymus (Ai). In colon cancer, an inverse increase trend appeared compared to healthy colon data (Aii). In both, the trend was conserved across all annotation levels-core, extended and full (Ai and Aii). Specifically, core level exons decreased in MG-thymoma compared to colon tumorgenesis events (Aiii). Median gene-level global exon array population exhibited decrease in MG-thymoma tissues compared to healthy thymuses, corresponding to exon-level changes (Aiv). The change was significant and differed from that of permutated populations . (B) The(1.17 MB TIF)Click here for additional data file.Figure S3Comparison of total global median core exons change in MG-thymoma and colon cancer. A) The median core exonic gene level signal is shown for MG-thymoma and colon cancer, for all the UniGene clusters represented on the array. Generally, colon cancer (N = 10) increased compared to MG-thymoma changes (N = 4) significantly . MG-thymoma and colon cancer samples were compared to matching healthy samples accordingly, and then to one another. B) (i) Median core exonic gene level signal, for MG-thymoma compared to healthy thymus samples (blue), with permutated patients and healthy samples changes (green and red). (ii) Median core exonic gene level signal, for colon cancer compared to healthy colon samples (blue), with permutated patients and healthy samples changes (green and red).(0.63 MB TIF)Click here for additional data file.Figure S4Comparison of median expression change in MG-thymoma 3′ 60 to 600 base pairs. Median exonic change of all array transcripts, was considerably smaller within the 3′ 60 nucleotides  then in the 3′ 600 bps . Statistical information under Supplementary Material.(0.36 MB TIF)Click here for additional data file.Figure S5Comparison between expression differences in various gene edges between the 3′ to 5′ of genes. A. MG-thymoma compared to healthy thymus samples (i)Median Exonic change of 3′ compared to 5′ 10% edge fragments of all array transcripts. There was a slight increase (CDF plot) in the 5′ Exonic change (high KS<0.05) with a change in location and dispersion. (ii) Median Exonic change of 3′ compared to 5′ 5% edge fragments of all array transcripts. There was a slight decrease (CDF plot) in the 5′ Exonic change (high KS<0.05) with a change in location and dispersion. (iii) Median Exonic change of 3′ compared to 5′ 2.5% edge fragments of all array transcripts. A striking difference between edges is observed, at both distributional tails , and both dispersion and location. (iv) In the 5′ edge, no change between median exonic changes observed between 10%, 5% and 2.5% of all the transcripts. B. Colon cancer compared to healthy colon samples Median Exonic change of 3′ compared to 5′ 2.5% edge fragments of all array transcripts. There was a decrease  in the 5′ Exonic change compared to the 3′ edge.(0.56 MB TIF)Click here for additional data file.Figure S6Specific GO category change compared to its parent terms. The change in RNA binding (N = 83) category compared to both direct and indirect GO parents. The blue and red bars indicate UniGenes that decrease and increased more then 2-fold, accordingly. (i) RNA binding decreased  compared to the global parent term, MF . (ii) RNA binding decreased  also compared to its direct parent term, nucleic acid binding (N = 280).(0.60 MB TIF)Click here for additional data file.Figure S7Median core exons change of ASF/SF2, SC35, Myosins and AChE-R. The median core exonic log fold change as compared between MG-thymoma to healthy thymus samples, for the validated genes. Myosin (MYH10), lymphocyte-specific myosin (MYLC2PL) and AChE-R increased, whereas SC-35 and ASF/SF2 decreased.(0.05 MB TIF)Click here for additional data file.Text S1Functional Analysis of Colon Cancer Data Set Exhibits Large Overlaps with Other Colon Cancer Data Sets. Results of comparisons between our continuous and discrete ad-hoc functional GO statistical analysis to other ad- and post- hoc functional GO analyses on colon cancer microarray and proteomic data sets.(0.10 MB DOC)Click here for additional data file.Text S2Global Exons Probe Sets Changes in Different Gene Portions of MG-Thymoma and Colon Cancer. Statistical results of the comparisons between different gene edges (3′ and 5′ regions) expression changes between MG-thymoma to healthy thymuses, and differs from permutated MG-thymoma data set.(0.03 MB DOC)Click here for additional data file.Text S4The Intron in which the New ACHE Exon is Located is Enriched with ALu Repeats. Results of repeat masker (0.04 MB DOC)Click here for additional data file.Text S3Core Probe Sets Normalized Signals Intensity of Detected MG-Thymoma genes. Expression signals intensity of core probe sets of the MG-thymoma genes detected by the specific term to parent ad-hoc approach combined with fold change threshold are given for MG-thymoma and healthy thymuses.(0.09 MB DOC)Click here for additional data file.Text S5Additional (0.04 MB DOC)Click here for additional data file.DataSet S11D-Gel Mass Spectrometry Proteomic Analysis Results on the MG-Thymoma and Healthy Thymus Samples. All the identified peptides are given, with the corresponding proteins identified by clustering analysis and NCBI database (1.72 MB XLS)Click here for additional data file.Table S1Ad-hoc Global Functional BP and MF GO Analysis of Exon Array Colon Cancer Data. Results of Affymetrix exon array colon cancer data set (0.62 MB XLS)Click here for additional data file.Table S2Ad-hoc Global Functional BP and MF GO Analysis of Exon Array MG-Thymoma Data Set. Results of Affymetrix exon array MG-thymoma data set are given for BP and MF. The columns of the Excel table are as follows: Name (GO term title) ,GO ID, KS-P (high), Fishex (INC),KS-P (low),Fishex (DEC), KS-P (high), percent changed genes and number of changed genes. The red and blue cells in the tables indicate the GO terms that significantly increased or decreased, respectively (p<0.05). Detailed description of each column is given under S1 description.(0.37 MB XLS)Click here for additional data file.Table S3Specific, Term-To-Parent Functional GO Analysis Results on MG-Thymoma Data Set. Results of hierarchical comparison between GO terms to their direct and indirect (BP/MF) parent GO terms in MG-thymoma exon array data set are given for all the terms represented on Affymetrix exon 1.0 S_T array. The columns of the Excel table are: Name (of GO term), GO ID (of the term), Parent name : p-value for the one tailed KS test , testing the hypothesis that the tested distribution contains larger values than the global distribution, Fishex (INC): p-value for the Fisher exact test using the hypergeometric distribution , testing the hypothesis that the tested distribution contains a larger than expected number of increased (INC) genes (using a 2-fold cutoff), KS-P (low)-a p-value for the one tailed KS test , testing the hypothesis that the tested distribution contains smaller values than those observed with the global distribution, Fishex (DEC): a p-value for the Fisher exact test using the hypergeometric distribution , testing the hypothesis that the testes distribution contain larger than expected number of decreased (DEC) genes (using a 2-fold cutoff), N-the number of transcripts associated with the GO term, KS accur (KS accuracy)-a number indicating the reliability of the KS test. The p- value is most accurate when KS accur >4.VAR: P value for variances (VAR) test indicating a change in dispersion and of Kruskal Wallis (KW) test indicate a change in location. The percent of transcripts that changed more than the threshold (both increased and decreased) of all the GO term transcripts (% change genes), the number of changed genes (# changed genes)-the number of transcripts that changed more than the threshold (both increased and decreased) out of all the GO term transcripts. The red and blue cells in the tables indicate the GO terms that significantly increased or decreased, respectively, at significance level of 0.05.(5.76 MB XLS)Click here for additional data file.Table S4Samples Description of MG-Thymoma and Healthy Thymus. Detailed description of the MG-thymoma and healthy thymus data set.(0.03 MB XLS)Click here for additional data file.Table S5www.frodo.wi.mit.edu/) and were used for both gel electrophoration of RT-PCR and for Real-Time RT-PCR.PCR Primers for ACHE New Exon. PCR primers design obtained from Primer3 software ((0.03 MB XLS)Click here for additional data file.


Microarrays are a powerful tool used for the determination of global RNA expression. There is an increasing requirement to focus on profiling gene expression in tissues where it is difficult to obtain large quantities of material, for example individual tissues within organs such as the root, or individual isolated cells. From such samples, it is difficult to produce the amount of RNA required for labelling and hybridisation in microarray experiments, thus a process of amplification is usually adopted. Despite the increasing use of two-cycle amplification for transcriptomic analyses on the Affymetrix ATH1 array, there has been no report investigating any potential bias in gene representation that may occur as a result.Here we compare transcriptomic data generated using Affymetrix one-cycle (standard labelling protocol), two-cycle  and IVT-Express protocols with the Affymetrix ATH1 array using Arabidopsis root samples. Results obtained with each protocol are broadly similar. However, we show that there are 35 probe sets  that are misrepresented in the two-cycle data sets. Of these, 33 probe sets were classed as mis-amplified when comparisons of two independent publicly available data sets were undertaken.Given the unreliable nature of the highlighted probes, we caution against using data associated with the corresponding genes in analyses involving transcriptomic data generated with two-cycle amplification protocols. We have shown that the Affymetrix IVT-E labelling protocol produces data with less associated bias than the two-cycle protocol, and as such, would recommend this kit for new experiments that involve small samples. Arabidopsis research involved using large quantities of tissue in order to generate the required amount of RNA. For example, Schaffer et al. . Other data sets used in this manuscript were obtained from GEO http://www.ncbi.nlm.nih.gov/geo/ with the accession numbers: [GEO: GSE5749] [GSE432] [GSE3350] [Data produced in these experiments have been made available from Arrayexpress [GSE432]  and [GEOGSE3350] .The authors declare that they have no competing interests.TJH produced the material, generated and analysed the data, and wrote the manuscript. MHW carried out bioinformatic analyses and wrote the manuscript. KK produced the statistical analyses and wrote the manuscript. ILD helped with the statistical analyses and initial direction of the investigation. TCH provided ideas for bioinformatic analyses. ATAW helped with the statistical analyses and the direction for the manuscript. MJH conceived the idea for a manuscript and aided the direction of the analyses. All authors read and approved the final manuscript.Pairwise comparisons of MS and EZ samples using the three different protocols. Pair-wise comparisons of biological replicates of Log2 data in the meristem  and elongation zone  tissues using the three different labelling protocols - A. 1-cycle; B. 2-cycle; C. IVT-E. R2 values are indicated in each comparison.Click here for fileQuantile plots to investigate 5' and 3' bias. For a given probability q between 0 and 1, the q-quantile of a data vector is the value cq, such that the proportion of the observations less than cq is equal to q. For example, the median is the 0.5-quantile. Some of the biases are very large, so we trim the bias observation vectors and only consider the quantiles between 0.05 and 0.95. These plots are shown above. From the plots, it is clear that the 3' bias is significantly higher for the 2-cycle data. This can be seen by the fact that in both of the plots in the figure, the red lines (representing the two-cycle labelling protocol) are lower than either the black or blue lines (one-cycle and IVT-E protocols). The IVT-E protocol shows marginally more 3' bias than the one-cycle protocol, but this is much less marked a difference than in the case of the two-cycle protocol.Click here for fileLoci with absolute residual values greater than 3.5 in pairwise comparisons. A table showing the ID numbers of probes sets classed as mis-amplified  in pairwise comparisons samples using different labelling techniques. The numbers of loci in each list are shown also in Table Click here for fileLoci classed as mis-amplified. A table showing the loci that have been classed as mis-amplified, along with the fold change of the mis-amplification (1-cycle vs. 2-cycle data) for MS and EZ tissues, AGI codes and locus descriptions. Positive fold changes indicate over-amplification, negative fold changes indicate under-amplification.Click here for fileRMA expression values of mis-amplified probes. The RMA normalised expression values of loci identified as mis-amplified in Table Click here for fileRelative expression levels of the over- and under-amplified probe sets. Relative expression levels (compared to median of the array) of the over- (above the break line) and under-amplified (below the break line) probe sets in other plant root data sets.Click here for fileRelative expression level of probe IDs classed as over- and under-amplified in publicly available one- and two-cycle data sets. Relative expression level of probe IDs classed as over- (above the break line) and under-amplified (below the break line) in publicly available one- and two-cycle data sets  and Birnbaum et al. (2003) respectively).Click here for file


High-throughput real-time quantitative reverse transcriptase polymerase chain reaction (qPCR) is a widely used technique in experiments where expression patterns of genes are to be profiled. Current stage technology allows the acquisition of profiles for a moderate number of genes (50 to a few thousand), and this number continues to grow. The use of appropriate normalization algorithms for qPCR-based data is therefore a highly important aspect of the data preprocessing pipeline.We present and evaluate two data-driven normalization methods that directly correct for technical variation and represent robust alternatives to standard housekeeping gene-based approaches. We evaluated the performance of these methods against a single gene housekeeping gene method and our results suggest that quantile normalization performs best. These methods are implemented in freely-available software as an R package qpcrNorm distributed through the Bioconductor project.The utility of the approaches that we describe can be demonstrated most clearly in situations where standard housekeeping genes are regulated by some experimental condition. For large qPCR-based data sets, our approaches represent robust, data-driven strategies for normalization. Recent technological advances have greatly expanded the total number of genes that can be analyzed in a single assay; qPCR experiments now regularly analyze "moderate" numbers of genes, in the range of fifty to a few thousand -3. Howevβ-actin (ACTB), and 18S ribosomal RNA [Nearly all normalization methods are based on the assumption that one or more genes are constitutively expressed at near-constant levels under all experimental conditions and the expression levels of all genes in a sample are adjusted to satisfy that assumption. The most widely used control genes are those selected from among an assumed set of "housekeeping" genes and typical selections include highly expressed transcripts such as glyceraldehyde-3-phosphate dehydrogenase (GAPDH), omal RNA -8. In moomal RNA ,10.More sophisticated normalization methods use multiple housekeeping genes where their respective measures are combined to represent a "virtual" housekeeping gene.. While tImprovements in qPCR technology have allowed significantly larger numbers of genes to be profiled simultaneously for each sample. This allows not only more experimental genes to be tested, but also provides an opportunity for a larger number of control genes, spanning a wide range of expression levels, to be used. However, these broad surveys also offer the possibility of introducing potential biases in normalization by using the data themselves to identify a set of appropriate controls. In many ways, this parallels widely used normalization methods developed for DNA microarray expression analysis, where data-driven methods have become the standard for most experimental designs. Here we present two normalization methods for high-throughput qPCR-based data adapted from those commonly applied in DNA microarray analysis: rank-invariant set normalization and quantile normalization ,13. As an-fold partition of the data, where n is the number of data points, and assumes that the data for individual samples have the same overall rank-order quantile distribution. Quantile normalization then adjusts the overall expression levels so that the distribution for all samples is equal.Quantile normalization is one of the most widely used methods in the analysis of microarray experiments . QuantilIn high-throughput qPCR experiments, the number of genes assayed in each sample can exceed the capacity of a single microtiter plate so assays for one sample are often distributed across multiple PCR plates. Consequently, in normalizing data, one must also consider plate-specific effects that may introduce a bias. To correct for plate effects in raw qPCR data, one natural solution is to apply a quantile normalization approach where we make the assumption that the distribution of gene expression measures is the same across all plates for the same experimental condition. This assumption will be reasonable when the gene allocation was based on factors that are unrelated to their expected expression levels; for example, genes are randomly assigned or the assignment is based on an alphabetical ordering by gene name. By forcing the distribution from each plate to be equal we remove the variability associated with plate-specific effects in the data. Note that alternative solutions to this problem exist, namely one could take a linear model-based approach which incorporates a covariate term to explicitly account for plate-specific effects.Quantile normalization proceeds in two stages. First, if samples are distributed across multiple plates, normalization is applied to all of the genes assayed for each sample to remove plate-to-plate effects by enforcing the same quantile distribution on each plate. Then, an overall quantile normalization is applied between samples, assuring that each sample has the same distribution of expression values as all of the other samples to be compared. A similar approach using quantile normalization has been previously described in the context of microarray normalization . BrieflyM of dimension k (maximum number of genes or primer pairs on a plate) rows by p (number of plates) columns. Plates with differing numbers of genes are made equivalent by padded plates with missing values to constrain M to a rectangular structure.1. qPCR data from a single RNA sample are stored in a matrix M'. The sorted columns correspond to the quantile distribution of each plate. The missing values are placed at the end of each ordered column. All calculations in quantile normalization are performed on non-missing values.2. Each column is sorted into ascending order and stored in matrix M'. Each column in M' is replaced by this average quantile distribution and rearranged to have the same ordering as the original row order in M. This gives the within-sample normalized data from one RNA sample.3. The average quantile distribution is calculated by taking the average of each row in N of dimension k  rows by n (number of samples) columns. Steps 2 and 3 are then applied to this matrix.4. Steps analogous to 1 – 3 are repeated for each sample. Between-sample normalization is performed by storing the within-normalized data as a new matrix et al.[The use of "housekeeping genes" in normalization is based on the assumption that one or more genes are expressed at near constant levels across all conditions assayed in a particular experiment. However, DNA microarray and high-throughput qPCR analyses have shown that many genes assumed to be constant in their expression can vary between conditions ,15,16. Ret al. for DNA R of dimension g  rows by s .1. qPCR data from all samples are stored in matrix et al.[s pairwise comparisons, relative to a common reference. We take the intersection of all s sets to obtain the final set of rank-invariant genes that is used for normalization.2. We first select gene sets that are rank-invariant across a single sample compared to a common reference. The reference may be chosen in a variety of ways, depending on the experimental design and aims of the experiment. As described in Tseng et al., the refαj represent the average expression value of the rank-invariant genes in sample j.  then represents the vector of rank-invariant average expression values for all conditions 1 to s.3. Let βj for sample j where βj represents the ratio of the rank-invariant average expression value in the first sample versus sample j, i.e. j = 1 to s.4. We calculate the scale factor j of R by the scale factor βj for j = 1 to s.5. Finally, we normalize the raw data by multiplying each column For our qPCR PMA time series data, we identified five rank-invariant genes to be used for normalization Table . glyceraThe advantage of the rank-invariant approach becomes very clear when you consider that with a standard normalization method, we would only normalize the raw data based on GAPDH or ACTB expression. However, not only does our approach validate the assumption of near constant expression of GAPDH and ACTB for this experiment, it also provides other stable genes that can be used for a more robust normalization without the need for any a priori selection.In order to compare our approaches with existing methods, we also performed normalization based on a single housekeeping gene, which for the PMA data set, was GAPDH. GAPDH was the only gene assigned to every plate and to take advantage of this design, we performed normalization in the following way:δj represent the average expression value of GAPDH in sample j.  then represents the vector of average GAPDH expression values for all conditions 1 to s.1. Multiple GAPDH expression measures are averaged within each sample. Let λj for sample j where λj represents the ratio of the GAPDH expression in the first sample versus sample j, i.e. j = 1 to s.2. We calculate the scale factor j with the scale factor λj for j = 1 to s.3. Finally, we normalize the raw data by multiplying the vector of data from sample The approach adopted represents an alternative version from how housekeeping gene normalization is typically performed in qPCR experiments. The standard approach taken is the delta-delta Ct method which involves two subtractions, first between the housekeeping gene expression value (in this case GAPDH) from the gene of interest measured in the control sample and second, the gene of interest measured in the experimental sample and the housekeeping gene in the experimental sample. We took this approach to exploit the design which involved GAPDH measurements being available for all plates in this data set. In doing so, our implementation of the housekeeping gene more closely parallels the other two methods.The PMA time series experiment described previously provided an opportunity to assess the performance of these methods relative to the use of the single housekeeping gene approach. The data was normalized using all three methods and the average gene-specific coefficient of variation (CV) was calculated to assess the overall reduction in technical noise associated with each approach. The CV measures the ratio of the standard deviation to the mean and captures the level of dispersion in the data. Therefore, a normalization method that better reduces technical noise will have a lower average CV. The results are shown in Figure Q1 and Q2 represent two normalization algorithms (e.g. Q1 = quantile). We calculate the variance of each gene expression profile normalized under Q1 and Q2 and plot the log2-transformed ratio of these variances as a function of the average expression of each gene for all genes. The red line represents a smoothed lowess curve that has been fitted to reflect the overall trend of the data [Q1 effects a greater reduction in the variation of the data relative to method Q2. Similarly, when the red curve is above Y = 0, method Q2 is more effective in reducing the variation. If the data from both methods have similar variances then the red curve should remain at Y = 0. Bolstad et al.[We also examined the variance of the raw and normalized data as a function of expression level. Let the data . When thad et al. originalFigure Comparisons of normalized to non-normalized data Figures  also supAs an example of how our normalization strategies perform in action, we highlighted the effects the different normalization methods had on four individual genes: E2F transcription factor 1 (E2F1), early growth response 1 (EGR1), v-myb myeloblastosis viral oncogene homolog (avian) (MYB), tumor necrosis factor, alpha-induced protein 3 (TNFAIP3). These genes were selected as examples because they were typical genes of interest for PMA-activated THP-1 cells. Comparing the variances of each profile . We expect the robustness of both the rank-invariant gene method and the quantile method to break down when the number of genes or primer pairs decreases and drops below this threshold. We were unable to assess this quantitatively however due to the lack of available data.The quantile method first focuses on the data within a sample, and applies a normalization correction to ensure the quantile distribution from different plates for the sample have the same quantile distribution. If there were a small number of genes or primer pairs assigned to each plate, then this normalization would be more susceptible to outliers. It is assumed that when there is a large number of genes or primer pairs, that the corresponding quantile distribution will be reasonably smooth and cover a realistic range of values expected for qPCR data. When the number of genes shrinks, the validity of this assumption may become questionable.et al.[et al. and colleagues suggest. In the case of rank-invariant set normalization, genes which do satisfy the properties of being a good control gene are easily identified. Quantile normalization corrects for plate-specific effects in the data by requiring samples to have similar distributions, although it does not have the aesthetic advantage of identifying control genes. Overall, our analyses indicate that both methods outperform approaches using a priori sets of housekeeping genes and that quantile normalization gives the best overall performance. Although we used a time course experiment to test these normalization approaches, these methods are applicable to any high-throughput qPCR setup.The data-driven normalization alternatives that we have presented have clear advantages when widely-used housekeeping genes are regulated by some experimental factor or condition. Vandesompele et al. use a paμL volumes in 384-well plates; each reaction contained: 0.5 U of HotStar Taq DNA polymerase (Qiagen) and the manufacturer's 1× amplification buffer adjusted to a final concentration of 1 mM MgCl2, 160 μM dNTPs, 1/38000 SYBR Green I (Molecular Probes), 7% DMSO, 0.4% ROX Reference Dye (Invitrogen), 300 nM of each primer (forward and reverse), and 2 μL of 40-fold diluted first-strand cDNA synthesis reaction mixture . Polymerase activation at 95°C for 15 min was followed by 40 cycles of 15 s at 94°C, 30 s at 60°C, and 30 s at 72°C. The dissociation curve analysis, which evaluates each PCR product to be amplified from single cDNA, was carried out in accordance with the manufacturer's protocol. Expression levels were reported as Ct values.The THP-1 cell line was sub-cloned and one clone (#5) was selected for its ability to differentiate relatively homogeneously in response to phorbol 12-myristate-13-acetate (PMA) (Sigma). THP-1.5 was used for all subsequent experiments. THP-1.5 cells were cultured in RPMI, 10% FBS, Penicillin/Streptomycin, 10 mM HEPES, 1 mM Sodium Pyruvate, 50 uM 2-Mercaptoethanol. THP-1.5 were treated with 30 ng/ml PMA over a time-course of 96 h. Total cell lysates were harvested in TRIzol reagent at 1, 2, 4, 6, 12, 24, 48, 72, 96 hours, including an undifferentiated control. Undifferentiated cells were harvested in TRIzol reagent at the beginning of the LPS time-course. One biological replicate was prepared for each time point. Total RNA was purifed from TRIzol lysates according to manufacturer's instructions. Gene-specific primer pairs were designed using Primer3 software , with anThe large number of genes assayed and the replicates measures required that samples be distributed across multiple amplification plates, with an average of twelve plates per sample. Because it was envisioned that GAPDH would serve as a single-gene normalization control, this gene was included on each plate. All primer pairs were replicated in triplicates.Raw qPCR expression measures were quantified using Applied Biosystems SDS software and reported as Ct values. The Ct value represents the number of cycles or rounds of amplification required for the fluorescence of a gene or primer pair to surpass an arbitrary threshold. The magnitude of the Ct value is inversely proportional to the expression level so that a gene expressed at a high level will have a low Ct value and vice versa.Replicate Ct values were combined by averaging, with additional quality control constraints imposed by a standard filtering method developed by the RIKEN group for the preprocessing of their qPCR data. Briefly this method entails:Ct1, Ct2, Ct3. Calculate differences between consecutive Ct values: difference1 = Ct2 - Ct1 and difference2 = Ct3 - Ct2.1. Sort the triplicate Ct values in ascending order: 4 overrides the other regions):2. Four regions are defined .JCM and JQ conceived of the study and the overall approach and were primary authors of the manuscript. JCM developed and tested the algorithms and evaluated the results and created the R package. KS and DH designed the laboratory experiments and KS generated the RNA samples. YK, KMI, HS, and YH and were responsible for designing and conducting the qPCR experiments; YK also contributed to testing and evaluating the normalization algorithms.All authors read and approved the final manuscript.Figure S1. Expression Profiles for Four Genes Normalized by Different approaches.Click here for fileTable S1. Variances of 4 Gene Expression Profiles Normalized by Different Approaches.Click here for fileTutorial and qpcrNorm Software. The R package qpcrNorm and a tutorial outlining its use.Click here for file


Gene expression microarrays provide a snapshot of all the transcriptional activity in a biological sample. Unlike most traditional molecular biology tools, which generally allow the study of a single gene or a small set of genes, microarrays facilitate the discovery of totally novel and unexpected functional roles of genes. The power of these tools has been applied to a range of applications, including discovering novel disease subtypes, developing new diagnostic tools, and identifying underlying mechanisms of disease or drug response. However, this technology necessarily produces a large amount of data, challenging us to interpret it by exploiting modern computational and statistical tools. In this brief review, we aim to indicate the major issues involved in microarray analysis and provide a useful starting point for new microarray users. Careful experimental design is crucial for a successful microarray experiment Slightly different oligonucleotide array platforms are manufactured by companies such as Affymetrix, Agilent, and NimbleGen , or instead to compare relative expression levels between a pair of samples on each microarray (two-color or two-channel arrays). There are tradeoffs between the two approaches. Single-color arrays allow for more flexibility in analysis, while two-color arrays can control for some technical issues by allowing a direct comparison in a single hybridization Design issues for two-color arrays are more complex As attractive as it might seem financially to run just one microarray for each “class” of samples  under consideration, replicates are essential for providing meaningful results During the experimental design stage, it is important to identify all the variables to be compared and to ensure that the proposed design allows their measurement. Be aware of other variables, such as patient age or date of sample collection, that might confound the distinction between the compared classes. One option is to randomize confounding variables related to experimental conditions under your control.The task of analyzing microarray data is often at least as much an art as a science, and it typically consumes considerably more time than the laboratory protocols required to generate the data. Part of the challenge is assessing the quality of the data and ensuring that all samples are comparable for further analysis.Normalization of the raw data, which controls for technical variation between arrays within a study, is essential Clustering is a way of finding and visualizing patterns in the data. Many papers and indeed books have been written on this topic . A range of methods to adjust for multiple testing are available . A) If the distributions are of the same overall shape, they can simply be scaled to the same mean. B) Quantile normalization imposes the same distribution on all samples. C) A known quantity of RNA is spiked-in to each sample  and is then used as a scaling factor.(1.57 MB EPS)Click here for additional data file.Text S1In this section we further discuss some of the issues raised in the main text.(0.23 MB RTF)Click here for additional data file.Table S1Comparison of commercial microarray manufacturers.(0.05 MB RTF)Click here for additional data file.


In order to study the molecular biological differences between normal and diseased tissues,it is desirable to perform classification among diseases and stages of disease usingmicroarray-based gene-expression values. Owing to the limited number of microarraystypically used in these studies, serious issues arise with respect to the design, performanceand analysis of classifiers based on microarray data. This paper reviews some fundamentalissues facing small-sample classification: classification rules, constrained classifiers, errorestimation and feature selection. It discusses both unconstrained and constrained classifierdesign from sample data, and the contributions to classifier error from constrainedoptimization and lack of optimality owing to design from sample data. The difficulty withestimating classifier error when confined to small samples is addressed, particularlyestimating the error from training data. The impact of small samples on the ability toinclude more than a few variables as classifier features is explained.


A novel and empirical method for normalization of RNA-seq data is presented The fine detail provided by sequencing-based transcriptome surveys suggests that RNA-seq is likely to become the platform of choice for interrogating steady state RNA. In order to discover biologically important changes in expression, we show that normalization continues to be an essential step in the analysis. We outline a simple and effective method for performing normalization and show dramatically improved results for inferring differential expression in simulated and publicly available data sets. The transcriptional architecture is a complex and dynamic aspect of a cell's function. Next generation sequencing of steady state RNA (RNA-seq) gives unprecedented detail about the RNA landscape within a cell. Not only can expression levels of genes be interrogated without specific prior knowledge, but comparisons of expression levels between genes within a sample can be made. It has also been demonstrated that splicing variants ,2 and siAn important aspect of dealing with the vast amounts of data generated from short read sequencing is the processing methods used to extract and interpret the information. Experience with microarray data has repeatedly shown that normalization is a critical component of the processing pipeline, allowing accurate estimation and detection of differential expression (DE) . The aimet al. [et al. [et al. [et al. [et al. [et al. [Current RNA-seq analysis methods typically standardize data between samples by scaling the number of reads in a given lane or library to a common value across all sequenced libraries in the experiment. For example, several authors have modeled the observed counts for a gene with a mean that includes a factor for the total number of reads -8. Theseet al.  use the  [et al. , both of [et al.  adjust t [et al.  log-tran [et al.  normaliz [et al.  illustraScaling to library size as a form of normalization makes intuitive sense, given it is expected that sequencing a sample to half the depth will give, on average, half the number of reads mapping to each gene. We believe this is appropriate for normalizing between replicate samples of an RNA population. However, library size scaling is too simple for many biological applications. The number of tags expected to map to a gene is not only dependent on the expression level and length of the gene, but also the composition of the RNA population that is being sampled. Thus, if a large number of genes are unique to, or highly expressed in, one experimental condition, the sequencing 'real estate' available for the remaining genes in that sample is decreased. If not adjusted for, this sampling artifact can force the DE analysis to be skewed towards one experimental condition. Current analysis methods ,11 have The fundamental issue here is the appropriate metric of expression to compare across samples. The standard procedure is to compute the proportion of each gene's reads relative to the total number of reads and compare that across all samples, either by transforming the original data or by introducing a constant into a statistical model. However, since different experimental conditions  express diverse RNA repertoires, we cannot always expect the proportions to be directly comparable. Furthermore, we argue that in the discovery of biologically meaningful changes in expression, it should be considered undesirable to have under- or oversampling effects (discussed further below) guiding the DE calls. The normalization method presented below uses the raw data to estimate appropriate scaling factors that can be used in downstream statistical analysis procedures, thus accounting for the sampling properties of RNA-seq data.Estimated normalization factors should ensure that a gene with the same expression level in two samples is not detected as DE. To further highlight the need for more sophisticated normalization procedures in RNA-seq data, consider a simple thought experiment. Imagine we have a sequencing experiment comparing two RNA populations, A and B. In this hypothetical scenario, suppose every gene that is expressed in B is expressed in A with the same number of transcripts. However, assume that sample A also contains a set of genes equal in number and expression that are not expressed in B. Thus, sample A has twice as many total expressed genes as sample B, that is, its RNA production is twice the size of sample B. Suppose that each sample is then sequenced to the same depth. Without any additional adjustment, a gene expressed in both samples will have, on average, half the number of reads from sample A, since the reads are spread over twice as many genes. Therefore, the correct normalization would adjust sample A by a factor of 2.The hypothetical example above highlights the notion that the proportion of reads attributed to a given gene in a library depends on the expression properties of the whole sample rather than just the expression level of that gene. Obviously, the above example is artificial. However, there are biological and even technical situations where such a normalization is required. For example, if an RNA sample is contaminated, the reads that represent the contamination will take away reads from the true sample, thus dropping the number of reads of interest and offsetting the proportion for every gene. However, as we demonstrate, true biological differences in RNA composition between samples will be the main reason for normalization.Ygk as the observed count for gene g in library k summarized from the raw reads, μgk as the true and unknown expression level (number of transcripts), Lg as the length of gene g and Nk as total number of reads for library k. We can model the expected value of Ygk as:A more formal explanation for the requirement of normalization uses the following framework. Define Sk represents the total RNA output of a sample. The problem underlying the analysis of RNA-seq data is that while Nk is known, Sk is unknown and can vary drastically from sample to sample, depending on the RNA composition. As mentioned above, if a population has a larger total RNA output, then RNA-seq experiments will under-sample many genes, relative to another sample.Ygk unspecified. Depending on the experimental situation, Poisson seems appropriate for technical replicates [Lg is generally absorbed into the μgk parameter and does not get used in the inference procedure. However, it has been well established that gene length biases are prominent in the analysis of gene expression [At this stage, we leave the variance in the above model for plicates ,7 and Neplicates . It is apression .Sk, cannot be estimated directly, since we do not know the expression levels and true lengths of every gene. However, the relative RNA production of two samples, k = Sk/Sk'f, essentially a global fold change, can more easily be determined. We propose an empirical strategy that equates the overall expression levels of genes between samples under the assumption that the majority of them are not DE. One simple yet robust way to estimate the ratio of RNA production uses a weighted trimmed mean of the log expression ratios ). For sequencing data, we define the gene-wise log-fold-changes as:The total RNA production, and absolute expression levels:To robustly summarize the observed M values, we trim both the M values and the A values before taking the weighted average. Precision (inverse of the variance) weights are used to account for the fact that log fold changes  from genes with larger read counts have lower variance on the logarithm scale. See Materials and methods for further details.fk) is required. It can be used to adjust both library sizes .We applied our method to a publicly available transcriptional profiling data set comparing several technical replicates of a liver and kidney RNA source . Figure The application of TMM normalization to this pair of samples results in a normalization factor of 0.68 . As ano [et al.  dataset, [et al.  exhibitset al. data set [Spike-in controls have the potential to be used for normalization. In this scenario, small but known amounts of RNA from a foreign organism are added to each sample at a specified concentration. In order to use spike-in controls for normalization, the ratio of the concentration of the spike to the sample must be kept constant throughout the experiment. In practice, this is difficult to achieve and small variations will lead to biased estimation of the normalization factor. For example, using the spiked-in DNA from the Mortazavi data set  would ledata set ).To investigate the range in utility of the TMM normalization method, we developed a simulation framework to study the effects of RNA composition on DE analysis of RNA-seq data. To start, we simulate data from just two libraries. We include parameters for the number of genes expressed uniquely to each sample, and parameters for the proportion, magnitude and direction of differentially expressed genes between samples . Figure et al. [et al. [To further compare the performance of the TMM normalization with previously used methods in the context of the DE analysis of RNA-seq data, we extend the above simulation to include replicate sequencing runs. Specifically, we compare three published methods: length-normalized count data that have been log transformed and quantile normalized, as implemented by Cloonan et al. , a Poisset al.  with libet al.  with lib [et al.  since th [et al.  or RPKM  [et al.  normalizet al. procedure requires both. We made the simulation data Poisson-distributed to mimic technical replicates  followed by next generation sequencing (ChIP-seq) may require a similar adjustment to compare between samples containing different repertoires of bound targets. Interestingly, the PeakSeq method  uses a lSimilar to previous high throughput technologies such as microarrays, normalization is an essential step for inferring true differences in expression between samples. The number of reads for a gene is dependent not only on the gene's expression level and length, but also on the population of RNA from which it originates. We present a straightforward and effective empirical method for normalization of RNA-seq data.k relative to sample r for gene g) and by absolute intensity (Ag). By default, we trim the Mg values by 30% and the Ag values by 5%, but these settings can be tailored to a given experiment. The software also allows the user to set a lower bound on the A value, for instances such as the Cloonan et al. dataset . SpecifYgk = 0 or Ygr = 0 are trimmed in advance of this calculation since log-fold-changes cannot be calculated; G* represents the set of genes with valid Mg and Ag values and not trimmed, using the percentages above. It should be clear that The cases where Mg, as is commonly done with log relative risk, and the inverse of these is used to weight the average.As Figure We compared the weighted with the unweighted trimmed mean as well as an alternative robust estimator (robust linear model) over a range of simulation parameters, as shown in Figure S4 in Additional file et al. [Human housekeeping genes, as described in , were doet al. .P-value for each gene. To apply TMM normalization, we replace the original library sizes with 'effective' library sizes. For two libraries, the effective library sizes are calculated by multiplying/dividing the square root of the estimated normalization factor with the original library size.For a two-library comparison, we use the sage.test function from the CRAN statmod package  to calcuet al. study [For comparisons with technical replicates, we followed the analysis procedure used in the Marioni l. study . Brieflyg in experimental condition zk. Their analysis utilizes an offset to account for the library size and a likelihood ratio (LR) statistic to test for differences in expression between libraries . In order to use TMM normalization, we augment the original offset with the estimated normalization factor. The same LR testing framework is then used to calculate P-values for DE between tissues. We modified this analysis to use an exact Poisson test for testing the difference between two replicated groups. The strategy is similar in principle to the Fisher's exact test: conditioning on the total count, we calculated the probability of observing group counts as or more extreme than what we actually observed. The total and group total counts are all Poisson distributed.where et al. [We re-implemented the method from Cloonan et al.  for the et al.  script.Yg). The mean is calculated from the sampled read counts divided by the sum Sk and multiplied by a specified library size Nk (according to the model). The simulated data are then randomly sampled from a Poisson distribution, given the mean. We have parameters specifying the number of genes common to both libraries and the number of genes unique to each sample. Additional parameters specify the amount, direction and magnitude of DE as well as the depth of sequencing . Since we have inserted known differentially expressed genes, we can rank genes according to various statistics and plot the number of false discoveries as a function of the ranking. Table S1 in Additional file The simulation is set up to sample a dataset from a given empirical distribution of read counts (that is, from a distribution of observed Software implementing our method was released within the edgeR package  in versiChIP: chromatin immunoprecipation; DE: differential expression; LR: likelihood ratio; RPKM: reads per kilobase per million mapped; TMM: trimmed mean of M values.MDR and AO conceived of the idea, analyzed the data and wrote the paper.A Word document with supplementary materials, including 11 supplementary figures and one supplementary table.Click here for file


The within-subject coefficient of variation and intra-class correlation coefficient are commonly used to assess the reliability or reproducibility of interval-scale measurements. Comparison of reproducibility or reliability of measurement devices or methods on the same set of subjects comes down to comparison of dependent reliability or reproducibility parameters.In this paper, we develop several procedures for testing the equality of two dependent within-subject coefficients of variation computed from the same sample of subjects, which is, to the best of our knowledge, has not yet been dealt with in the statistical literature. The Wald test, the likelihood ratio, and the score tests are developed. A simple regression procedure based on results due to Pitman and Morgan is constructed. Furthermore we evaluate the statistical properties of these methods via extensive Monte Carlo simulations. The methodologies are illustrated on two data sets; the first are the microarray gene expressions measured by two plat- forms; the Affymetrix and the Amersham. Because microarray experiments produce expressions for a large number of genes, one would expect that the statistical tests to be asymptotically equivalent. To explore the behaviour of the tests in small or moderate sample sizes, we illustrated the methodologies on data from computer-aided tomographic scans of 50 patients.It is shown that the relatively simple Wald's test (WT) is as powerful as the likelihood ratio test (LRT) and that both have consistently greater power than the score test. The regression test holds its empirical levels, and in some occasions is as powerful as the WT and the LRT.A comparison between the reproducibility of two measuring instruments using the same set of subjects leads naturally to a comparison of two correlated indices. The presented methodology overcomes the difficulty noted by data analysts that dependence between datasets would confound any inferences one could make about the differences in measures of reliability and reproducibility. The statistical tests presented in this paper have good properties in terms of statistical power. An extensive literature has been developed on procedures for testing the equality of two or more independent coefficients of variation as measures of reproducibility -5. TheirAn important application from molecular biology research in which correlated/dependent reproducibility coefficients are compared is when microarray technologies are compared in terms of reproducibility of gene expression measurements. DNA Microarrays are powerful technologies which make it possible to study genome-wide gene expressions and are extensively used in biological research. As the technology evolves rapidly a number of different platforms became available, which introduces some challenges for researchers to know which technology is best suited for their needs. There have been various studies that directly compared the performance of one platform with another in terms of cross-platform comparability and agreement of gene expression results. However the results of these studies are conflicting: some demonstrate concordance, others discordance between technologies -17. ThusMost of the microarray reliability or reproducibility and cross-platform studies use Pearson's correlation, as an index of reproducibility or agreement. However, it has long been recognized that application of procedures such as the paired t-test and Pearson's correlation are not appropriate tools for measuring agreement between measuring devices ,20. RathThe main focus of this paper is to develop several procedures for testing the equality of two dependent within-subject coefficients of variation computed from the same sample of subjects, which is, to the best of our knowledge, has not been dealt with in the statistical literature, and to evaluate the statistical properties of these methods via extensive Monte Carlo simulation. We propose two approaches; one is likelihood based , and the other is a regression based approach coined as PM test. After evaluating the statistical properties  of these tests using Monte Carlo simulation, the methodology is illustrated on data from two biomedical studies.xijl be the jth measurement of the ith subject by the lth instrument, j = 1,2,... ml, i = 1,2,... n, and l = 1, 2. To evaluate the WSCV we consider the one-way random effects modelSuppose that we are interested in comparing the reproducibility of two instruments. Let μl is the mean value of measurements made by the lth instrument, bi are independent random subject effects with bi ~ N θl = σl/μl as a measure of reproducibility. It determines the degree of closeness of repeated measurements taken on the same subject either by the same instruments or on different occasions under the same conditions. It is clear that, the smaller the WSCV, the better the reproducibility. We distinguish the WSCV from the coefficient of variation CVl involves ρl is population based. Therefore, more heterogeneity in the population would result in a large value of CVl. For that reason we shall focus our work on the WSCV rather than the CV. We also note that there is an inverse relationship between the ICC (ρl) and the corresponding within subject variance where iability ,23. Quaniability  argued tLetith subject, i = 1,2,....,n where m1 measurements obtained by the first method (platform), m2 measurements obtained the second method (platform). We assume that Xi ~ N, where denote the measurements on the k is a column vector with all k elements equal to 1, Ik is a k × k identity matrix and Jk and Jkxt are k × k and k × t matrices with all the elements equal to 1. Thus the model assumes that the m1 observations taken by the first platform have common mean μ1, common variance ρ1, whereas the m2 measurements taken by the second platform have common mean μ2, common variance ρ2. Moreover, ρ12 denotes the interclass correlation between any pair of measurements xij  and In these expressions 1lth method, the WSCV, which will be denoted as θl in the remainder of the paper is defined asFor the H0:θ1 = θ2 taking into account dependencies induced by a positive value of ρ12. We restrict our evaluation to reproducibility studies having m1 = m2 = m.Our primary aim is to develop and evaluate methods of testing X1, X2,.... Xn is a sample from the above multivariate normal distribution, then the log-likelihood function l, as a function of ψ =  is given by:If where,w = u1u2 - m2 ul = 1 + (m - 1)ρl, l = 1, 2 and,m - 1)ρ1}{1 + (m - 1)ρ2} > m2 m - 1) <ρl < 1 must be satisfied for the likelihood function to be a sample from a non-singular multivariate normal distribution.From  the condThe summary statistics given in (3) are defined as:μl and l = 1, 2. Clearly, m > 1. Therefore we shall assume that m > 1 throughout this paper. From [nm2 pairs .The maximum likelihood estimates (MLE) for er. From , we obtaH0:θ1 = θ2 requires the evaluation of variance of l = 1, 2, and The WT of a method ,27. On wψ = ',ψ1 = ', and I = -E⌊∂2l/∂ψ∂ψ'⌋ has the following structure:I12 = E(∂2l/∂ψ1∂ψ'2) = 0. Therefore, from the asymptotic theory of maximum likelihood estimation we have:This is based on a result from  100% confidence interval on (θ1 - θ2) given as:is approximately distributed under zα/2 is the (1-α/2)100% cut-off point of the standard normal distribution.H0 : θ1 = θ2 was developed numerically, and computed by first setting μl = σl/θl, l = 1,2 in Equation (3), and then adopting the following algorithm:An LRT of μl = σl/θl, l = 1,2 in Equation (3), thereafter;1- Set θ1 = θ2 = θ in (3)2- Set σ1, σ2, ρ1, ρ2, ρ12, θ) and;3- Minimize the resulting expression with respect to all six parameters  in the model.4- Subtract the minimum from the minimum of -20.It then follows from standard likelihood theory that the resulting test statistic is approximately chi-square distributed with 1 degree of freedom under HH0 : θ1 = θ2 is to set θ2 = θ1 + Δ, so that the null hypothesis is equivalent to H0 : Δ = 0, where Δ is unrestricted. Replacing μl by σl/θl, the log-likelihood function L is then independent of μl.One of the advantages of likelihood based inference procedure is that in addition to the WT and the LRT "Rao's score test" can also be readily developed. The motivation for it is that it can sometimes be easier to maximize the likelihood function under the null hypothesis than under the alternative hypothesis. A standard procedure for performing the score test of L = L = L and Let From  the scorwhereA1•2 are obtained from partitioning the Fisher's information matrix A so that A1•2 evaluated at Δ = 0. When an estimator other than the MLE is used for the nuisance parameters ψ*, provided that the estimator S is that of a chi-square with 1 degree of freedom [and  freedom ,30.A1•2 is quite complicated and we cannot obtain a simple expression for S that can be easily used. Moreover, we have also found through extensive simulations that while the score test holds its levels of significance, it is less powerful than LRT and WT across all parameter configurations. We therefore focus our subsequent discussion of power to LRT and WT.The score test has been applied in many situations and has been proven to be locally powerful. Unfortunately, the inversion of Pitman  and MorgLet di on si is linear [Direct application of the multivariate normal theory shows that the conditional expectation of s linear . That iswherewhereThe proof is straightforward and is therefore omitted.ρ12.It can be shown then from direct application of the multivariate normal theory that the conditional expectation (11) is linear, and does not depend on the parameter α = β = 0 if and only if μ1 = μ2 and σ1 = σ2 simultaneously. Therefore, testing the equality of two correlated coefficients of variations is equivalent to testing the significance of the regression equation (11). From the theory of least squares, if we define:From (11.a) and (11.b), it is clear that TSS - RSS)/(n - 2),H0 : α = β = 0 is rejected when RSS/EMS exceeds Fv,1,(n-2), the (1 - v) 100% percentile value of the F-distribution with 1 and (n-2) degrees of freedom [the hypothesis  freedom .The theoretical properties of the test procedures discussed thus far are largely intractable in finite samples. We therefore took a Monte Carlo study to determine the levels of significance and powers of these tests over a wide range of parameter values. For this study we generated observations from a multivariate normal distribution with covariance structure defined as in (2). Simulations were performed using programs written in MATLAB .n), the number of replications (m1 = m2 = m), and various values of . For each of 2000 independent runs of an algorithm constructed to generate observations from multivariate normal distribution, we estimated the true level of significance and power of the LRT, Wald, Score and PM tests using a nominal level of significance 5% (two sided) for various combinations of parameters.The parameters of the simulation included the total number of subjects  procedures for sample size of n = 50 and n = 100, respectively. It is seen that all procedures provide satisfactory significance levels at all parameter values examined. The empirical significance levels for smaller sample sizes  were also estimated. All test procedures provided empirical levels that are very close to the 5% nominal level (data not shown).Tables 2000 simulated datasets for WT and LRT in sample sizes n = 30 and 50, respectively. As alluded to earlier, the score test is excluded from the power Tables θ1 and θ2 is large, and showed greater power than both the Score and PM tests. The power of Score test was generally very low.Tables θ1 and θ2. We note that the equality of the means and variances implies the equality of the WSCV, but the reverse is not true. This strong assumption might explain the relatively poor performance of the PM test, particularly when the means are not well separated.For selected parameter values, power levels of PM, Wald, and the score tests for n = 50 subjects are given in Table To assess the effect of non-normality on the properties of the proposed test statistics we generated data from a log-normal distribution, and evaluated the performance of the four procedures for 2000 simulated datasets. The empirical levels of the regression based PM test were quite close to the 5% nominal level, but the power was poor. However, the likelihood based procedures  did not preserve their nominal levels for the majority of the parameters combinations (data not shown).We illustrate the proposed methodologies by analyzing data from two biomedical studies. In the first data sets we illustrate the methodology on the gene expression measurement results of identical RNA preparations for two commercially available microarray platforms, namely, Affymerix (25-mer), and Amersham (30-mer) . The RNAThe results presented in this section were not restricted to the group of differentially expressed genes, and we used the "control" part of the data for both technical and biological replicates. The normalized intensity values are averaged for genes with multiple probes for a given Gene ID. Hence, we have a sample size of n = 2009 genes measured three times (m = 3) by each of the two platforms (or instruments). We have used the within- gene coefficient of variation as a measure of reproducibility of a specific platform.ρ) are quite high. Using benchmarks provided in [The results of the data analyses are summarized in Table vided in , both plHere we demonstrate the statistical methodologies of this paper on a much smaller data set than the microarray gene expression example. The data are from a study using the Computer-Aided Tomographic Scans (CAT-SCAN) of the heads of 50 psychiatric patients ,34. The A comparison between the reproducibility of two measuring instruments using the same set of subjects leads naturally to a comparison of two dependent indices. In this paper, several procedures are developed for testing equality of two dependent within-subject coefficient of variations computed from the same sample of subjects. We proposed two approaches; one is likelihood based , while the other is regression based approach (extension of Pitman-Morgan). We assessed the powers and the empirical levels of significance of these methods via extensive Monte Carlo simulations. It is shown that the relatively simple Wald's test (WT) is as powerful as the likelihood ratio test (LRT) and that both have consistently greater power than the score test. A simple procedure based on results due to Pitman  and MorgDependence between the datasets would confound any inferences we could make about the differences in correlations. ... determination whether differences in correlation were statistically significant could not be made". In this paper, we have used the within- gene coefficient of variation as a measure of reproducibility of a specific platform. Therefore, a comparison across platforms leads naturally to a comparison of two dependent within-subject coefficients of variation.We illustrated the proposed methodologies with the analyses of data from two biomedical studies. The majority of microarray reproducibility and cross-platform agreement studies use Pearson's correlation, as an index of reproducibility and agreement, which would not be an appropriate measure of reproducibility. Because of the large heterogeneity among the genes in the data set, the intra-class correlation coefficient as an index of reproducibility of the platform would also not be an appropriate index of reliability as highly heterogeneous populations artificially produces high reliability index. Therefore, WSCV should be used as an index of reproducibility. In addition, the methodology presented in this paper overcomes the difficulty noted by Tan et al.  in whichTwo issues need to be discussed in this section. The first is related to the nature of the data to be analyzed while the other is related to situations when the assumed underlying model generating the data deviates from the normal distribution.First, a frequently occurring question in the planning of biomedical investigations is whether to measure the response or the trait of interest on a continuous scale  or dichotomous scale . In the case of two measuring devices and two dichotomous responses, the most commonly used measure of test-retest reliability or agreement is the kappa coefficient introduced in . Donner Second, it should be noted that the inference procedures discussed in this paper (except the PM test) are likelihood based and their statistical properties may not be appropriate in small samples. The difficulty is that the sampling distribution of a test statistics is unknown. Alternatively, one may use the bootstrap technology to estimate the sampling distributions of the test statistics. When the data are hierarchical in nature with variance covariance matrix Σ as shown in (2), one may use model-based approach to generate bootstrap samples , which iComparison of reproducibility or reliability of measurement devices or methods on the same set of subjects comes down to comparison of dependent reliability or reproducibility parameters. Testing the equality of two dependent WSCV has not been dealt with in the statistical literature. The presented methodology overcomes the difficulty noted by data analysts that the issue of dependence when ignored, would confound the inference on measures of reliability or reproducibility. It should also be emphasized that when comparison among platforms reliability indices the ICC is not an appropriate measure of reliability due to the large heterogeneity among the genes. Because the magnitude of the ICC depends on the degree of heterogeneity among the subjects it is not an appropriate index of reproducibility. We therefore recommend the WSCV in similar settings.The LRT and WT procedures presented in Section 2 may also be extended in a straightforward manner to compare more than two platforms . A further advantage of the LRT in this context is that it may easily be extended to deal with the case of an unequal number of replicates for each platform.The codes developed (in MATLAB) can be used to do power calculations for planning a reproducibility study when comparing two methods (or devices), and can be obtained on request from the authors.m1 = m2 = m)Elements of Fisher's information matrix (I22 is therefore given by:The matrix The authors declare that they have no competing interests.MMS conceived of the study problem and derived the analytical results. DC conducted the simulations and analyzed the data. All authors contributed to the writing of the manuscript, and approved its final format.The pre-publication history for this paper can be accessed here: