Over the past few years, both the U.S. Food and Drug Administration (FDA) and the pharmaceutical industry have recognized the potential importance of pharmacogenomics and toxicogenomics to drug development. To resolve the uncertainties surrounding the use of microarray technology and the presentation of genomics data for regulatory purposes, several pharmaceutical companies and genomics technology providers have provided the FDA with reports of genomics studies that included supporting toxicology data . These studies were not associated with any active drug application and were exploratory or hypothesis generating in nature. For training purposes, these reports were reviewed by the Nonclinical Pharmacogenomics Subcommittee consisting of the Center for Drug Evaluation and Research pharmacology and toxicology researchers and reviewers. In this article, we describe some of these submissions and report on our assessment of data content, format, and quality control metrics that were useful for evaluating these nonclinical genomics submissions, specifically in relation to the proposed MIAME/MINTox (minimum information about a microarray experiment/minimum information needed for a toxicology experiment) recommendations. These genomics submissions allowed both researchers and regulators to gain experience in the process of reviewing and analyzing toxicogenomics data. The experience will allow development of recommendations for the submission and review of these data as the state of the science evolves. To share and gain expertise in toxicogenomics and to develop a framework for reviewing data, pharmacology and toxicology reviewers and researchers within the U.S. Food and Drug Administration (FDA) Center for Drug Evaluation and Research (CDER) formed the Nonclinical Pharmacogenomics Subcommittee (NPSC) . The NPSa) microarray data quality using a data set generated for a single compound, b) assessment in a single tissue of the effects of several compounds in a pharmacological class, and c) use of a proprietary database to help classify a compound of unknown toxicity. None of the submissions involves a compound that is the subject of an active investigational new drug (IND) or new drug application (NDA). Further, these studies were not conducted under Good Laboratory Practice (GLP) regulations for nonclinical studies regulations  Society in their proposals of the minimum information about a microarray experiment (MIAME) . The fieWe describe in detail the three submissions in the sections that follow, with the sponsor’s conclusions and the NPSC comments for each submission. CYP)1B1 and CYP2B2 gene expression in rat liver. The report included all the data except for the histopathological findings in kidney and liver. Detailed protocols for microarray assay steps from RNA isolation to array scanning were provided in the submission. Gene expression analysis of liver samples collected 1, 7, or 30 days after drug administration was performed using Affymetrix  Rat Genome U34 Microarrays. Only the concurrent control and high-dose groups were analyzed for gene expression changes.The mechanism of action of a compound purported to reduce cholesterol levels by inhibiting HMG-CoA (3-hydroxy-3-methylglutaryl coenzyme A) reductase was investigated in female rats. The compound was administered daily by oral gavage at three dose levels for 1 month. Observations and measurements included clinical assessments (including body weights and food consumption), serum chemistry, macroscopic and microscopic evaluations of kidney and liver, and quantitative polymerase chain reaction (qPCR) of HMG-CoA reductase and cytochrome P450  to illustrate how this information might be incorporated in an actual microarray data submission. Hybridization controls to determine the sensitivity and quality of hybridizations included the measurement of BioB, BioC, and BioD genes  of the Escherichia coli biotin synthesis pathway and the cre recombinase gene for P1 bacteriophage . Data from these hybridization controls were provided in tabular and graphic form, including an analysis of the dynamic range. The sponsor also discussed the possible use of batch controls to monitor technical variability and equipment controls to ensure microarray image quality.The submission addressed the potential use of controls to monitor different steps in microarray experiments. Although labeling controls to monitor the efficiency of target preparation were not included in the mock data set, the sponsor included proprietary data on the use of polyadenylated transcripts of three in vitro transcription yield and the ratio of absorbance at 260 and 280 nm were proposed as QC metrics. QC metrics generated by Affymetrix Microarray Suite 5.0 (MAS5: Affymetrix) software from scanned images  were used to assess the quality of the hybridizations.The submission included a comprehensive section on quality control (QC) metrics. The sponsor provided examples based on their general laboratory experience to demonstrate the possible use of QC metrics for RNA and cRNA because these metrics were unavailable for the submitted data set. RNA quality was assessed by 28S:18S ratios and percent area . For cRNA, These metrics informed a paradigm for rejection of array data. Further, hybridization images were inspected for signal level, presence of image defects, and appropriate grid alignment at the edges and corners. Reproducibility between biological replicate hybridizations  was presented as a scatter plot and in tabular form. The bias in the data due to scanning of control and treated samples on different days was identified through examination of the data for the hybridization controls. The bias was mitigated by using a permutation analysis to reduce the number of false positives and by considering the biological relevance of the differentially expressed genes.Analysis of changes in liver gene expression was conducted to further understand the pharmacological mechanism of action of the compound through examination of its primary genomics target in the cholesterol biosynthesis pathway. The compound showed no significant treatment-related toxicity in the liver at the tested doses. A heat map was generated to compare samples collected from treated and control animals at different time points after drug administration. The heat map illustrated possible co-regulated genes that clustered by direction and extent of regulation at each time point. Changes in expression levels of genes encoding enzymes from relevant biochemical pathways were shown in figures and discussed in the text. The data were consistent with current understanding of cholesterol biosynthesis.The content and organization of the mock submission were a collaborative exercise between the NPSC and the sponsor. A preliminary technical plan was presented to the NPSC and followed by a draft proposal before the final mock submission. The sponsor submitted very extensive information on laboratory and informatics infrastructure. Such extensive information may not need to be included in all genomics data submissions where a summary is generally sufficient. However, this information should be well documented and available on request, possibly as part of a device master file or standard operating procedures (SOPs). Although the text of the submission contained information allowing a reviewer to trace a sample from the animal source to its corresponding array data set, it would have been helpful for the sponsor to provide a key in electronic file format to link these files.The sponsor submitted a comprehensive section on QC measurements. The NPSC concluded that these parameters appeared adequate. The electronic files were organized to be consistent with FDA guidance . The NPSThe NPSC observed that, in general, the protocols for the in-life portion of toxicogenomics studies should follow practices used for standard toxicity studies. These include the use of both sexes (unless there is scientific justification to limit the study to one sex) and using appropriate doses of drug. The number of replicates and power needed will vary depending upon the sponsor’s intended claim and the experimental objectives . HoweverIn general, the NPSC agreed that sufficient data were available to support the mechanism of compound action proposed by the sponsor. The NPSC considered the use of clustering analysis to identify co-regulated genes to be primarily hypothesis generating. Pharmacogenomics data that explore the presumptive mechanism of action of a compound may enhance traditional toxicology studies.In vivo study observations included clinical evaluations (including body weight and food consumption), clinical chemistry and hematology, and toxicokinetics. At necropsy, organ weights were determined and macroscopic evaluation conducted. The sponsor used SOPs in conducting the toxicology portion of the study. The livers were examined using both light microscopy and transmission electron microscopy (TEM). Enzymatic activity of components of β-oxidation pathways was also assessed in these samples.The potential human liver toxicity of compounds used to treat dyslipidemia was evaluated by treating male cynomolgus monkeys with one of three drugs. The test articles included two marketed peroxisome proliferator α-agonists  and a proprietary PPAR pan-agonist compound (PPARpan) with activity on PPARα, γ, and δ receptors. A vehicle control was also included. In the definitive study, monkeys (four per group) were treated for 15 days with a single dose of PPARpan or several dose levels for fenofibrate or ciprofibrate, respectively. Doses for fenofibrate and ciprofibrate in the 15-day study were based on a 4-day dose-range–finding study. Liver sections were collected for toxicogenomics analysis from all animals. RNA was isolated and the expression profile was examined using the manufacturer’s standard protocols with Affymetrix GeneChip HGU95Av2 arrays. The mock submission provided limited information about the protocols because these protocols are publicly available. Samples reported included control and high-dose animals from the range-finding study and all animals in the definitive study.To further understand the toxicity associated with PPARpan, the sponsor analyzed microarray data using principal component analysis (PCA) to examine the relationship between control and treatment groups in both the 4-day dose-range–finding study and the 15-day definitive study. The samples from animals treated for 4 and 15 days clustered differently. The sponsor attributed these differences to technical factors based on a similar separation among 4-and 15-day control animals. One 15-day control animal clustered away from other samples in the same treatment group and thus appeared to be an outlier. Ciprofibrate-treated samples clustered more tightly by dose level than fenofibrate-treated samples and demonstrated more pronounced dose–response alterations in gene expression. The single-dose level of PPARpan induced a modest alteration in gene expression. Using PCA, PPARpan effects appeared similar to those of the higher dose levels of fenofibrate.The sponsor assumed that the human-based probe sets with a MAS5 detection call of “present” were hybridizing to the correct homologous monkey sequence and that the data were interpretable because the objective was to compare gene expression between control and treated groups. Previous studies have demonstrated that cRNA from macaque and rhesus monkeys effectively hybridizes with a human Affymetrix GeneChip , althougTreatment-related effects were noted in clinical observations and serum chemistry, particularly at the highest doses. Liver changes reported at some or all drug-treatment groups included hypertrophy, increased liver weights, eosinophilia and granularity of the cytoplasm, and single-cell necrosis consistent with apoptotic cell death. This last finding was not observed with PPARpan. TEM evaluation indicated that all compounds increased peroxisome and mitochondria number as well as mitochondrial area. Activity of some enzymes of β-oxidation was increased in livers of ciprofibrate- and fenofibrate-treated animals. Further interpretation of the gene expression results, for example, regarding induction of apoptosis, has subsequently been performed by the sponsor .The initial submission consisted of separate toxicology and gene expression study reports of the same experiments. Although some experimental data such as animal treatment and husbandry were available through cross-reference to other protocols, this submission would have been easier to review if it had been organized as an integrated report using a format like MIAME/MINTox. Data for QC metrics were provided in the submission, but graphical presentation of some metrics would have aided the review. The toxicology data were very helpful in interpreting the difference in gene expression results between the PPARα compounds and PPARpan. The electron microscopy and morphometry analysis of peroxisomes and mitochondria provided additional detail relevant to the toxicogenomics assessment.Inclusion of electronic files in the mock submission made it possible to independently analyze the sponsor’s results. The NPSC used Spotfire software  to analyze the microarray data from the 4- and 15-day experiments. The NPSC chose not to analyze these data sets together because the significant technical variation between them might confound the analysis of biological relatedness between PPARpan and PPARα agonists. PCA by the NPSC of individual samples from the 15-day experiment showed that most samples did not form tight clusters within their treatment groups . This inAlthough cross-species hybridization is not an ideal approach, there are limitations in the available arrays for specific animal models. The sponsor acknowledged this fact and attempted to address it by providing confirmatory evidence with additional studies. With the advent of arrays for additional laboratory species, including the monkey, this technical challenge is not expected to be a major regulatory concern in the future.Despite the technical problems encountered and the different conclusions reached by the sponsor and the NPSC, this submission provided an example of the incorporation of gene expression analysis into an overall assessment of the potential toxicity of a new drug entity compared with a known drug class. Such an integrated submission potentially could be used to support a possible mechanism of action or to distinguish compounds within a pharmacologic class.The potential toxicity of a selective serotonin reuptake inhibitor (SSRI) was assessed in male rats treated daily for 5 days with vehicle and two dose levels of a test compound. One dose was a nontoxic pharmacologic dose, whereas the high dose was equivalent to maximum tolerated dose based on decreases in body weight gain. The results of molecular pharmacology assays, clinical observations (including body weights and food consumption), clinical chemistry and hematology, selected organ weights, necropsy observations, and liver histopathology were provided for four time points (6 hr to 6 days after the first dose was administered). Liver samples from individual animals were obtained and processed separately for microarray analysis. Gene expression studies were conducted using CodeLink RU1 Expression BioArrays . Expression data for three to six animals per time point per treatment group dosed for 1, 3, and 5 days were analyzed and compared with a contextual reference database of microarray data for approximately 600 compounds. The sponsor provided access to its proprietary database to allow NPSC members to independently confirm the gene expression findings.The toxicogenomics analysis indicated that the SSRI was relatively nonhepatotoxic, as confirmed by the histopathology findings. The sponsor defined a drug signature as “a small set of genes that delineates a property of one class of compounds from another or from vehicle controls.” The use of drug signature analysis confirmed several effects of the SSRI class that were observed using traditional pharmacology and toxicology assays (ion channel blocking and serum creatinine increase). Signature analysis suggested potential safety risks of perturbed blood pressure regulation and phospholipidosis, but ancillary data were not available to confirm these findings.This mock submission demonstrated the use of a contextual database containing genomics data annotated with toxicity data for the interpretation of the potential toxicity of an SSRI. The comparison of the pharmacology data and gene signature profile for this SSRI with similar compounds in the database also provided an example of gene profile specificity. In this example, independent verification of matches to some gene signatures was provided by clinical chemistry and molecular pharmacology assay results. Matches to other gene signatures could only suggest potential toxicities, as these are not probable valid biomarkers. The signatures would require additional experiments for confirmation. Potential toxicities suggested by gene signatures would be addressed in different sections of full IND/NDA submissions . The submission contained adequate information on sample and array quality assessment and on the statistical analyses that were applied to the data. The NPSC agreed that sufficient data were available to support the hypothesis proposed by the sponsor.The opportunity for the NPSC to access and be trained on the database provided an in-depth understanding of the use of a contextual database to generate gene expression signatures. Voluntary submissions may be a suitable route for sponsors to introduce FDA reviewers to similar approaches that use large data sets to generate gene signatures. The NPSC anticipates increased use of contextual databases as a resource for the generation of gene expression signatures or motifs that could eventually become validated biomarkers of toxicity. However, the gene components of these signatures would need to be available for review by regulators if not generally accepted, that is, a known valid biomarker, as defined by the FDA pharmacogenomics guidance (FDA 2005b).The NPSC is aware of several nonclinical pharmacogenomics submissions provided as part of IND and NDA submissions. These gene expression studies were designed to explore the pharmacology of the pharmaceutical or as an additional tool to investigate an observed toxicity. As described by the FDA pharmacogenomics guidance (FDA 2005b), most of these submissions likely would have qualified as voluntary submissions to the IND or as synopses to the NDA because they were exploratory studies in which no regulatory decisions were or could have been made. The sponsors were not using the pharmacogenomics data to make claims regarding safety or efficacy. The submissions did not involve either known or probable valid biomarkers. MIAME/MINTox criteria were not used in these submissions. All submissions were provided as paper reports, making it difficult to assess drug-related effects on gene expression. Most of these submissions reported only a subset of affected genes, which may be acceptable, although all data should be available upon request. In the opinion of the NPSC, submission of the data in an electronic format would have facilitated review of these analyses. Some of these studies were inadequately designed. Issues included insufficient numbers of animals per group per time point, too few time points, use of pooled tissue samples, and lack of QC metrics . Adequate documentation and evaluation of these parameters will be important for future regulatory submissions.The application of pharmacogenomics and toxicogenomics in drug development has primarily been used in compound selection and for identification of possible biomarkers of safety or efficacy. The NPSC anticipates that as confidence grows in the technology and guidelines for its use in a regulatory context are further delineated, sponsors will increasingly use it to address issues of regulatory importance. These may include studies of a drug’s mechanism of action or further investigation of a specific toxicity observed in a clinical or non-clinical study. The mock submissions described here serve as a basis for dialog within and outside the FDA to address how data are to be submitted, what data should be submitted, and what regulatory decisions are likely to be made with the data submitted.A structure such as that described by MIAME/MINTox  would beAnalyses such as PCA were helpful for identifying general similarities or differences among samples within or across treatment groups. Information not normally included in most submissions of toxicology data and not specified in MIAME/MINTox, such as information on laboratory informatics and equipment settings, may not be needed for review but should be available upon request. For example, information on array design description for commercially available arrays may not be necessary, but specifications should be provided for custom arrays. Quality metrics that were used for technical evaluation of the microarrays in these submissions were generally acceptable, but additional standards may be necessary for use of gene expression analysis in nonclinical toxicology assessments during drug development. It is not clear how much information regarding technical variation and equipment efficiency will be needed in a regulatory submission.The mock submissions were a useful tool for the NPSC to gain experience in how to best review toxicogenomics data. Additional voluntary genomics data submissions as described in the pharmacogenomics guidance (FDA 2005b) are encouraged so that the best practices for handling these data can continue to be developed. Many issues remain to be addressed. These include but are not limited to the amount of data needed to support a study’s conclusion, methods for the statistical evaluation of microarray data, the complexity of pathway analysis, and the need to make decisions concurrently with advances in related disciplines such as cell biology and molecular medicine.The FDA has co-hosted several workshops on pharmacogenomics and drug development with groups representing pharmaceutical manufacturers and the biological industry. Additional workshops are planned in the near future. Consortia have been formed to continue dialogue between regulators, industry groups, and academicians on these topics under the umbrella of organizations such as the International Life Sciences Institute (ILSI) through the ILSI–Health and Environmental Sciences Institute initiative . The Cli


We compared the relative precision and accuracy of expression measurements obtained from three different state-of-the-art commercial short and long-oligonucleotide microarray platforms . The design of the comparison was chosen to judge each platform in the context of a multi-project program.All wet-lab experiments and raw data acquisitions were performed independently by each commercial platform. Intra-platform reproducibility was assessed using measurements from all available targets. Inter-platform comparisons of relative signal intensities were based on a common and non-redundant set of roughly 3,400 targets chosen for their unique correspondence toward a single transcript. Despite many examples of strong similarities we found several areas of discrepancy between the different platforms.We found a higher level of reproducibility from one-color based microarrays (Affymetrix and CodeLink) compared to the two-color arrays from Agilent. Overall, Affymetrix data had a slightly higher level of concordance with sample-matched real-time quantitative reverse-transcriptase polymerase chain reaction (QRT-PCR) data particularly for detecting small changes in gene expression levels. For the last decade, genome-wide expression analysis technologies have been rapidly evolving. With the growing number of large data sets derived from a variety of technological platforms  cross-laboratory studies are becoming more challenging. These kind of studies are greatly effected by differences in target type (cDNA versus short or long oligonucleotide), target production and design, the efficiencies of target labeling and sub-optimal array production/hybridization protocols -4. TheseNevertheless, increasing the precision and accuracy of a given microarray study will ultimately increase the yield of meaningful biological information and thus reduce the cost of genome-based biomedical discovery. Since commercial expression profiling technologies, which typically offer a higher precision compared to homemade arrays, are not only becoming more abundant but also more affordable comparative studies are useful in making a cost-efficient choice. Several of these types of studies have been published -22. HoweWe present here, a comparison of the precision and accuracy of the latest and most exploited commercial high-density, pan-genomic short- and long-oligonucleotide reporter technologies. We compared data derived "in-house" from a common source of RNAs from 3 different human cell lines between Affymetrix HG-U133 plus 2.0 GeneChip™ (one-color), GE Healthcare Codelink™ (Amersham) (one-color) and Agilent Technologies 44 k whole human genome arrays (two-color). This comparison was conducted for the multi-project transcriptome program "Cartes d'Identite des Tumeurs"  launcheAll raw data sets were generated by each respective commercial entity  from a common source of biologically different RNAs . Data from a pre-release version of a fourth commercial platform  was also included in and limited to the evaluation of the level of discordance between the target measurements between the 3 platforms. The term "available targets" hereafter refers to target measurements that were not filtered out by a given platform. Different lists of targets (spotted or synthesized oligonucleotide located on the arrays) were used for the following different aspects of the analyses: all available target measurements were used for the comparisons of intra-platform reproducibility (precision); all available targets out of a common lists of 3,471 targets ("3.4 K" set) were used for all inter-platform comparisons of the accuracy of target measurement ; a sub-list of 14 genes were used for the comparison of microarray data and quantitative QRT-PCR data.2-ratios for one-color platforms compared to their corresponding CVs this transformation did not change the overall degree of variation relative to the two-color data obtained from Agilent. As another evaluation of intra-platform precision we calculated correlation coefficients (Pearson coefficient) stratified by intensity quantiles . A summary of the maximum (max) and minimum (min) correlations for a given quantile are shown in Figure In order to compare the precision of the measurements derived from each platform we used the following three criteria: 1. the percentage of targets having a 2-fold difference (or higher) of expression measurements obtained between each pair-wise combination of replicate slides; 2. the distribution of inter-slide (intra-cell line) variation measures; 3. inter-slide (intra-cell line) correlation of expression profiles analyzed for 4 intensity quantiles. One of the major observations from this study is that the data from all platforms were highly reproducible. We found that one-color platforms globally yielded lower variation measures compared to the two-color platform of Agilent Fig.  and the The evaluation of platform accuracy was based on the following criteria: 1. concordance between the expression levels of commonly represented targets on each array type; 2. agreement in the identification of genes discriminating the three cell lines; 3. concordance of relative expression compared to data obtained from QRT-PCR for subset of 14 genes. The first 2 evaluations were based on the available targets from the 3.4 K set. This 3.4 K set was specifically chosen in order to reduce the confounding effects of multiplicity (N targets: transcript (RefSeq identifier) relationship, see Methods). This approach afforded the most straightforward study of the concordance across the different platforms and did not favor platforms with high feature-to-target and/or target-to-transcript ratios. Moreover, this 3.4 K set of targets is representative, at the array feature-intensity level, of overall target measurements for each array type. Roughly 25% of the available 3.4 K targets were attributed to each of the 4 intensity quantile bins for each platform . Expres2-ratio data from all platforms yielded clusters of samples grouped according to the three cell lines showing a high overall correlation between the 4 platforms  and inter-platform (0.72) correlation, see Fig. Cluster analysis of logNevertheless, we found a significant number of aberrant target measurements unique to particular platform. Here the data from PX was included in order to confirm the discordance in target measurement observed for a particular platform. We found that while most of the 2,218 targets had a similar overall expression values between the 4 platforms 965 targets were found to be discordant across the different platforms Fig. . Of thesDifferences in target availability and discordant expression values between the platforms may be related to the differences in feature design, sequence and their placement along the target transcript. Recent sequence-based identification studies have suggested that annotations attributed to target sequence is a non-negligible source of discrepancy between technologies ,28. UsinStarting from all available targets out of the 3.4 K set for each platform we performed independent ANOVA and t test analyses. From the ANOVA analyses, we obtained 1,696, 1,727 and 1,781 significant targets (p < 0.001) for Affymetrix, Amersham and Agilent, respectively  using real-time QRT-PCR. These genes passed the pre-processing filters in at least one out all of the samples studied. In addition, these genes were selected based on either 1. complete concordance (CCNB1) or discordance between either their measured expression levels  and/or being identified as a discriminating gene between the different platforms ; 2. three genes  known to be highly expressed in MCF7 cells -32. ERBBDespite a high overall correlation between array and RT-PCR data Table  we foundA gene by gene analysis of the expression relative to the reference sample revealed a high agreement between array data and QRT-PCR data Table . This ob2-ratio (cell line/REF) array data to -ΔΔCt QRT-PCR data by -ΔΔCt FC bins. To optimize comparisons with single intensity platforms we also compared -ΔCt QRT-PCR values to log2-intensity values as a function of -ΔCt FC bins. Figure 2-ratio array data and -ΔΔCt data we observed a closer relationship of Agilent FCs to QRT-PCR FCs  25% of the FCs obtained from Amersham ratio data were below 0. If, however, we base our assessment of single-intensity platforms on single-intensity data we see a much closer correlation with QRT-PRC data, particularly for Affymetrix which yielded a near linear relationship with QRT-PCR data ranging from the minimal to the maximal FC bin . For this, the relative expression levels (ratios) between all pairs of genes across all 3 cell lines from the QRT-PCR data were calculated. Values that were filtered out by either Agilent or Amersham were not included in the comparison and thus we did not penalize these 2 platforms for missing values. A total of 3,321 and 3,828 combinations were considered for ratio and single intensity array data, respectively (see Methods). Each gene pair combination was attributed to 1 out of 9 FC bin depending on its QRT-PCR value. We then analyzed the concordance of log FCs Fig . We alsobin Fig. .Our study compares the latest, state-of-the-art short and long oligonucleotide human-microarray commercial technologies. The results presented here were used for the program CIT, a centralized, multi-tumor program as well as for a similar program launched by the Canceropole Ile-de-France (France's cancer research network) in choosing a human genome-based microarray technological platform that can meet a high-throughput demand. Therefore, the design of the study was adapted to judge each company in the context of these types of programs. In this context, our results show that all platforms yielded reproducible and comparable data. Nevertheless, between the 3 technologies we found that one-color platforms were more precise while Affymetrix and Agilent were more concordant based on the agreement of expression measurements and identification of differentially expressed genes. In addition, Affymetrix was found to have a slightly higher sensitivity at detecting relative differences in gene expression levels (irrespective of the magnitude of change) and an overall higher level of concordance with the QRT-PCR data.Human cell lines Jurkat (T cell acute lymphoblastic leukemia), MCF7 (breast cancer) were provided by INSERM U462, Institute Universitaire d'Hematologie, Hospital Saint Louis, Paris, France) and BattP12 (mesothelioma) was established and kindly provided by Dr. Marie-ClaudeJaurand . All cell lines were cultured in RPMI 1640 supplemented with 10% fetal calf serum, 10 U/ml penicillin, 100 μg/ml streptomycin and 2 mM L-glutamine. Cells were harvested during exponential phase growth and total RNAs were extracted as previously described . UniversBarring Agilent, all platforms use a one-color labeling approach. Agilent arrays were co-hybridized with cell line samples and the reference sample. Each platform followed protocols that are publicly available at their respective web sites -37. Brie2-ratios using the median value of the universal reference samples as the denominator.Filtering information with the aim of removing data obtained from non-viable targets (spotted or synthesized oligonucleotide) was provided by Amersham, Agilent and PX. Briefly, for Agilent, 20 different criteria were used to flag a specific target based on signal/noise ratio, spot morphology or homogeneity. For Amersham only spots with intensities above the negative control threshold (20% trim-mean of negative controls + 3 standard deviations using the trim-mean set) were included in the analysis. PX filtered targets based on a signal/noise threshold ratio of 3 and those called "not-detected" by their spot-detection software. The criteria used to flag targets were not imposed or suggested but rather derived from the respective platforms. We did not receive pre-processed or filtered data from Affymetrix. The following methods were used to normalize each data sets: robust multi-average (RMA) normalization and expression summary [39] for Affymetrix HG-U133 plus 2.0 raw data; median normalization (BRB-Array Tools v3.2 beta5.0) for Amersham CodeLink and PX raw, filtered data; lowess non-linear normalization [40] for Agilent raw, filtered data. Only available data from each platform were taken into consideration in our evaluation of platform discordance of expression measurements. When necessary, all single-intensity platform data  were transformed into logmean) between 1–1.25; cQ1.5: Qmean = 1.5; cQ2: Qmean = 1.75 – 2.25; etc.). We also determined the number of targets having "discordant" quantile attributions unique to a given platform (min Q – max Q = 2 while "similar" for 3 out of the 4 platforms) as well as discordant between all platforms  to each available targets out of all available targets per platform. The rank of the median intensities across all 3 cell lines for all targets were used to attribute an intensity quantile bin to a given target. Therefore targets in first quantile (Q1) have the 25% weakest expressed median intensities and Q4 corresponds to the targets having 25% strongest median intensities. We then determined the number of targets having similar inter-platform (consensus) quantile attributions  between all 4 platforms. Here "similar" refers to a situation where the minimal and maximal quantile attributions to a given target doesn't differ more than 1 . Using the quantile information for each platform 1 of 7 consensus quantiles  was attributed to each of these targets using the following partitioning: cQ1 for targets with mean quantile attributions (Qrms (see  for a liIn order to eliminate the problem of multiplicity (1: N and N: 1 relationships between targets and transcripts), only non-redundant targets (one feature per target ID) in which there was a 1 target: 1 transcript (RefSeq identifier) correspondence represented on the slide were considered . Sequence-based mapping between the different platforms was not feasible as oligonucleotide sequence information was not available for each of the 4 platforms (only for Affymetrix and Agilent). We used the provided representative sequence identifier (Genbank and RefSeq accession number) to map each oligonucleotide to an Entrez Gene identifier that was available at the time (April 2004). We found that out of the 6,664 unique sequence identifiers that were found in common between the different platforms, 6,404 mapped to Entrez Gene identifiers. Only single transcripts associated with a Entrez Gene identifiers represented only once on the arrays ) were used for direct comparisons of relative expression measurements. This set of 3.4 K targets were representative, at the feature-intensity level, for each array type .2 single-intensity data for Affymetrix and Amersham and log2-ratio data obtained for Agilent. "Discriminating targets" were determined using a nominal significance level of each univariate test of p < 0.001. The F test was based on comparing the differences in mean log2-intensities (or log2-ratios) between the 3 cell lines relative to the variation expected in the mean differences. Due to small sample population for each platform we used a random variance model for the univariate tests when distribution assumptions underlying this model were met [41]. These assumptions were satisfied for all platforms except for F tests conducted for the Affymetrix data .To determine the genes that discriminate the different cell lines we performed independent ANOVA analyses (F tests) and t tests (MCF7 versus Batt P12) for the 3 platforms. All univariate t and F tests using BRB ArrayTools (v3.2 b5) using the logFor the clustering analysis shown in Figure ® probe. After initial steps for 2 min at 50°C and 10 min at 95°C, 50 cycles at 95°C for15 sec and 60°C for 1 min were performed on an ABI Prism 7700 . Primers and Taqman® probes for the selected set of genes were chosen using Primer Express software  based on the primer-probe combinations prioritizing the 5' end of the transcript and in which the sense and anti-sense primers corresponded to 2 different exons separated by an intron of at least 500 bases. This yielded several potential combinations. The final combination for each gene was chosen based on thermodynamic criteria. All QRT-PCR reactions included in the analysis yielded amplification curves with ΔRN above 1.0, slopes between -3.1 and -3.8 and correlation coefficients above 0.98. Optimisation of each reaction was based on the performance of a given primer-pair/probe combination and the resulting standard curves from dilution series experiments . Absence of a specific transcript in a given cell line (eg. RAG2 transcript expression in Batt P12 and MCF7 cells and CASP5 transcript expression in Jurkat cells) was assessed based on an absence of amplification in the corresponding QRT-PCR assay. For RAG2 and CASP5 transcripts, sensitivity was experimentally determined to be at 1:10,000 dilution of Jurkat cDNA, and 1:1,000 dilution of peripheral blood mononuclear cell (isolated from blood samples after ficoll purification) cDNA in water, respectively. Based on this dilution series we obtained an optimal slope (-3.68 for Rag2 and -3.43 for CASP5). Normalization of the Ct values was carried out by subtracting the Ct values of the control gene ABL1 yielding ΔCt values for each gene [43]. ΔΔCt values were then derived by subtracting each ΔCt by the ΔCt value obtained from the same gene in the REF sample.Quantitative RT-PCR (QRT-PCR) reactions were performed as previously described [42] for each of 15 genes  between each array data sample replicate and the corresponding QRT-PCR data were calculated  are the HUGO gene symbol (Gene Symbol) and Entrez Gene public data bank identifiers. Also given are the intra-platform intensity quantile attribution calculated for each platform respectively  as well as the consensus quantile (cQ) calculated based on the mean quantile attribution obtained for each platform. ANOVA targets (p < 0.001) are indicated for each platform ("1") where "0" indicates a target not filtered and not an ANOVA target. AMER_MCF_BATT, AFFY_MCF_BATT, AGILENT_MCF_BATT indicates the targets that passed the p < 0.001 filter for the respective t tests comparing the MCF7 and Batt P12 samples , Amersham (Amer) or the filtered log2-ratios obtained for Agilent (Agilent44K). "NA" is shown for all empty fields corresponding to filtered data points.Click here for fileCluster analysis using the common ANOVA targets. Sample cluster dendrograms of all 36 platforms samples alone (A) or with other data including 66 tumor samples (B) and gene cluster dendrogram with associated heat map based on the expression profile of 1,096 (A) common ANOVA targets  found between Affymetrix, Agilent and Amersham. Mapping to the Affymetrix U133A array was based on RefSeq identifiers. Clustered sample groups corresponding to the cell line and similar tumor type are indicated by colored bars. Log2-ratios were centered and color coded (below heat map B). MCF-7 samples are represented with blue bars, BattP12 and mesothelioma samples are represented with red bars while Jurkat and TALL samples are represented by green bars. BattP12 is a mesothelioma cell line and clusters with the MESO samples (red bar) which as a group clusters with the MCF7 samples (blue bar). Jurkat is a T-cell acute lymphoblastic leukemia and clusters with T-ALL samples (green bar). As a part of the CIT program we obtained raw Affymetrix HG-U133A data for 66 samples corresponding to either pooled tumors of a particular cancer type and/or derived cell lines (unpublished data). These samples include the universal reference RNA from Stratagene (see above). Tissues represented are as follows: B-cell acute lymphoblastic leukemia ; colon cell line ; follicular lymphoma ; hepatocarcinoma ; myeloblastic acute leukemia ; epidermotrophal lymphoma ; chronic lymphoblastic leukemia ; myeloblastic chronic leukemia ; mesothelioma cell line ; epithelial cells ; lung (POUM n = 3); T-cell acute lymphoblastic leukemia T ; thyroid (THYR n = 4). Labeled cRNA reactions, hybridizations and image analyses for all 66 samples were carried out at the IGBMC, Strasbourg, France . Raw data was normalized and summarized expression values were generated using RMA (described above). For all cluster analyses, normalized intensity values were transformed into log2-ratios using the reference sample as the denominator. Cluster analysis was performed using log2 ratios  and DNA-Chip Analyzer (dChip) Version 1.3, with the following parameters: distant metric = 1-pearson correlation; linkage = centroid; row standardization.Click here for fileArray and QRT-PCR data for the 15 genes analyzed by QRT-PCR. This table is divided into three sections. Section I shows the filtered log2 intensities (Affymetrix and Amersham) or log2-ratios (Agilent) for the 15 genes analyzed by QRT-PCR. RNA batches are indicated above each table. Multiple targets were available for ERBB2 and RPS6KB1. Section II shows Ct values for each RNA batch obtained for each of the 2 batches . For sections I and II A red X is given for data points that were either filtered out by a given platform (microarray data) or for genes that were below the level of detection in a given cell line determined by QRT-PCR. Section III shows the number of possible versus available gene pair combinations QRT-PCR fold change bins used in Figure 2-ratio array data  or using -ΔCt values to compare with log2 intensities . Nine different bins were created with the following code (in bold) corresponding the following log2-ratios (x) intervals: <-5: x < -5; -3: -5 = x < -1.25; -1: -1.25 = x < -0.75; -0.5: -0.75 = x < -0.25; 0: -0.25 = x < 0.25; 0.5: 0.25 = x > 1.25; 1: 0.25 = x > 1.25; 3: 1.25 = x > 5.0; > = 5: x = 5.0.Click here for fileDescription of the 3 array formats used in the comparisons. The numbers represent the counts made of total number of targets represented on the respective microarrays. 1 The number of unique public sequence identifiers (ID) represented on the respective microarrays. These counts are based on the annotation table provided at the time the experiments were carried out . Control targets were excluded from the counts.Click here for filePrimer/Probe information for the 15 genes analyzed by QRT-PCR. Shown in the table are the 15 genes (Genes Names), accession number, forward and reverse primers and probes with associated 5'-3' sequence, 5' position and size analyzed by QRT-PCR.Click here for file


Accurate interpretation of data obtained by unsupervised analysis of large scale expression profiling studies is currently frequently performed by visually combining sample-gene heatmaps and sample characteristics. This method is not optimal for comparing individual samples or groups of samples. Here, we describe an approach to visually integrate the results of unsupervised and supervised cluster analysis using a correlation plot and additional sample metadata..We have developed a tool called the HeatMapper that provides such visualizations in a dynamic and flexible manner and is available from The HeatMapper allows an accessible and comprehensive visualization of the results of gene expression profiling and cluster analysis. Gene expression profiling by applying microarrays followed by cluster analyses is a powerful way to define pathobiologically relevant relations between the expression of sets of genes and disease classes. Unsupervised methods such as cluster analysis  and prinA second type of heatmap that is frequently used is a matrix of pair-wise sample correlations in which anti-correlation or correlation is indicated by a color-scale, e.g. blue to red -6. AlthoTo be able to correctly interpret both the sample versus gene expression heatmap and the sample versus sample correlation plot, data of the type of samples profiled, e.g. clinical parameters, karyotypes, mutations in particular genes, or gene expression data should be available. This information might then be included in a visual overview, as is frequently seen with sample versus gene heatmaps ,8. Such The HeatMapper, written in JAVA (version 1.4.2), uses comma-separated or tab-delimited text-files as input. It requires two files: one file containing a matrix of sample-sample similarity, i.e. Pearson correlation, Spearman correlation or Euclidean distance, and one file with sample related data. In both files, similar sample ID's are used. Correlation files can be generated using tools such as Omniviz, GeneMaths and R/BioConductor, while sample data files can for instance be created in Microsoft Excel. Example files are available from the website. Alternatively, the tool can be adapted to communicate with a database. In our laboratory, the HeatMapper is connected to a MySQL database which further optimizes the workflow. This version is available on request.As the upper right part of a traditional sample versus sample heatmap is in fact a mirror image of the lower left part, it is redundant. Therefore, when data are loaded, the HeatMapper only displays a triangular heatmap Figure . Sample-Our tool provides several advantages over more traditional means of presenting results obtained gene expression profiling and clustering analysis ,8. The pNPM1) mutations), indicated in red in the fifth column and (3) mRNA levels of CD34 are low in samples with NPM1 mutations.As an example, Figure In our laboratory the HeatMapper code has been coupled to a database containing gene expression profiling results, from which gene expression levels can dynamically be obtained. This allows the quick and accurate visual inspection of the distribution of expression levels in different clusters, and making the tool even more powerful. The database implementation, is available on request.Our visualization method has been successfully applied in several studies , 11, 12.With the increase of the number of samples profiled, particularly in patient-cohort studies, specialized visualization methods for microarray studies are indispensable. Our tool allows the accurate inspection of combinations of dataset characteristics, i.e. correlations and clustering results and sample related characteristics, i.e. survival time and gene expression levels. Summarizing, the HeatMapper tool results in powerful visualization tool that allows the accurate and rapid interpretation of the data obtained by large scale gene expression profiling. The HeatMapper tool has already proven to be very useful in several studies , 11, 12.Project name: HeatMapperProject homepage: Operating system: Platform independentProgramming language: JAVAOther requirements: JAVA 1.4.2 or higher.License: The tool is available free of charge. Source code is available upon request.Any restrictions to use by non-academics: NoneAML Acute Myeloid LeukemiaPNG Portable Network GraphicsNPM1 NucleophosminRGWV designed the software, participated in all phases of research and wrote the manuscript; MAS wrote the majority of the JAVA code; MAB contributed to software design and earlier code; RD gave intellectual contributions and revised the manuscript; SB contributed to the software code; MJM and PJS were involved in an earlier implementation of the software; BL gave intellectual contributions; PJV initiated the idea, gave intellectual contributions and revised the manuscript.


Here we outline advanced and universally applicable models for relative quantification and inter-run calibration with proper error propagation along the entire calculation track. These models and algorithms are implemented in qBase, a free program for the management and automated analysis of qPCR data. Since its introduction more than 10 years ago [To address the shortcomings of the available software tools and quantification strategies, we modified the classic delta-delta-Ct method to take multiple reference genes and gene specific amplification efficiencies into account, as well as the errors on all measured parameters along the entire calculation track. On top of that, we developed an inter-run calibration algorithm to correct for (often underestimated) run-to-run differences.Our advanced models and algorithms are implemented in qBase, a flexible and open source program for qPCR data management and analysis. Four basic principles were followed during development of the program: the use of correct models and formulas for quantification and error propagation, inclusion of data quality control where required, automation of the workflow as much as possible while retaining flexibility, and user friendliness of operation. Our quantification framework and software fit exactly in current thinking that places emphasis on getting every step of a real-time PCR assay right , especiqBase employs a proven, advanced and universally applicable relative quantification model. An important underlying assumption is that PCR efficiency is assay dependent and sample independent. While this may not be true in every experimental situation, there is currently no consensus on how sample specific PCR efficiencies should be calculated and used for robust quantification. Most evaluation studies attribute a lack of precision to these sample specific efficiency estimation methods. Hence, the gold standard is still the use of a PCR efficiency estimated by a serial dilution series , at least if one aims at accurate and precise quantification. Sample specific PCR efficiency estimation has its usefulness, but currently only for outlier detection -5.Calculation of relative quantities from quantification cycle values requires knowledge of the amplification efficiency of the PCR. As stated above, amplicon specific amplification efficiencies are preferably determined using linear regression  of a serial dilution series with known quantities (either relative or absolute). However, the error on the estimated amplification efficiency is almost never determined, nor taken into account. This error can be calculated using linear regression as well (formulas 2 to 4 and 6), and should subsequently be propagated during conversion of the quantification cycle values to the relative quantities. The formula for the error on the slope provides the mathematical basis to learn how more accurate amplification efficiency estimates can be achieved, that is, by expanding the range of the dilution and including more measurement points.E of the exponential function) and uses a single reference gene for normalization:Methods for the conversion of quantification cycle values  into normalized relative quantities (NRQs) were first reported in 2001. The simplest model described by Livak and Schmittgen  assumes NRQ = 2CtΔΔPfaffl  modifiedThis model constituted an improvement over the classic delta-delta-Ct method, but cannot deal with multiple (f) reference genes, which is required for reliable measurements of subtle expression differences . TherefoThe calculation of relative quantities, normalization and corresponding error propagation is detailed in formulas 7-16.The basic principle of the delta-Cq quantification model is that a difference (delta) in quantification cycle value between two samples  is transformed into relative quantities using the exponential function with the efficiency of the PCR reaction as its base. In principle, any sample can be selected as calibrator, either a real untreated control, or the sample with the highest or lowest expression. In addition, any arbitrary cycle value can be chosen as the calibrator quantification cycle value. The choice of calibrator sample or cycle value does not influence the relative quantification result; while numbers may be different, the actual fold differences between the samples remain identical, so results are fully equivalent and thus only rescaled. However, the choice of calibrator quantification cycle value does have a profound influence on the final error on the relative quantities if the error on the estimated amplification efficiency (see above) is taken into account in the error propagation procedure. To address this issue, we developed an error minimization approach that uses the arithmetic mean quantification cycle value across all samples for a gene within a single run as the calibrator quantification cycle value. As the increase in error is proportional to the difference in quantification cycle between the sample of interest and the calibrator formula 12), the overall final error is minimized if the mean quantification cycle is used as the calibrator quantification cycle value  and coefficients of variation (CV), we calculated these normalization quality parameters for our previously established reference gene expression data matrix obtained for 85 samples belonging to 5 different human tissue groups [in situ calibration); or normalizing against DNA . ClearlTwo different experimental set-ups can be followed in a qPCR relative quantification experiment. According to the preferred sample maximization method, as many samples as possible are analyzed in the same run. This means that different genes (assays) should be analyzed in different runs if not enough free wells are available to analyze the different genes in the same run. In contrast, the gene maximization set-up analyzes multiple genes in the same run, and spreads samples across runs if required Figure . The latWhatever set-up is used, inter-run calibration is required to correct for possible run-to-run variation whenever all samples are not analyzed in the same run. For this purpose, the experimenter needs to analyze so-called inter-run calibrators (IRCs); these are identical samples that are tested in both runs. By measuring the difference in quantification cycle or NRQ between the IRCs in both runs, it is possible to calculate a correction or calibration factor to remove the run-to-run difference, and proceed as if all samples were analyzed in the same run.Inter-run calibration is required because the relationship between quantification cycle value and relative quantity is run dependent due to instrument related variation , data analysis settings (baseline correction and threshold), reagents  and optical properties of plastics. Important to note is that inter-run calibration should be performed on a gene per gene basis. It is not sufficient to determine the quantification cycle or relative quantity relation for one primer pair; the experimenter should do this for all assays.To provide experimental proof of the advantage of sample maximization over gene maximization with respect to reduction in variation, we designed and performed an experiment consisting of five different runs Figure . The resTo our knowledge, there is only one instrument software that can perform such a correction, but the algorithm is based on the Cq values of a single IRC. Although it can be valid to calibrate data based on Cq values, this method has the drawback that the same template dilution needs to be used in all the runs to be calibrated . It is often much more straightforward and easier to calibrate the runs based on the NRQs of the IRCs (formulas 13-16). The quantity  of the calibrating input material is adjusted after normalization. This has the important advantage that independently prepared cDNA of the same RNA source can be used as a calibrator in the different runs . To some extent, even a biological replicate  can be used for inter-run calibration when doing the calibration on the NRQs, provided that the experimenter realizes this introduces some level of biological replicate variation (but still adequately removes inter-run variation). The validity of using independently prepared cDNA as calibrator is demonstrated by the experiment described in Figure It is also advisable to use multiple IRCs. A failed calibrator does not ruin an experiment if two or more are available. In addition, calibration with multiple IRCs gives more precise results with a smaller error. Based on our real calibration experiment, inter-run calibration using a single IRC inherently increases the uncertainty on the relative quantity by about 70% whereas a set of 3 IRCs increases it by only 40% Table . AlthougIt is important to note that formulas 13'-16' can only be used for inter-run calibration if the same set of IRCs is used in all runs to be calibrated. For more complex experimental set-ups (whereby different combinations of IRCs are used in the various runs), advanced inter-run calibration algorithms are currently being developed in our laboratory .The process of inter-run calibration is very analogous to normalization. Normalization removes the sample specific non-biological variation, while inter-run calibration removes the technical run-to-run variation between samples analyzed in different runs. As such, the same formulas can be used to calculate the inter-run calibration factor , and the same quality parameters can be applied to monitor the inter-run calibration process . Calculation of the IRC stability measure allows the evaluation of the quality of the calibration, which depends on the results of the IRCs. Our experiment shows that, with low M values , virtually identical results are obtained for the different selections of IRCs  are stored in the run layer; the experiment layer groups data from different runs that need to be processed and visualized together; and the project layer combines a number of related experiments . This hierarchical structure provides a clear framework to manage qPCR data in a straightforward and simple manner. The qBase Browser window is split into two parts: the bottom of the screen provides an explorer-like window to browse through the data; and the top of the screen contains a separate window displaying the annotation of the selected run, experiment or project. The qBase Browser allows the deletion and addition of projects, experiments and runs. The facility for exporting and importing projects and experiments is a convenient way to exchange data between different qBase users.Each qPCR instrument has its own method of data collection and storage, accompanied by a large heterogeneity in export files with respect to file format, table layout and used terminology. During import into qBase, the different instrument export files are translated into a common internal format. This format contains information on the well name, sample type, sample and gene name, quantification cycle value, starting quantity values (for standards), and the exclusion status. The last field indicates whether the measurement should be excluded from further calculations without actually discarding the measurement.Data can be imported from a number of data formats. Two standards ) and a number of instrument specific formats are supported. The qBase standard consists of a Microsoft Excel table in which the columns correspond to the information that is used internally by qBase. RDML is a universal format under development for the exchange of qPCR data under the form of XML files .The import wizard guides users through the process of data import Figure . To addrThe Analyzer is the data processing module for experiments. It performs relative quantification with proper error propagation along all quantifications, provides a number of quality controls and visualizes NRQs. This process involves several consecutive steps, some of them to be interactively performed by the user, others automatically executed by the program. Users are guided through the analysis by means of a simple workflow scheme in the main screen of the qBase Experiment Analyzer Figure .The first step in the workflow is the (automatic) initialization of an experiment, during which raw data from all individual run files from the same experiment are combined into a single data table. The initialization procedure also generates a non-redundant list of all the samples and genes within the experiment. There are no limits on the number of replicates, genes or samples contained within an experiment, except for those imposed by Excel . The absence of such limitations is a major improvement compared to the existing PCR data analysis tools, which are usually limited to processing data from a single plate or run with a fixed number of sample replicates. In qBase, data points with identical sample and gene names are automatically identified as technical replicates, except when the wells are located in different runs. In the latter case, they are interpreted as IRCs and renamed as such, that is, an appendix is added to indicate the run in which they are analyzed. Within the sample and gene lists on the main screen, a color code is used to label the reference genes and special sample types  layout with well annotation. It allows the modification of gene and sample names, as well as sample types and quantities in individually selected cells or in a range of neighboring cells. Together these tools allow users to review and correct the input annotation.Accurate relative quantification requires appropriate normalization to correct for non-specific experimental variation, such as differences in starting quantity and quality between the samples. The current consensus is that multiple stably expressed reference genes are required for accurate and robust normalization, especially for measuring subtle expression differences. While different tools are available to determine which candidate reference genes are stably expressed  is present for all genes (primer pairs); the quantification cycle values of NTCs are larger than a user defined threshold; the difference in quantification cycle value between samples of interest and NTCs is larger than a user defined threshold; the difference in quantification cycle value between replicated reactions is less than a user defined threshold; and genes are spread over multiple runs .After data quality control, a message box reports all quality issue alerts and the involved data points are color-coded in the data list. This allows users to easily evaluate their data and to select data points for exclusion from analysis without actually removing the data themselves.During initialization, samples are ordered alphanumerically, but the order of the samples can be adjusted in a user defined way. Samples can be re-ordered in the list by using the up and down keyboard arrows or the sample context menu. Samples that do not need to show up in the results can be excluded by using the delete button on the keyboard or the sample context menu. Apart from changing the default sample order and display selection in the Analyzer main screen, this can also be modified in a temporary gene specific manner when reviewing the results (see below).All quantification models transform (logarithm) quantification cycle values into quantities using an exponential function with the efficiency of the PCR reaction as its base. Although these models and derivative formulas have been used for years, no model or software has taken into account the error (uncertainty) on the calculated efficiency. qBase is the first tool that takes the error on the amplification efficiency into account by means of proper error propagation.Within qBase, gene specific amplification efficiencies can be specified in three ways. A default amplification efficiency (and error) can be set to all genes, or it can be provided for each gene individually. In the latter case, the efficiencies and corresponding errors can be simply typed , or calculated from a standard dilution series. qBase provides an interface for the evaluation of standard curves whereby outlier reactions can be removed. Amplification efficiencies are calculated by means of linear regression and can be saved to the gene list, in order to be taken into account during further calculation steps Figure .After raw qPCR data  quality control, reference gene(s) selection and amplification efficiency estimation, qBase can calculate the normalized and rescaled quantities. This process is fully automated and involves the following steps: calculation of the average and the standard deviation of the quantification cycle values for all technical replicates  - the program automatically detects the number of replicates for each sample-gene combination and can deal with a variable number of replicates (formulas 7-8); conversion of quantification cycle values into relative quantities based on the gene specific amplification efficiency (formulas 9-12); calculation of a sample specific normalization factor by taking the geometric mean of the relative quantities of the reference genes (formulas 13-14); normalization of quantities by division by the normalization factor (formulas 15-16); rescaling of the normalized quantities as requested by the user   , the PfaNormalization can be monitored by inspecting the normalization factors for all samples, or by calculating reference gene stability parameters. In an experiment with perfect reference genes, identical sample input amounts of equal quality, the normalization factor should be similar for all samples. Variations indicate unequal starting amounts, PCR problems or unstable reference genes. The qBase normalization factor histogram allows easy identification of these potential problems.One of the unique features of qBase is the option to normalize the relative quantities with multiple reference genes, resulting in more accurate and reliable results. In addition, qBase evaluates the stability of the applied reference genes  by calculating two quality measures: the coefficient of variation of the normalized reference gene expression levels; and the geNorm stability M-value. Both values are only meaningful, or can be calculated only if multiple reference genes are quantified. The lower these quality values, the more stably the reference genes are expressed in the tested samples. Based on our reported data on the expression of 10 candidate reference genes in 85 samples from 13 different human tissues [qBase is especially useful and unique for analysis of experiments containing multiple runs. As users are usually interested in comparing the expression for a given gene between different samples, the sample maximization experimental set-up is the preferred set-up because it minimizes technical (run-to-run) variation between the samples. Nevertheless, the gene maximization set-up is also frequently used. To correct the inter-run variation introduced by this set-up as much as possible, qBase allows runs to be calibrated (on a gene specific basis) using one or multiple IRCs Figure . If no sNormalized and rescaled relative quantities can be presented in three ways: a single-gene histogram, a multi-gene histogram, or a table. The default sample order and sample selection is defined in the main qBase window by editing the sample list. For the single-gene histogram Figure  the defaqBase is freely available for non-commercial research and can be downloaded from the qBase website .For the training of new qBase users we have designed a demo experiment that is explained in detail in a step-by-step tutorial. Demo experiment 1 consists of 4 runs (96-well format) containing 16 samples, 5 standards, and a no template control to be analyzed for 5 genes of interest and 3 reference genes. Demo experiment 2 adds two runs to the initial experiment, expanding it with eight additional samples and three calibrators for inter-run calibration. After training, complete analysis of these six plates can be performed in less than an hour. This includes data import, correction of well annotation, quality control, determination of amplification efficiencies, inter-run calibration, calculations and results interpretation. To our knowledge, there are no other tools available that can perform all these functions. Conventional spreadsheet calculations would take considerably longer, are error prone and do not include quality control.Although qPCR has been around for more than ten years, the employed calculation models are still amenable for improvement. Here we report our advanced, and proven, model for relative quantification that uses gene-specific amplification efficiencies and allows normalization with multiple reference genes. Errors are propagated throughout all calculation steps, and previously ignored errors, such as the uncertainty on the estimated amplification efficiency, are now taken into account. In addition, we developed an inter-run calibration method that allows samples analyzed in different runs to be compared against each other.We implemented these improved and innovative methods in an easy to use, Microsoft Excel based tool for the management and the automated analysis of qPCR data, coined qBase. This freely available software package incorporates several data quality controls and uses an advanced relative quantification model with efficiency correction, multiple reference gene normalization, inter-run calibration and error propagation along each step of the calculations. A configurable graphical results output and the possibility to import and export experiments allow easy results interpretation and data exchange, respectively.As a final comment, we would like to point out that, although our framework and program help management and interpretation of mRNA data, assessment of biological relevance or statistical significance requires the correlation of these mRNA data with protein levels or activity, and the measurement of biological replicates, respectively.According to the Real-time PCR Data Markup Language (RDML) we used the proposed universal terms for the plethora of available descriptions  instead of cycle threshold value (Ct), take off point (TOP) or crossing point (Cp)).Error propagation is performed using the delta method, based on a truncated Taylor series expansion.2 transformed  relative quantity ratio; V, geNorm pairwise variation; M, geNorm stability parameter.N, number of replicates i; g, number of genes j; c, number of IRCs m, m'; r, number of runs l, l'; s, number of samples k; f, number of reference genes p, p'; h, number of standard curve points q with known quantity Q; Cq, quantification cycle; CF, calibration factor; NF, normalization factor; RQ, relative quantity (relative to other samples within the same run for the same gene); NRQ, normalized relative quantity; SE, standard error; IRC, inter-run calibrator; CV, coefficient of variation; A, column matrix in which each element consists of the logA standard curve can be generated from the Cq and quantity values of a dilution series measured for the same amplicon within a single run. The slope and its standard error can be calculated for this curve by means of linear regression:E, and its standard error SE(E) are calculated from these values:The base for exponential amplification jk within a given run l:Calculation of the average Cq value for all replicates of the same gene/sample combination Ejl, with minimization of the overall error:Transformation of mean Cq value into RQ using the gene specific PCR efficiency Cqjkl = Cqreference, jl - Cqjkl     (formula 10)ΔThe procedures for normalization and inter-run calibration are highly analogous and are therefore described in parallel.NF for sample k based on the RQs of the reference genes p.Calculation of the normalization factor CF for gene j in run l based on the NRQs of the IRCs m:Calculation of the calibration factor NRQ, see formula 15)Conversion of RQs into NRQs.Conversion of NRQs into CNRQs:k and a given reference gene p:Calculation of the mean NRQ for all samples CV of a given reference gene p across all samples k:Calculation of the coefficient of variation Calculation of the mean coefficient of variation for all reference genes:Since normalization and inter-run calibration are highly analogous, quality evaluation using the stability parameter M is similar as well. Therefore, both methods are explained in parallel.Agene in which the kth element is the log2 transformed ratio between the relative quantities  of two reference genes p and p' in sample k; matrix Asample is calculated in an analogous manner.Calculation of the s × 1 matrix Airc in which the jth element is the log2 transformed ratio between the NRQs of two IRCs m and m' for the same gene j within a run l; matrix Arun is calculated in an analogous manner:Calculation of the g × 1 matrix Vgene for all samples k and a given reference gene combination . Vgene is the geNorm pairwise variation V for two reference genes.Calculation of the mean log transformed ratio and the standard deviation Virc for all runs l and a given IRC combination  and a given gene j. Vsample and Vrun are calculated similarly from Asample and Arun, respectively:Calculation of the mean log transformed ratio and the standard deviation Mgene of all pairwise variations Vgene of a given reference gene p with all other tested reference genes p'. Mgene represents the geNorm gene stability measure M for a particular reference gene p.Calculation of the arithmetic mean Mirc of all pair wise variations Virc of a given IRC m with all the other IRCs m', for the same gene j. Msample and Mrun are calculated similarly from Vsample and Vrun, respectively:Calculation of the arithmetic mean Calculation of the mean stability measure for all reference genes.Calculation of the mean stability measure for all IRCs:The calculations for Figure For the calculation of the effects of inter-run calibration, NRQ values were retrieved from qBase for runs 1, 2 and 3 independently. Inter-run calibration was performed as described in formulas 13'-16', using one, two or three IRCs . The effect of inter-run calibration with two IRCs was calculated on the three sets of two IRCs . Similarly, the effect of inter-run calibration with one IRC was calculated over all individual IRCs.The increase in error is defined as the ratio of the relative error after and before calibration. The 95% confidence interval (CI) for this increase was calculated on log-transformed ratios. For the investigation of the effect of the selection of (sets of) IRCs from the three available calibrators, CNRQs for the different calibrated data sets were rescaled to allow them to be compared. The fold difference between the data sets was log transformed and a 95% CI was calculated. The effect of calibration with identical or independently prepared cDNA was studied similarly to the effect of the selection of IRCs. The IRC stability measure was calculated as described in formulas 21'-25'.The following additional data are available with the online version of this paper. Additional data file Data and calculations leading to the results presented in Figure 3Click here for fileThe conclusions of these calculations are represented, in part, in Table 2Click here for file


If you’re going through an identity crisis, you might not want to consider the fact that, at the cellular level, you’re really more microbe than human. A hundred trillion single-celled organisms, representing some 400 bacterial species, inhabit the adult human gut, out-populating the cells in your body by a factor of ten. Nonetheless, it might comfort you to know that in exchange for room and board, your microbial cohorts offer several essential services, from pathogen protection to nutrient metabolism, and likely others yet to be discovered.The colonization of the sterile newborn gastrointestinal tract by a thriving microbial community is a pivotal milestone in human development. Yet, many aspects of this process remain obscure. Where do the microbes come from? What affects how the colony changes over time? These are just a few of the questions that Chana Palmer, Patrick Brown, and colleagues shed light on in a new study. To their surprise, the researchers found that healthy babies had remarkably different microbial communities living in their guts during their first months of life.With the help of 13 mothers, Palmer et al. collected stool samples from 14 healthy babies , starting with daily samples just after birth, and tapering off over the first year. From these specimens—the next best thing to sampling the colonic space directly—the researchers profiled the microbial communities present during colonization. To determine the likely provenance of the microbes, they also collected stool samples from many of the parents and siblings, as well as vaginal and breast-milk samples from the mothers.The researchers extracted genetic material from the samples, and used DNA microarrays and gene sequencing to reconstruct the colonization process. DNA microarrays are usually used to examine the expression levels of thousands of different genes in a cell at once, but Palmer et al. designed a custom DNA microarray that used a single gene capable of recognizing thousands of distinct microbial species. Acting as a species fingerprint, the small subunit ribosomal DNA (SSU rDNA) gene is found in every species and evolves relatively slowly, making it a favorite tool for classifying microbes, based on variations in this gene’s sequence across organisms.To assess the performance of this novel tool, the researchers created a pool of SSU rDNA sequences from 12 diverse samples and generated community profiles of each sample using both the sequencing data (the “gold standard” approach) and their microarray. The two methods produced remarkably similar results, demonstrating that the microarray could reliably be used to classify microbes and estimate their relative abundance. Notably, nearly all of the 4,000-plus sequences analyzed were dominated by just three major taxonomic divisions.Estimates of bacterial density during the first week of life  revealed no significant link between mode of delivery and onset of colonization. Interestingly, however, the only two babies delivered by cesarean section, before the amniotic membranes had ruptured—and thus without exposure to the microbial flora present in their mother’s birth canal—had lower bacterial counts than the others for that first week.The researchers went on to analyze all of the baby stool and familial samples with their microarray. Again, just three major taxonomic divisions predominated, yet the specific members of those large taxonomic groups, the timing of their appearance, and the duration of their stay varied widely among the babies. The fraternal twins were a striking exception—their microbial flora was by far more similar to each other than to other babies. Despite these individual variations, the profiles for each child showed a surprising degree of continuity—each baby could be recognized by its distinctive microbial flora for weeks and months at a time. By the time the babies were one year old, however, their profiles had converged toward a characteristic “adult-like microbiota,” which, the researchers point out, still retained a stamp of individuality.The idiosyncratic nature of the early stages of colonization suggests that a baby’s initial bacterial profile largely results from incidental microbial encounters. The fact that some of the early stool samples matched their mother’s breast milk or vaginal sample supports this interpretation. Shared environment may also explain the coincidental appearance of microbes in the twins. The researchers explain the tendency of these communities to eventually converge by hypothesizing that the human–microbe symbiosis has likely evolved under strong selection and that certain well-adapted microbes repeatedly “win” the battle over the opportunistic early colonizers.By comparing the surprising range of microbial profiles found in these healthy babies to the microbiota of infants born prematurely or with health problems, future studies can explore how diet, delivery method, or other factors might spell the difference between health and disease. The researchers’ custom-designed microarray provides a valuable resource for exploring such questions, as well as the environmental and genetic factors that shape and personalize the amazing “alien” ecosystem that lives within us.


Commonly employed clustering methods for analysis of gene expression data do not directly incorporate phenotypic data about the samples. Furthermore, clustering of samples with known phenotypes is typically performed in an informal fashion. The inability of clustering algorithms to incorporate biological data in the grouping process can limit proper interpretation of the data and its underlying biology.k-prototypes algorithm, for clustering biological samples based on simultaneously considering microarray gene expression data and classes of known phenotypic variables such as clinical chemistry evaluations and histopathologic observations. The strategy involves constructing an objective function with the sum of the squared Euclidean distances for numeric microarray and clinical chemistry data and simple matching for histopathology categorical values in order to measure dissimilarity of the samples. Separate weighting terms are used for microarray, clinical chemistry and histopathology measurements to control the influence of each data domain on the clustering of the samples. The dynamic validity index for numeric data was modified with a category utility measure for determining the number of clusters in the data sets. A cluster's prototype, formed from the mean of the values for numeric features and the mode of the categorical values of all the samples in the group, is representative of the phenotype of the cluster members. The approach is shown to work well with a simulated mixed data set and two real data examples containing numeric and categorical data types. One from a heart disease study and another from acetaminophen  exposure in rat liver that causes centrilobular necrosis.We present a more formal approach, the modk-prototypes algorithm partitioned the simulated data into clusters with samples in their respective class group and the heart disease samples into two groups (sick and buff denoting samples having pain type representative of angina and non-angina respectively) with an accuracy of 79%. This is on par with, or better than, the assignment accuracy of the heart disease samples by several well-known and successful clustering algorithms. Following modk-prototypes clustering of the acetaminophen-exposed samples, informative genes from the cluster prototypes were identified that are descriptive of, and phenotypically anchored to, levels of necrosis of the centrilobular region of the rat liver. The biological processes cell growth and/or maintenance, amine metabolism, and stress response were shown to discern between no and moderate levels of acetaminophen-induced centrilobular necrosis. The use of well-known and traditional measurements directly in the clustering provides some guarantee that the resulting clusters will be meaningfully interpretable.The mod Elevated levels of ALT and AST correlate with liver injury . MissingA data set comprised of numeric data with 64 features and 33 objects was simulated from three distinct probability distributions. Normal deviates (mean 0 and standard deviation 20) were drawn at random from the 3 probability distributions generating 11 objects in each. Samples that belong to the same class are #s12–22, 33–43, and 44–54 respectively.p >= 0.6) of visiting or remaining in the no/low toxicity state. An illustration of the HMM, a curve of the log-likelihood from the training, the transition and emission probabilities are in the supplemental materials  and 5 severity levels  of centrilobular necrosis observed in rat livers exposed to acetaminophen. An independent cDNA microarray gene expression data set  acquiredA mixed data set  for the microarray, clinical chemistry and histopathology data domain dissimilarity measures, respectively. The weights for data domain d at the nth step (Wd [n]) are adapted (for controlling how much each data domain contributes to the clustering of the samples) as follows:where τ) is the exponential weighting update factor in the range  and avecorr is the average correlation coefficient  between the samples and the prototypes based on the feature values from domain d.where tau  feature values, the dissimilarity measure between For categorical  times, the modk-prototypes algorithm initialization is seeded by the domain data vector of a randomly selected sample for each of the k clusters. For adaptive clustering, recursion was used to update the prototypes in order to find the configuration of the initial k-prototypes which ultimately results in the reduction of the objective function closest to the global minimum. Matlab code and a stand-alone executable program for the modk-prototypes algorithm to simultaneously cluster gene expression data with clinical chemistry and pathological evaluations are available [For vailable .To determine the number of clusters in a data set, the DVI of Shen et al.  was usedwherek is the number of clusters, N is the number of samples and intra is the average Euclidean distance between samples and the prototype Q of the cluster each sample is assigned to.For mixed data with numeric and categorical values, the DVI was modified to include a CU measure  that defP(Ai = Vij) is the unconditional probability of feature Ai taking on the value Vij, P(Ai = Vij | Ck) is the conditional probability of Ai = Vij given cluster Ck, and k is the cluster number from 1 to m. The DVI modified with CUwhere (8)DVI_CU = (DVI + 1/CU)     k sets for each run of the modk-prototypes clustering algorithm. Validation of cluster assignment was carried out using R', the adjusted Rand index [is minimized over all nd index -44. Whength gene  from the gene expression component of prototype q for ith and jth  cluster  be observedg = (qgi - qgj) and the expected change in expression beA cluster's prototype is formed from the mean of the values for numeric features and the mode of the categorical values of all the samples in the group. Hence, the cluster's prototype is taken as a representation of the feature values that depicts the phenotype of the samples in the group. The process for obtaining phenotypic prototypes is to extract all the histopathologic feature value labels and clinical chemistry measurements as well as significant genes from the prototypes of the clusters that can distinguish between pathological outcomes and best represent the underlying biology of the group of samples. Let the observed difference between the expression ratio of the X2) goodness-of-fit test [Averaging over all the genes gives an estimate of the expected difference between a gene's ratio values in the prototypes of two clusters being compared. Assuming independence and an approximately normal distribution of differences, genes which have expression ratios which significantly distinguish between prototypes of clusters are evaluated using a standard chi-square  where χ2 is the α-level critical value of a χ2-distribution with 1 degree of freedom. An α of 0.05 gives reliable results. Genes from a comparison of two prototypes which significantly distinguish the clusters are annotated for biological function and process(es) using the GO database [where the null hypothesis is that the expression value of the database ,47.k-prototypes, modified k-prototypes; ALT, Alanine aminotransferase; AST, Aspartate aminotransferase; SDH, Sorbitol dehydrogenase; BUN, blood urea nitrogen; TBA, total bile acids; Creat, Creatine; ALP, Alkaline Phosphatase; TP, total protein; ALB, albumin; DVI, dynamic validity index; CU, category utility; R', Adjusted Rand Index; DVI_CU, dynamic validity index with category utility; SOM, self organizing map; HMM, hidden Markov model; GO, Gene Ontology.modThe author(s) declare that they have no competing interests.k-prototypes algorithm for gene expression and phenotypic data, implemented the modk-prototypes clustering algorithm and DVI_CU, applied them to the mixed type data and wrote the paper. RDW provided statistical guidance for the work. GG provided advice and guidance throughout the project. Both GG and RDW assisted in the evaluation of results. All authors read and approved the final manuscript.PRB performed the analysis of the gene expression data, considered the utilization of the DVI_CU measure and the Generation and clustering of simulated mixed data and real data with reduced dimensions of the microarray data. Supplemental_materials.pdf is a pdf file to be viewed with Adobe Acrobat.Click here for fileCluster assignment of the heart disease samples using equal weights. Table_S1.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for fileCluster assignment of the acetaminophen-treated samples using equal weights. Table_S2.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for fileHistopathology observations and clinical chemistry measurements. Table_S3.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for fileSignificant and unique genes that distinguish between levels of centrilobular necrosis of the rat liver. Table_S4.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for fileApproximately 3100 genes determined to be significantly differentially expressed by ANOVA modelling. DEGs.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for fileSimulated mixed data of different types . sim_mixed_data.txt is a tab-delimited text file to be opened and viewed with any standard spreadsheet software.Click here for file