Recent advances in antibody microarray technology have made it possible to measure the expression of hundreds of proteins simultaneously in a competitive dual-colour approach similar to dual-colour gene expression microarrays. Thus, the established normalisation methods for gene expression microarrays, e.g. loess regression, can in principle be applied to protein microarrays. However, the typical assumptions of such normalisation methods might be violated due to a bias in the selection of the proteins to be measured. Due to high costs and limited availability of high quality antibodies, the current arrays usually focus on a high proportion of regulated targets. Housekeeping features could be used to circumvent this problem, but they are typically underrepresented on protein arrays. Therefore, it might be beneficial to select invariant features among the features already represented on available arrays for normalisation by a dedicated selection algorithm.We compare the performance of several normalisation methods that have been established for dual-colour gene expression microarrays. The focus is on an invariant selection algorithm, for which effective improvements are proposed. In a simulation study the performances of the different normalisation methods are compared with respect to their impact on the ability to correctly detect differentially expressed features. Furthermore, we apply the different normalisation methods to a pancreatic cancer data set to assess the impact on the classification power.The simulation study and the data application demonstrate the superior performance of the improved invariant selection algorithms in comparison to other normalisation methods, especially in situations where the assumptions of the usual global loess normalisation are violated. While gene expression microarrays are now a standard tool in biological and medical research, microarray technologies for measuring protein expression are still in development. Antibody microarrays represent a technology that has potential for the screening of hundreds of protein expressions in parallel on large sample sets from minute sample volumes -3. By spDue to the similar experimental setup, scanning and data acquisition infrastructure of cDNA microarrays can be utilised. Thereby, data are generated in a standard format, which facilitates the use of well-researched data handling, processing and statistical analysis tools of cDNA gene expression data, e.g. the open-source and open-development Bioconductor project .For dual-colour cDNA array data the following steps are a vital part of the data pre-processing procedure to prevent technical artefacts from introducing unwanted systematic bias and variation e.g. -9). Thes. Thes9])However, antibody arrays have certain characteristic features which need to be taken into account specifically. First, it is much more difficult to quantify protein expression in a multiplex manner than for gene expression, due to the larger variability in the physico-chemical properties of proteins. Even after careful optimisation and tuning of the entire experimental design, the highly diverse electric charges and hydrophobicities of proteins which occur in complex samples usually lead to higher unspecific background binding than in DNA-microarrays. In addition, protein sizes as well as binding kinetics of the different antigen/antibody pairs vary much more than in DNA hybridisation experiments and the typical concentrations of proteins span a much broader range of magnitudes than for mRNAs. Consequently, it is much harder for protein arrays to design the array in such a way that the fluorescence intensities of all proteins are within the measurement limits of the scanner, increasing the likelihood of satiated data. Therefore, for a data analyst dealing with protein array data it is even more important to incorporate all sources of variation and bias properly in the data processing and modelling. Out of the data processing steps (i) to (iv) listed above, within-array normalisation is arguably the most important step with respect to the potential to help remove common sources of bias and is therefore the focus of this paper.Due to splicing and post-translational modifications, the complexity of the human proteome is expected to range from the order of a hundred thousand to a million different protein molecules . Apart f(A) most probes are not differentially expressed and(B) differential expression is symmetric, i.e. that over-expression and under-expression occur equally frequently2 expression ratios M = log2 (Cy5/Cy3) are plotted against the average log2 intensity values A = 0.5 × (log2 Cy5+log2 Cy3) . After normalisation, the M-values should not depend on the A-values. Loess normalisation uses a robust local scatterplot smoothing method based on locally regressing the M-values on the A values and subsequent replacement by the regression residuals, which will have mean zero independently of the A-values. Due to nonlinear effects of the intensity-dependent bias, loess regression usually performs better than normalisation methods using linear regression or simply applying a constant shift in M-values independent of intensity values  [Both these assumptions might fail for boutique antibody arrays. Additionally, the number of non-regulated housekeeping controls represented on protein arrays will usually be small due to two facts. First, compared to transcriptional levels, there is only limited knowledge available on protein abundances in a variety of different tissues or body fluids in the presence or absence of a certain disease or treatment making it hard to select appropriate controls. Second, in antibody microarrays the costs per probe are about a factor of ten higher than for DNA-probes with unrestricted re-amplification possibilities. Therefore, usually the data analyst faces the problem, that only an inappropriate number of control features is available on the arrays, which could otherwise help to reliably estimate and remove systematic error. In this paper, we will concentrate on the crucial step of within-array data normalisation, which is important for removing systematic variations and biases within an individual array. The most important source of such variation arises from biases associated with the different fluorescent dyes ("dye-bias"). These biases can be dependent on the intensity levels, which are caused by scanning instruments, the label reaction as well as the chemical characteristics of the dyes them-selves. In addition, spatial variation across the array between the two dyes can occur. The intensity-dependent bias is typically addressed within the representation of the data in MA-plots , where tisation) ,13. Howeisation) ,15, anotisation) . Hence, isation) ,18. Prevg is considered to be rank-invariant on an array, if the difference of the ranked Cy5 and Cy3 intensities is less than a threshold d and the average of the ranked intensities is not among the highest or lowest l ranks. For each array j, the set of rank-invariant genes In order to normalise boutique dual-colour microarrays, for which a set of possible reference or house-keeping genes is not available, Tseng et al.  suggestejg) and r(Cy3jg) are the ranks of the intensities and G is the number of spotted genes. To select a more conserved set of rank-invariant genes, Tseng et al. [where r - r (Cy3jg)| is the absolute difference of the ranked intensities of protein g in array j.where ΔThe weighting scheme described in equation (5) was applied to all probes on the array in order to perform a global loess normalisation with weighted probes. Here the term 'global loess' means that all probes are used in the loess fit  which fits a non-linear loess curve where equal weight is assigned to all probes. The second method is the variance stabilising normalisation (VSN) of Huber et al.  that utiIndividual slide effects are corrected for by A-quantile normalisation between arrays. This is done with data sets derived from all within-array normalisation methods except VSN, as the VSN method already incorporates a between-array normalisation procedure. A-quantile normalisation performs quantile-transformation of the A-values, so that the empirical distribution of the A-values is the same across all arrays.To identify proteins that are differentially expressed between groups we used linear models and the empirical Bayes method by Smyth et al. (limma) . The resIn the application on pancreatic cancer data multivariate classification rules were constructed for discriminating between different sample types. Multivariate classifiers were built by applying the nearest shrunken centroid classification method called PAM  , which sTo compare within-arry normalisation procedures for boutique dual-colour antibody microarrays a simulation study was performed. The simulation was based on data generated by self-self incubations of plasma samples on twenty antibody microarrays in a dual-colour mode. The array layout and protocols are described in detail elsewhere ,27. In bN distribution left-truncated at zero. This procedure leaves the mean intensity values  unchanged. After shifting the M-values, the modified dye intensities are Cy3* = 2A (M/2) +and Cy5* = 2A-M/2)  data Figure . This inWe compared different within-array normalisation methods for the normalisation of boutique dual-colour antibody microarrays comprising several hundreds to some thousands of features. The focus was on situations where due to a likely selection bias the usual assumptions for most within-array normalisation methods are violated, i.e. the number of differentially regulated proteins is large and/or the regulation is asymmetrical. In these situations a global loess normalisation method, that is based on all features of the array, will artificially shrink the M-values to zero and thus possibly hide present differential expression. A possible solution is the use of control features which are known to be regulated in a constant manner independently of the experimental settings. However, in current protein arrays such control features are often missing or are underrepresented. In the case of boutique gene expression arrays titration series of a microarray transcript pool (MSP) constructed from transcript libraries have been proposed for building control features for normalisation ,14. For MS conceived the improved invariant selection algorithm and was responsible for the simulation study and bioinformatic analysis of the pancreatic cancer data set. MS, CS, MZ, and AB were involved in the design of the simulation study. CS and JDH designed the antibody microarray, and performed the experiments. MS, CS and MZ prepared the manuscript, which was revised and approved by all authors.R-implementations of the normalisation methods. The file contains R-functions for the different normalisation methods described in the article.Click here for fileR-script to perform the simulation study. The file contains the R-script to perform the simulation study described in the article. The script is organized in three parts. The first part will generate semi-artificial data sets based on the self-self hybridised dual-color microarray data. The second part describes the simulation study and the third part will generate figures of the results, e.g. MA-plots, the ROC-plots and the boxplots of the AUC values.Click here for fileRData file containing the self-self hybridised dual-color microarray data set. The file contains the self-self hybridised dual-color microarray data set used in the simulation study  and to generate boxplots of the bootstrap estimated misclassification errors.Click here for fileRData file containing the pancreatic cancer data set. The file contains the pancreatic cancer data set (see [Additional file Click here for file


There are drawbacks to using this measurement, including unstable ratios when the denominator is small. This paper suggests an alternative estimate based on a proportion that is just as simple to calculate, just as intuitive, with the added benefit of greater numerical stability.In genetic transcription research, gene expression is typically reported in a test sample relative to a reference sample. Laboratory assays that measure gene expression levels, from Q-RT-PCR to microarrays to RNA-Seq experiments, will compare two samples to the same genetic sequence of interest. Standard practice is to use the logAnalysis of two groups of mice measured with 16 cDNA microarrays found similar results between the previously used methods and our proposed methods. In a study of liver and kidney samples measured with RNA-Seq, we found that proportion statistics could detect additional differentially expressed genes usually classified as missing by ratio statistics. Additionally, simulations demonstrated that one of our proposed proportion-based test statistics was robust to deviations from distributional assumptions where all other methods examined were not.2-ratio under most circumstances and better results than the log2-ratio when expression values are close to zero.To measure relative expression between two samples, the proportion estimates that we propose yield equivalent results to the log Regardless of technological platform, laboratory assays of gene expression first extract mRNA from a test sample and a control sample. These samples may be labeled with a tag or dye and hybridized to amplified cloned sequences that represent a gene of interest. The amount of mRNA in each sample is usually measured by examining the amount of dye remaining after hybridization. Researchers use Q-RT-PCR to measure expression when there are only one or a few genes of interest. Several lab protocols from various companies exist to quantify gene expression such as RT-PCR assays using intercalating dyes like SYBR Green, the TaqMan Gene Expression Assays, LightCycler, and QuantiGene -3. When  at once .More recently, sequence-based technologies provide more efficient and accurate expression measurements on a genome-wide scale. Evolving from early techniques such as Serial Analysis of Gene Expression (SAGE) to modern techniques such as Massively Parallel Signature Sequencing (MPSS) and RNA Sequencing (RNA-Seq), these approaches now rival microarray-based gene expression analysis for efficiency, cost, and accuracy . Sequenc2-ratio to measure relative mRNA expression between two samples. The estimate is as follows. Let ij Rrepresent a summary expression value for gene j in the reference sample i where i = 1,..., n and j = 1,..., K. Let ij Grepresent a summary expression value for gene j in the test sample i. The value n is the number of paired samples or experiments and K is the number of genes studied. To summarize relative expression between two samples, the log2-ratio isResearchers commonly use the log2-ratio is commonly interpreted as the average "log-fold-change" in gene expression between the reference sample and the test sample. Its estimate will be denoted by j r= 1, then the ratio between the two samples is 21 = 2, meaning that the expression of gene j in the test sample is two-fold that of the reference sample on average. If j r= 2, then the ratio between the two samples is 22 = 4, meaning that on average the expression in the test sample is four-fold that of the reference sample. Other values of j rare interpreted similarly.or other similar variants on the theme. The log2-ratio is appealing, the statistic has an important drawback. When expression in the reference sample is low, ij Rare small. As ij Rapproaches zero, j rincreases drastically, approaching infinity. When ij R= 0, then j ris undefined. Thus, when reference sample expression is low, we get extreme estimates or missing values for jr. This phenomenon is especially common when measuring gene expression in simple organisms. In bacteria, for example, transcription may be binary; either on or off. The log2-ratio is least reliable for these systems. This problem persists in human genomics research for certain experimental conditions and genes of interest.While the interpretation of the logj pfor each gene. The proportion takes the amount of mRNA in the test sample and compares it to the total amount of expressed mRNA represented by the sum of the test and reference samples. One formula for estimating the proportion isThis article proposes a new estimate to compare mRNA expression in two samples. This estimate is the proportion of mRNA in the test sample ij Rand ijG. For example, when ij R= 0, then ij/G(ij G+ ijR) = 1. We can interpret the number as follows: mRNA expression is observed only in the test sample and not in the reference sample. Similarly, if ij G= 0, then ij/G(ij G+ ijR) = 0 and this means that mRNA expression is observed in the reference sample only. Figure 2-ratio and the proportion estimates, which follows a logistic function. The relationship is roughly linear near the center point but non-linear at the extreme values. A detailed description for the estimate of the proportion, The proportion is well-defined for all values of j pcan be interpreted as the proportion of mRNA from gene j expressed in the test sample. As j pdeviates from 0.5, then there is differential expression between the test and reference samples. As j papproaches one, then gene j is up-regulated in the test sample. As j papproaches zero, then gene j is down-regulated in the test sample. The proportion statistic j pcan also be transformed into a percentage: j p× 100% for reporting. For example, if j p= 0.75 then we can say that 75% of the mRNA expressed in the experiment comes from the test sample. The proportion estimate can easily be used to test for differential expression between groups. Under the null hypothesis of no gene expression, j p= 0.5. The alternative hypothesis is differential expression, j p≠ 0.5. The log2-ratio estimate requires a different hypothesis test. Under the null hypothesis, j r= 0 and under the alternative, j r≠ 0.More generally, j pto describe relative expression for gene j instead of the log2-ratio j rmaintains the ability to interpret differential expression and test for differences. The added benefit of the proportion is the ability to preserve all data points, even for experiments with very low expression values. Typically when values of ij Rare very small, researchers eliminate the th jprobe of the th iexperiment from their analysis. Eliminating missing data results in a loss of information and potential bias and loss of power. The proportion estimate does not require the removal of extreme, but legitimate, data points.Using a proportion jp. The section also provides several test statistics for hypothesis tests of jp. Estimation and testing are developed in the frequentist context but the Bayesian context can also be used, as described in the Appendix. The Results section compares the testing scenarios in simulations and two datasets. The first dataset consists of expression data from a cDNA microarray platform and the second dataset uses RNA-Seq. Both the log2-ratio and proportion statistics achieve roughly equivalent results under usual conditions, but one of the proportion statistics performs better across a variety of distributional assumptions. Proportion statistics also detect differentially expressed genes that would typically be classified as missing data.The Results section provides details that describe the estimation of statistics for We propose a new strategy for the comparison of expression values that is tied to the underpinnings of the hybridization process and its natural interpretation using a binomial distribution. Figure ij Mclones, where j is the gene probe index and i is the sample number, in order to co-hybridize the extracted mRNA sequences from the reference and test samples. Usually ij Mis in the millions, but the exact value will be unknown. For each probe, suppose it hybridizes to a test target with probability j pand to a reference target with probability 1 - jp. This reflects the proportion of available test sequences versus references sequences. We assume that each probe must hybridize to mRNA extracted from either the test or reference sample. Then, the number of hybridizing test target sequences ij Yfollows a binomial distribution with size ij Mand probability jp. We wish to estimate j pto calculate the proportion of hybridized test target sequences. The maximum likelihood estimate for j pis ij Y= ij Gand ij M= ij G+ ij Rwhere ij Rrepresents the expression value for gene j in the reference sample i and ij Grepresents an expression value for gene j in the test sample i when there are i = 1,..., n paired experiments and j = 1,..., K genes. Therefore, to summarize n experiments the estimated proportion for each gene j isFor each gene sequence, suppose that researchers amplify sequences resulting in H0: p = 0.5 versus the alternative hypothesis H1: p ≠ 0.5. The test derived for this binomial distribution has a test statisticTo test for differential expression we set up a decision with the null hypothesis j zis compared to a quantile from the normal distribution z1-α/2. If the type I error is α = 0.05, then z0.975 = 1.96. If j| >|z1.96, then gene j is declared differentially expressed between test and reference samples. The z1-α/2 quantile is replaced by a jp. This test of binomial proportions, however, is not robust to deviations from the binomial distribution. Indeed, we do not believe that expression data will always follow a binomial distribution, but we include this derivation to motivate the choice of this statistic. Instead, we recommend an alternative test statistic that can be used whether distributional assumptions are met or not. The alternative test statistic simply uses a normal approximation to the binomial distribution and calculates a sample variance estimate. Then the test statistic for differential expression isThe test statistic jt| >tα/2,n-11-, then gene j is differentially expressed between test and reference samples. This test is valid for sufficiently large sample sizes  . Since t . Since ij G= 0 and ij R= 0, which made ij Gand ij Rwere never zero for all samples of a specific probe. For this data, none of the 158 unanalyzed probes were in the top eight when using α = 0.05, then When using Therefore, To examine the performance of our methods on RNA-Seq data, we analyzed the expression values reported in Marioni et al (2008) . This daα = 0.05/32000 for all tests. In this way, the threshold can be universally applied to all genes and methods while controlling the genomewide error rate.In order to compare the methods in the original paper to those we are proposing, we used a type I error rate of Table estimate . In factal paper , the mos2-ratio estimates 2-ratio may miss many possibly important differentially expressed genes. When using a LRT as suggested in Marioni et al (2008), 3,979 sequences were omitted from analysis because of missing data. The edgeR and DESeq packages did not generate any missing data. Of the 3,979 missing values generated by the LRT, our proposed methods detected 3,064  where c was the fold-change value, c = 1,2,3,4,5. The four statistics, c and sample sizes n = 3, 5, 10, 15, 20, 25, 30, 40, 50. Additionally we evaluated the ratio statistic n = 50 give simulation results similar to those for sample sizes of 50. In order to compare results, the p-value for an independent t-test was computed, with a null hypothesis of no difference between the two sample means. The null hypothesis was rejected if the p-value was below α = 0.05 and the proportion of rejections out of 1000 simulations was recorded  distribution, to study the effects of changing the scale.The proposed test statistics were evaluated under four different distributions. Though sophisticated simulations can be used to mimic expression data, the simulations below use simple scenarios so as to examine the performance of test statistics under basic distributions and to compare the eight different methods clearly and meaningfully. The first set of simulated intensity values were sampled from an exponential distribution that mimics the values from a 16-bit TIFF image of a cDNA microarray with respect to center and spread. The reference sample was taken from an Exp(1/4000) and the test sample was taken from a perfomed . For simperfomed ,19. The M = 10000, p = 0.5) distribution were obtained. The choice of this distribution was motivated by the derivation behind the maximum likelihood estimate of the proportion M = 100, p = 0.5) distribution, to study the effects of change in the size parameter. A third set of simulated sampled intensity values from a Poisson(λ = 3) distribution were obtained. This distribution is motivated by the derivation behind the likelihood ratio test used in Marioni et al (2008) [A second set of simulated sampled intensity values from a Binomial(l (2008) . The parμ = 5, σ = 1) distribution were obtained. This distribution was included since many analyses assume expression data to be normally distributed. The center was chosen to mimic values from cDNA data with mean 5,000 and standard deviation 1,000, scaled to Normal. Analogous simulations to the exponential above were conducted with respect to statistics, sample sizes, and fold changes. For the normal distribution, fold-changes of 2, 3, 4, and 5 correspond to test samples of Normal. The simulations were repeated for a Normal distribution, to study the impact of changing the parameters.A fourth set of simulated sampled intensity values from a Normal . This se008 . Thij, the likelihood ratio test compares H0 : λKj = λLj versus the alternative hypothesis that expression rates differ H1 : λKj ≠ λLj. The likelihood ratio test isThe methods to test for differential expression from RNA-Seq data in the original paper used a likelihood ratio test (LRT) for inference . Their tj. The maximum likelihood estimate for the alternative hypothesis from the above LRT is denoted by for gene  of 0.1% . More re of 0.1% ,19.β, with densityIn order to compare our proposed methods to previously suggested test statistics in the data analysis sections, we evaluated the proportion statistics within a frequentist testing framework. It is also possible to conceive the model in a Bayesian framework. Given the binomial assumption presented in the Results section, a Bayesian analysis can be conducted. Let the beta distribution be denoted by a) is the gamma function with parameter a. We denote the Bayesian estimator of j pby j pwith parameters a and b, the posterior distribution of β , where Where Γ (. To comprived in  can be uTLB and JW contributed equally to the research. Both authors read and approved the final manuscript.Additional materials. Additional tables for each of the simulation scenarios are provided in the file exprPropSupp2011.pdf. This file was generated using LaTeX.Click here for fileAdditional materials. The script written in R http://www.r-project.org to conduct simulations are provided in the file exprPropSimCode.R.Click here for file


Beta diversity analysis showed a species-based differentiation between GP-PR and M manufactures indicating differences between the preparations. Nevertheless, all the samples shared a naturally-selected core microbiome, that is involved in the curd acidification. Type-level variability within S. thermophilus species was also found and twenty-eight lacS gene sequence types were identified. Although lacS gene did not prove variable enough within S. thermophilus species to be used for quantitative biotype monitoring, the possibility of using non rRNA targets for quantitative biotype identification in food was highlighted.Mozzarella (M), Grana Padano (GP) and Parmigiano Reggiano (PR) are three of the most important traditional Italian cheeses. In the three cheese manufactures the initial fermentation is carried out by adding natural whey cultures (NWCs) according to a back-slopping procedure. In this study, NWCs and the corresponding curds from M, GP and PR manufactures were analyzed by culture-independent pyrosequencing of the amplified V1–V3 regions of the 16S rRNA gene, in order to provide insights into the microbiota involved in the curd acidification. Moreover, culture-independent high-throughput sequencing of  Streptococcus thermophilus is one of the most important bacteria in the dairy industry and it occurs in NWCs used in the manufacture of several traditional cheeses lacS gene showed a high level of heterogeneity in a previous study S. thermophilus.Mozzarella (M), Grana Padano (GP) and Parmigiano Reggiano (PR) are three of the most important traditional Italian cheeses. They are all protected designation of origin (PDO) cheeses and the technology of manufacture, as well as the microbiota involved, have been described in previous works lacS gene amplicons pyrosequencing for S. thermophilus biotype monitoring.In this study, we used culture-independent high-throughput sequencing (HTS) of 16S rRNA gene amplicons to study in depth the microbial diversity of NWCs from three Italian traditional cheeses and its evolution during curd fermentation. Moreover, a new approach was considered, to verify the possibility of using Samples from M cheese manufactures were collected from twelve dairies producing top-quality traditional water buffalo mozzarella PDO cheese, located in the Campania region  in the provinces of Salerno and Caserta. Samples from GP and PR manufactures were collected from six and seven dairies located in different places within the GP and PR area of production . Samples of NWCs and curds at end of the ripening were aseptically collected, cooled at 4°C, and analyzed within 6 h. NWC samples were from the manufacture of the previous day and used for the production of the corresponding curds according to the traditional back-slopping procedure. Curd samples were collected after 5 h from the adding of the NWC for M cheese and after 24 h for GP and PR cheese.All the samples were collected and used with the permission of the dairies.No animals were involved in the present study, but only animal products.Total DNA extraction from the dairy samples was carried out by using a Biostic bacteremia DNA isolation kit . The extraction protocol was applied to the pellet  obtained from 2 ml of NWC or from 2 ml of a homogenized 2-fold dilution of the curd in one-quarter-strength Ringer’s solution .The microbial diversity was studied by pyrosequencing of the amplified V1–V3 region of the 16S rRNA gene amplifying a fragment of 520 bp using primers and PCR conditions previously described S. thermophilus lacS gene sequencing, the 454 Universal Tailed Amplicon protocol was used with a double PCR step . The variable region of 250 bp upstream from the lacS gene was amplified using the primers LCS62f 5′- GGCTTCCAATACTTTAATT and LCS312r 5′- AAGTGAGTTGTCACAAACAT5′-TGTAAAACGACGGCCAGT and M13r 5′-CAGGAAACAGCTATGAC were included at 5′ and 3′ ends of the LCS primers −1 of each deoxynucleoside triphosphate, 2.5 mmol l−1 MgCl2, 5 µl of 10 X PCR buffer and 2.5 U of native Taq polymerase . The following PCR conditions were used: 94°C for 5 min, followed by 20 cycles at 94°C for 1 min, 45°C for 1 min, 72°C for 2 min. A final extension was carried out at 72°C for 7 min. After agarose gel electrophoresis, PCR products were purified with a QIAquick gel extraction kit  and 20 ng of the purified product were used as template in a second PCR where primers M13f and M13r were used, with the addition of 454-adaptors and a 10 bp sample-specific Multiplex Identifier (MID). The PCR mixture was prepared as above described and the PCR conditions were the same, except for the annealing temperature, that was increased to 50°C. PCR products were purified twice by Agencourt AMPure kit  and then quantified using the QuantiFluor™ . An equimolar pool of amplicons was prepared and it was used for pyrosequencing on a GS Junior platform  according to the manufacturer’s instructions by using a Titanium chemistry and a bidirectional sequencing.In order to prepare amplicon libraries for lacS gene reads, the script was carried out twice, in order to demultiplex both forward and reverse reads, after obtaining the reverse complement. In order to guarantee a higher level of accuracy, the reads were excluded from the analysis if they had an average quality score lower than 25, if there were ambiguous base calls, if there were primer mismatches and if they were shorter than 300 and 200 bp, for 16S rRNA and lacS gene reads, respectively. Two different pipelines were used to analyse 16S rRNA and lacS gene reads. For 16S rRNA gene reads, the analysis was carried out as follows: sequences that passed the quality filter were denoised http://www.r-project.org) using gplots package. Representative sequences belonging to clusters identified as Lactobacillus spp. were double-checked using the BLAST (BLASTN) search program (http://www.ncbi.nlm.nih.gov/blast/).Raw reads were first filtered according to the 454 amplicon processing pipeline. Sequences were then analyzed by using QIIME 1.6.0 software Weighted UniFrac distance matrix were used to perform Adonis and Anosim statistical tests through the compare_category.py script of QIIME, in order to verify if there were differences among the three types of cheese. Moreover, the otu_category_significance.py script was run in order to test whether the presence/abundance of any OTUs was significantly associated to a specific cheese.lacS gene reads, the analysis pipeline carried out was the following: forward and reverse-complemented demultiplexed sequences that passed the quality filters were denoised lacS gene sequence types defined by a 100% of similarity were picked using the uclust method lacS sequence of the strain A147 (accession no. M23009) by using MEGA 5.2.2 software lacS sequence types were defined as having at least one point of mutation compared to the reference sequence.For the All the sequencing data were deposited at the Sequence Read Archive of the National Center for Biotechnology Information (SRP033419).A total of 296,385 raw reads were obtained after the 454 processing; 221,903 reads passed the filters applied through QIIME, with an average value of 4,191 reads/sample and an average length of 469 bp. The number of reads obtained for each sample, the number of OTUs, the Good’s estimated sample coverage (ESC), the Chao1 and the Shannon indices are reported in Lb. delbrueckii and Lb. helveticus were the major OTUs in GP and PR samples . S. thermophilus was also always present, but its abundance was never above 24%, with a higher abundance in GP samples. Lb. fermentum occurred only in some samples, but at very low percentage. On the contrary, samples from M manufactures were characterized by abundance of S. thermophilus , while Lb. delbruekii and Lb. helveticus were present at lower extent compared to GP and PR samples  search program (http://www.ncbi.nlm.nih.gov/blast/). Although the identity scores were quite low, they were identified as Lb. acidophilus (87–95%), Lb. johnsonii (90–93%), Lb. gasseri (92–94%), Lb. crispatus (86–95%).After QIIME analysis, 82 OTUs were identified, but only 6 had a relative abundance higher than 1% in at least two samples . Lb. del samples . Lactocoectively . Many saEnterobacteriaceae family or to the LAB group (Lactococcus sp. and Leuconostoc sp.). Acinetobacter johnsonii and Acinetobacter sp. reached 0.5% in some curd samples from all the three different manufactures. Propionibacterium acnes was found only in GP and PR samples (0.02–0.07%), while S. suis was present only in M samples (0.02–0.2%). Bifidobacterium longum occurred only in one PR NWC and in the relative curd.Moreover, sub-dominant populations were also identified and 25 OTUs occurred with an abundance higher than 0.01% in at least 2 samples . M samplS. thermophilus, Lb. delbrueckii, Lb. helveticus, Lb. fermentum, Lactobacillus sp., and some sub-dominant OTUs previously discussed) while some M or GP/PR specific OTUs can be also identified. The same conclusion was drawn from β-diversity analysis (data not shown). The statistical Adonis and Anosim tests run by QIIME showed that the samples significantly differed (P<0.001) according to cheese type. Moreover, the ANOVA and g test run through the otu_category_significance.py script of QIIME showed that S. thermophilus, Lc. lactis and Lb. helveticus abundance was significantly different in the three cheeses (P<0.001). On the contrary, no significant difference was found between M samples from Salerno (MS) and Caserta (MC) area (P>0.05).The OTU network in S. thermophilus abundance was different in the three cheese types. A further analysis was carried out in order to investigate S. thermophilus variability at biotype level, through pyrosequencing of lacS gene amplicons. A total of 226,008 raw reads were obtained from lacS gene pyrosequencing; 195,315 passed the filters applied through QIIME, with an average value of 3,488 reads/sample and an average length of 250 bp. Clustering at 100% of similarity allowed identification of 28 different sequence types, but only 13 of them had a relative abundance higher than 1% in at least one sample. The average percentage of the 6 most abundant lacS types in the three manufactures is reported in lacS types, while GP the lowest. The mutations identified are reported in lacS types is shown in lacS type (lacS type 1) occurred very often, with abundance ranging 87–99% in almost all the samples. A total of 60 mutation points could be detected that allowed the differentiation of 28 lacS gene sequences. Most of the differences were found in the promoter region upstream from the lacS gene. Within the promoter region, −10 regions did not show sequence variability, whereas region −35 turned from TTGACT to TTGACA in 9 out of 28 reference sequences. Twenty-seven points of mutations were found in the protein coding sequence of the gene  of the whey substrate Lb. delbrueckii and Lb. helveticus. Accordingly, Lb. helveticus and Lb. delbruekii were reported as dominant and Lb. fermentum and S. thermophilus as sub-dominant species in NWCs for GP manufactures S. thermophilus was generally present at very low concentration in NWCs for PR, reaching the 12% only in one sample. Moreover, it did not increase in abundance during curd fermentations, suggesting a minor contribution to acidification. On the contrary, GP curd ripening conditions seemed to be more suitable to the development of this microorganism, since a higher amount of S. thermophilus was found in all the curds, compared to the corresponding NWCs. This was more likely related to the curd ripening conditions, rather than to the abundance of this OTU in the NWC. In fact, a higher abundance of S. thermophilus in the NWC, did not lead to a higher amount of this OTU in the relative curd (S. thermophilus was found more frequently in GP compared to PR NWCs S. thermophilus in GP curds could be due to a lower cooking temperature (51–48°C vs 53–54°C). A thermal gradient that starts with a lower temperature can cause a reduction of the heat stress within the molded curd, favoring the presence of S. thermophilusLb. fermentum was more often found in GP than PR. Accordingly, Lb. fermentum was even more abundant in M since no curd cooking is employed in mozzarella cheese production.16S rRNA sequencing revealed a very simple microbial community in all the types of cheese. GP and PR cheeses were characterized by a very similar microbiota and curd fermentation seemed to be driven by ive curd . In agreLactobacillus sp., that was not possible to identify at species level, and that possibly belonged to non-starter LAB (NSLAB) group. NSLAB are often isolated from whey cultures S. thermophilus, that often increased in abundance during curd ripening, together with thermophilic lactobacilli. Lc. lactis and Lb. fermentum were present at lower concentration and not in all the samples. In particular, Lc. lactis was present only in M samples, even if not in all the manufactures , which had the highest variability in lacS types, indicating a higher diversity of S. thermophilus lacS types in this area of production. In fact, lacS type 2 was present only in MC, while the type 3, although present also in some MS at very low concentration, was a dominant lacS type in MC. Sequence types 2 and 3 represented 14 and 15% of the total lacS sequences of MC samples, respectively (lacS types were found both in the NWC and in the relative curd (data not shown). The dominance of lacS type 1  overlapping the −10 box was previously highlighted cre site Click here for additional data file.Table S1Number of sequences analyzed, observed diversity and estimated sample coverage for 16S rRNA amplicons analyzed in this study.(DOCX)Click here for additional data file.


Biomarker discovery holds the promise for advancing personalized medicine as the biomarkers can help match patients to optimal treatment to improve patient outcomes. However, serious concerns have been raised because very few molecular biomarkers or signatures discovered from high dimensional array data can be successfully validated and applied to clinical use. We propose good practice guidelines as well as a novel tool for biomarker discovery and use breast cancer prognosis as a case study to illustrate the proposed approach.We applied the proposed approach to a publicly available breast cancer prognosis dataset and identified small numbers of predictive markers for patient subpopulations stratified by clinical variables. Results from an independent cross-platform validation set show that our model compares favorably to other gene signature and clinical variable based prognostic tools. About half of the discovered candidate markers can individually achieve very good performance, which further demonstrate the high quality of feature selection. These candidate markers perform extremely well for young patient with estrogen receptor-positive, lymph node-negative early stage breast cancers, suggesting a distinct subset of these patients identified by these markers is actually at high risk of recurrence and may benefit from more aggressive treatment than cur-rent practice.The results show that by following good practice guidelines, we can identify highly predictive genes in high dimensional breast cancer array data. These predictive genes have been successfully validated using an independent cross-platform dataset. The goal of biomarker discovery from high dimensional array data is to find an individual or a set of genes (or any other molecular variables) whose expression pattern can predict certain phenotype or clinical outcome. Biomarker discovery holds the promise for advancing personalized medicine as the biomarkers can help match patients to optimal treatment and thus improve patient outcomes. During the past 15 years, numerous biomarkers and gene signatures have been published in the literature. However, few of these biomarkers can be successfully validated and applied in clinical setting, which have caused serious concerns in biomedical research community [The lack of success in biomarker discovery is mainly due to three issues.(1) Many published gene signatures cannot be validated independently. This is mainly due to flawed data analysis. For example, fail to keep completely untouched validation data or improperly use cross-validation technique can all cause overly optimistic results being reported.(2) Some gene signatures do not have additional clinical benefit over known clinical variables even though the gene signatures can be validated. For example, gene signatures discovered from a breast cancer patient population including both ER + and ER- patients for predicting treatment response could probably end up approximating ER status, an important clinical variable for predicting treatment response.(3) Some gene signatures contain large numbers of genes making them difficult to be applied in clinical setting. These gene signatures may include lot of unimportant genes due to the inefficiency of the standard feature selection techniques. For example, standard t test will either miss or rank certain important features below many non predictive features, if these important features do not conform to normality.DLDA) classifier [Array Data Analyzer (ADA). The detailed method is described in [The first two issues have been addressed in ,2. To adassifier . When chribed in .ADA tool has the following advantages.Compared to standard feature selection tools, our (1) Because our approach does not select features based on any single statistic or fixed tradeoff of two statistics, it can adapt to different data and parsimoniously identify informative features from different distributions. Experimental result presented in  shows th(2) By using grid-search to explore different tradeoffs of a pair of statistics, our tool is very flexible to address different feature selection requirements in real world situations. For example, researchers at times are willing to sacrifice performance in order to gain other logistic properties, such as smaller numbers of features and larger fold change. These properties enable more tractable assays for clinical use (e.g. qPCR). By visually examining the performance at different combinations of cut points of the two statistics and checking the size of feature sets, researchers can decide which feature set would be best to use.(3) Our Java based tool is very efficient to run even though it conducts rigorous cross validation and grid search to find optimal feature sets. For example, finding an optimal gene signature from microarray gene expression dataset with 130 samples takes about 50 seconds on a modest laptop PC (Dell Latitude E4300 with 2GB of RAM).  to measure performance of each combination).ADA tool has been applied to various GSK drug discovery projects [AUROCC) across all 11 endpoints among the participating DATs. Using ADA tool, we also achieved the second place in the "The Sage Bionetworks/DREAM breast cancer prognosis challenge 2012" (https://sagebionetworks.jira.com/wiki/display/BCC/Home) by simply identifying 8 genes from the provided training set.The projects . It has projects  by the GAs a case study, we performed detailed prognostic analyses on the van de Vijver breast cancer dataset . An indeADA tool allows users to choose different pairs of statistics to perform the two-way filtering. In this paper, the default pair of non parametric statistics is used, which are the mean difference test and the Mann-Whitney U test. More detailed information of the ADA tool is given in [The given in .van de Vijver dataset and TRANSBIG dataset were downloaded from the public domain. For all datasets generated from Affymetrix platforms, if a gene intensity value is smaller than 40, we floor it to 40. The log transformed ex-pression values are used for analysis.For the discovered genes that can be successfully mapped from Agilent Hu25K platform to Affymetrix HG-U133A, we arbitrarily decided to keep top 10 genes from each direction  to validate. If a gene can be mapped to multiple probe sets of the Affymetrix HG-U133A chip, we use the probe set that has the largest interquartile range (IQR). To avoid bias, the scoring of validation samples using the 20 gene signature was done without any scaling or tuning. We simply calculated the scores by summing up the log expression values of genes of one direction and subtracting those of genes of the other direction. To classify the samples into "good prognosis" and "poor prognosis", we use zero as the threshold.As a case study, we use the van de Vijver data set  to discoThe van de Vijver data set contains samples from 295 patients with stage I and II breast cancer. The gene expression data was generated using Agilent Hu25K platform. A subset of the samples was used to develop a 70-gene signature for predicting breast cancer early relapse [ADA tool, we were able to generate gene lists of various sizes for different subgroups of patients. After studying the gene lists, we found that many of the predictive markers were indeed more sig-nificant in certain subgroups. For example, the ER+/lymph node+ group harbors unique genes that are predictive of relapse. The prognosis of this group of patients was also more predictable (based on nested CV results) compared to that of the other groups. Some of the discovered genes are listed in Table By using the To validate the candidate biomarkers discovered from van de Vijver data set, we download the TRANSBIG data set , which is based on the Affymetrix HG-U133A platform instead of the Agilent Hu25K platform. An original objective of TRANSBIG data set was to validate the 70-gene signature and it is similar to the van de Vijver data set in term of patient age distribution - i.e., both data sets contains young breast cancer patients. Because all patients of the TRANSBIG data set are lymph node negative (Node-) patients, we evaluate the candi-date biomarkers discovered from the Node- patients.AUROCC based on commonly used endpoints: time to distant metastasis(TDM) at 5 years and 10 years. Because 70% of Node- patients in the training data belong to the subgroup Node-/ER+, in addition to measure the performance on the whole validation sets (consisting of Node- patients), we also measured the performance on the Node-/ER+ subset of patients. Both results are shown in Table We picked the top 20 genes from our Node- gene list that can be mapped from Agilent Hu25K to Af-fymetrix HG-U133A. The performance of the 20 gene-signature as well as each of the individual gene was measured using From the result we can see that our 20-gene signature performs better than other gene signature or clinical variable based prognostic factors. Most of the 20 candidate genes can also be successfully validated. About half of the 20 genes can individually achieve similar or better performance compared to clinical variable based risk scores, which further shows that the proposed approach can parsimoniously select high quality features.To further illustrate the added value of these markers over the traditional clinical criteria, we use survival curves to compare the 20-gene signature to the Adjvant! 10 year overall survival probability, which is based on known clinical markers. Figure AUROCC 0.83; and the genes CCNB2, BUB1, CENPA, BIRC5 and CYP4B1 can each achieve AUROCC above 0.78.It is also worth noting that the 20-gene signature and the top performing genes based on the whole vali-dation set have even better performance when validated using Node-/ER+ patients only. For example, for end point TDM at 5 years, the 20-gene signature can achieve ADA tool was able to effectively discover both biologically relevant and class predictive genes. The tool has two important features: (a) because the model performance estimation is based on rigorous nested cross-validation, the tool does not return any feature when the dataset does not produce a strong signal. (b) It has a unique iterative procedure that attempts to find all important genes, which can be used for further analyses, such as pathway and ontology analysis.Using the breast cancer prognosis experiment, the DLDA is preferred. We believe the best way to gain better performance is through improving performance of feature selection, rather than tuning modeling parameters of complex models. Complicated learning schemes can make proper cross-validation too computational expensive to run. Without proper cross-validation, the result can be overly optimistic. Important guidelines for biomarker discovery are presented in Richard Simon's work [When doing biomarker discovery using the van de Vijver datasets, our approach relies heavily on robust cross validation to control over fitting rather than leaving out an artificial validation set. This is a more efficient way to use the relatively small number of samples - artificially defining a small validation set is unlikely to achieve much due to the large test data variability and reduced power of model development in the smaller training set. However, it is crucial that the cross-validation is done properly. For example, the feature selection must be performed within each run of cross validation; and the nested cross validation is often required when evaluating model performance . As we can see here, the proper cross-validation procedures can often be quite computationally expensive. This is one of the reasons that a simple modeling technique such as n's work .As shown in Table To validate our findings from the van de Vijver data, we used the cross-platform validation dataset TRANSBIG. In addition to evaluate our 20-gene signature, we also evaluate individual genes. By doing so, we can carry forward those validated genes for further clinical validations. Table Table Based on our past experience, we are confident that biomarker discovery can play important role in ad-vancing personalized medicine if the study design and data analysis is done properly. We would like to propose the following good practice guidelines for biomarker discovery. Most of the points have already been made in ,2.(1) Make sure no information leak from validation set. If a separate validation set is available, one need to make sure that the validation set is not used in any way in feature filtering or model building. The same rule also applies to cross validation.(2) Model performance evaluation and model parameter tuning cannot be done using the same cross va-lidation loop. In such case, nested cross validation is needed .(3) When sample size is not very large, cross validation is a preferred validation technique.(4) Consider patient stratification using known clinical variables.(5) Use simple modeling techniques. Simple models performs well  and are (6) When choosing feature filtering or feature selection methods, do not automatically assume that all va-riables are normally distributed. Many highly predictive biomarkers are far from normally distributed, especially in cancer research ,15.AUROCC: area under the receiver operating characteristics curve; CV: cross-validation; DAT: data analysis team; DLDA: diagonal linear discriminant analysis; ER: estrogen receptor; MAQC-II: MicroArray Quality Control phase II on predictive modeling.The authors declare that they have no competing interests.JC designed and implemented the proposed feature selection algorithm and performed the experiments. All authors participated in discussing and drafting the manuscripts. All authors participated to and approved the final manuscript's preparation.


Several DNA microarray based expression signatures for the different clinically relevant thyroid tumor entities have been described over the past few years. However, reproducibility of these signatures is generally low, mainly due to study biases, small sample sizes and the highly multivariate nature of microarrays. While there are new technologies available for a more accurate high throughput expression analysis, we show that there is still a lot of information to be gained from data deposited in public microarray databases. In this study we were aiming (1) to identify potential markers for papillary thyroid carcinomas through meta analysis of public microarray data and (2) to confirm these markers in an independent dataset using an independent technology.We adopted a meta analysis approach for four publicly available microarray datasets on papillary thyroid carcinoma (PTC) nodules versus nodular goitre (NG) from N2-frozen tissue. The methodology included merging of datasets, bias removal using distance weighted discrimination (DWD), feature selection/inference statistics, classification/crossvalidation and gene set enrichment analysis (GSEA). External Validation was performed on an independent dataset using an independent technology, quantitative RT-PCR (RT-qPCR) in our laboratory.-10) in PTC and identification of papillary carcinoma was possible with 93% accuracy . We also show that the extracellular matrix pathway is strongly activated in the meta analysis data, suggesting an important role of tumor-stroma interaction in the carcinogenesis of papillary thyroid carcinoma.From meta analysis we identified one gene (SERPINA1) which identifies papillary thyroid carcinoma against benign nodules with 99% accuracy . In the independent validation data, which included not only PTC and NG, but all major histological thyroid entities plus a few variants, SERPINA1 was again markedly up regulated  of 181 samples. Thyroid nodules are endemic in iodine deficient areas, like Europes alpine regions, where they have a prevalence of 10-20%. They are classified by their histology into five main classes: the benign types Nodular Goiter (NG) and Follicular Thyroid Adenoma (FTA), and the malignant entities Follicular Thyroid Carcinoma (FTC), Papillary Thyroid Carcinoma (PTC), Medullary Thyroid Carcinoma (MTC) and Anaplastic Thyroid Carcinoma (ATC). Only approximately 5% - 10% of thyroid nodules are malignant ,4. This In other types of cancer it has been shown that gene expression profiling can add substantial value to the discrimination of the different clinically relevant tumor-entities ,6. To daHere we focus on Papillary Thyroid Carcinoma (PTC), which is the most common and therefore most extensively studied form of thyroid cancer. We hypothesised, that feature selection based on a larger sample cohort will be more robust than using single studies, so we decided for a meta analysis approach which allows us to analyse all publicly avail-able datasets (99 samples) for PTC with one common analysis approach. From this dataset we were able to identify SERPINA1 as a single gene which allows to discriminate between PTC and benign nodules or healthy tissue with 99% accuracy (one misclassification). To validate these findings and to quantify the discriminatory power of SERPINA1 for the detection of PTC, we performed RT-qPCR experiments on an independent set of thyroid nodules (instantaneous sections) measuring the mRNA levels of SERPINA1. In contrast to the meta analysis data, we included not only PTC and benign nodules but also all other histological entities of thyroid nodules plus rare histological variants . Despite the marked overrepresentation of these difficult to diagnose histological variants, the classification accuracy was still as high as 93%.Encouraged by the high classification accuracy in the independent validation data, showing the validity of our meta analysis approach, we went on to perform Gene Set Enrichment Analysis (GSEA) on the meta-analysis microarray data to elucidate some of the specific mechanisms involved in papillary thyroid tumorigenesis.Data Integration was performed using Distance Weighted Discrimination (DWD) . Figure He dataset, none of these classifiers can be applied to any of the other datasets. Classification results are rarely ever higher than expected by chance. If, however, one uses the DWD-integrated data, the classifiers already fit much better  and a pamr classification and leave-one-out cross-validation (loocv) was performed. The results of the cross-validations are near perfect with single samples classifying wrongly. However, with the exception of the classifier from the Then a pamr - classifier for the distinction of PTC versus various types of benign thyroid tissue was built for the complete DWD-integrated dataset and validated in a leave-one-out crossvalidation. This identified a one gene classifier, which classifies 99% of samples correctly in leave-one-out-crossvalidation (loocv). The discriminative gene is SERPINA1. If one removes the SERPINA1-probe from the analysis, pamr again finds a classifier with 99% accuracy in loocv, this time using a 9-gene signature. Removing these 9 genes yields another 9-gene classifier with a similar performance (99% accuracy), and further an 11-gene classifier with 99% accuracy. In all of these cases, the same sample is misclassified. The expression profile of these genes is visualised in the Figure <2 * 10-16) and 36-fold (p = 1.3 * 10-10) in the qPCR data . In the meta analysis data SERPINA1 expression accurately distinguishes PTC from benign nodules for 99% of the cases. Note that this is the DWD-integrated data. In the independent validation RT-qPCR data, where we picked samples from all histological entities, the papillary carcinomas again show a distinctly elevated expression of SERPINA1 compared to all other histological groups under investigation. A threshold value of 1.08  was used as decision boundary for the decision PTC vs nonPTC. At this threshold, PTC can be detected with a sensitivity of 1 and a specificity of 0.905 (93% prediction accuracy). False Positive were 2 MTCs, 1 ATC, 2 FTCs (1 recurrent) and 1 FTA (oxyphilic). The meta analysis data classified one PTC falsely as benign, all others were correct.Next, we used RT-qPCR to test the discriminative power of SERPINA1 on an independent dataset generated in our own laboratories. Figure Encouraged by the high reproducibility of the meta analysis results in the external validation data, we went on to perform inference statistics on the meta analysis data. Table The present microarray meta analysis makes use of the latest methods for microarray data integration and classification. Nevertheless, meta-analysis of microarray data still poses a challenge, mainly because researchers ask at least partly different questions and hence use different experimental designs. Moreover, the number of thyroid tumor microarray data available to researchers to date is still comparably low . Therefore, when doing meta analysis one is forced to use all data available, even if the patient cohorts represent a rather heterogeneous and potentially biased population. More specifically, it is difficult to obtain a homogenous collection of control material . These are usually taken from patients who were operated for other thyroid disease which is in turn very likely to cause a change in gene expression as measured on microarrays. In other studies, healthy tissue surrounding the nodule was takes as control material. The generation of homogeneous patient cohorts is further hampered by limited availability of patient data like age, gender, genetic background, etc.Nevertheless we believe to have shown that there still is a great deal of information to be discovered in the wealth of DNA microarray data deposited in databases like ArrayExpress and GEO ,12. As sThis is also supported by another meta analysis on papillary thyroid carcinoma data which has been conducted by Fujarewicz et al. . They usWhen doing meta analysis of microarray data, researchers have also based their approach on modeling of inter-study variation  or on coMost of these lists were generated from microarray analysis. However, even when comparing the genes in the classifiers to gene lists generated with independent technologies, like cDNA library generation as performed by Kaserer et al. , there iSERPINA1 has been reported before as a potential diagnostic marker for papillary thyroid carcinoma in a study investigating its protein expression in thyroid biopsy tissue by Immunocytochemistry and Western Blot . In lineIn GSEA, the most apparent feature of PTC is an upregulation of extracellular activities, like cell communication and adhesion and extracellular matrix receptor interactions. While the gene lists and the derived classification rules of the 4 studies under investigation here show little over-lap, we found the same pathways to be overrepresented when each of the single studies is being analysed separately. Upregulation of ECM microenvironment genes have been described as an effect of BRAF mutations through phospho-ERK1/2 signalling, which in turn may be involved in the up-regulation of some ECM remodeling genes like TSP-1 . While EThe power of the meta analysis approach adopted here is demonstrated by a 99% loocvaccuracy  for the distinction between papillary thyroid carcinoma and benign nodules. One sample was classified wrongly, and although it is not possible to correctly map the samples from this analysis to the original analysis [In the qPCR data we achieved a prediction accuracy of 93%. We chose the threshold to allow for false positives rather than false negatives. For this threshold 6 samples were false positives and none false negative. Three follicular nodules were false positive, one of them was a recurrent FTC nodule and one was an oxyphilic FTA nodule. The third was a classical FTC. Two MTC and one ATC nodules were falsely positive. The latter two entities are rather easy to distinguish by alternative methods like cytology or serum calcitonin testing (MTC). However, the raw, unnormalised data (not shown) suggests, that the slightly elevated levels in the MTC and ATC group may not be down to elevated SER-PINA1 levels, but to a different cellular turnover in these cells which leads to a problem in the normalisation procedure. A more thorough search for appropriate normalisation genes may circumvent this problem.Gerhard et al.  studied These results are very encouraging, however, more studies need to be carried out in order to be able to transfer these findings into clinical applications for routine diagnostics. Firstly, markers with sufficient sensitivity and specificity for the other histological subtypes need to be found. There are few studies reporting molecular classifiers for follicular thyroid disease ,26, and Datasets were downloaded either from websites or from public repositories  identifiers. This, however, disregards all possible splice variations in the genes under investigation. For example, if a gene had 2 splice variants, one of which was Differentially expressed in the experiment and the other not and if one platform would contain an oligo specific only to the Differentially expressed variant and the other platform only an oligo to the other variant, then a matching based on UniGene would merge probes which measure different things.The first step in any meta analysis of microarray data is to find the set of genes which is shared by all microarray platforms used in the analysis. Traditionally, overlap is assessed by finding common Entrez-Gene or UniGene . Probes which annotate to the same set of RefSeq identifiers were deemed to be similar. The median value was used, if one set of RefSeqs was represented by multiple probes on the array. 5707 different sets of RefSeqs were present on all arrays.To overcome this problem, the approach adopted here merges only probes which annotate to the same set of RefSeq identifiers. To this end all matching RefSeqs were downloaded for each probe(set) via the Bioconductor annotation packages , FTA (n = 18), FTC (n = 13), MTC (n = 6), normal thyroid (n = 5), PTC (n = 19) and NG (n = 18). The choice of nodules also included a high rate of histological variants, like follicular variant PTC, tall cell PTC and oxyphilic follicular nodules. Approximately 30 mg of tissue was lysed in RLT-buffer on a FastPrep FP120 instrument (Qbiogene p/n 6001-120) and extracted using Qiagen All-Prep DNA/RNA extraction (Qiagen p/n 80204) according to manufacturers instructions. RNA quality was assessed on the Agilent 2100 Bioanalyser (p/n G2938C). μl of the ABI 2× Gene-Expression-Mastermix, 20  μl cDNA and 42,5  μl RNase free water for the PCR mastermix. Aliquots of 15  μl (for 4 assays in triplicates) were added to a 1  μl+4  μl dilution of each assay (5  μl diluted assay plus 15  μl cDNA Mastermix). Cycling conditions were 2 sec 50°C, 10 sec 95°C and then 15 sec 95°C, 1 sec 60°C for 50 cycles (ABI 5700 Sequence Detection System). Using the method of Vandesompele et.al [We had chosen following ABI TaqMan Assays: ACTB (Hs99999903_m1), CASC3 (Hs00201226_m1), DAD1 (Hs00154671_m1), PPIA (Hs99999904_m1) and SerpinA1 (Hs01097800_m1). Oligo-dT primed reverse transcription was done using Superscript III (Invitrogen), following the suppliers instructions. RT reaction with 180 ng RNA was used for three controls  and the SerpinA1 gene in triplicates for quantitative PCR (12 reactions). We used 125 le et.al , DAD1 wa®, Natick, MA) was done in R/bioconductor http://www.bioconductor.org[http://www.methcancerdb.net/methcancerdb/img/ThyroidMetaAnalysis.zip.First, inference statistics was calculated using the bioconductor package limma . This inuctor.org. R-codesThe authors declare that they have no competing interests.KV, CN, KK and FL carried out the conception and design of the study, statistical analysis was carried out by KV under supervision of FL, conception and completion of molecular analysis was done by MM, collection and assessment of clinical material and data was done by KK and OK. KV prepared the manuscript including critical intellectual input and revisions by all authors. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1755-8794/4/30/prepub


High-throughtput technologies enable the testing of tens of thousands of measurements simultaneously. Identification of genes that are differentially expressed or associated with clinical outcomes invokes the multiple testing problem. False Discovery Rate (FDR) control is a statistical method used to correct for multiple comparisons for independent or weakly dependent test statistics. Although FDR control is frequently applied to microarray data analysis, gene expression is usually correlated, which might lead to inaccurate estimates. In this paper, we evaluate the accuracy of FDR estimation.Using two real data sets, we resampled subgroups of patients and recalculated statistics of interest to illustrate the imprecision of FDR estimation. Next, we generated many simulated data sets with block correlation structures and realistic noise parameters, using the Ultimate Microarray Prediction, Inference, and Reality Engine (UMPIRE) R package. We estimated FDR using a beta-uniform mixture (BUM) model, and examined the variation in FDR estimation.The three major sources of variation in FDR estimation are the sample size, correlations among genes, and the true proportion of differentially expressed genes (DEGs). The sample size and proportion of DEGs affect both magnitude and precision of FDR estimation, while the correlation structure mainly affects the variation of the estimated parameters.We have decomposed various factors that affect FDR estimation, and illustrated the direction and extent of the impact. We found that the proportion of DEGs has a significant impact on FDR; this factor might have been overlooked in previous studies and deserves more thought when controlling FDR. With the advent of high throughput technologies, research has focused on the systematic genome-wide study of biological systems. Microarray technology has been used to measure the mRNA expression of thousands of genes simultaneously. Concurrently, new statistical methods have been developed to analyze the data generated by these experiments. These methods involve both data preprocessing  and specific tools for different types of studies .p-value associated with each test; and they estimate the FDR using the distribution of p-values. The methods are usually based on the assumption of independent or weakly dependent test statistics. However, when dealing with microarray data, we know that genes are usually correlated either for biological or technical reasons. Recent studies have demonstrated the non-negligible effects of correlation in microarray data on large-scale simultaneous hypothesis testing and pointed out how variable the FDR could be in the presence of strong correlation [The canonical class comparison problem involves the identification of lists of DEGs. The evolving consensus  on the arelation -13.p-values. However, those p-values are not independent. Since gene expression is often correlated, the effective number of independent measurements used to estimate the distribution may be quite small. Strategies that have been proposed to improve the estimation of FDR include resampling [Our study sheds more light on possible reasons for the (lack of) precision in the estimated FDR. Our results provide two concrete examples of this imprecision. First, we look at an example where univariate Cox proportional hazards (CPH) models are used to determine which genes appear to be related to survival. By resampling ~ 100 patients at a time (out of a set of ~ 200 patients), we find that the estimates of the percentage of genes that appear to be related to survival range from 0% to 20%. In a simpler example of univariate t-tests between two groups of samples, we find that the estimate of the percentage of DEGs ranges between 13% and 43%. This range of estimates is much wider than one would anticipate from fitting a distribution based on thousands of sampling ,14,15 ansampling . Howeverp-values [p-value distributions that we encounter suggests that our results are driven by intrinsic variability in the p-value distributions from one sample set to another, and thus are likely to affect all known methods to estimate FDR.Throughout this paper, we estimate FDR using a BUM model for the distribution of p-values . We seleIn many cases, it is difficult to evaluate analytical methods for microarray data because of the complex— and unknown—nature of the underlying biological phenomena. Thus, simulated data sets with known “ground truth” are needed in order to assess the performance of computational algorithms for the analysis of high throughput data. To address this problem, many groups have developed microarray simulation software ,16-20. Hp-value distributions and FDR estimates found in real data.We have already introduced a package of microarray simulation software called UMPIRE . The curN = 95) and testing (N = 96) sets. We downloaded the processed microarray data and associated clinical annotations from the supporting web site (http://llmpp.nih.gov/FL/).The Affymetrix microrray data were collected as part of a study to predict survival in follicular lymphoma patients . The dathttp://cmgm.stanford.edu/pbrown/). We used intensity-dependent loess normalization to normalize the background-corrected channels from each microarray, after which the log ratios between experimental and reference channels were transformed by computing the base-two logarithm.The two-color fluorescent cDNA microarray data were collected as part of a study to identify clinically relevant subtypes of prostate cancer . The datGenes could be correlated when they are involved in active biological pathways, or are regulated by the same set of factors. We consider the correlation in gene expression to be “clumpy”, meaning that there are gene groups with high correlation within groups but no or little correlation between groups. In order to mimic this correlation feature, we applied block structure. Both the block sizes and the correlations within a block vary in order to mimic different sized pathways/networks, and loosely or strongly correlated genes within a particular pathway/network. Distinct blocks are assumed to be independent. Please refer to our previous publication for detailed description of the block structure .G genes and N samples. We allowed N to vary in order to study the effect of the number of independent observations on various test statistics. The same number of cancer samples were generated with a portion of DEGs, where differential expressions were simulated as changed mean expression. Rather than focusing on individual genes, we altered the means of blocks of genes in order to mimic the effect of cancer pathology on pathways or networks. We let ѱ denote the percentage of differentially expressed blocks, which were randomly selected from the transcriptionally active blocks. Although it is possible for an inactive block of genes in normal samples to be turned on in cancer samples, or vice versa, we kept transcriptionally inactive blocks inactive in both normal and cancer samples in this simulation. The absolute changes of the mean expression values on log scale for a block of genes were given by Δg ~ Gamma. Both parameters for this gamma distribution were set to 10 so that the absolute fold change on the log2 scale was 1, and the long tail on the right hand side of the distribution allowed a few genes to have large fold changes. A gene in the changed block was randomly assigned to be up-regulated or down-regulated in cancer samples.Using the UMPIRE package, additive and multiplicative noise were incorporated, and correlated blocks were implemented. We simulated normal samples as a homogeneous population with N = 95) and testing (N = 96) using a method that guaranteed that the survival curves for the two halves of the data would be the same. Their method for developing a model to predict survival began by computing p-values for each gene, derived from univariate CPH models in the training set. In order to better understand their method, we began our own analysis by fitting CPH models on both the training and testing sets. Figure p-values. The distribution of p-values in the training set is uniform, which suggests that there is no evidence that any individual gene is associated with survival. However, the histogram of p-values in the test set shows clear evidence of an enrichment of small p-values, and is well-fit by a BUM model [p-values from two-sample t-tests contrasting gene expression between training and testing sets was also uniform, suggesting that no genes were differentially expressed. We found this collection of results surprising: although the survival curves and the expression of individual genes were the same in training and testing sets, there appeared to be a difference in the ability of individual genes to predict survival. To resolve this conundrum, we hypothesized that the two distributions of p-values must actually be more variable than the histograms suggest. We repeatedly split the data into half, computed gene-by-gene CPH models on each half, and fit BUM models to the p-values. We found that the estimate of the number of significant genes range between 0% and 20%, with a peak near 10%.Dave and colleagues  collecteUM model  that sugp-value distributions across experiments, we turned to a simpler example. Lapointe and colleagues [p-values, and fit a BUM model to the distribution of p-values. We found that the appropriate p-value cutoff to achieve an FDR of 10% varied between 1.6 × 10–7 and 0.0434 when using ten samples per group, with a median of 0.00628. The cutoff ranged from 0.0089 to 0.0439, with a median of 0.0229, when using twenty samples per group. If we tried to use the median cutoff across all random resamplings from the full data set, we also found that the effective FDR varied widely from one data set to another , eight different mean block sizes , and four different θ scenarios, where θ denote the portion of negatively correlated genes within a block. In addition, we tested five levels of differential expression  with three different mean block sizes  and 25 samples in each biological condition. Our first goal was to study how sample size, block size, ѱ, and θ affect FDR. Based on the results, different θs do not have a pronounced impact on various parameters of interest (data not shown). Thus in the following sections, we will only show the results obtained from θ that is uniformly distributed between 0 and 0.5 ). We performed gene-by-gene two-sample t-tests in order to identify DEGs between normal and cancer samples. Then we modeled the p-values using a BUM model. Several parameters of interest were recorded for each data set.We simulated 128 sets of normal and cancer data, using four different sample sizes , but the effect is not dramatic. Figure ѱ = 10%. For a fixed sample size, the variance of Both the mean and the variance of ѱ = 10%. Like ѱ (proportion of DEGs) equal to 0.1 in the simulations summarized in Figure ѱ, of non-DEGs. Since ѱ results in smaller Figure π using the formula:From p-values arises from the alternative hypothesis; that is, π = 1 – ѱ = 0.9 since ѱ = 10%. The distribution of the calculated upper bound of ѱ = 10% has a pattern similar to the one shown in Figure The remaining  studies .We also studied the correlation between p-value thresholds, we counted the number of significant DEGs that were called “positive”. Because we know which genes were truly differentially expressed in the simulation, we calculated the number of true positives (TP) and false positives (FP) for each p-value threshold. Next, we calculated the observed false discovery proportion: FDP = FP/(FP + TP). We simulated cancer samples with a proportion of DEGs having altered mean expression values. For given  ѱ (portion of DEGs). Figure ѱ. We observe clearly that larger ѱ is associated with smaller and better estimated FDP (less variable). This is not surprising because more true positives render more power during estimation. Another observation from Figure One factor that dramatically affects FDP is ѱ (10%). Since we know that the mean of FDP is not significantly affected by block size, we calculated mean FDP by averaging over all block sizes for each sample size. Additional file To illustrate the influence of sample size on FDP estimation, we performed another set of simulations with different sample sizes and block sizes, but fixed A. The central peak of the null distribution widens when A > 0, and narrows when A < 0. We recorded A for each simulation; Additional file A grouped by sample size and block size when ѱ = 10%. The mean value of A is essentially unaffected by either block size or sample size, but the variance of A increases with larger block size. This observation is consistent with Efron’s finding that more correlation leads to larger variance.Efron  showed tA is dramatically affected by ѱ , the distribution of the test statistic widens, so A is larger. The values of Â are almost always positive in this set of simulations, which is supposed to mean that gene correlations usually widen the distribution. However, Figure One novel finding from our study is that the dispersion variate ѱ Figure . With laA, which is the opposite of the conclusion in Efron’s paper. Figure A vs FDP for varying values of ѱ. For fixed ѱ, FDP and A exhibit weak positive correlation. However, when ѱ is allowed to vary, FDP and A are negatively correlated.We also found that FDP is negatively correlated with the dispersion variate corr.std). We found that corr.std increases with larger block size (Additional file corr.std decreasing significantly with larger sample size.Also following Efron, we calculated the standard deviation of the empirical correlation densities (From the two concrete examples, we observed a lack of precision in the estimation of FDR. In order to study the sources of variation during FDR estimation, we simulated microarray data with more realistic parameters. In our simulation, block-wise structure with different block sizes and intra-block correlation are used to mimic the molecular networks or biological functional groups where large scale correlation of gene expression arises. A particular block of genes could be transcriptionally active or inactive depending on specific biological conditions. When the block of genes are transcriptionally active, their expression levels follow a multi-variate normal distribution with parameters estimated from real microarray data. Certain portion of genes will be differentially expressed between normal and cancer samples, where the magnitude of changes follows a gamma distribution which allows some large magnitude changes while the majority have two fold change on average. With this setting, we simulated microarray data sets with different sample sizes, correlation structure, portion of negtively correlated genes within a block, and portion of DEGs between two biological conditions.For each simulated data set, we have recorded the parameters related to FDR estimation. Different portions of negtively correlated genes within a block do not affect the parameter estimations. Thus, the three major sources of variation in FDR estimation are the sample size, correlation structures and the portion of DEGs. With large sample size, the variances of all parameters decrease due to increased estimation power. However, the percentage of non-DEGs is always under-estimated, even though it approaches the true portion with larger sample size. Large block size results in less precise estimation of all the parameters due to less independent measurements. However, the block size does not affect the mean estimation of the parameters. Thus the FDR estimation are less precise with more correlation, but the average FDR estimation is not affected.Our study suggests that an important factor affecting FDR estimation is the portion of DEGs. With larger portion of DEGs, the distribution of test statistic is widened by the larger portion of true positives, thus resulting in smaller FDR and more precise FDR estimation.In summary, the correlation structure is not the only factor affecting FDR estimations. The portion of DEGs, which varies under different biological conditions contributes to both the precision and the magnitude of FDR estimation.FDR: False Discovery Rate; FDP: False Discovery Proportion; UMPIRE: Ultimate Microarray Prediction, Inference, and Reality Engine; BUM: beta-uniform mixture; DEGs: differentially expressed genes; CPH: Cox proportional hazardsThe authors have no competing interests.JZ performed the simulations, gathered the simulation results, and completed the first draft of the manuscript. KRC initiated the project, provided ideas, developed UMPIRE, supervised the progression, and was involved in the manuscript development.Boxplot of pearson correlation between Click here for fileFDP for different sample sizes and block sizes Solid black lines represent the mean FDPs from all simulated data for the same sample size. Dashed lines represent standard deviations of FDPs for different block sizes that are distinguished by colors shown in legend of bottom-right figure.Click here for fileÂ grouped by sample size and block sizeBoxplot of Click here for fileBoxplot of Click here for file


Accurate tumor classification is crucial to the proper treatment of cancer. To now, sparse representation (SR) has shown its great performance for tumor classification. This paper conceives a new SR-based method for tumor classification by using gene expression data. In the proposed method, we firstly use latent low-rank representation for extracting salient features and removing noise from the original samples data. Then we use sparse representation classifier (SRC) to build tumor classification model. The experimental results on several real-world data sets show that our method is more efficient and more effective than the previous classification methods including SVM, SRC, and LASSO. Tumor is a solid lesion caused by the abnormal growth of cells. A timely accurate treatment is very important clinically. The premise of an accurate treatment is an exact diagnosis due to the heterogeneity of cancer. That is, we need to classify them accurately before treating tumors. Current methods for classifying cancer malignancies mostly rely on a variety of morphological, clinical, or molecular variables. Despite recent progresses, there are still many uncertainties in diagnosis. The advent of DNA microarray and RNA_seq  makes itMany methods for molecular data classification or clustering based on gene expression data have appeared in this area –14. Huanl1-regularized least square optimization is used to calculate an SR coefficient vector with only a few significant coefficients. In theory, a test sample can be well represented by only using the training samples from the same class. However, there is too much noise in gene expression data, which causes that the discriminative features are not obvious and the test samples can also be represented by some training samples from different classes. This will decrease the classification accuracy. To reduce noise , the whole data set can be reexpressed as W = . Suppose that a new testing sample y ∈ Rm belongs to ith class. Based on the theory of sparse representation, y would lie in the linear span of the training samples Wi; that is,αi,j ∈ R is a scalar and j = 1,2,…, ni.Let x0 ∈ Rn,  y can be also rewritten asx0, only the entries corresponding to the same class as y are nonzero.Supposing a linear representation coefficient vector y according to x0. So the key problem is how to calculate x0 in 4) has not method .X =  ∈ RD×n, where each column vector xi is a sample, and a dictionary A =  ∈ RD×m, where ai is also a sample. X can be linearly represented by the dictionary. That is,Z =  ∈ Rm×n is a coefficient matrix and each zi is the representation of xi. Equation , which is an improvement of LRR.By solving , the twoXO and XH, then by solving  as the characteristic function which selects the coefficients associated with the ith class in the coefficient vector x. By only using ith class coefficients to reconstruct the test sample Ly as Ly into the minimum residual class between Ly and  Input. Observed data X ∈ Rm×n for k classes; test sample y. Step  1. Normalize the columns of X. Step  2. Extract the salient features of X and remove to some extent noise to get data D defined in  = ||Ly−Dδi(x)||2. Output. Identity(y) = arg miniri(y).Using the result of , we consOur method can be seen as the combination of SRC  and lateTo evaluate our proposed method, we compare our method with SRC , 34, LASIn this subsection, three two-class microarray data sets are used to evaluate our method: colon cancer , prostatThe colon data set contains 62 samples consisting of 40 tumor and 22 normal. The prostate data set contains prostate tumors and normal prostate samples, each consisting of the expression levels of 12600 genes. For the DLBCL data set, the gene expression values were measured by high-density oligonucleotide microarrays. An overview of the three data sets is given in The classification results by using SVM, LASSO, SRC, and the proposed SRC-LatLRR are listed in To further evaluate our method, in this experiment, we also introduced BW feature selection in our method to classify these three data sets. The results are listed in In this subsection, we use four multiclass data sets to further check the classification performance of SRC-LatLRR. The four data sets are lung cancer , leukemiIn lung cancer data set, there are four classes of lung cancer and normal class. This data set contains 203 samples. For leukemia data set, all the samples are classified into acute myelogenous leukemia, acute lymphoblastic leukemia, or mixed-lineage leukemia. The data set includes 72 samples with 11225 genes. For 11_tumors, there are 11 classes of samples, which are ovary, bladder/ureter, breast, colorectal, gastroesophagus, kidney, liver, prostate, pancreas, adeno lung, and squamous lung. This data set includes 174 samples. For the 9_tumors data set, there are 60 samples with 5726 genes. These 9 types of tumors are non-small-cell lung, colon, breast, ovarian, leukemia, renal, melanoma, prostate, and central nervous system. The detailed descriptions about these four data sets are listed in The experimental results are listed in We then introduced BW feature selection before applying our method. The obtained results are listed in λ in (λ in Figures λ is, the bigger the noise level is removed. For these three figures we use ||E||1 to represent the level of the removed noise. From these three figures we can see that the noise that we remove from the original data can not be too much, or it will reduce the accuracy. The reason is that if λ is set to be too small, useful information may be also removed besides noise. On the contrary, if λ is too big, the noise that was removed is too little, and we still can not get a good classification result. The experiment suggests that for colon data sets, λ = 0.011 is the best choice and λ = 0.096 and λ = 0.1 for the prostate and DLBCL data sets, respectively.In this section, we use the data sets described in λ in  affect tFor gene expression data, cancer diagnosis is one of the most important clinical applications. In this paper, we have proposed a new SR-based method for tumor classification which uses the noiseless salient features extracted from the original samples to classify a test sample. We compared our method with several state-of-the-art methods including SVM, LASSO, and SRC on seven data sets. The results of experiments show that the proposed method is better than SVM, LASSO, and SRC in a way. These demonstrate that SRC-LatLRR is effective and efficient for tumor classification. We also introduced gene selection into our method. The results show that gene selection can improve the classification accuracy to some extent.Z* represents the affinity matrix of samples [During the study we also found that, for the optimal result of LatLRR on the observed samples,  samples . In theo