When the number of groups involved is more than two, however, a more general approach is needed.Two major identifiable sources of variation in data derived from the Serial Analysis of Gene Expression (SAGE) are within-library sampling variability and between-library heterogeneity within a group. Most published methods for identifying differential expression focus on just the sampling variability. In recent work, the problem of assessing differential expression between two groups of SAGE libraries has been addressed by introducing a beta-binomial hierarchical model that explicitly deals with both of the above sources of variation. This model leads to a test statistic analogous to a weighted two-sample We describe how logistic regression with overdispersion supplies this generalization, carrying with it the framework for incorporating other covariates into the model as a byproduct. This approach has the advantage that logistic regression routines are available in several common statistical packages.The described method provides an easily implemented tool for analyzing SAGE data that correctly handles multiple types of variation and allows for more flexible modelling. The Serial Analysis of Gene Expression (SAGE) methodology introduced by Velculescu et al.  is a seqBriefly, mRNA transcripts are converted to cDNA and then processed so as to isolate a specific subsequence; starting from the poly-A tail, the subsequence is the 10  or 14 (long SAGE) bp immediately preceding the first occurrence of a cleavage site for a common restriction enzyme. Ideally, this subsequence, or "tag" is sufficiently specific to uniquely identify the mRNA from which it was derived. Tags are sampled, concatenated and sequenced, and a table consisting of the tag sequences and their frequency of occurrence is assembled. The complete table derived from a given biological sample is referred to as a SAGE "library". As most tags are sparse within the entire sample, most libraries contain numbers of tags in the tens of thousands to allow the expression levels to be estimated. Due to the current costs of sequencing, however, the total number of libraries assembled for a given experiment is typically small: often in the single digits and occasionally in the tens.While the type of information, gene expression, being investigated in a SAGE experiment is the same as that in a cDNA or oligonucleotide microarray experiment, there are some qualitative differences in the approaches. First, SAGE uses sequencing as opposed to competitive hybridization. Second, while the expression value reported for an array experiment is a measure of fluourescence and is loosely continuous, SAGE supplies data on gene expression in the form of counts, potentially allowing for a different type of "quantitative" comparison. Third, SAGE is an "open" technology in that it can provide information about all of the genes in the sample. Microarrays, by contrast, are "closed" in that we will only get information about the genes that have been printed on the array.Y, the number of counts of that tag in the library, and n, the total number of tags in the library. In analyzing SAGE data across a series of libraries, interest typically centers on assessing how the underlying true level of gene expression is changing as we move from one library to the next.Mathematically, the information pertaining to the abundance of a particular tag in a sample is summarized in two numbers: Yi} and the set of library sizes {ni}, where the subscript i denotes the specific library. Unless otherwise specified, we will restrict our assessment of differential expression to the case of a single tag. This approach is common to all of the procedures described below. In a real analysis the chosen test is applied to all tags individually and a list of those tags showing differential expression is reported. Different tests will provide altered assessments of significance for individual tags, and hence the list provided will depend on the test employed.When surveyed across a series of libraries, the sufficient statistics containing all of the information about the change in expression for a single tag are the set of counts {Xi describing properties of library i. The most common case involves comparing two groups of libraries, such as cancer and control. In this case the information Xi simply defines which group library i belongs to. If there are more than two groups, Xi can have more levels or can even be vector valued, but as before interest centers on assessing how and whether the expected proportion changes with X.In most problems of interest, there is also covariate information Much work has been done on the problem of comparing expression between two groups. Most of the approaches -9 deal wt-test used to compare two groups of samples in [pi} = {Yi/ni}, capturing the variation between members of a group but losing track of the inherent count sampling nature and variability of the data. The two-sample t-test results can be dramatically different from the pooled test results, as they focus on two different types of variation. The effects of these two approaches on a single group of four libraries are shown in Table This approach, while it captures the count nature of the data, loses information in that variation of the proportions within a group is ignored. As noted by both Man et al.  and Ruijmples in  reduces t-test, tw. Computing the value of this test statistic requires all 8 of the numbers in the main body of Table tw gives results close to those supplied by pooling tests, which focus on within-library variation. Conversely, when the between-library variation is much larger than the within-library variation, tw gives results very similar to those of a two-sample t-test, which focuses on between-library variation. The tw model also allows the relative contributions of the two types of variation to be assessed. Baggerly et al. [Baggerly et al.  proposedy et al.  found thWhile cases with more than two groups have been described in the literature ,13-15, tWe seek to construct a method that takes the count nature of the data into account, deals with multiple groups simultaneously, and allows for variability in the proportions beyond that due to sampling alone. Fortunately, this is not the first time such a problem has arisen.t-test. When the data are counts (SAGE data), and sampling variability needs to be dealt with, the ANOVA test can be adapted to give logistic or Poisson ANOVA. The multi-library test for differential expression proposed by Stekel et al. [The problem of assessing differential expression for multiple groups corresponds to the classical statistical problem of the analysis of variance (ANOVA). When the values of interest are continuous (e.g., microarray log ratios), the test statistics become F-tests, higher-dimensional generalizations of the two-sample l et al.  correspol et al. ,19. Suchdefines both the function of the data that is to be modeled in terms of the covariates (the logit of the proportions) and the precision of each of the measurements. The maximum likelihood estimates of the parameters of this model can be found through iteratively reweighted least squares (IRLS)., into the variance equation, so thatExcess variation within a level, or overdispersion, can be introduced into a logistic regression framework in a number of ways. The most common and most widely implemented approach is to replace the binomial likelihood function being maximized above with a "quasi-likelihood" which differs from the initial formulation solely through the introduction of a scale term, V(Yi) = nipi(1 - pi). values will be the same – just the associated standard errors will be inflated. Logistic regression with quasi-likelihood overdispersion is implemented in a wide variety of statistical packages, including S-PLUS, R, GLIM, and SAS. Another method of introducing overdispersion is to assume a hierarchical model in which the proportions at a given level of the covariate are drawn from a nondegenerate distribution, and the distribution of the observed counts is binomial conditional on the value of the drawn proportion. When a beta distribution is assumed for the proportions, the final unconditional distribution of the observed counts is beta-binomial. This is the model suggested by Baggerly et al. [This approach has the advantage that it inflates the variance of each of the observations by a like amount, so that the estimated y et al.  for modey et al.  in genery et al.  p.201) tV(Yi) = nipi(1 - pi)[1 + (ni - 1)φ],ni are the same. While approximate equality may suffice, even this assumption may be questionable for SAGE data, particularly if some of the libraries are drawn from experiments conducted at different times. Williams [φ only and need not assume further structure from the beta distribution, making the procedure slightly more general. This form of overdispersion is implemented in R as part of the dispmod package.which is equivalent to the quasi-likelihood formulation when all of the library sizes Williams  shows hot-statistics applied to the proportions, in that approximate normality is assumed to hold for the β values rather than the proportions themselves. When we have worked with a beta model for the pi's, we have been led to choices of parameters which yield quite skewed distributions, suggesting that the logit scale may be more appropriate. Working with the β values has the additional advantage that confidence intervals are naturally interpreted in terms of fold changes.In the logistic regression framework, assessing differential expression reduces to a case of deciding whether a set of regression coefficients is different from zero. This can lead to slightly different inferences than models such as 2 test applied to the pooled counts from each of the two groups yields a test statistic of 444.27; the 95% cutoff for the null  distribution is 3.84, with values above this being deemed "significant". The two-sample t-test applied to the two groups of proportions yields 1.60; the 95% cutoffs for the null t6 distribution are ± 2.45, so this test suggests that the difference is not significant, showing the possibility of stark disagreements between tests focusing on different portions of the variability. The tw statistic proposed by Baggerly et al. [t6 distribution, and the qualitative results are far closer to those of the t-test than those of the pooled tests, reflecting the relative dominance of patient heterogeneity in driving the total variation for this tag. We note in passing that this disagreement between the two types of tests is not an isolated incident. When we surveyed all of the tags in this group of libraries we found 10 tags with |t| < 2 and χ2 > 200, and 48 tags with |t| < 2 and χ2 > 50. In Baggerly et al. [t-test was tried, and that in this case the t-test was more likely correct.We begin by comparing the counts of the tag ATTTGAGAAG in 8 colon libraries initially described in Zhang et al. . These 8y et al. , which iy et al.  it was fX are 0 or 1 as the library is in the first or second group, respectively. In models 1 and 2, the fitted proportions are  = e-4.66/(l + e-4.66) = 0.94% and  = e-4.66 - 0.89/(l + e-4.66 - 0.89) = 0.39% for the first and second groups, respectively, and the proportions are only slightly altered in model 3. We note that the estimated coefficient values are exactly the same for the first two models, and this is true for these two approaches in general. Fitting the model with no allowance for overdispersion gives a z-value of β1/s.e. (β1) = -20.42, which is definitely significant. Note that the square of this value is of the same order as the value found by the χ2 test. The Pearson residuals from this model, , however, show a problem. If the model fits well, these should be approximately distributed as a standard normal, with extreme values from a set of 8 observations around 3 or 4 in magnitude. The actual values, -14.6 and 19.0, are far too extreme. When the model is fit with allowance made for overdispersion, the point estimate of the dispersion parameter is  = 187.57; this value should be close to 1 if there is no overdispersion. With this allowance made the t-value of -1.49 is no longer significant. This t-value can be found from the first z-value (-20.42) by dividing by  = 13.70. Similarly scaling the residuals yields values far more commensurate with a standard normal. We note that due to the differences in the models employed, the presumed distributions of the test statistics have changed. If we assume that the standard logistic model with no overdispersion holds, the test statistic has an approximately normal distribution. This is because the number of total successes is driving the binomial distributions to approximate normality. When we shift to a model where we presume the existence of overdispersion, the test statistic now has a t distribution. This is because our estimate of the variance is now strongly dependent on the precision with which we can estimate the overdispersion parameter, and this precision depends on the number of libraries, not the number of successes. Fitting this model with the hierarchical type of overdispersion, model 3 in Table  is still not significant. The difference in  values from those found before is due to the fact that in this model the amount of overdispersion attributed to each proportion changes slightly with library size, thus altering the weights used in the regression model. The point estimate for the hierarchical dispersion parameter φ is  = 3.399e - 03, so the multipliers for the binomial variances areThe results from three logistic regression model fits to the data are shown in Table ni} - 1) =1 + ({(169.62, 165.78, 141.62, 190.32,207.26, 190.12, 175.35, 208.84).averaging these gives 181.11 which is close to the value found for the quasi-likelihood dispersion parameter. We note that the differences in coefficient values for models 2 and 3 are largely cosmetic, but the differences in significance between model 1 and the others are not. Choosing to account for overdispersion is more important than the precise model used to achieve this.t-values about -1.49, whereas the two-sample t-test and the modified version tw suggested by Baggerly et al. [t-values of about -1.6 (as noted earlier, agreement between t and tw suggests that for this tag, the between-library variation is much larger than the within-library variation). There are two reasons for this difference. First, the t statistic works on the proportion scale, and logistic regression works on the β scale, which is roughly the log proportion scale. Second, the tw statistic used here,We note that the overdispersed logistic regression approaches give y et al.  both givt test and the weighted tw test both give -1.57, and the logistic regression t value is -4.16.does not assume that the overdispersion factor is the same in the two groups being compared; the variance estimate is not pooled. The latter difference is actually the more important for this contrast, particularly as the variance estimate from the first group of size 2 is very unstable. This effect is not always subtle; if we consider instead the tag GCGAAACCCT, with counts given in Table β coefficient scale. Second, when the number of libraries in a group is quite small, which will often be the case with SAGE data, we prefer the pooled estimate of the variance. This preference is due in large part to its greater stability through the use of more degrees of freedom. It is possible to explicitly incorporate levels of overdispersion that change with the covariates in logistic regression, but we have not pursued this here.Of the two answers, we tend to prefer the one given by the logistic regression fit, for two reasons. First, when we have fit the parameters of the beta distributions for the proportions directly, we have found the distributions to be quite skewed. As such we find it better to assume rough normality on the Above, we treated the colon libraries as if they came from two groups, but it is more natural to view them as coming from three: normal samples, primary tumors, and cell lines. When we have data from multiple groups, there are two different ways in which this changes the nature of the problem. First, if we are only interested in comparing two of the groups, it is often nonetheless worthwhile to incorporate the data from the other groups into the model. The reason for this is that when overdispersion is driving the variance, the significance of our results depends strongly on the precision with which we can estimate the overdispersion parameter. The libraries in the groups not directly involved in the comparison of interest can still supply information about the overdispersion parameter and increase the degrees of freedom of the associated t-test. Second, by examining the fitted proportions for all groups, the relative sizes of the transitions can be assessed.We begin by looking at the results for a single tag flagged as interesting in the paper by Zhang et al. , namely We first attempt to compare the levels in normal colon and primary tumor while ignoring the cell lines (ie, using just four libraries), and then using a model incorporating all three groups. The results using logistic regression with hierarchical overdispersion are shown in Table x1 = (0,0,1,1) denoting which of the two groups the library belongs to. This model produces an overdispersion estimate of  = 8.938e - 05, for inflation factors ofIn the model with only two groups, we have a single covariate vector ni} - 1) = (5.43, 5.33, 4.70, 5.98).1 + . In general, if we have k groups, we need to use k - 1 covariate vectors. Here, we use x1 = (0, 0, 1, 1, 0, 0, 0, 0) and x2 = (0, 0, 0, 0, 1, 1, 1, 1). The set of all 0s (x1 = 0, x2 = 0) corresponds to the first group, here normal colon, and the other groups are defined by which one of the other covariates is nonzero: Group 2 (primaries), (x1 = 1, x2 = 0), Group 3 (cell lines), (x1 = 0, x2 = 1) As we are still focused on the difference between normal colon and primary tumors, for which the logit values are β0 and β0 + β1 respectively, the main interest remains on whether β1 is significantly different from zero, and the predicted logit for the cell line group, β0 + β2, does not enter the problem directly. Fitting this model produces an overdispersion estimate of  = 1.160e - 04, for inflation factors ofThe fact that these factors are significantly larger than one suggests that the within-group heterogeneity is the dominant component of the variance not explained by the model. In the model with three groups, we cannot use a single covariate vector ni} - 1) =1 + ({(6.76, 6.62, 5.80, 7.46,8.04, 7.45, 6.95, 8.09)., appear significant once allowance is made for overdispersion, but there is an interesting point to note. Even though the point estimate of overdispersion increases when the cell lines are included, and the value of the t-statistic (/s.e) associated with the difference declines, the associated p-value indicates an increase in significance. Without using the cell lines, we have just 4 libraries, and after estimating the mean proportions in each group just 2 degrees of freedom for estimating φ. When we use the cell lines, we have 8 libraries and 5 degrees of freedom for estimating φ. Thus, the degrees of freedom in the t-tests shift from 2 to 5. The t2 distribution has very wide cutoffs, and the t5 is much closer to normal. In general, the inclusion of related groups can improve estimation by increasing the precision of our estimate of overdispersion.In neither case (considering two groups or three) does the contrast between normal colon and primary tumor, represented as the magnitude of β0 and β0 + β2 respectively, by checking the significance of . Likewise, we can look at the difference between primary tumors and cell lines, for which the logits are β0 + β1 and β0 + β2 ,by testing the significance of the difference . While this significance is not listed in the table directly, we can compute the standard error of this contrast, s.e. , divide the estimate by its standard error to get a t-statistic with the degrees of freedom listed (here 5), and compute a p-value accordingly.In fitting the model with three groups, of course, we have also gained the ability to look at other contrasts. For example, we can look at normal colon versus cell lines, for which the logits are F-test. When dealing with generalized linear models, the quantity -2 * log (likelihood ratio), known as the deviance, plays a role analogous to the variance in ANOVA and thus we can speak of the analysis of deviance. The analysis of deviance is complicated by the inclusion of overdispersion in the model, requiring a multi-step approach in which several different models are fit in succession. These models are listed in Table x1 and x2, and fitting the full model with both present, β0 + β1 + β2, gives  = 1.160e - 04 as noted above. Second, submodels are fit with the value for overdispersion fed in as fixed. In this case, the submodels are β0 + β2, using x2 as the only covariate, β0 + β1, using x1 as the only covariate, and β0, using no covariates and simply fitting a single proportion to all of the data. The results are shown in Table F distribution. Here, for example, testing whether the overall model including β1 and β2 explains things significantly better than just fitting the same proportion throughout (β0) reduces toIt is also possible to perform an omnibus test of whether there exists any significant difference among the groups, which is logistic ANOVA for proportions. The regular ANOVA test looks at the amount of variance explained by the terms of interest in the model and compares this to the amount of residual variance. Adjusting for the degrees of freedom in each group gives an β1 in addition to the constant appears to explain very little; this is due to the way in which we have chosen the entries of X, so that including β1 isolates the effect of the primary tumor group, but excluding β2 still combines the normal colon group with the cell line group for the contrast. This latter grouping blurs the normal colon vs primary tumor distinction found to be a bit larger earlier.indicating that the overall difference between groups is not significant at the 5% level. It may be noted that the submodel including just x1 being 0 or 1 as the sample is colon or pancreatic, respectively, and x2 being 0 or 1 as the sample is a primary tumor or a cell line, respectively. Inference reduces to testing the significance of β2, with the scale of natural variation being assessed only after the effects of the change in tissue type, β1, have been factored out.It is possible to use the logistic regression approach to partition the variance amongst multiple effects of interest. For example, in the above section we considered a case with colon libraries taken from both primary tumors and cell lines. Such data is also available for other organs, eg pancreas. If we are interested in identifying consistent differences between primary tumors and cell lines, it would be natural to use libraries from both organ types. However, if these were then compared as two groups, primary vs cell lines, the differences would be difficult to isolate due to the large differences between tissue types within both the primary and cell line groups. The solution is to fit a model with two covariates, with In the above example, we allowed for the effect of one other effect, tissue type. In principle, multiple factors can be allowed for through the inclusion of other covariates. Likewise, though the two covariates in the above example were both "factors" having a finite number of unordered levels, it is possible to include continuous covariates in the modelling process as well.x1 = (0, 0, 1, 1, 0, 0, 1, 1), indicating whether the library is normal (0) or primary tumor (1), and x2 = (0, 0, 0, 0, 1, 1, 1, 1), indicating the organ from which the library was derived. In the second example, we posit that in addition to the above information, we have access to the levels of a biomarker potentially predictive of survival. These levels are supplied as the values of a third covariate vector, x3 = (0.89, 0.35, 0.66, 0.23, 0.30, 0.54, 0.90, 0.90). The values for x3 were generated as random draws from a uniform distribution. In terms of fitting the models, the mechanics are similar to those presented earlier. The model fits are presented in Table To illustrate this, we give two hypothetical examples using the counts for the GCGAAACCCT tag from Table β involves the fold change to the proportion in the second group, leading to division by zero. When the proportions are this small, the binomial variability dominates the heterogeneity and the values are completely noninformative with respect to the estimation of overdispersion.The logistic regression fitting procedure can break down, or exhibit lack of convergence. Typically this means that all of the proportions in one of the groups are zero or one; only the former is realistic in the context of SAGE data. This is natural, in that the maximum likelihood point estimate for the group proportion is 0, and inference for We propose a fix that is iterative in nature in that it requires the logistic fitting routine to be run three times. To illustrate this procedure, we will use the data from tag ATTTGAGAAG in Table  = 3.71e - 03.The first run of the fitting procedure serves to estimate the overdispersion parameter. This fit uses just the groups that have nonzero counts, omitting the problematic group(s). Here, this involves fitting a single proportion to the six libraries in group 2. The fitted proportion is 0.40%, and the overdispersion estimate is ni trials is 1/(ni + 1); with multiple trials, it is 1/((∑ni) + 1). This is the value we use. This value is actually quite conservative here, for two reasons. First, the uniform distribution places far too much chance on the possibility of proportions greater than a few percent, which will never be observed with SAGE data. Restricting the distribution to be uniform over the range [0, 0.02] should be more than adequate. Second, the presence of overdispersion means that pooling the samples underemphasizes the evidence of a small proportion being supplied by the zero variance of the observed proportions. While we could pursue a more optimal proportion, we choose in this case to simply use the simplistic bound noted above. Here, as the library sizes in the first group are 49610 and 48479, the proportion is 1/(49610 + 48479) and the faked counts are 0.506 = 49610/(49610 + 48479) and 0.494, respectively. Some reformatted results from this fit are shown in Table The second run of the fitting procedure takes the overdispersion parameter as given, and fits the data after replacing the zero proportions in a group with the same small nonzero proportion, giving us a hopefully conservative estimate of the fold change. This type of replacement is commonly used, and is most often justified via the assumption of a vague prior distribution for the proportions, with the point estimate being derived as the posterior mean or mode. A common assumption for a prior in dealing with proportions is the uniform distribution. The posterior mean after 0 successes are observed out of t-value  relies on the approximate normality of the likelihood function in the vicinity of the maximum, and this shape assumption breaks down severely if the number of counts in one group is small. Tests based on changes in the scaled deviance, corresponding to likelihood ratio tests, are better.The results for this fit are ridiculously "insignificant". The problem lies in the fact that the use of a The third run of the fitting procedure fits a simpler submodel, in this case a single proportion for all eight libraries, using the same overdispersion estimate so as to measure the change in deviance. The results of this fit are shown in Table Here, we cannot conclude (given the level of overdispersion) that the difference is real. Note that the degrees of freedom used in the denominator is 5; this follows from the fact that only 6 libraries were used to estimate the overdispersion parameter, and one of those 6 degrees of freedom was needed to estimate the proportion.In general, when any of the groups has very small counts, checking the change in deviance is a good idea.Logistic regression with overdispersion addresses three issues with SAGE data: simultaneously modelling multiple types of variance, dealing with multiple groups at once, and allowing for the incorporation of covariates. This procedure is widely implemented in available software. Further, and most importantly, viewing SAGE data in the logistic regression setting supplies the framework for thinking of models that describe such data.t-tests. The regression setting carries with it other benefits, such as a well-developed body of work regarding model checking, residual analysis, and detection of outliers. For example, the influence of any given library tag count on the overall analysis can be assessed, and methods can be made more robust by bounding these functions so that no single library drives the results.Dealing with multiple types of variance yields significance estimates we believe to be superior to those derived from pooled counts or from There are some areas in which we can identify difficulties and see room for improvement.First, the model that we are using for the error may be improved. For SAGE data, the proportion associated with a specific tag is rarely on the order of a percent, sopi) ≈ log(pi)logit = β0 + β1xi + εi,logit, where "mixed" refers to the fact that we have both fixed effects of interest, the changes with the covariates, and random shocks whose variance needs to be estimated and allowed for. Williams [where Williams  suggestsWilliams , p.272, Williams ). We areφ over like groups. Work on these issues is ongoing  = log(pi/1 - pi) = β0 + β1Xi + ε. This particular choice is suggested by the form of the likelihood function for binomial data (see McCullagh and Nelder [V(pi) = pi(1 - pi)/ni, so that the precision with which an observation is known depends on both the value of that observation and on the size of the total ni from which the proportion was derived. In the case where the observations are known with differing precisions, then the standard adjustment is to fit a weighted version of least squares, minimizing a weighted sum of the squared differences between the observations and their fitted values, where the weights are inversely proportional to the variances of the observations. Thus, at the first step we fit a logistic curve using weighted least squares where the weights are inversely proportional to the variances associated with our initial estimates of the proportions, (Yi + 0.5)/(ni + 1). After this first fit, we now have predicted values for each of the observations, and these predicted values in turn suggest new values for the variances and hence the weights. Thus, the second step is to refit the data using the new weights. This process is iterated (iteratively reweighted least squares, IRLS) until the changes in the predicted values from one fit to the next are small enough that the procedure is said to have converged.We want to fit the observed proportions, d Nelder , p.28–32V(pi) = nipi(l - pi), for  > 1. The added mechanics for computing the hierarchical form are somewhat involved and we refer the reader to Williams [, and significance tests need to be adjusted accordingly. In order to estimate , we return to the weighted squared deviations between observations and predictions noted above. Ideally, the sum of the squared weighted residuals will have a chi-squared distribution with k - p degrees of freedom, where k is the total number of libraries and p is the number of β terms being estimated. As the mean of a chi-squared distribution is equal to its degrees of freedom, we get our initial estimate of  by dividing the sum of squared weighted residuals by the posited degrees of freedom:Even after the process has converged, it is often the case that the sizes of the squared deviations will be substantially larger than might be expected if the variances were of exactly the form given above. In this case, the data are said to exhibit overdispersion relative to the postulated model, and we seek to estimate the scale of the overdispersion. We deal with the quasi-likelihood case of overdispersion here, where the variance is really of the form Williams  for deta, the test statistics are scaled by  and the significances recomputed. In the cases below, we outline the procedure and couple the descriptions with scripts for the freeware package R. In each case, the approach begins by loading the data corresponding to the tag counts Yi and the library sizes ni, which are used to supply the observed proportions. The main distinction between the cases resides in how the covariate X values are defined. All of the models assume the presence of a constant vector X0 of all ones; this produces the corresponding estimate for β0. Our discussion will likewise treat this covariate as present in all modelling steps.Given the estimated value of # Source code for models used in the paper# "Overdispersed Logistic Regression for# SAGE: Modelling Multiple Groups and# Covariates", by Baggerly et al.########################################### First, we deal with the case of two# groups, and introduce the methods for# fitting the logistic regression models.##########################################if(0){# Load the tag counts for ATTTGAGAAG (y)# from the 8 libraries in Zhang et al.# , the ass# and the covariate vector indicating# which of two groups the librares# belong to, normal or cancer (x).y <- c(320, 600, 312, 549,246, 65, 41, 52);n <- c(49610, 48479, 41371, 55700,60682, 55641, 51294, 61148);x <- c(0, 0, 1, 1, 1, 1, 1, 1);# Now fit a standard logistic regression# model to the data, with no allowance# for overdispersion. This is done# through a call to the generalized# linear model (glm) routine. help(glm)# provides more information about the# nature of the arguments here.fit1 <- glm(cbind(y, n-y) ~x,family=binomial);# check the resultssummary(fit1);# Next, we refit the model while# allowing for overdispersion of the# quasilikelihood type; all variances# are inflated by a common factor. This# call differs from the first only in# the definition of the glm "family" to# be used.fit2 <- glm(cbind(y, n-y) ~x,family=quasibinomial);# check the resultssummary(fit2);# Ideally, the sum of the squared# Pearson residuals should have a chi-# squared distribution, with mean equal# to its degrees of freedom. Dividing# the sum by the degrees of freedom# gives our initial estimate of the# overdispersion parameter.varQL <- sum^2)/fit2$df.residual;# Finally, we refit the model using the# overdispersion method suggested by# Williams , where t# inflated by factors that are slightly# different depending on the underlying# library sizes. This routine is# implemented in the R package "dispmod"# which is available at# library("dispmod");fit3 <- glm.binomial.disp(fit1);# check the resultssummary(fit3);phi <- fit3$dispersion;# Note that the reported p-values from# this fit are incorrect. This is due to# the assumption that the test-stats# have normal distributions, even though# we have had to estimate the# overdispersion parameter. When we have# to perform this estimation, the# correct test is a t-test, with a# number of degrees of freedom# corresponding to the number of# libraries less the number of estimated# parameters. As the number of libraries# is typically not large, this can# create a large difference.sumfit3 <- summary(fit3);t.values <- summary(fit3)$coefficients [,"z value"];p.values <- 2 * pt,fit3$df.residual);}########################################### Next, we deal with three groups##########################################if(0){# We begin by focusing on gains# available when multiple groups are# present, even if the other groups are# not directly part of the contrast of# interest, due to the additional# information that the added groups can# provide about the scale of the# overdispersion.# Here, we use the data from the tag# TGCTGCCTGT, and this time we note that# there are 3 groups of libraries:# normals (libraries 1–2), primary# tumors (libraries 3–4), and cell lines# (libraries 5–8). If we are interested# in the contrast between normals and# primary tumors, we can fit this using# only the data from those two groups,# or using the data from all three.# First, fit the model as if there were# just two groups present.y <- c(0, 1, 1, 15);n <- c(49610, 48479, 41371, 55700);x <- c(0, 0, 1, 1);fit1 <- glm(cbind(y, n-y) ~x,family=binomial);fit2 <- glm.binomial.disp(fit1);# get the correct p-valuesfit2.t.values <- summary(fit2)$coefficients [,"z value"];fit2.p.values <- 2 * pt, fit2$df.residual);# Next, fit the model assuming that# there are three groups. In this case,# we cannot use a single covariate# vector x, as this is not suited to# indicating 3 or more groups in an# unordered fashion # In general, if we have k groups, we# need to use k-1 covariate vectors.# Here, we use# x1 <- c(0, 0, 1, 1, 0, 0, 0, 0);# x2 <- c(0, 0, 0, 0, 1, 1, 1, 1);# The set of all 0s (x1 = 0, x2 = 0)# corresponds to the first group, here# the normals, and the other groups are# defined by which one of the other# covariates is nonzero:# Group 2 (primaries), (x1 = 1, x2 = 0),# Group 3 (cell lines), (x1 = 0, x2 = 1)y <- c(0, 1, 1, 15, 9, 1, 12, 27);n <- c(49610, 48479, 41371, 55700,60682, 55641, 51294, 61148);x1 <- c(0, 0, 1, 1, 0, 0, 0, 0);x2 <- c(0, 0, 0, 0, 1, 1, 1, 1);fit3 <- glm(cbind(y, n-y) ~x1 + x2,family=binomial);fit4 <- glm.binomial.disp(fit3);# get the correct p-valuesfit4.t.values <- summary(fit4)$coefficients [,"z value"];fit4.p.values <- 2*pt, fit4$df.residual);# The above approach has fit the model# with all of the covariates available,# but in order to perform an analysis of# deviance we want to fit various# submodels using the same estimate of# overdispersion as found here. In this# case, there are 3 submodels:fit5 <- glm(cbind(y, n-y) ~x1,family=binomial,weights = fit4$disp.weights);fit6 <- glm(cbind(y, n-y) ~x2,family=binomial,weights = fit4$disp.weights);fit7 <- glm(cbind(y, n-y) ~1,family=binomial,weights = fit4$disp.weights);# alternatively, the anova function can# be used, but this only considers the# submodels obtained by adding terms# sequentially. Thus, we get the# deviances for beta_0 (the null model),# beta_0 + beta_1 (adding the x1# covariate only), and beta_0 + beta_1 +# beta_2 ;}########################################### Next, we deal with the case of other# covariates, possibly continuous.##########################################if(0){# Here, we are using the counts from the# GCGAAACCCT tag, but we are treating# the 8 libraries as coming from tissue# type 1 (libraries 1–4) and tissue type# 2 (libraries 5–8), with normal tissue# of both types (libraries 1–2, 5–6) and# primary tumor of both types (libraries# 3–4, 7–8). In this hypothetical# example, we are able to partition the# changes into effects associated with# normal/primary differences (x1) or# tissue 1/tissue 2 differences (x2).y <- c(167, 566, 64, 98, 33, 47, 40, 27);n <- c(49610, 48479, 41371, 55700,60682, 55641, 51294, 61148);x1 <- c(0, 0, 1, 1, 0, 0, 1, 1);x2 <- c(0, 0, 0, 0, 1, 1, 1, 1);fit1 <- glm(cbind(y, n-y) ~x1 + x2,family=binomial);fit2 <- glm.binomial.disp(fit1);# get the correct p-valuesfit2.t.values <- summary(fit2)$coefficients [,"z value"];fit2.p.values <- 2*pt, fit2$df.residual);# Next, again using the tag as above, we# posit that we also have access to the# levels of a biomarker potentially# predictive of survival, supplied as# the levels of another covariate x3.# The values supplied here were# generated as random draws from a# uniform (0,1) distributionx3 <- c(0.89, 0.35, 0.66, 0.23,0.30, 0.54, 0.90, 0.90);fit3 <- glm(cbind(y, n-y) ~x1 + x2 + x3,family=binomial);fit4 <- glm.binomial.disp(fit3);# get the correct p-valuesfit4.t.values <- summary(fit4)$coefficients [,"z value"];fit4.p.values <- 2*pt, fit2$df.residual);}KAB, LD and JSM developed the main ideas and the methodology; LD did most of the coding. CMA supplied SAGE data and provided practical feedback on aspects of earlier approaches found to be wanting, thus guiding further development.


Bacillus Calmette Guérin (BCG)-immunotherapy has a well-documented and successful clinical history in the treatment of bladder cancer. However, regularly observed side effects, a certain degree of nonresponders and restriction to superficial cancers remain a major obstacle. Therefore, alternative treatment strategies are intensively being explored.Mycobacterium tuberculosis for the treatment of bladder cancer. The phosphate transport protein PstS1 which represents the phosphate binding component of a mycobacterial phosphate uptake system is known to be a potent immunostimulatory antigen of M. tuberculosis. This preclinical study was designed to test the potential of recombinant PstS1 to serve as a non-viable and defined immunotherapeutic agent for intravesical bladder cancer therapy.We report a novel approach of using a well defined immunostimulatory component of Mononuclear cells (PBMCs) were isolated from human peripheral blood and stimulated with PstS1 for seven days. The activation of PBMCs was determined by chromium release assay, IFN-γ ELISA and measurement of lymphocyte proliferation. The potential of PstS1 to activate monocyte-derived human dendritic cells (DC) was determined by flow cytometric analysis of the marker molecules CD83 and CD86 as well as the release of the cytokines TNF-α and IL-12. Survival of presensitized and intravesically treated, tumor-bearing mice was analyzed by Kaplan-Meier curve and log rank test. Local and systemic immune response in PstS1-immunotherapy was investigated by anti-PstS1-specific ELISA, splenocyte proliferation assay and immunohistochemistry.in vitro experiments showed that PstS1 is able to stimulate cytotoxicity, IFN-γ release and proliferation of PBMCs. Further investigations showed the potential of PstS1 to activate monocyte-derived human dendritic cells (DC). In vivo studies in an orthotopic murine bladder cancer model demonstrated the therapeutic potential of intravesically applied PstS1. Immunohistochemical analysis and splenocyte restimulation assay revealed that local and systemic immune responses were triggered by intravesical PstS1-immunotherapy.Our in vitro activation of human immune cells by recombinant PstS1. In addition, intravesical PstS1 immunotherapy induced strong local and systemic immune responses together with substantial anti-tumor activity in a preclinical mouse model. Thus, we have identified recombinant PstS1 antigen as a potent immunotherapeutic drug for cancer therapy.Our results demonstrate profound  Urothelial carcinoma of the bladder accounts for about 4% of all cancer related death in man. The large majority of tumors (70–80%) is superficial at diagnosis and has a high rate of local recurrence (70%) and progression (30%) after local surgical therapy. Therefore, patients require lifelong medical follow-up examinations and effective prophylactic treatment to prevent recurrences and progression of the tumor.Mycobacterium bovis – has a well-documented and successful clinical history. Immunotherapy with BCG is performed by six weekly instillations of viable Mycobacteria (for induction course) into the bladder of patients after initial transurethral resection of the tumor. Until now various clinical trials have shown that this type of therapy is superior to topical chemotherapy and transurethral resection of the tumor alone to prevent recurrences and local progression especially in patients with high risk tumors [In this type of cancer, the immunotherapeutic use of Mycobacteria – specifically Bacillus Calmette-Guerin (BCG), a non-pathogenic strain of k tumors -3. Despik tumors . Furtherk tumors . BCG's ak tumors . Roughlyk tumors ,8. Thus,M. tuberculosis belonging to the family of ABC (ATP-binding cassette) transporters [In order to limit toxicity recent endeavors are therefore focused on the development of alternative non-viable products and some of those have already been tested in preclinical and clinical studies -12. The sporters ,15. It isporters ,17. Moresporters .in vitro system and in a previously described murine model of experimental bladder cancer therapy [We hypothesized that this highly immunogenic protein antigen could function as an effective biological response modifier in immunotherapy of bladder cancer. To test the tumortherapeutic potential of recombinantly expressed PstS1  we perfo therapy . Because therapy -23, we pThe data reported herein demonstrate that PstS1 is a potent activator of human tumor-cytotoxic MNCs, induces maturation and activation of human dendritic cells and most importantly is very effective in the treatment of experimental orthotopic bladder cancer. While local and systemic immune responses were observed in sensitized and non-sensitized mice, immunotherapy of cancer was only successful in non-sensitized animals.2 in RPMI 1640 (PAA Laboratories, Linz/Austria) containing 10% FCS (Linaris, Bettingen/Germany), 1% glutamine, 100 U/ml penicillin and 100 μg/ml streptomycin. The murine bladder tumor cell line MB-49 was cultured at 37°C and 5% CO2 in DMEM (PAA Laboratories, Linz/Austria) containing 10% FCS (Linaris, Bettingen/Germany), 1% L-glutamine, 100 U/ml penicillin and 100 μg/ml streptomycin.The human bladder tumor cell line T-24 was cultured at 37°C and 5% CO6/ml in RPMI-1640 medium containing 5% human serum, 100 U/ml penicillin and 100 μg/ml streptomycin. Recombinant PstS1 in concentrations from 0.1 μg/ml – 100 μg/ml, BCG (Connaught substrain, Immucyst, 4 × 104 cfu/ml), or PBS (100 μl) was added and the cells were cultured for 7 days in 6-well microtiter plates at 37°C and 5% CO2.MNCs from heparinized blood of healthy human donors were obtained by discontinuous gradient centrifugation using Biocoll Separating Solution (Biochrom, Berlin/Germany) and adjusted to a concentration of 1 × 10251CrO4 (ICN, Irvine/USA) for 1 h at 37°C, washed and resuspended at 5 × 104 cells/ml. Effector cells were added to a total of 100 μl of target cells at an effector:target ratio of 40:1. The radioactive content of the supernatant was measured in a gamma-counter (Berthold, Wildbad/Germany). The specific lysis was determined according to the formula: spec. lysis (%) = 100 × (Exp – Spo)/ (Max – Spo) where Exp is the experimental release, Spo the spontaneous release and Max the maximum release.Cytotoxicity was determined in a standard 4-hour chromium release assay. Target cells were labeled with NaSupernatants of PstS1, BCG and PBS stimulated PBMCs were recovered at days 2, 5 or 7 and examined for the presence of IFN-γ. Each experiment was carried out several times with different donors. Detection was performed with an anti human IFN-γ ELISA-Kit according to the manufacturers' instructions (eBioscience, San Diego/USA).5 cells / well of the stimulated PBMCs were cultured in a microwell plate and 1 μCi 3H-Thymidine (five wells per sample) (Amersham, Freiburg/Germany) was added during the last 18 h of stimulation. DNA was harvested on filter membranes and thymidine incorporation was measured by liquid scintillation counting .PBMCs were stimulated for 2 to 7 days with PstS1, BCG, PBS or PHA (Sigma). 2 × 106 monocytes were cultured for seven days with 2 ml RPMI 1640, 10% FCS (Linaris, Bettingen/Germany) 1% penicillin/streptomycin and IL-4/GM-CSF (500 U/ml each) (TEBU/PeproTech, Offenbach/Germany) in 24-well cell culture plates (Nunc, Wiesbaden/Germany). Exchange of medium was carried out at day three and day five. This procedure resulted in full differentiation of monocytes with no undifferentiated monocytes present in the cultures after seven days. Immature DCs were stimulated for three days with 10 μg/ml PstS1 or BCG (MOI = 0.01) and subjected to flow cytometry. Debris was excluded from the analysis according to FSC/SSC gating.After separating peripheral blood mononuclear cells (PBMCs) from heparinized blood of healthy donors by Ficoll-Paque centrifugation, monocytes were elutriated by counterflow centrifugation. For generation of immature DCs, 2 × 10Expression of cell surface molecules was analyzed by flow cytometry  using phycoerythrin (PE) conjugated monoclonal antibodies (mAbs): anti-CD1a (Biosource, Camarillo/USA), anti-CD83 (Pharmingen, San Diego/USA), anti-CD14 and anti-CD86 (Dianova, Hamburg/Germany).5 DCs suspended in PBS containing 5% human serum and 0.1 % sodium acide were incubated with mAbs for 30 minutes on ice. Then cells were washed with PBS and resuspended in 400 μl 1.5 % paraformaldehyde in PBS.1 × 10Supernatants of PstS1, BCG and PBS stimulated DCs were recovered at day 3 after stimulation and examined for the presence of TNF-α and IL-12. Each experiment was carried out several times with different donors. TNF-α detection was performed with a quantitative ELISA, provided by Dr. H. Gallati (Intex, Muttenz/Switzerland).IL-12 detection was carried out with an ELISA-Kit from eBioscience (San Diego/USA).E. coli and purified by standard chromatography.PLG-particles (10 mg/ml) (Lionex, Braunschweig/Germany) were diluted 4:1 with PBS and the pH was adjusted to 7.0 with NaOH. PstS1 was recombinantly expressed in 250 μg of PLG-particles were loaded with PstS1 or BSA by coincubation with 60 μg of the respective protein for 15 h at RT on a shaker . Afterwards the particles were spun down and washed for 5 min with 100 μl PBS. The supernatant was removed and the particles were diluted in 100 μl of PBS. 10 μl of loaded particle solution were centrifuged and the pellet was treated for 30 min at RT with 0.1 M NaOH/ 10% SDS. After centrifugation in a minifuge, supernatant was removed and analyzed on a 10% SDS page. Using recombinant PstS1 as a calibration standard it was determined that 250 μg PLG-particles bound approximately 50 μg of PstS1 protein.4 MB-49 cells were instilled into the bladder. Intravesical immunotherapy with 100 μg PstS1 in 100 μl PBS was performed on days 1, 8, 15 and 22 after tumor implantation. Control groups were instilled with PBS (100 μl) alone. The viability status of the mice was checked daily. Surviving mice were sacrificed on day 70. Survival of mice was compared using Kaplan-Meier analysis and log-rank test. Trial II: Performance similar to trial I with the following differences. 11 animals per group were subcutaneously injected with a) 250 μg empty PLG-particles b) 250 μg PLG-particles coated with 50 μg BSA c) 100μl of PBS. Intravesical treatments were performed with 100 μg PstS1 or PBS only.Female C57BL/6 mice were purchased from Charles-River Laboratories (Sülzfeld/ Germany) at the age of 6–8 weeks. To test efficacy of PstS1 therapy in C57/BL6 mice a published syngeneic, orthotopic bladder cancer model was used . Trial IC57/BL6 mice (5 per group) were treated as described in table Murine blood was obtained immediately after sacrificing mice by heart punctation and centrifuged in a microcentrifuge to obtain the serum for antibody analysis which was performed with the PstS1-ELISA-kit (Lionex GmbH, Braunschweig/Germany) according to the manufacturer's instructions.2O. Afterwards 2 × 105 cells / well were cultured in a microwell plate and stimulated with 10 μg/ml PstS1 for a period of five days (five wells per sample). Then 1μCi 3H-Thymidine (Amersham, Freiburg/Germany) was added and cells were incubated for 15 h at 37°C and 5% CO2. Finally the DNA was harvested on filter membranes and 3H-Thymidine incorporation was measured by liquid scintillation counting .Splenocytes were isolated, washed with PBS and erythrocytes were lysed with Hin vitro systems. As in these systems the immunostimulatory mechanisms of whole BCG bacteria have been thoroughly studied in the past [To assess the immunostimulatory properties of our PstS1-preparation we analyzed several human PBMC the past ,25-27 weIn a first set of experiments the optimal concentration of PstS1 to stimulate human PBMC cytotoxicity Fig. , IFN-γ rAs depicted in figure Dendritic cells are central cellular mediators for the induction of anti-tumor immune responses. We tested the potential of PstS1 to induce activation and maturation of human monocyte-derived DCs.After 7 days of differentiation human monocyte-derived dendritic cells showed the typical immature phenotype with low or absent expression of the monocyte marker CD14 and the maturation markers CD83 and CD86 Fig. . At the in vitro we conducted a series of studies to test its immunotherapeutic potential in vivo. Prompted by the controversy about the role of prior exposure to mycobacterial antigens in BCG immunotherapy we wanted to assess the in vivo activity of PstS1 in sensitized and non-sensitized mice. For this purpose two independent in vivo experiments were carried out. PLG-particles have previously been described as useful agents for sensitization of mice to mycobacterial antigens and were used as such in our study [After we have shown profound immunostimulatory properties of PstS1 ur study .In a first series of experiments we analyzed mice which had been s.c. sensitized with empty control particles or with particles loaded with PstS1 antigen. Ten days later mice received inoculation of tumor cells and subsequent intravesical treatment with either PstS1 or PBS control solution (four weekly instillations). Mice which received s.c. PBS followed by intravesical PBS served as negative controls. Using this experimental set up we could show that pre-vaccination with empty particles and intravesical treatment with PstS1 protein significantly prolonged survival of mice and thus provided a clear therapeutic benefit in this model of experimental bladder cancer Fig. . When miAfter we had demonstrated successful immunotherapy of bladder cancer by local instillation of PstS1 we next analyzed the local and systemic immune response of the various treatment groups to obtain initial insight into the immunological basis of this novel immunotherapy. To achieve this we analyzed the systemic antibody response by a semiquantitative anti-PstS1 IgG ELISA, the systemic lymphocyte response by splenocyte restimulation assays and the local immune response in the bladder by immunohistology.in vitro stimulation with PstS1. In contrast, the four groups of mice which had been exposed to PstS1 either s.c. or intravesically or via both routes strongly responded to specific restimulation of their splenocytes. Again these data indicate that systemic immune responses to PstS1 can be achieved by both s.c. and intravesical challenge. While a s.c. sensitization of mice with PstS1 abrogated the anti-tumor effect of intravesical PstS1  declare that they have no competing interests.CS carried out most of the experiments, provided experimental protocols and prepared the initial version of the manuscript. ABu contributed to figure The pre-publication history for this paper can be accessed here:


The objective of this study was to demonstrate the use of the Heckman two-step method to assess and correct for bias due to missing health related quality of life (HRQL) surveys in a clinical study of acute coronary syndrome (ACS) patients.We analyzed data from 2,733 veterans with a confirmed diagnosis of acute coronary syndromes (ACS), including either acute myocardial infarction or unstable angina. HRQL outcomes were assessed by the Short-Form 36 (SF-36) health status survey which was mailed to all patients who were alive 7 months following ACS discharge. We created multivariable models of 7-month post-ACS physical and mental health status using data only from the 1,660 survey respondents. Then, using the Heckman method, we modeled survey non-response and incorporated this into our initial models to assess and correct for potential bias. We used logistic and ordinary least squares regression to estimate the multivariable selection models.We found that our model of 7-month mental health status was biased due to survey non-response, while the model for physical health status was not. A history of alcohol or substance abuse was no longer significantly associated with mental health status after controlling for bias due to non-response. Furthermore, the magnitude of the parameter estimates for several of the other predictor variables in the MCS model changed after accounting for bias due to survey non-response.Recognition and correction of bias due to survey non-response changed the factors that we concluded were associated with HRQL seven months following hospital admission for ACS as well as the magnitude of some associations. We conclude that the Heckman two-step method may be a valuable tool in the assessment and correction of selection bias in clinical studies of HRQL. The potential impact of missing survey responses is often ignored in health-related quality of life (HRQL) studies ,2. MissiNewer statistical techniques have been developed to assess and correct for bias resulting from missing HRQL surveys ,3. One tThe objective of this study was to demonstrate the use of the Heckman two-step method to assess and correct for bias due to missing HRQL surveys. To accomplish this goal, we evaluated HRQL outcomes in a cohort of patients with acute coronary syndromes (ACS).We analyzed data from the VA Access To Cardiology study, which was a multi-center prospective cohort study of 2,733 veterans with a confirmed diagnosis of acute coronary syndromes (ACS), including either acute myocardial infarction or unstable angina . BaselinThe outcome variables were the Physical Component Summary (PCS) and Mental Component Summary (MCS) scores from the 7-month SF-36 health status survey. The PCS and MCS scores reflect a patient's overall physical and mental health status, respectively ,11. The Baseline characteristics of the patients who did and did not complete an SF-36 were compared using analysis of variance for continuous variables and chi-square for categorical variables Table . Then, aThere are two steps in the Heckman method. The first step is the development of a selection equation (i.e. a model of factors associated with survey non-response). This step includes derivation of a variable from the selection equation called the Inverse Mills Ratio (IMR). The second step of the Heckman method is the insertion of the IMR variable into the initial regression models  from a given study in order to assess for, and attempt to control for, selection bias.The first step in the Heckman method is to create the selection model, which estimates whether or not the quality of life survey was completed. The Heckman selection equation is usually estimated using a probit estimator ,21. The Once the Heckman selection equation is estimated, the residuals (error term) from this equation are used to form a new variable called the Inverse Mills Ratio (IMR). The formula to create the IMR variable depends on the distributional assumption in the outcome equation. In most HRQL applications, the quality of life score is the outcome of interest and is usually estimated using multivariable linear regression. In this case, the distributional assumption of the error term is the standard normal distribution, so that the ratio of the standard normal probability density function (pdf) and cumulative density function (cdf) applied to the residuals for each individual in the data set is created. The ratio of pdf/cdf is the IMR.Each individual in the study sample receives an individual value of the IMR based on the residual observed for that individual in the selection equation. In this study, the value of the IMR for each individual represented the predicted probability that they completed the 7-month SF-36 survey. It is important to note that the IMR is a function not only of observed or measured variables that are included in the selection equation, but also of unobserved or unmeasured variables. These are captured through the error term or residual in the selection equation, and included through the non-linear function used to estimate the IMR. As a result, adding the IMR into the outcome equation introduces a term that attempts to capture both observed and unobserved variables that affect selection, or non-response.We estimated the Heckman model using the maximum likelihood estimation method in Stata version 8.0. In this approach, the outcome and selection models are estimated jointly, which can result in slightly different selection models for different outcomes, in this case the PCS and MCS scores from the SF-36. However, for clarity of presentation of the Heckman process, we present only one table of selection equation results (the probit estimation of the probability of returning the SF36), assuming that the patient survived to the 7-month survey point.The second step of the Heckman method is to include the IMR as a separate predictor variable in the initial regression models. In this study, the IMR variable derived from our Heckman selection model was inserted into the initial PCS and MCS models. Once this variable is inserted, two factors can be evaluated to help determine whether there is significant bias from missing responses in the initial models. First, one can examine the significance of the IMR variable itself. If significant, it suggests there was significant bias in the initial model. However, one potential limitation of the Heckman method is that if the Heckman selection model is not well-specified, and the variables in the selection model do not predict response/non-response well, the IMR may be weaker than expected and the Heckman method may have limited power to detect bias. Therefore, a second factor to examine following the addition of the IMR variable into the initial outcome models is whether or not there have been significant changes in any of the parameter estimates of the other predictor variables in the model. While somewhat arbitrary, changes in parameter estimates of >10% may indicate that these estimates were biased due to missing surveys. Where possible, one should apply clinical judgment about whether changes in parameter estimates are 'biologically important' .With these factors taken together, the insertion of the IMR variable into the initial risk models allows the assessment of whether or not there was bias in the initial models, and suggests which initial predictors may have been most associated with this bias. Furthermore, by including the effect of unmeasured as well as measured variables from the selection equation, bias due to selection is controlled.Compared to patients that completed the 7-month SF-36 survey, patients who did not respond to the survey were older, more likely to be current smokers and more likely to have a history of alcohol or substance abuse, dementia, stroke, or depression Table . FurtherThe initial multivariable PCS and MCS models not accounting for potential selection bias from missing HRQL surveys are presented in Tables Variables significantly associated with better 7-month physical health status included ST-segment elevation MI on electrocardiogram and coronary revascularization during the index ACS hospital admission. Variables significantly associated with worse 7-month physical health status included older age, history of prior CABG surgery, chronic heart failure, arthritis, COPD, stroke, depression, diabetes, and elevated serum creatinine during index ACS admission.Variables significantly associated with better 7-month mental health status included older age and ST-segment elevation MI on electrocardiogram. Variables significantly associated with worse 7-month mental health status included a history of prior MI, alcohol and/or substance abuse, COPD, and arthritis.The Heckman selection model (modeling response to the SF-36) is presented in Table The final multivariable models for PCS and MCS (after addition of the IMR variables from the Heckman selection model) are presented in Tables By contrast, when the IMR variable was inserted into the initial MCS model, we found evidence of selection bias. In this case, the parameter estimate for history of alcohol or substance abuse changed from significant to insignificant with the introduction of the IMR variable. Therefore, it appears that a history of alcohol or substance abuse was associated with lower likelihood of responding to the survey, but not directly associated with mental health status. In addition, a number of parameter estimates changed quantitatively, with larger changes than those observed in the PCS findings. Finally, the coefficient on the IMR variable itself was significant, and was negatively associated with MCS, implying that unobserved variables in the selection equation appear to be associated with worse MCS scores.The goal of this study was to demonstrate the use of the Heckman two-step method to assess and correct for bias due to missing HRQL surveys in a clinical study of acute coronary syndrome patients. We created initial multivariable models of 7-month post-ACS physical and mental health status using data only from survey respondents. Then, using the Heckman method, we modeled survey non-response, derived an Inverse Mills Ratio variable for each patient that captured the likelihood of survey response, and incorporated this variable into our initial models to assess and correct for potential bias from survey non-response.We found that our initial model of 7-month physical health status was not biased due to survey non-response. In contrast, our initial model of 7-month mental health status was biased. After controlling for bias due to non-response, a history of alcohol or substance abuse was no longer associated with mental health status. Furthermore, the magnitude of the parameter estimates for several of the other predictor variables in the MCS model changed after accounting for bias due to survey non-response.Given these results, biased parameter estimates of the association between these variables and mental health status would have been reported if we had used the standard approach to evaluating the predictors of HRQL outcomes in this population. Furthermore, we might have concluded that alcohol/substance abuse was significantly associated with mental health status outcomes following ACS, and may therefore be an important target for interventions to improve post-ACS HRQL . While alcohol/substance abuse may be important for other reasons, it would have been incorrect to conclude that it was associated with HRQL in our study population. Rather, it was a marker for survey non-response. This analysis demonstrates the utility of the Heckman method in its application for assessing and correcting survey response bias in clinical studies of HRQL.Health-related quality of life data are usually not missing at random, and failure to account for missing HRQL assessments can bias estimates of associations and may lead to inappropriate conclusions about the determinants of HRQL outcomes -3. OftenThere are other statistical techniques, or approaches, to assess and correct for bias resulting from missing HRQL surveys, including index function models, propensity scores, instrumental variables, and multiple imputation methods -4,23. ThPropensity score approaches can be analogous to the Heckman method in that a multivariable model of survey non-response is developed and the probability of survey non-response is used to stratify the study population and/or the propensity score is used as an independent variable in the primary HRQL models. In other words, propensity scores are similar to the Heckman method in that the predicted probability of non-response is used as the basis for assessing the impact of missing data and controlling for it . Unlike Instrumental variable approaches are also used to address similar questions to those addressed by the Heckman method. In the full instrumental variable approach, a single exogenous variable  is used to stratify the full sample, removing the effect of the correlated error terms that lead to biased estimates . An instFinally, multiple imputation methods can be employed to address missing HRQL data . In contThe Heckman method has several limitations. First, the selection equation must have at least one variable that is associated with survey response but not the outcome of the study (i.e. HRQL). In some clinical applications, the inability to identify such a variable may make it difficult to use the full Heckman method to control bias. However, it is still possible to use the first step of estimating a selection equation to assess the degree to which selection bias may affect the parameter estimates in an outcome equation. If there are variables that are significant in both the selection equation and the outcomes equation, it is likely that there is bias due to selection effects in the outcome equation. Acknowledging this and commenting on the likely magnitude of effect may provide helpful guidance to clinicians and other researchers. Another limitation of the Heckman method is that this technique depends heavily on the quality of the data available for the selection equation. If the amount of variance explained is relatively low, then there is a possibility that selection bias in the outcomes equation may not be detected. In other words, the Heckman method can be under-powered for the detection of bias in some cases.Finally, the Heckman method is very sensitive to how the model is specified; in other words, omitting variables that are associated with either non-response or with the outcome of interest  can lead to inaccurate findings and biased estimates of the parameters in the final models. Careful attention to specifying the models, and ensuring that model specification follows what is known in the literature to be associated with the outcomes of interest is essential .This study demonstrated the use of the Heckman two-step method to assess and control for bias from missing HRQL surveys in a clinical study. We found that our mental health status model was significantly biased due to missing HRQL assessments. Recognition and correction of this bias changed the parameter estimates of association and the factors that we concluded were associated with HRQL seven months following hospital admission for an acute coronary syndrome. We conclude that the Heckman two-step method may be a valuable tool in the assessment and correction of selection bias in clinical studies of HRQL.AES conceived of the study, participated in design and coordination, conducted analyses, and drafted the manuscript; MEP conducted analyses and contributed to the manuscript; DJM and JAS reviewed and contributed to the manuscript; JSR participated in the design and coordination of the study and participated in the drafting and revision of the manuscript. All authors read and approved the final manuscript.


Amidst the hoopla over the exact number of genes we have in our genome—more than a fruitfly, fewer than a rice plant—a more fundamental genetic truth has often been obscured. The expression of 20,000–30,000 genes is under the control of an uncounted host of non-coding sequences, which bind transcription factors and thereby regulate when and where genes are expressed. Unlike coding sequences, whose signatures are easy to spot, the characteristic features of non-coding regulatory elements are largely unknown, making their discovery by simple sequence analysis difficult. In this issue, Greg Elgar and colleagues attack this problem by comparing the non-coding sequences of the human and the pufferfish.Since the last common ancestor of these two species existed 450 million years ago, the authors reasoned that non-coding sequences conserved between them are likely to be fundamental to vertebrate development. Through sequence alignment with increasingly strict criteria, they identified 1,373 highly conserved non-coding elements (CNEs), with an average length of about 200 base pairs. The average sequence match is 84%: not perfect, but much higher than for coding regions shared by humans and pufferfish. A quick check showed that virtually all the sequences also occurred in rodents, chickens, and zebrafish, but not in the nematode, fruitfly, or even the sea squirt, a primitive non-vertebrate chordate.CNEs are not spread uniformly throughout the genome. Instead, they are bunched together in fewer than 200 clusters, most of them in close proximity to genes implicated in transcriptional regulation or development. This clustering of CNEs suggests they may not only attract transcription factors, but may also influence the local topology of the DNA, thereby controlling access to their associated gene. Several clusters also appear in regions without any known genes—the identification of these clusters might lead to the discovery of new developmentally significant genes.While “in silico” discoveries such as this can be the jumping-off point for whole new areas of investigation, their validity must be tested “in aqua,” in the wet biology of real organisms. For this Elgar and colleagues chose the zebrafish, because its transparent embryo is ideal for observing developmental events. They injected individual CNEs into embryos, along with a green fluorescent protein (GFP) reporter. By day two of development, 23 out of 25 CNEs injected had upregulated GFP expression, indicating interaction of these sequences with endogenous transcription factors. Different CNEs caused different regional patterns of expression, in keeping with their presumed roles in distinct developmental processes.The discovery of these developmentally important sequences opens several avenues of new research. For example, analyzing the sequence and location of these CNEs may help point the way to other non-coding elements that remain undiscovered. It is also likely that mutations in these critical sequences cause human diseases. Studying how such mutations drive development astray may lead to better understanding not only of these diseases, which are likely to be rare, but also of normal human development.


Analysis of heart rate variation (HRV) has become a popular noninvasive tool for assessing the activities of the autonomic nervous system (ANS). HRV analysis is based on the concept that fast fluctuations may specifically reflect changes of sympathetic and vagal activity. It shows that the structure generating the signal is not simply linear, but also involves nonlinear contributions. These signals are essentially non-stationary; may contain indicators of current disease, or even warnings about impending diseases. The indicators may be present at all times or may occur at random in the time scale. However, to study and pinpoint abnormalities in voluminous data collected over several hours is strenuous and time consuming.This paper presents the spatial filling index and time-frequency analysis of heart rate variability signal for disease identification. Renyi's entropy is evaluated for the signal in the Wigner-Ville and Continuous Wavelet Transformation (CWT) domain.This Renyi's entropy gives lower 'p' value for scalogram than Wigner-Ville distribution and also, the contours of scalogram visually show the features of the diseases. And in the time-frequency analysis, the Renyi's entropy gives better result for scalogram than the Wigner-Ville distribution.Spatial filling index and Renyi's entropy has distinct regions for various diseases with an accuracy of more than 95%. Bio-signals are essentially non-stationary signals; they display a fractal like self-similarity. They may contain indicators of current disease, or even warnings about impending diseases. The indicators may be present at all times or may occur at random – in the time scale. However, to (study and) pinpoint anomalies in voluminous data collected over several hours is strenuous and time consuming. Therefore, computer based analytical tools for in-depth study and classification of data over day long intervals can be very useful in diagnostics.Electrocardiography deals with the electrical activity of the heart. Monitored by placing sensors at defined positions on chest and limb extremities of the subject, electrocardiogram (ECG) is a record of the origin and propagation of the electric action potential through cardiac muscle. It is considered a representative signal of cardiac physiology, useful in diagnosing cardiac disorders. The state of cardiac health is generally reflected in the shape of ECG waveform and heart rate. It may contain important pointers to the nature of diseases afflicting the heart. However, bio-signals being non-stationary signals, this reflection may occur at random in the time scale. Therefore, for effective diagnostics, the study of ECG pattern and heart rate variability signal (instantaneous heart rate against time axis) may have to be carried out over several hours. HRV is a useful signal for understanding the status of the autonomic nervous system (ANS).The interest in the analysis of heart rate variability (HRV), that is, the fluctuations of the heart beating in time, is not new. And much progress was achieved in this field with the advent of cheap and massive computational power, which provoked many recent advances.HRV is a non-invasive measurement of cardiovascular autonomic regulation. Specifically, HRV is a measurement of the interaction between sympathetic and parasympathetic activity in autonomic functioning. There are two main approaches for analysis: time domain analysis of HRV [for standard deviation of normal to normal intervals (SDNN)]; and frequency domain analysis [for power spectrum density (PSD)]. The latter provides high frequency (parasympathetic activity) and low frequency (sympathetic and parasympathetic activity) and total power  values -3. RecenThe cardiovascular system is a complex system that includes heart and vessels. ECG and HRV are two methods for study it. Hence, many attempts have been made to analyze these signals and extract information about the cardiovascular system. Most of the methods used are linear and it has been recognized that nonlinear methods may be more suitable for analyzing signals that originate from complex nonlinear living systems . Recent A complex system like cardiovascular system can not be linear in nature and by considering it as a nonlinear system can lead to better understanding of the system dynamics. Recent studies have also stressed the importance of nonlinear techniques to study HRV in both health and disease. The progress made in the field using measures of chaos has attracted scientific community applying these tools in studying physiological systems, and HRV is no exception. There have been several methods of estimating invariants from nonlinear dynamical systems reported in the literature. Recently, Fell et al and Radhakrishna et al have tried the nonlinear analysis of ECG and HRV signals respectively ,14. Alsopremature ventricular contraction (PVC), Complete Heart Block (CHB), Sick Sinus Syndrome (SSS), Congestive heart failure (CHF), Ishemic/Dilated cardiomyoapathy (ISCDIL), Atrial Fibrillation (AF), and ventricular fibrillation (VF).ECG data for the analysis was obtained from MIT-BIH arrhythmia database . Prior tr-r seconds) and the heart rate (beats per minute) is given as:Each dataset is taken consists of more than10,000 samples and the sampling frequency of the data is 360 Hz. The interval between two successive QRS complexes is defined as the RR interval (tr-r     (1)HR = 60/tX(k) in phase space. Then the dynamical behavior of the signal is reconstructed by succession of these points X(k) in the phase space. Phase space reconstructions are based on the analysis of dynamic systems by delay maps. The vectors X(k) in the multidimensional phase space are constructed by time delayed values of the time series, which determine the coordinates of the phase space plot.Let the signal be represented by the coordinates of a point X(k) = {x (k), x((k + τ), ...,x (k + (E-1)τ)} for k = 1,2,...,N - (E - 1)τ     (1)X(k) is one point of the trajectory in the phase space at time k, x(k + τ) are the coordinates in the phase space corresponding to the time delayed values of the time series, τ is the time delay between the points of the time series considered and E is the embedding dimension, which is the number of coordinates of the phase space plot. The attributes of the reconstructed phase space plot depend on the choice of value of τ. One way to choose τ is to take it as the time it takes the autocorrelation function of the data to decay to 1/e [τ is the best choice for ECG signals and 5 for HRV signals [where y to 1/e . Anothery to 1/e , which a signals .x (1), x(2), ..., x (N), a matrix AE is obtained asFrom the given signal E is the number of dimensions and M is related to N by the equation:where M = N - (E -1)τ     (3)A against column 1 (for the case E = 2), the phase space plot for two dimensions is obtained.By plotting column 2 of matrix A3 represent a phase space plot in three dimensions. Now, a normalized matrix BE is obtained by dividing each element of AE by xmax whereSimilarly, the first three columns of matrix xmax = max |x(k)| 1 ≤ k ≤ N     (5)B2 (in two dimensions) is hence represented asThe matrix R × R |R ∈ Real, 2/R ∈ Integer}. Then the number of grids in the normalized phase space is n = 2/R. A matrix C is now obtained with its elements c(i,j) equal to the number of phase space points falling in a grid g(i,j). The matrix C is called the phase space matrix and its elements are divided by m, whereIn two dimensions, the phase space plot corresponding to the normalized matrix spans from -1 to +1 on either axis. The phase space area is now divided into small square areas of size {P(i,j), the probability of a phase space point falling in a grid g(i,j). A matrix Q is now formed by squaring each element of P to get q(i,j) as the elements of Q. The sum of elements of matrix Q is calculated asThis division yields η is defined as:The spatial filling index η = s / n2     (9)η is used to quantify the degree of variability in the test signals.Now the value of There are three common approaches to generating the time-frequency (TF) plots. These are the short Time Fourier Transform; the Wigner-Ville based bilinear distributions and the Continuous Wavelet Transform. In this investigation the latter two were used.The Wigner-Ville distribution (WVD) is defined as:z(t) is the analytic signal and h(τ) is a window function. The results where obtained using a Hamming window. This window attenuates the interferences oscillating perpendicularly to the frequency axis. The WVD satisfies a large number of desirable mathematical properties. In particular, the WVD is always real-valued; it preserves time and frequency shifts and satisfies the marginal properties. Moreover, the WVD conserves the Energy of the signal. We obtain the Energy (Ex) by integrating the WVD of z all over the time frequency plane:where With the Energy conservation property the WVD can be interpreted in terms of probability density: expression (10) is the Fourier transform of an acceptable form of characteristic function for the distribution of energy. Therefore, the WVD can be used to obtain the information content of a signal; this thought is further extended in Section 4.3.'wavelet' implies a small wave of finite duration and finite energy, which is correlated with the signal to obtain the wavelet coefficients [mother wavelet, and the coefficients are evaluated for the entire range of dilation and translation factors [wavelet is shifted (translated) continually along the time scale for evaluating the set of coefficients at all instants of time. In the next phase, the wavelet is dilated for a different width – also normalized to contain the same amount of energy as the mother wavelet – and the process is repeated for the entire signal. The wavelet coefficients are real numbers usually shown by the intensity of a chosen color, against a two dimensional plane with y-axis representing the dilation  of the wavelet, and the x-axis, its translation . Thus the wavelet transform plot  can be seen as a color pattern against a two dimensional plane. In the CWT the wavelet coefficients are evaluated for infinitesimally small shifts of translation as well as scale factors. That is, the color intensity of each pixel in the scalogram is separately evaluated, and the resulting pattern contains information about the size and location of the 'event' occurring in the time domain [scalogram representation of even high frequency, low energy 'events' occurring in the time scale are more conspicuous than in the Fourier Transform. Thus the color patterns in the scalogram can be useful in highlighting the abnormalities specific to different types of disease. MATLAB version 6.1 is used to plot the various scalogram plots.A ficients . The ref factors . Initiale domain ,27. SincΨa,b(t), the coefficients are evaluated using Eq. (12):For a given wavelet  is energy normalizing coefficient, and Ψa,b(t) is the mother wavelet; a → scale factor ; b → translation factor.The wavelet, defined as ,...small wave of finite duration and finite energy...' has also zero mean value, Ex) is obtained by integrating over all scale and translation factors:Just like the WVD, the CWT representation preserves also the energy of the signal. The total energy - is a continuous and strictly monotonic function subclass of Kolmogorov-Nagumo functions. To satisfy the constraints of an information measurewhere I(pk)- any information measureSimplifying the above relation, we haveα = 3) is calculated from the WVD time-frequency representations as follows:The third order Renyi's entropy  is expressed in bits: If one elementary signal yields zero bit of information (20), then two well separated elementary singles will yield one bit of information (21), four well speared elementary singles will yield two bits of information (22), and so on. It shows that for different cardiac signals the Renyi's entropy in the time-frequency domain is different.The result produced by this measure • εij is a matrix of random disturbances.• The model posits that the columns of y are a constant plus a random disturbance. You want to know if the constants are all the same.η) defined in Equation (9). For WVD the parameter is the third order Renyi's entropy  defined in Equation (14). For CWT the parameter is also the third order Renyi's entropy  defined in Equation (15). For each dataset these three parameters (,  and η) where calculated. Table N-1 where N is the sequence length) of these parameters for each of the data classes. The p-value, also shown in Table The result section compares the three different analyzing methods. Each of these methods results in a single parameter for each of the datasets. For Phase Space the parameter used for comparison is the spatial filling index  seems to provide very useful information for detection of abnormalities in the condition of the heart that is not available by conventional means. In this paper, a phase space and the time-frequency analysis of these cardiac signals using spatial filling index and Renyi's entropy has been proposed for detecting cardiac dysfunction. The ANOVA test was used to compare the different analyzing methods. The Renyi's entropy gives better result for the scalogram than the Wigner-Ville distribution. The evaluation of the proposed technique on a larger data set will improve the efficacy of the technique. It is left as future work to compare the different methods with more sophisticated statistical methods, such as post hoc comparisons. It is hoped that the graphical representation along with its corresponding analytical index and Renyi's entropy proposed here will find potential applications in computer analysis of cardiac patients' status in intensive care units.