Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> BACKGROUND AND OBJECTIVE ::: To describe the association between values for a proportion and the intraclass correlation coefficient (ICC). ::: ::: ::: METHODS ::: Analysis of data obtained from the General Practice Research Database (GPRD) for variation between United Kingdom general practices and results from a Health Technology Assessment (HTA) review for a range of outcomes in community and health services settings. ::: ::: ::: RESULTS ::: There were 188 ICCs from the GPRD, the median prevalence was 13.1% (interquartile range IQR 3.5 to 28.4%) and median ICC 0.051 (IQR 0.011 to 0.094). There were 136 ICCs from the HTA review, with median prevalence 6.5% (IQR 0.4 to 20.7%) and median ICC 0.006 (IQR 0.0003 to 0.036). There was a linear association of log ICC with log prevalence in both datasets (GPRD, regression coefficient 0.61, 95% confidence interval 0.53 to 0.69, P < 0.001; HTA, 0.91, 0.81 to 1.01, P < 0.001). When the prevalence was 1% the predicted ICC was 0.008 from the GPRD or 0.002 from the HTA, but when the prevalence was 40% the predicted ICC was 0.075 (GPRD) or 0.046 (HTA). ::: ::: ::: CONCLUSION ::: The prevalence of an outcome may be used to make an informed assumption about the magnitude of the intraclass correlation coefficient. <s> BIB001 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> The intra-cluster correlation coefficient (ICC) of the primary outcome plays a key role in the design and analysis of cluster randomized trials (CRTs), but the precise definition of this parameter is somewhat elusive, especially in the context of non-normally distributed outcomes. In this paper, we provide a unified treatment of ICC as used in CRTs. We present a general definition of the ICC that may be expressed in different ways depending on the modelling approach used to describe the data, illustrating how this general definition is applied to continuous and dichotomous outcomes. Greater complexity arises for dichotomous outcomes; in particular, the usual definition of the ICC cannot be related directly to the parameters of the logistic-normal model that is commonly used for dichotomous outcomes. We show how the definition of the ICC is different when covariates are introduced. Finally, we use our framework and definition of the ICC to draw out implications for those interpreting and choosing values of the ICC when planning CRTs. Copyright (c) 2009 The Authors. Journal compilation (c) 2009 International Statistical Institute. <s> BIB002 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> OBJECTIVES ::: To evaluate oral poliovirus vaccine (OPV) coverage of the November 2009 round in five Northern Nigeria states with ongoing wild poliovirus transmission using clustered lot quality assurance sampling (CLQAS). ::: ::: ::: METHODS ::: We selected four local government areas in each pre-selected state and sampled six clusters of 10 children in each Local Government Area, defined as the lot area. We used three decision thresholds to classify OPV coverage: 75-90%, 55-70% and 35-50%. A full lot was completed, but we also assessed in retrospect the potential time-saving benefits of stopping sampling when a lot had been classified. ::: ::: ::: RESULTS ::: We accepted two local government areas (LGAs) with vaccination coverage above 75%. Of the remaining 18 rejected LGAs, 11 also failed to reach 70% coverage, of which four also failed to reach 50%. The average time taken to complete a lot was 10 h. By stopping sampling when a decision was reached, we could have classified lots in 5.3, 7.7 and 7.3 h on average at the 90%, 70% and 50% coverage targets, respectively. ::: ::: ::: CONCLUSIONS ::: Clustered lot quality assurance sampling was feasible and useful to estimate OPV coverage in Northern Nigeria. The multi-threshold approach provided useful information on the variation of IPD vaccination coverage. CLQAS is a very timely tool, allowing corrective actions to be directly taken in insufficiently covered areas. <s> BIB003 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> BACKGROUND AND PURPOSE ::: Power and sample size calculations for cluster randomized trials require prediction of the degree of correlation that will be realized among outcomes of participants in the same cluster. This correlation is typically quantified as the intraclass correlation coefficient (ICC), defined as the Pearson correlation between two members of the same cluster or proportion of the total variance attributable to variance between clusters. It is widely known but perhaps not fully appreciated that for binary outcomes, the ICC is a function of outcome prevalence. Hence, the ICC and the outcome prevalence are intrinsically related, making the ICC poorly generalizable across study conditions and between studies with different outcome prevalences. ::: ::: ::: METHODS ::: We use a simple parametrization of the ICC that aims to isolate that part of the ICC that measures dependence among responses within a cluster from the outcome prevalence. We incorporate this parametrization into sample size calculations for cluster randomized trials and compare our method to the traditional approach using the ICC. ::: ::: ::: RESULTS ::: Our dependence parameter, R, may be less influenced by outcome prevalence and has an intuitive meaning that facilitates interpretation. Estimates of R from previous studies can be obtained using simple statistics. Comparison of methods showed that the traditional ICC approach to sample size determination tends to overpower studies under many scenarios, calling for more clusters than truly required. ::: ::: ::: LIMITATIONS ::: The methods are developed for equal-sized clusters, whereas cluster size may vary in practice. ::: ::: ::: CONCLUSIONS ::: The dependence parameter R is an alternative measure of dependence among binary outcomes in cluster randomized trials that has a number of advantages over the ICC. <s> BIB004 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> Estimation of vaccination coverage at the local level is essential to identify communities that may require additional support. Cluster surveys can be used in resource-poor settings, when population figures are inaccurate. To be feasible, cluster samples need to be small, without losing robustness of results. The clustered LQAS (CLQAS) approach has been proposed as an alternative, as smaller sample sizes are required. We explored (i) the efficiency of cluster surveys of decreasing sample size through bootstrapping analysis and (ii) the performance of CLQAS under three alternative sampling plans to classify local VC, using data from a survey carried out in Mali after mass vaccination against meningococcal meningitis group A. VC estimates provided by a 10 × 15 cluster survey design were reasonably robust. We used them to classify health areas in three categories and guide mop-up activities: i) health areas not requiring supplemental activities; ii) health areas requiring additional vaccination; iii) health areas requiring further evaluation. As sample size decreased (from 10 × 15 to 10 × 3), standard error of VC and ICC estimates were increasingly unstable. Results of CLQAS simulations were not accurate for most health areas, with an overall risk of misclassification greater than 0.25 in one health area out of three. It was greater than 0.50 in one health area out of two under two of the three sampling plans. Small sample cluster surveys (10 × 15) are acceptably robust for classification of VC at local level. We do not recommend the CLQAS method as currently formulated for evaluating vaccination programmes. <s> BIB005 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Evaluating the designs <s> Lot quality assurance sampling (LQAS) has a long history of applications in industrial quality control. LQAS is frequently used for rapid surveillance in global health settings, with areas classified as poor or acceptable performance on the basis of the binary classification of an indicator. Historically, LQAS surveys have relied on simple random samples from the population; however, implementing two-stage cluster designs for surveillance sampling is often more cost-effective than simple random sampling. By applying survey sampling results to the binary classification procedure, we develop a simple and flexible nonparametric procedure to incorporate clustering effects into the LQAS sample design to appropriately inflate the sample size, accommodating finite numbers of clusters in the population when relevant. We use this framework to then discuss principled selection of survey design parameters in longitudinal surveillance programs. We apply this framework to design surveys to detect rises in malnutrition prevalence in nutrition surveillance programs in Kenya and South Sudan, accounting for clustering within villages. By combining historical information with data from previous surveys, we design surveys to detect spikes in the childhood malnutrition rate. <s> BIB006
There are two primary differences between the methods above: the distributional assumptions (binomial-scaled, beta, and quasi-binomial models) and clustering parameterization using σ versus ρ (that is, the difference between treating σ or ρ as known and the same for p l and p u ). These differences are summarized in Table 1 . We compare the impacts of these differences across the Pezzoli, Hedt, and Hund methods and determine situations in which the choice of method results in a substantively different design. As example designs, we use three couplets for p l and p u : 55-70%; 75-90%; and 90-95%. The first two couplets are from the 6 × 10 twostage cluster designs for vaccination coverage proposed in BIB003 . The last couplet is consistent with the 5-10% couplet used in the 33 × 6 or 67 × 3 malnutrition LQAS surveys. We parameterize these designs according to Eq 1 and specify X as the number of successes (vaccinated children or children with no evidence of malnutrition) out of n. As noted above, both the Pezzoli and Hund models are subject to some rounding error; we use nearest integer rounding for all calculations. All analyses have been conducted using R 3.1.1  , and code for reproducing all figures and tables, along with the R package from BIB006 , is available at https://github. com/lbhund/ClusterLQAS_PLOS. Comparing the distributional assumptions. First, we conduct a simulation study to evaluate whether the choice of distributional model (binomial-scaled, beta, and quasi-binomial) makes a substantive difference in the classification precision of the cluster LQAS designs. For the 2 p l and p u threshold couplets from BIB003 , we use the 6 × 10 design with decision rules d = 38 and 50; for the 90-95% couplet, we use the 33 × 6 design with d = 13. We fix σ = .1 for all models (we repeat the analysis fixing σ = .05; results are similar and are not presented). That is, we select the clustering parameters at p l and p u such that σ = .1 for all models and thresholds (deviating from the specific methods described above) to isolate the impact of the probability distribution choice. For each p l , p u couplet, we generate 10,000 draws from each of the three distributions fixing m, k, d, and σ = .1 and examine how α and β change with the distributional assumption. For the quasi-binomial and betabinomial models, we calculate ρ using Eq 2. We then repeat the simulation fixing ρ = .1 (rather than fixing σ = .1) and calculate σ for the binomial-scaled model using Eq 2. Fixing σ versus ρ. Historically, survey sample size calculations rely on the design effect and thus assume ρ is known and constant across levels of coverage p BIB002 BIB004 . The Hund and Hedt methods also model clustering assuming ρ is known and fixed as a function of p. The Pezzoli method parameterizes clustering differently, assuming σ is fixed as a function of p. Fixing ρ results in less cluster-level variance (smaller σ) as p moves away from .5. Fixing σ results in the same variance of p j for all levels of p (and larger ρ as moves away from .5). Intuitively, variability in p j may decrease as p moves away from .5 (e.g. less variability in the coverage of a rare event), though there is also evidence that ρ decreases as p moves away from .5 BIB004 BIB001 . It is thus not immediately clear as to whether fixing σ, ρ, or neither is appropriate. To examine design differences according to the clustering parameterization, we compare the Pezzoli, Hund, and Hedt designs when σ = .1 and when ρ = .1. That is, we first calculate ρ l and ρ u such that σ = .1 at p l and p u , respectively, and compare fixing both ρ l and ρ u for the Hedt and Hund designs to the Pezzoli design with σ = .1. We then repeat this exercise, calculating σ l and σ u such that ρ = .1 at p l and p u , respectively, and compare fixing both σ l and σ u for the Pezzoli design to the Hedt and Hund designs with ρ = .1. We again evaluate the three couplets 55-70%, 75-90%, and 90-95% and specify α = β = .1 and m = 10. We then examine reported estimates of p, σ, and ρ from BIB003 and BIB005 to assess how estimates of ρ and σ vary as a function of estimated vaccination coverage in practice. Before delving into this data example, we note that the standard deviation of the estimatedp j s, s ¼  From Eq 6, it is clear that using s to estimate σ is only appropriate when m is large (which is seldom the case in cluster LQAS surveys). We estimateŝ using reportedp and standard errors s= ffiffi ffi k p for 20 different 6 × 10 surveys in BIB003 . Minetti et. al BIB005 reportp, s, and intraclass correlation estimatesr from 41 different 10 × 15 vaccination coverage surveys. Rather than using Eq 6, we estimateŝ ¼rpð1 ÀpÞ for the data in BIB005 . For each dataset, we estimate the relationship betweenr andp and betweenŝ andp using a nonparametric loess smooth  based on the loess function in R. We use the default parameters in the loess function, using a weighted least squares fit based on 75% of the data for the local smoothing. For each data example, we then estimate operating characteristic (OC) curves for the 6 × 10 design with d = 50 for the 75-90% couplet using the binomial-scaled distribution but varying the values of ρ and σ. First, we assume ρ is fixed at the value corresponding to the mean ofr across all SAs; next, we allow ρ to vary as a function of p, estimating ρ from the nonparametric loess smooth forr. We then repeat for σ, resulting in 4 OC curves for each data example.