Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> BACKGROUND AND OBJECTIVE ::: To describe the association between values for a proportion and the intraclass correlation coefficient (ICC). ::: ::: ::: METHODS ::: Analysis of data obtained from the General Practice Research Database (GPRD) for variation between United Kingdom general practices and results from a Health Technology Assessment (HTA) review for a range of outcomes in community and health services settings. ::: ::: ::: RESULTS ::: There were 188 ICCs from the GPRD, the median prevalence was 13.1% (interquartile range IQR 3.5 to 28.4%) and median ICC 0.051 (IQR 0.011 to 0.094). There were 136 ICCs from the HTA review, with median prevalence 6.5% (IQR 0.4 to 20.7%) and median ICC 0.006 (IQR 0.0003 to 0.036). There was a linear association of log ICC with log prevalence in both datasets (GPRD, regression coefficient 0.61, 95% confidence interval 0.53 to 0.69, P < 0.001; HTA, 0.91, 0.81 to 1.01, P < 0.001). When the prevalence was 1% the predicted ICC was 0.008 from the GPRD or 0.002 from the HTA, but when the prevalence was 40% the predicted ICC was 0.075 (GPRD) or 0.046 (HTA). ::: ::: ::: CONCLUSION ::: The prevalence of an outcome may be used to make an informed assumption about the magnitude of the intraclass correlation coefficient. <s> BIB001 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> Results Hypothesis tests showed GAM as <10% for the 33 � 6 design and GAM as 510% for the 67 � 3 and Sequential designs. Point estimates for the 33 � 6 and 67 � 3 designs were similar to those of the 30 � 30 cluster-survey for GAM (6.7%, CI ¼ 3.2–10.2%; 8.2%, CI ¼ 4.3–12.1%, 7.4%, CI ¼ 4.8–9.9%) and all other indicators. The CIs for the LQAS designs were only slightly wider than the CIs for the 30 � 30 cluster-survey; yet the LQAS designs required substantially less time to administer. Conclusions The LQAS designs provide statistically appropriate alternatives to the more timeconsuming 30 � 30 cluster-survey. However, additional field-testing is needed using independent samples rather than a nested study design. <s> BIB002 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> This article describes the development of a novel sequential sampling method for the surveillance of transmitted HIV drug resistance by cross-sectional survey. Two commonly used sequential sampling methods are described and their applicability to the problem of classifying the prevalence of transmitted HIV drug resistance investigated. Both methods are rejected due to insufficient savings in sample size and operational complexity. A novel method is proposed and this is tested using computer-based simulation. This method provides useful sample size savings and operational simplicity and could provide the basis for a rapid and reliable survey method for classifying the prevalence of transmitted HIV drug resistance in circumstances where monitoring HIV drug resistance is an important issue, but resources do not allow fullscale surveillance to be established. The method is currently being used in several such settings. <s> BIB003 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> The conventional method to collect data on the health, nutrition, and food security status of a population affected by an emergency is a 30 × 30 cluster survey. This sampling method can be time and resource intensive and, accordingly, may not be the most appropriate one when data are needed rapidly for decision making. In this study, we compare the precision, time and cost of the 30 × 30 cluster survey with two alternative sampling designs: a 33 × 6 cluster design (33 clusters, 6 observations per cluster) and a 67 × 3 cluster design (67 clusters, 3 observations per cluster). Data for each sampling design were collected concurrently in West Darfur, Sudan in September-October 2005 in an emergency setting. Results of the study show the 30 × 30 design to provide more precise results (i.e. narrower 95% confidence intervals) than the 33 × 6 and 67 × 3 design for most child-level indicators. Exceptions are indicators of immunization and vitamin A capsule supplementation coverage which show a high intra-cluster correlation. Although the 33 × 6 and 67 × 3 designs provide wider confidence intervals than the 30 × 30 design for child anthropometric indicators, the 33 × 6 and 67 × 3 designs provide the opportunity to conduct a LQAS hypothesis test to detect whether or not a critical threshold of global acute malnutrition prevalence has been exceeded, whereas the 30 × 30 design does not. For the household-level indicators tested in this study, the 67 × 3 design provides the most precise results. However, our results show that neither the 33 × 6 nor the 67 × 3 design are appropriate for assessing indicators of mortality. In this field application, data collection for the 33 × 6 and 67 × 3 designs required substantially less time and cost than that required for the 30 × 30 design. The findings of this study suggest the 33 × 6 and 67 × 3 designs can provide useful time- and resource-saving alternatives to the 30 × 30 method of data collection in emergency settings. <s> BIB004 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> Sampling theory facilitates development of economical, effective and rapid measurement of a population. While national policy maker value survey results measuring indicators representative of a large area (a country, state or province), measurement in smaller areas produces information useful for managers at the local level. It is often not possible to disaggregate a national survey to obtain local information if that was not the intent of the original survey design. Cluster sampling is typically used for national or large area surveys because sampling in clusters lowers the cost of a survey. Lot Quality Assurance Sampling (LQAS) is used to measure results at a local level, since it requires small random samples and produces results useful to local managers. However, current LQAS methodology requires all local areas (strata) be included in the survey in order to be aggregated to produce point estimates for the nation or state. In large countries it is not feasible to sample all strata for logistical and financial reasons. This paper resolves this problem by presenting Large Country (LC)-LQAS, a method with two concurrent objectives: 1) provide local managers with accurate local information to enable data driven decisions, and 2) provide central policy makers with the aggregate information they require. These are achieved by integrating cluster sampling with LQAS methodologies. Two examples of the implementation of LC-LQAS are provided, in an HIV/AIDS program in Kenya and a Malaria Booster Project in Nigeria. Classifications of local health units into performance categories and aggregate estimates of coverage, with associated confidence intervals, are provided for select indicators in order to demonstrate its use, analysis, and costs. This paper is written as a manual to support the use of LC-LQAS by others. <s> BIB005 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> Traditional lot quality assurance sampling (LQAS) methods require simple random sampling to guarantee valid results. However, cluster sampling has been proposed to reduce the number of random starting points. This study uses simulations to examine the classification error of two such designs, a 67×3 (67 clusters of three observations) and a 33×6 (33 clusters of six observations) sampling scheme to assess the prevalence of global acute malnutrition (GAM). Further, we explore the use of a 67×3 sequential sampling scheme for LQAS classification of GAM prevalence. Results indicate that, for independent clusters with moderate intracluster correlation for the GAM outcome, the three sampling designs maintain approximate validity for LQAS analysis. Sequential sampling can substantially reduce the average sample size that is required for data collection. The presence of intercluster correlation can impact dramatically the classification error that is associated with LQAS analysis. Copyright (c) 2009 Royal Statistical Society. <s> BIB006 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> The intra-cluster correlation coefficient (ICC) of the primary outcome plays a key role in the design and analysis of cluster randomized trials (CRTs), but the precise definition of this parameter is somewhat elusive, especially in the context of non-normally distributed outcomes. In this paper, we provide a unified treatment of ICC as used in CRTs. We present a general definition of the ICC that may be expressed in different ways depending on the modelling approach used to describe the data, illustrating how this general definition is applied to continuous and dichotomous outcomes. Greater complexity arises for dichotomous outcomes; in particular, the usual definition of the ICC cannot be related directly to the parameters of the logistic-normal model that is commonly used for dichotomous outcomes. We show how the definition of the ICC is different when covariates are introduced. Finally, we use our framework and definition of the ICC to draw out implications for those interpreting and choosing values of the ICC when planning CRTs. Copyright (c) 2009 The Authors. Journal compilation (c) 2009 International Statistical Institute. <s> BIB007 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> BACKGROUND ::: Large investments and increased global prioritization of malaria prevention and treatment have resulted in greater emphasis on programme monitoring and evaluation (M&E) in many countries. Many countries currently use large multistage cluster sample surveys to monitor malaria outcome indicators on a regional and national level. However, these surveys often mask local-level variability important to programme management. Lot Quality Assurance Sampling (LQAS) has played a valuable role for local-level programme M&E. If incorporated into these larger surveys, it would provide a comprehensive M&E plan at little, if any, extra cost. ::: ::: ::: METHODS ::: The Mozambique Ministry of Health conducted a Malaria Indicator Survey (MIS) in June and July 2007. We applied LQAS classification rules to the 345 sampled enumeration areas to demonstrate identifying high- and low-performing areas with respect to two malaria program indicators-'household possession of any bednet' and 'household possession of any insecticide-treated bednet (ITN)'. ::: ::: ::: RESULTS ::: As shown by the MIS, no province in Mozambique achieved the 70% coverage target for household possession of bednets or ITNs. By applying LQAS classification rules to the data, we identify 266 of the 345 enumeration areas as having bednet coverage severely below the 70% target. An additional 73 were identified with low ITN coverage. ::: ::: ::: CONCLUSIONS ::: This article demonstrates the feasibility of integrating LQAS into multistage cluster sampling surveys and using these results to support a comprehensive national, regional and local programme M&E system. Furthermore, in the recommendations we outlined how to integrate the Large Country-LQAS design into macro-surveys while still obtaining results available through current sampling practices. <s> BIB008 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> BACKGROUND AND PURPOSE ::: Power and sample size calculations for cluster randomized trials require prediction of the degree of correlation that will be realized among outcomes of participants in the same cluster. This correlation is typically quantified as the intraclass correlation coefficient (ICC), defined as the Pearson correlation between two members of the same cluster or proportion of the total variance attributable to variance between clusters. It is widely known but perhaps not fully appreciated that for binary outcomes, the ICC is a function of outcome prevalence. Hence, the ICC and the outcome prevalence are intrinsically related, making the ICC poorly generalizable across study conditions and between studies with different outcome prevalences. ::: ::: ::: METHODS ::: We use a simple parametrization of the ICC that aims to isolate that part of the ICC that measures dependence among responses within a cluster from the outcome prevalence. We incorporate this parametrization into sample size calculations for cluster randomized trials and compare our method to the traditional approach using the ICC. ::: ::: ::: RESULTS ::: Our dependence parameter, R, may be less influenced by outcome prevalence and has an intuitive meaning that facilitates interpretation. Estimates of R from previous studies can be obtained using simple statistics. Comparison of methods showed that the traditional ICC approach to sample size determination tends to overpower studies under many scenarios, calling for more clusters than truly required. ::: ::: ::: LIMITATIONS ::: The methods are developed for equal-sized clusters, whereas cluster size may vary in practice. ::: ::: ::: CONCLUSIONS ::: The dependence parameter R is an alternative measure of dependence among binary outcomes in cluster randomized trials that has a number of advantages over the ICC. <s> BIB009 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Discussion <s> BACKGROUND ::: The Uzbekistan 1996 Demographic Health Survey reported 60.4% of women of reproductive age (WRA) had low hemoglobin concentrations (<120 g/L), and anemia was an important public health problem. Fortification of wheat flour was identified as an appropriate intervention to address anemia due to the ubiquitous consumption of wheat flour. A National Flour Fortification Program (NFFP) was implemented in 2005. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: After 3-years of the NFFP, a national survey using large country-lot quality assurance sampling was carried out to assess iron, folate, hemoglobin and inflammation status of WRA; the coverage and knowledge of the fortified first grade UzDonMakhsulot (UDM) flour/grey loaf program; and consumption habits of women to investigate the dietary factors associated with anemia. Estimated anemia prevalence was 34.4% (95% CI: 32.0, 36.7), iron depletion 47.5% (95% CI: 45.1, 49.9) and folate deficiency 28.8% (95% CI: 26.8, 30.8); the effect of inflammation was minimal (4% with CRP >5 mg/L). Severe anemia was more prevalent among folate deficient than iron depleted WRA. Presence of UDM first grade flour or the grey loaf was reported in 71.3% of households. Among WRA, 32.1% were aware of UDM fortification; only 3.7% mentioned the benefits of fortification and 12.5% understood causes of anemia. Consumption of heme iron-containing food (91%) and iron absorption enhancers (97%) was high, as was the consumption of iron absorption inhibitors (95%). ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: The NFFP coincided with a substantial decline in the prevalence of anemia. Folate deficiency was a stronger predictor of severe anemia than iron depletion. However, the prevalence of iron depletion was high, suggesting that women are not eating enough iron or iron absorption is inhibited. Fortified products were prevalent throughout Uzbekistan, though UDM flour must be adequately fortified and monitored in the future. Knowledge of fortification and anemia was low, suggesting consumer education should be prioritized. <s> BIB010
After reviewing three methods for implementing cluster sampling in LQAS surveys, we conclude that the cluster LQAS methods result in similar designs when p l and p u are far from 0 or 1 or when clustering is minimal. In these situations, the user should not be concerned about which of the cluster methods to use, as the methods should all result in similar designs. Our simulation study suggests that the choice of distributional assumptions also does not appear to substantively impact the survey design. Choice of the clustering parameterization is the major factor that can impact the final cluster LQAS survey design. Specifically, if p l or p u is near 0 or 1, the design (sample size) changes substantially depending on the clustering parameterization, namely whether ρ or σ is fixed as a function of p. That is, when p l .1 and p u !.9, clustering parameters should be selected carefully to avoid severe misspecification of classification risks. As an example, assume p l , p u >.5. When σ and ρ result in the same amount of clustering for p = p l (that is, σ = ρp l (1 − p l )), the design with fixed σ will require a larger n; for p = p u , the design with fixed ρ will require a larger n. Differences in n between these options can be substantial when p u is close to 1 or p u −p l is large. We did not find evidence that ρ is constant as a function of p (assumed by Hedt and Hund) nor that σ is constant as a function of p (assumed by Pezzoli). In two vaccination coverage examples, estimates of both ρ and σ decrease as coverage increases, consistent with BIB009 BIB001 . Using historical data to plot clustering estimatesŝ andr as a function of estimated coveragep can help evaluate the clustering parameterization assumptions. Our data analysis suggests that, if clustering design parameters (σ or ρ) are determined from historical data, only estimates ofŝ andr from SAs with estimated coverages near the range of p l and p u should be incorporated. To facilitate incorporation of historical data, we provide R functions that estimate σ using summary statistics commonly available in the literature. We strongly encourage cluster LQAS users to publish their estimates of these summary statistics (p, s, andr) for each SA to provide more historical data for specifying these design parameters in the future. When the Hund or Hedt methods are used, we propose the following steps to choose a design: 1. Define p m as whichever of p l or p u is closer to .5. Estimate ρ using existing data or existing literature using SAs with coverage estimates near p m . If data are not available, consider how much variability is expected around p m to select ρ. 2. Determine the cost of sampling a cluster and sampling each individual per cluster. 3. Iterate through choices of m and k to minimize cost for α, β, p l , p u , and ρ. The same steps could be applied to the Pezzoli method, replacing ρ with σ. The R package accompanying this paper can help achieve Steps 1 and 3. Additionally, users can examine the sensitivity of their designs to the selected model (as in Tables 3 and 4) by calculating different possible designs with varied clustering parameterizations (after selecting the relevant values of p l , p u , α and β) using the R package. Knowledge of the design's sensitivity to the clustering parameterization contributes to understanding the robustness of the design. In Step 1 above, determining ρ or σ based on p m has two advantages: (1) σ and ρ are likely to decrease as p moves away from .5, so this clustering estimate is conservative (note that, using this algorithm, fixing σ is more conservative than fixing ρ); and (2) visualizing a highly skewed distribution in terms of a mean and standard deviation is not straightforward, and the skewness of the beta and binomial-scaled distributions increases as p moves away from .5. Returning to Fig 1, the beta and binomial-scaled distributions with mean p = .5 are symmetric, and one can easily visually check that σ = .1 by examining the spread of the distribution. With mean p = .9, there is no way to visually inspect these highly skewed distributions and conclude that σ = .1. If the clustering parameters are underestimated in practice, the survey design will not have the desired precision; that is, the true α and β risks will be higher than those specified by design. One way to protect against this situation is to employ a sequential design  BIB003 . In a sequential design, the data are collected in multiple stages. In theory, the clustering parameters could be estimated as the data are collected and the design could then be changed to accommodate the observed clustering values. However, the variance of clustering parameters is high in small samples. Further, sequential designs are logistically more challenging. Future research should evaluate such sequential designs in simulation and in practice. Users should keep in mind that overestimating clustering parameters at the design phase is always more conservative and adds to the cost of the survey. In practice, to select an LQAS survey design, the user first must decide whether SRS is feasible and practical. If SRS is used, survey design can proceed using a standard binomial model and the simple steps described in the Introduction. If SRS is not used, two-stage cluster sampling is a commonly used practical alternative. In some situations, the cluster sampling design can be ignored when small within-cluster sample sizes are used  BIB002 BIB004 BIB006 or when clustering is known to be negligible  . Survey design is much simpler when the cluster sampling design is ignored, because the user can act as though the data are a simple random sample and use standard LQAS procedures for design; that is, the user does not need to specify a clustering model or specify the clustering amount (i.e. ρ or σ). When the cluster sampling design cannot be ignored, choice between the cluster LQAS methods (Pezzoli, Hedt, and Hund) is not straightforward. Historically, ρ, not σ, is fixed as a function of p in design calculations BIB007 BIB009 . Uniting the methodology for LQAS surveys with other M&E estimation surveys may help de-mystify LQAS. The binomial-scaled model is not used in the statistics or survey sampling literature, possibly due to the discrete nature of this distribution. However, based on the empirical calculations in this article, choice between the Pezzoli, Hedt, and Hund methods should depend more upon the anticipated clustering patterns in the study population than the distributional specification (though prior information on clustering is likely quite limited). Hence, further research should attempt to characterize clustering patterns in specific applications and provide suggestions for best-practice cluster LQAS designs on a setting-specific basis. We only considered data examples in the context of vaccination coverage cluster LQAS surveys in this paper, but the validity of different clustering parameterizatons should be evaluated in other contexts, for instance nutrition. Cluster sampling in LQAS has also been recently discussed in the context of Large Country LQAS (LC-LQAS) BIB010 BIB005 BIB008 . These LC-LQAS surveys select SAs using cluster sampling with the goals of estimating prevalence over a large geographic region and classifying indicators at the SA-level. The LC-LQAS surveys are distinct from cluster LQAS surveys discussed in this paper which sample clusters within SAs, rather than sampling SAs. Future work could explore accommodating uncertainty in ρ or σ at the design phase. In the data examples in this paper, both ρ and σ vary with p, complicating selection of a cluster LQAS method. All three methods could be modified to allow σ or ρ to vary as a function of p, though this modification would introduce an additional design parameter into the LQAS procedure. Future work should investigate the merits of considering designs with different clustering parameters corresponding to p l and p u or exploring different clustering parameterizations BIB009 . Additionally, all of the cluster surveys discussed in this paper rely on using a self-weighting sampling design. Future work should explore how to extend LQAS designs for unequal probability weights.