Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> Emphasizing methods for practitioners with little expertise and no background in sampling this paper presents a set of guidelines to follow in planning cluster-sample surveys of appropriate size in developing countries without undue bias. A self-weighting design based upon the World Health Organizations Expanded Program on Immunization is employed. The paper covers the topics of sample design methods of random selection of areas and households sample-size calculation and estimating proportions ratios and means with standard errors appropriate to the survey design. Extensions including stratification and multiple stages of selection are also discussed. Giving guidance on possible values the authors pay close attention to allow for survey structure in estimating sample size using the design effect and the rate of homogeneity. A spreadsheet is finally included to aid in calculating standard errors. <s> BIB001 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> On the basis of theoretical considerations, population-based nutrition surveys of 30 clusters of 30 children should provide reasonably valid estimates of the prevalence of malnutrition with at least 95 per cent confidence that the estimated prevalence differs from the true value by no more than 5 per cent. In areas of famine in Africa, where an urgent need often exists for rapid nutritional assessment to determine the extent and severity of the problem, visiting 30 sites is often logistically difficult. To determine the effects of using fewer than 30 clusters on the validity and precision of the estimated level of undernutrition, we used data from the 1983 Swaziland National Nutrition Survey and from rapid nutrition surveys performed in 1984 and 1985 in Burkina Faso, Guinea, and Niger. Fewer than 30 clusters may result in point prevalence estimates that differ dramatically from the true prevalence and, in most instances, are less precise. In contrast, little is gained by collecting more than 30 clusters. In summary, around 30 clusters provides relatively valid and precise estimates of the prevalence of undernutrition, and every effort should be made to obtain the logistic support required to study this number of clusters. <s> BIB002 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> BackgroundTimely and accurate data are necessary to prioritise and effectively respond to humanitarian emergencies. 30-by-30 cluster surveys are commonly used in humanitarian emergencies because of their purported simplicity and reasonable validity and precision. Agencies have increasingly used 30-by-30 cluster surveys to undertake measurements beyond immunisation coverage and nutritional status. Methodological errors in cluster surveys have likely occurred for decades in humanitarian emergencies, often with unknown or unevaluated consequences.DiscussionMost surveys in humanitarian emergencies are done by non-governmental organisations (NGOs). Some undertake good quality surveys while others have an already overburdened staff with limited epidemiological skills. Manuals explaining cluster survey methodology are available and in use. However, it is debatable as to whether using standardised, 'cookbook' survey methodologies are appropriate. Coordination of surveys is often lacking. If a coordinating body is established, as recommended, it is questionable whether it should have sole authority to release surveys due to insufficient independence. Donors should provide sufficient funding for personnel, training, and survey implementation, and not solely for direct programme implementation.SummaryA dedicated corps of trained epidemiologists needs to be identified and made available to undertake surveys in humanitarian emergencies. NGOs in the field may need to form an alliance with certain specialised agencies or pool technically capable personnel. If NGOs continue to do surveys by themselves, a simple training manual with sample survey questionnaires, methodology, standardised files for data entry and analysis, and manual for interpretation should be developed and modified locally for each situation. At the beginning of an emergency, a central coordinating body should be established that has sufficient authority to set survey standards, coordinate when and where surveys should be undertaken and act as a survey repository. Technical expertise is expensive and donors must pay for it. As donors increasingly demand evidence-based programming, they have an obligation to ensure that sufficient funds are provided so organisations have adequate technical staff. <s> BIB003 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> OBJECTIVE ::: To estimate the yellow fever (YF) vaccine coverage for the endemic and non-endemic areas of Bolivia and to determine whether selected districts had acceptable levels of coverage (>70%). ::: ::: ::: METHODS ::: We conducted two surveys of 600 individuals (25 x 12 clusters) to estimate coverage in the endemic and non-endemic areas. We assessed 11 districts using lot quality assurance sampling (LQAS). The lot (district) sample was 35 individuals with six as decision value (alpha error 6% if true coverage 70%; beta error 6% if true coverage 90%). To increase feasibility, we divided the lots into five clusters of seven individuals; to investigate the effect of clustering, we calculated alpha and beta by conducting simulations where each cluster's true coverage was sampled from a normal distribution with a mean of 70% or 90% and standard deviations of 5% or 10%. ::: ::: ::: RESULTS ::: Estimated coverage was 84.3% (95% CI: 78.9-89.7) in endemic areas, 86.8% (82.5-91.0) in non-endemic and 86.0% (82.8-89.1) nationally. LQAS showed that four lots had unacceptable coverage levels. In six lots, results were inconsistent with the estimated administrative coverage. The simulations suggested that the effect of clustering the lots is unlikely to have significantly increased the risk of making incorrect accept/reject decisions. ::: ::: ::: CONCLUSIONS ::: Estimated YF coverage was high. Discrepancies between administrative coverage and LQAS results may be due to incorrect population data. Even allowing for clustering in LQAS, the statistical errors would remain low. Catch-up campaigns are recommended in districts with unacceptable coverage. <s> BIB004 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> OBJECTIVE ::: Vaccination programmes targeting disease elimination aim to achieve very high coverage levels (e.g. 95%). We calculated the precision of different clustered lot quality assurance sampling (LQAS) designs in computer-simulated surveys to provide local health officers in the field with preset LQAS plans to simply and rapidly assess programmes with high coverage targets. ::: ::: ::: METHODS ::: We calculated sample size (N), decision value (d) and misclassification errors (alpha and beta) of several LQAS plans by running 10 000 simulations. We kept the upper coverage threshold (UT) at 90% or 95% and decreased the lower threshold (LT) progressively by 5%. We measured the proportion of simulations with < or =d individuals unvaccinated or lower if the coverage was set at the UT (pUT) to calculate beta (1-pUT) and the proportion of simulations with >d unvaccinated individuals if the coverage was LT% (pLT) to calculate alpha (1-pLT). We divided N in clusters (between 5 and 10) and recalculated the errors hypothesising that the coverage would vary in the clusters according to a binomial distribution with preset standard deviations of 0.05 and 0.1 from the mean lot coverage. We selected the plans fulfilling these criteria: alpha < or = 5% beta < or = 20% in the unclustered design; alpha < or = 10% beta < or = 25% when the lots were divided in five clusters. ::: ::: ::: RESULT ::: When the interval between UT and LT was larger than 10% (e.g. 15%), we were able to select precise LQAS plans dividing the lot in five clusters with N = 50 (5 x 10) and d = 4 to evaluate programmes with 95% coverage target and d = 7 to evaluate programmes with 90% target. ::: ::: ::: CONCLUSION ::: These plans will considerably increase the feasibility and the rapidity of conducting the LQAS in the field. <s> BIB005 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> OBJECTIVES ::: To evaluate oral poliovirus vaccine (OPV) coverage of the November 2009 round in five Northern Nigeria states with ongoing wild poliovirus transmission using clustered lot quality assurance sampling (CLQAS). ::: ::: ::: METHODS ::: We selected four local government areas in each pre-selected state and sampled six clusters of 10 children in each Local Government Area, defined as the lot area. We used three decision thresholds to classify OPV coverage: 75-90%, 55-70% and 35-50%. A full lot was completed, but we also assessed in retrospect the potential time-saving benefits of stopping sampling when a lot had been classified. ::: ::: ::: RESULTS ::: We accepted two local government areas (LGAs) with vaccination coverage above 75%. Of the remaining 18 rejected LGAs, 11 also failed to reach 70% coverage, of which four also failed to reach 50%. The average time taken to complete a lot was 10 h. By stopping sampling when a decision was reached, we could have classified lots in 5.3, 7.7 and 7.3 h on average at the 90%, 70% and 50% coverage targets, respectively. ::: ::: ::: CONCLUSIONS ::: Clustered lot quality assurance sampling was feasible and useful to estimate OPV coverage in Northern Nigeria. The multi-threshold approach provided useful information on the variation of IPD vaccination coverage. CLQAS is a very timely tool, allowing corrective actions to be directly taken in insufficiently covered areas. <s> BIB006 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> In November 2009, Sierra Leone conducted a preventive yellow fever (YF) vaccination campaign targeting individuals aged nine months and older in six health districts. The campaign was integrated with a measles follow-up campaign throughout the country targeting children aged 9–59 months. For both campaigns, the operational objective was to reach 95% of the target population. During the campaign, we used clustered lot quality assurance sampling (C-LQAS) to identify areas of low coverage to recommend timely mop-up actions. We divided the country in 20 non-overlapping lots. Twelve lots were targeted by both vaccinations, while eight only by measles. In each lot, five clusters of ten eligible individuals were selected for each vaccine. The upper threshold (UT) was set at 90% and the lower threshold (LT) at 75%. A lot was rejected for low vaccination coverage if more than 7 unvaccinated individuals (not presenting vaccination card) were found. After the campaign, we plotted the C-LQAS results against the post-campaign coverage estimations to assess if early interventions were successful enough to increase coverage in the lots that were at the level of rejection before the end of the campaign. During the last two days of campaign, based on card-confirmed vaccination status, five lots out of 20 (25.0%) failed for having low measles vaccination coverage and three lots out of 12 (25.0%) for low YF coverage. In one district, estimated post-campaign vaccination coverage for both vaccines was still not significantly above the minimum acceptable level (LT = 75%) even after vaccination mop-up activities. C-LQAS during the vaccination campaign was informative to identify areas requiring mop-up activities to reach the coverage target prior to leaving the region. The only district where mop-up activities seemed to be unsuccessful might have had logistical difficulties that should be further investigated and resolved. <s> BIB007 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> BACKGROUND ::: Lot Quality Assurance Sampling (LQAS) is a provably useful tool for monitoring health programmes. Although LQAS ensures acceptable Producer and Consumer risks, the literature alleges that the method suffers from poor specificity and positive predictive values (PPVs). We suggest that poor LQAS performance is due, in part, to variation in the true underlying distribution. However, until now the role of the underlying distribution in expected performance has not been adequately examined. ::: ::: ::: METHODS ::: We present Bayesian-LQAS (B-LQAS), an approach to incorporating prior information into the choice of the LQAS sample size and decision rule, and explore its properties through a numerical study. Additionally, we analyse vaccination coverage data from UNICEF's State of the World's Children in 1968-1989 and 2008 to exemplify the performance of LQAS and B-LQAS. ::: ::: ::: RESULTS ::: Results of our numerical study show that the choice of LQAS sample size and decision rule is sensitive to the distribution of prior information, as well as to individual beliefs about the importance of correct classification. Application of the B-LQAS approach to the UNICEF data improves specificity and PPV in both time periods (1968-1989 and 2008) with minimal reductions in sensitivity and negative predictive value. ::: ::: ::: CONCLUSIONS ::: LQAS is shown to be a robust tool that is not necessarily prone to poor specificity and PPV as previously alleged. In situations where prior or historical data are available, B-LQAS can lead to improvements in expected performance. <s> BIB008 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> Traditional Lot Quality Assurance Sampling (LQAS) designs assume observations are collected using simple random sampling. Alternatively, randomly sampling clusters of observations and then individuals within clusters reduces costs but decreases the precision of the classifications. In this paper, we develop a general framework for designing the cluster(C)-LQAS system and illustrate the method with the design of data quality assessments for the community health worker program in Rwanda. To determine sample size and decision rules for C-LQAS, we use the beta-binomial distribution to account for inflated risk of errors introduced by sampling clusters at the first stage. We present general theory and code for sample size calculations. The C-LQAS sample sizes provided in this paper constrain misclassification risks below user-specified limits. Multiple C-LQAS systems meet the specified risk requirements, but numerous considerations, including per-cluster versus per-individual sampling costs, help identify optimal systems for distinct applications. We show the utility of C-LQAS for data quality assessments, but the method generalizes to numerous applications. This paper provides the necessary technical detail and supplemental code to support the design of C-LQAS for specific programs. <s> BIB009 </s> Choosing a Cluster Sampling Design for Lot Quality Assurance Sampling Surveys <s> Introduction <s> Lot quality assurance sampling (LQAS) has a long history of applications in industrial quality control. LQAS is frequently used for rapid surveillance in global health settings, with areas classified as poor or acceptable performance on the basis of the binary classification of an indicator. Historically, LQAS surveys have relied on simple random samples from the population; however, implementing two-stage cluster designs for surveillance sampling is often more cost-effective than simple random sampling. By applying survey sampling results to the binary classification procedure, we develop a simple and flexible nonparametric procedure to incorporate clustering effects into the LQAS sample design to appropriately inflate the sample size, accommodating finite numbers of clusters in the population when relevant. We use this framework to then discuss principled selection of survey design parameters in longitudinal surveillance programs. We apply this framework to design surveys to detect rises in malnutrition prevalence in nutrition surveillance programs in Kenya and South Sudan, accounting for clustering within villages. By combining historical information with data from previous surveys, we design surveys to detect spikes in the childhood malnutrition rate. <s> BIB010
When designing global health monitoring and evaluation (M&E) surveys in resource limited settings, the key objective is to balance cost, precision, and feasibility. Lot quality assurance sampling (LQAS) surveys were originally proposed for industrial quality control  and have more recently been applied in global health M&E for classification of health indicators  . In LQAS surveys, samples of size n are collected from supervision areas (SAs), and the number of successes X is compared to a decision rule d to classify an SA as acceptable or unacceptable [3] . Because LQAS is a classification procedure, LQAS surveys typically require smaller sample sizes than estimation-based surveys. Further, the simplicity of classification facilitates local-level data analysis, reducing the time between data collection and data-driven actions. LQAS designs are typically determined by four parameters p l , p u , α, and β. Specifically, the sample size n and rule d are selected such that the risk of misclassifying an SA as acceptable or unacceptable is small, namely: The risk α is the probability of classifying as low when p = p u , i.e. coverage is truly high; and the risk β is the probability of classifying as high when the health indicator coverage p = p l , i.e. coverage is truly low. The risk α can then be interpreted as the 'service provider risk,' namely the risk of intervening in an area that actually has high coverage; and the risk β interpreted as the 'population risk,' namely the risk that a truly under-performing area is classified as having adequate coverage. It is important to note that the LQAS equations in 1 are often parameterized differently in the literature depending on the application, and the interpretation of the risks changes with the parameterization. The α and β risks are not restricted for SAs with p l < p < p u , i.e. in the "grey region." Choice of the design parameters p l , p u , α, and β has been discussed elsewhere [3] BIB008  and is not addressed in this paper. Typically, the probabilities in Eq 1 are calculated by assuming data are collected using simple random sampling (SRS) from the target population and then modeling X * Binomial(n, p). Implementing SRS requires enumerating and randomly sampling individuals from the target population, a challenging feat in geographically disperse and hard to reach populations. Cluster sampling surveys are popular in M&E, where enumerating clusters (often villages) and sampling within clusters is less costly BIB001 . Commonly used rapid M&E cluster surveys include 30 × 30 emergency nutrition surveys BIB002 BIB003 and 30 × 7 Expanded Program of Immunization (EPI) surveys  . Recently, three methods have been proposed to adjust LQAS sample sizes and decision rules to accommodate cluster sampling. We refer to these methods as the Pezzoli BIB004 BIB005 BIB006 BIB007 , Hedt BIB009 , and Hund BIB010 methods. In this paper, we contrast these methods and provide recommendations for choosing a cluster LQAS design. First, we review two-stage cluster sampling surveys and describe the three methods for melding cluster sampling with LQAS. We then evaluate these methods using example designs from vaccination and nutrition surveys. Finally, we provide recommendations for choosing a cluster LQAS design based on these results.