Up to 1. 45 billion people currently suffer from soil transmitted helminth infection, with the largest burden occurring in Africa and Asia. Safe and cost effective deworming treatment exists, but there is a debate about mass distribution of this treatment in high prevalence settings. While the World Health Organization recommends mass administration of anthelmintic drugs for preschool and school-aged children in high (>20%) prevalence settings, and several long run follow up studies of an influential trial have suggested large benefits that persist over time, recent systematic reviews have called this recommendation into question. This paper analyzes the long-term impact of a cluster-randomized trial in eastern Uganda that provided mass deworming treatment to preschool aged children from 2000 to 2003 on the numeracy and literacy skills of children and young adults living in those villages in 2010-2015. This study uses numeracy and literacy data collected seven to twelve years after the end of the deworming trial in a randomly selected subset of communities from the original trial, by an education-focused survey that had no relationship to the deworming study. Building on an earlier working paper which used data from 2010 and 2011 survey rounds, this paper uses an additional four years of numeracy and literacy data (2012,2013,2014, and 2015). Aggregating data from all survey rounds, the difference between numeracy scores in treatment versus control communities is 0. 07 standard deviations (SD) (95% CI -0. 10,0. 24, p = 0. 40), the difference in literacy scores is 0. 05 SD (95% CI -0. 16,0. 27, p = 0. 62), and the difference in total scores is 0. 07 SD (95% CI -0. 11,0. 25, p = 0. 44). There are significant differences in program impact by gender, with numeracy and literacy differentially positively affected for girls, and by age, with treatment effects larger for the primary school aged subsample. There are also significant treatment interactions for those living in households with more treatment-eligible children. There is no evidence of differential treatment effects on age at program eligibility or number of years of program eligibility. Mass deworming of preschool aged children in high prevalence communities in Uganda resulted in no statistically significant gains in numeracy or literacy 7-12 years after program completion. Point estimates were positive but imprecise; the study lacked sufficient power to rule out substantial positive effects or more modest negative effects. However, there is suggestive evidence that deworming was relatively more beneficial for girls, primary school aged children, and children living in households with other treated children. As this analysis was conducted on secondary data which is publicly available, no research approval was sought or received. All individual records were anonymized by the data provider prior to public release.
The theory that deworming could improve educational outcomes goes back at least as far as the 1930s [18], but there was little rigorous research on the topic until a series of trials in the 1990s testing the impact of deworming on cognitive outcomes over short periods (1-7 months) [19,20,21], followed by a series of mass treatment trials which recorded educational or cognitive outcomes. The Cochrane and Campbell reviews [11,12] identify a relatively modest number of multiple dose mass deworming trials with cognition outcomes. Both reviews note their limited ability to aggregate these trials due to incomparable outcome measures, but find that there is no evidence of positive effect from deworming on these outcomes. However, these trials largely studied school age children, while the deworming program examined in this paper targeted pre-school aged children. There is a priori reason to believe that improved early childhood nutrition can have larger effects on a range of outcomes, including cognition, than improved nutrition for school aged children [22,23,24]. Going back to the Barker hypothesis [25], evidence has accumulated that early child nutritional deprivation, such as from famine, has long run consequences for long run health, well-being, and cognition [26,27,28,29]. This implies that interventions that mitigate early childhood nutritional deficits could have long run benefits [24,30,31]. Another theme is the broad range of shocks that appear to have important effects, including some seemingly mild shocks [32]. For this age group, the findings from the deworming meta-analysis literature are sparser. A smaller group of studies in the Campbell review examine cognitive outcomes for preschool aged children [33,34]. Neither finds a significant benefit, although both papers note that the studies in question were not powered for cognitive outcomes. Evidence of short run benefit on education-related outcomes from deworming of school-aged children comes from Miguel and Kremer [5], who use a cluster randomized design to show increases in school attendance resulting from deworming. In addition, they identify spillovers both within treatment schools, and at schools up to 3 km away from treatment schools. Replication of this paper and subsequent exchanges with Aiken et al. [35] confirmed their finding of within-school spillovers and geographic spillovers up to 3 km, but showed that spillover benefits did not extend beyond 3 km range as the original paper argued. High levels of STH prevalence in this setting could explain the benefits; by contrast a recent trial in a low prevalence setting did not find comparable educational gains [36]. Evidence for longer run benefits of deworming comes from follow up data collection on the cohorts from the Miguel and Kremer study. Baird et al. [7] find marked gender-related patterns to gains in their long run follow up of the original trial: while men see employment related gains, they do not gain more education. By contrast, women are one third more likely to attend secondary school, and have improved self-reported health and increased body mass index. Ozier [6] examines whether younger siblings of children treated in the original Miguel and Kremer trial show cognitive benefit, conditional on the age at which they were exposed to the spillover. Children whose communities were treated before they were 1 year old show large benefit (via spillovers from sibling treatment), while children treated after that age show no evidence of benefit. Furthermore, the effects were twice as large for those likely to have had treated older siblings in their household, suggesting that children may benefit more from deworming when their siblings are also dewormed.
With 36 clusters and an intra-cluster correlation (ICC) of 0. 02 for numeracy scores, we calculate an intention-to-treat minimum detectable effect of 0. 18 standard deviations (for the numeracy outcome) if all children were treated in the treatment group and none were treated in the control group (the ICC is calculated from Uwezo data in study clusters, and the power calculation is implemented using Stata’s “clustersampsi” command). Inflating this to account for incomplete take up in the treatment group (74% of the treatment group reported attending Child Heath Days) implies a minimum detectable effect (MDE) for numeracy of 0. 33 standard deviations (SD). (Some control group deworming was reported from private drug shops. However without information about the frequency or intensity of this deworming we cannot precisely adjust power calculations to reflect it). As this analysis was conducted on secondary data which is publicly available, no research approval was sought or received. All individual records were anonymized by the data provider prior to public release. Outcome data in this study comes from the Uwezo initiative, which conducts large-scale annual surveys to test basic literacy and numeracy in Kenya, Tanzania, and Uganda. Every year, Uwezo randomly samples 30 villages in every district in all three countries, and all children between ages 6 and 16 in 20 households per village are tested on basic literacy and numeracy [38]. Further details about Uwezo’s sampling procedures are available in Supporting Information Section 1. In the Uwezo survey, children are given a numeracy test with seven skill areas of ascending difficulty, and a literacy test with six skill areas, for a maximum of 13 points. Following standard practice in the education literature, standardized math, literacy, and total (math plus literacy) scores are created, with mean zero and standard deviation of 1, as outcome variables. In all rounds, the basic math test consists of seven categories, representing counting objects up to 10, number recognition of numbers from 10-999, addition, subtraction, multiplication, and division. There are bonus questions across all years but since these vary in content substantially (from practical questions about buying items in market in some years, to place value questions in other years), we omit them. The literacy test includes five categories, from letter recognition, to the ability to read words, sentences, and paragraphs. There are two comprehension questions, for which one point is awarded, in early rounds, while in later survey rounds two points are awarded for the two questions. Since the meaning of a comprehension bonus point appears to change over time (and is missing for most respondents in 2010), we omit the bonus comprehension questions for all rounds and simply use the 5 point literacy scale which remains fundamentally unchanged across all six survey rounds. The Uwezo survey captures numeracy and literacy outcomes for a small fraction of the population exposed to the original program (the Uwezo sample is 2,210 while there were 27,995 individuals in the original trial). However, Uwezo’s sampling method makes it likely that the 2,210 individuals surveyed are in effect a random subsample of individuals from the communities in the original sample. This is because according to its procedures, the Uwezo survey sampled a random subset of villages from the original trial districts, and a random group of households from within these villages. Exploiting the random allocation of treatment in the original trial, and the random sampling of a subset of original trial communities by Uwezo, the effect of the deworming program is estimated by regressing test scores on a treatment indicator variable (unadjusted model) and on the treatment variable together with dummy variable controls for each age category (ages 6 through 16), each survey round (2010,2011,2012,2013,2014, and 2015), and gender, and all interactions of gender, age, and survey round (adjusted model). The specifications are: y i c = β 0 + β 1 t r e a t c + ε i, c (1) for unadjusted specifications (odd numbered columns), and for adjusted specifications (presented in even numbered columns), y i c = β 0 + β 1 t r e a t c + χ i c ′ · γ + ε i, c (2) where i is for example an individual and c is a community (parish). yi, c is the learning outcome of interest, treatc is a dummy for treatment communities from the original trial, and χ i ′ is a vector of indicator variables for each age, gender, and survey round, and all interactions of these age, gender, and survey round indicators. The coefficient of interest is β1. εi, c is the error term, clustered at the parish level. Based on reported age, a child’s age in the years of 2000,2001,2002, and 2003 is calculated. If the child was between age 1 and age 7 during any one of those years, he or she is considered to be in the age group potentially exposed to the program. Motivated by the broader deworming literature, we also examine potential modifiers of effect. We seek to do so in ways which are robust to concerns about multiple hypothesis testing. Therefore we first conduct the main analysis, and the heterogeneity analysis, using the specifications used in the 2014 working paper which was mentioned previously. In effect we treat this 2014 working paper as analogous to a pre-registered analytical plan for the full paper. Then, in more exploratory vein, we examine additional hypotheses which we believe are appropriate given changes in the sample across rounds, as well as hypotheses which have been generated by other recent papers in the literature. We examine the following dimensions of heterogeneity on the treatment effect of deworming: primary school vs. secondary school age, potential exposure to other dewormed children, and age at first eligibility for the program. Given the exploratory nature of these analyses, we calculate q-values for the coefficients of interest, limiting the false discovery rate using the method proposed by Benjamini and Hochberg [39]. (We also examine these effect modifiers on school enrolment and attendance in Supporting Information S5 Table). For hetereogeneity analysis, we interact the treatment variable with variables created to represent poverty, gender, and cumulative years of program eligibility (for the pre-specified analyses), and age at program initiation, primary school age vs. post-primary school age, and number of other deworming-eligible children in the household (for the exploratory analyses). As a proxy for poverty, we calculate the first principal component of a wealth index made up of the household asset variables which are measured consistently across all survey rounds (access to electricity, and ownership of a television, radio, and phone), and create a “low assets” indicator which represents the bottom two wealth quintiles. To represent households with multiple treatment-eligible children we create a variable which represents the number of other treatment-eligible children in the household (excluding the respondent him or herself). The “years of program eligibility” variable is calculated by summing up the number of years that a respondent was of program-eligible age (age 1 to age 7) between 2000-2003, and creating a binary variable equal to one for respondents who were eligible for 2,3, or 4 years of the program. The primary school age variable is binary, equal to one for respondents aged 7-13 at the time of survey, and equal to 0 for respondents aged 14-16. The “age at first eligibility” variable is a binary variable equal to 1 for respondents who were age 1 when the program was initiated (in 2000), and equals 0 for those who were older than age 1 in when the program was first initiated. We also conduct a range of robustness checks for the main results. Since survey non-response could bias results, we re-run the main analysis using predicted scores (for respondents who did not take the Uwezo test) generated by the Uwezo team based on observable respondent characteristics [40], as well as using inverse probability weighting to minimize any bias due to non-response. In addition, we use a different dataset, the Uganda National Panel Survey, to examine whether migration out of the study sample is correlated with being of post-primary school age, and with literacy. We also conduct all additional robustness checks which were implemented in the 2014 working paper. Finally, we also run the main analysis using population weights provided by Uwezo. Since Uwezo conducted stratified sampling (with equal numbers of communities sampled from both densely and thinly population districts), observations can be re-weighted to be more representative of the population [40]. All of these robustness checks are presented in the Supporting Information.
Across the three outcome variables, there is no statistically significant treatment effect of deworming in the full sample. In adjusted specifications, the coefficients for numeracy (0. 07 SD, standard error (s. e.) 0. 08), literacy (0. 05 SD, s. e. 0. 11) and total score (0. 07 SD, s. e. 0. 09) are not significantly different from zero (Table 2). The other educational outcomes captured in the Uwezo dataset relate to whether the child is currently in school, and whether he or she ever enrolled in school. There is no significant relationship between deworming program exposure and attendance or school enrollment (Table 3); the coefficients in adjusted specifications on being in school at the time of the survey or never having enrolled are -0. 01 (s. e. 0. 01) and 0. 01 (s. e. 0. 01). In this section we focus on several dimensions of heterogeneity in treatment effect which were examined in the 2014 working paper: gender, poverty (proxied by asset ownership), and number of years eligible for treatment, as well as three additional dimensions of heterogeneity (presence of other program-eligible children in the household, primary vs. post-primary school age, and age at first program eligibility), which are exploratory in nature.
This study addresses the question of whether a mass deworming program in a high prevalence setting had a measurable impact on basic academic skills, such as numeracy and literacy, for children who live in villages which were part of this program during their preschool years. While an earlier working paper using the 2010 and 2011 Uwezo data suggested that deworming can have important long run impact on these skills, analysis incorporating all data from 2010–2015 shows that there is no statistically significant effect of early childhood deworming on numeracy or literacy 7-12 years later. However, there is suggestive evidence of differential impact along several dimensions. We find that the program differentially affected female respondents, and that treatment effects for females alone, although imprecisely estimated and not statistically significant at conventional levels, are substantively large. In exploratory heterogeneity analysis, we note that treatment effects for the primary school age (7-13 year old) respondents are significantly larger than for the post-primary (14-16 year old) cohorts. In addition, motivated by recent findings on spillovers in deworming programs, we find that the treatment effect increases as the number of treatment-eligible children in a household increases. However, we do not find evidence that initiation of program eligibility at age 1 results in larger effects than initiation after age 1, and we do not find evidence of differential gains for age cohorts who had more years of program eligibility. What factors might explain the difference between the findings of the original working paper and this analysis? A published critique of the 2014 working paper [41] criticized it for only having information about a subsample of original trial communities, and for potential imbalance. However, as noted by several participants in 2016 International Journal of Epidemiology symposium about this and related papers, the Uwezo sampling procedure leaves no reason to expect imbalance in which of the trial communities were sampled by the Uwezo survey [42,43,44], and the full sample demonstrates no significant imbalance between the treatment and control groups across a broad range of covariates. There are no statistically significant differences on any of the variables tested, and an F test of a regression of all the variables in the table below on treatment has a p value of 0. 85. A potential explanation is suggested by the heterogeneity analysis, which showed larger effects for primary school aged children. A key difference between this paper and the original working paper is the age profile of the sample. Compared to the sample used in the original working paper, the 2010-2015 sample is now more heavily composed of post-primary school age cohorts. In the first three years of the sample (2010-2012), 34% of the sample were aged 14-16, compared to 54% in the 2013-2015 surveys. If 14,15, and 16 years olds are more likely to migrate away from their home community than primary school aged children, it is more likely that they are interviewed in a community in which they did not live at the time the program was implemented (2000-2003). This introduces measurement error into the independent variable (potential exposure to the deworming program in early childhood), and therefore biases any potential treatment effects towards zero [45]. It could also bias effects downward if more literate and numerate individuals have higher propensity to leave their home communities. Differential migration rates by age and skill are impossible to document in the Uwezo sample, since Uwezo household rosters only include children and young adults living at home; those who have migrated are simply not observed in the data. However, using the 2009-2010 Uganda National Panel Survey (which includes both household members living both at home, as well as those living outside the household for education or work purposes), we document differential migration among the aged 14-16 cohorts, and among the more literate. This is consistent which evidence from other developing countries: For example, Young [46] aggregates DHS data from 65 developing countries to shows that education is correlated with rural-urban migration, while Hicks et al. [47] use panel surveys in Kenya and Indonesia to show that cognitive ability is strongly associated with rural-urban migration, even after adjusting for education. In Uganda, Mensah and O’Sullivan [48] show associations between education and migration in the Uganda National Panel Survey. A weakness of this study is that we do not know specifically that the children and young adults surveyed by Uwezo were directly exposed to the deworming program. We do know that over 70% of children in treatment communities attended Child Health Days and were dewormed, but some unknown fraction of the respondents may have been born in different communities than the ones they were surveyed in, or else did live in the communities but never attended the deworming events. Another limitation of the study is that, after the trial from 2000-2003, the government of Uganda incorporated deworming into Child Health Days, meaning that some percentage of children in the control group received some deworming treatments after the original trial period concluded [49]. This would attenuate any benefits experienced by children in the original treatment group, since both treatment and control communities were potentially exposed to national deworming programs after 2003. How should this pattern of results be interpreted, in light of the broader long run deworming literature? In the main treatment effect specifications, much depends on the approach taken to treatment estimates which are positive but do not reach conventional levels of significance. In this case, the power of the study and the 95% confidence intervals of the estimated treatment effects should also be considered. The main regressions for numeracy have 95% confidence intervals of (-0. 1,0. 24), meaning that the study lacks sufficient power to rule out large positive effects, as well as more modest negative effects. These wide confidence intervals are a function of the fact that this study took key design parameters (especially number of clusters) as fixed by the original trial and the subsequent Uwezo data collection exercise. While p values above or below 0. 05 are often interpreted in binary terms, this has long been criticized by epidemiologists and statisticians [50,51]. In this “vote counting” paradigm, Baird et al. [7] and Ozier [6] count as evidence for the long run benefits of deworming (p<0. 05) while the results presented here count as evidence against (p>0. 05). A better approach would be to formally aggregate these results via meta-analysis. However, whereas short-run deworming trials have typically measured a small set of outcomes, such as height and weight, consistently across trials, making meta-analysis very feasible, comparable meta-analysis is more challenging for long run studies. The treatments differ (direct deworming versus exposure via spillovers), as do the populations of interest (all exposed cohorts versus children under age 1, as in [6]), and the primary outcomes studied differ (labor market outcomes [7] versus generalized cognitive ability [6] versus numeracy and literacy (this paper). While the deworming treatments differ and the outcomes measured are not directly comparable, like Baird et al. [7] we find educational gains concentrated among females, and like Ozier [6] and Miguel and Kremer [5] we find suggestive evidence of very local spillovers. The smaller point estimates of treatment effect are also consistent with this literature, given that the worm burden in the Ugandan setting, while high in absolute terms, was notably lower than the burden in western Kenya where the original deworming trial (on which Ozier [6] and Baird et al. [7] are based) took place. Relating this back to policy, a frequentist policymaker might dismiss the evidence from this setting as too imprecise to use. A more Bayesian-minded policymaker, with imprecise information about potential benefits, but needing to decide about a deworming program on the basis of this information, might compare the expected benefit of the policy to the expected cost [52]. This policymaker might be primarily concerned about the effect of deworming on the health of the STH-infected, but in a high prevalence setting, it also makes sense to consider potential long run benefits to learning. A Bayesian policymaker would not consider a point estimate that is statistically indistinguishable from zero as effectively a zero (evidence of no effect), but would consider it as the most likely of a distribution of potential effect sizes which also includes zero and negative numbers. Then the decisionmaker would compare the expected costs and benefits of this intervention to other policies to increase learning. To give context for cost-effectiveness, McEwan [53] estimates cost effectiveness ratios for a subset of learning interventions. Most studies require between 10 USD and 100 USD (PPP adjusted) to generate 0. 2 SD of learning gain. A version of Uganda’s Child Health Day program which only included Vitamin A supplementation and deworming cost $0. 22 per child served, which includes both financial outlays, staff time, and in-kind contributions [54]. These costs as were 73% driven by the deworming component ($0. 16 per recipient) and 27% by Vitamin A supplementation ($0. 06 per recipient). Since CHDs are conducted two times per year, this implies $0. 32 per child per year for the deworming component, or $0. 96 total per child for a three year program like the one studied here. A cost of $0. 96 for 0. 07 SD for learning implies an expenditure of $2. 74 per 0. 2 SD of learning. Even allowing for imprecision or differential assumptions between the cost estimates from Uganda’s Child Health Day deworming [54] and the various education trials [53], these cost ratios are favorable to deworming. In conjunction with the policymaker’s prior views on the value of deworming and the local prevalence and intensity of worm infection, a policymaker who significantly discounted the probability that 0. 07 SD was the true effect on numeracy might still choose to invest in preschool deworming [55]. More generally, how should these finding affect interpretation of the long run impact of deworming? Clearly, the estimated treatment effects here are smaller than the large effects estimated in the original working paper using 2010 and 2011 data, and are no longer statistically significant. The case for meaningful long run effects is therefore at least somewhat less convincing than it would be if the results of this study, like that of Baird et al. [7] and Ozier [6], still provided strong evidence of large effects of mass deworming. Yet this study should be interpreted in the context of the serious challenges involved in generating evidence about the long run effects of interventions such as deworming. The ideal research design for long run effects involves well-powered trials followed by detailed longitudinal data collection. This is the model of the original Miguel and Kremer [5] trial and the Baird et al. [7] follow up, and studies stemming from this original experiment have generated all of the experimental evidence on the long run effect of deworming to date. Outside of this setting, and apart from conducting new trials and waiting a decade or more for new long-run results, another approach is to identify experimental interventions, such as the one on which this paper is based, for which existing data can be used to assess long run impact. The benefit of this approach is the limited marginal cost of the exercise, and the (relative) speed with which results can be generated. The downside is that in this type of research design, control over study parameters such as measurement, survey design, and statistical power are limited. Results should therefore be interpreted in this light.
Mass deworming is recommended by the World Health Organization for health benefits to communities where soil-transmitted helminth infection is endemic. In addition to health benefits, several recent studies find long run educational or economic benefits for cohorts dewormed as children. In this paper, treatment and control communities which formed part of a cluster randomized deworming trial in eastern Uganda from 2000-2003 were surveyed from 2010-2015 to measure children’s basic numeracy and literacy. We analyze this data to see if there are detectable improvements in basic academic skills among children in the dewormed communities. We find that mass deworming of preschool aged children in high prevalence communities in Uganda resulted in no statistically significant gains in numeracy or literacy 7-12 years after program completion. However, there is suggestive evidence that deworming is relatively more effective for girls, primary school aged children, and children living in households with other treatment-eligible children.
children
medicine and health sciences
education
sociology
geographical locations
social sciences
numeracy
uganda
neuroscience
parasitic diseases
pediatrics
age groups
academic skills
research design
cognitive psychology
literacy
surveys
africa
families
research and analysis methods
public and occupational health
schools
child health
people and places
helminth infections
psychology
survey research
biology and life sciences
population groupings
cognitive science
The long run impact of early childhood deworming on numeracy and literacy: Evidence from Uganda