This paper used a blended approach that involves multiple techniques to, first, test a set of assumptions around a health behavior change communication intervention theory of change (ToC) and, second, surface some unidentified assumptions involving the local context. The intervention was integrated with women’s self-help groups (SHGs) in Uttar Pradesh, India. The key assumption tested in this paper was the linkage between SHG membership, program exposure, and maternal, newborn, and child health practices. Learnings were substantiated through empirical investigations, including structural equation modeling and mediation analysis, as well as ‘co-learning’ workshops within the community. The workshops aimed to capture and interpret the heterogeneity of local contexts through deep dialogs with the community and program implementers at various levels. Statistical analyses indicated a significant association between the amount of women’s program exposure and their health practices. SHG membership was shown to affect maternal health practices; however, it did not have a direct effect on neonatal or child health practices. The ‘co-learning’ workshops revealed crucial aspects, such as prevailing socio-cultural norms, which prevented pregnant or recently delivered women from participating in SHG meetings. This paper encourages evaluators to work with the community to interpret and co-construct meaning in unpacking the contextual forces that seldom appear in the program ToC. •Contextual factors and social norms play important role in program theory of change.•Unpacking the local context is essential while testing theory of change assumptions.•Co-learning workshops and community dialogs help surface contextual heterogeneity.•Program ToCs are dynamic and to be revisited at different phases of implementation.•Evaluators must integrate such approaches to assess programs and interpret findings. A ToC dUsing an example of a health behavior change communication intervention implemented through women’s self-help groups (SHGs) in rural India, this paper tested an assumption of the program ToC using statistical techniques and then used an approach of engaging with the community and program implementers to identify the ‘unidentified’ assumptions related to the local context.2Based on existing evidence regarding the barriers to strong maternal, newborn, and child health practices , and conThe program operated in a very diverse and complex socio-cultural setting. The state of Uttar Pradesh has a population of more than 200 million, consisting of 75 districts and 822 development blocks, and an agriculture-based economy . It has Guided by the assumption that health behavior change communication intervention, delivered through groups and federations, would improve health behaviors, a program ToC was prepared to guide program implementation. Evaluation plans were also designed at the beginning of the project, in 2012. Based on the ToC, an implementation model was prepared to reach the target women who were pregnant, had recently given birth, or had a child less than 2 years of age – through different mediums or channels with information on correct health practices. The program had eight operational regions covering 203 administrative blocks of Uttar Pradesh. Among these, the health intervention was implemented in 120 blocks and the remaining 83 blocks did not receive the health intervention. The evaluation included sampled blocks from the 120 intervention blocks with both SHGs and a health behavior change communication intervention integrated within the SHGs, as well as from the 83 comparison blocks with SHGs but without the health behavior change communication intervention.The interventions were designed to shift community norms toward health behavior change through three sets of activities. First, discussion on maternal, newborn, and child health messages in SHG meetings by trained peer educators at least once every month. Second, interpersonal communication with target women and their family members through identification of target women and providing health information through home visits. Each SHG member was supposed to identify target women in their neighborhood and share their names in the group meetings. Eventually, each SHG member was connected with at least one target woman, acting as a friend to provide information and support on correct maternal, neonatal, and child health practices. This member was called ‘Attach-a-Sakhi’, who was supposed to do home visits to provide health information at the family level. The third set of activities included community meetings and outreach activities such as Godhbharai (a ceremony to celebrate pregnancy) and distribution of letters and leaflets containing health messages to target women. These activities were actively facilitated and arranged by the federations of the groups at the village level using audio-visual aids, such as health video shows . Women’sIn the comparison blocks, groups were engaged in micro-finance activities but discussion on maternal and child health was not part of their activities. However, the provision of normal health services from frontline health workers was common in both intervention and comparison blocks.3To illustrate the importance of the methodology in testing the ToC, we focus on two key assumptions of the program ToC – namely, that the health behavior change communication activities through the SHGs and their federations improve women’s knowledge and practices around maternal, neonatal, and child health in the SHG families. The first assumption was that the SHG members would follow and share the messages received from the discussion on health in the group meetings with their families and neighbors. The second assumption was that the home visit strategy would increase interpersonal communication with target women and their family members. Further, the program assumed that outreach activities would increase awareness in the community and create an enabling environment around correct maternal, neonatal, and child health practices.The key assumptions of the ToC we have examined here are: a) behavior change communication program activities, implemented through SHG, increase women’s exposure to maternal, neonatal, and child health information. and b) program exposure acts as a mediating factor between SHG membership and correct health practices.4A core assumption in the ToC was that women’s exposure to behavior change communication program activities, implemented through groups and federations, would help them follow correct maternal, neonatal, and child health practices. Yet, we need to ask whether enough consideration was given to the ‘unidentified’ socio-cultural norms that have a bearing not only on the health practices themselves, but also on participation in the health programming? What assumptions were made about the unidentified time  it may t5Typically, the program ToC that is framed at the beginning of a project gets evaluated at the end of the project. In this case, after two years of the program intervention, using two waves of quantitative primary surveys in the intervention and comparison blocks , we testGodhbharai event, (5) community meeting, (6) night meeting; (7) viewing of health video shows. Women’s membership in the SHG was the key independent variable.The key outcome variables for testing the assumptions were women’s maternal, neonatal, and child health practices as assessed in the two waves of the survey. In each of the two survey waves, for each respondent, two composite indices – one for maternal health and another for newborn/child health practices were computed. The health program exposure was an intermediate variable in the assumed linkage between SHG membership and maternal, neonatal, and child health practices. For each woman, a summative index on health program exposure was constructed using seven dichotomized variables on the program activities a woman was exposed to during her pregnancy and after childbirth. These include (1) women’s participation in SHG meetings with health discussions; (2) home visitation by SHG cadres to provide health information/advice; (3) receipt of letters/leaflets containing health messages; participation in community outreach activities, specifically attendance at (4) We used structural equation modeling  to test By way of some contextual background on the women respondents, average age was 26 years, and the women had three children on average. Approximately, 60% of women had attended school for formal education, in 2015; this improved by 10% points, in 2017. Women’s exposure to mass media – radio, television, or newspaper – was less than 50%. The proportion of Scheduled Caste/Tribe respondents was approximately 50%. Women in the intervention area were from households with a relatively longer association with the SHG than the comparison area, both at wave 1 (36 months versus 16 months) and wave 2 (73 months versus 46 months).In terms of women’s program exposure, approximately 60% of women in the wave 1 survey were exposed to any program activities and received health information; this increased to 64% in two years. Women’s participation in SHG meetings with health discussions reduced from 47% in 2015, to 28% in 2017. Home visits by SHG members/cadres increased from 38% in 2015, to 55% in 2017. Approximately, one-quarter of women participated in community outreach activities in 2017. The maternal, neonatal, and child health practices of women in both areas improved over time, yet the magnitude of improvement was greater in the intervention area than the comparison area for eight of the ten indicators. Analysis of these results by regions, among the blocks where the health intervention was implemented, revealed substantial variation. For example, women’s exposure to program activities and the magnitude of improvement in the maternal, neonatal, and child health practices were much higher in some regions than others. The dialog with implementers revealed that the variation in the program exposure across regions could be due to the presence of SHGs for a longer duration, the high volume of SHGs in those regions, and proximity to the program headquarter.Findings from the structural equation modeling suggested that SHG membership had a direct and positive association with health program exposure. Further, women’s exposure to program activities had a significant positive influence on maternal, neonatal, and child health practices. Delving more deeply, we assessed the effect of program activities on each of the maternal, neonatal, and child health practices and found that various program activities affected the practices differently. For example, delivery practices and current use of any contraceptive method were not influenced by women’s exposure to any of the program activities. On the contrary, health meetings significantly and positively influenced women having at least four antenatal care visits and postnatal care within the week after delivery. Health discussions in SHG meetings seem to influence women practicing kangaroo mother care, cord care, exclusive breastfeeding, and age-appropriate child immunization. Home visits were found to be the most effective medium toward improving all the maternal, neonatal, and child health practices.The mediation analysis confirmed the positive role of SHG membership on maternal, neonatal, and child health practices. Health program exposure emerged as the mediator explaining the association between SHG membership and maternal, neonatal, and child health practices. Approximately 34% of the overall effect of SHG membership on maternal health practices is mediated through women’s program exposure . For neo6Despite the positive association between women’s program exposure and maternal, neonatal, and child health practices that emerged from the statistical analyses, women’s participation in many of the activities were low. If participation were higher, an even stronger effect on health practices might have been seen. To learn more about the underlying contextual factors, we wanted to explore why, despite so much effort, did program exposure not move ‘much’? In order to shed light on such questions, we decided to engage with the community women and cadres, and staff at all levels of the program implementation team.In February 2019, we organized a community dissemination meeting with the ground-level program staff and cadres to share the key evaluation results and the geographic variation across the eight program regions, with a particular interest in learning more about the local context and challenges faced during program implementation. We aimed to make the event informal in the hopes of creating an engaging and open space for the participants. To this end, we presented the evaluation results in the local language, Hindi, and sat together on the floor with all participants. After a gap of four months, we organized a 4-day ‘co-construction and learning workshop’ with program cadres and program managers to learn from their reflections on the evaluation results. We deliberately tried to make the interactions less formal and used several qualitative techniques, such as key informant interviews with top level program managers, focus group discussions separately with a similar set of program personnel, and engaging sessions with group activities using flipcharts, color coded postcards, sticky notes, etc. We also made field visits to interact with women and SHG leaders to learn about the positives and negatives of the program activities and program coverage from their perspectives.Our interactions with the program community made us realize that the ToC had failed to acknowledge that a large proportion of SHG members are older women, and most of them have achieved their desired family size. The discussions revealed that despite the SHGs’ efforts to conduct meetings and invite target women to those meetings for health discussions, the prevailing socio-cultural norms acted as barriers. For example, newly married and/or pregnant women face mobility restrictions in many rural parts of the program area. The ground-level staff reported that women were not allowed to move outside of the home during pregnancy, as it is believed that some ‘evil spirit’ may harm the mother and to-be-born child. Cultural practices, such as postnatal confinement of the mother and newborn for at least six days following the birth, did not allow interaction of program functionaries with recently delivered women.‘parcha’ (papers) would be kept in a ‘safe’ place -- meaning they would likely never be seen or read. The community events, such as health video shows, also suffered from several logistical challenges not visualized at the beginning of the program. As the program rolled out beyond 2017, both these approaches – distribution of health leaflets and health video shows – were discarded from the program plan.We also learned about the ‘credibility’ of the information provider. The discussion revealed that for health information and advice, families trust frontline health workers more than SHG members or Attach-a-sakhi, who they rely on more to accompany them to the health centers to access health services. In addition, the absence of other  members in the family during home visits to target women by a SHG member defeated the purpose of the home visit in terms of sensitizing the entire family. Following this evaluation, a new ‘positive family meeting’ strategy was adopted with the specific intention of engaging family members in the health discussion during a home visit. In the same phase of the evaluation, we learned that while, in keeping with the ToC, letters and leaflets with health messages were given to the target women, in most cases those Our discussions around the geographical diversity of output and outcome results with the program managers of all the eight regions that exhibited heterogeneity of local contexts revealed many explanations. For example, the duration of SHGs were not uniform across the program areas. Recall from the summary of the quantitative results that there was substantial variation across districts in the relationship between program activities and improved maternal, neonatal, and child health practices. While some of the regions had SHGs since 2002, other regions had relatively newly formed SHGs. It seems likely that, unless the SHGs are matured and continue with their core functions, their health program activities will be hampered.Further, it emerged from those discussions that ‘adequate’ time needs to be given to the community before they begin to accept something new. The program started with a key focus on neonatal health; later, maternal health, and, subsequently, child health was added. Newer program activities were also ‘experimented’ with by the program management. The program managers as well as community cadres mentioned that the continuous addition of health modules, newer health messages, newer program activities and approaches caused confusion among the grassroot level workers. The duration of discussions on each message was greatly reduced and reinforcement of messages was difficult.7This paper uses a blended approach – applying techniques such as structural equation modeling and mediation analysis in testing already explicit assumptions empirically, on the one hand, while using a ‘co-learning’ approach to unpack the contextual factors explaining and more deeply understanding the ToC assumptions on the other. It illustrates that, on its own, statistical testing of the program ToC assumptions provides an incomplete picture, even if the assumed paths seem to hold true. Adopting an approach such as ‘co-learning’ that creates an open and safe space for the program functionaries at various levels is important. The community engagement process revealed certain nuances around the assumptions, which were not considered at the time the ToC was formulated yet play a critical role in explaining key paths. A ToC, which is prepared at the beginning of a program based on existing knowledge, should not be treated as not a static framework; rather, it needs to evolve organically during the implementation process. Our knowledge base should be strengthened in an ongoing way. This means going beyond statistical techniques and adopting some unconventional approaches to learn from the community as the program progresses, and using the resulting learnings to sharpen/revise program strategies or future interventions.To understand the success or failure of a program, evaluators must engage with and listen to the voices of ground-level program staff and cadres right at the outset, at the formulation of the theory of change. And given that human behaviors and ecology are complex, insights from the community are crucial in connecting the dots in the ToC and identifying missing links in the set of assumptions. The socio-cultural context is often a neglected factor in a sophisticated program ToC. In this paper, we showed that while empirical testing of ToC supported the assumed path of SHG membership, program exposure, and maternal, neonatal, and child health practices, it failed to identify the role of socio-cultural norms that influenced women’s participation in health programming and practices. For example, norms such as restricting pregnant women to the house presumably reduced exposure to SHG-led health activities, resulting in smaller effects than expected. Defying cultural norms and traditions invites serious punitive measures, especially in a patriarchal society. The discussion on geographical variation in program results pointed to the fact that it may be unfair to assume that all the CRDCs will perform in the same way and that the magnitude of improvement will be similar. Due to the contextual heterogeneity, even a smaller amount of change in one region may represent an equivalent degree of advancement, in light of initially poorer standing to other regions. Our dialogs with community women and program managers clearly suggest that more serious measurements of socio-cultural norms and contextual diversity are needed.The program ToC did not consider the role of time in bringing about change in social norms and creating an enabling environment for behaviors to be sustained. Organizing and mobilizing women in a hierarchical society is a challenging and daunting task. Bringing health as a priority in their life is far more challenging. The program assumed that when a large pool of women in a community start giving health information to influence the health behavior of a target population, an environment will be created to bring change in people’s attitudes and shift community norms. The SHGs and federations can act as change agents for strengthening community capital that can remove the barriers and help in the diffusion of health messages. The co-learning workshop pointed out that challenging social norms and following correct practices may work at a slower pace than assumed.8Undoubtedly, the ToC is a critical element in any program planning; but the ToC needs to be revisited at different phases of the program. It is imperative to measure the nuances of contextual factors that may play a crucial role in the pathways of change. Researchers and evaluators often examine and present the results of a program at an aggregate level, using sophisticated statistical techniques to test pathways. Yet, in doing so, the heterogeneities of the local context and the existing norms, and their influence on project outcomes, are frequently overlooked. If we view learning as a dynamic act, an evaluation should not stop after an initial set of statistical tests. This study shows that despite being able to test the assumed ToC paths, the robust statistical analyses failed to provide answers as to ‘why’ certain building blocks of the ToC were at different levels with variation across program geographies, or ‘how’ the assumptions followed or deviated from the ToC path. Understanding these ‘why’ and ‘how’ elements of program outputs and outcomes demands a detailed engagement with community and implementers, and a much deeper probe into local contexts.Community dialogs and co-learning workshops facilitated a deeper engagement between the evaluators and program implementers. These approaches helped surface the heterogeneity of socio-economic and cultural context and norms in explaining the assumed paths. For large and complex programs, it is necessary to revisit the program ToC jointly with implementers, evaluators, and other partners at appropriate intervals to identify important local contextual factors that may have been missed in the initial ToC.9We are students of John Mayne, and we were influenced by the discipline he brought to thinking about theories of change – for example, his focus on different kinds of assumptions helped us formulate our research questions more clearly. Our contribution in this paper will perhaps most closely correspond to his focus on learning and building an evaluative culture . We add 1)Contexts matters in learning about assumptions:This is not a new insight and was a focus of Mayne’s work in the last few years of his life. Approaches such as realist evaluation  make con2)Lack of upfront knowledge of the timeline of change for linkages:A second kind of incompleteness regarding some linkages in a theory of change is the lack of knowledge of temporality. For example, most theories do not describe how long it would take an intervention to impact an outcome. Further, anticipated timelines of impact might differ across different contexts. We believe there is need for more focused discussion of temporality in the linkages of the theory of change.3)A space for dialog with key stakeholders:Perhaps one of the most central lessons for us is that learning about assumptions cannot simply involve data analysis. We needed spaces for dialog, and it was important that such a space was created in a way that promoted dialog without implied asymmetries in power -- for example, we insisted that, like all participants, we also sat on the floor during the dialog. Such dialog and spaces must be created with humility and openness to uncertainty. The pretense that all knowledge resides in statistical data can interfere with interpretation and co-construction.4)The importance of mixed methods:This approach also strongly reinforced for us the critical importance of mixed methods. We work in organizations that tend to prefer statistical methods over qualitative methods or methods based in dialog and co-construction. In our experience, concepts and measurement, including properties of important concepts like social norms, are often adopted on the basis of the psychometric properties. We think greater care needs to be spent in ensuring that we understand how the participants in an intervention understand concepts such as social norms. Our lesson here is that distant views of measurement should not be reified over the views of an underlying concept as the clients themselves see it. Given that the program recipients are the ones with the greatest ‘skin in the game,’ it is important to make them more central in the definition of key measures.5)Learning needs of clients and other stakeholders:Another key learning that the co-construction dialog highlighted was: Who needs to learn from exploring the assumptions in a theory of change? Our experience has been that often the needs of a funder or an academic/consultancy organization are often implicitly considered higher than the needs of the program participants. In the co-construction dialogs, it became obvious that community members themselves appreciated the dialogs, and they had far deeper insights on the weaknesses of our statistical models including key gaps in our theorizing and measurement. Such knowledge only surfaced after we created spaces in which their views were treated not only as validation of the results but as a constant process of refinement and development in understanding the assumptions in a theory of change.6)Starting with the program clients:We believe our paper highlights the following six kinds of learnings:Finally, one of our key learnings was that it is important to conduct these dialogs up front before we complete an initial theory of change. Theories of change fundamentally inform the key constructs that define how programs work and also provide one foundation for measurement. It is important that we are informed by how key program participants view a program in both our conceptualization and measurement. Engagement with clients should not be only of refinement or validation but of creating a space in which they become the key owners of a theory of change. It is neither a funder nor an evaluation consultant who owns a theory of change, but the theory of change fundamentally reflects stories of what could happen to program recipients. Given the limited knowledge we have of participant/recipient/client contexts, it is only fair to construct such stories by first involving the participants.10.13039/100000865Bill & Melinda Gates Foundation, Seattle, WA [grant number: OPP1033910].The evaluation work was supported by the None.


Vouchers that reduce the cost of sport and active recreation participation have been shown to increase children’s and adolescent’s physical activity levels. Yet, the influence of government-led voucher programs on the capacity of sport and active recreation organisations is unclear. This qualitative study explored the experiences of stakeholders in the sport and recreation sector that were engaged in implementing the New South Wales (NSW) Government’s Active Kids voucher program in Australia. Semi-structured interviews were conducted with 29 sport and active recreation providers. Interview transcriptions were analysed by a multidisciplinary team using the Framework method. Overall, participants reported that the Active Kids voucher program was an acceptable intervention to address the cost barrier to participation for children and adolescents. Three main steps influenced the capacity of organisations to deliver their sport and recreation programs and the voucher program: (1) Implementation priming—alignment of the intervention aims with stakeholder priorities and early information sharing, (2) Administrative ease—enhanced technology use and establishment of simple procedures, and (3) Innovation impacts—enablement of staff and volunteers to address barriers to participation for their participants. Future voucher programs should include strategies to enhance the capacity of sport and active recreation organisations to meet program guidelines and increase innovation. The World Health Organization has called for stronger partnerships between stakeholders to enable more children and adolescents to be physically active more often . The spoOrganisations in the sport and recreation sector are vital stakeholders in the promotion of physical activity, particularly structured programs. Structured physical activity programs delivered by organisations in the sport and recreation sector include team sports ; individual sports ; and structured recreation . The typical operating model in the Australian context has three tiers of sport and recreation organisations: National Sporting Organisations (NSO) and State Sporting Organisations (SSO) govern and support affiliated community-based, grass roots clubs and associations to deliver structured physical activity programs. Other operating models in the sport and recreation sector include businesses or independent not-for-profit organisations that an NSO or SSO does not govern. The objective of most sport and recreation organisations is to provide enjoyable, accessible, inclusive, and affordable structured physical activity programs while maintaining financial sustainability . TypicalOrganisations in the sports and recreation sector facilitate the participation of over 3.5 million Australian children in structured physical activity outside of school time each week . AustralCapacity is broadly defined as an organisation’s ability to achieve its objective with available assets and resources . The abiGovernments can use economic tools (such as taxes and subsidies) to influence the costs associated with structured physical activity participation ,21,22. FThis cross-sectional qualitative study aimed to understand the implementation process of an Australian financial incentive (Active Kids voucher) program from the perspective of sport and recreation organisations and explore the impacts of the voucher program on their organisational capacity.In 2018, the New South Wales (NSW) government in Australia launched a four-year voucher program titled ‘Active Kids’ that aimed to reduce the cost of participation in structured physical activity programs outside of school time for children and adolescents . All schTo register as a provider with the Active Kids program, organisations must deliver a sport or structured active recreation program and adhere to the Active Kids Provider Guidelines . RegisteEvaluation of the Active Kids program has shown that children who redeemed an Active Kids voucher to participate in a structured physical activity program increased their days achieving physical activity guidelines from 4 days per week at registration to 5 days per week 6 months after using a voucher . The mosIn June 2019, the NSW Government Office of Sport database of Active Kids providers contained 10,037 approved Active Kids providers offering activities to children across the state. The Office of Sport staff selected 81 organisations from the Active Kids database for inclusion in this study using a quota sampling technique, randomly selecting providers from their database until each quota was reached. The sample of providers was selected based on the number of voucher redemptions recorded in the database ; location where they delivered most of their activities ; and SSO affiliation (Yes/No/SSO). SSOs were defined by whether the provider organisation was a recognised SSO in NSW, and affiliation was defined as a grass roots club/association affiliated with a recognised SSO  hadn’t thought through all the processes to implement it … none of the sports had actually budgeted for the cost to get this implemented because of the staff time to collect information and the API that was required. Even though the government provided it, we as a state sporting organisation still had to pay for the implementation of that through our service providers.” Manager, SSO“Operationalising the Active Kids program universally across all sports, using existing systems, within the government timeline was not considered straightforward. SSOs reported a sense of obligation from the government to ensure most of their affiliated clubs/associations participated in the program. Following this, non-SSOs felt pressure from others in the sport and recreation sector to be registered so they were not seen as disadvantaging their membership base.We know there were other sports that were signing up for it, so we thought we’d better do it with [our sport]. And it’s a good service, obviously, to the players to be able to get a discount.” Business owner, Large affiliated organisation“Those who became involved with the program after it had been launched were affiliated with smaller, less-resourced SSOs or independent businesses or franchises. These smaller providers reported the registration process as simple; rather than having an SSO do it for them, those who registered themselves reported the process as being easy to undertake.[The registration process was] no trouble at all. It wasn’t hard to do. There was a whole list of things we had to send through. And then we waited a month and then we got a notification that we were a registered provider.” Business owner, non-affiliated large organisation“In the program’s second year, providers reported registration and re-registration were simple compared to the initial set-up.The voucher redemption process, where providers log on and redeem the voucher through an online government portal, generally exceeded expectations. Active Kids providers reported that the process was simple if they had automated the system or were redeeming vouchers in small numbers.Initially, there was those couple of, let’s call them logistical hurdles. But once they were overcome, it ran very smoothly. It was quick to redeem. It was quick to get the money back into the bank.” Affiliated SO, regional“Organisations that received large volumes of Active Kids registrations had typically developed sophisticated systems for redemption in partnership with the Office of Sport within the first year of implementation. Large organisations reported the administration changes came at a significant expense to the organisation; however, they deemed this cost worthwhile to ease the administrative burden on their affiliated clubs/organisations and simplify the voucher redemption process. Non-affiliated organisations were still refining the ongoing voucher redemption process, which had mixed effects on organisational resources. Medium/Large organisations that did not have sophisticated online systems in place or did not have the budget to change their registration IT systems expressed frustrations in processing the vouchers manually. Small organisations did not share the same frustrations as medium/large organisations without sophisticated IT systems due to the reduced frequency of the task.It became such a gigantic job that I’ve had to get one of my staff members now to take over [redeeming the vouchers] … its’ an admin nightmare. I now have to pay a staff member to do all the inputting, because I just don’t have time to do it. It’s become an additional cost for the business.” Business owner, non-affiliated organisation“While, overall, the voucher redemption had been refined by the time the interviews took place, confusion remained among providers when something went wrong during the redemption process. The providers and their affiliated organisations, or individual providers, did not know where to report and solve administrative issues. The different roles of the two government agencies, the Office of Sport and Service NSW, led to confusion in the communication pathways.The level of information that we were provided was pretty much sub-standard. It was difficult because we were told one situation but then our members were told something else. And it just sort of went around in a, in a circle.” Manager, SSO“Stakeholders reported that implementation of the Active Kids program positively impacted their finances through participation and memberships, improved the motivation and organisational climate (human resources) and strengthened external relationships. There was limited innovation in the planning and delivery of structured physical activity programs by stakeholder organisations, which appeared to be related to the alignment between the program aims and sector activities.Provider’s perception of the Active Kids program on their membership was largely positive, with no anecdotal reports of reductions from participating Active Kids providers. Some participants had easily accessible records to monitor voucher use (n = 4), while others did not have access to the information or were not monitoring (n = 25) voucher redemption rates in their sport. Anecdotally, eight providers reported increased participation numbers.It’s significant. We’ve seen 25% growth in our club, literally this year. Last year, we took a big step up.” Affiliated SO, President“Three providers described that the voucher was attracting more family members of their usual participants to begin memberships after the organisation became a provider.Siblings join them who wouldn’t have before. We’ve got one family whose kids are foster care kids, and so they might not have had that opportunity to [play sport together], but now they do because [the Active Kids voucher] gives them that extra bit of financial help to be able to do that.” Committee member, SSO“Two providers reported they thought the voucher might increase their older children’s participation and retention. However, the majority did not perceive the voucher to impact the number of registrations or memberships at their organisation.I don’t think it’s made an impact in terms of numbers or, even the demographics. I think it’s really just been helping out in our existing members financially.” Manager, Large organisation“The Active Kids vouchers stimulated new motivation for some providers and their staff members to reach and engage new participants. Those providers working in socioeconomically disadvantaged areas and/or with low-cost activities most reported this shift in approach. Knowing that families would be assisted financially through the Active Kids program and that membership would not result in less money for essential items empowered staff to encourage families to invest in their child’s sports registration.It’s been a positive impact on staff because, they feel that they can better sell our programs to other people and know the fact that everybody can be involved in these programs, even if they are a little bit socio-economically disadvantaged … it makes staff feel more confident and empowered when talking to people on the phone.” Manager, Non-affiliated large organisation“I think it’s the first time in many years memberships have all been paid up, and I’m not chasing families for money during the year.” Committee member, SSO“This empowerment of staff and volunteers facilitated further promotional activities to increase participant recruitment. Providers took the Active Kids program as an opportunity to promote their activity, as well as ensure their community members were aware of how to utilise the vouchers. Many providers (n = 16) reported increasing their marketing activity once registered as providers and including Active Kids in this activity promotion.I’ve now added the approved provider logo that the New South Wales government said that we were allowed to add…. certainly, having that logo on our marketing is reducing the barrier to entry.” Committee member, Medium Affiliated organisation“Through delivering the Active Kids program, the collegiality of the sports industry may have adjusted towards increased collaboration between providers within one structured physical activity and/or across different organisations. For example, providers who adopted the program early could assist their peers in registering for the program in the absence of SSO guidance. Providers reported promotional activities were primarily focused on ensuring their community members were aware of the Active Kids program and that engaged children could utilise the vouchers to be active. Providers were still satisfied if their participants had used their voucher with another provider, as long as they had used the voucher. One participant reported holding a forum in her regional community after noticing that local families were unaware of the voucher or how to access it. They conducted sessions to increase the registration of children in any structured physical activity opportunity, not just specifically their sport.In our community hall, we set up three stations of laptops. And we got parents to come in with a group of community people and show them how to use the laptops to get onto their sites to get the access, and then they could print them out there and then, or they could just write their number down and access them later on and lodge it wherever they wanted to” Committee member, Medium Affiliated organisation“Another organisation reported doing presentations at local schools to increase awareness of their sport and the Active Kids program. Most providers (n = 26) reported asking individuals whether they had an Active Kids voucher when they began the registration process with a new or existing member. Those who actively asked parents and caregivers if they had a voucher would often help people to sign up their children for the Active Kids program so that it could be used when paying their registration fee. Some providers (n = 3) were less proactive in promoting the Active Kids voucher, although they would agree to redeem vouchers when customers presented them. These passive Active Kids providers were the organisations who reported administrative challenges redeeming larger volumes of vouchers.With the introduction of a second voucher in the latter part of the year, some providers reported developing informal partnerships with other sports for their off season; however, the majority (n = 25) of providers did not report ongoing changes in their partnerships with others in the industry.a lot of the other codes are starting to realise that what we’re training from a skill basis is enhancing their skills and their own codes. Some of the clubs in some of the areas including ours are actually actively promoting to go and play [approved activity] if you want to up your skills. We’re no longer looked at as the enemy anymore. We’re looked at as supplementary…The cross promotion has 100% been influenced by the Active Kids program” Committee member, Affiliated organisation“Providers that reported leveraging the Active Kids program to achieve these innovation impacts were only able to do so after achieving administrative ease. Often these organisations were part of an SSO or affiliated with an SSO. Challenges with organisational infrastructure and the expenses required to administer the program reduced the capacity of providers to do things differently.This study was the first to explore the impact of implementing a large-scale financial incentive intervention on the capacity of structured physical activity organisations. As part of a comprehensive evaluation of the Active Kids program, this study provides evidence with practical implications for policy makers planning and designing financial incentive programs in partnership with stakeholders in the sport and recreation sector. The introduction of the Active Kids voucher program initially reduced organisational capacity particularly regarding financial, infrastructure (IT systems), and human resource dimensions of capacity. During ongoing implementation, these challenges were mostly resolved. Three themes were identified from the data that demonstrated practical supports that should be considered when implementing a financial incentive or similar intervention. Firstly, organisations reported intervention priming helped enhance their ability to respond to the government program. Second, support to update administration systems were required, particularly for medium-large organisations and those not affiliated with national or state sporting organisations. Third, stakeholders who were well supported had the capacity to leverage positive impacts from the intervention. Government interventions that require structured physical activity providers to achieve their objectives should include strategies to build their partners’ capacities through financial assistance or training. This study highlights the unintended impacts of a large-scale financial incentive program on implementation partners. It suggests ways to strengthen the capacity of structured physical activity providers for enhanced implementation.Research has shown that sport and recreation sector organisations have limited capacity yet, financial incentive interventions have not previously assessed the impact of the intervention on organisational capacity ,25,26,40The sport and recreation sector comprises a range of different organisations in NSW, including SSOs, non-affiliated organisations, and business owners that provide structured physical activity programs. Traditionally, the sub-national government engage most with SSOs however a broader definition of stakeholders was adopted for the Active Kids program to remove barriers to sport and other types of structured active recreation. This broader definition resulted in a more diverse group of stakeholders being involved in the program, many of whom had not previously had a direct relationship with the Office of Sport. It also increased the number of potential stakeholders that could be engaged as partners in the implementation process. Nichols et al. demonstrated the need for different types of organisations in the sport and recreation sector to receive different types of support to achieve government objectives . This waDetails of implementation processes have been under-reported in previous studies of financial incentives encouraging children and adolescents to participate in physical activity outside of school. For the Active Kids program, promotion was the responsibility of stakeholders, which achieved high awareness among parents and caregivers . Staff aThere is great potential for interventions delivered by the sport and recreation sector to increase physical activity levels and improve public health ,3,33. TrThis study is part of a complex pragmatic evaluation of the impact of the Active Kids program . StructuThis study took place during the second year of the four-year Active Kids program (2018–2023) and is not without its limitations. There were over 10,000 stakeholders involved in implementing the Active Kids program across NSW; the NSW Government Office of Sport limited the recruitment process to a sample of 81 participants. The research team asked that organisations be selected from the larger database using a quota sampling technique; however, they were not involved in the selection process due to privacy constraints. The Office of Sport staff selected the 81 participants using the appropriate sampling technique; however, with only 35% of invited Active Kids providers giving consent, there is a potential for response bias. Saturation was not achieved in this study, and we acknowledge that the representative sample includes a small proportion of the total number of providers registered in the Active Kids program. Therefore, the results should be interpreted with caution and may not reflect the experiences of all Active Kids providers. Future research should monitor stakeholder experiences throughout the implementation of financial incentive interventions to understand program adaptations, the impacts of these on organisational capacity, and program maintenance in real-world contexts.This qualitative study is the first to explore the stakeholder experience and organisational impact of implementing a universal financial incentive (voucher) program focused on increasing children and adolescents’ participation in structured physical activity programs. Organisations across the sport and recreation sector engaged as partners in implementation were able to adapt to the government’s requirements for administering the program; however, support for the infrastructure and finance dimensions of capacity would have been beneficial. Once administrative ease was achieved, implementation became institutionalised among Active Kids providers. Stakeholders reported that government action to increase children and adolescents’ participation in structured physical activities outside of school hours through the voucher program was acceptable, aligned with their goals, and supported its continuation. Similar government-led interventions should embed capacity-building strategies that address the dimensions of organisational capacity most substantially impacted by the new program and monitor the response of structured physical activity providers. The further identification of mutually beneficial interventions to enable more children and adolescents to participate in structured physical activity should be undertaken and implemented considering the stakeholder capacity.


Levamisole is an anti-helminthic drug with immunomodulatory properties that is added to cocaine to increase its potency and weight. Levamisole-adulterated cocaine (LAC) may cause an antineutrophil cytoplasmic antibody (ANCA)-associated systemic small vessel vasculitis (AAV). We aimed to characterize the phenotype of persons developing pulmonary-renal syndrome (PRS) in LAC-induced AAV and summarize its treatment and outcomes. Pubmed and Web of Science were searched (until September 2022). Reports that described co-existing diffuse alveolar hemorrhage and glomerulonephritis in an adult (age ≥ 18) with confirmed or suspected LAC exposure were included. Reports, demographics, clinical and serologic features, treatment and outcome characteristics were extracted. Of the 280 records identified, eight met the inclusion criteria, including eight unique cases. Persons were aged 22–58 years, and 50% were women. Cutaneous involvement occurred in only half of the cases. Other associated vasculitis findings and serologies were heterogeneous. All patients received immunosuppression with steroids, with cyclophosphamide and rituximab commonly added. We concluded that PRS could occur from LAC-induced AAV. Distinguishing LAC-induced AAV from primary AAV is challenging as clinical and serologic presentations overlap. Asking about cocaine use is requisite in persons presenting with PRS to guide diagnosis and appropriately counsel on cocaine cessation in conjunction with immunosuppression as treatment. Levamisole is an anti-helminthic drug with immunomodulatory properties that was previously used in the treatment of various medical conditions, including autoimmune disorders and as an adjuvant chemotherapy agent for cancer . The theThe typical presentation of LAC-induced vasculitis is inflammation of small vessels associated with necrosis and disruption of the vessel walls, leading to retiform purpura with or without hemorrhagic bullae and/or skin necrosis , arthralgia, otolaryngologic involvement  and constitutional symptoms ,14,15. TThe habitual use of cocaine may cause complications affecting virtually any organ system , and theIn contrast, smoking cocaine leads to a myriad of lower respiratory tract manifestations, including acute respiratory symptoms , exacerbation of asthma, thermal airway injury, deterioration in lung function, pneumothorax, pneumomediastinum, bronchiolitis obliterans with organizing pneumonia, noncardiogenic pulmonary edema, interstitial pneumonitis, pulmonary hypertension and diffuse alveolar hemorrhage (DAH) . DAH, asPulmonary-renal syndrome (PRS) is a potentially life-threatening condition defined by the combination of DAH and rapidly progressive glomerulonephritis (RPGN) . While dAcute renal failure and alveolar hemorrhage independently are each well-described but uncommon manifestations of LAC-induced vasculitis, affecting approximately 12.5% and 3.1% of cases, respectively ,15. CausThe purpose of this literature review was two-fold:  to charaTwo primary databases (PubMed and Web of Science) were searched from inception until 2 September 2022 using the following keywords: (“levamisole” OR “cocaine”) AND “pulmonary-renal syndrome” OR “glomerulonephritis” OR “alveolar hemorrhage” OR (“vasculitis” AND “kidney”) OR (“vasculitis” AND “lung”) OR (“vasculitis” AND “pulmonary”) OR . The search was limited to studies written in English and involving human subjects. Additional citations were added through a Google Scholar search and review of references from included reports. Case reports and case series were eligible for inclusion if they  reportedOne author  individually screened each article , and subsequently, the full texts of records recognized as potentially eligible by abstract screening. A second reviewer and subject matter expert (R.E.E.) was included to resolve discrepancies. Predetermined fields for data extraction from each report included demographics , the time between onset/diagnosis of vasculitis and onset of PRS manifestation, the method of ascertainment of levamisole exposure, evidence of DAH, evidence of glomerulonephritis, cutaneous manifestations of vasculitis (if present), other organ system manifestations of vasculitis (if present), ANCA profile and other serologies, renal indices, treatment(s) described, and outcome(s) of the case.This study was reviewed by the Augusta University Institutional Review Board and determined not to be research involving human subjects.Two hundred eighty records were identified from the two primary databases. Thirty-five additional records were identified through a Google Scholar search and review of references from included reports. Of these records, eight met the inclusion criteria. A flow chart of the literature search strategy is depicted in Age ranged from 22 to 58 years with a mean age of 41 years (standard deviation 11.7 years). Half of the cases occurred in men and half in women. Race and ethnicity were reported very infrequently. The majority of cases (63%) presented with PRS as the initial manifestation of LAC-induced AAV, but others reported PRS onset after a long-standing LAC-induced AAV diagnosis. Two of the eight cases reported a pre-existing autoimmune diagnosis in the past medical history: one patient had a history of reactive arthritis , and anoMost cases (75%) confirmed DAH with bronchoscopy, but two cases diagnosed DAH based on hemoptysis and compatible chest imaging without bronchoscopy. Renal biopsy confirmed glomerulonephritis in seven cases (88%). The remaining case was suggestive of glomerulonephritis by reporting the presence of hematuria and proteinuria on urinalysis . Of caseThe most commonly reported associated organ involvement was otolaryngologic (63%), with epistaxis and destruction/perforation of nasal septum reported. Cutaneous and musculoskeletal involvement each occurred in 50% of cases. Two cases 25%) had both arthralgia and skin lesions consisting of erythema to purpura/necrosis on the face and extremities % had bot. The mosThe majority of cases were positive for myeloperoxidase  antibodyCreatinine at presentation with PRS varied widely from normal to very high (up to 11.07 mg/dL). When urinalysis results were reported in six of eight cases, both proteinuria and hematuria were present. Half of these cases reported nephrotic range proteinuria, one case reported non-nephrotic range proteinuria, and the remaining two cases did not specify the amount of proteinuria. Dysmorphic and crenated red blood cells were reported on urine microscopy.All eight of the cases reported were treated with high-dose steroids. Six of eight cases received additional immunosuppression with cyclophosphamide (63%) and/or rituximab (25%). One person who initially received IV cyclophosphamide was transitioned to maintenance azathioprine therapy but did not tolerate this medication . The maiPulmonary-renal syndrome (PRS) can occur from levamisole-adulterated cocaine-induced antineutrophil cytoplasmic antibody-associated vasculitis (LAC-induced AAV). In this literature review, we determined that PRS from LAC-induced AAV is gender indiscriminate and highly heterogeneous in phenotype with regards to age, associated clinical features and analytical results, which may lead to diagnostic and treatment difficulties.The diagnosis of LAC-induced AAV is a diagnosis of exclusion based on compatible clinical presentation in the context of a reasonable ascertainment of exposure to levamisole-adulterated cocaine. The classic presentation of LAC-induced AAV includes fever, arthralgia, otolaryngologic symptoms, and, in the vast majority of cases, cutaneous involvement, most commonly as retiform purpuric lesions with or without central necrosis, with a special predilection for the face and earlobes ,15,35. TAntineutrophil cytoplasmic antibodies (ANCA), autoantibodies directed against antigens in the cytoplasmic granules of neutrophils and monocytes, may be helpful in distinguishing types of ANCA-associated vasculitis (AAV) with often overlapping clinical presentations . GranuloThere are limited reports of levamisole-induced AAV in the absence of concomitant cocaine exposure ,7,35,42,In this review, our findings in patients with PRS secondary to LAC-induced AAV were consistent with existing literature regarding serologic profiles of LAC-induced AAV in general. However, the serologic profiles reported were non-uniform. The most commonly seen ANCA serologic profile was P-ANCA positivity without or without MPO antibody; however, this accounted for only half of the cases. Dual PR3 and MPO antibody positivity was reported in only one case. One case displayed typical C-ANCA targeting PR3-positivity indistinguishable from the typical ANCA profile of granulomatosis with polyangiitis. Positive APLA, ANA, dsDNA, RF, Anti-Ro, and Anti-La were all reported, but none consistently across more than two studies. No serologic features predominated in this review that would consistently distinguish PRS secondary to LAC-induced AAV from PRS secondary to primary AAV. Anti-HNE was not reported in any of the cases in our review. Lood and Hughes have demonstrated that the pathogenesis of LAC-induced AAV may be driven by the ability of cocaine and levamisole to induce the release of inflammatory neutrophil extracellular traps (NETs), leading to neutrophil elastase (HNE) autoantigen production . SystemaLevamisole was confirmed in the urine by liquid chromatography-mass spectrometry (LC-MS) in only one of eight cases of PRS due to LAC-induced AAV in this review. Instead, cocaine exposure was determined either by history alone or with a conventional urine drug screen. The pharmacokinetics of levamisole, with a short plasma elimination half-life of 5.6 h, extensive metabolism, and minimal unchanged drug urinary excretion (2–5%), make it a difficult agent to detect in biological samples ,45. ClinMore information is needed about the optimal treatment of PRS secondary to LAC-induced AAV. The existing paradigm of treatment for LAC-induced AAV, in general, has been a complete cessation of cocaine  use ,35,46. IPlasmapheresis was used twice in these cases, once with reported recovery in a patient with the primary manifestation of DAH over renal involvement and once in a patient who required hemodialysis and remained dialysis-dependent at the time of publication. In both instances, the cases occurred before the PEXIVAS trial in 2020 demonstrated no reduction in death or kidney failure with the use of plasmapheresis in persons with severe primary AAV . HoweverThis review has limitations. The data on evidence for DAH, evidence for glomerulonephritis, or associated vasculitis symptoms may have been incompletely reported in manuscripts or published abstracts, leading to incomplete case ascertainment or lack of detail to inform clinical phenotypes in this review. Data on ANCA profiles was incomplete in several included published reports which may have impaired our ability to characterize the serologic profile of PRS in LAC-induced AAV. We were unable to distinguish in each case with otolaryngologic involvement whether these manifestations were due to LAC-induced AAV or direct toxicity of cocaine, i.e., CIMDL, given their substantial overlap in clinical presentation, as is often the case in clinical practice. With the exception of Pecci et al. , where bAsking about cocaine use is requisite in persons presenting with PRS. PRS may be due to LAC-induced AAV. Differentiation of PRS secondary to LAC-induced AAV versus primary and other secondary causes of AAV can be challenging due to the heterogeneity of clinical and serologic presentations in PRS secondary to LAC-induced AAV and their substantial overlapping features with primary AAV. A detailed history eliciting any cocaine exposure may be the best and sometimes only clue to separating these entities. Recognizing the diagnosis of LAC-induced AAV is difficult but crucial. The prognosis for PRS secondary to LAC-induced AAV may depend on the patient’s understanding and willingness to discontinue cocaine use, although data on cocaine reintroduction and recurrence of PRS was beyond the scope of this review. Although PRS secondary to either LAC-induced AAV or primary AAV are treated with similar immunosuppressive regimens, it is necessary to identify LAC-induced AAV in order to appropriately counsel on cocaine cessation to improve the chance of recovery and avoid recurrence.


Realist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how program implementation triggered outcomes that could strengthen understanding of its effect on Health System Strengthening in specific settings. In low- and middle-income countries, like Nigeria, there is a paucity of realist evaluation studies, despite the implementation of multiple intervention programs and projects in these countries. This article is aimed at unveiling the black-box of program implementation and Health System Strengthening of the “Abiye” Safe Motherhood Program in Ondo State, Nigeria. Specifically, it identified the role of contextual factors in the “Abiye” program in Ondo State, determined the mechanisms that facilitated or constrained outcomes of the “Abiye” program, and developed a Context Mechanism Outcome (CMO) Configuration from which a Middle Range Theory (MRT) can be framed.This was qualitative research structured along with the realist domains . The Initial Program Theory was validated by the qualitative study, after which a new MRT was developed. The study population comprised key stakeholders, secondary stakeholders, and primary stakeholders in the Abiye safe motherhood program. Data was collected through 10 key informant interviews, 28 in-depth interviews, and six focus group discussions sessions. Thematic analysis was used to analyze all the qualitative data collected, and seven themes with 19 subthemes emerged in the study.We identified 13 contextual factors under five principal areas, with most of the factors playing enabling roles, some playing inhibitory roles, while very few played both roles. We elicited eight mechanisms, and some of these facilitated the outcomes, while some constrained the outcomes of the program. Health system strengthening was a key feature of the outcome of the program. We developed a middle-range theory based on the 6 CMO configurations we elicited from the study.Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC. Realist evaluation is a form of theory-based evaluation and is unique because of the assumption that nothing works everywhere for everyone . It usesRealist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how outcomes were triggered by program implementation, which led to health system strengthening in specific settings –8. In loAbiye” (Safe Motherhood in the Yoruba language) program was initiated in 2009 and stopped in early 2017 in Ondo State, Nigeria to address the challenges of poor maternal and child health outcomes in the state. The initiative sought to address the four levels of delay  was framed.Three steps are involved in the realist evaluation approach , 6, 7:  were developed. The initial program theories were framed around the core program strategies, which are the demand and supply components of the program. These theories were developed by adapting methods previously published in realist evaluation IPT development studies –24, and “Community engagement strengthened referral, and an effective ambulance system without delay in reaching or receiving quality health care whenever or wherever it is needed would build trust, commitment, and participation of the community women (and their households) in the health system and translate into improved access and utilization of the health facilities, thereby improving maternal health and reducing maternal and infant mortality in Ondo State.”The availability of highly motivated, skilled healthcare personnel with an appropriate service delivery environment and functional equipment that was previously unavailable would result in quality maternal care in Ondo State, thereby reducing maternal and child mortality. See The initial program theories developed were tested by the qualitative method which allowed the exploration of the context, mechanisms, and outcome of the study using Key Informant Interviews, In-Depth Interviews, and Focus Group Discussion sessions applied during the study. This method tested both the demand and supply components of the IPT along with the developed Logic Model. This was done by exploring the items listed under the IPT domains and the constructs in the Logic Model.The initially developed program theory was refined and remodeled after validation by the qualitative method applied to generate a CMO Configuration from which a new theory, the MRT, was developed.This is qualitative research that was conducted in Ondo State, South-West Nigeria. The state has three senatorial districts and eighteen local government areas . The study population was comprised of (i) key stakeholders, who were policymakers, such as the “Immediate Past Governor” of the State, the Immediate Past Commissioner of Health in the State, Directors in the Ministry of Health, and relevant departments who were present during the implementation of the program. They were included because they were critical in the agenda-setting, design, decision-making, and implementation of the “Abiye” program in the state. (ii) Secondary stakeholders included service delivery personnel, such as Maternity In-Charges, Health Rangers , nurses, and midwives who were directly involved in the state's implementation of the “Abiye” program. They were included because of their experiences and perspectives on the role of contextual factors and mechanisms in the implementation process and the outcomes that occurred. (iii) primary stakeholders, who were the program beneficiaries, particularly mothers who delivered during the program. Mothers who delivered before and during the program period, as well as older women in the community, were also included. The mothers who delivered during the program had experiences of how contextual factors and mechanisms affected the outcomes, while the mothers who gave birth before and during the program had broad perspectives on contextual factors and mechanisms that were different during their two delivery experiences. The older community women had experiences and perspectives regarding the pre-“Abiye” program period as well as witnessed the unfolding and implementation of the “Abiye” program and they provided insight into these periods.The qualitative data collection methods that were used in this study were focus group discussions (FGD) in which the primary stakeholders participated, in-depth interviews (IDI) in which the secondary stakeholders participated, and key informant interviews (KII) in which the key stakeholders were the interviewees. The participants for the FGD and IDI were selected through a probable sample procedure—multistage sampling with the senatorial districts, Local Governments Areas (LGAs), wards as the sampling frames, and then participants were selected purposively from the wards. Furthermore, three LGAs  were selected for the study and six wards  were selected for the study. The interviewees for the key informant interviews were purposely selected from the eligible directors and policymakers.The study instruments were the FGD guide, the In-depth interview guide, and the KII guide, which was developed based on the contextual factors that influenced the agenda-setting, design, decision-making, and implementation of the program and on mechanisms that facilitated or constrained outcomes of the program.During the study, 10 KII sessions were held. Key informant interviews were held with the immediate past “governor” of Ondo State, the immediate past commissioner of health of Ondo State, five directors from relevant ministries, departments, and agencies in Ondo State, and the local government primary health care coordinators in the selected LGAs. Data collection for the key stakeholders was stopped when data saturation was reached, that is when a further interview with additional key stakeholders did not yield any new information for the study . The KIIDuring the study, 28 IDI sessions were held. IDIs were held with six health rangers (one from each of the six selected wards) and six facility health workers (one from each of the selected wards). Also, in-depth interviews were held with 15 women (five in each LGA) who gave birth before and during the “Abiye” program. This was to help elicit the experiences and perspectives before and during the program, and this was in keeping with the logic of using a small sample in interview-based qualitative research . A tradiDuring the study, six focus group discussion sessions were held. In each of the selected LGAs, two FGD sessions were held, sessions for (i) mothers who had delivered in the “Abiye” program 2 years prior (2015–2016) to the end of the program and (ii) older women (age 60 years and above) in the community. Each FGD consisted of 6–12 participants. The pre-designed guide for FGD was translated into Yoruba, the predominant local language in the state, for ease of communication using a two-way translation approach and to ensure proper understanding. Each of the FGD sessions lasted about 30–45 min and was tape-recorded after seeking permission from the participants. The FGDs were conducted in the Yoruba language and the tape-recorded form was translated to English using a two-way translation approach. This was to ensure an accurate reflection of perspectives. All translations were done by two postgraduate students in the Department of African Languages at Obafemi Awolowo University, Ile-Ife . One of the authors (OJ) was the moderator in all the FGD sessions and made use of a trained note-taker who recorded key issues (through jottings and audio recording) during the sessions.The measures for analyzing the role of contextual factors in the “Abiye” policy in Ondo State were derived from the perspective of key stakeholders, secondary stakeholders, and primary stakeholders on the role of contextual factors which were elicited during the KIIs, IDIs, and FGD sessions.The measures for identifying the mechanisms that facilitated or constrained outcomes of the “Abiye” safe motherhood program were derived from the perspectives of the key stakeholders, health workers, mothers who delivered before and during the program, older women in the community, and TBA which were elicited from the KII, IDI, and FGD.Thematic analysis was carried out for all the data collected. Thematic analysis is a process of identifying patterns or themes within qualitative data . It is aThe qualitative data collected in the Yoruba language from the FGDs and some of the IDIs were translated to the English language by a postgraduate student of the Department of African Studies of Obafemi Awolowo University and back to the Yoruba Language by another postgraduate student of the Department of African Studies using a two-way translation method to ensure the perspective and ideas in the collected data is not lost. All the qualitative data in English in this study were transcribed verbatim using a rigorous process that entailed listening multiple times to the recorded tapes thereby retaining the actual meaning in the verbal accounts which is essential in qualitative analysis . The traInitial codes were generated from the transcribed dataset of this study. Codes are the basic feature of data that can be interpreted in a meaningful way concerning the topic in focus . The codThe long list of codes generated was sorted into potential themes according to the different patterns in the different data extracts. Essentially, the different codes generated from the KII, IDI, and FGD sessions were combined together, organized, and reorganized to form a different pattern to enable themes to emerge from the data set. A visual representation , a themaThe candidate themes initially identified in this study were further reviewed to refine and assess their uniqueness. Internal homogeneity within the theme and external homogeneity among the different themes identified were assessed using a dual criterion judging category . HomogenThe developed themes were described by writing detail on what each theme captures in the data set. This was done to ensure that the details in the themes were within the scope of the objectives of this study. Appropriate subthemes were also developed for the themes where they were needed. The seven final themes that emerged from this thematic analysis were given appropriate titles that fully represent the details of their contents and how they fit into the study.The findings that emerged from the analysis were shared with some of the participants to ensure the findings were credible and consistent with the perspectives of the study participants. This was to prevent the researchers' bias from influencing findings and the final report of the study. The themes with their subthemes that emerged in the thematic analysis of this study are shown in We structured our findings along with the three domains of realist evaluation: context, mechanism, and outcome, and formed CMO Configurations that guided the development of the middle-range theory.We identified the roles of contextual factors in the “Abiye” safe motherhood program in Ondo State using a simplified framework . ThirteeWe determined mechanisms that facilitated or constrained the outcomes of the “Abiye” program. The mechanisms were grouped broadly into eight groups, each having a specific focus. Each of the mechanisms identified was justified based on the themes that emerged during the thematic analysis, and the relevance to the contextual factors which influenced the implementation of “the Abiye” program with the resulting outcomes. The lack of law for the sustainability of the “Abiye” program, demotivation due to irregular payment of salary and incentives, and poor health-seeking behavior, which was a mechanism within the community constrained the implementation of the program. Other identified mechanisms that facilitated the implementation of the program are policy entrepreneurship, commitment and will, legal nudge, central funding system, financial protection, knowledge acquisition, task shifting, health rangers' mechanisms, bonding, perceived trust, and good health-seeking behavior. The “Abiye” safe motherhood program in Ondo State apart from achieving the set goals, also contributed to the strengthening of the Ondo State health system. It affected all the building blocks of the health system in the state. The health workforce, health financing, service delivery, information management system, medicine and technology, and leadership were all strengthened in the state during the implementation of the program. The supply and demand components of the IPT are related to the health system building blocks and this is specified in As a key policymaker, I knew I had to do something to reduce maternal mortality. The short period I spent in clinical practice exposed me to harrowing experiences of maternal mortality which I have never forgotten. These experiences made me take the bull by the horns when I had the mantle of leadership. I decided to make a change in maternal mortality in Ondo State.                    Key Informant 01Recruiting health professionals of different cadres was a priority for the implementation of the “Abiye” program. We recruited CHEWs, nurses, midwives, and doctors for the program in the various facilities. We employed eight obstetricians and six pediatricians in the two MCH hospitals specifically built for the “Abiye” program. The number of consultants in the state workforce increased from 4 in 2007 to 72 by 2016.                    Key Informant 01At the onset of the statewide implementation, the “Abiye” safe motherhood program in Ondo State was a priority of the state government. The political priority it generated in the state affected the focus and prioritization of some other programs within the health sector. Funding for the health sector was also skewed in favor of the “Abiye” program. The other non-“Abiye” facilities and non-maternal child health services were not well funded. This initial outcome was adjusted by the coordinating activities of the Ondo State Primary Health Care Development Board (SPHCDB), which ensured funds were appropriated to all units in primary health care in Ondo State.The “Abiye” committee coordinated how funding was done from the SMOH. We had a few sources of funds and had to strategize according to our priorities. We had some periods of skewed funding in favor of the “Abiye” program at the expense of other areas and in favor of designated “Abiye” facilities in the wards. The committee tried to achieve balanced funding during this period, but it was challenging.                    Key Informant 04The “Abiye” program had a community engagement framework and power play in all the LGA which involved community participation in the program implementation and involved some community members in sensitization campaigns. The stakeholders' forum and Ward Health Development Committee (WHDC) formed during the “Abiye” program functioned for the development of the communities beyond just the “Abiye” program. For example, in the development of the sanitary system, water system, road and security networks in some of the communities.The “Agbebiye” program (meaning “Safe Midwife” in the Yoruba language), a subsidiary of the “Abiye” safe motherhood program, had vocational skill training for Traditional Birth Attendants (TBAs) as part of the central plan of the policy when it was initiated. The TBAs were trained in a new specific vocational field for some time and were given start-up money for them to establish their new trade. They were economically empowered by this process, which led to some of them leaving the practice of delivering pregnant women. Furthermore, the recruitment of qualified health professionals like physicians, midwives, and nurses into the state as part of the policy of the intervention brought with it more socio-economic activities and investment opportunities into the state.The massive recruitment of health professionals into the state healthcare sector increased the health wage bill in the state resulting in an unintended outcome for the state government.A new government that prioritized reducing maternal mortality in Ondo State gave a window of opportunity for the implementation of the “Abiye” safe motherhood program. This favorable policy environment further ensured the enactment of the Confidential Enquiries in Maternal Deaths in Ondo State (CEMDOS) law in Ondo State. The commitment and will of the political actors  to the program and the nudge by the legal system in the state facilitated the “Abiye” safe motherhood program that ensured the full implementation of the program. However, the lack of a definite legal sustainability plan for the “Abiye” program was a constraining mechanism that affected its implementation of the program.The data on maternal deaths in the state was an embarrassment to South-West Nigeria. This drove us to action. We also knew from different pieces of evidence that these deaths were highly preventable if we did the right things in our setting, and that is what we did.                    Key Informant 02We actually had plans to sustain “Abiye” in the state through the social protection bill but it never worked out. Also, the residential card, (“Kaadi Igbeayo”) we implemented had some challenges which affected sustainability funding plans for the “Abiye” program.                    sKey Informant 01The burden of neighboring state patients at the later stage of the program made us include “kaadi Igbeayo” (residency card) as a requirement to access free maternal health. We treated every pregnant woman but discharged them after presenting the card or full payment.                    Key Informant 03The recruitment of qualified health professionals, training and retraining of in-service staff, initiating the Health Ranger scheme, and multiple incentives for the specific service delivery rendered by the health care workers were the context of the program. The motivation from the multiple incentives was a mechanism that facilitated the implementation of the program making the workers always available and ready to ensure the pregnant women were delivered safely. Continuous improvement and development of the knowledge and skills of the health workers was another facilitating mechanism in Human Resources for Health in the implementation of the “Abiye” program. Another mechanism was the task shifting process during the “Abiye” safe motherhood program which was initially resisted, but when it was accepted and implemented, ensured more skilled attendants were at the delivery of pregnant women in the state. The state also used the mechanism of the pooling system for human resources for health within the state when the midwives of the Midwives Service Scheme (MSS) from the Federal government were integrated into the “Abiye” program.Health Rangers (CHEWs) were a game-changer in “Abiye” program. They were recruited and were tasked with ensuring safe delivery for all pregnant women. We did a task-shifting process to equip them to be skilled attendants proficient at managing pregnancies, but this became a battleground for some time with our midwives and nurses. We had 5 midwives and 23 nurses in a local government with over 10,000 pregnant women. There was an obvious need for task shifting and equipping of these CHEWs to help reduce maternal deaths.                    Key Informant 02The “Abiye” safe motherhood program was implemented in the context of multiple sources of funding, different funding models , and financial risk protection for pregnant women and their children. This removal of financial risk facilitated facility delivery among the women. The central funding system mechanism was an interplay of power and policy which facilitated the implementation and ensured effective use of resources in the state to achieve their outcomes.We partly funded the program from the state fund. This showed development partners our commitment and sincerity of purpose to reduce maternal mortality within the state. We had limited funds available for this but we consistently increased our total health financing from 2.9% of the state budget in 2009 to 11% by 2014.                    Key Informant 01State funding declined during the national economic recession period. This affected our implementation during this period, and state full ownership of the program.                    Key Informant 04The renovation, equipping, and building of facilities specifically for the “Abiye” program motivated health workers to be committed and ensured availability of needed life-saving interventions and effectiveness in service delivery The referral ambulance service of the program ensured pregnant women requiring lifesaving interventions get to the health facilities immediately, while the use of Toll-free Closed User Group (CUG) in the “Abiye” program by the different participants ensured utilization of the program and thereby facilitated its implementation.Accountability and monitoring in the “Abiye” program were mechanisms that facilitated the outcome achieved because of regular data collection, monitoring, and review of the activities of the health rangers in the communities. The tracking system in the program by the State Primary Health Care Development Board and performance-based financing, which required data to measure performance, also facilitated its outcome.A word that describes “Abiye” is tracking. Tracking of pregnant women in the communities, keep tracking them during their period of pregnancy and track them in labor to delivery, and keep track of them to the postnatal period.                    Key Informant 07PBF (performance-based financing) is very good. It helped our facility to grow and render better maternal services. We employed temporary staff, including a medical doctor through it, which helped improve the utilization of our services. We did not have to go through the bureaucracy of the SPHCDB in Akure  to make decisions on some funding and recruitment issues.                      IDI 01PBF is a strict funding system. Our funders are results-driven. We had to put in our best to achieve these results or we might lose our funding. We also needed to collect accurate data like phone numbers, house addresses (description to the house), and names , which we did not routinely do formerly. Falsified or unverifiable data is not accepted and I know some facilities lost their funding because of this.                    IDI 02The mobilization campaign in the communities was a mechanism that facilitated the implementation of the program in the state. The community members and engaged before the implementation started and were involved in the decision-making process at the community level through the activities of the WHDC, which was another mechanism at the community level. The TBAs within the communities were also motivated by incentives to bring pregnant women to the health facilities to deliver thereby further reducing maternal mortality. The “carrot and stick” approach was used for the TBAs in the communities initiated by the Agbebiye program. This facilitated the outcome of the “Abiye” program in Ondo State. Discouragement of some community members because of the bad road terrain in their communities was a constraining mechanism that affected the implementation of the “Abiye” safe motherhood program in such communities. See “Provision of sustainable free quality maternal care to all pregnant women by motivated health professionals within a comprehensive health design in fully engaged communities, with a background of strong political will and prioritization, would improve utilization of the health facilities and skilled delivery thereby sustainably reducing maternal mortality in Ondo State.”Based on this new program theory, a Framework for maternal mortality reduction program was developed, which can be adopted in LMIC where maternal mortality is a challenge .The gap between policymaking and evidence generated from studies in LMIC is a challenge. Multiple evaluations of intervention programs have focused on the effectiveness and not on “what works for whom, how, why and under what circumstances”. The realist approach, which we applied in this study, explicitly explored the implementation of the “Abiye” safe motherhood program in Ondo State identifying the roles of contextual factors, the effects of mechanisms that triggered various outcomes. Furthermore, we unveiled the “black-box” of program implementation and Health System Strengthening.The contextual factors of the implementation of the “Abiye” safe motherhood program in Ondo State that emerged in this study are situational factors , structural factors , design factors , cultural factors , and international community factor . Most of these factors with their context have been described as contextual factors in previous realist evaluation studies. Marchal et al.  focused However, the “Agbebiye”, sustainability, and financial context reported in this study have not been reported in previous realist evaluation studies. This might be because the “Agbebiye” program was specific to the “Abiye” safe motherhood program, and the sustainability context reported in this study was also specific to the program. Nonetheless, we propose that for similar safe motherhood projects, program implementers could adapt the concept of Agbebiye into their setting, as this study shows it to be a program enabler.The financial context reported in this study is different from the previously reported financial context in realist studies because of the multiple funding models, central funding system, and coordination system of the financial context in the “Abiye” program. The multiple sources of funds coupled with good coordination probably contributed to the good outcomes of the Abiye safe motherhood program.In this current study, several facilitating and constraining mechanisms were identified to have affected the Abiye safe motherhood program implementation/outcomes. As found in this study, motivation and knowledge acquisition by health workers as program facilitating mechanisms has been previously reported by a prior realist evaluation study in Nigeria on the use of neonatal and pediatric pulse oximeters in 12 hospitals . SimilarThe policy entrepreneur's commitment and nudge by legal provisions were facilitating mechanisms identified in the political theme that emerged in this study. Similar facilitating mechanisms have been reported in a London realist evaluation study on modernizing health service . The awaThere are three facilitating mechanisms identified in this study that were not reported in previous realist evaluation studies to the best of the authors' knowledge. These are the task-shifting mechanism, central funding system mechanism, and tracking mechanism. However, they have been identified as forms of causal/ actor process mechanisms –41 whichThe constraining mechanisms identified in this study were demotivation of the health workers when program funds dwindled and the health workers' payments were adversely affected, and poor health-seeking behavior which was a cultural factor in certain riverine communities with poor road access. Demotivation and poor health-seeking behavior have been reported as constraining mechanisms in some realist evaluation studies in Nigeria , Ghana , South AThe lack of legal nudge for sustainability identified in this study as a constraining mechanism in this study was not reported in previous realist evaluation studies. This is possibly why the successful implementation of some programs starts to dwindle with time as found in this study. It is thus imperative that legal nudges for sustainability and identified and incorporated at the planning stage of intervention programs. Limited literature exists on the interplay of power, politics, and policy in the health system , 43. InsThe framework for maternal mortality reduction programs developed in this study could be applied in different settings. This framework is a guide around which maternal mortality reduction programs and also other public health interventions can be built. The key elements of this framework are: (a) policy championing and political prioritization, (b) comprehensive health system design, (c) strong data monitoring, (d) strengthened community systems and ownership, (e) elimination of non-skilled deliveries, and (f) adequate funding.Policy championing and sustainable political prioritization in maternal health programs have been reported in previous studies in Nigeria and sub-Saharan Africa . SimilarThis study helps in filling some of the research gaps in the realist evaluation approach. It used a realist approach to evaluate the implementation and possible outcome of a safe motherhood initiative in a low-middle income. It has brought to the fore how specific contextual factors, their mechanisms, and outcomes influence program implementation. In addition, a framework for maternal mortality reduction programs was developed which could be applied in different settings, and also for other public health intervention programs. Thus, findings from this study are useful for government at all levels, policymakers, and program managers in public health programming in Sub-Saharan Africa and other LMICs.The realist evaluation approach of the study did not allow for any causal relationship/ inferences to be made but it showed the different factors (context and mechanisms) that potentially influenced the outcome of the program. Analyzing a public policy intervention program like the “Abiye” safe motherhood program is prone to social desirability bias. However, a detailed explanation of the objectives of the study to the participants, highlighting the possible benefits to Ondo State and reassurance of confidentiality is believed to have sufficiently encouraged the participants to give valid responses. Thematic analysis was used for qualitative analysis, and it is prone to the researchers' subjective views but this was addressed by reviewing every phase of the thematic analysis process with the recorded audiotape of the data set, thereby ensuring all the detail in the themes that emerged represent the data collected in the study.Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC, especially for maternal mortality reduction programs.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Health Research and Ethics Committee of the Institute of Public Health, Obafemi Awolowo University, Ile-Ife, Osun State and its HREC Number is IPHOAU/12/830; Health Research and Ethics Committee of Ondo State Ministry of Health and its HREC Number is OSHREC/25/09/2017/019. The patients/participants provided their written informed consent to participate in this study.OO developed the concept of the study, co-designed the proposal and methodology, analyzed the data, interpreted the data, drafted the article, and approved the final version. AF reviewed the concept of the study, co-designed the proposal and methodology, reviewed and interpreted data, repeatedly reviewed the draft article, and approved the final version. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.