Preventing obesity is an international health priority and women living in rural communities are at an increased risk of weight gain. Lifestyle programs are needed as part of a comprehensive approach to prevent obesity. Evaluation provides a unique opportunity to investigate and inform improvements in lifestyle program implementation strategies. The Healthy Lifestyle Program for rural women  is a large scale, cluster randomized control trial, targeting the prevention of weight gain. This program utilises multiple delivery modes for simple lifestyle advice . Here, we describe the acceptability of these various delivery modes.n = 28] via thematic analysis and quantitative methods (program checklists and questionnaires [n = 190]) analysed via chi square and t-tests.A mixed-method process evaluation was undertaken measuring program fidelity, recruitment strategies, dose delivered, program acceptability and contextual factors influencing program implementation. Data collection methodologies included qualitative semi-structured interviews for a sub-group of intervention participants [p = 0.15). A face-to-face group session was the most commonly reported preferred delivery mode for receiving lifestyle advice, followed by text messages and phone coaching. Multiple sub-themes emerged to support the value of group sessions which included: promoting of a sense of belonging, mutual support and a forum to share ideas. The value of various delivery modes was influenced by participant’s various needs and learning styles.We recruited 649 women from 41 rural townships into the HeLP-her Rural program with high levels of program fidelity, dose delivered and acceptability. Participants were from low socioeconomic townships and no differences were detected between socioeconomic characteristics and the number of participants recruited across the towns  contains supplementary material, which is available to authorized users. The global obesity epidemic represents a great public health challenge. The Australian Preventative Taskforce has advocated the need for obesity prevention programs amongst all population groups . ReproduThe International Obesity Task Force highlights the need for monitoring and evaluating all obesity prevention and management programs . In thisThe value of conducting obesity prevention program evaluations has been established . There hAnother key research gap is the value and “audience appeal” of obesity prevention programs in women overall and the acceptability of various modes of delivering lifestyle advice  . To our Understanding participant’s value and preferences for various modes of delivering lifestyle advice, can inform translation and impact on participation rates and program reach, effectiveness and sustainability . In addrThe Healthy Lifestyle Program  is an integrated community cluster randomised controlled trial (RCT) designed to prevent weight gain in a population of reproductive-aged women living in rural Victorian communities in Australia. Detailed study design methodology are comprehensively explained elsewhere . In summThe sample size calculations were completed a-priori as outlined in the published protocol  and clinParticipant recruitment commenced in September 2012 and was completed in April 2013  Table .Table 1SFour types of process evaluation data were collected here utilising qualitative and quantitative methods. This is comprehensive for this type of program evaluation. Data collection methods used included: 1) administration data, 2) checklists and observations completed by the research team, 3a) surveys and questionnaires completed by participants and 3b) qualitative semi-structured interviews .To explore program reach and context, data from the Australian Bureau of Statistics (ABS) measuring Socio-Economic Indexes for Areas (SEIFA) of relative disadvantage was used. Potential scores ranged from 1–10 with a lower score indicating a greater level of social disadvantage relating to household total income, education attained and unemployment rates . We cateBuilding on this, we explored the relationship between participant’s perceptions of the supportiveness of their environment and SEIFA index of their township. Participant’s perceptions of the supportiveness of their environment measured via a baseline questionnaire, “do you believe your local area takes an active role in promoting healthy lifestyle to women”. Exploration of the relationship between participant’s environment and SEIFA scores were tested using chi-square .Devised program specific evaluation checklists, research team field notes and staff observations were utilised to evaluate program fidelity, dose delivered and program context. Program checklists were informed by previous literature and pilot tested and modified during the early phases of implementation –36. The This program was delivered by research staff  working in pairs; with one staff member facilitating the group education sessions, with the other observing and recording their observations. The staff member observing the delivery of the group education session provided feedback to the program facilitator, ensuring consistency of program delivery across all communities. Fidelity was addressed by all staff undergoing a one day training workshop, receiving on-going support and the use of standard delivery resources. During training staff were advised of the core elements to be delivered and where they could adapt the program to their local audience.All program participants (control and intervention) were invited to complete validated and devised questionnaires at baseline and 12-months post program enrolment. Questionnaires included items on demographic characteristics, socio-cultural and physical environment, health status , food intake and physical activity. To assess program satisfaction and acceptability participants completed a program devised satisfaction survey and ranked each intervention component  on a likert scale (1–5), where 1 = not at all helpful to 5 = extremely helpful. To see if there were differences in satisfaction scores across the various program components paired T-tests were conducted. A two-sided value of 0.05 was considered statistically significant. Data was analysed using SPSS version 19.0 for Window  with resThe acceptability of each program component was assessed through qualitative semi-structured interviews conducted in a sub-group of intervention participants only at six months post intervention commencement. A criteria-based, purposive sampling approach was performed regarding the following criteria; 1) towns allocated to receive the intervention only, 2) local government region  and 3) town population size (2000–7500). Twelve towns were eligible for participation in the qualitative sub-study and six were randomly selected for participation. All participants from the six selected communities were invited to participate in the semi-structured interviews.Participants who did not receive the full intended dose of HeLP-her Rural program  were not eligible for participation in the semi-structured interviews. This was because the focus of the semi-structured interviews was to determine the acceptability of the various HeLP-her Rural program components. Subsequently, control participants were also excluded as they did not receive the various intervention components.All participants were provided with information regarding their involvement in the qualitative semi-structured interviews, prior to participation. Written consent was provided by all participants and a letter and follow–up phone call sent to consenting volunteers. To ensure consistency between all interviews a single researcher conducted all interviews guided by a developed interview guide  and intervention group (n = 348) with a mean baseline age and BMI of 39.6 ± 6.7 years and 28.8 ± 6.9 kg/m2 respectively, with no significant difference between groups. This represented approximately 7–10 % of the eligible target population  were excluded post screening. Reasons for exclusion included pregnancy, breastfeeding, taking weight control medication, diagnosis of a serious physical or psychological condition or were not contactable.Participant recruitment commenced in September 2012 and was completed in April 2013  with 89 withdrawals primarily related to pregnancy as well as non-completers. Of these 259 women who completed the intervention, 190 women completed the associated satisfaction evaluation questionnaires at 12 months. At 12 months, 233 control participants remained in the study (77 %) with 68 withdrawals primarily related to pregnancy as well as non-completers.p = 0.15).We reached women from 41 small rural Victorian townships with diverse socioeconomic statuses. SEIFA indexes for relative disadvantage varied between scores of 1–7 Fig. . Overallp = 0.3).In addition, no statistical significant differences existed between SEIFA categories  and participant’s perceptions of environmental supportiveness for promoting healthy lifestyles (n = 289) attended the group education session.The intended dose for control participants was the attendance of one group education session. Overall, 96 % of the control participants (n = 341). During the group education sessions all participants engaged with the program manual and completed self-management activities with support from the program facilitators. Most group education sessions were delivered within 45–60 min. Phone coaching was completed by 74 % of participants and the average numbers of calls taken to complete phone coaching was 2.84. Text messages were received by 323 intervention participants with 23 participants receiving the same information via mailed post cards, as they reported not having mobile phone. Results on dose received are published elsewhere [The intended dose delivered for the HeLP-her Rural intervention arm included: one group education session, receiving a program devised manual, one phone coaching session and monthly text messages. The group education session was attended by 98 % of the intervention participants . During this workshop the underlying program theory was discussed in depth  and motivational interview training was provided.Contextual factors influencing program delivery were documented via researchers completing program devised checklists. The completed checklists indicated that there were a greater number of enablers to program delivery, in comparison to barriers (across control and intervention communities). Observed program contextual enablers included, an interactive and interested group, indicated by participants being actively involved in group discussions and asking numerous questions, evidence of social interactions, friendships, and participant’s encouraging each other during the session. The most frequent barrier to program delivery included reduced group size limiting group discussions and significant background noise made by children present at the sessions, distracting program participants and the program facilitator. However, study design and sample size limited group size in each town and other barriers were addressed by one staff member looking after the children, whilst the other delivered the program to participants.P = 0.63). There was no statistical difference found between satisfaction scores for the group session and text messages (P = 0.13).Program satisfaction was measured quantitatively by questionnaire at 1-year on a likert scale of 1–5 with a higher score indicating greater participant satisfaction. The mean ± SD score (intervention participants only) for the group session (3.6 ± 0.9), text messages (3.5 ± 1.1), phone coaching 3.2 ± 1.1) and program manual (3.2 ± 1.1) Table  ± 1.1 an2. Participants were representative of all intervention communities.Data saturation for all themes and sub-themes was achieved following 28 participant interviews with a mean age of 39.9 ± 6.2 years and BMI of 28.6 ± 5.2 kg/mParticipants demonstrated a uniform preference for group face-to-face education as the most preferred delivery mode, followed by text messages and phone coaching. The acceptability of the program devised manual’s value varied between intervention participants. Additional supportive quotes can be found in  resonated strongly with participants. Participant’s agreed that the combination of various program components led to behaviour modifications and improved program acceptability, revealing that “it’s a complete” program and “it all works together”.“I found the group session brilliant. I got probably the most out of that and then…combination of text messages and phone call”.However, the value of the group session in combination with other components was a repeated theme.Interviewees also explained that “people have different learning styles”, and “preferences” which influences how they will respond to different delivery modes. Participants emphasised that as “every individual is just so different and different things work for different people” they “wouldn’t remove any” HeLP-her Rural program components in the future.The evaluation results from the HeLP-her Rural program highlight the acceptability of delivering healthy lifestyle programs via mixed face-to-face and remote delivery modes. We reached women experiencing relative socioeconomic disadvantage and report a high level of program fidelity, dose delivered and program acceptability. The most commonly preferred method of receiving lifestyle advice was via the face-to-face group session. Whilst text messages and phone coaching also had high reported value, they appeared to be most helpful when used in combination with face-to-face contact. Overall, participants emphasised that the combination of various delivery modes maximised program acceptability and value. Based on our evaluation findings, key learnings to optimise the future implementation of the HeLP-her Rural program are described in Table In this healthy lifestyle program, we have engaged women from broad socioeconomic backgrounds including the most socially disadvantaged communities in the State of Victoria, Australia. This is in contrast to previous literature highlighting the challenges of engaging low income population groups into research trials , with moMultiple strategies were employed to ensure high program fidelity and dose delivered. We report high program fidelity as result of all program facilitators attending program specific training and standardisation of resources used during program delivery. Dose delivered has been shown to be a limiting factor in intervention success with results from an intensive obesity prevention trial, reporting that only 50 % of participants received the intended program dose . The litSupporting the value of utilising various delivery modes, two systematic reviews have reported increased program efficacy with combined program delivery modes , 8. PrioRegarding face-to-face delivery, group-based healthy lifestyle programs appear advantageous at the individual and systems level. A systematic review revealed that group-based education sessions produced significantly greater weight management effects over 12 month, compared to individual based treatments . These pThe HeLP-her Rural program is one of the largest prevention trials in Australia presenting logistical challenges but providing an opportunity to learn valuable translation and scale-up lessons. To ensure our evaluation would yield useful and reliable results, extensive pre-planning was essential. Vital components included defined evaluation questions, aligning the evaluation with program objectives and prioritising and building an evaluation plan within our program resources . Table Strengths of the current study included the application of a mixed-method evaluation approach to a rigorously designed large-scale obesity prevention RCT, targeting a healthy population at risk of weight gain. The use of robust qualitative data analysis methods  and a range of quantitative data collection tools  strengthened results. Moreover, the purposeful sample utilised in the qualitative data collection increases the generalisability of the results to the wider RCT cohort. We chose to focus on the acceptability of numerous delivery modes for lifestyle advice, as this was a key gap in the literature , 21 and Broad scale evaluations of healthy lifestyle obesity prevention programs are not routinely conducted, limiting the ability to maximise on research investments and translate evidence into practice to address the obesity epidemic. Here, results from a comprehensive evaluation of a lifestyle obesity prevention program, demonstrate strong fidelity and high level dose delivery. We reached women from relatively low income rural townships and highlight the acceptability of combined delivery modes including mixed face-to-face and remote delivery modes in this population. Group education sessions were the most highly valued program component and including at least one face-to-face session within a healthy lifestyle program appears integral. Phone coaching and text messages were also valued delivery modes, reinforcing program messages and encouraging commitment to the program. Delivery of lifestyle advice through multiple delivery modes is recommended to optimise program acceptability and ultimately effectiveness.


Nepal’s Female Community Health Volunteer (FCHV) program has been described as an exemplary public-sector community health worker program. However, despite its merits, the program still struggles to provide high-quality, accessible services nation-wide. Both in Nepal and globally, best practices for community health worker program implementation are not yet known: there is a dearth of empiric research, and the research that has been done has shown inconsistent results.Here we evaluate a pilot program designed to strengthen the Nepali government’s FCHV network. The program was structured with five core components: 1) improve local FCHV leadership; 2) facilitate structured weekly FCHV meetings and 3) weekly FCHV trainings at the village level; 4) implement a monitoring and evaluation system for FCHV patient encounters; and 5) provide financial compensation for FCHV work. Following twenty-four months of program implementation, a retrospective programmatic evaluation was conducted, including qualitative analysis of focus group discussions and semi-structured interviews.Qualitative data analysis demonstrated that the program was well-received by program participants and community members, and suggests that the five core components of this program were valuable additions to the pre-existing FCHV network. Analysis also revealed key challenges to program implementation including geographic limitations, literacy limitations, and limitations of professional respect from healthcare workers to FCHVs. Descriptive statistics are presented for programmatic process metrics and costs throughout the first twenty four months of implementation.The five components of this pilot program were well-received as a mechanism for strengthening Nepal’s FCHV program. To our knowledge, this is the first study to present such data, specifically informing programmatic design and management of the FCHV program. Despite limitations in its scope, this study offers tangible steps forward for further research and community health worker program improvement, both within Nepal and globally. Community health workers (CHWs) are key components to public sector health systems in many low- and middle-income countries –3. They The Nepali Ministry of Health and Population (“Ministry”) began its Female Community Health Volunteer (FCHV) program in 1988 . Within Despite its successes, the FCHV program faces challenges: community health services have been inconsistent and of variable quality between districts across the country –10. PrevIn this context, we evaluated an innovative pilot program to strengthen the FCHV network within a rural district in western Nepal. This program was assessed at the completion of the first two years of implementation. The goal of this evaluation was to determine whether the five core components of the program were valuable additions to the pre-existing FCHV program, as perceived by the program participants and community members, and to identify programmatic challenges and implementation barriers. The lessons learned from this evaluation have implications for the improvement of the FCHV network in Nepal and for other similar CHW programs globally.Since 2009, the Ministry has been working in a formal public-private partnership with non-governmental organization Possible to bolster clinical and community health services at the Ministry’s Bayalpata Hospital, in the remote western district of Achham, Nepal. Achham is the third poorest district in the country, and is home to a primarily agrarian population of 287,000 people who were heavily affected by the recent civil conflict .Local FCHV Leadership: Under the pre-existing Ministry program, FCHVs reported once per month to their local health post. Outside of this, there was limited oversight and support for FCHV work on a daily basis. In order to better facilitate and oversee FCHV work, this program established the role of Community Health Worker Leader (CHWL). CHWLs are literate residents of their villages who were nominated by pre-existing village Mothers’ Groups and hired as salaried staff members of Bayalpata Hospital. Each CHWL oversees an average of 9 FCHVs within their village and is responsible for communicating directly with the staff at the local health posts and at Bayalpata Hospital. CHWLs are overseen by a Director of Community Health (DCH) based at Bayalpata Hospital. This enhanced village-level leadership structure was designed to increase oversight and support for FCHVs while simultaneously improving ties between individual community members and their local health centers.Weekly Village-level FCHV Meetings: In the pre-existing FCHV system, FCHVs had met with the health worker at their health post once per month. Given other work demands and time constraints, health post staff members had limited capacity to spend time with them to provide guidance or review patient problems. To address this, CHWLs conduct weekly meetings with their village’s FCHVs, reviewing patient encounters and any challenges involved in their work. CHWLs subsequently review these patient encounters with the DCH during weekly DCH-CHWL meetings, receiving guidance from the DCH and Bayalpata Hospital clinical staff, which they are then able to convey to FCHVs at follow-up meetings.Weekly Village-level FCHV Trainings: Over 70% of FCHVs in the pilot program are illiterate with minimal formal education, which limits their health literacy capacity. To address this, the program provides thirty minute weekly trainings for FCHVs on health-related topics relevant to their patients. Weekly DCH-CHWL meetings cover materials that CHWLs then re-teach to FCHVs. Additionally, every two months, FCHVs from multiple villages come together for more intensive day-long trainings.Weekly Monitoring and Evaluation: Previously, FCHVs had reported tally-based data of their patient encounters to their health post once per month. This data was often not reviewed and FCVHs rarely, if ever, received feedback about the quality or content of the data. To address this, CHWLs collect FCHV data regarding weekly patient encounters, which are then evaluated by the DCH to monitor trends and provide feedback to CHWLs and FCHVs. At the end of each month, the DCH presents aggregate monthly data to FCHVs, CHWLs, and Bayalpata Hospital staff which, in turn, helps to identify areas for specific patient needs or for more general programmatic improvement.FCHV Compensation: Most FCHVs in Achham live in extreme poverty. Because FCHV work is not salaried in the pre-existing Ministry system, competing household, agricultural, and other professional responsibilities pose barriers to consistent, high-quality FCHV work. To overcome these barriers, financial and non-financial incentives are provided to FCHVs through this pilot program. FCHVs receive NR 200 (approximately USD $2.25) for weekly responsibilities (8–12 hours of work per week). Non-financial incentives are provided in the form of weekly meeting lunches, uniforms, health equipment, supplies, and community recognition. CHWLs and the DCH receive salaries from Bayalpata Hospital.In 2010, the Ministry and Possible began a pilot program to strengthen the pre-existing FCHV network in the villages surrounding Bayalpata Hospital. The program was composed of five core components, designed specifically to address known problems reported within the FCHV program throughout the country:We conducted a twenty-four month  retrospective study of the implementation of this pilot program. We obtained ethical review for this research from the Nepal Health Research Council  and the Partners Healthcare Institutional Review Board . Focus group discussions (n = 3) and semi-structured interviews (n = 27) were held with involved staff and community members. Two villages’ FCHV’s (n = 18), all CHWL’s (n = 9), and community members from Bayalpata Hospital’s Community Advisory Board (n = 5) were invited to participate in the focus group discussions. These discussions and interviews examined the program’s impact on FCHV supervision and leadership, training, work participation, monitoring and evaluation, and work incentives and satisfaction. Verbal informed consent was received from all participants. No children or patients were involved as study subjects.A grounded theory approach was used for qualitative data analysis. The content of focus group discussions and interviews were reviewed and coded. Primary categories were identified, which included the effects of the program on leadership, regular work meetings and consistent participation, trainings, monitoring and evaluation, and compensation for work. The results of these analyses are described below. Programmatic process metrics were assessed using weekly monitoring and evaluation data. Descriptive statistics were calculated for programmatic process metrics and cost data.Qualitative data analysis revealed that the program was received positively by focus group participants and interviewees. In the focus group discussions, there was consensus from the participants that the core components of the program were valuable additions to the pre-existing FCHV network. Specific benefits of the program were identified and are summarized, along with selected quotations from focus group discussions and interviews:Local Leadership Structure: Strong local leadership of FCHV work improves consistency and accountability of FCHV work. DCH: “Yesta saral ebam bistrit rup le byakhya gariyeka ra niyamit suchana aadan pradan ka pranali le jawabdeyita tatha samuhik ekata ra bal ko bridhi ma sahayog puryauchhan. Pahile mahila swasthya swayam sewika haru le aafno gaabisa ka didi bahini haru sanga ahile jattiko sahakarya garna paunu bhayeko thiyena.”FCHV: “(Aafno ghar ko) aru kaam bhaye pani ta jaai nahune ho bhane jaaincha!”. .Weekly Village-level FCHV Meetings: FCHVs value the opportunity for regular, structured contact. FCHV: “Yo baithak hapta ko ek choti matrai hoina din dinai ya hapta ko dui tin choti basna mildaina? Kati birami haru lai hami le aaitabar nai herchau, usko barema ek dui din bhitrai khabar puryauna paye ramro hunthyo ki?”.Weekly FCHV trainings: FCHVs value continued training opportunities, both new lessons and review of previously-taught lessons. FCHV: “Sarhai ramro kura haru sikaunuhuncha  tara hami lai badelgada bata sikaune kura chuttai kisimka hunchan. Badelgada ra hamro yeta (gaun ma) hune baithak ma pahile nasuneka, pahile nasikeka kura sikchau.”CHWL: “Taalim dohoryaune kallai ichhya hudaina? Kuda gari ekohoro gaye ta tyo kuda ko matlabai bhayena. Ekohodi kuda gareko jindagi ma ka rahala ta?”. .Weekly Monitoring and Evaluation of FCHV Patient Encounters: FCHVs value the opportunity to get regular feedback and guidance regarding their patient encounters. FCHV: “Pahile-pahile mahina ma ekchoti report bujhauna gainthyo, didi-bahini bhet hune nahune thaha hunthena, kahile bhetinthyo kahile bhetinthena. Aajkal ta haptai pichhe ekchoti sangai basincha, chalphal garincha, afno kaam ko dukha sukha badincha, naya naulo kura sikincha.”.Compensation for FCHVs: Financial and non-financial compensation, including community recognition, are important motivators for FCHV work, especially in the setting of competing domestic and professional demands. FCHV: “Kaam dherai chha tara subidha kehi chhaina..chora-chori chhan, khuwaunai padyo.”FCHV: “Pahile kohi sarhai birami pareka, marnai lageka bhaye matrai bhetna jaainthyo, aajkal sabai ko sancho bisancho bela bela bheterai shodhincha, kallai ke kasto chha thaha bhairakcha.”. FCHV: “Gaun ma birami parda sab bhanda pahile hamilai bolauchhan. Aafna chimeki ra samudaya ko herchah garna paunu hamro lagi garba ko kura ho.”. .Illiteracy: The limited literacy of the majority of FCHVs constrained the extent of detailed data collection for monitoring and evaluation. CHWLs are all literate, but by themselves, were not capable of logging all individual encounters, nor verifying each individual encounter, thus creating a reliance upon FCHVs’ memory and/or their assistance by literate family members/colleagues.➢ Professional respect: Historically, there has been a lack of professional respect for FCHVs by some of the more highly-trained healthcare workers at health posts and hospitals. In tandem with the novel CHWL position , this disrespect posed barriers of professional legitimacy, and occasionally led to an unwillingness to work with the FCHVs and CHWLs by the healthcare workers.➢ Geographic limitations: The long distances between some of the villages and Bayalpata Hospital  limited the ability of FCHVs to consistently attend group trainings, and their ability to effectively refer/transport patients from their villages to the hospital when the health posts’ services were insufficient.➢ While the program was received positively, three specific challenges to successful program implementation were identified through focus group discussions and interviews:During the study period, the pilot program expanded from three to nine villages. At the completion of the study period, the program included 92 FCHVs, nine CHWLs, one DCH, two Assistant DCHs, and a catchment area of 20,905 people. Throughout the study period, weekly village-level CHWL-FCHV meetings and trainings maintained high attendance rates: aggregate average weekly attendance for all villages was 99%. FCHVs conducted an average of 183 patient encounters per village per month, with pregnancy-related care (42%), pediatric diarrheal disease (33%), and pediatric acute respiratory infections (10%) being the three most common encounter types . Financial compensation accounted for 95% of total costs, with 45% for FCHVs, 19% for CHWLs, and 31% for the DCH and Assistant DCHs. The remaining 5% was administrative costs. Based on monthly expenditures in the program’s 24th month, costs are approximately NR 154 (USD $1.72) per patient served per annum.The Nepali FCHV program has been internationally acclaimed as a model CHW program, but within Nepal, has been inconsistent in its ability to deliver high-quality services on a national scale. We have presented qualitative data from an innovative pilot program to strengthen Nepal’s FCHV network in one district of the country. The data from this evaluation demonstrate potential value for the five core program components, suggesting that these components may be important for improving Nepal’s FCHV program. To our knowledge this is the first study to present such data, which may inform further programmatic design for the FCHV program.The data presented in this study may inform potential improvements to the FCHV program, as reported by program participants and community members. However, this study was conducted in a retrospective manner, in a single district in Nepal, within a single socio-cultural and economic context. These factors limit its external validity on a national level throughout Nepal and also on an international level as far as other similar CHW programs are concerned. Additionally, given the lack of health outcomes data associated with this study, it is not possible to assess whether these programmatic improvements would lead to changes in population-level health indices.This study provides further insights into FCHV programmatic design and implementation which may improve community-based service provision, but is limited in its scope and impact assessment. To further clarify best practices within Nepal, or globally in other similar CHW programs, additional operational and outcomes research is needed. Testing the program presented in this article regionally within western Nepal in a large-scale effectiveness trial  will hel


Con Amor Aprendemos (CAA) is an intervention created to educate the Latino community to reduce their risk for cervical cancer and diseases related to human papilloma virus (HPV). CAA was adapted to With Love We Learn (WLWL) to prevent cervical cancer and HPV in African American communities.From 1999 through 2009, African American women in the United States had the second highest incidence rates of cervical cancer and were more likely to die from cervical cancer than women of other races. Health ministries of 2 churches in the Atlanta area partnered with the Spirit Foundation Inc to adapt CAA to WLWL by tailoring the curriculum to the African American faith-based community.The National Cancer Institute’s Research to Reality (R2R) mentorship program pair collaborated with program staff on an adaptation summary form, a tool to document and assist with adapting the program curriculum with fidelity. Trainers, faith leaders, and participants adapted the program in 4 phases: 1) review of the CAA curriculum, 2) a focus group discussion to determine changes for the WLWL curriculum, 3) train-the-trainer sessions on program delivery, and 4) a pilot intervention and follow-up focus group to evaluate the new curriculum.The CAA/WLWL curriculum was adapted and piloted in a faith-based setting. Adaptations to the CAA program included pictures, games, statistics on cervical cancer, dialogues, and delivery of curriculum.Community engagement in the adaptation of WLWL through various methods was critical to tailoring an evidence-based program to a new population and setting. An estimated 12,340 cases of invasive cervical cancer are expected to be diagnosed and 4,030 deaths from cervical cancer are expected for 2013 , one-on-one education, provider assessment and feedback, and provider reminders as strategies to increase cervical cancer screening  program for Latino couples to increase knowledge about risk and behaviors leading to cervical cancer and HPV-related diseases. This 7-week intervention is implemented by trained promotoras, or community health workers, in faith-based and community organizations. Sessions last an average of 2 hours, and each session follows a specified curriculum covering topics on anatomy, sexually transmitted infections (STIs), cervical cancer and HPV, dialogues, role playing, presentation of skits to community or church members, and education about the HPV vaccine. An extensive 2-day train-the-trainer course is a vital component of the CAA program to train a team of community members in health ministries on how to deliver the curriculum to the faith-based community. Couples are targeted to participate in the program in efforts to reduce the anxiety of the male–female partner dialogue regarding HPV and risks associated with contracting cervical cancer. The program uses innovative tools, such as a “ring of knowledge” — where program participants collect small notecards with information throughout the course of the program — games to build awareness about STIs, anatomy labeling, and a “parking lot” for participants to ask personal questions anonymously.In response to the high prevalence of cervical cancer and human papilloma virus (HPV) among Latino women, the Spirit Foundation Inc, in partnership with the American Cancer Society, South Atlantic Division, created the With Love We Learn (WLWL).CAA has been implemented in Latino communities in Georgia, Nicaragua, El Salvador, and Bolivia. This intervention has been piloted in El Salvador with preliminary results demonstrating increased knowledge about HPV and Cervical Cancer . This arAfter successful implementation in multiple Latino communities, the Spirit Foundation Inc. considered it essential to replicate the program in the African American community in light of the high incidence of cervical cancer among African American women. Without appropriate education reinforcing the importance of screening and follow-up, the rates of cervical cancer will continue to rise among African American women. As it was in the Latino community, a faith-based setting was chosen for adaptation of CAA in the African American community because of the strong historical ties the African American community has with the church. Faith-based organizations have become one of the most common vehicles for the dissemination of prevention efforts and addressing health concerns for that population ,12.In October 2011, the Spirit Foundation partnered with 2 churches in the metro Atlanta area that would adopt WLWL in their health ministries. These churches have predominantly African American populations with congregations of 4,000 members or more . Health In 2011, the National Cancer Institute piloted the Research to Reality (R2R) mentorship program as a capacity-building initiative for public health practitioners to gain hands-on experience in evidence-based programs and decision-making practices. Mentees were paired with seasoned cancer control practitioners and worked on a year-long cancer control and prevention project to learn and apply new skills in evidence-based public health practices. The project for the Georgia R2R pair focused on the adaptation of CAA to WLWL. Orientation meetings were held with program developers and staff members to learn the details of the program, and worked for several months to gather information that would inform the key phases at which data would be gathered in the adaptation process. The adaptation summary form was the primary tool used among project staff to document the changes at each phase . The proSix health ministry leaders from the 2 partner churches reviewed the CAA program manual for general understanding of the overall curriculum. Reviews were conducted over a 2-week period, and leaders were provided with a deadline by the program developers to provide their initial feedback. Detailed notes were written throughout the manual on specific areas suggested for revision for the African American audience.Following the manual review, 6 health ministry leaders met at one of the partner churches for a 2-hour focus group. During the focus group, leaders and program developers discussed the detailed components of the curriculum and explained how each session was to be conducted. Participants talked through the notes written throughout the manual on sections that needed revisions. A group of stakeholders from academia, government, and community collaborated on developing the focus group guide to gather detailed information about key changes to the curriculum. There were specific inquiries were made about changes to the core content and pedagogical and implementation components . The guiAfter incorporation of the recommended changes, the WLWL manual was revised and a 2-day training workshop was conducted to train the 6 health ministry leaders on the revised curriculum. The objective of the train-the-trainer program for CAA/WLWL is to establish community educators who are motivated to address the community with accurate knowledge of HPV and cervical cancer. The training sessions were conducted on location to provide a familiar environment for the trainers. Refresher sessions were provided closer to program implementation to ensure that trainers remained comfortable teaching information accurately and did not deviate from the set topics of the curriculum.The first pilot of WLWL in the churches began in May 2013 with 15  participants. Participants were recruited through church announcements, fliers, and word of mouth from the health ministry leaders. Not all of the participants were couples, however all 15 participants completed the adapted program in June 2013. On completion, participants and trainers were invited to participate in a focus group discussion to gather their feedback. Participants were asked their opinions about the marketing of the program to their congregation, their comfort level with questionnaires that asked about sensitive and personal information, receptivity to the session topics and components, and their recommendations for recruiting participants in future programs. Data gathered from the pilot group of WLWL participants served as the final data collection point to include on the adaptation summary form for the last manual revisions.The overall program curriculum and the focus group guide was submitted under the Emory University Social and Behavioral Institutional Review Board and received exemption from full review. Trainers were compensated for their time for the manual review, focus group discussion, and train-the-trainer session. and participants in the pilot program were compensated for their feedback during the focus group. The R2R mentorship pair, program developers, and WLWL project staff followed adaptation guidance from the literature  to summaOverall, the program was well received by church leaders, trainers, and participants of the pilot intervention. Each phase of the adaptation process included valuable recommendations for the WLWL curriculum . SuggestProgram adaptation is an important process in the translation of evidence-based programs into practice. A systematic process for adaptation is necessary to learn what potential changes can be made to a program to ensure its suitability for another population and also balance fidelity to the original program. Community engagement is equally important for this adaptation process to garner insights from key constituents and understand contextual issues in matching interventions to particular communities. Castro and colleagues summarize key processes in program adaptation including assessing your audience, selecting an evidence-based approach, preparing for adaptation with focus groups or topical expert review, adapting the program, testing adaptation materials, and refining adaptation ,17. We fOne of the most important considerations for adapting community programs is to understand both the intervention and the components required for program fidelity to remain uncompromised. Additionally, it is equally important to take the time to foster the right partnerships with faith-based institutions and involve the right stakeholders’ perspectives to ensure the programs’ success. Substantial organization and investment of time are needed to collect data at multiple points. Working closely with the program developers and staff on organizing the data collection throughout the adaptation phases was extremely helpful in keeping the project on track and without losing the fidelity of the original program. Having a resource like the adaptation summary form referenced throughout this case study, is highly recommended to keep records of the changes in a multiphase process.The data collection and subsequent adaptation of the CAA program involved a variety of stakeholders from program developers to key members of the faith-based organizations to identify areas of original program mismatches  to the African American population. The multiple types of data collection generated many recommendations for program changes to increase the fit of the cervical cancer prevention program to African American church members. Many of our program changes fit into the area of context modification  and content modification  found in a recent review of adaptation of evidence-based interventions .We involved the program developers in this adaptation process in order to speak to the dynamics of the program, the role of the faith-based organization that would be implementing it, and to make informed decisions on what to change in the program. This strategy has been suggested as a way to ensure that the modifications are justifiable and that changes to the content, duration, or delivery style of the program will not diminish the program’s effectiveness ,20. LinkDissemination of evidence-based practices is increasing in the community. Many community organizations are attempting to adapt packaged programs for their own populations and settings. This project has illustrated a method for sequential data collection to inform the adaptation process. Future research should further explore how community-based participatory processes with key community members and organizations can inform program adaptations of evidence-based interventions and testing of adapted programs in communities for effectiveness.


Motor vehicle crashes decreased and seat belt use, including car seat use, increased in an American Indian and Alaska Native community through a multidisciplinary approach using strong partnerships among public health and law enforcement agencies; community outreach; mass media campaigns; and enactment and high-visibility enforcement of key laws, such as lowering the legal blood alcohol concentration limit for drivers and mandating use of occupant restraints. Motor vehicle crashes decreased and seat belt use, including car seat use, increased in an American Indian and Alaska Native community through a multidisciplinary approach using strong partnerships among public health and law enforcement agencies; community outreach; mass media campaigns; and enactment and high-visibility enforcement of key laws, such as lowering the legal blood alcohol concentration limit for drivers and mandating use of occupant restraints. Unintentional injury is a significant public health burden for American Indians and Alaska Natives and was the leading cause of death among those aged 1 to 44 years between 1999 and 2004. Of those deaths, motor vehicle-related deaths cause the most mortality, justifying the need for intervention at an American Indian Reservation in Arizona (United States). We describe motor vehicle injury prevention program operations from 2004 through 2013. This community-based approach led by a multidisciplinary team primarily comprised of environmental public health and law enforcement personnel implemented evidence-based strategies to reduce the impact of motor vehicle-related injuries and deaths, focusing on reducing impaired driving and increasing occupant restraint use. Strategies included: mass media campaigns to enhance awareness and outreach; high-visibility sobriety checkpoints; passing and enforcing 0.08% blood alcohol concentration limits for drivers and primary occupant restraint laws; and child car seat distribution and education. Routine monitoring and evaluation data showed a significant 5% to 7% annual reduction of motor vehicle crashes (MVCs), nighttime MVCs, MVCs with injuries/fatalities, and nighttime MVCs with injuries/fatalities between 2004 and 2013, but the annual percent change in arrests for driving under the influence (DUI) was not significant. There was also a 144% increase in driver/front seat passenger seat belt use, from 19% in 2011 before the primary occupant restraint law was enacted to 47% during the first full year of enforcement (2013). Car seat checkpoint data also suggested a 160% increase in car seat use, from less than 20% to 52% in 2013. Implementation of evidence-based strategies in injury prevention, along with employment of key program approaches such as strong partnership building, community engagement, and consistent staffing and funding, can narrow the public health disparity gap experienced among American Indian and Alaska Native communities. Between 1999 and 2004, unintentional injury was the leading cause of death among American Indians and Alaska Natives (AI/AN) ages 1 to 44 years in the United States.Unintentional injury is a leading cause of death among American Indians and Alaska Natives in the United States.Injuries—whether fatal or non-fatal—impose obvious impacts in AI/AN populations on the affected victims, families, and communities. However, injury can also have less obvious impacts. Due to the higher incidence of injury in younger age groups, fatal injury may involve premature death. Years of Potential Life Lost (YPLL) measures premature death before age 65 to illustrate lost potential productivity and income. Unintentional injury is the leading YPLL cause of death category for AI/AN, accounting for 28% of all YPLL causes of death, compared with 8% each for the next leading causes of heart disease and cancer.The Indian Health Service (IHS) has long recognized the problem that injury presents in AI/AN populations and has focused attention and resources on injury prevention activities. The US Centers for Disease Control and Prevention (CDC) has similarly recognized the injury problem in AI/AN populations and has collaborated with the IHS and with individual tribes. However, until relatively recently funding to support tribes to develop their own local infrastructures to address local injury problems based on the individual tribe’s needs was rare.,A focus of both the IHS Injury Prevention Program and the CDC’s National Center for Injury Prevention and Control is the implementation of effective injury prevention strategies that are evidence-based or that are considered best practices. Among AI/AN populations, appropriate attention has been focused on the prevention of motor vehicle-related injury. Risk factors for motor vehicle crash-related injuries and deaths in AI/AN communities include low rates of occupant restraint use and a high prevalence of alcohol-impaired driving.Risk factors for motor vehicle-related injuries include low rates of occupant restraint use and high prevalence of alcohol-impaired driving.In 2004, the San Carlos Apache Tribe in Arizona received funding from the CDC to develop, implement, and evaluate a multiyear program using such effective strategies to combat impaired driving. Administratively located within the San Carlos Police Department, this Motor Vehicle Injury Prevention Program (MVIPP) expanded in 2010, with funding from the IHS under the Tribal Injury Prevention Cooperative Agreement Program (TIPCAP), to include strategies to increase occupant protection. Funding under TIPCAP ended in August 2015. In this paper, we describe the specific program intervention components, provide data to illustrate the impact of the program 10 years after it began operation, and offer reasons for its success.The San Carlos Apache Indian Reservation is located in east-central Arizona, 110 miles east of Phoenix, Arizona. During the program period, there were an estimated 10,000 to 12,000 tribal members residing on the reservation’s 2,812 square miles. According to the 2010 census, the unemployment rate was 64%, and about 41.5% of the population lived below the poverty line.The San Carlos MVIPP was originally funded in 2004 as one of four tribal pilot programs nationally.Due to the demonstrated success of the MVIPP, in 2009 the San Carlos Police Department self-funded the program while it sought other funding options. The activities initiated during the 5 CDC-funded years continued during the 1 year of self-funding.In 2010, the MVIPP received funding from TIPCAP in the amount of $65,000 per year for 5 years. The TIPCAP program had a similar intent as the earlier CDC program, namely to develop injury prevention programs based on effective strategies or best practices. As part of the TIPCAP grant, the MVIPP expanded its efforts to include strategies to increase occupant protection.Following in approximate chronological order are the major MVIPP injury intervention components that have been accomplished from 2005 through 2013.The MVIPP developed a comprehensive media campaign in 2005 and 2006 that provided a foundation from which to operate for the duration of the program. Adopting a social marketing approach, the outreach made the public aware of the program and its activities. It used a variety of local media outlets to advertise its message about alcohol-impaired driving and occupant protection. These included paid advertisements in local newspapers, paid messages on the local cable access channel, messaging on the local casino marquee, and unpaid messaging via flyers provided to residents and posted on bulletin boards. Similar media outlets were used to advertise scheduled checkpoints. These culturally appropriate messages were derived from focus groups that we conducted with the public, and messages were advertised more frequently during tribal and national holidays in alignment with the Bureau of Indian Affairs Indian Highway Safety’s campaign and the National Highway Traffic Safety Administration’s campaigns.The motor vehicle injury prevention program in San Carlos used a variety of local media outlets to spread information about alcohol-impaired driving and occupant protection.An associated media campaign activity was the development and use of specific slogans and logos associated with the MVIPP. One logo  referencuse a sober driver.” Messaging was clarified because police officers discovered during the first few DUI checkpoints that non-sober drivers would state they were the least intoxicated occupant of the vehicle and were therefore designated the most likely to drive safely.Media messages evolved to ensure the right information reached the target audience to help promote safe behaviors. Initial “don’t drink and drive” messages were revised to “don’t drink and drive, The MVIPP participated in occasional community events, including health fairs, parades, and community meetings, to advertise the program and its activities.The MVIPP emphasis on impaired driving led to the development of a DUI task force, known as the 390 Task Force, which planned and organized the DUI checkpoints at various locations on the reservation. The high-visibility checkpoints used standard operating procedures for consistency, stopped all traffic during a specified time period to assess drivers’ levels of impairment, and involved the use of field sobriety tests and breath alcohol testers to assess alcohol impairment. The times of day and days of the week for the checkpoints were determined by anecdotal evidence contributed by the Task Force members and by police crash report data.A task force organized regular DUI checkpoints at various locations on the reservation.Some DUI checkpoints were conducted by San Carlos Police Department officers and support staff alone. Others were conducted with the assistance of regional enforcement partners, such as neighboring tribal police departments, neighboring city police departments, county sheriffs, the Arizona Department of Public Safety, and enforcement officers from the US Immigration and Customs Enforcement.When police officer staffing levels or competing work priorities precluded the staff-intensive DUI checkpoints, saturation patrols were used as an alternative to checkpoints. Saturation patrols involved increased patrols at high-risk times in high-risk areas to identify impaired drivers, as well as to enforce all traffic safety laws.Lowering the legal BAC limit for vehicle drivers from 0.10% to 0.08% is a recommended intervention to reduce alcohol-impaired driving.Lowering the legal blood alcohol concentration limit for drivers from 0.10% to 0.08% is recommended to reduce alcohol-impaired driving.The MVIPP coordinator used a variety of creative incentives—beyond overtime pay—to encourage and maintain police officers’ sustained participation and motivation in the program. Such incentives, valued by officers, included home-cooked meals before and during checkpoint events, awards of food and promotional items such as windbreakers and jackets to recognize exceptional participation and performance, an expense-paid trip to a national conference for the officer with the most DUI arrests in a calendar year, and recognition for outstanding performance in off-Reservation organizations such as Mothers Against Drunk Driving (MADD). These incentives were especially important during periods of shortages of police officers on staff.In November 2011, the San Carlos Apache Tribe enacted a primary occupant restraint law, to become effective in January 2012. The tribe was the fifth Arizona tribe to pass a primary law more stringent than the state’s secondary law. With primary laws, police officers may stop vehicles solely for seat belt law violations. In contrast, with secondary laws, officers must have another reason to stop a vehicle before citing an occupant for failing to wear a seat belt.The San Carlos Apache Tribe enacted a primary occupant restraint law, which allowed police officers to stop vehicles solely for seat belt law violations.As part of planning the implementation of the new primary occupant restraint law, the MVIPP and San Carlos Police Department leadership realized the public needed time to adapt to the new law. Simultaneous to a media campaign, the San Carlos Police Department began a 3-month enforcement grace period, during which officers would stop drivers who were not wearing a seat belt but would only issue a warning citation. Public awareness via media outlets continued beyond the 3-month grace period.Full enforcement of the new primary occupant restraint law began in late April 2012. At this time, anyone stopped by a police officer for not wearing a seat belt was issued a citation.Evaluation of the MVIPP was an integral part of its development. Several data elements were tracked throughout the operation of the program to measure its impact and are reported here. These included DUI arrests, total San Carlos Police Department-reported crashes, nighttime crashes, crashes involving injury or fatality, and nighttime crashes involving injury or fatality. A motor vehicle crash was any vehicle incident that involved a motor vehicle occupant, pedal cyclist, pedestrian, or other transport on a public highway, street, or road. Motorcycles, all-terrain vehicles, and other off-road vehicles, were not included unless a motor vehicle was also involved in the crash.The number of DUI arrests and the total number of crashes are self-explanatory. The injury crashes and nighttime crashes were tracked since alcohol-involved crashes are associated with greater likelihood of injury,P values were calculated.Joinpoint Regression Analysis Software models were used to calculate trend changes in these key data elements. Joinpoint is statistical software for the analysis of trends using joinpoint models, i.e., models where several different lines are attached at the "joinpoints." Joinpoint models linear and non-linear trend lines, which can be used to best determine significant variations in data. We report one joinpoint relevant to our DUI arrest data. For each trend segment, the annual percentage change (APC) and corresponding We also tracked seat belt use to measure the impact of the primary occupant restraint law, which became effective in January 2012. We observed driver and front seat passenger seat belt use according to the local IHS observational seat belt survey protocol. The protocol standardizes how surveys are conducted to ensure local seat belt use surveys are reasonably consistent and provides guidance for the selection of observation locations, survey procedures, and data summary and reporting.In 2013, 3 police-coordinated traffic checkpoints, which stopped all traffic at a particular location, provided an opportunity to obtain an estimate of car seat use for a sample of children in the stopped vehicles. It is important to note that these observations indicated only if a child was occupying a car seat; they did not assess car seat installation or optimal restraint use .Finally, knowledge, attitudes, and practices (KAP) surveys of the public were conducted in 2005, 2006, and 2008. KAP surveys were conducted to document the level of the public's support of a lower BAC limit, a primary occupant restraint law, enhanced enforcement of new laws, and perceptions related to DUI and occupant restraints. Questionnaires were designed and piloted locally then administered by trained surveyors who completed 1 survey in 5 minutes or less. Occasionally, participant incentives such as free bottles of drinking water or t-shirts were provided. KAP survey locations were selected based on the high likelihood of being able to find a population primarily comprised of tribal members . We surveyed 200 participants in 2005, 139 in 2006, and 140 in 2008. In addition, a cost-benefit analysis of the San Carlos MVIPP, that included the program years 2005 through 2008, was conducted by a third party to estimate the value of the program’s effects.P=.02) than the linear model, implying the DUI trend is not linear. The annual percentage change (APC) increased 9.08% per year until 2009, then decreased 14.95% per year afterwards, but neither change was statistically significant .Overall, the change in DUI arrests was non-linear—the trend for the first segment (between 2004 and 2009) was not significant nor was the trend for the second segment (between 2009 and 2013). The model with a joinpoint at 2009 is significantly better (P=.002), for nighttime MVCs -7.43 (P=.001), for MVCs with injuries/fatalities -5.37 (P=.01), and for nighttime MVCs with injuries/fatalities -6.89 (P=.02).However, the remaining outcomes regarding motor vehicle crashes were linear and significant, all decreasing about 5% to 7% per year. The number of total crashes, nighttime crashes, crashes involving injury/fatality, and nighttime crashes involving injury/fatality all generally dropped by substantial proportions from 2004. The APC for total MVCs was -6.34  to 47% during the first full year of enforcement (2013) . While tSeat belt use increased from 19% in 2011 to 47% in 2013.The tribe’s primary occupant restraint law also encompasses child car seat use. Efforts to assess the extent of car seat use prior to 2013 varied by methodology and venue. Many of these assessments involved voluntary car seat distribution or check events, which did not provide representative estimates of the population but found car seat use at less than 20%. In 2013, 3 police-coordinated traffic checkpoints that stopped all traffic at a particular location provided an opportunity to obtain a better estimate of car seat use for a sample of children in the stopped vehicles. In 2013, the combined observations from 3 checkpoints found that 52% of the children were in car seats. If we use 20% car seat use from previous years as the baseline measure, this reflects a 160% increase in car seat use since the occupant restraint law was enacted.KAP surveys revealed high levels of support for programs to prevent DUI. For example, in 2005 81% of KAP survey respondents were in favor of sobriety checkpoints, increasing to 93% in the 2008 KAP survey . Nearly The cost-benefit conducted by an independent third party of program years 2005 through 2008 concluded that the MVIPP showed a lifetime cost-benefit ratio of 1:9.86, meaning that for every US$1 dollar spent to implement the program, there was almost $10 in savings from reduced medical and other costs.The San Carlos MVIPP clearly demonstrates the application of evidence-based injury prevention strategies is feasible and has a positive impact on public health in a Native American community. Use of such effective strategies as instituting DUI checkpoints to enforce sobriety laws, lowering the legal BAC limit for drivers, and passing and enforcing primary occupant restraint laws significantly reduced motor vehicle crashes, including those involving injuries or fatalities, and increased occupant restraint use, including car seat use. In 2013, the data on motor vehicle crashes suggest an upward turn, which may be due to reduced police officer staffing at the San Carlos Police Department.Changes in DUI arrests over time were not statistically significant. However, the number of DUI arrests generally increased over the first several years of the project, but then the number declined or plateaued over the remaining project period. Possible explanations for this decline include that the San Carlos Police Department had significantly fewer sworn officers or that officers had multiple responsibilities other than traffic enforcement that competed with, and often negated, opportunities for involvement in DUI and/or occupant protection checkpoints. On the other hand, the decline in DUI arrests could be related to increased awareness among community members about the danger of DUI or the risk of DUI arrest leading to less impaired driving incidents.We believe several factors contributed to the program’s overall success:Extensive partnershipsTechnical expertiseProgram managed by a civilian employee within the police departmentConsistent staffing and fundingDUI task forceUse of incentivesCommunity and stakeholder supportAlthough these factors are valuable assets individually, all of them combined likely led to the program’s significant success. Partners included federal agencies , multiple law enforcement agencies , and a private marketing firm. A high level of local technical expertise existed. The public health staff involved with the program contributed expertise in the collection of qualitative and quantitative data, and in the management, analysis, interpretation, and presentation of that data. The law enforcement staff involved with the program contributed expertise in the development, enactment, and implementation of laws; with the enforcement of laws in general; and with specific enforcement operations such as DUI checkpoints.The establishment of the MVIPP was the result of proactive foresight from the police department to become involved in a “prevention” effort. This highlights the point that not all prevention programs should or must be managed by a public health program to be successful. The department also hired a civilian  full-time program coordinator who was able to dedicate all her talent and resources to implementing the interventions. Both the department and the coordinator possessed unwavering dedication to the project and to achieving its goals from the program’s inception.The creation of a DUI task force within the police department emphasizes the department’s dedication to the issue. Reliable financial resources, through the multiyear funding cycles, enabled the program to function at a level necessary to implement the effective strategies. This included money for police officer overtime pay and the use of funding sources outside the department to procure equipment used at sobriety checkpoints.,Many of these factors are consistent with what Fell et al.The support from tribal and police leadership was critical throughout the life of the program. Such ardent support was evident in the passage of the 0.08% BAC law and the primary occupant restraint law as well as the enforcement of these new laws. Support was also manifested in less obvious ways, such as police officer participation in injury prevention-focused training courses, conferences, and program activities such as DUI and occupant restraint checkpoints. We believe these interventions worked because they were championed by local leaders and because law enforcement and public health officials alike shared the common goal of reducing motor vehicle-related injuries, encouraging close collaboration on achieving this goal throughout the program period.Support from tribal and police leadership was critical to the success of the injury prevention program.-A novel element of the program’s data and monitoring strategy was not only to provide monitoring and evaluation updates to the funders but also to document the program’s approaches and share program successes and challenges with the broader public health community. For example, the program developed and presented several posters at the National Conference on Highway Safety Priorities – Lifesavers on topics ranging from overall program evaluation to unique program elements such as the police officer incentive program.Ensures data collection/analysis. Effective data management may be incomplete in under-resourced programs.Develops new skills. Improves written and oral communication skills, which provides long-term benefits for team members to better explain their mission and how they plan to achieve goals and overcome barriers.Reports back to stakeholders. Demonstrates program’s initiatives, successes, and challenges in different formats for local stakeholders/partners and for tribal leadership.Increases exposure. Allows local practitioners the opportunity to interact with highway safety and injury prevention colleagues from across the country.Enhances morale. Enhances sense of accomplishment and motivates partners, particularly when work such as conference posters is displayed in routine workspaces such as police officer patrol rooms or local hospital/clinic corridors, as well as markets the program to the public and to funding sources.A challenge that was overcome early in the program was the collection and management of data. Since the program coordinator received training in injury prevention and data management, she became the on-site data advocate within the police department. She worked closely with the officers to stress the importance of accurate and complete crash reports, and to emphasize the importance of the data they collect in measuring the impact of the MVIPP. She, in turn, collected and managed the collective crash data. This early emphasis on quality data resulted in the data component becoming a strength of the MVIPP. The data were used to demonstrate the program’s impact on several platforms: as feedback to the officers to reinforce the importance of their enforcement and data collection efforts, to police management and tribal leadership to demonstrate the value of the program in preventing injury, to the funding agencies to demonstrate the value of their investments, and to procure additional funds from multiple sources.Data variables are police-reported only, with no verification by health care sources. While such verification through health care sources may be ideal, it was beyond the scope of local resources to combine with local emergency room data. This was recognized in a 2010 report that surprisingly determined police-reported MVCs were a more reliable data source than accessible local health care facility data sources because many MVC victims were evacuated directly from the crash site to health care facilities that provided a higher level of care.A decrease in DUI arrests and increases in the 4 crash variables in 2013  coincideProgram sustainability beyond the time period we report in this paper is difficult to predict. However, continued collaboration among partners is likely and regardless of level of funding, interventions conducted may be scaled to meet local needs based on available resources .The San Carlos Apache Tribe’s Motor Vehicle Injury Prevention Program is an exceptional example that supports the "Community Guide’s" recommendations that a carefully planned, well-executed, multicomponent program, implemented in conjunction with community mobilization efforts, can reduce motor vehicle crashes and increase occupant restraint use. Key factors that contributed to the program’s success include strong partnerships between public health and law enforcement agencies, consistent financial resources over multiple years, and high-visibility enforcement of impaired driving laws.


The Mental Health Emergency Care-Rural Access Program (MHEC-RAP) is a telehealth solution providing specialist emergency mental health care to rural and remote communities across western NSW, Australia. This is the first time and motion (T&M) study to examine program efficiency and capacity for a telepsychiatry program. Clinical services are an integral aspect of the program accounting for 6% of all activities and 50% of the time spent conducting program activities, but half of this time is spent completing clinical paperwork. This finding emphasizes the importance of these services to program efficiency and the need to address variability of service provision to impact capacity. Currently, there is no efficiency benchmark for emergency telepsychiatry programs. Findings suggest that MHEC-RAP could increase its activity without affecting program responsiveness. T&M studies not only determine activity and time expenditure, but have a wider application assessing program efficiency by understanding, defining, and calculating capacity. T&M studies can inform future program development of MHEC-RAP and similar telehealth programs, both in Australia and overseas. The application of telehealth technologies for mental health care is not new and has been used to provide various clinical services and support, supervision and learning, and administrative activities from a distance for decades both in Australia and abroad ,6,7,8,9.The provision of specialist mental health care in rural and remote communities is hindered by vast distances, geographic isolation, and ongoing workforce shortages ,17,18,192 or 55% of the state ; a 24/7 rural emergency telepsychiatry program, is contacted through the state freecall mental health telephone access line (1800011511) and aims to improve access to specialist emergency mental health care ,26. It hhe state .Evaluation findings have already been reported on service activity and program reach ,28. The A T&M study was conducted to assess the time taken to complete MHEC-RAP activities and to use these data to inform program development. This was part of a larger study that aimed to describe the structure of MHEC-RAP, how it functions, its processes, and efficiency . T&M stuThe “STAMP” (Suggested Time And Motion Procedures) method was used to report our findings. STAMP is a checklist developed for T&M studies to improve consistency in research and reporting covering areas from the intervention, design, and analysis to the observer and subject . For thiThe study took place over 15 consecutive days in April 2013 at the MHEC-RAP facility on the Bloomfield Campus in Orange, NSW. The observations were conducted by ES as part of her PhD Candidature. She was also part of an initial study of MHEC-RAP in 2008 during which time she was co-located with the MHEC-RAP team; this provided useful background knowledge of the program . ES onlyAt the time of the study, MHEC-RAP operated using a 12-h shift schedule with two clinicians rostered on each shift (day/AM and night/PM). A third clinician worked on an overlapping shift each day between 1300–2100. All 17 members of the MHEC-RAP team were invited to participate in the study, including casual staff. As MHEC-RAP is a 24/7 program, observations were scheduled to accommodate non-consenting staff and to ensure observation of day and night shifts across all days of the week. ES observed seven 12-h shifts (3 day/4 night) and four morning sessions (0700–300) on 11 of 15 (73%) possible days during the study period.Data were collected in the form of detailed notes (who was acting and what they were doing) and timed activity records . Activity data were also extracted from the MHEC-RAP call logs and the clinical service dataset for the study period. All notes, activity data, and times were entered on a form created for the purpose, and transcribed into EXCEL spreadsheets for future analysis. Timed activity records were from the initiation of the activity and included all aspects of the activity  through to the completion of the case. For activities that involved more than one rostered clinician, a summary measure of person-time was calculated.clinical services (the telephone triage and video assessments),information services , andother program activity .For these analyses, MHEC-RAP activities were grouped, based on findings from the observational study, into:Geometric means and medians were calculated for all analyses because MHEC-RAP activity data were skewed. The Geometric mean normalizes the range of numbers. The median is the middle value of the range and is a robust statistic to report with skewed data.Three clinical service activities (two telephone triage only and one with video assessment) were managed across shifts so we used mean substitution for the unobserved tasks to calculate the total activity time. Actual time spent on activities during a shift was compared with an estimate based on the geometric mean times to identify outlier observations for more detailed analysis and to inform the use of T&M findings to estimate program efficiency.The total available time for MHEC-RAP service provision was calculated as follows: for each 12-h shift two hours were subtracted for meal and work breaks for each clinician, and one hour for the 8-h shift. This resulted in 2820 min of available time for MHEC-RAP service provision each 24-h.Program efficiency was estimated for the study period (to compare AM and PM shifts) and for a 12 month period using 2011 activity data which was previously collected for the evaluation ,28. ThesThere were 299 activities observed during the study period—18 clinical services, 239 information services, and 42 other program activities. Information services accounted for 80% of overall activity in terms of numbers, but only 21% in terms of time with each telephone call lasting 2–3 min on average. The clinical services accounted for 6% of all activities and 50% of the time for those activities.It took 65 min on average to complete a telephone triage, increasing twofold to 126 min when a video assessment was organized as well. Time required for the clinical care component ranged from a minimum of 8 min for telephone triage only to a maximum of 96 min with a video assessment, averaging 17 and 47 min respectively. Arranging care required 4 to 5 min on average, however in one instance this took 51 min. Completing the associated clinical paperwork accounted for more than half the time taken to deliver a clinical service—63% for telephone triage only and 52% with a video assessment .Case reviews were conducted each weekday (AM shift) and involved the supervising psychiatrist, the two rostered MHEC-RAP clinicians, and the Nurse Unit Manager. This activity required 81 person-min on average. Clinical handover occurred twice a day between shifts, and took 25 person-min on average for the AM shift and 13 person-min for the PM shift.10df = 2.35, p < 0.05). We reviewed the four shifts whose actual times exceeded the predicted values by more than 50%. For three, between 57% and 100% of the difference was accounted for by the additional time spent completing paperwork in excess of the average, and for the fourth, which had only one clinical service, all of the difference was attributed to a lengthy case review and other program activities.We compared the time spent on activities during the observation period with an estimate based on the mean times . The preThe estimated time spent providing MHEC-RAP activities varied between the AM and PM shifts during the study period . This reIn 2011, we estimated that a median of 26% (range 7%–61%) of the available time was spent providing MHEC-RAP activities on a daily basis. The estimated time spent on MHEC-RAP activities exceeded 50% of the available time for only 5 days in 2011. For 175 days that year, less than a quarter of the available time each day was spent providing program activities .This T&M study provided valuable information, not otherwise available, about the amount of time required for MHEC-RAP activities and program efficiency. It emphasized the importance of telephone triage and video assessment activity in relation to program efficiency (mean time required of 65 min and 126 min respectively for each activity). Even at low levels of activity they accounted for the majority of time spent on service delivery and thus represent the most important determinant of program efficiency. One area that would benefit from more efficient service delivery is the completion of paperwork, which accounted for at least 50% of the time required for a clinical service. This activity showed significant variability and has the potential to impact program performance if it is not effectively managed.The T&M study findings have wider application. During 2011, the estimated time required for MHEC-RAP activities accounted for about one-quarter of the available time and only exceeded half of the available time on five days that year. This indicated that the program had capacity to support a higher level of clinical service activity. Currently, there is no efficiency benchmark for emergency telehealth mental health care programs. T&M studies could be used to derive benchmarks for efficient service delivery and inform the development of similar telehealth programs, both in Australia and overseas. Like other emergency services, MHEC-RAP must manage efficiency against capacity and based on our data, an initial increase in activity to achieve a median of 40% of the available time in active service provision should be possible without adversely affecting program responsiveness. Increased activity would result in greater program efficiency and further address the aim to improve access to the service. For MHEC-RAP, this could be achieved by promoting greater use of the program within the existing service footprint or extending the program reach to additional communities in adjacent regions.Future research could provide more detailed breakdown of the different elements of MHEC-RAP activities including the time taken to coordinate the admission and transportation of patients requiring specialist care in a Mental Health Inpatient Unit, for supervision and support of generalist staff to manage patients in the local ED, and support of community mental health staff in smaller centres. Additional investigations could review MHEC-RAP against the standards for NSW Health Mental Health Telephone Triage Services such as the time to answer a call and compare the time taken to complete telephone triage and video assessment by referral source to provide a greater understanding of program performance .The design specifications for similar telehealth programs, including geographical reach and minimum staffing levels, influence demand for the service and their capacity to manage workload. Nonetheless, getting program efficiency right is primarily a function of maximizing the number of emergencies referred to the service, on average, over an extended period, with adjustments for the variability in numbers on a daily basis to maintain service responsiveness. Further work is required to generate metrics that would enable benchmarks to be established and calculations to be made that set realistic program efficiency targets and assist service planning.The study is based on a small number of clinical services collected during a period of low program activity which reduced the precision of study estimates. The lack of time pressure on some shifts may have influenced the amount of time taken to complete certain activities such as the completion of paperwork, resulting in prolonged times. Program efficiency measures were derived from routinely collected data accessed through MHEC-RAP service records for 2011. Data were limited to a daily count of clinical activities and a summary of in-coming and outgoing calls on a monthly basis restricting our analysis to daily measures of program efficiency and potentially reducing the variance of those daily estimates.The use of well accepted and robust T&M methods, based on independent observation of activity and administered by one observer, are study strengths that support the validity of the findings.MHEC-RAP is not the first telehealth service to be developed in Australia or overseas, but it is the first model to combine 24-h availability and access to regionally-based emergency specialist care using telehealth technology. The T&M study provided valuable information, not otherwise available, about the amount of time required for MHEC-RAP processes and program efficiency. The study emphasized the importance of telephone triage and video assessment activity in relation to program efficiency and that the program had capacity to support a higher level of clinical service activity. The T&M study findings have wider application and could be used to derive benchmarks for efficient service delivery and inform the development of similar telehealth programs, both in Australia and overseas.


Background: Intestinal stricture is an important complication of necrotising enterocolitis (NEC). We aimed to describe clinical profile and identify the risk factors for post-NEC intestinal strictures. Method: A retrospective study of infants with NEC over 10 year period. Results: Of the 61 infants with NEC, 18 (29.5%) developed intestinal strictures. Leucocytosis and longer length of bowel resection during acute stage of NEC was associated with a later diagnosis of intestinal stricture. Infants with NEC who did not develop stricture had non-specific intestinal dilatation on abdominal x-ray during acute NEC. Intestinal strictures were diagnosed at a median interval of 34 days after NEC. Majority of strictures (67%) occurred in the colon. A significant proportion (77%) of infants with intestinal stricture had associated co-morbidities. No mortality occurred in infants with intestinal strictures. Conclusion: The incidence of post-NEC intestinal stricture is high but development of stricture is difficult to predict. Leucocytosis during NEC and length of bowel resected at surgery may be associated with development of post-NEC intestinal stricture. A substantial number of infants with post-NEC intestinal stricture fail to thrive, have co-morbidities and need prolonged hospitalisation. Neonatal necrotising enterocolitis (NEC) is one of the most common surgical emergencies in neonatal intensive care units (NICU) with a reported incidence of 2.6% to 6.6%, depending on the gestational age of infant at birth . Most cases of NEC are managed conservatively and surgical intervention is required in infants with intestinal perforation or deteriorating clinical or laboratory parameters despite optimum medical treatment . With improved survival of many seriously ill infants with NEC, a substantial number of infants present with intermediate or long-term sequelae [5]. Intestinal stricture following NEC is a well-known complication of the healing process that follows ischaemic injury to the inner muscular layer of the intestine, during the acute episode [6]. Intestinal strictures develop irrespective of the mode of management of the acute episode of NEC and tend to have a strong preference for the left colon  [3]. Strictures manifest at a variable period after the acute NEC. Several studies have described significant mortality and morbidity secondary to intestinal strictures . However, there is paucity of recent literature describing the natural history of intestinal strictures following NEC [8]. The aim of this study was to describe clinical profile, complications and outcomes of post-NEC intestinal strictures in a surgical neonatal unit over a period of 10 years and to identify potential risk factors during the acute NEC episode that could predict development of post-NEC intestinal stricture.This was a retrospective study of infants who were treated for NEC in a neonatal unit of a children’s hospital from 1st January 2000 to 31st December 2010. All infants admitted to the neonatal unit during the study period with a diagnosis of NEC were eligible for inclusion. The study population was identified from the hospital’s electronic database by quizzing the database with search terms “Necrotising Enterocolitis”, “NEC” “Stricture” and “Intestinal Stenosis”. The infants with congenital anomalies and infants who died during acute NEC were not included in the study. The electronic medical records and discharge summaries were reviewed. Demographic information, clinical indicators of severity of NEC such as haematological investigations, findings on abdominal x-ray, mode of management of the acute episode of NEC, length of bowel resected at surgery and histopathology of resected bowel segment were collected. All the abdominal X-rays were reported by a paediatric radiologist. The infants were classified into Stage I, Stage II or Stage III NEC using the modified Bell’s criteria. For the infants who had post-NEC strictures, age at the diagnosis of stricture, time interval since diagnosis of acute NEC, weight at diagnosis of stricture, signs at presentation, results of intestinal contrast study, number and site of strictures, complications attributable to strictures, mode of management of the stricture, time required to attain full enteral feeds and the length of hospital stay were recorded. All Infants with NEC were followed up in the hospital’s outpatient clinic until at least 3 months of corrected age. Ethical approval was granted by the local Human Research Ethics Committee. After the database was accessed and the medical records examined, de-identified study data were recorded electronically and password protected by the primary investigator.Statistical Analysis:Data was analysed using SPSS software  Chicago, IL, USA. Chi Square test, Fisher’s Exact test and Mann Whitney test were used as appropriate. A p-value of less than 0.05 was considered statistically significant. We identified 72 infants with NEC during the study period. 11 infants died due to NEC and were excluded. 61 infants  with NEC were included in the study. The baseline characteristics of infants with or without strictures were similar (Table 1). Modified Bell’s stage I NEC occurred in 15/61 (24.5%) infants, Stage II in 22/61 (36%) and Stage III in 24/61 (39.5%) infants. Surgical intervention was required in 28/61 (46%) infants and 33/61 infants (54%) were managed conservatively. Intestinal strictures developed in 18/61 (29.5%) infants. The incidence of stricture was 35.5% (10/28) in the surgically managed subgroup and 24% (8/33) in the conservatively treated subgroup. The infants who developed intestinal strictures were more likely to have an elevated white cell count (> 25000/cmm) at the time of acute episode of NEC (p = 0.01) (Table 2). Infants with Bell’s stage I disease and non-specific intestinal dilation on x-ray were less likely to develop a stricture (p = 0.05 and 0.02 respectively) (Table 3). Of the intestinal perforations during acute NEC (17/61 infants) caecal perforation posed a borderline risk for development of strictures (P=0.05). In infants that required surgical intervention for NEC, longer length of resected bowel was significantly associated with development of strictures (p=0.02) (Table 3).In our series of 18 infants with intestinal strictures, the median time interval between NEC and diagnosis of the intestinal stricture was 34 days (range 27.5-62 days). In 7/18 (39%) infants the strictures were diagnosed by the routine contrast study prior to reversal of stomas to re-establish gastro-intestinal continuity. The remaining 11 infants had symptoms of intestinal obstruction and the diagnosis of intestinal strictures was established on a contrast study.In 12/18 (67%) infants the strictures were located in the colon and in 6/18 (33%) the site was small bowel. Amongst the isolated colonic strictures, four were located in the transverse and descending colon each and one in the ascending colon. Three infants had strictures involving more than one segment of colon. In six infants with stricture of the small bowel, five were located in the terminal ileum and one in the proximal ileum. All small bowel strictures were isolated. The complications noted in infants with intestinal strictures were intestinal perforation (2/18) bacterial sepsis (2/18), severe electrolyte imbalance in (4/18), hypoalbuminemia requiring albumin transfusion (8/18) and significant cholestasis in (9/18). Seven infants with intestinal stricture had growth failure with weight that dropped two major centile lines to well below 10th centile between the diagnosis of acute NEC and the diagnosis of stricture. Median time required to attain full enteral feeds after repair of stricture was 25 days (range 14-67.7 days) and the median duration of hospitalisation for stricture and related issues was 33 days (range 21-81days). No deaths occurred in the infants with strictures in our cohort.Intestinal strictures following NEC in neonates were first reported by Rabinowitz et al in 1968.[9] There is paucity of studies looking at risk factors for post-NEC intestinal strictures [8]. With improved perinatal care in the last 20 years we sought to evaluate the pattern of post-NEC intestinal strictures. Our study had a sizeable number (18) of infants with intestinal stricture in comparison to the previous studies [5-11]. Our findings of the incidence of stricture, the median time to diagnosis of stricture since the acute episode of NEC, location of strictures and lack of significant association between development of post-NEC stricture and mode of management of acute NEC were similar of the previous reports . The intestinal strictures result from collagen deposition, fibrosis and wound contraction over a variable period following an episode of acute inflammation. We examined various markers of inflammation and severity of illness at the time of acute NEC. We observed significant association between presence of high white cell count during acute NEC and length of bowel resected at NEC surgery and the occurrence of intestinal strictures. These findings have not been described in the literature previously. They could be of prognostic significance and are biologically plausible. We hypothesize that presence of large number of white cells resulting in heightened immune mediator response locally may precede development of stricture and more length resection to be marker of more extensive disease . Contrastingly, level of C- reactive protein (CRP) that has been reported as a part of a scoring system to predict development of stricture did not attain prognostic relevance in our study.[16] Thrombocytopenia and metabolic acidosis at the time of acute NEW were not predictors of later strictures. Growth failure and life threatening complications of strictures such as intestinal perforation and sepsis are well described in literature . We found severe electrolyte abnormalities, hypoalbuminemia, significant derangement of liver functions as associated morbidities in infants with intestinal stricture. Our observation of morbidity associated with intestinal stricture (in more than three fourths of infants) was higher than that reported by Janik et al (30%) [7]. This difference may be due to more variables studied in our study population and comparatively larger infants in the report by Janik et al. Mortality ranging from 8-25%, secondary to intestinal stricture related problems such as intestinal perforation, aspiration pneumonia and sepsis in the peri-operative period, has been reported in the literature [3’7]. We did not observe any mortality in infants with strictures. This difference could be due to improved neonatal care in the last two decades.This is a retrospective study from single neonatal unit with a relatively small sample size and might not be powered to detect small differences between the study groups. Infants included in this study were referred from other neonatal units for surgical evaluation and could represent the severe end of NEC spectrum making the observations not applicable to perinatal settings that medically manage infants with NEC.Our study offers a recent insight into the impact of improved care on the clinical profile and outcomes of post-NEC strictures. The overall incidence of intestinal strictures has remained high and similar irrespective of the mode of management of NEC. Prediction of development of intestinal stricture following NEC is difficult. However, leucocytosis during acute NEC and the length of bowel resected at surgery could be potential predictors of post-NEC stricture. A substantial number of infants with strictures failed to thrive; the majority had morbidity and required prolonged hospitalisation. Implications for current Practice:All infants with confirmed NEC should be observed closely and investigated for stricture in the presence of any symptom of persistent gastrointestinal disturbance irrespective of initial mode of management. Physical growth, liver functions and serum electrolyte levels of infants with post NEC strictures should be closely monitored and nutrition proactively managed. Source of Support: NoneConflict of Interest: None