Polystyrene nano- and micro-sized plastic particles (NMP) are one of the common plastic materials produced that dramatically pollute the environment, water, and oceanic habitats worldwide. NMP are continuously absorbed by the body through a number of routes, especially via intestinal ingestion, dermal uptake, and inhalation into the lung. Several studies provided evidence of NMP provoking oxidative stress and affecting cellular responses. Yet, the NMP effects on primary lung cells have not been studied. To this end, we isolated and cultured murine lung cells and exposed them short-term or long-term to polystyrene 0.2–6.0 µm-sized NMP. We studied cellular consequences regarding oxidative stress, morphology, and secretion profiling. Visualization, distribution, and expression analyses confirmed lung cells accumulating NMP and showed several significant correlations with particle size. Moreover, we found substantial evidence of biological consequences of small-scale NMP uptake in lung cells. Besides alterations of cytokine secretion profiles resulting in inflammatory responses, indicators of oxidative stress were identified that were accompanied by Nrf2 and β-catenin signaling changes. Our results serve as an important basis to point out the potential hazards of plastic contaminations and uptake in lung cells. Plastic diversity and versatility make its application indispensable across all areas of everyday life. This, of course, goes hand in hand with its ever-increasing role as a pollutant in aquatic systems and air, recognizing plastic as a serious global environmental concern ,2. EmergDue to their small size and strong biological penetration, NMP are easily ingested by organisms, causing severe damage to biological functions on a single-cell level ,10. AddiIn our study, the mechanisms and effects of the accumulation of fluorescence-labeled polystyrene NMP were monitored in cells freshly isolated from mouse lungs. Polystyrene was selected as it is a commonly produced and used plastic type and one of the primary pollutants found in the environment, which can remain there for a long time . Moreove2. In our experimental setups, early passages from 1 to 4 were used.The study was conducted in accordance with the regulations of the local ethics committee of the State of Mecklenburg-Vorpommern  and the guidelines for the care and use of laboratory animals. It was performed according to the recommendations of Good Laboratory Practice and within the guidelines and regulations of animal care guidelines in SKH1-hr hairless immunocompetent mice . Primary cells were isolated from the lungs by enzyme-mediated digestion according to the recommendations of a dissociation kit. The cell suspension was homogenized in gentleMACS C tubes using a gentleMACS dissociator for obtaining live cells . To separate individual cells, the suspensions were passed through 40 µm MACS SmartStrainers (Miltenyi Biotec). Lung cells were spun down. The resulting mix of lung cells, including pulmonary alveolar epithelial type I and type II cells, were cultured over a total of four weeks in complete Dulbecco’s Minimum Essential Medium  supplemented with 10% fetal bovine serum, 1% penicillin/streptomycin, and L-glutamine  in a humidified incubator at 37 °C with 5% COThe NMP used in this study were the fluoresbrite (FB) polystyrene microspheres 0.2 , 1.0 , 2.0 , and 6.0  , and polymer microspheres mix  in aqueous suspension . The hydrodynamic diameter was monitored with a dynamic light scattering system . Stocks were sonicated with a bath sonicator and strongly vortexed before use to avoid aggregation of NMP. Lung cells were incubated with NMP in corresponding cell culture dishes dependent on downstream applications ranging from 6 cm dishes for RNA isolation and protein lysates to 96 well plates for metabolic or viability measurements. Environmentally relevant concentrations of NMP are in the range between 10 and 1000 µg/mL to facilitate studies on biological effects ,29,30. I3 cells were NMP-treated, incubated for 24 h in a 96-well plate, and washed with PBS. Hydrogen peroxide  was used as a positive control. After staining with H2-DCF-DA  at 37 °C for 1 h, fluorescence intensity was assessed, which is associated with the amount of ROS formed. Cells were imaged by fluorescence microscopy . By flow cytometry, the thiol content was analyzed with a thiol-sensitive reagent , and the granularity of NMP-treated lung cells was determined using the side scatter (SSC) metric and compared to that of untreated cells. NMP-treated lung cells were incubated with resazurin . Resazurin is reduced to fluorescent resorufin by metabolically active cells, and the fluorescence was determined after 2 h using a microplate reader  at λex 535 nm and λem 590 nm. Afterward, the viability of lung cells was determined via live–dead analysis. Briefly, cells were stained after NMP exposure with FITC-Annexin V  along with 4′,6-diamidine-2-phenylindole  at 37 °C for 30 min. After washing with cold FACS washing buffer, samples were measured using flow cytometry . Data analysis was performed using Kaluza analysis software 2.1.3 (Beckman-Coulter).DCF staining was applied to measure intracellular oxidative stress levels. For this, 5 × 10GAPDH and RPL13A, were used as an internal normalization control. Gene expression was analyzed using the ∆∆CT method. For protein analysis, NMP-treated cells were lysed in RIPA buffer containing protease and phosphatase inhibitors . The protein expression levels of Nrf2, HO-1, Nqo1, Keap1, Cat, Sod1, β-catenin, and β-actin were determined using corresponding antibodies  and the WES system together with its Simple Western Software  according to the manufacturer’s instructions. Gapdh served as housekeeping control. Band intensities were quantified as fold change compared to the untreated control.After the lysis of cell NMP-treated samples in RNA lysis buffer, total RNA was isolated according to the manufacturer’s instructions , and mRNA expression levels were quantified by quantitative real-time PCR (qPCR). Briefly, 1 μg of RNA was transcribed into cDNA, and qPCR was conducted in triplicates using SYBR Green I Master . Murine gene-specific primers  were useLung cells were seeded on glass coverslips and exposed to NMP after one day. NMP uptake was studied with a live-cell high-content imaging system , and algorithm-driven image quantification was performed using dedicated imaging software (Harmony 4.9). For immunofluorescence microscopy of samples, cells were fixed in 4% paraformaldehyde  for 20 min and permeabilized with Triton X-100 . Fixed cells were incubated with primary antibodies: Nrf2, β-catenin, HO-1 , and FITC- or FR- (flash red) labeled phalloidin for actin cytoskeleton staining (both BioLegend), followed by staining with secondary antibodies conjugated to Alexa Fluor 488 for red NMP or Alex Fluor 594 for green NMP, respectively . DAPI was used for nuclear counterstaining, and samples were mounted onto glass slides  prior to fluorescence microscopy using an Axio Observer Z.1 (Zeiss).Using multiplex cytokine detection technology , cytokine profiles were measured in supernatants of lung cells exposed to NMP according to the manufacturer’s instructions. The bead-based sandwich immunoassay was measured using flow cytometry  targeting tumor necrosis factor-alpha (TNFα), interferon (IFN) α and γ, monocyte chemotactic protein (MCP) 1 (or CCL2) and chemokine (C-X-C motif) ligand (CXCL) 9, and nine interleukins . Appropriate data analysis software  was utilized for target quantification.Student’s t-test for comparison between two groups with p-values indicated by * p < 0.05, ** p < 0.01, and *** p < 0.001, as in figure legends. All analyses were Pearson-correlated against NMP sizes using prism software.Data are presented as mean + SE of at least three independent experiments. Graphing was completed using Prism software 9.5.0 . Statistical analysis was performed by one-way analyses of variants (ANOVA) for multiple group comparison and 2O2-treated lung cells , and the cell-cycle-related gene p21. p53 and BAX expression tended to increase after single and repeated NMP exposure for all particle sizes. In contrast, the observed lack of alterations of p21 after single incubation has reversed with increasing duration of NMP exposure except for 2.0 µm particles. Hypoxia-inducible factor 1α (HIF1A) expression was increased after single treatment for nano-sized particles but nearly unchanged for micro-sized particles and after repeated NMP exposure  or repeatedly for four weeks. Uptake, distribution, toxicity, metabolic activity, secretory pattern, and changes in downstream signaling were observed and analyzed using a broad spectrum of methods a. Green ng cells b. In conng cells c. The inng cells c, suggesng cells d to be oexposure e, indicaNrf2, heme oxygenase 1 (HMOX1), and NAD(P)H quinone oxidoreductase 1 (NQO1), two antioxidant proteins, as well as Keap1, a redox-regulated substrate adapter protein, which were modestly regulated in an NMP size-dependent fashion after single and repeated treatment (GSTA1) expression level was significantly increased, suggesting the protection of cells from ROS and peroxidation products in response to NMP treatment. Due to the antioxidant activity of several downstream targets of Nrf2 to protect cells from ROS-induced oxidative stress, we quantified the expression levels of catalase (CAT) and superoxide dismutase 1 and 2 (SOD1/2), which tended to be upregulated in an NMP size-dependent fashion. By contrast, glutathione peroxidase 2 (GPx2) was significantly down-regulated after single and repeated NMP exposure. In turn, glutathione reductase (GSR) was down-regulated upon single (1 week) NMP treatment but profoundly upregulated when cells were exposed over 4 weeks to NMP particles. Moreover, qPCR analysis of several targets such as thioredoxin 2 (TXN2), sulfiredoxin (SRXN), heat shock proteins (HSP) 70A and 90, and nitric oxide synthase 2  was divergently regulated upon NMP exposure, indicating a regulation of these targets through a redox-sensitive mechanism. The protein expression levels of Nrf2 were altered similarly to Nrf2 transcript levels. Other downstream targets were more modestly changed, except for Cat, Sod1, and GPx1, which were mostly upregulated after single or repeated NMP exposure -Keap1 signaling a in modereatment c. At theexposure d. Nucleaexposure e.ACTB mRNA (data not shown), as well as the staining of the cytoskeleton, was unchanged after single NMP exposure  gene and protein level after single NMP treatment, in contrast to repeated exposure. This was accompanied by a slight decrease in E-cadherin (CDH1) expression after single treatment in contrast to repeated exposure, along with a strong increase of WNT1 and WNT7a expression. Phosphorylated glycogen synthase kinase (GSK) 3β promotes the phosphorylation of β-catenin at key Ser/Thr residues by modulating its degradation and nuclear translocation [GSK3β was comparable between NMP-treated cells and controls, with a slight increase for 0.2 µm-sized NMP exposure. Consistent with previous data, a significant decrease in GSK3β mRNA was identified in repetitive NMP-treated lung cells. After nuclear β-catenin accumulation, it acts as a coactivator for transcription factors of the T-cell/lymphoid enhancer-biding factor (TCF/LEF) family. LEF1 and c-JUN, as further activators downstream of β-catenin stabilization, showed similar results in single- and repeated-treated lung cells with slightly increased mRNA levels in the NMP-treated lung cells (CLDN1) and vascular endothelial factor (VEGF) were significantly elevated in NMP-treated lung cells, particularly after single treatment and independent of particle size. For other molecules, such as the lipid-sensitive nuclear peroxisome proliferator-activated receptor δ (PPARD), mRNA expression increased after single treatment and decreased after prolonged NMP exposure, except for 0.2 µm NMP , IL6, and VEGF after single NMP exposure in all experimental groups. The expression of the tumor necrosis factor TNFα, another mediator of an inflammatory response, has shown a slight decrease except for the NMP mix  in humans , particuThe composition, size, and shape of NMP significantly influence their retention in the lungs and targeting properties. Plastic particles with fragmented or fiber shapes are more abundant in environmental exposure scenarios than spherical particles of controlled size . However3 [3 [2 for inhalation treatment experiments of pulmonary cancer [Due to the lack of analytical techniques, little information regarding airborne NMP is available. Previously, an indoor air concentration of 3–15 particles/m3  and 0.4–3 [3  was foun3 [3  and was y cancer . Studiesy cancer ; on humay cancer ; in hepay cancer , and inty cancer ; in zebry cancer ; and in y cancer . Neverthy cancer  or at 12y cancer . To faciAccording to previous studies, exposures lasting for at least four weeks can be considered long-term (chronic) exposure ,59,60, wP. nana [The penetration of the cell membrane in response to cellular NMP uptake is strongly linked to biological responses on the single-cell level, such as cytotoxicity, apoptosis, cellular damage, and activation of signal transduction . SeveralP. nana . A recenP. nana . GeneralP. nana . IntriguP. nana . AdditioP. nana . The repP. nana . It was P. nana . The duaP. nana . In a meP. nana , colorecP. nana , and nonP. nana . In thisIn conclusion, due to our findings, NMP accumulation in murine lung cells supports intracellular ROS formation along with modest alterations in cytotoxicity and apoptosis. In contrast, NMP exposure changed the transcriptional signature of the Nrf2-driven oxidative stress response and β-catenin signaling along with modulation of inflammatory mediators  and cytoskeletal factors , particularly following repeated NMP deposition. To obtain more data on toxic NMP effects, further in vivo studies using environmental plastic mixtures must be conducted.


The Stroke Recovery in Motion Implementation Planner guides teams through the process of planning for the implementation of community-based exercise programs for people with stroke, in alignment with implementation science frameworks.The purpose of this study was to conduct a field test with end users to describe how teams used the Planner in real-world conditions; describe the effects of Planner use on participants’ implementation-planning knowledge, attitudes, and activities; and identify factors influencing the use of the Planner.This field test study used a longitudinal qualitative design. We recruited teams across Canada who intended to implement a community-based exercise program for people with stroke in the next 6 to 12 months and were willing to use the Planner to guide their work. We completed semistructured interviews at the time of enrollment, monitoring calls every 1 to 2 months, and at the end of the study to learn about implementation-planning work completed and Planner use. The interviews were analyzed using conventional content analysis. Completed Planner steps were plotted onto a timeline for comparison across teams.tips from the field to help future teams optimize use of the Planner.We enrolled 12 participants  from 5 planning teams. The teams were enrolled in the study between 4 and 14 months, and we conducted 25 interviews. We observed that the teams worked through the planning process in diverse and nonlinear ways, adapted to their context. All teams provided examples of how using the Planner changed their implementation-planning knowledge , attitudes , and activities . We identified team, organizational, and broader contextual factors that hindered and facilitated uptake of the Planner. Participants shared valuable The Stroke Recovery in Motion Implementation Planner is an adaptable resource that may be used in diverse settings to plan community-based exercise programs for people with stroke. These findings may be informative to others who are developing resources to build the capacity of those working in community-based settings to implement new programs and practices. Future work is needed to monitor the use and understand the effect of using the Planner on exercise program implementation and sustainability.  With more people surviving stroke , communiPlanning and implementing new evidence-based practices and programs is a complex process requiring specific knowledge and skills. There is a growing body of literature and resources aimed at building the capacity for implementation science, but there is less support for building the capacity of those actually practicing implementation -10. Pracplanning, which is a process that turns strategy into action through three phases: phase 1: establishing a diverse, interdisciplinary planning team and working together to understand the community population, their needs, potential program options, and the available resources; phase 2: conducting a barriers and drivers assessment and developing tailored solutions (implementation strategies), as well as building an evaluation plan; and phase 3: launching, monitoring, and maintaining the exercise program.As a means to build implementation-planning capacity for community-based exercise programs for people with stroke, our team developed an implementation-planning guide, the Stroke Recovery in Motion Implementation Planner (hereafter referred to as the Planner) . The PlaMany strategies promoting uptake of evidence-based practices focus on modifying the individual or their environment. Throughout the Planner development and evaluation process , we prioThis study was part of a larger research program that developed and evaluated the Planner, which included a cross-sectional user evaluation of the Planner by diverse stakeholders (reported elsewhere ) and theThis field test study used a longitudinal qualitative design ,24, whiccurrent planners) and willing to use the Planner to guide their ongoing planning. We welcomed representatives from several occupations, including program managers and coordinators, health partners , and fitness professionals. As the purpose of this study was to observe how teams used the Planner to guide planning processes, teams were required to be at an early stage in their planning. A research team member (J Reszel) contacted potential participants by email with study information, with up to two reminders. If the contact was interested in participating, a memorandum of understanding was signed between the participating organization and our research institution. We asked the contact person to nominate other planning team members  whom we could invite to take part in the study. All participants signed an individual consent form. Participants did not receive any incentives to test the Planner or fund their program initiatives; however, participants were provided an honorarium to compensate them for time spent on study activities.Using the professional networks of the study team, we identified primary contacts  at Canadian community organizations that were intending to implement a community-based exercise program for people with stroke in the next 6 to 12 months  with a primary contact for each team to discuss planning work completed, Planner sections and tools used, and feedback on what was helpful and what was not. Each monitoring call started with the researcher summarizing the last call, allowing the participants to make corrections. Finally, primary contacts completed an end-of-study interview. At this time (February-March 2021), each team was at a different point in their planning process, but this end-of-study interview provided an opportunity for participants to reflect on their overall experience of using the Planner to date and share final feedback. The primary contact was the team member who was taking the lead in the planning process and had the most direct experience working through all Planner steps and activities. We anticipated that this person could therefore provide the richest updates on the team’s work throughout the process.The field test teams participated in 3 core data collection points. After reading the Planner and completing a questionnaire , all parIn keeping with a user-centered design approach, the baseline, monitoring, and end-of-study interview guides  focused Applying a cross-sectional approach , we usedWe received ethics approval for this study from the Ottawa Health Science Research Ethics Board (20190594-01H). Before starting any study procedure, each participant signed a consent form.We enrolled 5 planning teams in Canada. The teams were diverse in their geography and composition. There were teams representing the west coast to east coast of Canada in urban and rural settings. The teams ranged from a single person leading all aspects of planning to interdisciplinary teams sharing the planning work .To provide context for interpreting the findings, we provide a brief description of each planning team before describing the study participants.Team 1 included a physiotherapist in private practice and a program coordinator from the municipality who had previously collaborated to plan and implement other adapted fitness programs in the community. On the basis of their previously successful partnership and the perceived need for stroke-specific community programs, they decided to plan a new program together.Team 2 comprised a single program coordinator from the municipality who was working within its usual organizational model whereby an individual coordinator is largely responsible for all planning activities. The municipality had an existing suite of adapted fitness programs and wished to explore adding a stroke-specific exercise class to its model.Team 3 was led by a physiotherapist who had previously piloted a stroke-specific exercise program in a long-term care setting. The municipality had expressed interest in collaborating with the health authority on an adapted exercise program. The physiotherapist subsequently formed a new partnership with the municipality to begin planning a stroke-specific exercise program in the community.Team 4 comprised rehabilitation health professionals and a manager from a primary care clinic affiliated with the local hospital. The team members had experience planning and implementing other group programs tailored to various health conditions within their primary care setting and were considering offering a stroke-specific program in their clinic.Team 5 included 2 program coordinators from a nonprofit organization planning a web-based adapted exercise program. Before the COVID-19 pandemic, the program was offered in person. The onset of the pandemic provided an opportunity to explore whether the program could transition to a new web-based format reaching a broader geographical area.From these 5 teams, we enrolled 12 participants. Between February 2020 and March 2021, we conducted 25 interviews and focus groups with the 12 participants: the 7 baseline sessions lasted an average of 57 (range 40-75) minutes, the 13 monitoring calls lasted an average of 27 (range 18-35) minutes, and the 5 end-of-study sessions lasted an average of 44 (range 31-53) minutes. On average, the planning teams were followed by the research team for 9.6 (range 4-14) months .The 5 teams varied in how they used the Planner, ranging from methodically working through each step to using the Planner more as a reference guide when needed. The call to action was one of the first steps completed by each team. Although most  of the teams had previously been considering the value of planning a stroke-specific exercise program in their community, these teams described how participating in the field test prompted them to formally start their planning process.We observed that involving the community was the step that required the greatest amount of time , reflectI think it (the Planner) was quite good actually because it just allowed me to know where to focus and make some plans around how those meetings were going to look. It was really very helpful for meeting planning to focus from a team perspective. I think if it was just myself doing it I might have been able to manage without the Planner, but when you’re trying to actually organize your thoughts and articulate them to other people I think the Planner was really helpful in doing that.Physiotherapist, team 3, ID6, end of studyThe teams described several methods to conduct their community assessments, which were completed at various time points in the planning process. For example, based on an early assessment of services in the community and the number of people living with stroke, team 1 decided to move forward with planning and implementing a program. However, after launching the program and having lower than anticipated enrollment, this team appreciated the importance of taking the time to complete a detailed community needs assessment upfront:Make sure there’s a need for it in your community...do that research before you start so that you’re not launching a program that you get only three registrants for, and you don’t know if you can run it after all that work.Physiotherapist, team 1, ID1, end of studyAt the start of the planning process, team 4 felt confident that they would offer a future program in their clinic setting. However, as they took inventory of the programs currently offered, they realized that their initial idea was similar to an existing program at their organization. As a result, this team revisited their initial program idea and formed a new partnership with the municipal recreation center to address the service gap they identified. This marked the first time that their organization established a formal partnership with an external organization to plan  a program in their community.preadvertising for their future program and used the survey results in drafting their business case.Team 5 had started their community assessment before enrolling in the study and described administering a survey to potential participants and staff. This allowed them to collect information on community needs, interests, and preferences. They also saw these community surveys as a form of Of the 5 teams, 4 (80%) had selected an exercise program before joining the study. These decisions were made based on team members’ knowledge of, and experiences with, existing programs. Team 4 had not selected a program and undertook a comprehensive process to explore the various options, including reading the content in the Planner, exploring program websites and resources, and connecting with exercise program contacts and other teams who had previously used the programs.Although many  of the teams reported that their organization had a set participant enrollment fee, they indicated that the budget planning tool in the Planner helped them to assess what items and resources they had available to them and what unique start-up and ongoing costs they needed to budget for a program designed specifically for people with stroke.In total, 40% (2/5) of the teams completed a business case. Team 3 used the business case template in the Planner and adapted it to suit the needs of their team. Team 5 was required to use their organization’s template to prepare a business case; however, the team was able to use information from the Planner document and steps completed to produce a comprehensive business case in alignment with their usual organizational processes.The Planner included an implementation work plan template, although only team 4 used it. The reasons given by other teams for not using the implementation work plan included having another preferred method for tracking planning work or not being able to prepare a detailed work plan at this time because of the uncertainty and limitations imposed by the COVID-19–related restrictions.Of the 5 teams, 4 (80%) assessed barriers and drivers to program implementation and started planning for how challenges could be addressed. Although this step is described in phase 2, exploring potential barriers tended to occur early and simultaneously with phase 1 activities. In total, 60% (3/5) of the teams engaged with their community planning partners to explore potential challenges, which often resulted in the formation of practical solutions. For example, team 1 held a stakeholder meeting with representatives from local stroke and brain injury groups where they identified concerns related to the location, program time, transportation, and costs. Team 3 had a meeting with their stroke and caregiver planning partners and learned about potential barriers related to the accessibility of the building. This led to changes in their program screening form and development of an information sheet for future participants.Team 1 proceeded to launch a program but revisited their barriers assessment when they encountered enrollment issues. To understand the factors discouraging people from participating, they continued working with their community partners to identify potential issues with referral pathways, reassess barriers to participation, and consider how these could be addressed in future program sessions.Because of the pandemic, only 40% (2/5) of the teams worked on this step. Team 3 aimed to complete as much planning work as possible, including launch-readiness activities, with the intention of launching the program when pandemic conditions allowed. However, this site was never able to launch because of ongoing COVID-19–related restrictions. After approximately one year, team 1 was able to launch a session of their exercise program.Although unable to launch, 40% (2/5) of the teams did spend time exploring how they could evaluate their program. Team 3 prepared a participant satisfaction survey, selected client-centered before-and-after measures, and collaborated with their internal evaluation team to identify available data that could be pulled to show broader system impact. Team 5 reviewed the program-fidelity templates included in the Planner and subsequently selected and adapted one of the tools for use in their setting.Although at baseline team 1 indicated that the Planner made them think about the importance of developing a comprehensive monitoring plan that includes before-and-after participant outcome measures, at the end of the study the participants explained that they had overlooked this, and it was never implemented. However, they planned to administer a participant satisfaction survey.Because of pandemic-related restrictions and delays, none of the enrolled field test teams evaluated, made adjustments, or focused on sustainment during the study period.As some participants used the Planner, their impressions of the proposed process changed. For example, upon first reviewing the Planner, at least one member from every team expressed concerns about the length and potential complexity of the planning process and tools. However, after gradually working through the planning process over several months during field testing, the participants’ feedback became more positive as they saw the benefits. The following quotes illustrate how a program coordinator changed their views over time as they used the Planner:There is too much information. It’s overwhelming to read, review, use, and implement.Program coordinator, team 5, ID12, baselineOnce you get over the size of it, I can’t stress enough what a great resource it is...This [Planner] really gave me a great overview of the right way to plan something...When you see something laid out from start to finish it makes a real big difference. The Planner is going to be so helpful for all programming that I am involved in moving forward. I have learned so much.Program coordinator, team 5, ID12, end of studyready-set-go personality and indicated that the Planner helped them to pause and consider other activities to enhance the success of programs in their organization . Another participant identified several Planner steps that were not part of their organization’s usual practices  and acknowledged that although these steps would lengthen the planning process, it would be worthwhile. All teams indicated that they would use the Planner for program planning in the future, both to continue planning their current program  and for other program planning. We have summarized the participants’ usual planning process and provided examples of how the Planner changed their implementation-planning knowledge, attitudes, and activities . Finally, for some  of the teams, an unanticipated condition that supported Planner use related to the COVID-19 pandemic—with the closure of many services, team members described having On the basis of their field test experience, participants offered insightful suggestions about how to apply the Planner , which mSuggestions for how to use the Planner and illustrative quotesbig picture overview, use strategies to break up the content, and flag priority areas for your team.The Planner is a comprehensive document with a lot of information. Get the “I was trying to read it all at once when I wasn’t actively doing any planning. Now I am going through it one section at a time and just trying to tackle that section. I’m actively planning as I’m doing it and checking off the tasks, and it definitely feels more manageable.” “Even though it’s long, read the whole thing first...If someone’s just starting and they’re going to use the Planner, read it first to keep in mind what you’ve done in the past and then highlight and make notes on the sections that you know are really going to be useful to you. So that way when you do go back you know exactly where to go.” Do not treat the process as completely linear—give yourself permission to jump around the Planner.“Depending on their personalities, some people are very like ‘I must follow the steps. I must do the step-by-step-by-step’ and I am an example of that person. And so there are some times where it was like well I’ve gotten to here and I haven’t been able to do a proper community survey therefore stop, I cannot go any further because I haven’t done this step yet. Well no, you could still flip forward and see what else you can get started on.” Take time and get the early stages of the planning process right.“Even if you think you have this all in the bag and you have a program that’s going to start in a couple of months, it’s still really important to go through all of the process because you really need to have those evaluation bits in place from the outset...It’s the depth of quality. Having the time to put into this process will save you time down the road and avoids situations that you can’t really get yourself out of...would that have looked different if we had a different process in place from the beginning?” Engage with the Planner steps and tools beyond just ticking items off. Take an active approach with the Planner and use it to actually engage with people and organizations.[related to using a Planner tool to assess the facility] “Actually have them go in the building and go through [the checklist] instead of just doing it by memory and what they think it’s going to be. Have someone actually go and do a walk-through of the building.” Following the planning process can be more manageable if the load is shared among the team members. Identify a team lead who has the full view of the Planner and can delegate tasks and activities to other team members.“I would have looked at delegating a little bit more. I did a little bit of ‘okay you do this, you do that,’ but I think it could have been done more effectively where [other planning team members] could have taken on some of the bigger pieces...” tips from the field to help future teams optimize their use of the Planner. Given the paucity of literature reporting implementation toolkit evaluations [We followed 5 diverse teams as they used a newly created implementation-planning guide, the Stroke Recovery in Motion Implementation Planner, to plan for the implementation of community-based exercise programs for people with stroke. Because of the COVID-19 pandemic, none of the teams were able to work through the full planning process, with most activities focused on the first of the 3 phases. The findings of this field test study showed that teams took different approaches to applying and adapting the Planner in their settings. All teams indicated that the Planner influenced their approach to program planning and that they intended to continue using the resource in the future. We identified various team, organizational, and broader contextual factors that hindered or facilitated uptake of the Planner by teams. Study participants shared valuable luations ,31, the recipe that must be followed in a lock-step manner. The finding that diverse teams could, in fact, adapt and navigate the planning process in various ways suggests that the resource was applied as intended in real-world settings.A key finding was that the teams did not complete the planning process in a linear manner. All teams conducted multiple steps simultaneously, with some steps being revisited multiple times during the planning process, a pattern reported elsewhere . We alsoEven experienced planners described new things they learned from the Planner and how this improved their processes, suggesting that the Planner positively influenced end users’ planning capabilities, similar to other work reporting improvements in practitioner implementation skills . In our inclusiveness [Fitness professionals are another group essential to planning and implementing fitness programming; yet, it is largely unknown how fitness instructors engage in knowledge translation and implementation and what barriers and facilitators they encounter . Althougsiveness , we assesiveness . Future In this study we used a passive form of implementation; that is, the teams had access to the Planner guide and tools, but no active implementation facilitation or support or any financial incentives were provided. However, there is evidence to suggest that active facilitation can enhance implementation efforts ,38. GiveWe identified several cross-cutting barriers and facilitators to following the implementation-planning process proposed in the guide, including organizational context and support, team leadership style, and the value placed on community-partner engagement. These factors were also identified in a review as core capacity–building domains  that can be modified to build capacity for the implementation of evidence-based practices . Knowingblank slate at step 1, we revised the Planner to illustrate various starting points, with directions on how to use the Planner accordingly [Finally, it is noteworthy that 80% (4/5) of the teams had already selected the specific exercise program they wanted to implement before enrolling in the study. The finding that most teams had already selected a program speaks to the reality that for many teams, identifying a program is, in fact, the starting point that launches their implementation-planning journey. Observing that not every team starts with a ordingly .The greatest limitation of this study was conducting a prospective field test during a pandemic. The pandemic created significant recruitment challenges, and we were unable to recruit our originally planned sample size. Furthermore, for the teams we did enroll in the field test study, the closure of community facilities and the public health restrictions led to significant planning delays. The enrolled teams were therefore unable to progress through the entire planning process, with only 20% (1/5) of the teams able to launch a program during the study period. The purpose of this study was not to ascertain the effectiveness of the Planner related to program implementation and outcomes; rather, we sought to understand if, how, and why the Planner was used in practice. Despite the pandemic-related planning challenges, many teams were still able to work through many steps and provided valuable insight on how they used the Planner, allowing us to meet the field test study objectives. However, it is important to acknowledge that because the teams engaged much more heavily in the early phases and activities of the Planner, we do not have the same depth of understanding of how teams would use the Planner in the later stages .Furthermore, most  of the teams indicated that it was the study itself that prompted them to officially launch their planning activities. The study data collection procedures may have caused participants to engage more with the Planner . However, to model more typical program-planning scenarios, the research team did not provide any program funding and provided only minimal assistance with connecting study participants with resources . The interviewer (J Reszel) did not contribute to making decisions in the program-planning process. Finally, it is important to note that the monitoring calls and end-of-study interview were completed with 1 primary contact at each site, and their perceptions and experience of working through the Planner may not represent the experiences of their team members.A strength of the study was the diversity of the sample, which included planning teams with differing compositions based in both urban and rural settings across the country. This allowed us to observe the use of the Planner in a variety of settings, which enhances the transferability of the findings. In addition, despite the challenging pandemic conditions, we were able to maintain contact with 80% (4/5) of the teams throughout the study period, allowing us to collect comprehensive data on Planner use and experiences over time.Catalyzing the expansion of safe, effective, and sustainable community-based exercise programs is important to the long-term health of people with stroke. The Stroke Recovery in Motion Implementation Planner  is an ad


The substantial rise in non-communicable diseases (NCDs) over the last two decades poses a major concern to the healthcare services in Malaysia. This study aimed to evaluate the sustainability of the current NCDs programs and identify the challenges and factors impeding the sustainability of the NCDs program implemented under the National Strategic Plan.This study applied the mixed-method approach using the Program Sustainability Assessment Tool (PSAT) to assess the eight domains for program sustainability combined with 5 open-ended questions. The survey was administered to key leaders from the district health offices in Malaysia. The mean score for each sustainability domain and the overall mean sustainability score were determined. Descriptive statistics and thematic analysis were conducted using Statistical Package for the Social Sciences (SPSS) version 25 and NVivo version 12, respectively.A total of 80 key leaders responded to the survey. Overall seven domains scored an average of ≥ 4 with an overall mean sustainability score of 4.2. The highest domain mean scores were 4.5 (communications) and 4.4 . The lowest mean score domain was 3.8 (funding stability). The open-ended responses revealed challenges faced by department heads, including implementation difficulties, factors impeding the planning of the NCDs program for sustainability, lack of financial resources, lack of human resources, and support for staff training which are largely consistent with the scores of each domain.The sustainability factors affecting the NCDs program in Malaysia are qualitatively similar to other countries. For greater sustainability capacity, we should work towards strong leadership, strengthening funding stability, and incorporating evidence-based public health strategies in the implementation of the NCDs program. Chronic diseases are well recognized as the major health challenge in developed and many emerging economies. Chronic disease management programs (CDMPs) involve planned organization of care aiming to help patients better self-manage their health, reduce risk factors, facilitate more consistent and coordinated clinical care, and reduce associated disease risks. Such programs include health assessments, action plans, patient education, and health behaviour tracking, with ongoing support , 2. CDMPOver time, a CDMP ideally must sustain activities described as ‘sustainability outcomes’ such as community-level partnerships, organizational practices, benefits to clients, and the salience of the program’s core issue . There iet al developed a Program Sustainability Framework to assess public health program capacity for sustainability, which includes eight domains  [Sustainability is defined as “the existence of structures and processes that allow a program to leverage resources to effectively implement and maintain evidence-based policies and activities” . The suserships) , 7, 8.In Malaysia, most patients with NCDs are managed at the primary healthcare level. This activity is one of the main workload and organizational challenges faced by all primary healthcare providers. Malaysia has a dual primary health care system which consists of both public and private sectors with the public sector as the mainstream health provider. The Malaysian public primary healthcare service has developed significantly since the 1950s by establishing primary health clinics (PHCs). The PHCs provide easy access to health care for the community, with each clinic serving a population of approximately 15,000–20,000 population. The primary healthcare system in Malaysia provides basic or general healthcare focusing on the point at which a patient first seeks assistance from the medical care system. The referral system connects the primary healthcare facilities with hospitals (at the district and state level) and specialist centres [The primary healthcare system in Malaysia adopted the ‘Reviewed Approach of Primary Healthcare’ (REAP-WISE) to service delivery in 2007 . The REAIn response to the rise in NCDs, the Ministry of Health, Malaysia implemented “The National Strategic Plan for Non-Communicable Diseases (NSP-NCD) 2010–2014,” followed by the 10-year National Strategic Plan for Non-Communicable Diseases (NSP-NCD) 2016–2025 to address the burden of NCDs at national and state level . The iniThis study aimed to (i) evaluate the sustainability of the NCDs program implemented under the NSP-NCD in Malaysia and (ii) identify challenges and factors impeding the sustainability of the NCDs program implemented in the primary healthcare settings within the Ministry of Health (MOH). The paper covers the results of a survey of key managers in district health offices across Malaysia.A survey of leaders in district health offices (DHO) was conducted using a questionnaire compromising the Program Sustainability Assessment Tool (PSAT) (version 2) and five open-ended questions. Respondents were recruited from January to April 2019. The translation was not necessary as all respondents were fluent in English. Ethical approval for this study was obtained from the Medical Research and Ethics Committee (MREC)-NMRR-18–2542-44,097 (IIR), Ministry of Health, Malaysia.The PSAT is a 40-item multiple-choice instrument, that assesses the program’s sustainability capacity in 8 domains . The responses to each item are recorded on a 7-point Likert scale from “to little or no extent”  to “to aDescriptive statistics were used to summarize the data. Data analysis was performed using Statistical Package for the Social Sciences (SPSS) version 25. NVivo was used to manage the qualitative data. Thematic analysis was conducted for the five open-ended questions: (1) Who are your champions or advocates? In what ways do they advocate for the program ? (2) What organizations or individuals are invested in the success of the program? Why? (3) What are the strengths and weaknesses you see in terms of the organizational capacity to maintain the existing NCD program? (4) What is the current funding situation? (5) What have you learned (or done) about creating sustainable care coordination programs or improvements? Two researchers coded open-ended questions independently and discussed them to ensure convergence and divergence of the coding scheme. Subsequently, themes were identified for the eight domains of the sustainability framework.From the 114 DHOs across Malaysia, a total of 80 respondents agreed to participate in this study. The overall response rate was 70.2%. The median age was 47 years (range 36 − 60) and the majority of respondents were female (63%) . The mean score for 8 domains were Communications , Organizational capacity , Program evaluation , Environmental support , Program adaptation , Partnerships , Strategic planning  and Funding stability   and other stakeholders to improve the program. Some mentioned that lack of access to health record data and inadequate process evaluation limited the effectiveness of evaluation on sustainability.Of relevance to the domain of Environmental Support, open responses indicated it is a crucial and underlying element to implementing the NCDs program and its sustainability. The continuous economic support and internal politics  were crucial in implementing and sustaining the NCDs program at the district level. Some respondents mentioned good political support resulted in advocacy of the NCDs program within the community and strong participation from the community in the NCDs program. There is a need for external support from the community to gain broad public support for the initiative in sustaining the NCDs program.Of relevance to the domain of Program Adaptation, comments reinforced that it was critical in the success and sustainability of the NCDs program. Adaptability was seen as important for effective collaboration with key partners, adapting current workflow to accommodate the NCDs program, engagement of clinicians in health promotion activities apart from clinical management of NCDs, and leveraging resources. Currently, the nurses and medical assistants are tasked to carry out the screening, monitoring, and health promotion activities. Respondents emphasised the need for understanding local issues and challenges prior to implementing new programs.Of relevance to the domain of Partnerships, comments reinforced that these were crucial in implementing and sustaining the NCDs program. The core process for maintaining the programs was building strategic partnership management and creating coalitions among the community and other agencies. The DHO leaders recognised the importance of sustaining partnership, and building trust with the community was vital to maintaining activities for the community when overcoming challenges of limited resources and funding. Engaging community and religious leaders were a critical factor in program acceptability within the community.Of relevance to the domain of Strategic Planning, the respondents described the need for more effort and progress in planning to ensure implemented NCDs program would last. Some DHOs encountered implementation barriers in terms of funding and suggested a focus on sustainability planning of all programs prior to the rollout of new programs. In some instance, a lack of clear understanding of the roles and responsibilities among stakeholders impact the outcome of the NCDs program. There is a concern about the implementation of new programs with no integration or continuity with existing programs. Some reported that involving staff from districts in aspects of strategic planning was important to ensure success of the programs.Of relevance to the domain of Funding Stability, responses reinforced its importance in sustaining the NCDs program and providing greater flexibility of funding programs at the district level. Issues raised included strengthening funding by cost-sharing, support from external resources, and being flexible about which program components are supported. In certain districts, limited funds impacted the implementation and sustainability of NCDs programs.The sustainability of the NCDs programs is vital to the public health effort and a better understanding of the strengths and challenges of the existing programs will help guide the federal and state governments in policymaking decisions. As part of a broader evaluation of the NCDs program at the district level across all states in Malaysia, we used the PSAT to evaluate sustainability capacity in 8 domains with open-ended questions. The PSAT tool was designed to be easy to use for a wide variety of community and public health programs , 8. HighThe domains of communication and organizational capacity were rated highly at all sites. Funding stability is a known barrier or challenge described in the literature and was the most frequently reported barrier to sustainability in 48 projects with short-term foundation funding . CommuniNCDs management requires long-term care; accessibility and affordability are regarded as two critical factors for NCD treatment. Hence, public primary healthcare became the leading choice by the rural and urban communities. NCDs can be prevented by managing risk factors and early detection of the disease. The main challenge is addressing the issue and creating awareness among the public to visit health care facilities at the early stage of NCDs where they are free from any symptoms . InitiatImproving communication within the various levels of MOH is vital in program sustainability. Increased top-down communication would help to build an internal agency culture that would be more resilient to external conflicts, such as funding instability or complicated political environments. The communication process within the organization should reflect a process of discussion, engagement, clarification, negotiation, and perspective-taking, rather than solely as information exchange or directive. A review of administrative and management to improve local public health recognised participatory decision-making, involving communication with employees to get their input as an effective way to create a conducive working environment , essentiThe study found that the role of leadership in communication was essential and mentioned consistently, that constant communication of information and coordination of the program is vital for success in sustainability . The absThere are many factors that affect the NCDs program’s capacity for sustainability. Funding is crucial for the implementation and sustaining of the NCDs program, and factors such as organizational support, partnerships, and communications can aid in sustaining the NCDs program. The PSAT and planning process provide public health programs and their partners with a reliable method for rating their programs’ capacity for sustainability. This method is unique in that programs can identify their areas of strength and weakness across a range of sustainability factors and then make informed decisions about where to concentrate efforts. The study focused on leaders of the DHOs, which may limit the generalizability of the findings within the organization. The qualitative results provide additional support for the association between the domains and the findings from the empirical literature on sustainability , supportThe NCDs program is being evaluated by MOH regularly, and the study highlighted the factors and challenges of sustaining the NCDs program at the district level. The findings from this study may help improve the evidence-based approach for program sustainability in DHO settings. The evaluation was by the district leaders of the existing NCDs program, with the qualitative findings revealing the success and challenges faced at the ground level and aid policymakers at the federal level in the future implementation of policies and planning of the NCDs program.Since this is the first study evaluating the sustainability of the non-communicable disease program at the district level in Malaysia, future studies could adapt the PSAT to assess long-term program sustainment. Investments in leadership, improvement in communication from the top of the organization, and greater funding flexibility may enhance the sustainability of evidence-based public health non-communicable diseases programs.The findings from this study revealed that in the Malaysian context, a key area perceived as needing improvement for NCD program sustainability is the need for greater security for continued funding to support staff. Organizational capacity, program evaluation, program adaptation, communication, environmental support, strategic planning, and funding stability appeared to be lesser issues than whether a DHO can sustain programming. Increased top-down and bottom-up communication would help build an internal culture for better resilience to external conflicts such as uncertain political environments and funding instability. This study also demonstrated the usefulness of the PSAT for guiding a mixed-methods evaluation of sustainability capacity for the NCDs program. Findings from this study may help improve the sustainability of the NCDs program in DHO settings. Program managers can use the PSAT to assess the program design and implementation process to identify links between these and long-term program sustainment.


Many countries have adopted integrated community case management (iCCM) to reduce mortality among children under five years from common childhood illnesses. The 2016–2020 Malian Red Cross iCCM program trained 441 Community Health Workers (CHWs) to treat malaria, pneumonia, diarrhea, and malnutrition for children under five years of age in six districts. Implementation strength and quality of care (QoC) were assessed through the program’s supervision function, using the Malian Ministry of Health’s system.This paper compares methods and results of program supervision data and an independent evaluation to assess the effectiveness of program implementation and supervision and inform program improvement. It also presents the benefits and limitations of each method.RADAR evaluation. RADAR evaluation data collected in July and August 2018 were compared with program supervision data collected mostly between May and December 2018.An independent QoC evaluation was conducted using tools developed by the Real Accountability: Data Analysis for Results (RADAR) project, hereafter referred to as the The RADAR evaluation provided detailed findings on correct assessment, classification, and treatment per illness, medication type, and dosage. Program supervision combined the findings for all illnesses, medication type, and dosage due to limitations in the data collection process. Six indicators were comparable between both methods. Findings were similar for temperature and mid-upper arm circumference measurements but diverged between program supervision and the RADAR evaluation, respectively, on correct classification for all illnesses (87.1% vs. 65.3%), correct treatment for all illnesses (69.5% vs. 39.8%), correct respiratory rate counting (88.5% vs. 54.7%), and administering the first dose by CHW (75.4% vs. 65.0%). Findings from the RADAR evaluation guided improvements in program supervision.A robust program supervision system can serve as a credible method to assess QoC. However, a rigorous independent QoC evaluation provides a valuable benchmark to gauge the effectiveness of the supervisory process. Integrated community case management (iCCM) is a strategy endorsed by WHO and UNICEF to reduce mortality among children under five years by increasing access to services for common childhood illness . Over 30A review of 22 studies conducted in 2014 on iCCM impact and implementation considered supportive supervision as best practice, with defined elements such as record reviews, case management observations, constructive feedback, provider participation, problem-solving, and focused education influencing CHWs’ performance, motivation, and retention . Other sImproving supervision quality by ensuring that it includes key elements of supportive supervision – particularly performance monitoring, constructive feedback, problem-solving, and focused education – has a greater impact than increasing frequency of supervisions that lack such elements . A dose-Supportive supervision is a process of helping workers improve their own work performance continuously . It occuIn practice, however, supportive supervision is often weak and under-supported . In a reEssential Care in the Community, (the main guide for CHWs) and the Supervision Evaluation Form [Supplement Figure 1].Mali’s National Child Strategy 2007–2012 provided a strategic plan for newborn and child survival, including a full iCCM package and acknowledging for the first time the CHW’s role in delivering community case management. The strategy’s original tools and guidelines for CHWs’ training, service delivery, and supervision were revised in 2015. In 2016, the Malian Red Cross, supported by the Canadian Red Cross, scaled up its iCCM program  to a second five-year phase and enhanced its supervision methods. It used the Ministry of Health’s (MoH) iCCM training curriculum and new tools, templates, and guidelines, including The iCCM program trained 441 CHWs in rural communities in Koulikoro and Sikasso regions to provide preventive and curative care to children from 2 to 59 months of age for uncomplicated malaria, diarrhea, pneumonia, and malnutrition, as well as newborn care and family planning services. These CHWs covered 756 villages totaling around 650,000 people in six districts and were supervised by clinicians working in 130 Primary Health Centers (PHCs).The program hired District Counsellors (doctors or nurses) to mentor CHWs and provide technical support to the PHCs in supervising the CHWs. After initial training, some PHCs kept their CHWs for one to two weeks at the facility to solidify their technical skills and strengthen the relationship between the CHWs and PHC personnel.Centre de Recherche, d’Études et de Documentation pour la Suivie de l’Enfant (CREDOS) between July and August 2018. While supervision methods and the RADAR evaluation assessed both implementation strength  and QoC (correct classification and treatment), this paper focuses on QoC for childhood illnesses.The Real Accountability: Data Analysis for Results (RADAR) project conducted an independent, cross-sectional evaluation of the iCCM program  to measure the implementation strength and QoC provided by participating CHWs . The RADThis paper compares the methods and results of the Red Cross program’s supervision data and the RADAR evaluation to inform program improvement around QoC provided by CHWs. It also presents the benefits and limitations of using each method.This paper is part of the supplement “Strengthening Effectiveness Evaluations to Improve Programs for Women and Children”. The paper in the supplement “Marx MA et al. Tools and methods to Measure the Quality of Care for Maternal, Neonatal, Child, Reproductive Health and Nutrition Programs: Guidance from use in four sub Saharan African Countries through the RADAR project” describes the Quality of Care evaluation tool used in Mali.We compared findings from program supervision with those from the RADAR evaluation to assess the impact of supervision on QoC, which was provided by CHWs. We hypothesized that the closer the estimates are between similar indicators from program supervision data and from a rigorous, external evaluation, the more effective the program supervision in assessing the CHWs’ QoC level and improving it; wider findings indicate that the supervision process requires adjustments.Longitudinal secondary data collected during program supervision between 2017 and 2020 using the MoH’s iCCM strategy to evaluate QoC.Cross-sectional secondary data from the RADAR evaluation using tools developed by the RADAR project.A 2019 socio-demographic program survey of all CHWs that included findings on the relationship between CHWs and their supervisors.A survey of sampled CHWs as part of the iCCM program’s final evaluation that included findings on CHWs’ perception of program supervision.The study takes a data triangulation approach using the following secondary data sources.Supervision Evaluation Form. However, such frequency is not practical due to the lack of adequate financing. In this paper, ‘program supervision’ is defined as the program’s routine, supportive supervision. Supervision teams included three to nine members, but always at a minimum, the chief clinician from each PHC, a representative from the Community Health Association, and the program’s District Counsellor.The MoH program supervision of the CHWs by the PHC should be performed monthly, using the Supervision Evaluation Form .The program intended to adhere to MoH supervision guidelines, but it took more than three months to complete a supervision cycle in a district depending on the resources available, logistical constraints, and security incidents. As a result, some districts had fewer supervision cycles than others. During seven supervision cycles (October 2017 to September 2020), the program conducted 2,705 supervisory visits, 440 of which were in Cycle 3 . Data waCHWs in Mali are mandated to provide services in the following areas: childhood illnesses, newborn care, infant and child feeding practices, family planning, and hygiene and sanitation. Program supervision evaluated QoC provided by CHWs through records review and direct observation of case management. Records review measured five indicators on childhood illnesses, while observation measured 39 indicators of the full CHW mandate, of which 13 were on childhood illness. The MoH system uses the 39 indicators as a composite indicator, converting it to a scale of 100 points. CHWs with 80% scoring or more on the composite indicator demonstrated a satisfactory level of performance of their mandate. We identified 11 indicators from records review and observation of case management that were relevant to our objective (Supplement Table 1).The RADAR evaluation, implemented between July and August 2018, was a cross-sectional evaluation carried out on a sample of 300 CHWs that used two rigorous RADAR project tools to evaluate the implementation strength and QoC of the healthcare services that CHWs provided . The impBecause each supervision cycle took several months to complete, the start and end dates varied across the six districts. The second and third cycles overlapped with the RADAR evaluation. Cycle 3 data, due to its chronological proximity and higher sample size of CHWs (N = 440), was considered optimal for comparison to the RADAR evaluation data. Of the 66,654 sick child consultations carried out by CHWs in 2018, 38,887 (58%) were for uncomplicated malaria, 12,584 (19%) for cough/cold, 7,762 (12%) for pneumonia, and 7,421 (11%) for uncomplicated diarrhea, a pattern observed in subsequent supervisions.This section presents findings from the program supervision and RADAR evaluation on the characteristics of CHWs, the program’s implementation strength, the quality of care provided by the CHWs, and the impact of the RADAR evaluation on program supervision.Supplement Table 4 shows the characteristics of the CHWs assessed in the RADAR evaluation.The RADAR evaluation found that 98.3% of the CHWs received training with sick children and 81.0% received a supervision visit during the previous three months.Nearly every CHW had pneumonia and uncomplicated malaria medications in their kit. Both the RADAR evaluation and program supervision data found CHWs had the necessary equipment, records/registers, and adequate storage conditions for medications. The RADAR evaluation found medication availability for malaria and pneumonia to be higher than those for diarrhea, cough, and cold, a finding consistent with all supervision cycles since the MoH strategy prioritized commodities for malaria and pneumonia.A snapshot of key findings on assessment, classification, and treatment by CHWs is presented in We found six program supervision indicators that were comparable to indicators from the RADAR evaluation, with three significant differences ranging between 22 and 34 percentage points.The RADAR evaluation uncovered deficiencies that the program had not been aware of, because the MoH’s data collection form does not capture correct classification and treatment by individual illness type, but only for all illness combined. It also does not distinguish between correct medication and dosage when assessing correct treatment. After the RADAR evaluation findings were shared with them, program staff developed and implemented a plan of action to address these deficiencies, including a refresher course, reinforcement during subsequent supervisions, using monthly PHC meetings to coach CHWs, and a shift from focusing on the composite indicator to assessing individual skill areas.The program staff identified four benefits from the RADAR evaluation for iCCM program supervision.The first benefit was to initiate customized CHW supervision and coaching. For each supervision session after the RADAR evaluation, the supervision team reviewed the evaluation findings showing areas requiring improvement. They then reviewed the last supervision report of the CHW, which is evaluated to compare scores. If the CHW’s score of the corresponding indicator was low, the supervisor provided coaching on the knowledge and skill area that was lacking, practicing with the CHW repeatedly. The program’s Monitoring and Evaluation Officer supported the process by tracking all supervision reports and informing the supervisors on scores comparable with the RADAR evaluation lower performance areas.The second benefit was to complement CHW coaching during supervision visits with additional technical support during the monthly meetings at the PHCs when CHWs came to replenish their stock and submit their reports. Historically, these monthly meetings were seen as a platform to reinforce competencies; however, CHWs’ attendance was less consistent among those PHCs and their community health associations that were less capable of organizing them and less financially able to incentivize CHWs to attend, by covering their travel costs and offering refreshments. Therefore, the program offered one-time funding to these PHCs, enabling a more equitable opportunity for all CHWs to improve their knowledge and skills through refresher sessions that target their specific deficiencies.Although the fund existed only for the first fiscal quarter after the RADAR evaluation, the program leveraged this opportunity to advocate that all PHCs and their community health associations provide CHWs with a per diem to enable consistent attendance at subsequent monthly meetings and many agreed to do this. The program staff felt the RADAR evaluation findings reinforced these institutions’ responsibility to invest systematically in strengthening the competencies of their CHWs.The third benefit was to strengthen the competency of MoH supervisors. Testing of supervisors at the beginning of the program revealed technical areas that required improvement. The RADAR evaluation showed a correlation between the knowledge and skills gaps of supervisors and that of the CHWs under their responsibility. In response, program staff initiated a joint review of the RADAR evaluation findings with the supervisors and asked them to lead a refresher course for CHWs during the monthly meetings. This created an opportunity for supervisors to review the iCCM subject matter. As a result, program staff observed an improvement in the knowledge and skill level of supervisors and in the overall quality of supervision.The fourth benefit was to strengthen the relationship between CHWs and their PHC supervisors. While initial training promoted the connection between CHWs and their supervisors, the RADAR evaluation provided an opportunity for the program to reiterate that a CHW’s first point of reference for technical issues was their supervisor. Additionally, the informal contact opportunities that CHWs had with their supervisors beyond routine supervision visits  improved CHW performance. The program tasked the District Counselors, who provided technical support to the MoH staff overseeing iCCM in the field, by reinforcing the relationship between the CHWs and their supervisors, and supported the development of an individual CHW learning pathway that could be monitored consistently.The quality of supervision and CHW performance improved steadily for the remainder of the program. By Cycle 5, eight indicators on correct classification and treatment for all CHWs improved, on average, to 80% (MoH threshold) or higher.A socio-demographic survey conducted among all 441 CHWs in 2019 prior to disseminating the findings from the RADAR evaluation found 67.9% of CHWs contacted their supervisors ‘to seek information on technical issues’ (50.9% once a month and 17.0% once a week). In another survey that sampled 195 CHWs in December 2020 as part of the program’s final evaluation, 85.1% of CHWs reported receiving feedback from supervisors during supervision visits and rated the feedback as ‘excellent’ (38.7%) and ‘good’ (55.7%). CHWs also reported that supervisors corrected them when they saw them making a mistake ‘all the time’ (70.8%) and ‘sometimes’ (29.2%).The independent evaluation gave a detailed picture of the strengths and weaknesses of the QoC CHWs were providing, creating an opportunity to improve both the QoC and program supervision.Both program supervision and the RADAR evaluation measured the QoC provided by CHWs when assessing symptoms and danger signs, classifying illnesses, and treating or referring patients through observation of case management; program supervision carried out an additional assessment by reviewing the records of sick children. The RADAR evaluation provided the findings for each assessment step by illness, and the findings for treatment by the type of medication and dosage. Program supervision provided findings about each assessment for all illnesses combined and, because of the limitations of MoH’s data collection form, also combined the findings about medication and dosage.The RADAR evaluation showed that CHWs have good skills for correctly classifying uncomplicated malaria, diarrhea, cold, and cough, but not for pneumonia, and that they are skilled in prescribing the correct medication for all illnesses but were weak in prescribing the dosage, except for antimalarial medication. In contrast, program supervision data showed that the competencies of CHWs were acceptable overall, but the results were skewed because CHWs were more skilled at managing malaria, which is predominant for children in Mali . The use of a rapid diagnostic test for malaria must have contributed to the 91.5% rate of correct classification measured by the RADAR evaluation .We found six comparable indicators of QoC between program supervision and the RADAR evaluation . There wSeveral hypotheses could explain these differences. The timing of data collection and seasonal illness could account for the variance in correct classification and treatment. The RADAR evaluation, which was conducted between July and August 2018, showed that 71.8% of the cases of examined children were for cough and cold, 25.7% were for malaria, 24.3% for diarrhea, and 11.7% for pneumonia. Program data from 2018 showed that CHWs assessed 58.3% for malaria, 19% for cough and cold, 12% for pneumonia, and 11% for diarrhea. Cycle 3 spanned more than 15 months (May 2018 to July 2019), but 95% of CHWs had been supervised by December 2018. The malaria season is from July to October, peaking in August, while the season for cough and cold is from July to January. The RADAR evaluation showed that CHWs were more skilled in managing uncomplicated malaria, so it is possible that most of the cases assessed for QoC by program supervision were for malaria.The variance in correct respiratory counting could be attributed to the different counting methods used. In program supervision, the CHW and the supervisor counted simultaneously. In the RADAR evaluation, the CHW counted first and the data collector registered the count. After the CHW finished examining the child, an exit interview with the child’s companion was held, followed by the re-examination of the child by a clinician, counting the respiratory rate and comparing it with the CHW’s. The time lag between both counting methods could have been a factor. The differing competencies of supervisors in assessing correct respiratory counting could also have resulted in inaccurate counting. Moreover, CHWs could have felt more relaxed while being observed by their own supervisor than by unknown personnel assessing their competencies in the more formal RADAR evaluation.Care in the Community Guide and 95.8% had the Care of Sick Child Form, both of which included guidance on correct classification and correct prescription of medication type and dosage. Project staff observed that the form was complex, potentially preventing CHWs with lower literacy skills from mastering the content .One year after completing their initial training and after receiving two to three supervisions, some CHWs were still unable to correctly classify and treat children. The RADAR evaluation showed that 52.1% of children were classified correctly for pneumonia and that correct dosage for amoxicillin, oral rehydration salt, and zinc was 13.0%, 42.6%, and 48.9%, respectively. It is not clear why these deficiencies occurred. One hypothesis is that CHWs may not have been optimizing the job aids they were given. Program supervision Cycle 3 showed that 92.8% of CHWs had a copy of the Overall, the RADAR evaluation found that 39.8% of sick children presenting with iCCM illnesses were correctly treated, ranging from 80% for treatment of uncomplicated malaria to 13% for pneumonia. It found that CHWs prescribed correct medications at high rates for all illnesses, ranging from 90.4% for a combination of oral rehydration salt (ORS) and zinc to 97.8% for pneumonia. However, correct dosage was different, ranging from 13.0% for amoxicillin for pneumonia to 82.5% for artemether-lumefantrine for uncomplicated malaria.The RADAR evaluation found that only 15.9% of sick children cases received correct classification, medication, and dosage. In addition, all MoH training tools, forms, and manuals were in French, which is not the first language of CHWs and could have limited their use.Several studies have shown various findings on the effectiveness of program interventions to improve the QoC provided by CHWs, particularly the role of supervision. For example, two related studies on the Optimization of Health Extension Program Intervention in Ethiopia – which included training, supportive supervision, and performance reviews of health extension workers, the Ethiopian equivalent of CHWs, (HEWs) – found that the interventions were neither associated with an improved classification of childhood illnesses by HEWs nor improved the use of their services ,15. The The Last 10 Km Project in Ethiopia focused on and evaluated the use of job aids by HEWs in their supervision programs. Supervisors asked HEWs to refer to their job aids to find the correct answers to a standard set of questions on classification and treatment of key childhood illnesses . Such prA systematic review of diverse mentoring approaches in Africa found that mobile mentoring, both within the health facility and remotely by phone, improved competencies in clinical management of childhood illnesses .The independent evaluation provided a detailed snapshot of the strengths and weaknesses of the iCCM program in Mali. It guided the development of a plan of action to address the weaknesses through refresher training, program supervision tailored to CHWs’ needs, systematized monthly meetings at health facilities for performance review and remote coaching, and enhanced competence of supervisors in managing childhood illnesses. As a result, the quality of guidance by supervisors continued to improve. By the end of the program, 85.1% of CHWs reported receiving feedback from supervisors during supervision visits and rated the supervisor’s feedback as ‘excellent’ (38.7%) and ‘good’ (55.7%). These CHWs noted that the supervisor corrected them when he or she saw them making a mistake ‘all the time’ (70.8%) and ‘sometimes’ (29.2%).While the program incorporated several elements of supportive supervision, it provided a limited number of indicators on QoC. A review of the supervision process triggered by the RADAR evaluation indicates that its evaluation of the QoC is agile and credible enough to introduce corrective measures when needed.The study focused on the quality of care of childhood illnesses implemented through iCCM by CHWs in Mali but did not provide enough detail on the availability of commodities, equipment, and records that were assessed by either the RADAR evaluation or program supervision.Further research is needed to better understand how initial CHW training can be reorganized to optimize this as a learning opportunity that will enable CHWs to classify and treat illness more quickly, competently, and correctly.Differences in the assessment methods of the program supervision and RADAR evaluation can make it difficult to compare the estimates for similar indicators. These differences include the use of different tools, processes, and checklists and the amount of focused training for such quality-of-care assessments.The study suggests that the closer the estimates are between similar indicators from program supervision data and from a rigorous, external evaluation, the more effective the program supervision in assessing the CHWs’ QoC level and improving it; wider findings indicate that the supervision process requires adjustments. However, methodologies, context, and program interventions need to be factored when interpreting variances.A rigorous independent evaluation of QoC early in an iCCM program can provide a snapshot of strengths and weaknesses that can be compared with a program’s supervision data. Data from an independent evaluation can be used to assess the effectiveness of program supervision and inform program improvements. Ideally, if resources and time permit, a second independent evaluation at the end of the program to verify whether improvements have been achieved could increase learning on iCCM programs.The independent evaluation of the Red Cross’ iCCM program in Mali identified strengths and deficiencies. Adjustments to the process enhanced quality of care provided by CHWs and increased the effectiveness of program supervision.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.Click here for additional data file.


Online (HLaC Online), an online intervention supporting cancer survivors to set and meet their healthy living goals.The aim of the present study was to co-design Healthy Living after Cancer Adapted from an initial telephone-delivered Healthy Living after Cancer program, wireframes (PDF black and white mock-ups) of the proposed online program were presented in a series of focus groups and interviews to our stakeholder group, which consisted of cancer survivors, oncology healthcare professionals, and representatives from cancer support organisations. Stakeholders were prompted for feedback on the wireframe and given end-user scenarios to encourage deeper engagement with the co-design process. Transcriptions underwent thematic analysis to determine which features of the program needed change or expansion.Online prototype.27 participants took part in one of 8 focus groups or 10 interviews. Five themes were identified relating to (a) website design elements, (b) promoting and maintaining long-term engagement, (c) relatability and relevance, (d) navigating professional support, and (e) family and peer support. Recommended changes, such as simple activities and guidance videos, were integrated into the HLaC Online.Involving end-users in the co-design process ensured the intervention’s relevance and specificity to the needs of cancer survivors. Next steps include feasibility testing the prototype, prior to commencing a national randomised control trial of HLaC Online aims to support cancer survivors to improve their quality of life by making healthy lifestyle changes in their physical activity, healthy eating, weight management, mental health, and fatigue management.HLaC The online version contains supplementary material available at 10.1007/s11764-022-01284-y. Engaging in a healthy lifestyle after cancer, including regular physical activity and adequate nutrition, can reduce the risk of mortality, cancer recurrence –3, and cFace-to-face interventions have demonstrated efficacy in improving health behaviours; however, these interventions are not routinely implemented in clinical care at the completion of cancer treatment . This evThe telephone has previously be investigated as an accessible and acceptable modality for health interventions . One sucDigital health modalities, including patient portals, online support tools, and mobile applications, have emerged as a cost-effective and accessible way to deliver health-related services , 22. DigOnline (HLaC Online) research team commenced the co-design process with a group of stakeholders to adapt the program iteratively from its telephone-delivered format using a five-phase Design Thinking Research Process, comprised of empathising, defining, ideating, prototyping, and testing  expand on if they need to. You know beyond blue or, you know, Black dog institute or whatever. So, having those numbers there and Lifeline all that. You know, having that there as backup underneath all of all of this stuff for people that are having dark thoughts” (CS07).The stakeholder groups suggested embedding links to credible information. Cancer survivors in particular emphasised that this program should be viewed as a starting point for healthy lifestyle change, and it should provide links to additional resources or mobile applications for users who wish to continue exploring ideas introduced in the program.“Look at what the Cancer Councils already got and put some links in to those resources would be really good idea to be supportive rather than reinvent the wheel” (HCP08).The peer and family support theme encompassed (a) the stakeholders’ need to involve families in the program, both as a supporter of the cancer survivor and as individuals in need of support themselves and (b) to incorporate other various forms of peer support into the program.Offering support for families within the program was strongly identified as a need by the healthcare professionals and cancer support representatives. They recommended providing support either via the cancer survivors’ user portal or by offering family members the opportunity to also sign up to use the program.“What about the carers and what about the family members? They would really benefit from this. If you can click it can go, I’m the parent I’m the patient slash I'm the carer. Because, if the carer can do this and understand their emotions, often a patient and carer or patient and loved one that are looking at one another for support” (HCP03).Providing multiple avenues for peer support in the program was frequently identified by cancer survivors.“Because we all have different ways of looking for peer support. Some are one-on-one, some people like face-to-face support groups or can do it online, or sort of being online anonymously, you know, not like you and I, but where they can just use the discussion board. So, there’s a real wide variety of how people connect with a peer support group” (CS11).Cancer survivors provided recommendations for users to access peer support, often based on their own experiences of the peer support that they found helpful. These recommendations included face-to-face support  and Facebook groups. Healthcare professionals and cancer support representatives more frequently recommended peer support services offered by their organisations, such as Cancer Connect (a free telephone peer support service offered by various Cancer Councils).ideate and prototype stages of the Design Thinking and Research Process co-design framework [Online program. Consistent with the first round of co-design, stakeholders continued to emphasise the importance of addressing mental health, fatigue management, and peer support [This study fulfilled the ramework  by provi support . HoweverOnline, a self-managed intervention. The majority of stakeholders described typical online program users as highly engaged within the first few weeks of a program, before gradually tapering off in interest and engagement. Consequently, the majority of the feedback focused on program features to encourage uptake and longer-term adherence to HLaC Online. These findings support previous investigations into engagement design features, which have consistently found that interventions should be easy to use, relevant to the target population, and include personalisation features, avenues for social support, and some level of guidance through, for example, reminders or a web-support contact [A frequent observation made by all stakeholder groups was that maintaining engagement may pose a significant challenge to HLaC  contact , 36.The stakeholder co-design process generated modifications to several aspects of the program, including simplifying activities viewed as too complex for a self-guided format, allowing consumers to self-select program reminder frequency, and providing further information on locating support from peers and healthcare professionals. These findings were induced and strengthened by the iterative nature of the co-design methodology, in which the current prototype was derived from the initial consultation of stakeholders, and prototype-feedback was then sought from that same group. As a result, stakeholders were enabled to provide guidance as to whether the needs identified in the first round of engagement had been sufficiently met and which needs required further consideration or development.ideate and prototype stages of co-design [Online  in addition to end-users enabled diverse feedback to be collated from cancer survivors, healthcare professionals, and cancer support representatives. Feedback provided by cancer survivor stakeholders largely focused on how to make the intervention relevant and accessible to the diverse cancer survivor population who will ultimately be the end-users of the program . In contrast, the healthcare professional and cancer support representatives drew from their expertise on how to best support users to make and sustain healthy lifestyle and long-term behaviour changes . This diversification of feedback ensured that suggested behaviour changes are accessible to all cancer survivors  and that it included information about the potential cancer- and treatment-related side effects that can complicate the behaviour change process. The benefit of including multiple stakeholder groups, particularly healthcare professionals and representatives from support organisations, has been noted in previous digital health intervention research [The involvement of different stakeholder groups, rather than a single group, enhanced the o-design . Involviresearch .Online based on current stakeholder feedback may not suit the needs of Aboriginal and Torres Strait Islander or other culturally diverse Australian cancer survivors. Future iterations of the HLaC Online program should consider engaging stakeholders from Aboriginal and Torres Strait Islander and other cultural group communities, to ensure the program is culturally safe and meets the unique needs of these communities. Additionally, the study may have been improved with involvement of website design experts , who may have provided additional ideas about what would work within the program which end-user stakeholder could provide their perspectives on. This limitation will be addressed in the next stage of program development, whereby website design experts will be involved in the development of the HLaC Online website.Restrictions on stakeholders’ consultation time and limited cultural and professional diversity in the stakeholder group are two limitations of this study. Focus groups and interviews were time consuming, and engagement often felt rushed, especially with busy healthcare professionals. Consequently, stakeholders may have lacked adequate time to review each wireframe page in depth and only able to provide feedback based on their first impressions. Alternative co-design methodologies to reduce such time-constraints that could be considered in the future include providing the summary of the findings from the previous engagement and the wireframe ahead of engagement to allow more discussion time , or askiOnline. Key changes to the intervention design included offering HLaC Online with a responsive design for use on different devices, guidance videos to assist users completing activities, use of a mood rating as an alternative to the thought record, more information regarding the unique impacts of cancer and its treatment, guidance in each module on how to access relevant healthcare professions, and multiple options for accessing peer support. The feasibility and usability of this design iteration will be evaluated in a pre-post trial prior to testing the efficacy of HLaC Online via a randomised controlled trial.Stakeholder feedback was integrated into the website design of HLaC Online. Specific feedback and advice provided by the stakeholder group has been incorporated to ensure that the content best meets the needs of cancer survivors and supports their undertaking of the self-guided intervention. Future development of digital health interventions utilising the co-design approach should explore alternative co-design methodologies that address the potential time constraints of the stakeholder group and consider the recruitment of multiple, culturally diverse stakeholder groups to ensure the proposed intervention best meets the needs and expectations of their target population.In summary, continuing the co-design process through a second round of stakeholder engagement has further refined the development of HLaC Supplementary file1 (PDF 212 KB)Supplementary file2 (PDF 156 KB)Supplementary file3 (PDF 106 KB)Supplementary file4 (PDF 133 KB)Below is the link to the electronic supplementary material.