Nurses play an important role in promoting positive childhood development via early interventions intended to support parenting. Despite recognizing the need to deliver vital parenting programs, monitoring fidelity has largely been ignored. Fidelity refers to the degree to which healthcare programs follow a well-defined set of criteria specifically designed for a particular program model. With increasing demands for early intervention programs to be delivered by non-specialists, rigorous yet pragmatic strategies for maintaining fidelity are needed. This paper describes the step-by-step development and evaluation of a program fidelity measure, using the Attachment and Child Health (ATTACH™) parenting program as an exemplar. The overall quality index for program delivery varied between “very good” to “excellent,” with a mean of 4.3/5. Development of checklists like the ATTACH™ fidelity assessment checklist enables the systematic evaluation of program delivery and identification of therapeutic components that enable targeted efforts at improvement. In future, research should examine links between program fidelity and targeted outcomes to ascertain if increased fidelity scores yield more favorable effects of parenting programs. Parents influence children's affective and cognitive development, with lifelong impacts , 2. NursIn the United States and Canada, the development and implementation of parenting programs to bolster healthy development of children has increased during the last 20 years . There iThe practice of assessing fidelity in community-based interventions was adopted from trials of pharmacotherapy (drug) trials, in which strict adherence to protocol is a critical requirement , 19. In Evaluation of program fidelity answers the following question: Did facilitators deliver the program as intended? . StrategAssessing program fidelity includes consideration of content and process , 30–32. The cost and time required to develop a fidelity measure, training raters to code the intervention sessions, and establishing inter-rater reliability between the coders, contribute to the lack of reports of the systematic assessment of program fidelity. Although nurses may benefit from using extant program evaluation measures, instruments created for one program may not be promptly adaptable to others  as evaluHow fidelity is measured and how checklists are developed matters a great deal in terms of assessing adherence to any kind of practice –41. PsycTo develop and test the fidelity checklist for ATTACH™, we evaluated existing measures for monitoring fidelity of parenting programs. We determined their applicability to the ATTACH™ program and identified challenges that needed to be resolved for our fidelity checklist. Then we created the ATTACH™ fidelity checklist by tailoring standard recommendations , 44–47 tDetails of the ATTACH™ program and the guiding theory are published elsewhere , 17. BriTo examine utility for the evaluation of ATTACH™, twelve fidelity measures were reviewed to determine if and how content, process, adherence and competence fidelity were assessed . All meaOne of the major challenges or limitations with the extant fidelity measures was the vague boundary between fidelity to content elements (or adherence) and fidelity to process elements (or competence), expected to be consistent with the theoretical model of the program , 45. MosAnother challenge was the inattention to identifying unplanned program elements, an element of process fidelity . To descAdequate pacing, which allows the facilitators to deliver each step of intervention in a certain duration of time without rushing or dragging, is another important aspect of process fidelity . Howeverthinking  and feeling  and what the parent thinks their baby is thinking  and feeling  in those moments.In developing the ATTACH™ Fidelity Assessment Checklist, we extracted elements from the extant measures that best met fidelity requirements for our ATTACH™ fidelity checklist. We created a dictionary of operational definitions of the program elements or checklist items that is available on request. This includes, but is not limited to the definitions of: thoughts and feelings: liked, disliked, and interesting moments; the hypothetical and real life situations; and rushing, connecting, disconnecting, dragging, resistant, going along, and pacing. For example, during the video feedback component of the intervention, the facilitator invites the parent to choose a part of the free play interaction they like or found interesting to ascertain what the parent is Yes (attempted = implemented as intended) or No (not attempted = never asked or failed to perform). Additionally, to evaluate the overall adherence to program content elements, the occurrences of Yes or No elements were simply summed. These numbers were then divided by the total number of elements  , the ComFor ATTACH™ to be paced adequately, the facilitator was expected to deliver each of the five steps in dyadic sessions, and four steps in triadic sessions in 45–60 min. We analyzed video-recorded sessions from pilot studies to determine recommended durations for the steps . For exaAlthough the participants' responsiveness to the program is an aspect distinct from facilitators' adherence or competence, participants' acceptance of a program may impact fidelity . For exaThe overall quality index included an overall assessment of the program delivery after getting a sense of the entire session by reviewing to the video-recorded session for adherence intervention content elements, competence, pacing and participants' responsiveness , 66, 68.As described by , 70, typThe decisions about the percentages and ratings were adopted from fidelity assessments of psycho-educational interventions of both parenting and non-parenting programs , 66, 71.The resultant fidelity checklist was implemented as part of testing of the ATTACH™ program see , 4. TherTo select a representative sample, we selected video recordings from one dyadic and one triadic session for each of 18 participants who completed ATTACH™, by selecting alternate sessions from both dyadic and triadic sessions. Thus, we examined 36 videotaped sessions (from pilot studies #1 and #2). Two trained raters independently coded session fidelity using the ATTACH™ fidelity checklist. We assessed inter-rater agreement between the coders by computing intraclass correlation coefficients  usiThe ATTACH™ Fidelity Assessment checklist was developed to evaluate facilitators' adherence and competence in implementing an evidence-based, psycho-educational parenting program to high-risk families. We found that facilitators adhered to the program content and process fidelity close to 100%. They adequately paced or completed the sessions within 30–45 min without dragging or rushing and maintained excellent participant responsiveness. The overall quality index of program delivery ranged from 4 to 5 on a 5-point Likert scale . Coders for the ATTACH™ sessions helped to further refine the checklist by requesting minor changes.Supported by previous research, we developed the ATTACH™ fidelity checklist to measure the elements of adherence to content and process, as well as participant responsiveness , 62–64. As we reviewed in The ATTACH™ fidelity checklist will be specifically important in evaluating the efficacy of the program on outcomes, exploring ways to improve the program, and learning how to overcome challenges and barriers to program implementation , 55. TheAlthough only two facilitators delivered the ATTACH™ sessions evaluated here, the approach of monitoring and assessment is applicable to multiple facilitators. While we developed the fidelity checklist for training and monitoring fidelity in research contexts, it has not been evaluated in a clinical setting. Although inter-rater reliability was high , only twQuestions remain concerning whether variations in program implementation should be evaluated as part of program fidelity by taking participants' responsiveness into account as moderators of the program effects on outcomes, or both. In future research, weights for different elements may be taken into consideration when critical and core elements have been identified; however, this could be difficult to implement in clinical settings. There are important cost considerations in using fidelity measures , as we fIn future, an extension of this research would examine links between program fidelity and targeted outcomes  to help ascertain if the benefits outweigh the costs , 82, 83.The program content elements are unique to the ATTACH™ program; therefore, the process of developing and evaluating a fidelity checklist is specifically described for the ATTACH™ parenting program. However, the ATTACH™ fidelity checklist described in this paper, may inform and be adapted to fidelity evaluation tools for other parenting program. High fidelity scores in ATTACH™ sessions provided an evidence of validity for the measure. Establishing treatment differentiation between the ATTACH™ and other manualised interventions would provide another future direction or next step in validation of this measure . We haveThe original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.The studies involving human participants were reviewed and approved by Conjoint Health Research Ethics Board, University of Calgary, Calgary, Canada. The patients/participants provided their written informed consent to participate in this study.LA: formulated the research question and oversaw all aspects of manuscript preparation from literature review, found all of the relevant articles for fidelity assessment checklist development, developed and refined the checklist with the help of KB and NL, conducted the data analysis, interpreted the results, and undertook writing and submission. KB: helped oversee the research question, literature review, development of checklist, interpretation of the results, and writing. CE: helped oversee the literature review and reviewed the relevant articles, interpreted the results, and writing. MH: helped review the relevant articles, development and refinement of checklist, interpreted the results, and writing. NL: helped formulate the research question, oversaw the literature review and reviewed the relevant articles, helped in the development and refinement of fidelity checklist, interpretation of the results, and writing. All authors have read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Objective: We aimed to evaluate the fidelity and estimate the effectiveness of a novel health system employee weight-management program. Methods: Employees participating in a weight loss program consisting of self-monitoring, health coaching and meal replacements optionally enrolled in the 12-month study. Longitudinal, single-arm analyses were conducted evaluating change over time via survey, claims and programmatic data. Token participation incentives were offered for survey completion. Results: In total, 140 participants enrolled . During 1 year, participants attended 18.0 ± 12.2 coaching appointments and self-reported significant improvements in weight (−8.2 ± 10.5% body weight), BMI (−3.9 ± 6.5 kg/m2), fruit/vegetable intake, home food preparation, added sugar, sugar sweetened beverages and life satisfaction . No significant changes were reported in physical activity, weight-related social support, self-efficacy or healthcare utilization . Conclusions: The findings from this evaluation establish implementation fidelity. Clinically significant self-reported weight loss, coupled with improvements in many weight-related behaviors, suggest the program is an effective weight management tool when offered as an employee well-being program. Overweight and obesity are associated with increased morbidity and mortality, which in turn have economic impacts related to medical care, productivity and costs in human capital . In the To help address the impacts of overweight and obesity, the Institute of Medicine’s Committee on Accelerating Progress in Obesity Prevention recommends “Expand[ing] the role of health care providers, insurers, and employers in obesity prevention” . AlthougTo date, obesity-related worksite wellness programs have been shown to have a modest effect on weight reduction and healthy weight-related behaviors . Howeverhttps://www.profileplan.com/, accessed on 1 May 2021) is a behavioral weight management program designed and implemented by a large Midwest integrated health system . It is a theory-informed, employee-centered program designed to align diet, activity and lifestyle behaviors with weight management through in-person and phone coaching paired with meal replacement offerings and self-monitoring behaviors. The program, founded in behavioral health theory [Profile by Sanford  employee-centered outcomes ; and (2) health system- and employer-related outcomes  and to investigate program implementation fidelity including participant satisfaction. In the context of continued increases in obesity across the country , a singlThis evaluation was conducted in an integrated health system   distinct from the organization that developed and owns Profile. This member-governed not-for-profit health system, independent from Sanford Health and Profile, employs more than 26,000 people and serves over 2 million patients and members across the Midwest. The health system has an employer-centered Worksite Health & Population Well-being department housed within a larger health and care engagement division. This department currently offers a range of well-being products to its own employer group as well as to other outside employers and maintains a robust digital platform to provide annual health assessments and health promotion programs. These services are implemented in collaboration with health plan employers and customers at their sites for their employees and dependents. Completion of the health assessment and a qualifying program during the plan year allows participating employees to receive a financial incentive of reduced insurance costs and/or co-pays in the subsequent year, typically totaling $250 U.S. dollars. Within the Worksite Health & Population Well-being department and in partnership with the health system’s human resources department, a team of health program specialists are assigned to the health system’s employer group and offer tailored well-being programming for employees. The programming is a blend of on-site, digital and telephonic activities that support a comprehensive range of health and well-being needs. Activity options are personalized based on prioritized health needs identified through a health assessment and claims data.Profile is currently offered to all interested health system employees and adult dependents who receive health insurance through their employer. Employees are offered the program through onsite wellness fairs, digital well-being portal promotions, email communications and other employee newsletters. To enroll, interested individuals complete a screening and intake visit in person or via phone. Employees are eligible for the program if, after being screened by Profile coaches, they meet health criteria, do not have certain food allergies and are not receiving certain medical treatment regimens (such as active cancer treatment). Participants who complete at least 20 Profile coaching sessions during the year are eligible for the insurance incentive. The incentive was not tied to program outcomes , rather to participation in the program.Inclusion criteria were: (1) employed by the named health system above or adult dependent covered by the health system’s insurance and (2) enrolled in the Profile program between March 2018 and June 2019. Exclusion criteria included: (1) age younger than 18 or older than 64; (2) self-reported being pregnant or planning to become pregnant at the time of enrollment; and (3) participating in another weight loss or lifestyle management program. Profile program participants were eligible to enroll in the study regardless of their participation in any of the program components described below.Profile is a personalized weight loss and weight-loss maintenance program that utilizes a one-on-one lifestyle behavior coaching program focused on healthy diet and physical activity. The design of Profile is informed by behavior change theory including Self Determination Theory ,18 and SBehavior change and lifestyle modification are promoted through education and one-on-one coaching appointments with a certified Profile coach. Members have access to 30-min weekly coaching appointments for the duration of the program (up to 52 weekly appointments over the course of a one-year membership) and the ability to meet with a single coach or use different coaches. At coaching appointments, coaches and members review conversations, topics and goals from the previous week; discuss new education and behavior topics such as self-compassion and nutrition label reading; and update or set new goals for the upcoming week. Coaching appointments cover the program detail outlined here and focus on health behavior change strategies such as self-monitoring of behaviors and outcomes, goal setting and problem solving. All Profile coaches have at least a bachelor’s degree in a health-related field and receive extensive Profile training including shadowing and practice coaching sessions. All coaches must successfully complete certification which consists of onboarding and training of at least 40 h in their home location and via the Profile learning management system prior to a 3-day in-person Coach Certification course. Following successful completion of training and upon passing the certification exam, Profile coaches are on a 90-day monitoring plan where their consultations are shadowed and their plans and coaching notes are audited. Shadowing and audits continue throughout every coach’s employment. Lastly, all Profile coaches are required to complete 15 credit-hours of continuing education each year to maintain their certification to coach members.Coaches utilize meal plan protocols to meet dietary needs and personalize to fit individual members, show members how to safely increase their daily activity level and work with members on creating a healthier lifestyle with behavior and habit changes. Coaching is available by phone, video and in-person at Profile locations. Members may also email coaches or send messages over the mobile/web coach app. The program is individually tailored, but typically focuses initially on structured meal plans and meal replacement. Over time, this transitions to fewer meal replacements and more grocery foods as participants acquire and reinforce new knowledge, skills and behaviors to support these changes.This evaluation was designed to help health system leadership assess the effectiveness of Profile when offered as a worksite and health plan incentive program. To allow for adequate time to assess results prior to making subsequent vendor decision and in alignment with the annual Profile program membership, the study was designed to last 12 months. The study was designed by the implementation health system’s affiliated research and evaluation institute (“institute”), in collaboration with leaders from both health systems. The evaluation was also funded in part by both health systems, allowing for alignment of evaluation efforts with strategic goals for the health systems and Profile. However, to maintain appropriate employee confidentiality and minimize potential for perceived bias or coercion, the study was conducted solely by the Institute, which only shared summarized, de-identified results with either organization.This was a prospective, longitudinal study describing the outcomes associated with individuals who enrolled in Profile up to one year post enrollment. Self-reported outcome data were collected at baseline (immediately following program enrollment) and at 3, 6 and 12 months following enrollment. For parsimony, outcomes are presented for baseline, 3 months and 12 months only as findings at 6 months were not unique. With informed consent, retrospective insurance claims data were collected for one year prior to program enrollment and for the prospective year following enrollment. The study is a longitudinal pre/post design with no randomization or control group. Participants enrolled in the Profile program were responsible for the cost of the program ($99 per year membership) and food . The project was reviewed as human subjects research by the HealthPartners Institutional Review Board (IRB) and approved on 2 March 2018 (#A18-037).All eligible employees who enrolled in the Profile program during the study period were notified about the study through a study recruitment email from a Profile coordinator. The message may have also been reinforced by their Profile coach. The study opportunity was also included in the health system’s employee communications, announcing the Profile program as well as on the institute’s website. Profile enrollees were asked to complete a study interest form. Individuals who completed the interest form were then contacted by research staff to screen for eligibility and assess interest in the study. Eligible and interested individuals discussed the consent form over the phone and were asked to verbally provide informed consent to the study. Later, this was streamlined by offering an online consent process, with a phone number available should interested individuals have questions. To minimize the potential of coercion or undue influence, participants were informed that their participation would be confidential and that participation would not impact their relationship with their employer or health plan or their relationship with Profile and individual responses would not be shared. A copy of the consent form was also emailed to all participants who were enrolled in the study.For the purposes of this study, three data sources were triangulated: study-generated survey data, health system insurance claims data and Profile programmatic data. Each data source and the measures associated are described below.n = 20). For the limited constructs (n = 8) where existing questions or scales were not available, questions and response options were designed using best practice [Once individuals consented into the study, they were emailed a 34-item baseline survey to complete online  and weight status according to Center for Disease Control (CDC) guidelines. Two items asked participants about frequency of strenuous physical activity  and amouSix items with Likert scale or categorical response options asked participants to rate their current job performance , their eThree items were used to assess respondents’ emotional health, physical health and general personal goal attainment  and two Eight items asked participants to report their satisfaction with program enrollment, the importance of the insurance incentive offered by their employer for participating, the desire for employer-offered health and well-being programs and their satisfaction with various program components (at 3 and 12 months only). These measures were tracked throughout the program alongside other fidelity measures using programmatic data.Highest education status, race and ethnicity were collected as self-reported items from the survey, while gender and age were collected as self-reported data during the screening process.n = 3). Prescription drug pharmacy claims were categorized by selecting specific American Hospital Formulary Service (AHFS) therapeutic classes thought to be impacted by weight loss [Applicable healthcare utilization, prescription medication and disease diagnosis variables were collected from administrative health insurance claims and enrollment databases by a study team programmer for the year prior to the date of Profile enrollment for each participant. The same set of variables was also collected and summarized for the one year post-enrollment. Due to the COVID-19 pandemic disruptions in healthcare occurring at the end of this post-enrollment period, participants whose post-enrollment period spanned the state’s announced stay at home order date were excluded , Profile meals purchased  and self-weights reported .n = 800) would enroll in Profile, 35–40% of whom (n = 300) were likely to complete the evaluation study enrollment. An estimated 75% (n = 225) would complete 12-month study follow-up. Based on organization data and 2016 Minnesota Behavioral Risk Factor Surveillance System (BRFSS) data [2 with standard deviation of 2.54 kg/m2. After setting the study enrollment goal of 300, there was lower than expected Profile program enrollment, likely from lack of brand recognition and/or competing program options, which diminished the pool of study-eligible individuals. To address this, a post hoc power analysis was done to reassess sample size needs. Preliminary data from 53 respondents were used to understand BMI categories in target population. With 150 participants and an estimated 25% loss to follow-up the study has 80% power to detect a clinically meaningful 5% reduction in BMI using a two-tailed paired t-test with an alpha of 0.05.Profile was offered to all health system employees and covered adult dependents, estimated to total 30,000 individuals. Based on Profile enrollment in other settings, it was estimated that 2–3% of the eligible population (SS) data , the expn = 232) were interested in participating and initiated contact and 87% (n = 202) of these were screened for eligibility. In total, 56 were excluded after screening; 49 of those were ineligible and 7 declined post-screening. Of the 146 who were eligible and consented, 96% (n = 140) completed the baseline measures and were enrolled in the study. Overall, the recruitment rate was 40% of those originally invited to participate. The average number of days between program enrollment and baseline survey completion was 11.6 (SD = 12.0) days. Ninety-three percent (n = 130) of those enrolled completed the 3-month survey and 89% (n = 125) completed the 12-month survey. In total, 354 employees were documented as enrolling in Profile and invited to participate in the current evaluation. Sixty-six percent (n = 140) were non-Hispanic, White and female, with an average age of 51.2 ± 9.8 years and 17.9% had a graduate degree. Mean participant-reported BMI from surveys was 33.2 ± 6.5 kg/m2, with 25.0% categorized as having overweight and 64.7% as having obesity. Approximately one fifth had tried losing weight 10 or more times previously and the self-reported ideal weight loss goal was 22.0%. Ten percent of participants reported doing strenuous physical activity five or more times per week and 20.0% reported consuming five or more servings of fruits and vegetables per day. Most (64.0%) reported generally avoiding sugar-sweetened beverages, but few (9.4%) generally avoided added sugars. While weight-related social support was high (79.3%), 18.6% reported high weight loss-related self-efficacy and few (17.9%) reported preparing 10 or more meals a week at home. Twenty-seven percent of participants reported at least some health-related absenteeism and about half (42.9%) rated their job performance at a 9 or above on a scale of 0–10. Approximately a quarter (23.0%) rated their job fulfillment as a 9 or above on the same scale and even fewer reported having the energy to sustain a workday (19.3%). Some participants reported that emotional and physical health concerns do “not at all” impact their lives . On a scale of 0–10, life satisfaction of 9 or higher was reported by 17.6% of participants.Baseline survey results are presented in 2, p < 0.0001). This reduction was even larger at 12 months but had higher variability . p = 0.042, which did not hold after adjusting for multiple comparisons).Full results can be found in In the year pre-enrollment, 87% of included participants had at least one primary care encounter. Urgent care, emergency department outpatient and inpatient utilization was lower (ranging from 5% to 34% with at least some utilization in these categories). There were no differences in healthcare utilization outcomes in the year post-enrollment. The most common claims-measured disease outcome pre-enrollment was cardiovascular disease (44.0%). Fewer participants had a claim in the 12-month pre period related to asthma or type 2 diabetes  There were no differences in the proportion of people with these conditions pre and post enrollment. The most commonly prescribed medications were respiratory (33.6%), followed by gastroenterological (30.4%) and pain medications (26.4%). There were no statistically significant differences in medications pre and post enrollment. Program satisfaction was high at 12 months with 82% being somewhat or very satisfied (42% being very satisfied), which was slightly lower than overall satisfaction at 3 months . Likelihood to recommend the program was also high at 12 months . At baseline, participants reported high satisfaction with program enrollment , 42% said it was somewhat or very important that the program was offered as a way to get an insurance incentive and 66% strongly agreed that they desired future health and well-being programs from their employer. Full implementation fidelity results can be found in Utilization of program foods, which were not required but highly encouraged, was high with the average participant making 22 unique food purchases and purchasing on average 133 items over 12 months, but variability in these measures was also high. The average amount spent on purchases was $2348 (SD = $2352) over the year and the majority of participants reported using program foods at both 3 and 12 months . However, satisfaction with the food at 12 months was modest (31% very satisfied) falling from 46% at 3 months (data not shown). Participants completed an average of 18 coaching sessions in 12 months, with 69% completing 8 sessions in the first 3 months but only 39% completing 20 sessions in 12 months. Participants recorded an average of 75 weights (SD = 76 weights) over 12 months, with the average duration between first and last recorded weight being 263 days (SD = 111 days) or approximately 9 months. The average number of weights per month was 7.8 (SD = 5.8) with an average of 6.4 (SD = 7.6) days between weights. Almost all participants reported still meeting with a coach at 3 months, which fell to 63% at 12 months. Of those still meeting with a coach at 12 months, general satisfaction was high . Confidence in weight management and lifestyle behavior following coaching was more modest .Profile is a theory-informed, science-driven weight management program designed in close partnership with a healthcare system . This article presents the fidelity and outcomes of a pragmatic, multi-dimensional evaluation of Profile, a novel healthcare-designed behavioral weight management program that utilizes phone coaching, self-monitoring and meal replacements delivered in a worksite setting. The findings from this evaluation establish the fidelity with which this program was implemented when offered as a well-being incentive to a separate healthcare employee population by their employer and insurer. The fidelity of implementation and satisfaction were high, with some concrete opportunities for improvement. The evaluation also provides preliminary evidence of the effectiveness in this population with self-reported improvements in many key outcomes . Although self-reported height and weight have been shown to be impacted by self-presentation bias, these measures are generally accepted as appropriate for comparing longitudinal changes over time as they have very strong correlation with objectively measured height and weight ,40,41. Lprogram enrollment and researchers have consistently documented a higher proportion of women participating in behavioral weight management programs [Data reported here show that a high percentage of eligible participants were recruited, enrolled and retained in the study over 12 months. This type of frame data and ability to report true response rates are not often available in this type of evaluation, which provides greater confidence in the results. The high recruitment and retention rate also increases the likelihood that the study population is representative of the target population . This evaluation did not examine the reach of programs  such as programs .Program fidelity and satisfaction were examined as a part of this evaluation and showed high adherence across the three major components: coaching sessions, program food purchases and self-weighing. There was lower engagement with these components than has been shown in published evaluations of similar weight loss programs ,45. ThatThese variable participation and satisfaction results notwithstanding, self-reported weight loss was substantial with the average participant losing over 8% of their body weight (average weight loss was 17.5 pounds at 12 months) and almost three BMI points at 3 months and maintaining or slightly increasing this weight change at 12 months. Although these findings are limited by self-reported weight data and lack comparison to a randomized comparison group, the high response rate at 12 months and the longitudinal design increase confidence in the findings. Further, the documented upward trend in self-reported obesity across the country and in Minnesota provides additional confidence in the results from this study . SimilarThese pilot results should be used to conduct a future full-scale evaluation with a randomized design and gold-standard objective measurement to account for potential confounding or bias. Nearly three-fourths of participants lost a clinically meaningful 5% of body weight at 3 months and over half maintained 5% or greater weight loss at 12 months, results which are in line with similar clinically oriented weight management programs ,11. EighIn fact, important corresponding improvements were seen in the impact emotional and physical health concerns had on program participants’ lives at both 3 and 12 months. Overall life satisfaction increased over a half a point  as well. These are important changes to note, as baseline well-being and emotional health was lower in this population compared to other employed adults in this region covered by the same health plan . Claims-Lastly, this evaluation sought to understand this type of program’s impact on workplace productivity. Although health-related absenteeism was slightly worse at 12 months, this did not hold following adjustment for multiple comparisons, and participant-reported energy to sustain a full workday improved dramatically at 3 and 12 months . Similar improvements were not seen in reported job performance or job fulfillment, which were relatively high at baseline. This is the first published evaluation showing improved metrics of job performance following an insurance-offered weight loss program in a healthcare employee population . Lastly,This evaluation presented here highlights a number of key strengths and some limitations. Although the study population is relatively representative of the employer population, there is limited variability in the diversity of program participants. The evaluation was able to recruit a high percent of program participants but highlights the need to design and implement culturally sensitive engagement approaches. Similarly, employees who participated in Profile and enrolled in this study were highly motivated with significant prior weight loss attempts. Although this may limit the generalizability of this study, a recent review estimates 42% of the population has tried to lose weight, suggesting weight loss motivation is high in the general population as well . FurtherStudy participation could also have had an independent impact on outcomes. Other key strengths included the gathering of participant voice via surveys and the triangulation of multiple data sources to efficiently evaluate outcomes of interest to diverse stakeholders . Based on the findings of high implementation fidelity, significant and clinically meaningful weight loss and many positive changes in behaviors and well-being outcomes over time, this study found that Profile is a promising behavioral weight loss program when offered to employees in a healthcare setting.The results from this pragmatic evaluation show that the Profile weight management program was implemented with high fidelity and participant satisfaction. Improvements in many key outcomes were seen, with an approximate 8% loss in average body weight at 3 months, improvements in life satisfaction and energy to sustain a full workday and reductions in emotional and physical health concerns; however, there was no impact on claims-related outcomes. The results presented here are a first step in establishing the evidence base for this program and providing information for powering and implementing a full-scale randomized trial.


Given the limitations imposed by the COVID-19 pandemic, a better understanding of how nursing programs around the globe have implemented distance education methods and related initiatives to provide international collaborative learning opportunities as well as complementary aspects of practical education would be constructive for nursing students. It is expected that international collaboratives through web-based communication will continue to be increasingly utilized after the pandemic; therefore, it is time to discuss the effects and direction of these developments.We aimed to examine the impact of an online international collaborative learning program on prelicensure nursing students’ international and global competencies in South Korea.We conducted a mixed methods study (web-based surveys and focus group interviews). A total of 15 students participated in the study. The surveys were used to examine changes in participants’ global leadership competencies, and the focus group interviews were used to evaluate the program’s effectiveness and to identify opportunities for improvement. The online international collaborative program consisted of 7 synchronous web-based classroom sessions. Each session ran for 60 to 90 minutes. Faculty experts and nurses working in the United States discussed various topics with students, such as nursing education in the United States and evidence-based teaching and learning. The students gave presentations on the South Korean nursing education system. Data were analyzed with descriptive statistics, the Mann-Whitney U test, and content analysis methods.Participants reported improvement in their global leadership competencies. Four main categories emerged from analysis of the focus interviews: (1) realistic applicability, (2) clarification, (3) expansion of perspectives, and (4) initiative.The online international collaborative learning program had a positive impact on the development of students’ international competencies. The findings support the further development of international exchange programs through web-based meetings in the postpandemic era. Since the first COVID-19 infection was diagnosed in South Korea, rapid changes designed to contain the spread of the virus have affected all aspects of society ,3. The sUniversity international exchange programs have been especially affected by the COVID-19 pandemic . WhereasAttempts to promote and enhance global competencies via international exchange have taken various forms. The Korean Accreditation Board of Nursing Education  has addeDespite the importance of international exchange programs in developing nursing students’ global competencies, there has been insufficient research to evaluate program content and learning demands. In a study  that comGlobal nursing competencies are reinforced as students from different nursing education systems exchange information and ideas, compare and weigh differences and similarities in nursing curricula, broaden their perspectives, and develop more mature critical thinking abilities . Given tA mixed method design was used in this study. We used quantitative research methods to evaluate the program quality of online international collaborative learning programs and the global leadership competencies of nursing student participants; we used qualitative research methods (focus group interviews) to explore the impact of participating in the program.Undergraduate students applied to and participated in the online international collaborative learning program led by the Office of International Affairs at Ewha Womans University. Students who completed the program with at least 70% attendance were eligible to be included in the study. All 16 students were eligible to participate and were offered the opportunity to participate in the study. Of them, 15 agreed to participate in the study.Global leadership is a competency that positively influences the thoughts, attitudes, and behaviors of stakeholders beyond the national, cultural, and linguistic differences based on open-mindedness and diversity for organizational growth ,17. To mWe used the Student Evaluation of Educational Quality  to evaluThe Office of International Affairs at Ewha Womans University funds an international exchange program each semester; a lead professor plans, supervises, and facilitates the entire program, and students receive a small scholarship upon completion of the program. However, due to the spread of COVID-19, the program was conducted using a videoconferencing platform . The program was operated by the lead professor in collaboration with a nursing professor at Duke University in the United States; the program consisted of 7 synchronous sessions (running 60 to 90 minutes per session) presented from November 17 to December 22, 2020 . The firData were collected from December 23 to 29, 2020 from college-level nursing students who had completed the program. Quantitative data were collected through a web-based survey using Google Forms, and respondents were asked to complete the survey after providing consent to participate in the study. Focus group interviews were conducted via Zoom by a researcher who was not involved in supervising the educational program. Although the number of participants in a focus group interview varies depending on the literature, it has been found that 7 to 10 participants per group are desirable ; thus, 2During the interview, the interviewer summarized answers to the questions, asking whether the summary was accurate and if anything had been missed. Data collection through interviews continued until the meaning of experiences and subject matter reached theoretical saturation . After eU test).Quantitative data were analyzed using SPSS statistical software . The general characteristics of participants were represented by frequency and percentage or mean and standard deviation. Global leadership competency improvement  and program quality were represented by mean and standard deviation. Global leadership competency improvement and participant characteristics were analyzed using a nonparametric method . Participants were allowed to voluntarily access the web-based survey link. At the beginning of the survey, we provided instructions on the purpose of the study, contents, procedures, audiorecording for the interviews, anonymity of the data, and the right to withdraw participation at any time; after that, informed consent was obtained when the user clicked to indicated whether or not they agreed to participate in the study. Confidentiality and anonymity were maintained during data collection and analysis; personal information was not revealed when recorded data were transcribed. Collected data will be discarded after being stored for 3 years.Of the 15 students who agreed to participate, the data from 14 participants were used for quantitative data analysis; data were omitted from 1 participant due to a missing response on the questionnaire. All participants were female .open attitude toward diversity, which increased by a mean of 12.7 (SD 3.9 points) (on a possible range of 0 to 16 points) compared to the preprogram period. Analysis of the relationship between the characteristics of the participants and improvement in global leadership competencies showed no significant difference (P=.39) for second- or fourth-year students. There was a statistically significant difference in the subtheme performance improvement skills in improvement in global leadership competencies, depending on whether or not the participants participated in face-to-face international exchange programs before .Mean improvement of global leadership competencies was 51.1 (SD 17.9) . The mosThe mean program quality score was 48.1 points (SD 3.1) ; among trealistic applicability, clarification, expansion of perspectives, and initiative . This is consistent with the findings of a previous study [performance improvement skills in global leadership competencies was significantly higher (P=.04) for students with international exchange program participation experiences than that for students without these experiences. In this study, performance improvement skill was described as the capability to set organizational goals and utilize necessary information and resources to achieve results. Students with experiences showed more improvement due to synergistic effects of knowledge and information newly acquired through this program combined with knowledge and experiences acquired through previous international collaborative activities. Given these results, it seems necessary to continue to develop and implement similar international programs that can cultivate global competencies in nursing students; however, because this was a single-group postdesign study, quasi-experimental studies are needed to verify the effects of improving global leadership competencies.There was no significant difference in improvement in global leadership competencies between students who had participated in face-to-face international exchange programs before and those who had not  with qualitative data (focus group interviews).Limitations exist because this was a single-group postdesign study, and there is no feasibility study for pre and postcomparison verification. It is necessary to conduct research to confirm the effectiveness of the program based on practical experiences and college grade levels of student participants. In addition, because most research on international exchange programs has focused on field trips, there were limited tools to verify the effectiveness of our web-based programs; therefore, it is necessary to develop a tool that reflects the characteristics of distance education to measure the effectiveness of the program.Previous research on international exchange programs has focused on field trips, yet this study examined the program effectiveness of an online international collaborative learning program for nursing college students. We confirmed the effectiveness of the program in improving global leadership competencies during the COVID-19 pandemic, which had restricted the ability to operate traditional exchange programs between countries. We suggest conducting follow-up studies to verify the mid- to long-term intervention effects of continuous operation of the program, rather than one-off training, after planned incorporation into the nursing education global nursing course curriculum. We further suggest developing programs in connection with various organizations that utilize the advantages of web-based learning environments.


IntroductionTo provide high-quality care, physicians must effectively communicate with adolescents while addressing difficult and sensitive subjects. Our program aimed to (1) cost-effectively incorporate teenage actors into a pediatric simulation program and (2) increase residents' self-perceived comfort level with adolescent patients by practicing interview skills with teens.MethodsIn 2013, the authors established a Teen Acting Program, in which volunteer theater students created and simulated patient characters and provided feedback to learners. Residents on the adolescent medicine rotation participated in the program, completed a survey on self-perceived comfort level with adolescent patients, and answered open-ended questions regarding the program.ResultsA total of 70 residents participated in the program and pre-survey; 46 completed both the pre- and the post-survey. Of 46 participants, 37 (80%) reported that the program was helpful and four (9%) stated it was the best thing about the rotation; 35 (80%) described the program as “high-yield” for the time spent. Between the pre- and post-surveys, residents demonstrated statistically significant improvement in comfort interacting with adolescent patients, addressing confidentiality, and taking histories on sexuality, substance use, mental health, diet, and safety, whether they were the interviewer or observer. Residents that reported the teens taught them specific skills and concepts related to communication. A calculation of program costs demonstrated a 10-fold decrease in cost from traditional simulation patient encounters.ConclusionThe Teen Acting Program was cost-effective and improved resident comfort with interviewing adolescents about sensitive topics, while giving adolescent actors experience honing their acting skills. Adolescent health visits are uniquely challenging. Physicians in training can struggle with asking about mental health, eating disorders, sexual health, and peer influence topics . To commStudies using simulated adolescent patients have shown improvements in objective and self-perceived knowledge, comfort level with interviewing, and clinical skills -9. LearnIn 2013, we established the Teen Acting Program to train theater students from a local high school to create patient characters, act in these roles, and give feedback to residents.ActorsCollaborating with the high-school administration, we introduce the program at the beginning of the school year. Acting teachers recommend students, and we select four to six students annually. The student actors volunteered to participate but did not receive payment nor course credit, yet they did put the experience on their resumes and applications and often asked the faculty for a letter of recommendation.High-school actor training sessionsStudents receive in-person training with follow-up work at home Table . StudentResident educational sessionsEach session is comprised of two to three residents on their adolescent medicine rotation, a pediatric faculty mentor, and between four and six teen actors Table . We holdEvaluation designThe University of Utah Institutional Review Board exempted this study (UUIRB_00065432). We hypothesized that the program would increase residents’ perceived comfort interacting with adolescent patients, addressing confidentiality, and taking histories on sexual activity, substance use, mental health, diet (including eating disorder behaviors), social media, and safety (including dating violence). De novo, we created an 11-question pre-survey and a nine-question post-survey to evaluate the program. For the comfort-level questions, a five-point Likert scale was used, ranging from very uncomfortable to very comfortable. Faculty pilot tested the survey prior to distribution to the learners. No changes in the survey were made during the study period.Pediatric residents consented to the study; they answered surveys on the first (pre-survey) day and last (post-survey) day of the adolescent rotation. Both surveys asked about comfort level with the targeted topics. The post-survey included questions about resident experience with the program, which patients were interviewed/observed by each resident, and the program's usefulness. The post-survey included a final open-ended question: “What specific phrases and/or strategies did you learn work well in interacting with teenagers?” We used McNemar’s test to compare pre- and post-test responses. In an adjunct analysis, we summarized open-ended comments separately to describe important lessons learned . All autSince 2013, pediatric and family medicine residents have interviewed volunteer teen actors as part of the program; during these eight years, the program has been sustainable.During the 29-month evaluation period, 71 residents participated in the program and 70/71 (99%) completed the pre-survey. Of the 71 residents, 46 (65%) completed both the pre- and post-survey questions, of whom 41 (89%) were categorical pediatric residents and five (11%) were combined residents. Thirty-seven (80%) of 46 reported that the program was helpful and four (9%) stated that it was the best thing about the rotation; 35 (80%) described the program as “high-yield” for the time spent.Between the pre- and post-surveys, residents demonstrated statistically significant improvement in comfort discussing all topics, whether they were the interviewer or observer. Residents interviewing teen characters perceived improvements discussing mental health 86%; 18/21), substance use , sexual behavior , disordered eating , and safety . Resident observers reported perceived improvements discussing mental health , substance use , sexual behavior , disordered eating/diet , and safety   of residents reported that the teen actors’ feedback was helpful. Open-ended responses suggested that residents wanted more interaction with the teens and learned important skills related to body language and confidentiality statements. Residents reported the teens taught them specific skills and concepts related to communication Table . The conAnecdotally, teen actor volunteers reported their roles as simulated patients are of utmost importance, as they teach doctors who are often uncomfortable with sensitive topics. The teen actors noted that they benefited from the program by practicing “long-form improvisation” acting skills, developing characters, learning about adolescent issues, watching residents’ skills improve, and being exposed to the medical field.FeasibilitySpace at the high school or clinic and the teen actor time is donated in-kind. Faculty time above a traditional simulation program was four hours a year (approximately $400) to work with the teen actors. The adolescent rotation administrative coordinator scheduled the sessions as part of her existing responsibilities. Breakfast costs approximately $300 per year. Total cost was approximately $700. In contrast, the estimated project costs using our clinical skills teaching center would be $7,000 per year  .Through a novel Teen Acting Program, interviewing and observing simulated teen patients, pediatric residents reported improving their communication skills and comfort level with adolescent health issues. Because residents often feel ill equipped to interact with teen patients ,2, we deThe format of the educational sessions allowed the residents to have time interacting with teenagers, who were trained to give constructive feedback. Previous literature had demonstrated that directed feedback from actors improved long-term interview skills ,10, and With educational costs increasing and budgetary constraints, meeting standards  has becoLimitationsResults are self-reported rather than observed behavior changes. The post-survey may not have been representative as the response rate was only 65%. The program was limited to one pediatric residency program, and teen acting resources may not be readily available for other programs. As the program is part of an adolescent rotation, changes in comfort may not be entirely attributable to the Teen Acting Program.This study was limited in scope. Future study opportunities include observed encounters in vivo in a clinical setting and as an observed structured clinical examination to demonstrate changes in behavior. Using a single administration retrospective pre-post survey could improve the response rate while overcoming possible response shift bias . The teeUsing teen actors as simulated patients was a mutually beneficial arrangement for teaching and learning. The Teen Acting Program provides a unique setting to give feedback to learners and evaluate communication skills. Teen actors shared their expertise with medical professionals and hone their acting skills. Residents engaged with a teen trained to provide honest feedback that was directly applicable to clinical practice.Since the program uses volunteer teen actors, its financial costs are low, it provides a relatively authentic experience, and it has reported benefits. Medical schools and residency programs can replicate the program to teach important skills with a limited budget.


Web-based mindfulness programs may be beneficial in improving the well-being outcomes of those living with chronic illnesses. Adherence to programs is a key indicator in improving outcomes; however, with the digitization of programs, it is necessary to enhance engagement and encourage people to return to digital health platforms. More information is needed on how engagement strategies have been used in web-based mindfulness programs to encourage adherence.The aim of this study is to develop a list of engagement strategies for web-based mindfulness programs and evaluate the impact of engagement strategies on adherence.A narrative systematic review was conducted across the MEDLINE Complete, CINAHL Complete, APA PsycINFO, and Embase databases and followed the PRISMA  guidelines. Articles were screened using the population, intervention, comparator, and outcome framework. Adults aged >18 years with chronic health conditions were included in the study. Mindfulness interventions, including those in combination with mindfulness-based cognitive therapy, delivered on the web through the internet or smartphone technology were included. Interventions lasted at least 2 weeks. Studies with a randomized controlled trial design or a pilot randomized controlled trial design were included. Engagement strategies, including web-based program features and facilitator-led strategies, adherence, and retention, were included.A total of 1265 articles were screened, of which 19 were relevant and were included in the review. On average, 70.98% (2258/3181) of the study participants were women with a mean age of 46 (SD 13) years. Most commonly, mindfulness programs were delivered to people living with mental health conditions . Of the 19 studies, 8 (42%) used only program features to encourage adherence, 5 (26%) used facilitator-led strategies, and 6 (32%) used a combination of the two. Encouraging program adherence was the most common engagement strategy used, which was used in 77% (10/13) of the facilitator-led studies and 57% (8/14) of the program feature studies. Nearly two-thirds (63%) of the studies provided a definition of adherence, which varied between 50% and 100% completion across studies. The overall mean participant compliance to the mindfulness programs was 56% (SD 15%). Most studies  had a long-term follow-up, with the most common follow-up period being 12 weeks after intervention . After the intervention, the mean retention was 78% (SD 15%).Engagement strategies in web-based mindfulness programs comprise reminders to use the program. Other features may be suitable for encouraging adherence to interventions, and a facilitator-led component may result in higher retention. There is variance in the way adherence is measured, and intervention lengths and follow-up periods are inconsistent. More thorough reporting and a standardized framework for measuring adherence are needed to more accurately assess adherence and engagement strategies. Mindfulness is the act of bringing awareness to the present moment in a nonjudgmental and accepting way . MindfulEvidence shows that mindfulness skills can be improved through greater engagement with meditation, home practice, face-to-face contact with a facilitator, and a higher number of sessions per week . High adMindfulness programs are increasingly being adapted to web-based platforms, providing opportunities for more people to participate compared with conventional face-to-face sessions . TypicalAdherence to web-based programs in previous reports has varied between 39.5% and 92%   to develop a list of engagement strategies for web-based mindfulness programs and (2) to evaluate the impact of engagement strategies on adherence.This systematic review was guided by the PRISMA  framework . The folTo guide the eligibility and screening process, the PICO  framework  was usedAdults aged ≥18 years with a diagnosed chronic health condition or self-reported anxiety or depression were included in the study.Mindfulness interventions delivered on the web through the internet or smartphone technology were included. Mindfulness programs were defined as those focusing specifically on mindfulness-based practice, including programs using a combination of mindfulness and cognitive behavioral therapy (mindfulness-based cognitive therapy).To allow for engagement strategies and adherence to be analyzed, the interventions had to be at least 2 weeks in duration. There is limited research to describe how long interventions should be to warrant the inclusion of engagement strategies. Previously, engagement was measured by reflecting on the previous 2 weeks . TherefoStudies were required to have a comparison group with a randomized controlled trial (RCT) or a pilot RCT design.Mindfulness programs developed by research groups for specific populations or commercially available mindfulness programs were tested in controlled trial settings.Program adherence, study retention rate (%), and strategies such as web-based program features and facilitator-led features were included.Covidence to allow cross-checking between authors [Retrieved articles were uploaded and managed by Endnote X9 . Duplicates were removed, and titles and abstracts were screened by 1 author (NH). Full-text articles were uploaded to  authors . Full teA data extraction tool was developed in Microsoft Excel to standardize the extraction. Data were extracted by 1 author (NH), and 10% were cross-checked by the second author (PL).Study data including author, year of publication, country, design, number of participants, intervention type, intervention duration, follow-up measurements, prior mindfulness experience, recruitment method, financial compensation, commercial app name, primary outcome, and primary findings were extracted.Gender, age, race, ethnicity, type of chronic illness or condition, and patient and caregiver status were extracted.Studies were included in the review when they reported per-protocol and intention-to-treat analyses. Because of variance in reporting the intervention, adherence was assessed in 3 different ways depending on the data available:As a percentage of compliance with the intervention protocol. For example, some authors defined adherence as 80% program completion, and in this review, we recorded the percentage of the sample that was adherent with 80% program completion.In groups defined by the study authors. For example, in an 8-week program, some authors reported the percentage of people who were adherent with 0- 3 sessions, 4- 6 sessions, and 7-8 sessions. In this review, we recorded the percentage of the sample that was adherent with the highest group of completion, for example, 7-8 sessions.Summarized findings of the frequency and duration of use.Retention rates were reported for the intervention group at postintervention measurements and subsequent follow-up points.Engagement strategies were categorized into following 3 groups:Program features, including chat rooms, discussion boards, diaries and reflective processes, automated reminders, social support, goal setting, mood tracking, customization of content, demonstrations of meditation practice, and immediate feedback on meditation practice;Facilitator-led strategies, including reminders from the research team to continue practice, contact with the research team to discuss practice or monitoring, and response to well-being scores throughout the intervention; andA combination of program features and facilitator-led strategies.Study characteristics, participant characteristics, adherence, and retention rates were analyzed using descriptive statistics.Data analysis consists of the following:Exploring adherence: how adherence was defined, the impact of adherence on outcomes, impact of financial compensation on adherence, and impact of intervention length on adherence.Describing retention at postintervention measurements and the last data collection point.Describing engagement strategies : engagement strategies were categorized and summarized using frequency statistics.Assessing the impact of engagement strategies on adherence: the relationship between engagement strategies and adherence was analyzed by comparing the type of engagement strategy  with the percentage of people who reached program adherence or the percentage of people who adhered with the highest group of sessions .Assessing the impact of engagement strategies on retention: the relationship between engagement strategies and retention was measured by comparing the type of engagement strategy with the intervention length, retention at the postintervention measurement and retention at the last follow-up points.A total of 1922 articles were retrieved from the databases and reference lists. After removing duplicates, a total of 1265 articles were screened by title and abstract. Full texts were retrieved for 126 articles, of which 19 were included in the review . Most stA total of 8 (47%) studies focused on psychological measures as their primary outcome ,37-39,42A total of 34,601 participants were included in the trials. The mean sample size was 165 . On average, 71% of the participants were women  and 46 years old . A total of 8 (42%) studies reported the ethnicity of the participants ,29-35,39A total of 8 (42%) studies used only program features to encourage adherence, 5 (26%) used only facilitator-led strategies, and 6 (32%) used a combination of the two .Within the facilitator-led strategies (n=13) -38,42,43Within program feature strategies n=14)  14,24,2,33,37-42,37-4233,Contact initiated by facilitators or program reminders was most commonly delivered by email 9/14, 64%) [, 64% 14-38,40-42Nearly two-thirds 12/19, 63%) of the studies provided a definition of program adherence % of the ,33,36-42Among studies that used only program features (n=8) ,32,39-41When examining studies that used program features, of the studies that used 1 strategy (n=6) ,14,39-42A total of 10 (53%) studies analyzed the relationship between outcome variables and adherence ,38-40,42Of the 6 studies that provided any type of compensation, 2 measured adherence with a mean of 42%  ,36. AmonThe impact of the intervention length on adherence was analyzed. Of the 5 studies with an intervention <8 weeks, none recorded adherence. Those with an 8-week intervention recorded an average of 58% (SD 16%) adherence  ,39,40,42Most 10/19, 53%) studies conducted pre-post analysis with additional follow-up points ,38,39,42% studiesStudies that applied only facilitator-led strategies, on average, were 6 weeks in duration  and had a retention rate of 93%  compared with studies with a combination of program features and facilitator-led strategies with a mean duration of 16 weeks  and a retention rate of 75%  and those with only program features with a mean duration of 8 weeks  and retention rate of 67% .Of the studies that used facilitator-led strategies only, 40% (2/5) had follow-up periods after postintervention follow-up ,36. On aStudies that used only program reminders as engagement strategies (n=4) ,14,40,41In this review, we described the engagement strategies applied to web-based mindfulness programs and their impact on adherence rates. The use of program features only was associated with program adherence but not with maintaining study retention. Engagement strategies were largely reminders to use the program and, to a lesser extent, the ability to customize program content, interact with features, or engage with content on a deeper level through reflections, homework activities, and discussions of content with facilitators. There was little difference between the type of engagement strategy used and adherence to programs or retention rates.The need to accurately report study and program attrition to better understand the associations between program adherence and health outcomes has been established ,12. Our Similarly, the ability to measure the impact of engagement strategies on study attrition is limited. The findings suggest that studies using only facilitator-led strategies were favorable for maintaining study retention ,34-36,43Most studies in this review showed that web-based mindfulness resulted in improvements in either psychological or physiological outcome measures ,37,39-43Program features applied throughout studies to enhance engagement varied according to the type and number of features available to users. Furthermore, the number and type of features included had similar impacts on program adherence and study retention, suggesting that there may not be one superior feature to be included in programs. Features such as diaries, reminders, and social connectedness are commonly used in interventions as behavior change techniques , and webAcross studies, there was a large variance in interventions and in reporting adherence. These factors made it difficult to draw any firm conclusions from the data.The sample of the included studies was predominately White and female, which limits the generalizability of these findings to other population groups.This review aims to describe the influence of engagement strategies on adherence and retention among people living with chronic illnesses or conditions. Other studies measuring adherence to mindfulness in the general population may have provided additional information on the impact of engagement strategies. However, there is a need to evaluate engagement and adherence to web-based interventions, specifically in people living with chronic illness. People with chronic illness may be more likely to experience depression and anxiety symptoms than those without a chronic illness . Lower mFurthermore, the primary outcome of the review was to assess adherence, retention, and engagement strategies rather than to draw conclusions about the effectiveness of interventions on patient outcomes. As a result, the risk of bias assessment was less relevant.Engagement strategies in web-based mindfulness programs largely comprise reminders to use the program. The impact of other features such as personalization, self-reflection activities, and lesson summaries on adherence requires further investigation. There is variance in the way adherence is measured, and intervention lengths and follow-up periods are inconsistent. More thorough reporting and a standardized framework for measuring adherence are needed to more accurately assess adherence and engagement strategies.


It is important to improve verbal Working Memory (WM) in reading disability, as it is a key factor in learning. There are commercial verbal WM training programs, which have some short-term effects only on the verbal WM capacity, not reading. However, because of some weaknesses in current verbal WM training programs, researchers suggested designing and developing newly structured programs that particularly target educational functions such as reading skills. In the current double-blind randomized clinical trial study, we designed a new Verbal Working Memory-Balance (VWM-B) program which was carried out using a portable robotic device. The short-term effects of the VWM-B program, on verbal WM capacity, reading skills, and postural control were investigated in Iranian children with developmental dyslexia.The effectiveness of the VWM-B program was compared with the VWM-program as a traditional verbal WM training. In comparison with VWM-program, the participants who received training by the VWM-B program showed superior performance on verbal WM capacity, reading skills, and postural control after a short-term intervention.We proposed that the automatized postural control resulting from VWM-B training had a positive impact on improving verbal WM capacity and reading ability. Based on the critical role of the cerebellum in automatizing skills, our findings support the cerebellar deficit theory in dyslexia.Trial registration: This trial was (retrospectively) registered on 8 February 2018 with the Iranian Registry of Clinical Trials (IRCT20171219037953N1).The online version contains supplementary material available at 10.1186/s12868-021-00660-1. Developmental Dyslexia (DD) is characterized as a difficulty in learning to read accurately and fluently , despiteMany studies have supported that reading difficulties in children with DD are due to phonological deficits, explained by the phonological deficit theory , 5, 6. TThe cerebellar deficit theory, supported by several studies, has concluded that insufficient integration of information due to mild neurobiological impairment in the cerebellum is responsible for deficits in DD , 27–29. www.cogmed.com), Jungle Memory (www.junglememory.com), and Cognifit (www.cognifit.com) are currently used to improve the verbal WM capacity [Numerous computerized programs including Cogmed  or the same WM battery under dual-task balance conditions (intervention group) which are hypothesized to improve performance. In the present study, a dual-task, Verbal Working Memory-Balance (VWM-B) program, was designed and its short-term effects on verbal WM capacity, reading skills, and postural control were investigated in Iranian children with DD.This quasi double-blind randomized clinical trial study was performed with a between-subjects factor ‘group’ (control group vs. intervention group) and within-subjects factor ‘time’ (measurement at pre-intervention and post-intervention) and has adhered to CONSORT guidelines  was used to test the IQ and subjects with a WISC-IV total score ˂ 85 excluded from the study . Also, tAccording to Fig. For double-blinding in the current study, children and their parents were unaware of the group to which the children had been allocated. Also, an evaluator who was not a member of the research group blinded to the subjects’ groups performed the pre-intervention and post-intervention behavioral assessments. Recording the Center of Pressure (CoP) data using a force plate .Based on Baddeley’s theory, verbal WM includes encoding, maintenance and manipulation of verbal information, and retrieval sub-processes . FundameWhile training with the VWM-program, the subject sat on a chair in a relaxed mood with arms resting on the table. A 19-inch touch screen monitor ran the software, which was specially designed for the proposed training program, and a speaker was used to recite the words. As mentioned above, each training trial of the VWM-program includes all three sub-processes  of the verbal WM. Each trial began 3 s after touching the start button on the monitor . The mean (SD) for quantitative variables and the absolute frequency (%) for qualitative variables were reported. In the current study, both two groups had received the intervention. Hence, the mixed between-within ANOVA analyses were used to verify the treatment effects over time [p < 0.05) and effect size, with partial eta squared (ηp2), were reported [p2 is in the family of correlational effect size [The normality was tested by Shapiro–Wilk . Dependiver time . Signifireported . The ηp2ect size  and is mect size . There aect size . Using mp > 0.05), and the independent t-test showed no significant between-group difference at baseline scores. However, the hypotheses of normality for age and CSI-4 scores were rejected (p < 0.05), and the Mann–Whitney U-test showed no significant between-group difference at baseline scores. As a result, two groups were homogenized (p > 0.05) for age, height, weight, full-IQ, and attention, as possible confounders (Table p > 0.05) was found for baseline scores of clinical and CoP measures  age of 8 1.22) years. The Intervention group included 15 children with DD, with a mean (SD) age of 8 (0.96) years. The hypotheses of normality for height, weight, and full-IQ scores were accepted  of the dyslexic and non-dyslexic children . HoweverIn the present study, participants who had received training by the VWM-B program showed higher performance in verbal WM and reading skills. Also, the current research demonstrated that improved postural control was correlated with verbal WM and reading ability. Furthermore, verbal WM was correlated with reading ability. Although the current research has emphasized the cerebellar deficits hypothesis to interpret the results, the causal link between the balance deficits and reading problems is still under controversy , 92–94. The cerebellar deficit hypothesis has also emphasized the impairments in the procedural learning system, with specific deficits in the language/cerebellar procedural circuits . Almost Although it needs future neuroimaging studies to adequately investigate the changes in the activation of the cerebellar circuits after treatment with the VWM-B program, previous neuroimaging studies have confirmed the role of the cerebellum in verbal WM, reading, balance, and complex actions , 106. FoThere are limited studies that investigate balance training effects on children with DD. For example, Goulème et al. demonstrated the effect of balance training only on postural control ; howeverThe present study is a pioneer in investigating the effectiveness of a newly structured VWM-B training program. This program provides a dual-task condition including cognitive  and motor  tasks. This study demonstrated that the VWM-B program, after the short-term treatment, is more effective than the VWM-program in the improvement of verbal WM capacity, reading skills, and postural control in the children with DD. The improvement in postural control  probably had an effective role in improving the measured cognitive functions. It seems that the automatization in balance-related movements consequently led to assigning further neural resources to the cognitive task. The Cerebellum has a critical role in maintaining postural control and automatizing skills; therefore, the activation of the cerebellum regions may be changed after the intervention by the VWM-B program. Despite the results of the current studies that support the cerebellar deficits hypothesis, the role of the cerebellum in DD is still controversial.Additional file 1: Appendix S1. CONSORT checklist.


Out of school hours care (OSHC) services provide a unique opportunity to deliver early intervention programs to enhance primary school–aged children’s social, emotional, physical, and cognitive well-being; however, such programs are currently lacking.This study aims to address the lack of well-being programs for children accessing OSHC services in the research literature by using participatory design (PD) to collaboratively develop and test an OSHC well-being program—the connect, promote, and protect program (CP3).The study employed methods of PD, user (acceptance) testing, and iterative knowledge translation to develop a novel well-being program framework—CP3—with key stakeholders . Thematic techniques were used to interpret and translate the qualitative information obtained during the research and design cycles.The co-design process generated the CP3 model, which comprises a group-based mentoring approach to facilitate enhanced activities in OSHC settings. Activities are underpinned by 4 key principles of program delivery: build well-being and resilience, broaden horizons, inspire and engage, and connect communities.To our knowledge, the CP3 program is the first co-designed well-being program developed specifically for OSHC services. This co-design process is key to ensuring local community needs—particularly those of young people accessing OSHC—are met and that these individuals are meaningfully and actively involved in all stages of the research and design process, from conception to implementation, evaluation, and continuous improvement. In the most recent report by the Australian Early Development Census (AEDC), 22% of primary school–aged children were found to be vulnerable to experiencing a developmental delay in one or more areas of functioning . This inOut of school hours care (OSHC) services offer a safe and supervised environment for primary school–aged children before and after school. These centers provide vital services for many families by enabling parents and caregivers to achieve a balance between childcare, social responsibilities, and work . In Austin-betweeners, as they fall in between the toddler and postpubertal groups  need to capitalize on this and ensure that the program doesn’t just turn into another OSHC.Community worker, workshop 3The value of mentoring was also highlighted throughout the workshops:Including the mentoring component in the program might have positive impacts for the wider community, as it plants the seed for growth and can broaden perspectives.Community member, workshop 2The mentoring component was not only seen as beneficial to the children accessing the OSHC but also viewed as giving the mentors themselves skills, confidence, social connection, and “a feeling of ‘giving back’” .Concerns were raised about child protection, and an emphasis was placed on the need to ensure that the program uses “...the right people in the right capacity” . It was the prevailing view that such issues could be addressed through rigorous mentor recruitment, training, supervision, policies, and procedures.enhanced activities as they are “...more than just extracurricular activities” , which was subsequently accepted and adopted in the later workshops. Enhanced activities were viewed as the vehicle for carrying out the CP3 principle of broaden horizons—as the activities would be enriched, allow children to develop new skills, and contribute to their personal and social development. Some participants viewed this program component as particularly beneficial for more vulnerable children who might access CP3:In all workshops, the suggestions generated by participants highlighted that the OSHC activities on offer in CP3 should be enriched and enhanced, especially when compared with regular OSHC services. The term created for this component by participants in early workshops was Enhanced activities would be wonderful. Especially as they can be completely out of reach for some young people.Parent and community worker, workshop 1inspire and engage and went hand-in-hand with the mentoring component: “The mentoring and activities should create a spark for the child” .Enhanced activities were viewed as needing to be stimulating to ensure that the children were engaged and motivated to take part. This was directly related to the CP3 principle of The overarching, iterative feedback generated during the workshops was chiefly positive:This type of program could have huge benefits for wider community change as it sets out to make positive community connections—this can be powerful on a large scale and be a catalyst for huge community change.Community worker, workshop 3When prototyping the mentoring component design, participants developed a plan for group-based (collective) mentoring, otherwise defined as collective mentoring. The collective mentoring of children in group settings was viewed as more beneficial in an OSHC environment, compared with one-on-one mentoring, as it addressed concerns relating to program acceptability, matching children with mentors, mentor recruitment, and turnover, and this could easily run alongside general OSHC activities....mentors should be passionate about what they are teaching...”  to motivate, inspire, and engage children in CP3. The second type of mentor identified was a CP3 mentor, trained in CP3 principles, and could provide support to the enhanced group-based activities as well as the OSHC’s day-to-day running. Peer-to-peer mentoring was also proposed as an additional avenue for CP3 to engage primary school children attending OSHC to take on a leadership role, which reflected the inspire and engage CP3 principle.To enhance mentoring options for the children accessing OSHC and ensure CP3 was not a “...blanket one size fits all program...” , a 3-level approach to mentoring was generated during workshop discussions. This included skill-based mentoring, CP3 mentoring, and peer-to-peer mentoring. Skill-based mentoring meant that mentors with special skills would facilitate activities in their area of expertise. It was highlighted that these “Specialized CP3 training, designed for both staff and volunteer mentors, was seen as crucial to the delivery of CP3. Prototyped areas of training included vision and mission of CP3; mentoring processes and relationships; building emotional literacy; child development; working with special needs; managing challenging behaviors and situations; referral pathways and support; and risk management and safety.When prototyping the enhanced activity component, participants highlighted that during the implementation of CP3, the program would need to avoid activities being delivered in a “piecemeal manner...” , that is, there needed to be a coherent structure to the program, where activities link together to form a greater purpose of working toward the CP3 principles:The building blocks system or foundation as part of the program—where it’s not just one lesson and then move on will be important. It needs a framework that everyone is privy to.Educator, workshop 1My Time Our Place Framework and the National Quality Standards. The tool also supports reflective practice and sharing of ideas. An example summary page from the CP3 activity development guide is provided in On the basis of this feedback, a CP3 activity development guide was prototyped. This is a tool for selecting and designing enhanced activities. It ensures that the staff and children think purposefully about programming so that it provides every opportunity to enhance the experience in terms of the CP3 principles, the A total of 2 additional features of CP3 were identified, which included the provision of one-on-one well-being support for children with greater needs and involving families meaningfully.The idea generated by participants that CP3 could provide additional one-on-one psychological support for children with additional biopsychosocial needs, such as “...if there was a grief issue or if there was a diagnosis that required further support...” , received positive feedback when iteratively evaluated. Participants emphasized that if additional support was offered, it would need to be carried out by a registered psychologist or other qualified health professionals. The provision of such additional support was seen as particularly beneficial for the prevention and early intervention of social, emotional, physical, or cognitive difficulties....build[ing] the capacity of parents...” , which included developing a resource kit for parents, providing support pathways and “...link[ing] parents with counseling services...” , “...resources to support their children effectively...” , such as “...active parenting programs...” , “...positive parenting programs or circles of security...” . Providing clear communication channels such as a “...feedback cycle between the child, families and school...” , finding out “...positives about their children through feedback from the program...” , telling parents “... about the focus of the learnings... for example, we are going to talk about character and strength this week...” , and creating a CP3 newsletter or social media page  was recommended. Third, building a sense of community for parents, such as providing a “...chance to meet and interact with others of similar interests, problems etc...”  and having an “...open day...” .Participants also recommended that “...there needs to be a whole family approach...”  for CP3 implementation. Ideas generated included CP3 “A stepped approach to implementation was raised as a possibility in the workshops for the development and evaluation of CP3. In the knowledge translation phase, this idea was refined into 3 components: CP3 Lite, CP3, and CP3 Plus  using the CP3 activity development guide. CP3 Lite is facilitated by OSHC educators and qualified community experts. Example excerpts from the CP3 activity planning process, which led to the establishment of the CP3 activity development guide for training and trialing, are presented in CP3 Lite is the minimal viable product of CP3 (α-build). This component is the first implementation step and provides enhanced activities underpinned by the CP3 principles  research methods to develop a novel health and well-being program for primary school–aged children aged 5-12 years) to be delivered in OSHC: CP3. To our knowledge, CP3 is the first health and well-being program model designed specifically for OSHC settings that allows tailored interventions to be developed depending on the unique needs and preferences of the end users, including children (in later stages), their parents and guardians, staff, volunteers, and the broader community. CP3 adopts a holistic, community-focused approach, encouraging active participation of community members, peer-to-peer and adult-led mentoring, and interventions that not only focus on physical development but also foster social, emotional, and cognitive well-being. In this way, CP3 addresses the goals and objectives of the AEDC  and OECD years tobuild well-being and resilience, broaden horizons, inspire and engage, and connect communities) ensure that the goals of CP3 interventions can be clearly delineated. This is critical, as one of the pitfalls in the implementation of new well-being programs is that they often fail to adhere to the core components of best practice and frequently do not use a program model [connect communities. Currently, the available evidence in the literature indicates that for a program to be effective, it is necessary to follow best practices in recruiting, training, and providing ongoing support and supervision to mentors [CP3 addresses a major gap in the literature and in the delivery of universal health and well-being programs in educational settings. Unlike existing OSHC programs, which tend to be prescriptive, narrowly focused, and nongeneralizable, CP3 offers a framework for flexible program development and delivery while ensuring that a high standard of program development will be maintained. The 4 CP3 principles co-designed during PD workshops , which involves iterative user (acceptance) testing via a naturalistic formative service evaluation of the implementation CP3 combined with further PD workshops. This stage will test and refine the ideas generated in stage 1 in partnership with a wider group of stakeholders associated with the OSHC  to inform a more comprehensive CP3 model (β model). In the future, stage 3, a real-world cluster randomized controlled trial will be carried out on the CP3 model (β model).connect communities) and meaningfully engaging children in the decision-making process (inspire and engage) emphasize the importance of engaging end users at all stages of the intervention development process. Children themselves form part of the co-design process; however, this research is still underway, as it forms part of the evaluation and thus will be reported elsewhere. This co‐design and collaborative management means that the OSHC can be delivered according to the communities’ strengths while ensuring that the level of program consistency is maintained. Despite these benefits, the use of PD methods is also challenging. For example, in this research, PD workshops could only take place in English because of budget limitations, that is, this research did not have funds to provide translators and to translate all study materials (such as consent forms and participant information statements). This may limit the generalizability of the research, although people who spoke English as a second language participated. Interestingly, the percentage of individuals who only speak English at home  accurately reflected the demographics of the Illawarra region (80.6%) [In designing the CP3 α model, an iterative PD approach was employed that placed key stakeholders at the center of the design and development process. This process of co-design and development will continue to be used, as CP3 is implemented and evaluated in stages 2 and 3. These co-design research methodologies are also embedded in the program design itself in the continuous process of re-evaluation and re-responding to community needs as children and their communities grow and change over time. For instance, the CP3 principles of community collaboration ( (80.6%) . FurtherMy Time Our Place Framework [National Quality Standards [Research suggests that health programs can take up to 17 years to move 14% of original research into actual service delivery . Howeverramework  and the tandards . Using ttandards .CP3 is currently undergoing a formative evaluation, and plans are being made for future full-scale evaluation. These evaluation stages of research are crucial, as research suggests that many new mentoring programs are pursued without any supporting evidence from reliable or valid process or outcome evaluations ,38. Furtearly adopter sites may require a different focus. Specifically, the need for effective staff by in and change management may be required when CP3 is introduced into already-operational OSHC sites. Ultimately, the competence and capacity of local facilitators will be crucial for successful implementation. This will be evaluated as CP3 is rolled out further in already-established OSHC sites.Additional PD with children at multiple OSHC sites will occur from 2019 to 2021 as part of the formative evaluation of CP3 and thus are yet to be reported. Further plans are also being made to measure the effectiveness of the CP3 model in a large-scale randomized controlled cluster trial. The major challenge is ensuring that engagement continues to be high when research extends to new sites. There is a possibility that successful PD engagement is because of the nuances of the pilot OSHC community. For example, the first pilot OSHC site for CP3 was a brand new service; thus, a focus on culture change to move away from a traditional OSHC model toward the CP3 is not required, whereas other already-established OSHC To our knowledge, CP3 is the first co-designed health and well-being program to be delivered to primary school–aged children in an OSHC setting. The co-design process is key to ensuring that local community needs are met and that they are meaningfully and actively involved in all stages of the research and design process, from conception to implementation, evaluation, and continuous improvement. By providing a framework that encourages tailored interventions to be developed depending on the unique needs and preferences of the end users , CP3 takes an important step forward toward achieving universal access to a holistic health and well-being program for all children. The CP3 model is currently under evaluation, and the results will be used to determine the overall success and inform ongoing development and implementation.