During the COVID-19 pandemic, many psychiatry residencies  have adopted different learning modalities to preserve a high quality of educational training. There is minimal data on specific program adaptations, related change perspectives, and program type stratification. This study sought to examine trends in curriculum changes in accredited psychiatry residency programs in the United States.Program directors of accredited general psychiatry programs in the United States were surveyed to assess both general program details and changes in educational content, delivery, and perspectives with regard to program curricula.p = 0.13). Furthermore, changes were noted with respect to didactic content (60%), decreased rotation sites (38%), and increased telemedicine exposure (84%) with no change of wellness days (67%) or protected time (97%). Again, the above changes had no association with program type. Use of virtual educational platforms was described as positive (59.7%) with 9 programs noting a future transition to a hybrid learning model for didactics and grand rounds.A total of 63 program directors out of 264 eligible programs completed the questionnaire (23.9%). There was a significant shift to integrate virtual learning post-pandemic (98.5%) compared to pre-pandemic (3%). However, there was no association between these modality changes and program type (The findings suggest that pandemic-related curriculum adaptations occurred in all different types of psychiatry residencies and suggest a national trend to continue virtual educational platforms with regard to psychiatry didactics. However, future investigation of effectiveness of virtual learning programs in psychiatry residencies is warranted. Today, residents train in a system that has adapted to promote competency through clinical, didactic, and evidence-based learning. These include direct patient management, grand rounds, didactic education, and journal clubs. Historically, trainees have had the option to train at either community-based or university-affiliated academic hospitals, each with unique advantages. Community hospitals are generally smaller, more closely aligned with the community, and provide practical experience in settings residents are likely to practice in upon graduation.3 In comparison, academic hospitals are generally noted for increased patient diversity, resource-intensive services, and opportunities for post-residency training. However, the common mission to advance medical education and patient care has resulted in the production of hybrid models that have characteristics of both academic and community programs.Psychiatry residency education has evolved in response to social, economic, and cultural forces that shape the patterns of practice. Over the past 75 years, psychiatry education has shifted from irregularly structured curricula to more regulated process and requirement-driven models. Some residency programs substituted simulations for procedures and used team rotation systems to optimize the number of in-hospital residents for safety reasons. Non-immune to the pandemic, psychiatry programs may have needed to adopt many of these adaptations to maintain a quality educational experience for residents. Furthermore, the pandemic also negatively affected the mental health and quality of life for residents. Psychological pressure with managing COVID-19 positive patients with limited protective equipment, isolation from family in fear of infection transmission, and excessive working hours may have contributed to burnout and unhealthy mental states that impair learning.In March 2020, the COVID-19 pandemic completely shifted the landscape of residency education. In response to the lockdown, several tactics were used to maintain the delivery of quality education while mitigating the risks of the potential infection from the virus. Education shifted from the classroom to virtual spaces, clinical learning was replaced with remote modular learning, and services were reconfigured to ensure the mental health and well-being of residents.To date, there are minimal studies that have examined the adaptations made specifically for psychiatry residency programs secondary to the pandemic. The perception of these changes amongst educational psychiatry leaders also remains uncharted. Additionally, no studies have investigated the pandemic-induced changes stratified based on program type. It remains unclear whether certain adaptations demonstrated more benefit in a particular program type. This study aims to analyze curricular trends across accredited psychiatry residency programs in the United States to gain knowledge on how to continue the innovation of psychiatry education.The constructed survey aimed to quantify pandemic-induced psychiatry residency curriculum changes and assess psychiatry faculty perceptions towards the evolving educational landscape. Composed of 15 total items , the survey was developed into three sections: demographics, program changes, and director perceptions. The full survey form can be seen below .Table 1.First, respondents provided information regarding program institution type  and resident learning modalities (pre- and post-pandemic). Next, respondents were surveyed on curricular changes in didactic content, clinical rotation sites, resident wellness, and telemedicine exposure secondary to the pandemic. Finally, respondents were assessed on their attitudes towards respective pandemic changes and considerations for future curriculum adaptations.The study protocol was approved by the appropriate Institutional Review Board. A total of 277 psychiatry residency program director emails were manually located through the Electronic Residency Application Service (ERAS) and public contact information on respective program websites. Although ERAS primarily serves as a residency application platform, this study used it as a source to obtain the most updated contact information for psychiatry residency program directors. Any program director emails that were not listed on the ERAS database were further pursued through public internet Web site of each psychiatry residency program. Surveys were sent to the listed contact emails for psychiatry residency programs between November 2021 and December 2021. Of these, 13 emails (4.7%) were returned as undeliverable without any alternative method of contact. Of the remaining 264 eligible emails, 63 survey responses (23.9%) were obtained anonymously through Qualtrics software. Although the direct emails had clear instructions for program directors use, there was no official verification process to ensure that the actual program director of the psychiatric residency program the sole respondent to the survey.All quantitative data for statistical analysis was extracted from the Survey directly. Qualitative data was obtained as a format of freestyle elaboration.P < .05 through SPSS 26 software.Due to limited sample sizes, Fisher’s Exact test was used to examine differences between program types  across several variables . Chi-squared analysis was also performed similarly when an appropriate sample size was present. All analyses were conducted with a statistical significance level at Of 264 eligible programs, 63 program directors responded. Among these responses, the self-identified distribution showed 31 academic (49.2%), 19 community (30.2%), and 13 hybrid (20.6%) programs . Due to P = .2539] and post-pandemic [P = .1343] content delivery modalities showed no significant differences between academic, community, and hybrid psychiatry residencies. Furthermore, the majority of program directors reported making changes to their resident didactic content (60%) in response to the pandemic. However, there was no significant difference in didactic content changes across academic, community, and hybrid programs .For content delivery, the majority of respondents reported primary use of in-person modalities before the pandemic (97%). Few program directors reported using virtual (1.5%) or mixed (1.5%) modalities pre-pandemic. Post-pandemic, respondents reported primary use of virtual (52.5%) and mixed (46%) modalities to facilitate resident learning. One program directors reported continued use of an in-person format after the onset of the pandemic. When compared across program types, pre-pandemic [P = .7129] and protected time [P = .6984] showed no significant differences between academic, community, and hybrid psychiatry residencies.In terms of resident wellness days, most respondents reported no changes in the number of wellness days (67%) and allotted protected time (97%). Moreover, some programs respondents indicated an increase in the number of resident wellness days (30%). In contrast, a few program directors reported decreased wellness days (3%) and increased protected time (3%). When compared across program types, resident wellness days , current telemedicine use [P = .4173], and post-pandemic telemedicine exposure [P = .6074] between academic, community, and hybrid programs. This is the first study conducted to compare the curricular changes between academic, community and hybrid General Psychiatry residency programs post COVID-19 pandemic. Although no significant curriculum changes were found among above programs, it indicates a general trend of residency curriculum among different types of residency programs in future years.When surveyed about total rotation and elective sites, respondents reported an unchanged (62%) or decreased (38%) number secondary to the pandemic. No program directors reported increases in rotation sites. Additionally, most program directors reported the use of current telemedicine exposure (83%) with increased exposure post-pandemic (84%). Noteworthy, very few program directors reported lack of telemedicine use (17%) and unchanged exposure secondary to the pandemic (16%). No program directors reported decreases in telemedicine exposure after the pandemic. When compared across program types, there was no significant difference regarding changes in rotation sites [P = .3371] and virtual learning integration [P = .1038] between academic, community, and hybrid programs.When asked about the perspectives on individual program changes, most respondents reported positive (45.8%) or neutral (35.6%) attitudes towards respective changes. Surprisingly, few program directors reported negative perceptions (18.6%) on their individual respective changes. Furthermore, respondents reported positive attitudes (59.7%) towards the integration of virtual learning modalities into psychiatry residency training, in contrast only 24.2% of program directors reported neutral attitude and 16.1% of program directors reported negative attitudes towards the use of virtual learning platforms. When compared across program types, there was no significant difference regarding perspectives on individual program changes [n = 9), willingness to return to in-person activities when deemed safe (n = 5), increased telemedicine use and didactics (n = 2), and increased didactics on disaster management and contingency planning (n = 1).Respondents were also given the opportunity to elaborate on the considerations for future program changes through free response. We found the most common qualitative themes included usage of a hybrid learning model  for didactic-based learning . Investments in financial literacy, resources for implicit bias education, and the mitigation of potential stressors (child or elder care) can improve the learning experience of trainees. For psychiatry education, the narrative review from Ho et al. also supported the integration of technology-enhanced learning modalities . However, our results suggest that the pandemic decreased clinical rotation sites for residents. Some programs have also shifted more focus from traditional didactic topics to more recent pandemic-related topics. These factors may result in small but exigent deficits in clinical and didactic exposure.The review by Kaul et al. advocates for the digitalization of didactic sessions, grand rounds, and journal clubs in resident education. Field experts recorded 30 lectures focused on the assessment, diagnosis, and treatment of older adults aimed to address pandemic-induced learning gaps during in-person clinical experiences. The National Neuroscience Curriculum Initiative (NNCI) is another supplement to neuroscience education in psychiatry trainees. While it intended to focus on improving the dissemination of educational neuroscience resources, the curriculum has been implemented in over 200 psychiatry residency programs — even through the pandemic. When properly refined, online curricula may be an innovative way to supplement psychiatry learning where clinical experience is negatively affected.Online curricula may be quality supplements to traditional psychiatry residency training. The American Association for Geriatric Psychiatry (AAGP) created a web-based curriculum designed as introductory exposure to geriatric psychiatry for residents and trainees. The creation of consortia may provide cross-institutional opportunities for education and networking. The Otolaryngology Program Directors Organization (OPDO) created 3 consortia that offered live and recorded lectures over web-based teleconferencing by faculty from numerous national institutions. Lecture times were staggered for learning convenience in various time zones and supplemental materials were provided for structure. These tactics aimed to combat the reduced clinical experience, educational resources, and in-person contact secondary to the pandemic. However, Xie et al. noted significant obstacles of uncompensated time, expenses, management, and maintenance with the founding of these consortia. Consortia may improve psychiatry education, but it is imperative to consider sustainability. Local consortia composed of neighboring or regional institutions may be an intriguing concept that can provide enhanced resident learning, cross-institutional networking, and cheaper sustainability.Similar to the residency match, the pandemic may have restricted opportunities for psychiatry residents to travel and collaborate with colleagues from other institutions. Hence, residency programs have increasingly prioritized resident wellness to mitigate further deterioration of mental health. Generally, residents are allocated protected time and wellness days to address both personal and professional needs. However, our results showed that most programs did not change the number of wellness days and protected time in response to the pandemic. Only a subset of nearly one-third of program directions indicated an increase in the total number of wellness days. One review identified that resident well-being was associated with personal time availability (PTA), the reported ability of residents to find time to exercise, socialize, and tend to errands. Residents with higher PTA scores reported more positive experiences/emotions, increased career choice satisfaction, and less perceived stress. Given the impact of the pandemic on resident wellness, it is imperative to use evidence-based strategies to adapt to residents’ well-being needs.Mental health repercussions from the pandemic are on the rise. In primary and surgical residents, the pandemic has affected social interactions, spiritual well-being, and increased levels of mental and physical exhaustion. To develop wellness culture, the model prioritized open educational discussion of burnout symptoms and even classified burnout as an illness. Residents with any symptoms of illness, including burnout, were mandated to stay at home. In this model, residents reported increased feelings of support if they were physically or mentally unwell. Patient interactions were minimized to promote resident safety and practice efficiency. COVID-19 suspected or positive patients were only examined by either a senior resident or attending. While exposure to COVID-19 patients has been associated with increased physician trainee stress and burnout, it is also important to find an exposure balance that both promotes resident wellness and delivers high-quality clinical training.During the pandemic, one surgical residency program aimed to optimize residency wellness by using a model that emphasized a culture of wellness, personal resilience, and efficiency of practice. These challenges may contribute to decreased motivation, concentration, and learning effectiveness in virtual trainees. Additionally, these factors may be difficult to assess through remote learning and remain without intervention. For prevention, Ho et al. recommended the incorporation of adequate breaks, wellness check-ins, peer support, and live video feeds. These tactics may encourage trainee engagement, facilitate feedback communication, and improve the education quality trainees receive.While the pandemic heavily influenced the use of virtual learning, psychiatry residency program directors from our study heavily favored the integration of remote education into existing curricula, as concluded from the qualitative data from the survey. Our results show the presence of these shared perspectives on virtual learning across all program types. Despite our reliance on its adoption, virtual learning has several important weaknesses when compared to traditional education methods. One systematic review noted the issues of technical challenges, confidentiality issues, reduced student engagement, and loss of assessments.21 While in-person training may be more traditional, future considerations involving technological innovation and didactics on disaster management may allow psychiatry trainees to become better equipped in the case of another similar crisis.Due to the limitations of virtual learning, several psychiatry program directors reported considering the use of hybrid learning models — a blend of virtual and in-person learning. Where applicable, this combination allows for the achievement of a more optimal balance between the benefits and drawbacks of remote learning. Some non-psychiatry residency and fellowship programs have noted success with the model and report it as a method to adjust and maximize trainee learning.23 These students expressed disruptions due to cancelled in-person clinical instruction, willingness to accept the risks of COVID-19 infection to return to clinical rotations, and even considered repeating the final year of training to supplement decreased clinical experience.23 Specifically for psychiatry training, medical students in the United Kingdom experienced reduced educational demands such as experience documentations, attendance monitoring, workplace-based assessments, and case presentations, but noted no lowering or compromised of clinical standards. Final year medical students in Sri Lanka also reported great impacts on psychiatric training and exam performance. The pandemic had impacts on clinical aspects including patient availability, mental status assessments, rapport development, diagnostic skills, and clinical exam components. Although there is limited data specifically involving psychiatry residency training in the United States, psychiatry residency shares some similar clinical aspects of medical training. The pandemic impact on clinical involvement in medical student training suggests the possibility of interruptions in clinical residency training and risks of clinical competency despite further supplementation with virtual and hybrid learning modalities.The COVID-19 pandemic has induced the integration and increasing reliance on virtual platforms in residency education. However, concerns remain regarding the consequences of the pandemic interference of psychiatric training potential effects on clinical competency. There is limited current data that demonstrates the long-term outcomes of virtual education in psychiatry residency training. Based on the current available data, the literature shows dramatic interruptions in clinical medical student education in various countries including the United States.Our results should be interpreted in the context of some limitations. While appropriate for statistical power, the response rate we received was lower than expected. Due to anonymous response designed in the study, we cannot inquire those non-responded programs to respond. The small response rate (23.9%) limits the generalizability of our results to psychiatry programs throughout the United States. Additionally, the majority of responses to the open-ended question in the survey remained unanswered, opening the possibility of non-response bias. Initially, free response questions were limited to potentially encourage increased survey completion. However, knowledge of the reasoning behind responses regarding perspectives would have also been useful. Thus, respondents should have also been given the opportunity to justify the reasoning for their perspective selections to further inform future academic progress. Furthermore, the study evaluated qualitative changes and perspectives, but did not include measures on the effectiveness of these changes. This may suggest a limited scope with need for further investigation.The COVID-19 pandemic forced psychiatry residency programs to innovate education delivery. Our findings suggest that these innovations were ubiquitous, regardless of program type. Although difficult to navigate, the pandemic has highlighted several adaptations that may have the considerate potential to further innovate psychiatry education. Hybrid learning has remained a popular post-pandemic alternative that appears to streamline learning efficiency and promote trainee wellness. As changes continue to be made, it is important to use trainee feedback to improve chances for long-term sustainability. Regardless, psychiatry education must continue to seize the momentum of educational improvement created by the pandemic. Further investigation is needed on the effectiveness and long-term outcomes of these learning adaptations to ensure efficacy.


Depression in conjunction with cardiovascular disease (CVD) is associated with worsening in CVD, higher mortality, and poorer quality of life. Despite the poor outcomes there is a treatment gap of depression in CVD patients. Recently we found that an Internet-based cognitive behavioral therapy (iCBT) tailored for CVD patients led to reduced symptoms of depression. However, we still have little knowledge about CVD patients’ experiences of working with iCBT. The aim of this study was therefore to explore CVD patients’ experiences of engaging in a tailored iCBT program.A qualitative interview study using inductive thematic analysis. Data was obtained from 20 patients with CVD and depressive symptoms who had participated in a randomized controlled trial (RCT) evaluating the impact of a nine-week iCBT program on depression.Three main themes emerged: (1) Taking control of the disease, (2) Not just a walk in the park, and (3) Feeling a personal engagement with the iCBT program. The first theme included comments that the tailored program gave the patients a feeling of being active in the treatment process and helped them achieve changes in thoughts and behaviors necessary to take control of their CVD. The second theme showed that patients also experienced the program as demanding and emotionally challenging. However, it was viewed as helpful to challenge negative thinking about living with CVD and to change depressive thoughts. In the third theme patients reported that the structure inherent in the program, in the form of organizing their own health and the scheduled feedback from the therapist created a feeling of being seen as an individual. The feeling of being acknowledged as a person also made it easier to continuously work with the changes necessary to improve their health.Engaging in an iCBT program tailored for patients with CVD and depression was by the patients perceived as helpful in the treatment of depression. They experienced positive changes in emotions, thoughts, and behaviors which a result of learning to take control of their CVD, being confirmed and getting support. The patients considered working with the iCBT program as demanding and emotionally challenging, but necessary to achieve changes in emotions, thoughts, and behaviors.The online version contains supplementary material available at 10.1186/s12888-022-03939-7. Studies suggest that among patients with cardiovascular disease 20–40% have depressive symptoms, which is significantly higher than the rate of depression in the general population , 2. DeprThere are, however, several challenges in the treatment of depression in cardiovascular disease. One challenge is that the effects of pharmacological treatment are generally small  and can .Furthermore, studies report that patients with chronic somatic diseases who work in generic iCBT programs have difficulties to identify themselves when defining problems and in goal settings , 16. In Our research team recently published a randomized controlled (RCT) trial in which an iCBT program was tested in patients with CVD and depression. The program was tailored for this patient group and guided by nurses with brief introductory training in CBT and in collaboration with a clinical psychologist. One of the nurses was a specialist psychological nurse (JL) . The resA central aspect of CBT is changing the patterns of emotions, thoughts, and behaviors in order to decrease the symptoms of depression . It is tWe used a qualitative semi-structured interview study design when interviewing patients who had participated in an iCBT intervention. One-to-one interviews with open-ended questions were conducted and subsequently analyzed using inductive thematic analysis .The nine-week iCBT program tailored for CVD patients  consisted of 7 modules with weekly work and assignments according to the treatment steps in CBT including goal setting, disease-oriented psychoeducation, problem solving and behavioral activation. The present iCBT program was developed from a previous iCBT program targeting depression in patients with heart failure , 24. Eacn = 72) [The patients in the present study were recruited from a RCT evaluating the effect of iCBT on CVD patients with depression, and who were randomized to the iCBT arm (n = 72) . In thatInterviews were conducted between 1 and 6 months after iCBT completion. Patients were interviewed by telephone between November 2017 and April 2018 and the interviews had a mean duration of 28 min (range 15–49 min). All interviews were conducted by the first author (MW), a PhD student and registered nurse specializing in primary healthcare and experienced in conducting health assessments by telephone. At the time of the interviews MW was working part-time in a general practice health clinic. To ensure that the topics of interest would be addressed, an open-ended interview guide was created . The intTelephone interviews were used as they fitted the design of the present study, enabling a broad sampling among patients spread over the southeast part of Sweden. Telephone interviews have also shown to reduce emotional distress since the patients are not influenced by the presence of the interviewer . All intThe interviews were uploaded and transcribed verbatim. The first author (MW) checked the transcribed interviews for accuracy prior to the analysis. Thematic analysis with an inductive and latent approach was used to explore the experience of how engagement in an iCBT program could influence possible changes in emotions, thoughts, and behaviors. The inductive approach allows to generate themes from patterns in the data and not to prove or disapprove hypotheses or to test a previous finding, thus we did not use themes from headlines in the iCBT program. The latent approach involves interpretative work and admits exploration of underlying ideas and assumptions, and conceptualizations in order to find themes with a broader meaning in the data ..The analyses followed the six phases of thematic analysis described by Braun and Clark . In the To ensure the credibility, triangulation through multiple channels was conducted by having more than one researcher independently analyzing the same data set and thereby considering selective perceptions and interpretive bias . The trin = 11 (55%). The patients had completed five to seven modules out of seven possible modules included in the 9-week iCBT program. The patients lived in both rural and urban areas, and most of the patients (n = 17) were in a relationship  Taking control of the disease, (b) Not just a walk in the park, and (c) Feeling a personal engagement with the therapy program. Each of these three main themes has three corresponding sub-themes. Themes and sub-themes are presented in, Fig. This first main theme describes how the patients experience taking control of their health by working in the program. The patients described how the program gave them guidance through the different modules, which made them feel they were part of the treatment process. They also felt that they could make their own decisions in their treatment based on the new knowledge they had obtained from the program about their disease. The patients experienced a sense of control over the treatment due to the perceived freedom of being able to go back and forth in the program and work at their own pace.“Realizing that it’s up to me how I deal with life. Perhaps you can get that out of such a program. Not believing that the healthcare service should help me with everything. I’m not a child who needs their mom and dad to help, I’m an adult and I have to take charge of my life, and it’s up to me what I make of it” participant no.14 Female.The patients noted that being involved in the treatment and making their own decisions in the program created a feeling of being in control. The feeling of being in control made it easier to take difficult steps forward in order to improve their health by changing their negative thinking and behavior.The patients reported that learning about their disease helped them gain new insights. Working with the program was helpful to gain a new understanding of how to live with CVD and depression. This also helped them perceive their situation as less stressful and made it easier for them to make decisions in the program concerning the necessary steps to change their negative emotions, thoughts, and behaviors.“Because there’s a fear linked to this issue of the heart not behaving. But the fear disappeared, and I learnt that the heart can cope with much more than you think. And I also learnt that this little leakage I have, that’s quite normal in old age” participant no.11 Female.“One advantage… In other words, in a physical meeting there is… It’s perhaps that you have a physical meeting today, and then you meet again in two weeks. The advantage here was that you worked with the program during this time” participant no.10 Male.The patients experienced a feeling of freedom regarding the program. Having the possibility to influence the date and time to work with the program was perceived as flexible, but above all it contributed to a feeling that the program was constantly present during the whole treatment period and did not take breaks between the individual treatment sessions. The experienced freedom motivated the patients to continuously work with the treatment to achieve changes and take control of their own health.In this second main theme, the patients describe the work in the program as demanding and emotionally challenging but necessary to achieve a change in their negative emotions, thoughts, and behaviors. Not all the patients participating in the program were satisfied with the program and some of the content was experienced as demanding and difficult.“You have to read through, you have to reflect, you have to work for yourself. But you know that you’ll get an answer. Yes, it’s more… cognitive behavioral therapy is more work, but it also produces results” participant no.13 Male.The program was in some parts perceived as demanding and the patients had to work harder than they expected. For some patients, the amount of work and the active participation needed made it hard to complete the treatment despite experiencing positive changes in their negative thinking.“You try to repress things sometimes, unfortunately. Yes, it’s both good and bad, but eventually it re-emerges then, but no, I think what’s worth most is that you have to, err… think about it and above all look forward. That’s what has helped most, that there’s a change for the better in everything, both physically and mentally” participant no.3 Male.Working on the program evoked emotions and led to feelings of discomfort and stress. These emotions were a result of the increased insights about their health and the changes needed to achieve more positive thinking and behavior. Patients expressed these evoked emotions as a necessary part of the process towards improvement and recovery from depression.“I perceived it almost as if perhaps it wasn’t designed for a… someone who works, like, and has a lot of different types of activities. It became very complex, filling in these weekly schedules based on the module” participant no.17 Male.Not all the patients were satisfied with the program. The text in the program was sometimes experienced as difficult to read and to recognize, and not to be tailored to their own situation This made them find the program less credible and thereby less helpful.In the third main theme, patients emphasized the importance of recognizing themselves in the program and relating the content of the program to their CVD and depression. The patients perceived the work structure in the program, including support and feedback from a trusted therapist, as an important factor in achieving changes in emotions, thoughts and behaviors.“Sometimes you talk about depression and then if you don’t recognize it, so perhaps you don’t read that much, but if you recognize yourself there then you look up why you’re depressed and what you can do” participant no.1 Male.The patients felt that the tailored content of the program was personally directed towards patients with CVD and depression, creating a sense of being seen and listened to. Being able to relate the content of the program to their CVD and depression gave a feeling of safety and a realization of not being alone in their disease. This helped them to express their emotions and thoughts more openly about their disease, which in turn contributed to behavior changes, and thus improvements in their depression.“And it can be good to have the chance to express your feelings and so on, even if it’s online in the form of… You still get an answer from somewhere. I think that can be a positive thing. Just being able to express yourself can be enough” Participant no.4 Male.“Ideas about how to activate yourself, too. Like, planning your day better. And that one thing leads to the other, as it were. That you, I mean, just by activating yourself it’s easier to get out of a vicious circle, in a manner of speaking” participant no.12 Male.Patients reported that the structure of the program helped them set individual goals, organize, and perform activities, which was experienced as helpful in order to achieve changes in emotions, thoughts, and behaviors. For example, the psychoeducation regarding the cardiovascular disease, and creating an individual activity plan, were experienced as important support in order to break the vicious circle including negative emotions, thoughts, and behaviors. New insights motivated them to increase their physical and social activities and learn to accept their cardiovascular disease.The written feedback was experienced as trustworthy and made the patients perceive the therapist as credible, which also facilitated changes in emotions, thoughts, and behaviors. For most patients, the therapist was perceived as a close personal guide throughout the program, who made it possible to complete those parts of the program that were seen as difficult. The therapist also helped them become aware of and reflect upon, and thus change their negative emotions, thoughts, and behaviors.“Otherwise, I think that the feedback has been adequate, and I’m sometimes surprised how much the person providing the feedback has understood about what I’ve actually thought, when I’ve done my exercises. So, it’s been good” participant no.17 Male.“I felt that I was treated very personally then, and it was a bit like sitting next to the person. There was very close, direct contact” participant no.11 Female.The aim of this study was to explore CVD patients’ experiences of engaging in a tailored iCBT program. We found three main themes “Taking control of the disease”, “Not just a walk in the park”, and “Feeling a personal engagement with the therapy program”.An important finding was that the program helped the patients to change their emotions, thoughts, and behaviors, by learning to take control of the disease and treatment and learning to live with cardiovascular disease. This is in line with previous research , 32 inveAnother finding was the experience that participation in iCBT requires a great deal of effort to achieve positive changes in emotions, thoughts, and behaviors. The patients perceived the work in the program as stressful with regard to the need to read text and perform homework assignments according to a time schedule. They also described the program as challenging since they had to reevaluate how they were feeling, thinking, and behaving in relation to their CVD and depression. However, patients described the work in the program also as rewarding since the challenges made them achieve positive changes. This confirms that it is important that such programs require active engagement from the patients and that their beliefs are challenged in order to change emotions, thoughts, and behaviors . However.In one of the findings the patients describe a feeling of a personal engagement with the iCBT program. The patients stated that the work structure – including the tailored content, the homework assignments, and the feedback from the nurse therapist – helped them feel that they were being seen and supported in order to achieve positive changes in emotions, thoughts, and behaviors.The tailored content of the program helped the patient to create a feeling of recognition and being acknowledged as a person, thus making it easier to accept and work toward positive changes. In previous research, patients describe a desire for more disease-specific content related to the somatic chronic disease , 16. TheAn important part of feeling a personal engagement with the therapy program seemed to be the feedback in the program, which was described as motivating to achieve positive changes in emotions, thoughts, and behaviors. The importance of a therapist giving feedback has been established in previous studies and is an important factor in achieving good treatment results from iCBT , 39, 40.The strength of this study is that it is one of the first to report on patients’ experiences of participating in an iCBT program tailored to patients with CVD, which contributes to a better understanding of patients’ preferences when developing future programs.However, our findings should be interpreted with some caution. First, we sought to investigate experiences of a specific treatment using existing theories and therefore estimated the number of participants to be enough to contribute substantially to the understandings . The chaWe invited all the patients who had participated in spring 2017. However, those who agreed to participate in the interview study had completed at least four modules of the iCBT program. This is a limitation, as those who agreed to participate may have been more positive about the program. We did not measure depression severity when inviting the participants to this qualitative study.Another limitation is that the interviews were conducted 1 to 6 months after completion of the iCBT program. The reason for this was due to time constraints. This may have influenced the content of the responses due to difficulty remembering specific details. However, when the interviews were conducted, it never occurred that patients had to struggle to remember. On the contrary, the patients seemed to have a clear picture from the time in the program. When analyzing the interviews, the content did not seem to vary depending on when these were conducted.All the interviews were conducted by telephone, which was deemed appropriate for collecting the data. The procedure may, however, have affected the depth of the content of the interviews compared with conducting face-to-face interviews. Telephone interviewing has been reported as equally valid compared to face-to-face interviewing for data collection  and has Engaging in an iCBT program tailored for patients with CVD and depression was by the patients perceived as helpful in the treatment of depression. They experienced positive changes in emotions, thoughts, and behaviors. This was facilitated by learning to take control of their CVD, being confirmed and getting support. The patients described that the therapist enabled a positive change by guiding them through the program, including both psychosocial and somatic aspects associated with CVD and depression. The patients also experienced the work in the program as demanding and emotionally challenging. However, this was necessary to achieve changes in emotions, thoughts, and behaviors. Future improvements in iCBT for CVD patients with depression could be to make the program more person-centered by allowing them to tailor the content of the program to their own preferences.Additional file 1.


Employment agencies and funding systems commonly use distal outcome measures such as employed or not employed, full-time or part-time, and continuous measures such as wage, hours worked, and type of job to document the employment status of individuals with disabilities. These measures continue to demonstrate that individuals with disabilities fall behind individuals without disabilities in all employment outcomes. While there is utility in distal outcome measures, it is difficult to determine what intervention or program variables were responsible for a specific outcome. Moreover, outcome measures do not provide sufficient information about the quality of employment supports and services an individual with disabilities receives. One way to improve accountability in employment support programs is to link outcomes to specific processes for obtaining and maintaining employment. The purpose of this manuscript is to describe how employment programs can link short-term  and long term  outcomes measures to specific processes for employment. A customized employment framework is used to illustrate how systematically linking outcomes and processes improves accountability in programs that support job seekers with most significant disabilities. The health of economies is often measured by the number of individuals who are attached to the workforce and who are actively employed. An examination of current economic outcome measures in the United States suggest that the U.S. is currently in a changing and robust labor market with unemployment hovering around 3.5% at the end of 2021 , they experience improved outcomes in economic, psychological, and physical health factors . For exaBoth proximal and distal outcome measures are used to determine the impact of a program or intervention. Both measures attempt to answer, “what happened” after an intervention or program is implemented. Proximal outcome measures include data that is collected during program implementation and provide information about the most immediate and observable outcome of a program or intervention. Distal outcome measures include data that is collected after program or intervention implemented and are designed to determine the outcomes the program or intervention was intended to achieve . Policy The American Community Survey (ACS) is an example of a distal outcome measure that is used to inform policy. The ACS is an annual supplement to the U.S. Census and provides large-scale, aggregate distal outcome measures. The ACS provides information about demographics and social and economic statistics that serve as a base for the administration and evaluation of U.S. government programs . For disThe Rehabilitation Services Administration (RSA) 911 data is an example of an agency specific distal outcome measure. The RSA-911 data is mandated by the Rehabilitation Act as amended by the Workforce Innovation and Opportunity Act  and is uProcess measures attempt to answer and document “how something happened” and provide a robust assessment of how well or the fidelity to which practitioners implement a program or intervention . The earFor most working-age adults, the pathway to meaningful employment occurs over the lifespan , is baseCustomized Employment (CE) represents a departure from traditional employment support methods and is designed to support individuals with MSD to find and maintain competitive integrated employment. CE is a sequential, cumulative process consisting of discovery, customized job development, and ongoing training and support. CE begins with discovery, which is psychosocial rehabilitation process used to determine an individual's strengths, interests, skills, and support needs to obtain and maintain customized employment . The dis(JDFS)   are desi(JDFS)  used a t(JDFS)  conducteThe JDFS was designed to measure how to engage businesses that align with the job seeker's strengths and vocational interests. The JDFS consists of two sections: job development systems fidelity and job development services fidelity. Systems fidelity tenets cover the foundation for customized job development referrals; the incorporation of information gathered during discovery in the job development plans; and elements related to customized job development personnel, provider responsibilities, and transportation. Services fidelity tenets include building job-development plans based on discovery findings; using an informational interview approach to contact businesses; analyzing workplace cultures to ensure ecological fit; negotiating mutually beneficial and customized employment opportunities, including job creation through resource ownership or self-employment as appropriate; and maximizing opportunities for long-term career development and growth. Riesen et al.  used theA conceptualization for integrating and linking processes and corresponding proximal and distal outcomes for CE discovery and job development are outlined in After the discovery process is completed, a job seeker begins the process of customized job development. There are two components involved in the customized job development process: systems and services job development. The perennially low employment rates for individuals with MSD underscore the importance of expanding, connecting, and balancing measures used to determine the quality and success of employment programs for individuals with MSD. The most common framework for determining success of employment programs relies on distal outcome measures. Unfortunately, relying only on distal employment outcomes as an indicator of program success is problematic because distal outcomes are often influenced by other non-intervention/program factors , 35. WitCR and AS contributed to the manuscript by conceptualizing and writing specific sections. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This presentation describes the working relationships and steps that were taken to develop Dementia Friends for Incarcerated Settings. This included identifying population needs, approval processes, curriculum development specific to this population, learner coordination, implementation, measurement and reporting. Dementia Friends, a one-hour session to help everyone in a community understand five key messages about dementia, was used as the framework for an educational program targeting staff at correctional facilities. Dementia Friends is a standardized program that can be modified depending on the target audience or sector. Our sector-specific program was piloted in all 27 Ohio prisons to ensure that all staff are knowledgeable about ways to make a difference in the lives of those living with dementia.


As of 2020, 1 in 4 people in China is a domestic migrant. However, their lack of access to health care in destination cities has been largely neglected by the Chinese government until recently.Drawing on data from the 2010–2016 China Migrants Dynamic Survey, this study evaluated the impact of a pilot program of the Equalization of Basic Public Health Services launched in 2014 and focused on place of childbirth, an important indicator of delivery care. A difference-in-differences design was employed for statistical inference.The migrant pilot program increased the likelihood of a migrant childbirth at a migration destination by about 4 to 8 percentage points, depending on the model specification. Further analyses revealed that this positive effect was short-term and benefited relatively better-off migrant families.The migrant pilot program improved delivery care for migrant women. The Chinese government needs to expand this pilot program to more cities and improve its benefits to better serve the massive migrant population.The online version contains supplementary material available at 10.1186/s12889-022-14950-8. With rechildren . The lacchildren . In addichildren .In response to deteriorated health equity, the Chinese government has introduced the Equalization of Basic Public Health Services (EBPHS) program as part of its comprehensive health system reform that was launched in 2009 . One of In December 2013, the former National Health and Family Planning Commission of China  announced a pilot program to promote the EBPHS for all domestic migrants living in 40 selected cities  strategy to formally evaluate the performance of the program.Given that the EBPHS pilot program was announced in December 2013, and the selected cities started to roll out the pilot program in March 2014 after three months of preparations, we expect that:Hypothesis 1: The EBPHS migrant pilot program had a positive effect on maternal health care in 2014 and later.According to government bulletins and news reports, several provinces  began to expand the EBPHS migrant pilot program in the late 2015 and reached nearly universal coverage by the end of 2016.Hypothesis 2: The positive effect of the EBPHS migrant pilot program diminished over time.The primary data used in this study came from the 2010–2016 China Migrants Dynamic Survey (CMDS), a repeated cross-sectional survey conducted by the National Health Commission of China. Using a stratified, multistage probability proportional to size sampling strategy, the CMDS draws a nationally representative sample of about 130,000-200,000 domestic migrants who are Chinese citizens and aged between 15 and 59 years old annually from more than 1,500 cities and counties in 31 provinces  and Xinjiang Production and Construction Corps in mainland China . In the Childbirth was used as the unit of analysis because each migrant woman could have more than one birth during the study period. To derive the analytic sample, this study first identified a nationwide sample of 49,881 migrant women who had 50,365 births in the year of the interview or the year before . Among them, 21,966 births were delivered by 21,755 migrant women somewhere else rather than in any treatment or control city see the  section,The outcome variable was place of delivery, an indicator of reproductive health care that may be a matter of life and death for pregnant migrant women and their unborn children –27. ThisThe treatment in this study was the EBPHS migrant pilot program that was announced in November 2013 and rolled out in March 2014. Therefore, the time period prior to (and including) March 2014 was defined as pre-treatment, whereas the period after March 2014 was considered as post-treatment. The 40 cities that were chosen to pilot the EBPHS migrant program were the treatment cities, and the births given by migrant women in these cities comprised the treatment group. Fifty-one cities were picked as the control cities  or propensity score matching . The treatment group was also more likely to be urban-to-urban migrants rather than rural-to-urban or rural-to-rural migrants, compared with the control group. Third, on average, the counties in the treatment cities had considerably larger total and migrant populations, higher GDP per capita, and greater health care resources in terms of hospital beds than those in the control cities, regardless of the matching methods. These differences suggest that the treatment cities were carefully selected by the Chinese government to launch the EBPHS migrant pilot program because they were popular migration destinations and better equipped with economic and health care resources.Table Across all the models in Table After adding the interaction between time and city fixed effects in Model 2, the coefficient of the DID estimate nearly doubled, showing that the EBPHS program significantly increased the probability of a migrant childbirth at migration destination by 8.1 percentage points. The DID estimate was roughly the same after further controlling for the interaction between time and migration size in Model 3.The regression estimates remained qualitatively unchanged when control cities were selected and matched to the treatment cities based on propensity scores. Specifically, the DID estimates were statistically significant and positive across Models 1–3, although the point estimates changed slightly. For example, in the full model , the DID estimate based on propensity score matching showed that the EBPHS program significantly increased the probability of a migrant childbirth at migration destination by only 5 percentage points, smaller than the estimate based on manual matching (8 percentage points).Hypothesis1, no DID estimate was statistically significant in the pretreatment period (2010–2012) or the year when the EBPHS migrant pilot program began (2014). Second, DID estimates were positive and statistically significant, regardless of the matching methods, for 2015, one year after the EBPHS migrant pilot program was launched, again lending support to Hypothesis1. In fact, manual matching and propensity score matching yielded similar point estimates, indicating that the pilot program increased the probability of a migrant childbirth at migration destination by nearly 9 percentage points in 2015. Third, the DID estimates were no longer significant in 2016, supporting Hypothesis2 . In fact, the point estimates were negative in 2016. Together, these findings suggest that the EBPHS migrant pilot program had a short-term positive impact on improving the likelihood of migrant women giving birth at their migration destinations.The DID estimates reported in Table It is worth noting that the coefficient estimates for the year 2011 were positive though not statistically significant. This might be related to a national guideline released by the National Health Commission in October 2010. This guideline proposed several goals and principles for the EBPHS migrant pilot program announced three years late in December 2013.Last, we explored heterogeneity in the program’s effect by family income because in China, domestic migrants’ health-seeking behaviors are often constrained by the amount of economic resources at their disposal and the cost of health services at the destination city . We diviThe DID estimates reported in Table We conducted two sensitivity analyses to address concerns about omitted variable bias. At the individual level, employment status and health insurance coverage could affect migrant women’s decision on where to seek maternal and child delivery care . For exaAt the aggregate level, a migrant woman’s decision on the place of childbirth may be related to her long-term intent to permanently settle in the destination city. Such an intent often depends on how difficult it is to obtain local hukou status and how expensive the local housing market is . We borrLast, we conducted a placebo test in which we pretended that the EBPHS migrant pilot program was implemented in 2013. We then recoded 2010–2012 as the pretreatment period and 2013–2016 as the post-treatment period. We reestimated the DID regression models and expected the effect of the EBPHS program to be insignificant because in reality, the program was implemented in 2014 rather than 2013. As shown in Additional file Despite the massive rural-to-urban migration during recent decades, China’s long-lasting hukou system has restricted, if not denied, millions of migrants from receiving basic health care services at their destination cities –11. The Drawing on nationally representative, repeated cross-sectional survey data, we examined place of delivery for more than 20,000 births by domestic migrant women during 2010–2016. Our DID regression estimates suggest that during the study period, the EBPHS migrant pilot program increased the probability of a migrant childbirth occurring at the migration destination rather than the home of origin by approximately 4–8 percentage points, depending on whether additional control variables were included or how the control group were selected and matched to the treatment group. According to our regression estimates, every 1-year increase in educational attainment was associated with about a 0.9% point increase in the probability of a migrant childbirth at a migration destination (results not shown). Therefore, the EBPHS migrant pilot program is equivalent to receiving 4–8 years of education in terms of improving migrant women’s delivery care. This is quite substantial, given that the average years of education is about 10.6 among Chinese migrants .After unpacking the DID estimates averaged over 2010–2016, we found that the positive impact of the EBPHS migrant pilot program appeared to be short lived. The pilot program was officially announced in November 2013 and started to roll out in March 2014. It significantly increased the probability of a migrant childbirth at the destination by nearly 9 percentage points in 2015, but the effect lost its statistical significance in 2016. In fact, the point estimates were negative, showing that the probability of a migrant childbirth at the destination was reduced by 5.4–5.7 percentage points in 2016 compared to that in 2013, the year prior to the implementation of the EBPHS migrant pilot program. The decline in the impact of the EBPHS migrant pilot program might have resulted from the expansion of the program to other cities in 2016 . The potWhen we explored heterogeneous effects, we found that the EBPHS migrant pilot program mainly affected the middle- and upper-middle-income groups. It is possible that the benefit coverage of the pilot program was too limited to offset the cost of delivery care,  especially for the poorest migrants. On the other hand, the richest migrants might have already accumulated sufficient economic resources to afford delivery care at the destination city, especially at private hospitals where hukou-based discrimination is less a concern. This finding suggests that the EBPHS program should expand its coverage and increasingly focus on the most vulnerable migrants.These findings should be interpreted in light of several study limitations. First, we did not have access to CMDS data collected in recent years. Therefore, we were unable to evaluate the long-term impact of the EBPHS program. Second, we could not directly assess the receipt of the EBPHS program benefit at the individual level due to the lack of measures in the CMDS data. Therefore, our statistical inference relied on the assumption that all migrants living in the treatment cities had enrolled in the program, which may not be the case. Third, we only focused on the place of delivery as the outcome variable, which did not fully capture the quality of delivery care. Future research is needed to examine other indicators of maternal, prenatal, and postnatal care to provide a more comprehensive program evaluation.Despite these limitations, our findings suggest that the Chinese government is capable of implementing a public health policy that is effective in providing basic delivery care to domestic migrant women. This is a significant step toward serving the long underserved migrant population who have been treated as second-class citizens by the Chinese government . The preOne limitation of the EBPHS program lies in its targeted population restricted to domestic migrants who are Chinese citizens. Therefore, the growing population of immigrants from other countries to China are not covered by the EBPHS program. Many foreign immigrants, especially those from African countries, have faced various barriers, stigma and even discrimination in seeking healthcare services in Chinese megacities such as Beijing and Guangzhou , 13, 39.The migrant pilot program represents the first public health policy initiative by the Chinese government to address the massive migrant population’s inadequate access to basic health care services. It has made a positive impact on the delivery care for migrant women and needs to be expanded in its geographic and benefit coverages.Additional file 1: Table S1. List of treatment cities and control cities. Table S2. Tests of the differences-in-difference parallel trends. Table S3. Differences-in-difference regression estimates from robust check. Table S4. Differences-in-difference regression estimates from placebo tests. Figure S1. Geographic distribution of number of births in destination cities per year from the 2010–2016 China Migrants Dynamic Survey. Figure S2. Trends of average birth rates at migration destination.


An emergent intervention to address the opioid epidemic is the use of multidisciplinary outreach teams which connect an individual in the community to healthcare resources after the experience of an opioid overdose. While these interventions are receiving federal funding, less is known empirically to inform future interventions. Understanding the process and outcomes of these interventions is advisable due to the novel partnerships of public health and law enforcement agencies who sometimes hold divergent goals. The objective of the present review was to describe program structure and evaluated outcomes of community-based post-overdose interventions.A search of PubMed, PsycInfo, and Web of Science yielded 5 peer-reviewed articles that detail the implementation and outcomes of interventions delivered in the United States published from 2001 to July 2021. Most interventions used a multidisciplinary outreach team and referenced first responder data to contact individuals who recently experienced an overdose at their residence. Services offered often included referral to substance use treatment, recovery coaches, and social services. Method of outreach, evaluation measures, and outcomes varied. From the available literature, facilitators of program engagement included communication, information sharing, and leadership buy-in among multidisciplinary partners.Future studies could benefit from exploration of service provision in rural areas, for family affected by overdose, and for minoritized populations. Community-based post-overdose interventions utilizing a law enforcement partnership are emergent with promising yet limited examples in empirical literature.The online version contains supplementary material available at 10.1186/s40352-022-00201-w. In the United States, rates of non-fatal and fatal opioid overdoses have increased markedly since 2020 . MOUD is administered at hospitals, inpatient and outpatient programs, and other certified institutions in the United States  has prioritized funding for the planning, implementation, and delivery of post-overdose interventions to reduce overdose mortality in the US  and hospital settings (n = 11). Champagne-Langabeer et al. . In contrast to other review typologies, the scoping review methodology provides an overview of information on a limited, emergent concept. Scoping reviews do not seek to draw conclusions on efficacy or effectiveness but instead seek to clarify concepts  and identify gaps in research  Contacted individual(s) who had experienced or witnessed an opioid overdoseAttempted to connect or successfully connected individual(s) with health resources in the communityWe searched for articles published between January 2001 and July 2021 to capture the most recent two decades of addiction research. Included articles detailed the following intervention activities:Articles were included when process or outcome data was reported. Additionally, we used an additional key term to search for law enforcement to explicitly search for law enforcement-partnered interventions. To identify additional literature, we examined the reference lists of articles meeting the inclusion criteria, first by title and then by abstract.Interventions based primarily in a hospital , police station, or fire station were excluded. These programs were drastically different in design and needs than interventions taking place in the community. Articles were excluded if the intervention’s primary purpose was to deliver pre-overdose prevention services. Articles were excluded if not published in the English language or published on interventions outside of the United States. Scoping and systematic reviews were excluded. In particular, Formica and colleagues , encompassing urban areas   and public health personnel. Key partners typically represented police departments, fire departments, hospitals, and behavioral health organizations. Local universities often served as program evaluators.Every program’s target population was a person who had experienced an overdose. All programs received data from police or EMS to identify people who had overdosed. All programs had relationships with police or EMS to either directly call the program staff or directly enter 9–11 calls and supplementary reports into a shared database. While all programs defined overdose reports as incidents where naloxone was administered  induction and subsequent referral to long-term outpatient MOUD services.After initial participant contact, most programs utilized long-term peer recovery support, although the capacity for peer support varied by program. The Tempe First-Responder ORP behavioral health navigator maintained contact with participants for the 45-day period. HEROES research staff maintained daily participant check-ins for the 90-day study period. The RIMO intervention group received continuous check-ins with a linkage manager to discuss treatment barriers and progress over the study’s 30-day period.Many programs offered naloxone for those who experienced an overdose. All programs connected participants to wraparound services, such as employment, transportation, food, housing, and health insurance. One HEROES article mentioned connecting participants to mental health treatment services  intervention group participants after the 30-day intervention period. In the remaining studies, qualitative information was derived from key partner focus groups or observations. RIMO was the only study to randomize participants into a treatment or control group.Key partners provided insights into post-overdose intervention implementation. Of these insights, barriers included: stigma within organizations and within the community, lack of leadership buy-in or funding, issues with data sharing among organizations, and structural barriers to participant treatment engagement. Additionally, the HEROES program mentioned the slow growth of a new program coupled with potential participant mistrust of law enforcement involvement as a barrier to intervention utilization  and training to save lives   , especially MOUD (81% vs. 18%), and stay engaged in MOUD after the 30-day intervention period (44% vs. 6%), as compared to the control group.Law enforcement-partnered community-based post-overdose interventions are being implemented around the country with limited knowledge of effectiveness. This scoping review addressed current knowledge on these programs through review of empirical literature. Data on community-based post-overdose intervention context, method of outreach, and evaluation process can provide guidance for implementation of future interventions.Multidisciplinary teams are necessary to address the opioid epidemic , the MOUD retention rates were significantly higher  were excluded from the present search to capture only evidence-based program outcomes. The present review does not capture the potential diversity and variation of gray literature-documented post-overdose interventions. Programs included in this review represent few evaluated models and, therefore, may have limited generalizability to broader implementation and effectiveness of post-overdose interventions operating across the country. Previous post-overdose intervention reviews found programs lacked robust outcome data to suggest program effectiveness because programs were emergency responses to combat the opioid epidemic (Bagley et al., This scoping review found that the current knowledge base on community-based post-overdose interventions is emergent and requires further research. Post-overdose teams are multidisciplinary, comprised of law enforcement, public health, healthcare, and others, and aim to refer individuals who experience an opioid overdose to evidence-based treatment and wrap-around services. To work towards this goal, future programs can take guidance from facilitators documented in the peer-reviewed literature, including key partner collaboration and understanding across sectors, and the ability for key partners to communicate with participants to motivate and support recovery engagement. Challenges to program success were also documented in the literature including inability to contact hard-to-reach individuals and structural barriers to treatment engagement. In this scoping review, some gaps in service provision were identified including lack of rural programs, mental health support, and support for family affected by overdose. Future interventions could benefit from utilization of public health workers to reduce barriers for minoritized populations who may be mistrustful of law enforcement. To facilitate the growth of these innovative programs, more rigorous mixed-method evaluation of community-based post-overdose interventions are needed to establish a robust program model for future implementers.Additional file 1. Appendix A: PRISMA checklist for scoping reviews. Appendix B: Search terms.


The National Maternal and Child Health Workforce Development Center provides training, coaching, and consultation to Title V programs. The flagship experience is the Cohort program, a 6-8-month leadership development program where Title V programs convene a multisector team to address a pre-selected state/jurisdictional challenge related to health systems transformation. The overall objective of this paper is to demonstrate the impact of skills developed via the Cohort program on state/jurisdictional capacities to address complex challenges.Qualitative, post-Cohort evaluation data were analyzed using inductive and deductive coding and the “Sort and Sift, Think and Shift” method. Themes and supporting text were summarized using episode profiles for each team and subsequently organized using the EvaluLEAD methodology for identifying and documenting impact.Teams brought an array of challenges related to health systems transformation and 94% of teams reported achieving progress on their challenge six-months after the Cohort program. Teams described how the Cohort program improved workforce skills in strategic thinking, systems thinking, adaptive leadership, and communication. Teams also reported the Cohort program contributed to stronger partnerships, improved sustainability of their project, produced mindset shifts, and increased confidence. The Cohort program has also led to improved population health outcomes.Through working with the Center, Title V leaders and their teams achieved episodic, developmental, and transformative results through application of Center tools and skills to complex challenges. Investment in the MCH workforce through skill development is critical for achieving transformative results and solving “wicked” public health problems. The Cohort program from the National Maternal and Child Health Workforce Center has been previously described  Workforce Development Center (the Center), has provided training, coaching, and consultation to Title V programs. The Center’s mission is to enhance the capacity of Title V leaders and staff to engage in and lead health transformation efforts . The Cohort program features intensive skill-based learning  followed by individualized coaching to provide feedback, advice, referrals for Center experts, and assistance with measuring and tracking progress. Tools included in the Cohort program are briefly described in the Appendix and in Umble et al.  and internal factors  associated with advancing change. Developmental results are “sequences of steps taken by an individual, team, organization, or community that reach toward and may actually achieve some challenging outcomes” . Deductive codes were developed based on objectives of the Cohort program. Inductive codes were developed in response to emerging themes from the interviews. The “Sort and Sift, Think and Shift” approach developed by ResearchTalk, Inc was used throughout the coding process  approximately six months, as well as three years, after the conclusion of the Cohort program. Prior to the scheduled interview, the Center evaluator sent the interview guide to team lead(s) for review and encouraged them to solicit input from team members. The primary focus of the interview was to gain insight on how the Center advanced their MCH population health goals through their team project. The interview guide included listing, multiple choice, and open-ended questions. The interviews lasted approximately one hour and were recorded and transcribed. Between 2014 and 2021, 51 six-month interviews (51/53 = 96% of teams) were completed and analyzed using ATLAS.ti. Evaluators coded the transcripts using a combination of deductive and inductive coding  strategic thinking, (2) systems thinking, (3) adaptive leadership, and (4) communication.As noted in Table Participants described enhanced systems thinking throughout their Center experience. As noted in Table “…a lot of what we needed their help with was…organizing as a team and figuring out how to manage the work. It wasn’t really…the logistics of the projects that we ultimately needed their help with. It was…how do we come together as a team? Because a really interesting group of people [are] in this work group that all come from very different backgrounds and have different personalities and different workstyles.” (Team 3)Nearly half of teams reported enhanced adaptive leadership skills, including mutual learning Schwarz,  Table 33. Partic“It’s exciting to be able to go to things and…know how to tackle this challenge. [W]e have tools for this. We’ve done this before and we’re going to follow the process. We may not have the solution, but we…have the tools we need to explore it. We’ve been able to model that change management-type behavior…with [our partners].” (Team 4)Adaptive leadership skills also enhanced work beyond teams’ projects as participants were able to apply skills in work extending from their challenge. One participant, whose team’s project focused on developing and integrating regional networks of access and quality to improve care for children and youth with special healthcare needs (CYSHCN), used change management skills when working with stakeholders to improve their pediatric palliative care program.Learning and applying Center skills to another project not only improved their workforce capacity, but also their partners’ through sharing professional development opportunities and bringing their partners along through the learning process.Participants gained techniques to strengthen communication, recognizing such techniques as methods to help overcome complex challenges. Examples of improved communication skills included finding common language to build collective understanding and developing agreed upon, politically acceptable language. One participant, working to engage key stakeholders to realign MCH program priorities and needs with the MCH 3.0 Transformation .Title V promotes and improves the health and well-being of mothers, children, including CYSHCN, and their families through the creation of partnerships with governmental and non-governmental agencies and programs. Many stakeholders—early childhood, public instruction, Medicaid, nutrition—influence a broad array of determinants of the well-being of MCH populations. Title V programs can leverage the resources of the MCH Services Block Grant Program to enhance their impact through multi-sector partnerships. Teams described how Center tools and skills advance these types of partnerships.“We had conversations with the leads with a lot of these programs when we were doing our mapping activities around who was making referrals, who was doing screening services and such…at the stakeholder meetings. When we did those system support maps, I think that also helped to illuminate everybody’s role in this—that we hoped to eventually build an effective, integrate[d] early childhood development system.” (Team 7)Enhanced systems thinking contributed to stronger partnerships. Systems mapping tools, for example, brought together organizations and agencies which had not previously worked together. One team reported how system support maps were an effective tool to engage partners, such as Medicaid, private insurers, and policy and advocacy groups, on how to build a comprehensive development framework for early intervention screening:“[In the Cohort program] we have the opportunity to further enhance our collaborative skills…[W]e have not historically had the same collaboration with our Medicaid agency as we have in that past couple of years…this opportunity gave us a reason to pull in someone from Medicaid, and because there was this formalized application process…and travel to a two-day meeting, they had to designate someone and commit to a certain level of time and energy to be part of it. I think we got a higher level of buy in because of [the Cohort program].” (Team 8)Working with the Center also increased teams’ diversity of partners. For example, the Cohort program supported teams in developing deeper connections with Medicaid by including Medicaid leaders on the team. A number of teams reported systems tools created opportunities to work more closely with state Medicaid programs. Specifically, such tools provided a common language to explore strategies to improve care coordination for families of CYSHCN. In one state, for example, Medicaid, the Title V CYSHCN program, and managed care organizations came together to define and implement shared plans of care for families across agencies. This common language reduced duplication across systems and increased efficiency in care coordination services across the state.Having the Center itself as a partner increased the credibility of teams when approaching potential collaborators. For example, one participant described a challenging relationship with Medicaid on the issue of healthcare homes. Engagement with the Center legitimized their team’s status as a partner and allowed them to push back on Medicaid’s funding approach which failed to acknowledge the different service needs between children and adults with chronic conditions. “[It] was really helpful to have an outside entity help us with that because, internally, I think we would not have been as successful because there is a lot of history between the two sections” (Team 9).Several workforce development skills increased their opportunities to sustain projects and programs. Participants reported that sharing Center tools with stakeholders led to increased buy-in, ultimately facilitating sustainability of their work. One team focused on reducing adverse asthmatic experiences for children by coordinating a system of care through community partnerships. Center skills helped engagement, resulting in collaborative grant applications to continue their efforts. The team also reflected on how achieving sustainability of the project was a key indicator of success, even when the initial team members were no longer engaged in the work. “[I]t doesn’t need your skills at the time because you’ve kind of passed the baton and that’s an important thing to do in the state for sustainability and for having the community take the lead on projects” (Team 10).“You guys didn’t give us a magic pill or a magic answer. You gave us tools to help us critically think through this project…[and] applied those tools to help us critically think and tease-out other complex problems that we had…you made us sit down and…think outside the box. [N]ot only did you give us tool[s], but…you gave us the practice space to apply them. I’ve had implementation science as part of a college program, but being able to actually have didactic [learning] with real world problem[s] just helped it come to life.” (Team 6)Teams reported transformative results, that is, “fundamental shifts in individual, organizational, or community values and perspectives that seed the emergence of fundamental shifts in behavior or performance” . Specifically, the team shared how their perspectives have shifted to focus on the installation of adequate and appropriate training, support, and communication of new initiatives, all key elements of implementation.Another transformative result was increased confidence. One team described how increased confidence led to enhanced work with their community partners. “[S]he feels more comfortable [with] sharing the data, sending the data out, and getting feedback…I think it’s the level of confidence with themselves and…they’re more comfortable working together, supporting, participating, and providing more information [about] their initiative” (Team 12). Another Title V team, who worked on improving systems of care for CYSHCN through cross-agency collaborations, described how working with the Center increased their confidence to have a more direct conversation with their director about what can reasonably be achieved with existing funding sources. “From the cohort we were able to have confidence to stand up to the director and say, ‘MCH Title V money is not enough for all the things you want’” (Team 13).Collectively, developmental and transformative results were associated with improved population health. One team’s project focused on creating a state certification program for their Community Health Workers (CHWs), so their services could be reimbursed by Medicaid and other insurers (Team 14). Three years after completing the Cohort program, the state reported 232 CHWs had been certified and were working within multiple community agencies and programs. Families working with CHWs experienced tremendous improvements in population health measures, such as a nearly 50% decline in missed school days among children with asthma and increased use of clinician-advised asthma action plans from 20 to 80% . The team used systems mapping tools to create a roadmap to help families navigate systems and access services. Using quality improvement skills and tools from the Cohort program allowed the team to continuously review what was working and not working, adjust the roadmap to be more effective in connecting families with services, and obtain and integrate community feedback. The team’s work on responding to the Zika outbreak led to a greater focus on incorporating the voices of parents, more specifically among families with CYSHCN. Change management tools, including appreciative inquiry, were used to guide implementation of a family-to-family health information center to support greater family participation in their Title V program. Additionally, greater family engagement led to an agreement between Title V and the public works transportation program to register CYSHCN with a transportation system to assist with medical appointments and household errands.Title V leaders and their teams achieved episodic, developmental, and transformative results through application of Center tools and skills to complex challenges. The Center’s three core areas and tools provided an approach and the workforce capacity to address this complexity. This intentional approach provided teams with resources—space, time, coaching, tools, skills—to identify their aspirations, recognize how and where they are experiencing difficulties, and work together towards creating change. Further, this approach increased confidence by providing a mental “safety net” for individuals and teams to try hard things they might otherwise not attempt. The Center’s approach can also be a catalyst for eliciting deeper and more systemic workforce development, as the Cohort program enhanced work beyond their chosen state/jurisdictional projects.Our analyses have limitations. Given time and energy investments by teams and the Center, there may be social desirability in interview responses; however, interviewees were asked to provide concrete justifications/explanations to validate their responses. There was also no control group to allow for measurement of impact relative to states/jurisdictions not engaged in the Cohort program; however, many respondents noted participation in the Cohort program was more effective than other technical assistance programs. Teams may have also had exposure to other leadership development/capacity building programs or other technical assistance during their follow-up period of the Cohort program. Individual exposure to other programs was not collected and could be a potential limitation of our analysis.Collecting long-term follow-up data enabled us to identify how the Center’s capacity-building approach inspired positive changes in mindset, organizational structure, confidence, as well as population health outcomes. Future evaluation of workforce development programs should include long-term follow-up to illuminate improvements in population health, as well as unanticipated impacts that may be overlooked amidst more pressing, urgent needs for Title V programs. Additionally, state MCH leadership structures vary considerably across states. Future evaluation should seek to measure and contextualize findings within each state’s MCH leadership structure to better understand the influence of state leadership structures in addressing complex challenges. Evaluation tools should also be designed to collect more data on how workforce development programs influence diversity and equity outcomes.The Center has demonstrated that investment in the MCH workforce is critical for solving “wicked” public health problems, such as health equity, access to care, and attention to social determinants of health . The Center continually reviews the training and technical assistance needs of Title V leaders and adapts its’ approach, curriculum, skills, and tools accordingly. Through this iterative process and partnership, the Center seeks to empower Title V leaders with the skills and tools necessary to address “wicked” public health problems in their state.