The mental health of healthcare workers (HCWs) has been at the forefront throughout the COVID-19 pandemic. While workplace-based support programs have been developed in hospitals globally, few systematically collected data. While critical to their success, information on these programs and the experience of mental healthcare providers (MHP) who support colleagues is limited. The objective of this study was to explore the experiences of MHP caring for HCW colleagues within a novel workplace-based mental health support program during the COVID-19 pandemic, to provide insights on facilitators, areas for improvement and barriers to program sustainability.This qualitative study used semi-structured interviews conducted by videoconference between September 2020 to October 2021. UHN CARES  Program was developed during the first wave of the COVID-19 pandemic in March 2020. It supports over 21,000 staff members within the UHN, Canada’s largest academic health research institution, in Toronto, Canada. Purposive sampling was used to select 10 of the 22 MHP in the UHN CARES Program (n = 10). Using a critical realism framework, key components required to sustain a successful workplace-based mental health support program for HCWs and balance the needs of MHP were determined.Six psychiatrists and four psychologists (n = 10) with varying roles at UHN participated in 17 interviews, including seven repeat interviews exploring changes over time within the pandemic and program. Components which facilitated the success of the program included flexibility in scheduling, confidential health record storage, comprehensive administrative support, availability of resources and adaptive quality improvement approach. Recommendations for improvement included opportunities for peer supervision, triaging of cases, and managing HCW expectations. MHP found caring for HCWs to be meaningful and they utilized existing clinical skills during sessions. Challenges included working in a virtual setting, navigating boundaries when caring for colleagues, and managing the range of service users and their needs.These findings suggest how support programs can be structured for HCWs, how to provide support, and how to sustain this support, allowing health systems to balance the needs of HCWs and MHPs in preparation for future public health emergencies.The online version contains supplementary material available at 10.1186/s12888-022-04309-z. During previous coronavirus outbreaks, such as the 2003 SARS-CoV-1 outbreak in Toronto, Canada and 2014 MERS-CoV outbreak in Jeddah, Saudi Arabia, healthcare workers (HCWs) experienced emotional distress centered around inadequate institutional emergency preparedness . As suchThe COVID-19 pandemic has now been ongoing for two years. Early hopes that the pandemic would be short-lived are gone and there is increasing recognition that it will continue, even at endemic levels . New wavCoping and Resilience for Employees and Staff) Program in Toronto, Ontario. Embedded within a longitudinal program evaluation, we aimed to investigate the challenges and benefits of caring for HCW colleagues in a workplace-based mental health support program; understand strengths and weaknesses of the program; and determine the aspects of the program which contributed to its success and sustainability.These shifting stressors impact HCWs and the MHPs staffing workplace-based programs. Understanding the experiences of MHPs is critical if these programs are to continue to support HCWs . While oWe conducted semi-structured interviews with MHPs who provided individual psychological or psychiatric care to HCW colleagues. The study followed COREQ reporting guidelines . It was UHN is Canada’s largest academic health research institution with two general hospitals, a comprehensive cancer centre, a health education institute, and a multi-site rehabilitation institute. It has over 21,000 staff members, many of whom provided care during the SARS outbreak in 2003 . Since tAs shown in Fig. Purposive sampling was used, with researchers inviting specific providers for interviews to ensure a broad range of MHP experiences. The criteria included those who saw many versus fewer referrals, saw these referrals at different points in the pandemic, employed a varied number of sessions with HCWs and had a range of years in practice Table . Seven pA semi-structured interview guide was developed to explore MHPs’ experiences with UHN CARES, their perspectives on the mental health needs of HCWs in the pandemic, how to best meet those needs, and opportunities for program improvement. Interviews were conducted by multiple study authors  by videoconference, each 40–50 min. Each interviewer maintained a record of reflexive notes relating to their interviews and interviews were discussed amongst the researchers to capture immediate impressions after each took place. Interviews were audio recorded and transcribed verbatim. Seven interviews were conducted between September 2020 and March 2021 and an interim thematic analysis was conducted. The same MHP participants were re-interviewed between July 2021 and October 2021, with an additional three MHP interviews during this time to clarify and triangulate themes such as the evolution of the CARES users throughout the various waves of the pandemic, the navigation of boundaries between MHPs and HCWs, MHP coping strategies, MHP learning and growth as well as recommendations and feedback for the UHN CARES program  to have continuity between both aspects of the evaluation. CR identifies strength in quantitative analysis for the exploration of empirical regularities, and in qualitative methods for illuminating novel, complex relationships not captured by the predetermined categories of quantitative measures , 20. TraWe conducted 17 interviews with four psychologists and six psychiatrists. Sampling was stopped when data saturation was reached.Table As shown in Table As the pandemic evolved and MHPs returned to their normal duties, the program offered sufficient flexibility for providers to titrate their caseload individually Table . This coMHPs were concerned about record keeping and reported HCW concern about confidentiality. The program’s solution of having consultation and progress notes maintained separate from the electronic health record was appreciated, even though this deviated from usual patient care practice.Reliable administrative support made MHPs feel more at ease when taking on referrals. A single administrative assistant managed referrals, triaging, record keeping, communication with HCWs and bridged transitions of care between providers. The administrative assistant also managed a MHP webpage with all relevant documents  and other resources. Online clinical and community resources used by individual providers were shared amongst MHPs with the help of the program’s administrative staff. These were especially helpful for those not routinely involved in outpatient practice.Some MHPs recommended the addition of peer supervision to the program to ensure quality care and enhance cohesion and mutual support amongst the providers. Optimal frequency was identified as approximately quarterly, given the demands on time from other clinical and administrative obligations, which increased as various pandemic-related restrictions were eased. This was implemented following wave 1 and some noted that seeing this modification to the program occur provided confidence in the process of continuous quality improvement.UHN CARES was intended to be an easy-to-access, short-term service with a rapid response time to first appointment. Some MHP suggested that modified triaging would enable them to meet with acute users more quickly, whereas less acute users could potentially be seen at a farther time point, rather than MHPs feeling obligated to schedule into their personal time.Clarifying and managing the expectations of UHN CARES users was identified as a challenge. The scope of the CARES program was limited to staff working at UHN, with distress relating to COVID-19. Challenges involved users’ presenting issue being primarily focused on long-standing mental health challenges or family members. While providers did their best to connect those users to community resources, they reported that improving communication of scope and limitations of UHN CARES would be helpful.Even though it increased workload, MHPs found the experience of providing care to HCWs to be meaningful. Some even found the experience therapeutic and purposeful as they felt that they were contributing to pandemic-related mitigation efforts.Although prior to providing care, several wondered about being equipped to help HCWs cope, MHPs reported using skills developed in previous clinical work and were pleased it transferred to the pandemic setting. Some MHPs felt that UHN CARES service felt less complex than their usual day-to-day work and that they felt prepared to manage whatever issues arose. Users accessing the program presented with a wide range of concerns, including the inability to cope with the pandemic and its various stressors  and exacerbations of pre-existing mental health illnesses. Pandemic-related restrictions  meant MHPs had to accommodate and be creative with their treatment recommendations.At the outset of the pandemic, many MHP had limited virtual care experience which proved challenging for some. Some MHPs found it difficult to gauge affect and assess risk levels as many CARES users opted for phone calls rather than video communication. This contributed to a heightened sense of worry in some MHPs of “missing something”.Given that MHPs and UHN CARES service users belonged to the same institution, there was uncertainty about setting appropriate boundaries to ensure both groups felt comfortable in the therapeutic relationship. This involved discussion of these boundaries as well as acknowledgement of any feelings of role conflict (colleague versus provider) and self-reflection. Some MHPs chose to clarify this upfront in the first interaction with the users, while others did not feel the need to address it at all perceiving the risk of breaching any boundaries as low.Concerns of HCWs at times proved distressing to MHP as well. Given their shared workplace, there was inherently a greater sense of resonance in the issues raised and at times a sense of the user's experience being "close to" their "own identity".UHN CARES users represented diverse professions . There was also a wide variety of concerns expressed, and how those concerns evolved throughout the pandemic. MHPs frequently identified that users required additional support as they transitioned out of the program which was addressed by development of a dedicated referral pathway to community resources.Some MHPs who had never provided care to HCWs expressed initial apprehension, which later resolved as they gained experience within the framework of the program and were able to consult their colleagues for further support. MHPs also reported that CARES users were overall presenting as a population relatively well at baseline, which may have required a different approach to treatment. They also identified that recommended interventions need to be catered to the shorter-term nature of the program.Given the short-term nature of the program, MHPs creatively implemented effective ways to equip CARES users with the individualized support that they needed, within the CARES program itself and through community referrals. MHPs noted that this was a patient population that often had a particularly good sense of their needs and were collaborative in developing a treatment plan, including number and pacing of sessions. Attention to transitions, including termination of support was also identified as an important strategy (Box 1 ).With extensive social restrictions, numerous impacts on the healthcare system throughout the pandemic, and the added workload from UHN CARES, MHPs identified that developing ways to cope with their own stressors was important. Some MHPs expressed that providing support to frontline workers was, in fact, something that helped them cope. Others mentioned prioritizing sleep, physical activity, and using strategies to step away from work-related communication.It is known that mental health supports for frontline workers is a crucial component of institutional emergency preparedness to maintain infrastructure, as navigating through the turmoil of disasters often results in a substantial emotional burden on frontline staff as well as those providing the mental health services , 3, 11. Generally, MHPs described their involvement in UHN CARES to be rewarding as they felt it was their contribution to bolstering the mental resolve of their frontline colleagues. Flexibility, confidentiality, administrative support, access to resources, and adaptation of the program in response to MHP feedback were highlighted as key components contributing to providers’ positive experience. Providers’ dual role as MHP within their routine contexts, as well as pandemic distress responders for colleagues within their organisation was not experienced as burdensome but rather provided an opportunity for meaning-centred coping for providers . This apAll UHN CARES program MHPs were highly skilled clinicians with a range of 2 to 32 years’ work experience, plus many years of prior clinical training. This level of experience helped inform their approach and flexibly adapt to varying HCW support needs and allowed for easy access to more experienced colleagues for consultation and debriefs when needed. Providers without this breadth of training and access to colleagues may be more anxious, have less knowledge, and in turn, feel more burdened by their additional role . AdequatIn certain instances, the strengths of UHN CARES doubled as challenges for some MHPs. As both users and providers were affiliated with the same organisation, it often increased relatability, facilitating therapeutic rapport, and giving providers an intimate understanding of the local and institutional stressors and resources. Conversely, this also resulted in moral injury for some providers as they learned about their colleagues’ distressing experiences and institutional challenges. Providers also identified some challenges in working within the limitations of the scope of the program . Establishing clear guidelines and developing a compendium of external resources allowed MHPs to offer distressed users some form of support while still minimizing their feelings of helplessness and moral injury.Although the program’s flexibility was a major strength, not all providers had the same availability and scope to provide flexible appointments and expressed feelings of guilt as a result. These concerns were balanced by the knowledge that the program was a team effort, not reliant on a sole individual. This highlights the importance of having a large provider pool, offering each one more clinical flexibility. The opportunity for MHP flexibility in the number of new assessments they accepted and when they saw service users enabled providers to self-regulate their workload.These structural components of UHN CARES may enhance the sustainability of the program while the pandemic continues to impact HCWs, increasing provider satisfaction and potentially mitigating against provider burnout, which is accelerated in conditions of low worker autonomy . Our finAs would be expected in an institutional-based mental health support program, there is the added challenge of navigating and maintaining professional boundaries as roles between provider and colleague/user can become blurred . PersonaThis project was undertaken as a quality improvement initiative with a focus on program evaluation, not as a research project. As such, it was single site and reflective of the practice and culture of HCWs and MHPs at UHN. The emphasis of the analysis was on depth of understanding, rather than broad generalizability. That said, we sampled MHPs with a wide range of experiences to obtain a broader view of the program. A longitudinal interview format enabled us to track how issues evolved and member-check our findings as we developed the thematic analysis, enhancing the reliability of our results. As this program was designed and delivered using a quality improvement framework, feedback was used to rapidly adapt the program and assess the impact of those changes. While the evolving nature of the program itself might pose challenges for determining the impact of programmatic elements, the overall approach was in line with participatory effectiveness factors demonstrated in previous studies on the development of rapid response programs , 11.As HCW mental health is critical for ensuring high quality patient care and a resilient health system, understanding how workplace support programs can be sustained long-term and the experiences of MHPs working within them is necessary. In addition to insights into the program’s success from the perspective of developing and retaining a sizeable cohort of service providers, we have highlighted strategies and structures that may be applicable in other settings. Developing a body of evidence about how to provide mental health support to HCWs and sustain such programs over time will enable health systems to be better equipped for future crises , 26.Additional file 1: Supplemental Table 1. Personal Characteristics of Research Team.


Pathway programs engage students who identify with groups historically excluded from, and therefore underrepresented in, science, technology, engineering, math, and medicine (STEMM). We explored alumni-reported impacts of eight U.S. high school-to-college pathway programs funded by the Doris Duke Charitable Foundation (DDCF).A survey was sent to 499 alumni to evaluate their perceptions of the DDCF programs, which offer mentored experiences in clinical research. A multivariate analysis was used to compare Likert-style questions related to the programs' impact on education and career plans. Open-ended responses were analyzed using inductive analysis.Two hundred sixty-nine alumni responded to the survey, the majority of whom identified as Hispanic/Latinx or Black/African American. One hundred nineteen alumni (∼75%) currently in college reported majoring in STEMM fields. Of college graduates, 30 (∼65%) obtained a degree in an STEMM field. Participants identifying as Hispanic/Latinx had a significantly higher reported level of impact of the programs on their plans to attend college. Most alumni felt that the programs impacted their chosen majors and future professions and made them more confident to pursue careers in STEMM.Surveyed alumni perceive DDCF programs to have positively impacted their interest, confidence, and skills in STEMM-related areas. Our results support the benefit of DDCF programs and substantiate their funding, integration into higher education systems, and iterative redesign to ensure positive impacts on students with diverse backgrounds.Assessment and improvement of pathway programs may support underrepresented students in their STEMM aspirations and increase the diversity of the medical and scientific workforce. As such, there is a need to closely assess the value of these programs and their potentially differential impacts on students with various backgrounds.This article explores alumni-reported impacts of eight U.S. pathway programs funded by the Doris Duke Charitable Foundation (DDCF), which enroll students from underrepresented groups who have an interest in STEMM. The question driving this work was “How do program alumni perceive their experiences in these programs and the impact of these programs on their college and career choices?”https://www.ddcf.org/funding-areas/medical-research/clinical-research-continuum-high-school-to-college/ for a list of program sites). The programs were located in cities in the west coast, Midwest, and east coast areas.This study was approved by an Institutional Review Board. Survey data were collected from alumni of eight high school-to-college pathway programs throughout the United States, all of which received funding from the DDCF  to 5  or from 1 (much less confident) to 7 (much more confident). Qualitative themes from open-answer comments were summarized to enrich the quantitative survey results.Open-ended responses were analyzed using an inductive analysis, whereby one researcher with a background in sociology and education analyzed the responses to each of the open-ended questions and categorized them into themes. These themes were reviewed with a second member of the research team with a background in psychology and education.An important component of the qualitative analysis is the reflexivity of the researchers or our positionality as researchers with backgrounds in sociology, psychology, and education, as well as our personal experiences, such as that of a first-generation college student in the case of the educational psychologist. This is important to acknowledge given that while the goal of the data collection and analysis was to represent the experiences and perspectives of the alumni, our positionality undoubtedly influenced the questions that were asked and the themes that were developed from the data.Of the 499 alumni invited to take our survey, 269 responded (response rate 54%). The mode age of survey respondents was 19 years and the mean was 20.9 (range 18–26) years. The majority of alumni self-identified as Hispanic/Latinx or Black/African American . ReporteThere was variability in household income, with the mode reported household income between $25,001 and $50,000 . One hundred sixty-nine respondents (62.8%) reported an annual household income of $100,000 or less; however, 56 respondents (20.8%) reported an average household income of over $100,000.F=5.257, p=0.002, and η2p=0.077, a medium effect.Overall, alumni felt that the programs impacted their chosen college majors and future professions and that the programs made them more confident to pursue careers in clinical research and STEMM . Accordip<0.05 and d=0.65, a medium effect  were currently enrolled in a 4-year college or university. Overall, 119 alumni (∼75%) currently in college or university reported majoring in STEMM fields. Forty-six alumni (∼20%) reported having graduated from college or university. Of these graduates, 30 (∼65%) reported graduating with a degree in an STEMM field.Alumni currently in college majoring in an STEMM field or who graduated with a major in the STEMM field described five major themes related to their motivations for pursuing STEMM: (1) enjoying STEMM in general and wanting to contribute to science; (2) wanting to work in a medical or health care field; (3) being interested in a specific research field in STEMM ; (4) developing an interest in STEMM and the process of discovery through their research experience; and (5) experiences in the programs reinforcing that STEMM fields need more diversity.Alumni were asked to describe which parts of the programs impacted them the most and to explain how they were impacted. The following 10 categories summarize areas of greatest reported impact: (1) learning about research and the research process; (2) opportunities to engage in hands-on learning; (3) mentorship from researchers, clinical practitioners, and others ; (4) communication skill development, particularly the opportunity to read, write, and present scientific findings; (5) confidence in their skills and abilities to pursue their careers of interest; (6) having a unique opportunity that offers advantages and challenges; (7) experience working in a laboratory; (8) career exploration; (9) data collection and analysis experience; and (10) clinical experiences.The majority of alumni respondents were interested in opportunities to continue to engage with the programs as alumni. Alumni indicated interest in continuing to engage in the programs in multiple ways, including (1) attending networking or reunion events; (2) participating in panels to reflect on their program experiences and provide updates about their career trajectories; and (3) participating in research projects if there were further opportunities through the programs.Alumni independently suggested a variety of other ways they could potentially engage with the program following program completion, including engaging other alumni and current participants through social media, providing administrative support or help with website design, and making monetary donations.Moreover, alumni suggested that they could mentor students in some of the skills they have learned, particularly in regard to navigating the college/university process, through in-person or virtual mentoring. One hundred forty-two alumni reported they would be “very” or “extremely” interested in mentoring new students in the programs.Resources that respondents thought could be of most help if added as part of the programs were an online forum to engage with other program alumni ; more ways to reach back out and contact the programs following program completion; and access to counseling for school or personal issues that make degree completion difficult.When asked if they would recommend the programs to other students, alumni responses were overwhelmingly positive and reflected six distinct themes supporting reasons for program recommendation: (1) research experience; (2) developing skills in areas of research, communication, and leadership; (3) exploration of different careers and majors; (4) opportunity to network and connect with those working in the medical field, which they might not otherwise get; (5) developing confidence in themselves; and (6) programmatic elements, including mentorship, relationships with other students, and the positive program environment.We surveyed alumni of eight high school-to-college clinical research pathway programs funded by the DDCF throughout the United States to assess their current career trajectories and explore their perceptions of the programs. Our results suggest positive impacts of the programs on STEMM persistence, with 75% of responding alumni who are currently in college reporting majoring in an STEMM field and 65% of those who have graduated from college reporting having received an STEMM-related degree.Our results also indicate that program alumni perceive the programs to have positively impacted their interest, confidence, knowledge, and skills in STEMM-related areas and to have reinforced their appreciation for the importance of increasing diversity in STEMM. Moreover, many alumni indicated interest in continued engagement with past and present program participants. The ability to recognize and respond to the experiences and identities that students bring with them into the academic setting is an important factor in mentoring relationships.24As such, engaging interested alumni as mentors may prove helpful for program participants who are seeking mentorship. It will be important, however, to ensure that mentors are appropriately compensated for their time to avoid contributing to the “minority tax” phenomenon, in which individuals from underrepresented groups are asked to assist with recruitment and other diversity, equity, and inclusion efforts without adequate recognition.Systemic racism and discrimination have resulted in inequitable pursuit of and access to careers in the medical and scientific workforce for individuals from groups historically excluded from STEMM fields. Pathway programs have been increasingly developed and implemented to increase matriculation of students from historically underrepresented groups into these fields.While our survey results are promising and suggest an overall positive impact of the evaluated programs, pathway programs have limitations that represent areas in need of improvement.25–27First, programs may differentially impact participants from different groups and backgrounds. We found that participants identifying as Hispanic/Latinx had a significantly higher reported level of impact of the DDCF programs on their plans to attend college. More research is needed to understand this differential impact. Previously characterized educational barriers faced by some Latinx youth include low SES, low level of parental education, financial challenges, lack of knowledge about potential careers, language differences, discrimination related to immigration/legal status, lack of access to resources, and exposure to negative attitudes in academic settings.28 promoting early exposure to STEMM topics using hands-on approaches,28 and ensuring that students perceive equitable treatment.27Given these challenges, researchers evaluating education for Latinx students have highlighted the importance of engaging parents, families, and communities in educational efforts,DDCF programs actively encourage perceptions of equitable treatment through the emphasis on all students' abilities to succeed in science and medicine, and a positive program environment was one of six themes we identified in the present study for why respondents would recommend the programs to others. Furthermore, the DDCF programs promote hands-on exposure to STEMM fields: this was one of the 10 major areas of programmatic impact reported by alumni in our survey.11Future program iterations, however, should consider additional factors such as community and family engagement and other culturally sensitive strategies to continue to address specific challenges faced by participants. This is particularly important for Black, female, and low-SES students who, like Hispanic/Latinx students, are less likely to maintain interests in STEMM careers throughout high school.29 and the Health Disparities Summer Internship program, which resulted in more than half of the 73 participating students  reporting an increase in confidence to pursue a research or medical career.30Examples of programs that have positively impacted Black and female students include USSTRIDE , an undergraduate program with increased associated acceptance rates of Black and Latinx students to medical school,It will be critical to evaluate aspects of the DDCF programs that can be modified to ensure a positive impact on other groups of students who did not report as great an impact of the program on their career aspirations.27 and there is no single model that can address all the diverse needs of students.31 This reinforces the importance of programs assessing individual participants' needs, goals, and aspirations, acknowledging the range and variety of experiences that each student brings.Of note, while we and others strive to identify common barriers faced by students from underrepresented groups with aspirations in STEMM, the same challenges cannot be generalized to all students32 Ensuring similarly culturally sensitive programming may be an important next step for the DDCF programs.Pathway program research also emphasizes the importance of maintaining culturally inclusive practices to address cultural and societal barriers to STEMM participation and to improve the educational environment for students from underrepresented groups.11 yet some pathway programs do not have adequate minority representation.30 Students identifying with these groups may be most likely to benefit from program participation and mentorship and therefore these individuals, particularly those representing dual or triple underrepresented statuses,11 should be deliberately recruited.A second potential area for continued focus in our pathway programs is participant and mentor recruitment. Students at the intersection of female, Black, Hispanic, and low-SES identities have lower rates of maintaining aspirations in STEMM throughout high school,2 recommend establishing partnerships between colleges, universities, and medical schools to strengthen pathway programs for underrepresented students. Based on this recommendation, additional work in establishing and strengthening existing relationships with undergraduate and medical schools to ensure continuity for students throughout their training and career paths is warranted.Third, structural alignments with other pathway programs may help increase their relative impact. Wilbur et al33 Our results corroborate the positive impact of these programs and substantiate the need for their continued funding and integration into systems of higher education.Furthermore, despite long-standing evidence for the effectiveness of pathway programs, programs face waning governmental and philanthropic financial investment.8 and other aspects of structural racism. For this reason, it is clear that pathway programs represent one of many larger societal changes necessary to enact change.Finally, it is essential to acknowledge that a key limitation to these programs is that they do not mitigate all external factors that influence participants' future career trajectories, such as the bias and discrimination in recruitment for scientific and medical trainingAdditionally, while our results about the impacts of the programs are overwhelmingly positive, 3 of the 269 alumni respondents said they would not recommend the program to other students. The open-ended responses of two of these alumni revealed that this was due to issues of not feeling connected with and supported by program staff. Furthermore, we do not have information about the demographics and experiences of the alumni who chose not to respond to the survey.Continuing to assess and improve pathway programs is an essential component of supporting students from groups historically excluded from and therefore underrepresented in STEMM. To best serve our diverse communities, educational systems must facilitate equitable access to training in STEMM for individuals from historically underrepresented groups, promoting not only their application and matriculation into STEMM programs but also their retention and promotion.The continued study, improvement, and funding of pathway programs will serve as one strategy—of many that are necessary–to ensure the continued support of students from persistently underrepresented groups in their STEMM career aspirations.Alumni of eight DDCF high school-to-college clinical research pathway programs reported positive perceptions and impacts of program participation, including increased interest, knowledge, confidence, and skills, as well as increased intentions to pursue a career in STEMM. Alumni identifying as Hispanic/Latinx particularly felt the program impacted their intent to matriculate to college, and the majority of alumni wished to engage in future program iterations through mentorship.These findings support the beneficial role of pathway programs in increasing diversity and equity in health care, which is essential for provider well-being and quality of patient care. Pathway programs require continued study, redesign, and funding to remain impactful and beneficial for all participants.Furthermore, while these programs appeared to have an overall positive impact on participants, they represent only one of many levels of interventions necessary to dismantle systemic racism to improve equity and diversity in STEMM.


Modern manufacturing engineering requires quick and reasonable solutions during the production planning stage, ensuring production efficiency and cost reduction. This research aims to create a scientific approach to the rational choice of a locating chart for complexly shaped parts. It is an important stage during the manufacturing technology and fixture design process. The systematization of the designed and technological features of complexly shaped parts and the definition of the features that impact a locating chart create the fundamentals for justification. A scientific approach has been developed using the complex combination of the part’s features and a decision-making approach using the example of bracket-type parts. The matrix of design and technological features of parts was developed including steel AISI 3135 and cast iron DIN 1691. The classification of locating charts for bracket-type parts was defined. A mathematical model of the rational choice of the locating chart according to the structural code of the workpiece was verified in case studies from the practice. As a result, a decision-making approach was applied to the rational choice of the locating chart for any bracket-type part. The proposed solutions improve the production planning stage for machine building, automotive, and other industries. Up-to-date production is characterized by increasing requirements for the quality and accuracy of machine parts manufacturing. One of the key directions is the precision machining of complexly shaped parts from different materials according to the quality requirements. The importance of fixture design is an urgent issue during manufacturing, which affects its productivity and flexibility.According to current trends in creating highly automated, flexible manufacturing, there is a need for new methodological approaches to ensure quality parameters and reduce the cost of engineering products . FixtureThe material of the workpiece influences the machining modes. Simultaneously, machining conditions impact the proper selection of the locating chart. Therefore, locating chart selection is essential to the production planning stage during manufacturing process design. Therefore, locating chart choice should also include the impact of materials for bracket-type parts. Steel and cast iron are used to make such parts in the automotive industry .This research aims towards the creation a scientific approach to the rational choice of locating chart for complexly shaped parts based on a complex combination of the part’s design and technological features and a decision-making approach using the example of bracket-type parts. To achieve this goal, it is necessary to solve the following tasks. First, the matrix of the design and technological features of the parts should be proposed. Second, the classification of locating charts for bracket-type parts should be defined. Additionally, a mathematical model of the rational choice of the locating chart according to the structural code of the workpiece should be developed. This model should be verified using case studies from practice. Finally, a decision-making approach should be applied to the rational choice of the locating chart for any bracket-type part.Process planning is recognized as the critical problem between design and manufacturing in a computer-integrated manufacturing system. Recently, many types of computer-aided process planning systems using expert system techniques have been developed . NumerouImplementing a decision-making approach is a reasonable solution for finding a suitable option for the research tasks. Decision-making methods have proven to be an effective support for engineering design. The multi-objective genetic algorithms are well-suited to a service-oriented architecture implementation. The research study  presentsVirtual reality has proven to be a popular technology for engineering design and maintenance, providing novel ways for visualization and interaction. The paper  discusseIt should be noted that fixture design is a time-consuming process that requires a variety of information considering the capabilities of technological equipment , toolingBracket-type parts are used in various working conditions: at high and low temperatures and humidity, and with significantly high environmental levels of dust and salinity.Technical requirements for bracket-type parts are specified in most literature sources, particularly in . TraditiThe proposed design and technological classification  allow fol/d >1) and short (l/d <1) locating surfaces, which fundamentally determines the method of their locating during machining and, accordingly, the design of the fixture.According to the length of the locating surfaces, the parts are classified into brackets with long  and cast iron  can be classified into light (less than 1 kg), medium (1–10 kg), and heavy (more than 10 kg), as well as for non-metallic materials, which affects the choice of equipment, cutting tools, and the appointment of cutting modes during machining.Depending on the purpose, bracket-type parts are divided into parts with high (IT6–IT7), medium (IT8–IT10), and low (IT11–IT14) accuracy of locating surfaces. The machining of the locating surfaces with high accuracy, other things being equal, guarantees the more reliable and durable operation of the part and the product as a whole.Considering the overall dimensions, bracket-type parts are divided into small (less than 50 × 50 mm), medium (from 50 × 50 to 300 × 300 mm), and large (more than 300 × 300 mm), which determines the overall dimensions of the fixture and the required working space of the machine tool during machining. The surface roughness of bracket-type parts is in the range of values Ra = 0.8–6.3.Several theoretical locating charts can be proposed for each typical bracket-type representative. The choice of locating chart depends on the geometric shape and design features of the part , accuracy, size, shape, and spatial location of surfaces relative to each other, quality, roughness of surfaces, and stiffness .In practice, the number of implementation methods of locating charts can be extended according to the particular technical requirements and working conditions. After determining the most applicable implementation method, requirements for the set of locating elements are generated . The geometric shape, overall dimensions, and quality of locating surfaces are also considered when setting these requirements.W]—the matrix of transformation.The mathematical model for the rational choice of a locating chart is based on the following matrix equation:N × m, where N—the total number of considered parts; m—the total number of locating charts. The matrix [Ξ] of design and technological features is also a rectangular one, with the dimension of N × n, where n—the total number of design and technological features.The matrix [Ψ] of locating charts is a rectangular one with the dimension of W] with the dimension of n × m. Its elements are determined based on previous practical experience in the rational design of m various locating charts for N typical parts. The corresponding expression for the matrix [W] can be written according to the following regression equation:Notably, Equation (1) can be applied to the rectangular matrix , a total number of parts N = 12 has been considered. The total number of design and technological features: n = 11  has a dimension of 12 × 11:m = 6. Therefore, the rectangular matrix [Ξ] of locating charts has a dimension of 12 × 6Also, the total number of locating charts It should be noted that all of the columns have at least one non-zero value for the total consideration of all types of locating charts.For each typical representative of a bracket-type part, calculations of the automatic selection of the rational locating chart according to the proposed approach, based on Equations (2) and (4), were performed. Particularly, the matrix of transformation is determined according to Equation (2):N = 12 parts was determined by Equation (4). Their values are summarized in Also, the normalized decision-making factors 6 × 1 of locating charts for each part from the array of Analysis of the decision-making results is as follows. First, the numerical calculation results correspond to the assumed locating chart for all parts. This fact justifies the developed regression model (1)–(4) for decision-making in the rational choice of locating charts.Second, for the 10th part, the decision-making approach proposes choosing the fifth locating chart despite the considered third one. Evidently, the locating chart using two planes and a hole is more appropriate for the part presented in W], the decision-making approach should be evaluated according to an arbitrary part. The design and technological features are different from previously considered data. As a particular case study, such a bracket-type part is presented in After evaluating the transformation matrix [The structural code of the part  is “SAC1Y} of the locating chart is as follows:Therefore, according to Formula (5), the evaluated column-vector {It is also graphically presented in Y2 = 0.46 is the most valuable, the preferrable locating chart is No. 2—by an external cylindrical surface and two planes. The calculation results confirmed that, in practice, when machining bracket-type parts for similar machining conditions, the locating chart should be chosen by external cylindrical surface and plane.Since the second factor The scientific novelty of the present research is the development of a mathematical model for the rational choice of a locating chart by the design and technological features of the machined part based on the decision-making approach. Verification of the developed model was carried out on 12 examples of practical case studies and for an additional bracket-type part.Analyzing the calculation results , three cThe significance of the obtained results and their reliability is proven by their comparison with recent, globally produced results. Particularly, the locating system design module for freeform workpieces was developed for computer-aided fixture design . MoreoveAs a result, a general scientific approach to the rational choice of locating chart for complexly shaped parts is based on the complex combination of design and technological features and a decision-making approach. The case study for bracket-type parts proves the reliability of this approach.The practical significance of the obtained results is highlighted by the need to automate the fixture design process to reduce the time cost of the production planning stage. It is a crucial task in the implementation of the Industry 4.0 strategy.Further research will be aimed at investigating the effect of locating errors.The systematization of locating charts for workpieces in fixtures has been further developed. In particular, 6 locating charts and 23 implementation methods were identified for bracket-type parts manufacturing on CNC machining centers. Therefore, a scientific approach to the rational choice of the locating chart for complexly shaped parts has been developed to improve the fixture design process, including that performed in automated mode, based on a complex combination of the 11 design and technological features of the part and a decision-making approach.The number of numerical simulations is not less than the total number of design and technological features during modeling. Moreover, the following limitations were used: the minimum weight of the part—1.2 kg, the maximum weight—2.9 kg; materials—steel and cast iron; an accuracy range of locating surfaces from H6 to H11; overall dimension ranges varied from 48 × 44 × 40 mm to 187 × 189 × 149 mm; the surface roughness was in the range of values: Ra = 1.25–3.2.The developed methodology considers the different types of materials: steel AISI 3135 and cast iron DIN 1691. Its verification was completed using the example of bracket-type parts. Comparative analysis of the obtained results proved that the calculated solution corresponds to the variant assigned by an expert in the majority of cases. Practical implementation of the developed scientific approach was examined using bracket-type parts for the automotive industry.Further development of the research is related to integrating the developed scientific approach into a computer-aided fixture design system, which will allow it to be integrated into intelligent manufacturing.


This commentary discusses an article by Broekhuizen et al which assesses policy options for scaling up the SURG-Africa surgical team mentoring program in Malawi to increase access to surgical care. In modeling these scenarios, the authors assess the cost of scaling up surgical teams mentoring and the impacts of scaling the program on district hospitals (DHs) and central hospitals (CHs). The additional costs borne by DHs when increasing surgical volume remains a significant issue identified by the authors and could ultimately determine the success of the program. The piece indirectly advocates for an increased role for task-shifting. The Ministry of Health of Malawi will have to ensure the appropriate governance and regulatory processes are in place to maintain quality and accountability. Their study assesses scenarios for the continuation and scale-up of the SURG-Africa surgical team mentoring interventions in Malawi. To accomplish this, first, the authors organized two group model-building workshops with actors from district hospitals (DHs) and central hospitals (CHs) to assess factors that could influence the sustainability and replicability of the SURG-Africa surgical mentorship program in Malawi. Second, they model the impact of six different implementation scenarios of the mentorship program. For each scenario, the authors model the changes in referrals from DHs to CHs, financial implications for DHs and funders, and implications in terms of surgical specialist days lost and bed occupancy days averted at CHs.We read with interest the article by Broekhuizen et al titled “Improving Access to Surgery Through Surgical Team Mentoring – Policy Lessons From Group Model Building With Local Stakeholders in Malawi.”Our observation is that despite an exponential rise in publications relating to “global surgery,” there remains little evidence-based implementation science to apply and scale evidence-based interventions to improve quality or access to surgical care in low- and middle-income countries (LMICs) where the surgical burden is greatest. They have then honestly evaluated its applicability as a nationwide policy intervention, the cost and opportunity cost of implementation to policy-makers and implementers, and highlighted the many potential threats and limitations to its effectiveness. Since the Lancet Commission on Global Surgery publication in 2015, the number of articles relating to “global surgery,” defined as work to improve quality and access to surgical care worldwide, has grown significantly. However, the majority of these works could be classified as “advocacy” pieces as they spread the research of the Lancet Commission on Global Surgery to a wider audience or highlight new takes on publicly available information. Other studies have focused on descriptive assessments of surgical capacity in LMICs or review articles. Few contain empirical studies assessing the implementation and scale-up of surgical interventions in resource-constrained settings. This is often because acquiring primary data is challenging and expensive, requiring dedication and “hard work,” particularly in resource-constrained environments.In their paper, Broekhuizen et al have tackled this difficult knowledge gap head-on by taking the time and resources to implement and rigorously evaluate an intervention aimed at improving surgical care in a resource-constrained setting. NSOAPs have also been advocated to avoid the pitfalls of vertical disease programming and to create an overarching strategy to tackle each of the World Health Organization (WHO) health system building blocks  as it relates to surgical care. Such plans have been created in several countries, including Pakistan, Zambia, Tanzania, Rwanda, and more. However, despite the increased interest from Ministries of Health, particularly in Africa, to develop NSOAPs, there remains a severe lack of evidence-based implementation science on implementing and scaling effective interventions to be included as policy within NSOAPs. NSOAPs have effectively built coalitions in the surgical community and serve as an advocacy tool to promote the inclusion of surgery in National Health Plans. However, their effectiveness as truly implementable, effective policy documents is limited by this lack of evidence of what to contain within them. Although their article could be interpreted as dense or difficult to read, it is rigorous analyses such as these by Broekhuizen et al, that takes time and expertise to develop and interpret, that will significantly contribute to the implementation science literature and will form the substrate to take NSOAPs from a tool of advocacy to a credible tool for policy and ultimately tangible improvement in the surgical system.Within the global surgery literature, there has been an emphasis on creating National Surgical, Obstetric and Anaesthesia Plans (NSOAPs) to holistically and systematically strengthen surgical systems. In their group modelling exercise, the authors demonstrate the complexities of the healthcare system. The context and threats to the effectiveness of their intervention are then expanded on again in the discursive part of the paper. The number of threats to effective implementation are perhaps evidence of the inherently limited impact of programs that touch only one area of the complex healthcare system, in this case, the area of mentorship of existing personnel. In addition to the threats mentioned in this article, several health system building blocks need to be addressed in a coordinated way if the program is to sustain its impact.implementers [...] should focus on finding ways to incentivize actors so that new  behavior emerge,” Broekhuizen et al point to a threat to the effectiveness of their intervention being a lack of incentives for both the CHs and the DHs involved. For 90% of DHs in the southern region  and over 50% of all DHs overall, the cost of performing surgeries outweighs the cost of referral and the reimbursement from the government . As such, every additional surgery performed at the hospital is loss-making. This is in the context of many DHs already on the brink of insolvency, relying on donor funds and credit. The authors therefore propose an output-based or performance-based resource allocation to compensate DHs for the additional costs due to increased surgical volume.The financing piece of the health system is often the most crucial and consequential. The authors’ quote, “ For the DHs to increase their surgical volume and reduce referrals sustainably, the reimbursement for surgical services must be increased to match costs and overcome the financial disincentive to operate at the DH. While Broekhuizen et al consider the cost of scaling up the mentoring intervention in their model, they do not consider the additional costs of increased surgical volume and the cost of adequate reimbursement, which likely far outstrips the cost of scaling up the mentorship program alone. Particularly in a resource-constrained setting, calculation of these additional costs will be crucial to informing policy decisions.The recommendation for output-based resource allocation for surgery needs to be examined in the current Malawi health financing situation. Currently, Malawi spends approximately US$39 per capita per year on essential healthcare, a majority of which is from donor funding. As well as the disincentive to reduce referrals for the DHs, the authors also acknowledge that for the CHs, the immediately noticeable effect of the program may be the absence of mentors. In contrast, the reduced bed days which may result from decreased referrals may be subtle and not be attained until a year or more into the program. The authors have not suggested how these disincentives to the mentorship program are to be overcome. It is commendable of the authors to be so open about this limitation. A clear strategy to incentivize increasing surgical capacity at the DH and incentivize the mentorship program at the CH is required if the program is to work sustainably. However, as in many other contexts, this brings its own ethical challenges. Training of physicians is routed in basic sciences starting from undergraduate level up to specialisation. Non-physician training, including this mentorship program, lacks this component and concentrates on skills building. Most clinical decision making builds on an understanding of the basic sciences and pathology. This may be deficient in a skill-based mentorship program. However, once non-physicians have been trained to conduct some surgeries, there becomes a thin line on what they can do, and what their limits are. The Malawian legislator therefore needs to establish clear guidelines to make sure that non-physicians do not go beyond their scope of work but also that there is accountability. Should a non-physician clinician follow a surgeon’s recommendation if they disagree with it and are at risk of litigation even if they follow the surgeon’s recommendation? Given the unsolved questions around governance in task shifting, it is difficult for the Malawian patient to truly consent to their treatment without knowing who is ultimately responsible for their care. Therefore, the ethical and legal frameworks around task-shifting must be considered as interventions focused on scaling up task-shifting/sharing are considered in national policies to improve access to surgical care. In Malawi, task-shifting/sharing has been successfully used to scale up the treatment of other conditions such as HIV/AIDS. Lessons from task-shifting for such conditions, as well as lessons in surgical task shifting from other countries, could be considered in efforts to scale up the provision of surgical care by non-physician providers in Malawi.The SURG-Africa program consisted of mentorship of non-specialists to expand the surgical volume and decrease referrals to CHs in Malawi. The program, therefore, reinforces the important role of task-sharing/-shifting in increasing the surgical workforce density in LMICs and Malawi. This study has chosen referrals from DHs to CHs and costs as the measurable endpoints to the SURG-Africa interventions. These endpoints have merits as they are easily quantifiable and measurable. However, neither of these reflects the quality of care. Access to surgical care is defined as access to care that is safe, timely, and affordable. These include clinical, process and implementation outcomes that should be taken into consideration in any surgical program. The authors determined a threshold of diminishing returns at which increasing the surgical volume in DHs will cost more than referrals to CHs. The concept of diminishing returns can also be applied to safety because running a DH at its maximum profitable capacity does not guarantee logistical efficiency, infrastructural capacity, or safety. Can we safely assume that other components of the Malawian health system would evolve to accommodate the increasing surgical capacity of DHs? Additionally, in any scenario which advocates for increased responsibility and capacity of task-shifted non-specialists, assurance of the quality of care must be central as increased surgical volume without quality assurance is not a real increase in capacity. Alongside mentorship, there must be a system of prospective monitoring of the quality of care outcomes for patients to ensure this intervention is safe and effective at reducing referrals. The policy options presented in their findings could inform the Ministry of Health policies to scale up mentoring of non-physician surgical providers in Malawi. In developing a policy to scale up mentoring, the additional cost of output-based resource allocation to DHs, as well as ensuring the quality of care and the complex governance issues around task shifting should be taken into consideration as these are paramount to the impact and sustainability of the proposed policy recommendations.In summary, findings from the research conducted by Broekhuizen et al contribute to implementation research needed to sustainably scale up surgical interventions in resource-constrained settings. Not applicable. Authors declare that they have no competing interests. All authors contributed equally.


The univariate analyses revealed five app-feature clusters and four program-feature clusters. The bivariate analysis revealed five unique combinations of these clusters, called engagement personas, which represented 76% of users. These engagement personas differed in both member demographics and weight loss. Exploring engagement personas is beneficial to inform strategies for personalizing the program experience and optimizing engagement in a variety of digital health interventions.Digital health technologies are shaping the future of preventive health care. We present a quantitative approach for discovering and characterizing engagement personas: longitudinal engagement patterns in a fully digital diabetes prevention program. We used a two-step approach to discovering engagement personas among  Digital technologies are transforming health care and making preventive care broadly accessible to meet the needs of a growing population . The COVPrimary prevention of diabetes and its complications is an important public health priority. More than 96 million Americans have prediabetes , with anDigital DPPs have the potential to reach a large portion of the population. In 2021, 85% of individuals in the US reported owning a smartphone , which iUser personas are commonly used in user experience research to highlight salient characteristics that represent the characteristics and/or needs of a larger group of users or research participants. User personas are traditionally fictitious, and they are defined based on the expectations of user experience researchers or interview responses from a few archetypal members . User peTo further understand user engagement in digital health over time, there is a need for additional research on user personas and their utility. Specifically, new techniques are needed for discovering distinctive, longitudinal engagement patterns or engagement personas. We propose that data-driven approaches using longitudinal engagement data can be the most informative for discovering engagement personas. A data-driven approach can identify salient features of members based upon real-time behavior and actions that do not necessarily fit qualitative predeterminants. The detailed information from each member’s time series of interactions with a program enables the discovery of engagement personas using quantitative techniques that minimize human bias . EngagemThe goal of the present study was to quantitatively identify engagement personas based on distinctive, longitudinal engagement patterns in a fully digital DPP that is powered by conversational artificial intelligence. We employed a hypothesis-generating, inductive approach; thus, we did not have an a priori hypothesis as to the number of engagement personas we would discover. However, we did expect to observe significant differences in clinical outcomes between engagement personas that differed in their amount of engagement. For example, we hypothesized that members belonging to engagement personas that exhibited higher overall engagement would lose a greater amount of weight than members of engagement personas characterized by lower overall engagement. Finally, to validate the discovered engagement personas and investigate this hypothesis, we compared characteristics and weight loss between engagement personas and tested the persistence of the engagement personas both with an alternative algorithm and on an independent set of data.This study used data from a fully digital program called the Lark DPP, which has thousands of members participating monthly. This DPP has full recognition from the Centers for Disease Control and Prevention  (organization #4358176), which is the highest level of DPP certification based on approved curriculum, enrollment, engagement, and clinical efficacy outcomes. The educational content follows the PreventT2 curriculum [Lark’s coaching of these healthy behaviors is unique among the digital DPPs recognized by the CDC because it is powered by conversational artificial intelligence. Human experts and coaches designed the coaching conversations using evidence-based strategies including cognitive behavioral therapy and positive psychology. An artificial intelligence coach is uniquely positioned to provide personalized coaching since it can retain a perfect memory of all interactions with a member and refine coaching, feedback, and mechanisms of delivery as it learns the patterns and preferences of each member. Thus, an artificial intelligence platform is ideally suited to facilitate the discovery of engagement personas because it gathers and stores detailed information about individual patterns of use.The coaching in this digital DPP imitates a human coaching experience and provides (1) An educational curriculum on behavior change related to diabetes prevention, (2) Feedback and insights on goals and health behaviors, and (3) Daily and weekly summaries of member progress  so that users can track their progress over time. The artificial intelligence coach engages members in synchronous, text-based coaching conversations, so users can have a coaching exchange any time they open the app.2. Members also had to meet one of the following: (1) Blood test result (within the past year) of fasting plasma glucose between 100 and 125 mg/dL, two-hour plasma glucose after 75 g glucose load between 140 and 199 mg/dL, or hemoglobin A1c of 5.7–6.4% or (2) Risk score from CDC’s online risk assessment indicating high risk for type 2 diabetes [Members enrolled in the digital DPP as a covered service under their health insurance plans. All members agreed to the app’s privacy policy at registration, which included permission to use their de-identified data for research. This study received exemption status from the Advarra Institutional Review Board  for retrospective analyses of de-identified data. Members had to meet CDC requirements for prediabetes: (1) Aged 18 years or older, (2) No previous diagnosis of type 1 or 2 diabetes, and (3) Initial body mass index ≥ 25 kg/mdiabetes . CDC reqWe used data from members who enrolled in this digital DPP between 1 August 2019 and 1 October 2019. Using this time frame ensured that all members included in the analyses experienced the same version of the application and program features. A total of 2938 members enrolled during this time frame and downloaded the mobile application to their smartphones. For a member to be included in the analyses they also had to show extended engagement, defined as having at least one bidirectional conversation with the artificial intelligence coach after their first 90 days in the program. A total of 1613 members met this criterion for extended engagement. We required extended engagement for a couple of reasons. First, although this period of engagement was longer than other studies adopted  at program initiation using each member’s height and starting weight.Members entered their demographic data and additional characteristics in-app, including their age, gender, ethnicity, and height. All members included in the present analyses received a digitally connected scale upon enrollment that automatically uploaded their body weight to the app. We assessed starting body mass index . The app sends meal logging reminders to members outside of coaching exchanges to encourage members to log meals.Members use their connected digital scales to complete weigh-ins during the program. Members can weigh themselves as often as they wish but are encouraged to complete at least one weigh-in per week.Lesson check-ins refer to the completion of portions of educational lessons within the app that correspond to the National DPP PreventT2 curriculum . The digWe created four time series variables from the above-detailed engagement metrics: (1) The number of days per week with coaching exchanges, (2) The number of days per week with meals logged, (3) The number of days per week with a weigh-in, and (4) Whether a member checked in each day  with the educational lessons of the program. Time series 1–3 were “app features” and captured the major ways that a member could engage with the app. These time series were each 13 elements long per member, corresponding to the weeks (1–13) in the first 90 days of the DPP. Since there was no limit to the number of coaching exchanges, meals logged, or weigh-ins a member could have, we aggregated these app features weekly to capture high-level, cross-app engagement while reducing sensitivity to fluctuations in daily engagement. We further concatenated these app-feature time series into one 39-element series  to simplify the model and focus our clustering on the general shape of members’ app-wide engagement without emphasizing any single feature a priori. Time series 4 was a “program feature” reflecting the most important data used to measure a member’s progress through the educational lessons of the DPP. The app only allows members to complete one check-in per day, so this time series was 90 elements long to capture higher-frequency, daily changes in program engagement.We employed a hypothesis-generating, bottom-up approach to discovering engagement personas; thus, we did not have an a priori hypothesis as to the number of engagement personas we would discover. We used a two-step approach to discovering engagement personas. The first step was a univariate clustering method using an unsupervised k-means clustering algorithm with a Euclidean distance metric. We conducted two separate univariate clustering analyses, one for engagement with app features and one for engagement with program features. The second step involved cross-referencing the app- and program-feature univariate clusters to reduce noise in the final engagement personas and identify members who were strongly related across both app and program features. Each step is detailed below.We independently clustered the data for each time series: first for the concatenated weekly app-feature time series and second for the daily program-feature time series. We log-scaled the app features after observing improved cluster fit for the log-scaled vs. raw data. This was likely because the log-scaled data captured the percent difference between members’ frequencies of app actions rather than the absolute distance. For example, members engaging more frequently in an app behavior  were likely more similar than members engaging less frequently in that same behavior .We chose independent clustering steps for app features and program features to maximize the flexibility of engagement patterns represented by our clusters and enable each group of features to retain a unique time scale of observation . The general program feature also enabled this approach to be program agnostic. App features represent items that the Lark program has control over within the app: the amount of coaching, weigh-in reminders, and meal logging reminders. For the National DPP, the program material is defined by the PreventT2 curriculum. If a different program used this method for discovering engagement personas, the app-feature clustering could be cross-referenced to the educational content of a different program, as described below under “Bivariate Clustering Method”.Without an a priori expectation for an optimal number of clusters, we evaluated the resulting clusters at several candidate choices of k and selected the final value when the clusters were granular enough to capture meaningful differences in engagement patterns while remaining representative enough to avoid overfitting. For each univariate clustering step, we independently determined the optimal number of clusters, k, using silhouette coefficients and post hoc analyses. The silhouette coefficient ranged from 1 (best match to its own cluster) to −1 (worst match), with 0 indicating near-overlapping clusters. When determining the final k, we used the average silhouette coefficient of each cluster and selected a k that resulted in the highest coefficient per cluster.a, b) as the number of members with app-feature cluster label “a” and program-feature cluster label “b”. Then, for each label pair , we identified an engagement persona if:appk and programk are the numbers of clusters per method, and T is the minimum required percentage of the marginal total number of members. We determined the final cross-referencing threshold by evaluating the average silhouette coefficient among all members belonging to clusters that resulted in an engagement persona at each cross-referencing threshold and further selecting a threshold that captured a large percentage of members. We performed Mann–Whitney U tests to compare the average silhouette coefficients of members with versus without an identified engagement persona for each of the univariate clusters.The bivariate clustering method involved comparing the cluster label for each member across both univariate clustering steps. We constructed a resulting contingency table of cluster label counts per each label pair . We idenWe used Tukey post hoc tests to compare participant demographics and characteristics , aggregate engagement counts , and clinical outcome (weight loss at 4 months) between the engagement personas. We also compared engagement personas on membership in the digital DPP’s concurrent Facebook group to better understand if certain engagement personas were more inclined to seek social support from peers.We statistically validated the engagement personas to ensure that the distinctive longitudinal engagement patterns, which represent the unique ways that members engaged with the digital DPP, were both representative and consistently discoverable. This validation approach consisted of two steps: (1) Using an alternative clustering algorithm called hierarchical agglomerative clustering with Ward’s method and (2) Applying the same k-means clustering method on an independent test sample of 1071 DPP members from the same program using a different enrollment window of 1 February 2020 to 30 June 2020.As described above, when determining the final k, we used the average silhouette coefficient of each cluster and selected a k that resulted in the highest coefficient per cluster. Using this criterion, the univariate analysis revealed five app-feature clusters a and fouThe app-feature clusters revealed stratification across both the level of engagement and the type of engagement with app features. We discovered clusters of members who had high and low engagement relative to the sample average across all app features and members whose engagement differed depending on the app feature a. Weigh-The program-feature clustering revealed clear stratification of engagement with the educational lessons of the DPP. There were four program-feature clusters: two clusters emerged representing consistently low and high engagement with lesson check-ins over 90 days relative to the sample average  might be Learners (combined with app-feature cluster 3) or Enthusiasts (combined with app-feature cluster 5). Similar to the univariate clustering results, differences in the frequency of weigh-ins enabled us to distinguish the engagement personas. For example, a member with lower program engagement than the sample average had a lower frequency of coaching exchanges and meal logging, and vice versa, with only that member’s weigh-ins differentiating between each bivariate cluster and determining their final engagement persona.The bivariate clustering method identified engagement personas for members with the best fit in their respective univariate clusters. We compared the silhouette coefficients for the 1219 members with an identified engagement persona to those of the 394 members who did not belong to an engagement persona. For a display of the relationship between the goodness of fit for each cluster pair, The engagement personas differed in aspects of demographics, characteristics, and clinical outcomes . Enthusin = 1613 training set of members and when using the same k-means approach on an independent test set of n = 1071 members  Casual Members, (2) Mainstream Members, (3) Learners, (4) Data-Driven Members, and (5) Enthusiasts. We also provided both clinical and statistical validation of these engagement personas. Notably, the engagement persona with the highest engagement (Enthusiasts) had the greatest weight loss compared to other engagement personas, followed by Data-Driven Members and Learners. Engagement personas with lower engagement  and engagement drop-off over time (Mainstream Members) had the least weight loss. These findings add to the growing body of literature examining digital health engagement and demonstrate that longitudinal patterns of engagement are linked with clinical outcomes, similar to the findings of research on other types of digital health programs ,18.In one recent study, the authors examined variability in cardiovascular monitor use and found four engagement patterns based on the amount of use over time: consistently high use, consistently moderate use, early high use with a rapid decline to low use, and early moderate use with a decline to low use . There wThe engagement personas captured key differences in member characteristics and clinical outcomes. Two of the discovered engagement personas represented higher and lower engagement, respectively, with all features of the digital DPP. Enthusiasts exhibited higher app- and program-feature engagement than the sample average. These members likely have high levels of intrinsic motivation  and are We observed that Enthusiasts were significantly older than Casual Members. The National DPP has historically recruited large numbers of older adults (29.2% aged 55–64 years and 30.9% aged > 65 years) and demonstrated better retention and outcomes for older compared to younger members . One reaWe also observed two engagement personas that demonstrated a strong affinity for specific features of the digital DPP: Data-Driven Members regularly weighed in, and Learners had high engagement with educational content. Weight loss is the primary goal of the DPP, and Data-Driven Members seemed particularly motivated by tracking the data most related to this goal. Self-weighing has a well-established relationship with weight loss , and a gPeople who enjoy tracking their data tend to be motivated by seeing their progress expressed numerically and comparing their results to others’ . GamificThe digital DPP examined in this study is already a personalized program powered by artificial intelligence. The coach leverages a member’s unique data  to offer personalized insights and suggestions. However, much like a human coach, it takes time for the artificial intelligence coach to learn how a member prefers to be coached and in what areas they might require additional support. Rather than altering the content that a member receives, the discovery of distinctive engagement patterns can help the artificial intelligence coach to tailor the delivery of this personalized content, coaching, and feedback to be most palpable and beneficial to each member. Identifying distinctive patterns of engagement also enables the coach to determine if a member’s actions match their stated goals. The artificial intelligence coach can facilitate greater awareness of self through identifying mismatches between goals and behaviors . EngagemBivariate clustering enabled us to describe engagement patterns that would not have been clear had we limited our investigations to only univariate behaviors. For example, we observed that certain app behaviors, such as frequent weigh-ins, helped to distinguish groups of low and high program engagers. Thus, frequent weighing may be an important behavior indicative of program engagement. These types of insight facilitate the tailoring of coaching for specific patterns of engagement, as described above. For this study, we based the program features on the required National DPP curriculum, so the goal would be to optimize app features in the delivery of the DPP. However, the discovery of engagement personas is beneficial not only to inform future enhancements of a digital DPP but more broadly to inform a variety of digital health interventions. Digital health platforms are increasing in popularity for the management of a variety of chronic health conditions . DetermiDigital technologies are dynamic entities that are continuously refined and updated. There will always be a certain degree of heterogeneity in the program experience across different member cohorts. We plan to apply this method for discovering engagement personas to a much larger population to obtain a more stable sample average of longitudinal engagement patterns. We also have additional questions about the engagement personas discovered in this study. For example, does engagement persona membership at 90 days persist to the conclusion of the year-long program? Is it possible to identify an engagement persona earlier than 90 days? How malleable is an engagement persona? Can we use targeted engagement strategies to help a Casual Member or Mainstream Member to change engagement personas and increase their likelihood of retention? To address these questions, we plan to investigate engagement persona membership at different time points during the year-long program.We also plan to explore the engagement personas in the context of more detailed clinical outcomes to better elucidate the relationship between engagement in a digital DPP and clinical outcomes, which is a nascent area of research. While our preliminary investigation of weight loss suggests that there are differences among the engagement personas, would this remain true at one year? A member may get what they need from a program with very different engagement patterns; there may be more than one path to success. Indeed, members of all engagement personas achieved some weight loss over the first four months of the digital DPP. Thus, a critical next step is to recognize when and where it may be appropriate to make modifications to a member’s experience based on the information gained from their engagement persona membership.Indirect clinical outcomes, such as self-reported medication adherence, are also of interest. Recent evidence suggests that some individuals with prediabetes benefit from taking medications, such as Metformin, in addition to participating in the DPP . MedicatA complementary area for future exploration is identifying barriers to engagement specific to each engagement persona, along with corresponding interventions (facilitators) that effectively address these barriers. Specific barriers may include health beliefs, low intrinsic motivation, skills deficits, low insight or awareness, reactance, or other psychological factors that may affect engagement and outcomes . PinpoinWe employed an unsupervised learning approach for discovering engagement personas that imposed no a priori expectation on the number of engagement patterns and is flexible enough to be applied to other digital health programs. We used real member data from a commercially available, CDC-recognized DPP, and the collection of these data imposed no burden on members . We included members with a wide range of engagement patterns from Casual Members to Enthusiasts and investigated preliminary differences in the clinical outcomes associated with these distinctive engagement patterns.We did not identify an engagement persona for all members in this study and, further, did not observe complete persistence of members in each engagement persona during statistical validation. However, these differences do not reduce the utility of the univariate and bivariate clustering techniques. Engagement personas take time to refine, and we expect improvements in both the percentage of members with an identified engagement persona and the persistence of the engagement personas as the sample size for model training increases. The clinical validation of the engagement personas should be considered exploratory since we did not have complete data for some member characteristics and clinical outcomes. Finally, just as there are limitations to qualitative techniques for developing engagement personas, there are also inherent limitations to purely quantitative techniques. Behavioral patterns represent an important aspect of engagement but do not capture other aspects such as affective and cognitive investment in a program . We planDigital health technologies are shaping the future of preventive health care. To realize their full potential, we must identify the wants, needs, and preferences of members and the barriers to success. This study presented a method for discovering engagement personas based on distinctive, longitudinal engagement patterns in a digital DPP. The resulting engagement personas differed in member demographics, characteristics, and clinical outcomes. Enthusiasts represented the oldest members who engaged in all app- and program-specific features, and Enthusiasts achieved the greatest weight loss of 4.5% four months into the program. Other personas represented members who differentially favored engaging with specific app or program features, such as weigh-ins for Data-Driven members and educational lessons for Learners. We further identified engagement personas of members requiring additional support to increase their engagement: Casual and Mainstream Members. The engagement personas provide actionable insight into how to tailor the delivery of program content, coaching, and feedback for members with specific patterns of observed engagement. The method proposed for discovering engagement personas in this study will facilitate further tailoring of the digital program experience and, ultimately, ensure that individuals who choose to participate in a digital DPP achieve maximal clinical benefits.


Looked after children (LAC) are one of the most vulnerable groups in our society. Often, after experiences of neglect and abuse, they are more likely to experience poor mental health, attachment difficulties and problems in educational progress. This review aims to explore literature published over the past decade which addresses the mental health needs and management of LAC within CAMHS in the UK.A literature review was performed in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 structure. Selection criteria was used. The total number of papers identified after this initial search was thirty-six across the database.Literature titles and abstracts were then screened to exclude papers with an irrelevant focus. Full-texts of the remaining twenty-two papers were then assessed for relevant and conclusive information. The total number of full papers included in the research was thirteen.the mental health needs of LAC,factors relevant to the assessment of LAC in mental health servicesconsiderations associated with the management of LAC with mental health conditions.To analyse the literature identified, a framework of three themes has been highlighted. These include:Mental health needs of LAC included numerous emotional, behavioural and social problems. These were largely focused around substance misuse, emotional disorders and poor relationships with peers.the importance of supporting healthy social relationships with primary care-givers, peers and teachersmaximising the informal support of family, friends and petsthe provision of early, holistic and flexible mental health services, rather than disjointed agenciesThe main themes which have come out of research in relation to the treatment of LAC include:The literature published over the past decade has indicated the great number of adverse outcomes amongst LAC and has made useful suggestions for the assessment and treatment of these children within a CAMHS setting.Through targeted support into residential placements, offering intensive and direct psychological input at an early stage and continuing even after they have been adopted, as well as, working alongside schools to promote peer interaction could significantly reduce the adverse outcomes of LAC.Additionally, by referring patients along with their carers for psycho-education can be extremely beneficial. Alongside this, the young people ought to be directed to support groups with other LAC to meet peers who are in similar situations as themselves.