A pilot study evaluated the change in charge nurses' perception of their leadership skills after engaging in a 4-month structured leadership program. Based on a self-assessment, multimodal education using authentic leadership tenets and an appreciative inquiry framework increased participants' confidence in their skills. The hospital-based charge nurse (CN) can make or break the team dynamic on the shift. They have extensive responsibilities, but it's more than what they do that's critical, it's how they do it. CNs must use effective communication, influence, delegation, interpersonal relationship-building, and vision-setting skills while executing essential tasks. This is crucial because leadership skills have become more important than ever to reduce nursing turnover postpandemic.4The nursing leadership at a large, urban teaching hospital in Colorado recognized the value of the CN role. The hospital offered a CN orientation class. Beyond CN orientation, there were organizational development classes, but the content wasn't specifically taught to the role of the CN and participation was lacking. Because of the lack of continuous education focused on developing CN leadership skills within the organization, nurse leaders believed that creating ongoing CN-specific education could be beneficial. Nurse leaders proposed a CNO-approved evidence-based pilot program to support the strategic priority of investing in staff workforce development opportunities. The primary purpose of this pilot study was to evaluate CN participants' self-perception of improved confidence in leadership skills after engaging in a 4-month structured leadership program.The literature suggests that leadership development programs are beneficial. Kramer and Davies conducted a review of nine studies and concluded that formal orientation and ongoing leadership support is required for CNs to be successful.4Although the benefit of ongoing leadership development has been defined, there are no standardized competencies for CNs.It was important to design this program with theories that CNs could apply to day-to-day work and, in turn, use when leading improvement work on their clinical units. The project leader selected Patricia Benner's From Novice to Expert theoretical framework; although often applied to clinical skills, this theory also has relevance to nursing leadership.The program also used Appreciative Inquiry (AI) as the model for change management.The Charge Nurse Leadership Development Workshop Series (CNLDWS) was created to evaluate if implementation of a 4-month structured leadership development program increased CNs' self-perception of confidence in their leadership skills. The project leader also explored commonalities in factors that motivated participant involvement in leadership development. Participants were included if they worked in a hospital-based CN position regardless of years in the role, were in good employment standing, and were willing to attend the classes. There were 111 eligible CNs at the time of course offering; however, because this was a pilot program, the recruitment was limited to 12 CNs from diverse units. To recruit participants, the project leader sent targeted emails to eligible staff and advertised at daily CN patient staffing meetings. Participation was voluntary, without penalty for course discontinuation, and continuing-education credits were provided.The CNLDWS was a 4-month course that consisted of four 90-minute, in-person classes (one per month) followed by an email with related optional multimodal education, such as podcasts, articles, and blogs. CNs were compensated for attendance. All classes were taught by an experienced nurse leader within the organization who filled the role of course lead. The classes were held in four consecutive months (May through August).The CNLDWS curriculum was thoughtfully designed to be specific to the CN role, based on the evidence and the organization's leadership performance expectations. The courses were structured to build on the knowledge learned from class to class and enriched through group discussion. Course topics with subtopics are outlined in Table After each class session, the course leader distributed an email to participants with a brief course evaluation and optional self-driven learning. There were also six weekly emails sent after the fourth class to foster ongoing engagement in self-paced learning. Multimodal self-study options from nursing and nonnursing sources included journaling; free online podcasts and videos; self-assessment tools; and publications such as books, journal articles, website blogs, and professional documents. Participants had the option be compensated up to 2 hours per week for completing this work remotely with manager approval, as supported by organizational policy.Leadership Practices Inventory (LPI): Self.Model the Way, Inspire a Shared Vision, Challenge the Process, Enable Others to Act, and Encourage the Heart.Confidence in leadership skills was measured through a pre-post assessment using the This program evaluation was reviewed and approved as nonhuman subject research by the University of Colorado College of Nursing Bridge Committee and the hospital's Quality Improvement Committee with the authority of the Colorado Multiple Institute Review Board at the University of Colorado-Denver.t test analysis.There were 12 CN participants who were predominately White (67%) and female (100%), and 58% were 30 to 49 years old  to postintervention  with a mean difference of 23.7 , t = 4.03, P = .004). An independent t test (N = 12) showed similar results, with a mean difference of 32.3 . Analysis of the Five Practices of Exemplary Leadership found a statistically significant increase in all domains  found participants reported an increase in self-assessed leadership behaviors with preintervention LPI scores  to pQualitative survey responses revealed that CNs shared similar comments for why they participated in the program. Responses showed a desire for growth in leadership skills for personal improvement and to be better leaders on their units. One participant wrote: “To help in setting up an excellent unit culture, a successful and supported team, and to promote excellent practices on our unit.” The postintervention inquiry about how the program helped CNs on their leadership journey had an overwhelmingly positive response, revealing perceived value of the content and appreciation of the multimodal educational options. Eight participants completed a 45-day postclass survey, with all positively responding that they implemented one thing they learned into practice.After initiation of the course, one unexpected benefit was that some participants requested discussions about career progression, critical conversation planning consultation, and requests for ongoing mentorship from the course leader. The nature of these discussions was very positive and, despite working relationships in some cases, had not occurred prior to the course.The results of this evidence-based leadership skill development program support and expand on current literature. Although the small sample size is a limitation, the positive findings support the work of others.Enable Others to Act had the highest mean self-assessment score postprogram.The LPI has been used to assess the leadership skills of CNOs in two studies.There was value to the CNLDWS beyond self-assessment using the LPI. Participants reported that learning application of the principles of authentic leadership, improved communication skills, and diversity training were the most helpful. Homer and Ryan also found that communication was valued by CNs as a top skill set to be included in education.When designing the course, investigators didn't yet appreciate how often negative interactions associated with racism were occurring. After the conclusion of the CNLDWS, the American Organization for Nursing Leadership (AONL) published their Core Competencies for Nurse Leadership to guide the practice of all nursing leadership roles.23An impactful taught topic that emerged during group discussion was about “hidden rules,” which are unspoken social cues within a group that are expected to be followed.Examples shared by the participants led them to acknowledge that nursing units aren't always inclusive environments. This inspired further discussion regarding tactics for how the CNs could influence a culture of inclusivity, such as those proposed by Schmidt and colleagues in their Code of Conduct for Inclusion and Diversity.Participants found value in the multimodal self-study education to address generational preferences for teaching and learning. Evidence suggests modalities such as discussion boards are preferred by baby boomers and Generation X; however, Generations Y and Z prefer more digital sources with visually engaging platforms.There were 10-18 educational opportunities sent after each class. Participants discussed what they learned from the multimodal education at the beginning of each class, which revealed evidence that learners engaged with each type of education offered. There didn't appear to be one preferred modality within the cohort. Multimodal education provided participants the flexibility to engage with the material in the method, time, and location that was convenient to their busy lifestyles.The in-person classes were essential because they allowed for robust discussion and peer-led learning. Participants occasionally missed class due to scheduling conflicts or oversights. It was helpful to assess expected attendance for each class and reschedule when needed. Text message reminders for upcoming classes encouraged attendance.Finally, in alignment with Delamater and Hall's summation, it matters that an experienced clinical nurse leader taught the course and could provide real-life application of content to the CN role.Program strengths were that CNs with varying years of experience found value in the content and reported that they applied what they learned. This pilot program provided paid time for CNs who participated as a strategy to support professional development. Participants were compensated for attending the 90-minute sessions and were provided education time for self-paced learning. By offering paid time to learn, the organization demonstrated their commitment to professional growth by encouraging nurses to become more engaged in their professional development.Efforts to continue the program have been through budgetary processes at the organization to further develop and sustain this pilot program in support of CN leadership skill attainment and growth. However, there are some limitations to the program. A larger sample size of CN engagement may have provided a deeper understanding of the impact of the CNLDWS on the CNs' perceptions of their skills. The evaluation was limited to the participant's self-perception and didn't include a 360-degree assessment or unit-based outcomes. Finally, although it was informed by best evidence to teach leadership skills to CNs, the program evaluation was influenced by the organization's culture and isn't generalizable.The CNLDWS provided high-value support to CNs with diverse levels of experience. Ideally, growing CN self-perceived leadership strengths will also benefit the organization, unit, and patient populations served. The program was informed by and tied to professional nursing recommendations, the works of thought leaders, and the organization's expectations of CNs' daily work. Although the in-person sessions were reported to be essential for discussion of content being learned, offering multimodal learning opportunities provided flexibility and a model for moving the program forward in the future.CN participants reported that improvements in self-assessed leadership skills after the 4-month leadership course benefited their careers, patients, and teams. The AI framework prompted participants to engage in personal discovery and then create an actionable plan for application and implementation of content learned at each class. Compared with the average turnover cost of $46,100 per direct care nurse, the cost per CN participant was profoundly less, averaging $709 per participant or $8,508 for the program.


N = 289) and a comparison group (N = 160). We also conducted five focus group interviews with the leaders of nongovernmental organizations, social workers, NEY, parents, and employers to explore the program’s impacts on the CLD service provisions. Our quantitative results indicate that the piloting group showed greater improvement in two career-related competencies—youth career development competency and career and life development hope—than the comparison group. Meanwhile, our qualitative results suggest both the benefits and difficulties experienced by stakeholders in the program. The findings thus provide preliminary evidence of the Hong Kong Benchmarks (Community) Pilot Program’s positive impacts on NEY and other important stakeholders. The implications of expanding the existing program and theorizing the community-based benchmark approach are also discussed.In our study, aimed at examining the effectiveness and impact of the Hong Kong Benchmarks (Community) Pilot Program, a career and life development (CLD) intervention program targeting non-engaged youth (NEY) in Hong Kong, we employed a pretest–posttest quasi-experimental design to compare changes in career-related competencies between a pilot group ( It is aOwing to economic globalization and advanced technology, however, the world of work has increasingly been characterized by uncertainty and unpredictability , 2022 thGiven high unemployment and the difficulty of career environments, providing young people with career intervention programs has been identified as an important strategy for bringing young people, especially NEY, back into employment, education, or training. In the past decade, various programs targeting NEY or NEET youth have indeed been developed and implemented worldwide. For example, in 2013, to reduce challenges facing NEY, the European Union called for member states to provide more education, apprenticeship, training, and employment opportunities under the Youth Guarantee Scheme . In anotWhile career interventions or policies targeting NEY have mushroomed across the globe, most of them are limited to employment-focused job support and thus fail to fully respond to the complicated issues facing NEY . On the In our study, we evaluated a pilot program, the Hong Kong Benchmarks (HKBM) for Career and Life Development (Community)—the HKBM (Community) Pilot Program, for short—aimed at helping NEY become navigators and active agents of their pathways to desired careers and lives by providing them with CLD interventions that can be benchmarked with world-class standards (see Sections 2.2 and 2.3 for details about the program). In particular, we examined both the program’s practical effectiveness in enhancing NEY’s career-related competencies to navigate transitions through education into productive employment  and its 2.2.1.Career-related competencies refer to a range of capabilities needed to master the transition from education to productive employment and meaningful life. In addition to vocational skills and language proficiency, career-related competencies emphasize a strong sense of agency and commitment to work, the awareness to reflect on self, and the ability and motivation to actively explore career-related resources in a given environment . In our career and life development agency, refers to initiating and sustaining motivation to achieve desired outcomes. It also refers to young people’s gaining agency in their CLD and their ability to initiate, execute, and sustain motivation toward achieving their desired career and life goals. By contrast, the cognitive-behavioral component, career and life development pathways, refers to the perceived cognitive and behavioral ability to generate multiple routes or pathways to achieve desired goals. The two components are thought to reinforce each other during the pursuit of goals and in acquiring positive outcomes in young people’s CLD  to access more job opportunities and broader career networks. In that light, conducting a pilot program under the HKBM (Community) framework can provide a direct, systematic response to NEY-related issues, help them to circumvent the risk of becoming NEET, and support them in becoming active navigators of their careers and life journeys.Although school, community, and the world of work are three common touchpoints for young people in the school-to-work transition, the use of the benchmark approach has long been concentrated in schools and rarely emerged in other settings. Taking the Gatsby Benchmarks as an example, whether in the UK or in other countries around the world, their application primarily focuses on schools . For an The HKBM (Community) refers to a systematic self-improvement framework for NGOs to build quality CLD service provisions that can be benchmarked with global standards. The framework increases the number of benchmarks from the Gatsby Benchmarks from eight to ten, which together can be grouped into three parts: the core part, the youth-focused part, and the enabling environment part , the HKBM (Community) Pilot Program was launched under the territory-wide CLAP@JC project. CLAP@JC is a 10-year project and cross-sectoral support platform for CLD that aims at fostering a sustainable ecosystem by bringing together the education, community, and business sectors to facilitate the school-to-work transition for all youth . To ensuThe first mechanism was to form the BM Core Committee as the steering body for the program, which was designed to consist of experts from the abovementioned research team at the Chinese University of Hong Kong and experienced practitioners from six participating NGOs. Next, the BM Core Committee created a checklist to be used by the BM Working Groups, which consisted of NGO leaders and social workers within each partner NGO, to help them to conduct self-assessments on their existing CLD services. The checklist also served as a guiding tool by covering sets of criteria under each benchmark that need to be fulfilled to achieve positive outcomes in changing the CLD ecosystem for youth. Last, the BM Working Groups provided supporting proof  for each criterion on the checklist, and the criteria were ranked at three levels according to the achievement of each: fully achieved, partly achieved, or emerging .The BM Core Committee initiated the second mechanism to implement the three-day training program to train experienced social workers from the six participating NGOs to become BM Facilitators. Specifically, the training program provided a comprehensive understanding of the HKBM (Community) and its background, design, and principles of implementation to trainees. In addition, the program covered an in-depth explanation and discussion of the 10 benchmarks. Most important, after the training, the BM Facilitators were expected to fully understand the process and content of the checklist for the HKBM (Community) for the purpose of self-assessment and to have gained the skills to support the BM Working Groups in self-assessment to identify each NGO’s strengths and gaps in the implementation of the HKBM (Community). The BM Facilitators were also expected to have gained the knowledge to guide the NGOs in reflecting on the results of self-assessment and developing action plans for self-improvement. In other words, the role of the BM Facilitator was essential, for they were agents who imparted knowledge and supported the BM Working Groups’ members in gaining the skills and abilities in self-assessment by using the checklist, understanding the processes of the HKBM (Community), and offering constructive advice to BM Working Groups and NGOs.For the third mechanism, the BM Core Committee, BM Facilitators, and BM Working Groups conducted review meetings. During the meetings, they discussed the rankings of each criterion of the benchmarks and shared their feedback regarding the strengths, suggested areas for improvement, and recommendations for improvement to the pilot NGOs regarding each benchmark. For example, in the evaluation results for the first benchmark, “A stable and visible career and life development policy,” the BM Core Committee and BM Facilitators suggested collecting and consolidating feedback from different stakeholders  so that every party would help to enhance the policies to achieve smooth cooperation in improving CLD interventions. In addition, they encouraged the pilot NGOs to organize regular meetings with the stakeholders and collect feedback on CLD policies and challenges in implementing the benchmarks. Moreover, for the fifth benchmark, “Youth engagement and co-creation,” they advised strengthening NEY’s involvement in co-designing and co-implementing CLD activities with their responsible social workers. Meanwhile, for the ninth benchmark, “Meaningful encounters with further education opportunities,” they advised the NGOs to provide up-to-date information to the social workers about continuing education, especially information about continuing and higher education institutions, to enhance CLD consultation with NEY. Although the BM Core Committee provided recommendations to the pilot NGOs, the pilot NGOs were encouraged to formulate and implement their action plans for self-improvement according to their organizational needs and pace, which suggested an egalitarian and collaborative relationship between the BM Core Committee and the pilot NGOs. After the implementation of the action plans, the BM Facilitators held regular meetings with the BM Working Groups to review the strategies for the improvement of the HKBM (Community) in an effort to consolidate their practical wisdom and enhance their CLD interventions. In that way, the HKBM (Community) functioned as a framework geared toward self-improvement that enabled the NGOs to periodically measure their progress toward suitable CLD interventions against world-class standards. That assumption, however, needed to be supported by evaluations of programs with solid evidence.The current research was conducted to evaluate the preliminary outcomes of the HKBM (Community) Pilot Program, particularly whether it, as claimed, could enhance the community-based CLD intervention and thus promote NEY’s school-to-work transition. Researchers have indicated that a sound program evaluation should focus not only on outcomes that can be quantitatively assessed but also on the operational aspects of the program that can be examined using qualitative methods . Accordi3.3.1.M = 19.33, SD = 3.341) to complete a self-report questionnaire: 289 in the pilot group and 160 in the comparison group. The participants were informed about the purpose and procedures of the study, and parental consent was obtained for all participants less than 18 years old. The method employed in our study was assessed and approved by an ethical review committee before its administration.To evaluate the effectiveness of the HKBM (Community) Pilot Program, we employed a pretest–posttest quasi-experimental design in our study. The pilot group consisted of NEY who participated in the HKBM (Community) Pilot Program implemented by the six local NGOs. Each participating NGO was also asked to invite other NEY who had neither joined the pilot program nor received the CLD interventions to participate in the comparison group. The participants in the pilot group and their counterparts in the comparison group were invited to complete a self-report questionnaire at the pretest and posttest, which were separated by an interval of 4 months. Ultimately, we recruited a youth sample of 449 participants 13–29 years old (SD = 2.948). Most were of Chinese ethnicity (96.9%), whereas 3.1% were ethnic minorities of South Asian or Southeast Asian descent . A total of 15.2% of the participants were receiving public assistance from the government. By level of education, the majority (64.9%) had received senior secondary or higher education, whereas most of their parents (93.6% fathers and 94.4% mothers) had received no more than secondary education. Meanwhile, of the 160 participants in the comparison group, 73.1% were young men. Most (99.4%) were of Chinese ethnicity, whereas only 0.6% were ethnic minorities of South Asian or Southeast Asian descent . A total of 3.7% of the participants received public assistance from the government. By level of education, the majority (76.4%) had received senior secondary or higher education, whereas most of their parents (82.3% fathers and 89.5% mothers) had received no more than secondary education.Apart from surveys, our study also involved the qualitative method of focus group interviews with NGO leaders, social workers, NEY, parents, and employers to gather their views on and experiences with the HKBM (Community) Pilot Program. The first and second authors organized and conducted five separate focus groups, which, respectively, consisted of six NGO leaders, eight social workers, eight NEY, four parents, and seven employers. The focus group interviews were conducted in Cantonese and lasted for approximately an hour. Guiding questions for NGO leaders and social workers included “How do you see the impact of the HKBM (Community) on social workers?” and “How did the HKBM (Community) facilitate social workers’ effectiveness in helping youth?” By contrast, guiding questions for NEY included “What have you learned from the activities that you have attended?” and “Which of those activities impressed you the most? Why?” Meanwhile, guiding questions for parents included “Have you received parental services provided by NGOs?” and “Did the services help you to understand the CLD of your children?” Last, guiding questions for employers included “What type of workplace learning does your company provide for youth referred by NGOs?” and “What drives your company to provide workplace learning to youth?” Ethical approval was obtained from an ethical review committee prior to the commencement of the interviews.3.2.As mentioned in the Introduction, the effectiveness of the HKBM (Community) Pilot Program was assessed by two career-related measures in our study: YCDC and CLDH.3.2.1.not confident at all) to 5 (highly confident). YCDC, encompassing four dimensions—engagement, self-understanding, career and pathway exploration, and planning and career management—was measured using 17 items developed by 3.2.2.never) to 5 . CLDH, encompassing two dimensions—career and life development pathways and career and life development agency—was measured using 20 items developed by 3.3.For the quantitative part of our study, we conducted multivariate repeated-measures analyses of covariance (MANCOVA) by including the scores for YCDC and CLDH at baseline and at follow-up as dependent variables to test the effectiveness of the HKBM (Community) Pilot Program according to two hypotheses:H1: The pilot group is more likely to have an increase in YCDC than the comparison group.H2: The pilot group is more likely to have an increase in CLDH than the comparison group.The factors in the analyses were group  and time , while potentially confounding social demographic variables  were covariates.For the qualitative part, by contrast, the interview transcripts were first translated into English by research assistants who are native Cantonese speakers. The transcripts were then carefully checked by the first and second authors to ensure the validity of the translation before using it as the basis of data analysis. Thematic analysis was conducted following the steps described by 4.4.1.F = 33.220, p < 0.001, partial η2 = 0.069; see F = 27.829, p < 0.001, partial η2 = 0.059), self-understanding , career and pathway exploration , and planning and career management . Taken together, the analysis showed that the increases in the four dimensions of YCDC were significantly higher in the pilot group than in the comparison group, supporting Hypothesis 1.The repeated-measures MANCOVA involving baseline and follow-up scores was an omnibus test of the two indicators of career-related outcomes: YCDC and CLDH. F = 21.794, p < 0.001, partial η2 = 0.046, see F = 9.091, p < 0.01, partial η2 = 0.020) and career and life development pathways . Taken together, the analysis showed that the increases in the two dimensions of CLDH were significantly higher in the pilot group than in the comparison group, supporting Hypothesis 2.Likewise, we found a significant interaction of time and group for CLDH . Those themes are summarized and described below.4.2.1.The qualitative findings first highlighted that the HKBM (Community) provided global standards and guiding principles for NGO leaders and social workers to review and improve existing services with broadened visions.4.2.1.1.Because each benchmark comprised several measurable criteria, NGO leaders and social workers could easily assess the performance of their existing services based on each criterion and set specific goals for future actions accordingly. Participant 1, an agency leader, stated the following:It reminds us of which benchmarks we want to achieve when reviewing and planning existing services. (Participant 1).For that reason, the HKBM (Community) was also recognized by participants as a guiding framework. As Participant 5, who was an NGO leader, pointed out, a “benchmark language” had appeared in the workplace. Her colleagues, whether in her service unit or across service units, could talk about commonly recognized standards, which facilitated the collaboration of the entire NGO.4.2.1.2.NGO leaders and frontline social workers embraced the HKBM (Community) as a set of guiding principles also because they benefited from its comprehensive approach to CLD services. In particular, they were inspired to shift the focus of youth services from employment to multiple pathways, thereby crossing boundaries of thinking about the school-to-work transition and revealing new possibilities for practice. Participant 2 and Participant 6, both NGO leaders, stated the following:We used to be employment-focused, but we overlooked that they [NEY] might want to go back to school or try different pathways. (Participant 2).My colleagues all felt that the HKBM (Community) inspired us to envision multiple pathways to employment.... It has expanded our horizons and guided us to try out more possibilities. (Participant 6).4.2.2.The qualitative findings also revealed a youth-focused perspective in the HKBM (Community) Pilot Program. In other words, NEY were generally seen as active agents who were able to develop themselves instead of passive recipients of youth services provided by NGOs. Beyond that, NEY enjoyed a more egalitarian relationship with social workers and parents.4.2.2.1.The NEY participants generally praised the pilot program for enabling them to explore their interests, develop better self-understanding, and enhance their career competency. For example, Participant 21, an NEY who participated in various activities conducted by an NGO, remarked that his social worker has helped him explore his interests and get an internship:After learning about my interests, my social worker told me that a company would offer me an internship for a few months. (Participant 21).Often, youth encounter difficulties with identifying their strengths or articulating their values and skills. The pilot program facilitated youth in identifying and using their strengths while simultaneously motivating them to develop the skills that they need to succeed. Participant 16, an NEY participant in the pilot program, stated the following:I’ve accumulated work experiences here. I’ve also gotten subsidies. I’ve become more proactive in my career and life development and have learned different skills. (Participant 16).CV360 and the career fair have helped me understand my values and demonstrate my strengths. (Participant 16).The pilot program also allowed youth to interact with business stakeholders such as employers and employees, understand the various work environments and workplace standards, and pursue their CLD interests through experiential learning. With improved self-knowledge, NEY were more likely to stick with a clear career direction and obtain more stable employment experience. The following quotation from Participant 27, who provided NEY with encounters at a boxing gym, is a typical example:We started by providing them [NEY] with encounters at our gym. Afterward, they took internships, transitioned into part-time coaching, and eventually became full-time licensed coaches. We all felt that it was a meaningful journey. Their loyalty to our gym was greater than that of openly hired coaches. (Participant 27).Added to that, the participants appreciated the CLD tools provided to them, which could be used to scaffold opportunities for NEY to reflect positively on their workplace encounters and better understand themselves. For instance, NEY participants were guided to complete an online form called CV360® to generate their all-round profile . CV360® I have accumulated wonderful work experience.... CV360® has helped me to understand my values and myself and to demonstrate my strengths. (Participant 16).4.2.2.2.Nearly all participants agreed that the HKBM (Community) Pilot Program has made the relationship between NEY with social workers and parents far more equal than it used to be. A typical example is that NEY participants were treated as co-creators of activities in the process of participation. Instead of receiving the services passively, NEY were given a chance to design and implement CLD activities together with social workers. Both parties were pleased to see a more egalitarian relationship develop between them. Participant 8, a social worker who co-created activities with NEY, recalled:We asked about the young people’s career-related interests or their interests in different types of activities in advance. Then we organized the activities based on their interests. It wasn’t only about providing services. It was also about the relationship between social workers and service users, which has become more egalitarian. (Participant 8).In addition, Participant 17, an NEY participant who designed and implemented an activity titled “CeleBaby” for young mothers, shared how she has enjoyed an egalitarian relationship with social workers while co-creating the event:CeleBaby is a co-creation with social workers. I’m involved in the whole process, from initial planning to designing and organizing activities and preparing supplies. (Participant 17).Beyond that, parent–child relationships also improved. Parents reported that they learned how to understand their children’s needs from their perspectives and without imposing their expectations on their children. As a result, they were more willing to listen to NEY and provide more care and support according to their needs. As Participant 23, who supported her daughter through her school-to-work transition, indicated:[From the pilot program] I learned that when my daughter encounters difficulties, I should talk to her with an open mind. Parents need to listen to them [NEY] patiently and give advice as friends.... As a mother, all I can do is support and care for my daughter from her point of view. (Participant 23).It is worth noting that parents in Hong Kong envision that their children will be admitted to a university or further their education after finishing high school and believe that doing so is the only path that their children should pursue. For that reason, most youth in Hong Kong, in order to please their parents, view success as getting into higher education only. Youth unable to pursue higher education thus see themselves as failures and have no sense of direction for their careers. At the same time, their parents become disheartened, which causes the parent–child relationships to deteriorate. Under the influence of this unique sociocultural background, most young people in Hong Kong tend to conceive their career prospects too narrowly . Against4.2.3.As a pilot program, the HKBM (Community) Pilot Program presents stakeholders with challenges and barriers in its full application. Qualitative findings revealed such challenges and barriers faced by NEY, NGO leaders, social workers, and parents as they participated in the program.4.2.3.1.NGO leaders and social workers mentioned struggling to grasp the entire HKBM (Community) framework at an early stage, which hindered their implementation of the approach. For example, they were confused about whether they should achieve the 10 benchmarks simultaneously or in stepwise fashion. Fortunately, after the BM Facilitators clarified the rationale of the HKBM (Community), NGO leaders and social workers understood that they could formulate and implement actions to achieve benchmarks based on their organizational needs and pace. As one NGO leader recalled:Initially, we did not have a clear idea of the HKBM (Community).... We were confused about whether we had to achieve so many benchmarks simultaneously. (Participant 2).4.2.3.2.Although NEY participants appreciated the provision of encounters with the workplace, they found that the available industries were limited in type, which hindered them from exploring other career pathways. Therefore, the participating NEY hoped that the NGOs would cover more industries in the future, even marginalized and/or unpopular ones. Participant 16, for example, was impressed by a career-related workshop in which the organizer invited people from both the beauty and the funeral industries to share their work experiences. To him, creativity in workplace exposure mattered. He went on to say the following:There have been very different industries in the Hong Kong market. Maybe they [organizers] could invite professionals from different industries to talk about their career experiences in order to enrich or refresh our understanding of existing careers. (Participant 16).4.2.3.3.via their children, primarily because the information sharing between organizers and parents has not been well established. In response, a parent in the focus group stated:Parents expressed concerns with their ability to access CLD-related information. According to the HKBM (Community), parental support was a key component of making quality CLD services for NEY. However, the participants said that most parents could only receive CLD-related information I wonder whether CLD-related information can be spread to different groups through instant messaging applications. I think that would be more effective than the traditional promotion because we parents use mobile phones, too. (Participant 25).5.The HKBM (Community) Pilot Program was implemented as the world’s first CLD intervention program promoting a benchmark approach in communities. As a leading pilot program focusing on NEY’s problems in Hong Kong, where the unemployment rate among youth far exceeds the rate in the general population , the proThe analysis revealed that the development of YCDC differed significantly for the pilot group compared with the comparison group, thereby indicating that the HKBM (Community) Pilot Program had a positive effect on the improvement of NEY’s career development competency. In particular, the results show that increases in engagement, self-understanding, career and pathway exploration, and planning and career management were significantly different between the pilot and comparison groups over time. Those findings further demonstrate the positive effects of the HKBM (Community) Pilot Program on NEY and are consistent with past results showing the effectiveness of youth career intervention programs e.g., . HoweverThe success of the HKBM (Community) Pilot Program in cultivating NEY with a wide range of career-related competencies might be due to its emphasis on active participation, providing exposure to the world of work, and helping NEY to connect with supportive others  in the community. According to the eighth benchmark, “Meaningful encounters with the workplace” (see Appendix), for example, the six participating NGOs offered NEY opportunities for workplace visits and internship activities through which NEY were encouraged to interact with business stakeholders , experience real-world work environments, and learn workplace principles by active participation. Such meaningful encounters can provide important enabling environments for NEY to change their state of inactivity and lack of ambition. Moreover, because the HKBM (Community) Pilot Program managed to foster egalitarian youth–adult relationships, NEY may sense that are viewed as active social actors with capacities to handle various challenges in life. As a result, they might be more willing to take the initiative to engage in CLD-related activities and consequently develop different competencies needed for a smooth school-to-work transition. Although our evaluation did not address whether NEY ended up in meaningful education, training, or employment, the preliminary outcomes of the pilot program could be taken as promising signs that NEY were approaching productive pathways toward their desired career and life goals.The analysis also revealed that the development of CLDH differed significantly for the pilot group compared with the comparison group, thereby indicating that the HKBM (Community) Pilot Program had a positive effect on the enhancement of NEY’s sense of hope for their career and life journeys. To be specific, our results show that the development of the two dimensions of CLDH  was significantly different for the pilot group compared with the comparison group over time. That finding suggests that NEY who received CLD services in the HKBM (Community) Pilot Program may achieve a higher level of agency to initiate and sustain the motivation to pursue their desired goals and develop higher-level cognitive and behavioral abilities to generate multiple pathways to achieve goals, which together contributed to a more generic, positive expectation of goal attainment, namely CLDH. Previous studies evaluating youth career intervention programs have often shown young people’s improvement in emotional and psychological aspects e.g.,  but leftA possible reason for that effect was that the HKBM (Community) Pilot Program broadened the scope of CLD services from an employment-focused orientation to a multiple-pathway orientation, which enabled NEY to break the traditional framework of thinking about the school-to-work transition and thus rekindled their sense of hope about future career and life journeys. Today, career intervention theorists tend to consider the school-to-work transition to be a lifelong process within which multiple forms of transitions can occur e.g., , and thaIn general, the qualitative findings show the positive impacts of the HKBM (Community) on the ecosystem of CLD service provision. According to social workers and NGO leaders, having the HKBM (Community) helped them to review and enhance the quality of CLD services with a deeper understanding of what good CLD services look like. According to the social workers and NGO leaders, having the HKBM (Community) helped them to review and enhance the quality of CLD services with a deeper understanding of what good CLD services look like. In the pilot program, social workers do not work alone. On the contrary, they collaborate closely with other stakeholders, including parents and employers, to assist NEY in making informed career choices and accompany them in school-to-work transitions. At the same time, parents and employers receive support from social workers in the form of information about helping NEY to explore their multiple career pathways, unlock their full potential, and assist them in developing positive attitudes and values. In other words, the pilot program fosters cross-sectoral collaboration between social workers, parents, and business partners, which is supported by well-informed supportive measures and tools  to deliver CLD interventions.However, the qualitative findings also showcase several areas for improvement. First, social workers and NGO leaders initially reported difficulties in comprehending the HKBM (Community). Although the BM Facilitators responded quickly to clarify their confusion, the operational mechanism could be improved to support workers in using the HKBM (Community) more efficiently. One possible means to that end is to integrate the insights and experiences from the pilot program into different supportive measures. For example, participating NGOs in our pilot program have consolidated their good CLD practices into a tool kit with practical guidelines to facilitate the effective implementation of the HKBM (Community) among social workers . In addiSecond, NEY emphasized that encounters with the workplace provided by participating NGOs were limited in type and prevented them from fully exploring their career interests and finding their fit with specific jobs. Their emphasis on career interests and an in-depth understanding of the business world was consistent with what 6.Several limitations of our findings should be noted. First, although all of the interaction effects of time and group were significant, the effect sizes only reached small to medium levels . The smaDespite those limitations, our findings contribute to the practice and the theorization of the community-based benchmark approach. The HKBM (Community) Pilot Program is pioneering in implementing the benchmark approach through community-based NGOs to deal with NEY-related issues in Hong Kong. By exhibiting the positive impacts of the HKBM (Community) Pilot Program on NEY using rigorous research methods, our study can be used as solid evidence for advocating the expansion of the existing program in order to engage more NEY from diverse backgrounds. In addition, by investigating pretest and posttest outcomes and introducing a comparison group, our work may encourage evidence-based practice among social workers and local NGO practitioners by applying the HKBM (Community), which has been empirically piloted and evaluated in our study, to provide good CLD interventions and continuously improve the quality of those interventions against global standards . FurtherThe raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Survey and Behavioral Research Ethics Committee of the Chinese University of Hong Kong. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.SN, C-kC, and Y-hN: conceptualization. SN, C-kC, QZ, L-mW, and Y-hN: methodology. SN, C-kC, QZ, LW, WL, and EY: formal analysis. SN, C-kC, QZ, and HL: investigation. SN: resources, supervision, and funding acquisition. SN, QZ, and WL: data curation. SN, C-kC, QZ, WL, and L-mW: writing—original draft preparation. SN, C-kC, QZ, and EY: writing—review and editing. SN, C-kC, LW, and EY: visualization. SN and H-yT: project administration. All authors have read and agreed to the published version of the manuscript. All authors contributed to the article and approved the submitted version.This project was supported by a grant from the Hong Kong Jockey Club Charities Trust (Grant number: 2020–0111-002).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This program project arises from the consternation of psychotherapists at the increase in self-injurious behaviors in the child and adolescent population.Currently, in consultation, we are seeing many cases that do not match the conditions  described by current classifications. These cases have a common feature: emotional dysregulation.- Reduce the discomfort and emotional pain of the patient.- Increase the patient’s skills.- Improve patient motivation.- Generalize to the natural environment.- Structure the environment.The program is directed to peoople between the ages of 12 and 18. They are divided into two groups: one from 12 to 15 years old and another from 16 to 18 years old.The groups are a maximum of 8 adolescents. Parents also participate.These are closed groups.The duration of each session is one hour or one hour and a half.A therapeutic contract is signed.We will use different scales to measure the evolution of the patients. The following scales will be passed at the beginning and at the end: DERS, EGD, DASS 21, GHQ-12.Dialectical dilemmas in families will be worked on. These results will be collected and compared with those at the end of the program.People with emotional dysregulation sometimes do not have the necessary skills to regulate emotions. With this program, we intend to carry out training in skills , structured in modules, in addition to relying on individual therapy.Given this increase in deregulated children and young people, we see ourselves in the need to train ourselves and address these cases from a different point of view.None Declared


Projet Réaffiliation Itinérance Santé Mentale (PRISM), a shelter-based mental health and rehabilitation program intended to provide individuals experiencing homelessness and severe mental illness with transition housing and to reconnect them with mental health and social services.To explore the housing trajectory, personal recovery, functional level, and quality of life of clients at discharge and 1 year after completing Housing status, psychiatric follow-up trajectory, personal recovery , functional level , and quality of life  were assessed at program entry, at program discharge and 1 year later.Of the 50 clients who participated in the study from May 31, 2018, to December 31, 2019, 43 completed the program. Of these, 76.7% were discharged to housing modalities and 78% were engaged with psychiatric follow-up at the program's end. Housing stability, defined as residing at the same permanent address since discharge, was achieved for 62.5% of participants at 1-year follow-up. Functional level and quality of life scores improved significantly both at discharge and at 1-year follow-up from baseline.Admission to PRISM helped clients secure long-term stable housing and appropriate psychiatric follow-up. Stable housing was maintained for most clients at 1-year follow-up, and they benefited from sustained functional and quality of life outcomes in long-term follow-up. Projet Réaffiliation Itinérance Santé Mentale (PRISM) is a shelter-based mental health service launched in Montreal, Canada, in 20132 as a novel approach to provide services to homeless individuals with mental illness by establishing partnerships between the city's shelters and public mental health and social service providers. There are currently three PRISMs in Montreal. Each PRISM occupies a designated area of a shelter and has a capacity of 8 to 16 individuals at a time. The aim of the program is to provide short-term residential services and on-site psychiatric care to help clients transition to permanent housing after 8 to 12 weeks and to connect them with long-term support services. On-site psychiatric and nursing care is available, and a psychoeducator/social worker as well as an intervention worker helps clients reach reintegration goals. Clients benefit from various forms of subsidized housing, a quick access to universal health coverage as well as disability income support. PRISM also refers participants to scattered-site Housing First providers. A more exhaustive report of PRISM and its services is described in Laliberté et al. revealed that, of 579 clients, 63% were housed after program discharge (52% to permanent housing and 11% to temporary housing), 21% were not, and 16% were transferred to a hospital or rehabilitation centre. Furthermore, 85% had received some form of outpatient or community service support at discharge. The study also showed that 57% of clients came directly from the street or shelters, while the rest were referred by health care resources . Both community and health care partners were informed about PRISM for them to refer clients most likely to benefit from the program. Second, in a qualitative study based on interviews conducted with 20 clients at the time of program admission and again at discharge, it emerged that 3 aspects of the program played a role in patient recovery: the community-based approach and its flexible structure; the possibility of taking “a break” in a secure environment that allowed focusing on one's physical and mental health; and the multiple modalities of treatment at PRISM adapted to each client, which contrasted with less desirable experiences in hospitals. Whether the benefits of PRISM are maintained in long-term follow-up after program completion remains to be investigated.The PRISM program has been the subject of two studies to date. First, a descriptive study conducted from November 18, 2013, to May 31, 2019, based on a chart review of 4 operating PRISM partnerships38 or other programs. PRISM can thus act somewhat like Critical Time Intervention,10 in that it serves to help a client connect to housing and services that will enable them to function successfully in the community.An important value of PRISM is to provide an 8- to 12-week period of stabilization in the shelter, during which the team can make a careful assessment of the needs and strengths of the clients, as well as their housing and service-related preferences. PRISM is then able to refer appropriate clients to Housing FirstThe aim of our study was 2-fold: first, to explore the housing trajectory of PRISM clients 1 year on from program completion and, second, to assess the short- and long-term benefits of program admission for client personal recovery, functional level, and quality of life (QoL). We hypothesized that participation in PRISM would lead to sustained stable housing outcomes at 1-year follow-up and both immediate and sustained improvement in quantitative questionnaire measures after program completion. functional level, and QoL.For our purposes, we undertook a prospective cohort study in which we followed a group of clients admitted to PRISM-WHM, the partnership with Welcome Hall Mission, an all-male shelter where 8 beds were reserved for the program. All clients were admitted to PRISM-WHM from a situation of homelessness, either directly from the street or a shelter or following admission to an affiliated McGill University health centre see . ClientsFrom May 31, 2018, to December 31, 2019, every individual admitted to PRISM-WHM was eligible to participate in the study. Of these 57 individuals, 50 consented to participate in the study see  after meClients met at 3 different times with a member of the research team (BV) for data collection see : first, International Standard Classification of Education. Substance use was divided into 3 categories based on self-reported data  to measure functional level, and the Quality of Life Inventory 20 (QOLI-20).17 Additionally, at T2, each client completed the Adverse Childhood Experiences (ACE) questionnaire and the Montreal Cognitive Assessment (MoCA). During the interview at T3, information was sought regarding service utilization, housing and hospitalization since T2. All questionnaires at all times were administered by BV, a research team member independent of the clinical team, who completed training in MCAS administration prior to patient recruitment.Three other questionnaires were completed at baseline (T1), T2, and T3: the Canadian Personal Recovery Outcome Measure (C-PROM),All clients were recruited and completed the intake interview (T1), the PRISM program, and the exit interview (T2) in person at the shelter prior to the introduction of public health restrictions on social contacts on March 13, 2020. Following this date, a portion of our interviews was conducted by phone (11 of 28 interviews), and compensation was paid by way of electronic bank transfer alternatively when mailbox deposits were not an option. and R. Descriptive statistics were computed for questionnaire scores (mean and standard deviation [SD]) at all 3-time points and for sociodemographic variables . The MCAS and QOLI scores . Clients who completed the program spent an average of 70.7 days at PRISM-WHM. Those who were contacted at T3 were reached on average 434.5 days after program completion. A majority (55.9%) scored 4 or higher on the ACE questionnaire, M  =  4.09, SD  =  2.4, which is indicative of a significant history of childhood trauma. Only 27.0% obtained scored 26 or higher on the MoCA, M  =  24.16, SD  =  2.5, the score below which mild cognitive impairment is indicated.Two-thirds of all clients were discharged from the program to stable housing. Of the 43 participants who completed the program at T2, 33 (76.7%) were discharged to stable housing. Housing data were available for 32 participants at T3: at 1-year follow-up, 62.5% of these described a stable housing situation, and 56.3% were living at the same address that they had been discharged to. By the end of the program at T2, 78% of the study's 50 participants had engaged in psychiatric follow-ups of some sort.p  =  0.001. There was also a significant improvement from T1 to T3 of 6.62 , p < 0.001, which suggested a lasting effect of these benefits 1 year after program discharge. For participants for whom we had data at T2 and T3, a significant increase in MCAS was also observed between those 2 time points, 3.35 , p  =  0.002.Our study showed an improvement in participant functional level in the time spent at PRISM-WHM, the results of which are outlined in p < 0.001, as did the social skills subscore by 1.02 , p  =  0.020, which indicated participants made progress in these domains during their stay at PRISM-WHM. Furthermore, adaptability and social skills continued to improve in the year after program discharge, as suggested by the increases in the subscores from T1 to T3 of 2.51  for adaptability, p < 0.001, and 2.32  for social skills, p < 0.001.The MCAS behaviour subscore improved by 1.01  from T1 to T2, p  =  0.006, as well as the finances domain of 0.83  between T1 and T2, p  =  0.015. Improvements emerged at 1-year follow-up in other domains, namely leisure by 0.61  from T1 to T3, p  =  0.019, and family by 0.58  from T2 to T3, p  =  0.026.While there was no significant change in the global QOLI score from T1 to T2, a change of 0.15 , there was a significant improvement in the living situation domain of 1.01  between T1 and T2, Changes observed in the C-PROM between T1 and T2 of −0.03  and between T1 and T3 of 0.74  were not significant.We performed simple linear regressions to characterize group differences in questionnaire scores between clients with stable housing at T3 and those without at T3. No analysis reached statistical significance for MCAS, QOLI, or C-PROM.We set out to assess the impact of program participation on housing, functional, QoL and personal recovery outcomes for PRISM clients, as well as on their capacity to reconnect with psychiatric services. This was the first study to evaluate these outcomes both at the time of program completion and at 1-year follow-up, thereby providing fresh insight into the program's effectiveness. who found, based on a chart review that 63% of program users transitioned to housing after discharge while 85% had psychiatric follow-up care immediately after program completion. Given that follow-up care in the month after psychiatric discharge can be as low as 49% for patients who need to be reassessed in that timeframe, our findings reinforce the notion that the period of stabilization that clients benefit from in PRISM translates into a better transition to stable housing and more adequate follow-up. Moreover, in qualitative interviews, PRISM clients reported that the “break” from street life was a significant boon, as it afforded time and space to take care of their physical and mental health. Furthermore, establishing an environment where clients are engaged with stable housing and psychiatric follow-up has been demonstrated to predict housing status maintenance in homeless populations suffering from mental illness.At T2, 43 of 50 (86%) clients completed the PRISM program and 33 of 50 (66%) or 33 of 43 (76.7%) program completers were discharged to housing. Additionally, 78% of program completers were engaged with psychiatric follow-up of varying intensity prior to program discharge. Our results are congruent with the findings of Laliberté et al., and are more likely eventually to need acute-care services.2527 Our figure (66%) for permanent housing status immediately following discharge is comparable to that reported in a previous PRISM study (63%). The finding that a significant proportion of clients who emerged from the program with stable housing were able to maintain it after a year is consistent with the functional and QoL improvement over time reflected by the questionnaire scores in our study. These results echo also findings regarding Housing First interventions29 to the effect that clients who obtained stable housing also presented functional improvements.At 1-year follow-up, most clients maintained positive housing outcomes, though not necessarily the same permanent address. Of the 32 clients we reached, 18 (56.2%) had maintained stable housing and another 11 (34%) were presently housed but had experienced instability since T2  or were in supervised housing or group programs. Together, these numbers are quite impressive for this population, seeing how all study participants were admitted to PRISM from a precarious living situation. It has been demonstrated that homeless individuals with mental health issues do poorly without housing interventions and are at higher risk of remaining unsheltered for longer periods of time In our study, program completion was associated with improved client functioning at T2. Overall MCAS scores increased from program entry to program completion, which suggests that the client overall functioning improved over that period. More specifically, client health and behaviour subscores increased significantly from T1 to T2. Lastly, client QOLI living situation subscores, too, increased significantly from T1 to T2. These findings are in line with those of a previously published qualitative paper involving the same study population, where clients found that the PRISM program afforded an opportunity to engage with a variety of services without experiencing the taxing burden of life on the street. These observed changes over the course of their stay in the program may be indicative of functional growth. The fact that the program incorporates flexible therapeutic modalities within a stable structure provides clients with a platform that enables them to access permanent housing and psychiatric follow-up more readily. These gains may be reinforced to some extent by the presence of various multidisciplinary services directly in the PRISM environment, given this population's usual poor level of engagement with such services.32 The proximity of clients to these resources is maximized while they are completing the program, thus increasing the effectiveness of the mental health care and accessory services which are known to benefit clients.Evidently, the benefits of the PRISM program for clients who achieve stable housing include improved functioning and QoL. This is in line with the results of other studies that showed that housing interventions improved client confidence in their living situation and led to an increase in overall well-being.35 In that study, nonetheless, narrative interviews revealed meaningful differences in life trajectory during the study between the 2 groups. It may be that measures such as the RAS and C-PROM are inadequate to capture the real benefits that clients with a significant burden of life adversity experience. Other ways of measuring progress on what Deborah Padgett has called “complex recovery” may need to be found.Considering participants we were able to follow, functioning level and QoL outcomes were maintained at follow-up a year after program completion and even improved in the case of the MCAS total score, as well as the adaptability and social skills subscores. The finding that PRISM, an in-shelter program targeting homeless individuals with severe mental illness who engage with services after numerous unsuccessful interventions, can result in positive sustained changes in their functioning is promising and underscores the program's potential to enhance client long-term functional outcomes and improve their care trajectories. In contrast, C-PROM questionnaire scores remained stable over time. This result resembles that found in the At Home/Chez Soi study, where 2 years of recovery-oriented Housing First were not sufficient to achieve improvements on a different measure of recovery, the Recovery Assessment Scale (RAS). and even more so when it comes to reaching individuals experiencing both homelessness and severe mental illness and who often lack stability. Ultimately, this invites caution in the interpretation of our results.The principal limitation of our study derives from the fact that we lost participants at various follow-up points, especially from T2 to T3, and their outcomes are unknown. While the extent of the attrition made direct imputation of missing values unadvisable, as a sensitivity analysis we undertook a weighted analysis to account for the significant degree of attrition, and this did not materially alter the results. It is worth noting that considerable effort was made to reach clients at T2 and T3, including contacting backup contacts and members of care teams, and it is unlikely that more clients would have been reached had any particular study parameter been changed. Such limitations at follow-up are not unusual in long-term cohort studies of marginalized populations,In addition, while all participants were recruited prior to the onset of the COVID-19 pandemic, new public health measures were imposed in shelters during the study. Though this forced us to make minimal changes to the study protocol, no program client reported any material obstacle to participation. Still, the measures could have had an impact on some of the services offered in the shelter and on participant follow-up.PRISM is a shelter-based mental health service intended to help clients transition from acute-care settings and homelessness to sustained stable housing and appropriate psychiatric follow-up. This study demonstrated that benefits observed at program completion and positive functional and housing outcomes are maintained for the majority of clients over long-term follow-up. In a complementary study, we plan to analyse the data in provincial administrative databases for a larger pool of PRISM clients. This will allow us to examine participant characteristics more in-depth and to track housing and service use trajectories more closely. We will also conduct qualitative interviews with clients 1 year after program completion. Vulnerable homeless people with mental illness need significant support, and PRISM creates the appropriate environment and multidisciplinary service hub where clients can start or continue on the road to recovery.


There is an increased call for studies analyzing how implementation quality influences Social and Emotional Learning (SEL) program effectiveness.Mage = 12.41, S.D. = 1.06), from 36 classrooms , distributed between the control group (179 students), and three intervention groups (643 students) that reflected low, middle, and high implementer experience . Dosage and fidelity , gender, and classroom size  were also analyzed. Self-report questionnaires were administered pre- and post-intervention and at a 6-month follow-up.The current dissemination study analyzed the effectiveness of the Positive Attitude Upper Middle School SEL program on a Portuguese nationwide sample composed of 813 middle school students (7th and 8th grade; 51.7% boys; Multilevel models were employed, and results showed that participating in the PAUMS SEL program led to more positive trajectories in self-control, social awareness, relationship skills, and responsible decision-making when compared with control groups. Regarding implementation quality, only the implementer’s experience impacted the effectiveness of the PAUMS SEL program; students in the Gulbenkian Academies of Knowledge intervention group displayed a less positive trajectory in self-control than students in the Positive Attitude Torres Vedras intervention group.Altogether, results showed that the PAUMS SEL program is ready for dissemination in Portugal, although a higher level of implementer experience is needed to achieve the best effectiveness, and they support the importance of analyzing implementer experience in SEL programs’ effectiveness studies. The effectiveness of universal Social and Emotional Learning (SEL) programs has been well-established in several studies, including several meta-analyses . HoweverIn the literature, strong evidence supports that program success is moderated by implementation quality , which cAnother remaining issue in the literature is that a vast majority (87%) of effectiveness-based trials have been conducted in the United States of America . TherefoThe Collaborative for Academic, Social, and Emotional Learning (CASEL) defines Social and Emotional Learning (SEL) as ‘the process through which all young people and adults acquire and apply the knowledge, skills, and attitudes to develop healthy identities, manage emotions and achieve personal and collective goals, feel and show empathy for others, establish and maintain supportive relationships, and make responsible and caring decisions.’ . AccordiSeveral meta-analyses of SEL programs have provided robust evidence for the efficacy of SEL interventions . ConsistHowever, another common feature in these three meta-analyses was that the SEL programs analyzed were mainly applied in elementary schools and, in Portugal, middle school  is a period of upheavals and challenges for students’ social and emotional competencies that are associated with a decrease in the positivity of school climate perceptions . For exaThe Positive Attitude project has implemented the PAUMS SEL program (7th to 9th grades) since 2004. In the last 3 years, it has been implemented in two Lisbon municipalities . The PAUMS SEL program was designed to enhance children’s social and emotional competencies using the theoretical framework proposed by the Collaborative for Academic Social and Emotional Learning (CASEL) in 2005. So, the PAUMS SEL program is classroom-based , delivered weekly by an educational psychologist (in the presence of the head teacher), and integrated into the school curriculum. The current version of the program is delivered in 13 one-hour weekly sessions , according to each class profile. It follows a program manual, which contains detailed plans for each session. More information about the program can be found in The program’s efficacy has been analyzed in several previous publications , 2018. BAlthough many SEL programs have strong empirical support , many scAmong individual and classroom level variables, gender and classroom size have been identified by several authors as influencing SEL programs’ effectiveness . Gender Therefore, for the current study, we consider how a set of individual and classroom variables may influence the effectiveness of the PAUMS SEL program. Namely, implementation quality variables, gender, and classroom size were analyzed in the nationwide dissemination of the program.According to several authors , implemeAccording to Dosage is the level of exposure to an intervention, and it is widely accepted as being highly influential . MoreoveAuthors agree that among the components that influence program success we should include not only program’s characteristics, but also implementers’ characteristics, including previous program experience and their attitudes toward it . AccordiThere is no consensus in the literature regarding the differential gender effects of participating in SEL programs. While some studies  found noClassrooms are social settings where students are involuntary members and where they spend most of their time, interacting with other students, daily . FurtherDespite several meta-analyses supporting the effectiveness of school Social and Emotional Learning programs , severalTherefore, the current study had two main aims: first, to analyze the effectiveness of the PAUMS SEL in a dissemination study that used a nationwide sample under the GAK; second, to analyze the role of several elements of implementation quality and establish how they may influence the effectiveness of the PAUMS SEL program in the aforementioned nationwide replication of the program. For the first aim, we formulated the following three hypotheses—the PAUMS SEL program is effective in a nationwide sample (H1). Also, given previous program results , we alsoThe students who participated in this study were part of wave one (2019/2020 school year) of a nationwide dissemination initiative of the PAUMS SEL program. This initiative was part of the Gulbenkian Academies of Knowledge (GAK) program, which sponsored programs considered blueprints in Portugal. The sample was originally composed of 1,451 middle school (7th – 8th grade) students, who attended 15 Portuguese public middle schools, both in the continent  and in the Madeira Archipelago. The sample followed national population estimates for each region; 362 (24.9%) students were from the North region (Vizela); 388 students (26.7%) from the Center region ; 587 students (40.5%) from the Lisbon and Tagus Valey region , 71 students (4.9%) from the Algarve (Faro and Loulé), and 43 students (3.0%) were from Archipelago of Madeira.n = 638). Furthermore, there were sources of attrition other than the COVID-19 pandemic. Although the program was implemented as a part of a mandatory school subject dedicated to citizenship, 12 parents opted out of the assessments in the classrooms assessed. Therefore, the final sample was composed of 813 middle school (7th–8th grade) students, from 36 classrooms , 51.7% of which were boys (n = 420) and 47.9% girls (n = 390), the remaining (0.4%) classified themselves as “other” or opted not to answer. Participants’ age ranged from 12 to 16, with a mean age of 12.41 (SD = 1.06). Regarding school grade distribution, 453 were 7th graders and 360 were 8th graders. Six-hundred-thirty-four students participated in the PAUMS SEL program Attitude (78%) and 179 were in the control group condition (22.0%). Regarding the modality of program implementation, 220 students (27.1%) participated in Positive Attitude Torres Vedras, 115 (14.1%) participated in Positive Attitude Cadaval, and 299 (36.8%) were part of the GAK group. Classrooms varied in size, with the total number of students per class ranging from 13 to 28. Schools displayed a wide range of socioeconomic, between 24.4 and 60.4% of students eligible to receive free or reduced school meals . However, most schools had a relatively low level of students ethnic from minority backgrounds, from 2 and 18% . Further information about students is displayed in However, due to the lockdown implemented in Portugal due to the COVID-19 pandemic, it was not possible to finish the programs’ implementation in 29 classes  students completed the first assessment, 806 (96.0%) completed the second assessment, and 766 (91.2%) completed the third assessment. Students who did not complete an assessment had either moved to another school or were absent from school on the days of the assessments and could not be reached during the following week.The Social and Emotional Competences Evaluation Questionnaire QACSE;  was usedα = 0.73, 0.80 in the present study). The social awareness subscale evaluates the ability to understand other people, empathy, compassion, and social norms, and it is also composed of seven items . The relationship skills subscale assesses the capacity of initiating and maintaining positive interpersonal relationships, and the level of communication skills. It is composed of seven items . Finally, the responsible decision-making subscale measures the level of reflexive consideration when facing different choices, where the student must consider his and others’ well-being. It is composed of four items . The questionnaire’s reliability, validity, and factor structure have been validated in three different studies , which aimed to develop social and emotional competencies in children and youth, by disseminating blueprint Portuguese interventions. The PAUMS SEL program was one of the blueprint programs chosen for replication. After a two-stage selection process, six academies across Portugal were established in seven different municipalities: one in the North Region (Vizela), one in the Center Region , two in the Lisbon and Tagus Valley region , one in Algarve  and another one in the Archipelago of Madeira .To better support the GAK in the implementation of the PAUMS SEL program, the Positive Attitude team developed a package that included training, monitoring, and supervision, a standardized manual for PAUMS SEL program, and an online platform . The traLusíada University – North) under the project CIPD/2122/DSE/2 and it was conducted following the national professional code of ethics for psychologists.The educational psychologists who implemented the program were present in the meetings in back-to-school meetings (mandatory for parents), to explain the program and answer questions. All schools used passive informed consent because the program was considered part of the school curriculum, following national legislation. School boards (not the implementers nor the Academies personnel) assigned classrooms to the intervention and control groups. Parents had the choice to withdraw their children from the assessments, and therefore the data for those students was not collected. The study was approved by the Psychology for Positive Development Research Center (n = 49).Self-reports were filled at baseline, posttest, and six-month follow-up, while demographic data were recorded at the pretest. In the intervention group, questionnaires were administered in the first and last sessions of the program, and control groups were assessed in the same period, but proceeded with business as usual and, therefore, did not receive any social and emotional learning training. All intervention groups implemented the same curriculum and test applications for both groups were carried out under the same conditions, with the psychologist responsible for each class reading questionnaire instructions out loud to the students and the students responding on an online platform either using a computer provided by the school or a tablet provided by the project, which resulted in no missing data at the individual level. If a student was not present during the evaluation the questionnaires were administered in another class within 1 week , whereas, in the final models a series of cross-level interactions terms were specified using dummy coding to test hypothesis one, these cross-level interactions included Condition*TimeLinear, Condition*TimeQuadratic, and Classroomsize*Time. To assess hypothesis two three-way cross-level interactions were created; Gender*Condition*TimeLinear. To assess hypothesis three another three-way cross-level interactions were created; Classroomsize*Condition*TimeLinear.Students from the same class have a much higher probability of providing highly correlated responses . So, giv*TimeLinear, Fidelity*TimeLinear, Classroomsize*TimeLinear, Modality*TimeLinear, Modality*TimeQuadratic . These models are available in the To test research hypotheses four to six, the same steps were until the final models except that in Model 2 dosage was entered as a grand-mean centered individual variable, and in Model 3 fidelity was entered (grand-mean centered), whereas FRSM and ethnicity were removed. Additionally, in model 3 modality was entered instead of condition. Modality was dummy-coded , and all comparisons were made relative to the PA Torres Vedras group. In the final models, a series of cross-level interactions terms were specified using dummy coding including DosageThe descriptive statistics for the social and emotional competencies for every timepoint are displayed in Regarding implementation fidelity, the implementers reported a high degree of implementing the program as conceived. Although implementation fidelity varied from 70 to 100%, most classrooms reported a very high level of implementation fidelity; out of 28 classrooms, 16 (57.1%) reported 100% of implementation fidelity, and four reported 95%.There was also little variation in dosage, 73.1% of the students reported having been present in 12 to 14 sessions, which means that most students were either present in all sessions or missed one session. Only 5% of the students attended 11 or fewer sessions and 21.4% took part in an implementation where several extra sessions were needed to deal with all the material in the lesson plans (from 15 to 18 sessions).The first aim of the current was to analyze the effectiveness of the PAUMS SEL program in a dissemination study that used a nationwide sample. Under that aim, the first hypothesis was formulated to assess program effectiveness while controlling for socioeconomic status , ethnicity, and classroom size. The results of these analyses are displayed in β = −0.83, SE = 0.37; z = −2.23, p = 0.026) and a statistically significant decrease in responsible decision-making  that also accelerated . Gender was also a statistically significant predictor of self-control , social awareness , and responsible decision-making , with girls displaying higher levels than boys in these three social and emotional competencies.After adjusting for all individual and class-level variables, as well as cross-level interactions, linear time was a statistically significant predictor of social awareness and responsible decision-making, whereas quadratic time was a statistically significant predictor for responsible decision-making. After adjusting for all other variables, during the analyzed period, students displayed a statistically significant constant decrease in social awareness  and responsible decision-making . There were no statistically significant differences between students from schools according to their level ethnically diversity for any social and emotional competencies. Students from larger classrooms displayed higher levels of social awareness  and relationship skills .Regarding classroom level variables, students from schools where there was a higher percentage of students receiving free or reduced school meals reported statistically significantly lower scores in self-control  and responsible decision-making . For these competencies, the more positive evolution of intervention groups when compared with control groups decelerated between time points two and three.To analyze hypothesis one, cross-level interactions between condition and linear time, and condition and quadratic time were included. There were statistically significant results in all four social and emotional competencies for linear time; the intervention group displayed a more positive evolution during the period analyzed than the control group; self-control, β = 0.49, SE = 0.18; z = 2.30, p = 0.006). Therefore, boys who participated in the program showed a more positive trajectory in self-control than girls. Furthermore, the cross-level interaction between classroom size and time linear introduced to analyze hypothesis three yielded no statistically significant result for any of the analyzed variables.As seen in The second aim of the current study focused on the analysis of the influence of several elements of implementation quality on the effectiveness of the PAUMS SEL program. Accordingly, each of the three hypotheses posed was related to a dimension of implementation quality: fidelity (H3), dosage (H4), and implementers’ experience (H5). To test these hypotheses a series of cross-level interactions were added to the final models, which are displayed in β = −0.79, SE = 0.34; z = 2.30, p = 0.021. It should also be mentioned that there was no statistically significant difference in gains between students in groups where implementers’ experience was higher (PA Torres Vedras), and students in groups where implementers’ experience was medium .Neither hypothesis three nor hypothesis four were supported by the results of the current study because different levels of fidelity and dosage did not lead to statistically significant results in any of the analyzed social and emotional competencies. On the other hand, hypothesis five was partially supported by the results given that the students in the groups where the implementers’ experience was higher (PA Torres Vedras) displayed a more positive trajectory in self-control than the group where the implementers’ experience was lower (GAK), The current study had two primary aims. First, it analyzed the effectiveness of the PAUMS SEL program in a dissemination study that used a nationwide sample under the GAK initiative. Second, it analyzed the role of several elements of implementation quality and assessed how they influenced the effectiveness of the PAUMS SEL program. For these purposes, we formulated six hypotheses.We first hypothesized that the PAUMS SEL would be effective in a nationwide sample. We found that the program was effectively leading to better trajectories over time for the intervention groups when compared with the control groups in all four socioemotional competencies. The results are aligned with previous findings , 2017, wWe also analyzed if gender and classroom size impacted the program’s effectiveness (H2 and H3). Although program effectiveness did not differ by classroom size (thus negating hypothesis three), there was one statistical significance result due to gender; boys benefited more than girls in self-control from participating in the PAUMS SEL program. The current study’s results were not aligned with previous literature e.g., , which fThe remaining hypotheses were related to the second aim of the study because they focused on differential effectiveness—that is, which implementation quality variables influenced the effectiveness of the PAUMS SEL program in this nationwide replication. The fourth hypothesis (H4) predicted that the program’s effectiveness would be greater if its implementation fidelity was higher. However, the results did not support H4; the PAUMS SEL program’s effectiveness did not vary according to the degree of implementation fidelity. Although fidelity is recognized as an important feature in the successful delivery of SEL programs, influencing their effectiveness and outcomes , these rThe current study also assessed if the PAUMS SEL program’s effectiveness was greater when the implementation dosage was closer to the number of sessions prescribed in the manual (H5). In the current study, the level of exposure to the program was operationalized as the number of sessions in which the students were present. This hypothesis was supported by the results, as there were no significant differences in the PAUMS SEL program’s effectiveness due to higher or lower dosages than the ones prescribed in the manual. In other words, maximum effectiveness was achieved under the number of sessions prescribed in the program manual. This is consistent with The final hypothesis (H6) regarding implementation quality focused on the implementers’ experience. This hypothesis proposed that there was greater effectiveness of the PAUMS SEL program if the implementers’ experience was higher. The reports partially supported this hypothesis as they showed greater effectiveness of the PAUMS SEL program on self-control in the modality where the implementers’ experience was higher (PA Torres Vedras) than in the modality where the implementers’ experience was lower (GAK). However, there were no differences in effectiveness between the modalities where the implementers’ experience was higher (PA Torres Vedras) and the implementers’ experience was average .Altogether, although the current results support that implementers’ experience may influence the effectiveness of an SEL program, they do not support previous findings by In sum, the current study supported the effectiveness of the PAUMS SEL program with a nationwide sample, providing additional evidence of its effectiveness on a widespread scale. Additionally, the program yielded similar results for both genders in three of the four social and emotional competencies assessed, which underlines the universal nature of the program. However, students participating in the program from larger classrooms gained more self-control and responsible decision-making, thus highlighting the relevance of classroom size as a relevant variable in effectiveness studies. Furthermore, the current study supported the relevance of assessing implementation quality indicators. Although there were no differences in results between levels of implementation fidelity to the PAUMS SEL program, maximum effectiveness was achieved using the recommended dosage. Additionally, there was some support for the assumption that implementers with the highest experience level achieve more positive results.The current study supported the conclusion that the PAUMS SEL program is ready to be disseminated in Portugal. Although a previous meta-analysis  found thThe potential implications of the current study must be evaluated considering its limitations. A first limitation was that several implementation groups were not concluded due to confinement related to the COVID-19 pandemic, which disrupted normal schooling. Therefore, it was not possible to include almost a third of the groups that initiated the PAUMS SEL program in the current study. This circumstance could have impacted implementation quality, which according to In the current study, some independent variables that were analyzed were not possible to manipulate, such as dosage and fidelity. These variables were only assessed through implementers’ self-ratings and these depend on the accuracy of the implementers and their perspectives. An implementer may report that an activity was implemented, however, that does not provide information regarding how precise was the delivery and how much of the goals were achieved. Consequently, the variance found was relatively low, which allowed for very few conclusions to be drawn. It would have been adequate to complement the self-ratings with observations by the program developers, following Finally, implementer experience may have been confounded with access to program developers, because PA Torres Vedras and PA Cadaval implementers were in direct contact with the program developers in weekly meetings, whereas GAK implementers, after the initial training, only had two full days of supervision visits annually.Several authors  considerFurthermore, The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by the Center for Research in Psychology for Positive Development, Lusíada University. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.VC conceived the study and its design, drafted the manuscript, and performed the statistical analysis. MM and PB conceived the study, drafted the manuscript, participated in the interpretation of the data and its implications. All authors read and approved the final manuscript.The work was supported by the Fundação Calouste Gulbenkian who financed the open access publication fees, included in their Gulbenkian Academies of Knowledge initiative.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.