International partners reviewed the toolkit. Cognitive debriefing data were analyzed, revisions and new tools were integrated, and the final version was approved.twenty-two articles were identified and mapped as resources. The Model for Improvement, a data-driven approach to performance analysis, was selected for its widespread use and simplicity in carrying out the following steps: 1) organize a team, 2) assess improvement need regarding universal health and primary health care education, 3) set an aim/goal and identify priorities using a matrix, 4) establish metrics, 5) identify change, 6) carry out a series of Plan-Do-Study-Act learning cycles, and 7) sustain change.the Education Quality Improvement Toolkit, developed through stakeholder consensus, provides a systematic, and potentially culturally adaptable approach to improve student, faculty, and program areas associated with universal health coverage and access. Although various program strengths were identified, this collaborative work identified a need for ongoing monitoring, evaluation, reporting and development of quality improvement (QI) plans, to ensure graduate preparedness in PHC and UH. Recommendations included: 1) strengthening knowledge of integrating information technologies (IT) into healthcare, environmental and global health, emergency and disaster preparedness, complex and systemic thinking, problem solving and evidence-based care; 2) expanding use of clinical simulation and training experiences in PHC settings; and 3) adopting the principles of interprofessional education (IPE) using practical team-based experiences that reflect country specific healthcare contexts and priorities around PHC and UH.In 2016, the Pan American Health Organization (PAHO), in collaboration with three World Health Organization Collaborating Centers (WHOCCs) in Nursing and Midwifery, and the Latin American Association of Nursing Schools and Faculty (ALADEFE), conducted an “Universal Health and Primary Health Care: a Plan for Nursing and Midwifery Education Quality Improvement.” This plan included an education quality improvement (EQI) Toolkit to assist nursing and midwifery programs in evaluating and improving structure, process, and outcomes within the framework of transformative education and competency-based interprofessional collaborative practice (IPCP). The overarching goal for this international multi-centric partnership study was to improve UH and PHC in LAC countries. In this paper, we will describe the development, evaluation by partners, and revision of the EQI Toolkit aimed to guide QI activities in nursing and midwifery education.In the footsteps of this study, the PAHO/WHOCC in International Nursing at the University of Alabama at Birmingham (UAB) was tasked to develop (-). Transformative education encompasses principles of critical thinking, team work, creative adaptation, integration of education into health systems, resource sharing, networking and partnerships(-). Interprofessional education, where students learn from each other outside traditional disciplinary silos, is a pre-requisite to transformative education. Preparing the future healthcare workforce for IPCP, different disciplines working effectively together in teams with patients, families, caregivers, and communities, is a prerequisite to delivering the highest quality care. The High-Level Commission on Health Employment and Economic Growth, established by the United Nations (UN), WHO, and other agencies, recommends scaling up transformative and lifelong learning for the healthcare workforce. To move forward the UN 2030 Sustainable Development Goals (SDGs) agenda, nurses and midwives must be prepared to practice and thrive in an ever-changing environment, assessing evidence and working collaboratively with other healthcare professionals to meet the needs of diverse populations.In the era of a global economy, sweeping technological advances, growing disparities and increased focus on sustainability, higher education for health professionals is rapidly evolving. Pathways for transforming higher education to strengthen health systems in an interdependent and interprofessional world have been the focus of two simultaneous landmark reports-3. Trnerships-3. Inorkforce. To mov. Universal health coverage (UHC) is a financial protection arrangement for health for all. It is often achieved through some form of national health insurance program to serve the needs of the population. Closely tied to both concepts, PHC aims to achieve better health for all through reform in universal coverage, service delivery, public policy, and leadership. Organizing health services around peoples’ needs and expectations, and increasing stakeholder participation are two key PHC mandates. Recently, the potential contribution of nurses and midwives towards achieving UH has been a focus of the Nursing Now Campaign. Based on the premise that investment in nursing will enable nurses to achieve their full potential, nursing and midwifery education programs are urged to prepare graduates in promoting UH and PHC.Universal health (UH) access and coverage are two of the main indicators for ensuring healthy lives and promoting well-being for all (SDG #3). Defined as “the absence of geographical, economic, sociocultural, organizational, or gender barriers” universal access to health is achieved through “the progressive elimination of barriers that prevent all people from having equitable use of comprehensive health services determined at the national level”. Univerpulation. Closeladership. OrganiCampaign. Based “Midwifery Educator Core Competencies” and the “Nurse Educator Core Competencies” were developed in response to the World Health Assembly resolutions upon consultation with key partners. Both documents aim to support and guide educational institutions, worldwide, in modifying their competencies-based curricula within each country’s parameters, including diversity and availability of resources. Monitoring, assessment and evaluation of students and programs are listed as educator competencies for both nurses and midwives(-) linking core competencies to the ongoing debate about improving educational outcomes in nursing and midwifery schools.In order to prepare competent and skillful entry-level nurses and midwives in providing PHC and UH, educators must meet a minimum of core competencies. The encies” and theencies” were demidwives-10 li. Variation in levels of educational preparation affect workforce capacity, in both numbers and skill mix, which ultimately influences care quality. For example, some Eastern Caribbean countries have more than 40 professional nurses per 10,000 population, whereas most of the Spanish-speaking countries in the region have less than 10 per 10,000.Entry-level nursing and midwifery education in LAC countries reflects regional disparities in terms of geography, politics, economy, and culture. Variatr 10,000.. Midwifery can be practiced by obstetric nurses, nurse-midwives, professional midwives, and in some countries, by traditional midwives, with education ranging from lay midwifery to undergraduate and/or post-graduate preparation. For countries with low numbers of skilled birth attendants and high maternal mortality rates, it has been recommended that Ministries of Health consider professional midwifery as a key to improving maternal and perinatal health. Therefore, strategies to strengthen the quality of nursing and midwifery education are vital for establishing a competent workforce and must be set in the context of each country.Similarly, teaching and practicing midwifery varies greatly, with an ongoing debate over education and care models. Midwifparation. For col health. Theref. Studying what is occurring and its variation patterns was viewed as a first step before intervening to improve quality. The impact of specific QI approaches varies, depending on the context in which they are implemented. In general, success of a QI program is determined by an organization’s commitment to monitor, assess, improve, and embed continuous improvement into the organization’s culture for seeking higher levels of performance. Understanding systems and how they interact is critical, since improvement in one area of operations may induce unintended consequences, and even harm, in another area. Focusing improvement efforts on customers who benefit from the system is fundamental, whether they be patients, students, providers, or teachers. A team approach, involving more than one discipline, builds creativity, enthusiasm, and commitment to the work. Focus on collecting and using data, constant monitoring, and tracking the same elements throughout the iterative PDSA cycles is paramount to successful QI endeavors. The WHO has defined QI as “An approach to improvement of service systems and processes through the routine use of health and programme data to meet patient and programme needs.” This versatile definition can be applied to both healthcare delivery and educational programs. Improving quality in healthcare requires an appreciation and understanding of system complexity, critical components, and most importantly, the cultural context and population served. Educational program quality can be defined as the degree to which didactic and clinical education increases the likelihood of desirable outcomes, consistent with current knowledge. These educational outcomes should reflect competencies of nurses and midwives as they relate to providing patient care.Quality improvement, as we currently know it in healthcare and education, originated in 1939 with control charts to track variation in manufacturing defects and the “learning and improvement cycle” known in healthcare as the Plan-Do-Study-Act (PDSA) cycle. Studyilemented. In genher area. Focusi needs.” This ven served. EducatIn light of the above priorities and needs for highly competent nursing and midwifery professionals who are well-equipped to promote UH and PHC, this article aims to present the development of an EQI Toolkit for schools of nursing and midwifery in LAC countries.Following the Standards for Quality Improvement Reporting Excellence (SQUIRE), this multi-centric study is outlined in sequential phases .. For consistency with this previous PAHO commissioned work, the term UH was used as encompassing both universal access to health and universal health coverage. Donabedian’s model provided the framework for toolkit design and was operationalized through the QI process; a data-driven, formal approach to performance analysis in nursing and midwifery educational programs and the systematic efforts to improve it.A group of 10 faculty with expertise in the areas of QI, improvement science, program evaluation, competency-based education, IPE, IPCP, midwifery, primary health care, rural care, and global health were assembled. This expert group was tasked with reviewing previous work on educational resources in the LAC region, as well as the broader literature, to develop a plan for self-directed QI among nursing and midwifery schools/programs. As a starting point, an extensive survey carried out across LAC nursing school programs regarding preparation of graduates to promote UH and PHC was reviewed for baseline data and recommendations’s model provideprove it.A systematic literature search on the topic of QI, as it relates to nursing and midwifery education or nursing and midwifery education programs, was carried out in five databases; PubMed, Scopus, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Latin American & Caribbean Health Sciences Literature (LILACS), and Google/PAHO. Search terms included: “education, nursing, midwifery, baccalaureate, program accreditation, quality improvement, standards, Latin America, Caribbean”. A total of 313 peer-reviewed full-text articles, published within the last 10 years (from 2010 to 2018) in English language were retrieved. Search strategy results are presented in the PRISMA flow chart .Two investigators from the phase 1 expert group conducted independent title/abstract/full text reviews, discussed disagreements and a consensus was met. A total of 50 articles were screened for the following inclusion criteria: 1) focus on an educational program or school of nursing or midwifery; 2) focus on monitoring or evaluation or assessment of quality or implementation of a QI project; and 3) focus on a model or conceptual framework in relation to accreditation, competency-based education, IPE, IPCP or transformational education. Exclusion criteria were: 1) focus on a healthcare setting; 2) clinical practice focus; and 3) no full-text available online.With surveyed areas for improvement in mind, the same 10-member expert group: 1) identified existing QI resources for nursing and midwifery educational programs, and 2) prepared the model for a QI plan that could be tailored by LAC schools to address specific weaknesses in UH and PHC. Development of this model evolved into the EQI Toolkit with a target population of all nursing and midwifery schools offering undergraduate programs in the LAC region. Because of substantial educational differences among programs, it was clear that the toolkit would need to be adaptable to country/language/culture/setting, plus be user friendly and simple to use. For clarity, a glossary of terms with appropriate references was developed as an appendix to the EQI Toolkit.. For this project, ideas were recorded using a laptop and were immediately projected onto a screen for everyone to see and compare notes. Each work group member took responsibility for drafting content and tools for inclusion. An iterative process was used to select and connect the key elements to form a coherent resource for the next phase of review.The first toolkit version was developed using brainstorming and modified affinity grouping techniques. This process allowed the expert work group to generate, categorize, and choose among ideas by posting common themes on a flip chart. For thTo obtain user evaluation, the qualitative method of iterative processing cognitive debriefing was applied. Four international stakeholders were invited as reviewers through three partnering PAHO/WHOCCs. Individuals represented Brazil, Chile, and Mexico and were chosen for their diverse perspectives and technical expertise. They were asked to review the first version and to provide detailed comments, suggestions, and track-changes in the actual file. Further, reviewers were asked to complete the Cognitive Debriefing Form  during aTwo investigators asked questions and recorded responses. For two of the Spanish speaking respondents, questions and answers were translated back and forth by the Spanish speaking investigator. Respondent behaviors, such as asking for clarifications, being ambivalent or skipping items were recorded. Any difficulties experienced by subjects were used to modify the first toolkit version, thus making data collection an iterative process. Qualitative content analysis of cognitive debriefing data from the international reviewers was applied. In addition, the reviewers’ electronic track changes and added comments were also considered as part of the review process. Last, reaching consensus was used when reconciling reviewers’ input and data in order to agree on a final version of the EQI Toolkit.The UAB Institutional Review Board (IRB) designated this project as Not Human Subjects Research (IRB-300002006). The international review process was overseen by the leading investigator who safeguarded ethical standards of conduct and that all review comments were carefully considered..A total of 22 articles met all inclusion criteria and are listed in .Content analysis of selected articles revealed the following themes: 1) assessment - evaluation; 2) continuous quality improvement (CQI) - accreditation; 3) curriculum plan development; and 4) implementation - case study. Key points relevant to QI efforts in schools of nursing and midwifery, with a focus on UH and PHC, are presented below, grouped according to above themes.ssessment and evaluation are essential to the QI process, reflected in select articles. Despite the labor intensive nature of data mining, it is argued that gap analysis and feedback, as a method to assess core competency in the curriculum, could decrease content saturation and promote active learning. Program evaluation is achieved through course reviews, student surveys, pre-and-post-program assessment of students’ knowledge/skills, and faculty interviews about their experiences with new teaching methods. It is recommended that evaluation protocols include linkages to external criteria for evaluating the plan itself, while the teaching-learning process should be subject to ongoing QI. This is consistent with findings from an extensive survey of nursing schools in LAC countries showing nursing program evaluation, student evaluation, and outcomes as the top priorities for QI initiatives. Using an original self-assessment tool, the same survey revealed the heterogeneity of education, with a clear need to strengthen training in UH, PHC, and transformational education. Schools seeking accreditation are highly encouraged to develop periodic evaluations of curricula and programs with participation from their students, as well as to share the results with educational authorities and professional organizations.Aticles,-29. Dlearning. Progra methods,29. Igoing QI. This itiatives. Using izations.CQI-accreditation approach is described in select articles. It provides metrics and information to administrators and faculty regarding the rigor of programs and the potential of graduates. CQI as a hallmark of the accreditation process, identifies areas for improvement and allows for planning. Thus, the Standards for Accreditation of Baccalaureate and Graduate Degree Nursing Programs in USA, required by the Commission on Collegiate Nursing Education (CCNE), are also examined as an ongoing CQI process. Brown and Marshall use a quality enhancement plan (QEP) to assess key factors affecting program outcomes. They demonstrate CQI effectiveness in increasing NCLEX-RN pass rates, improving student advisement, and raising student satisfaction. Other effective interventions across programs include: early identification of students at risk, timely remediation, and program policy enforcement. Of equal importance is surveying past graduates of their opinion and feedback about strengths and weaknesses of the school/program.The planning. Thus,  process,42. BMarshall use a qorcement. Of equ/program.curriculum plan development, faculty collaborations and partnerships were instrumental in augmenting evaluation tools and processes for clinical learning, expanding clinical capacity, and improving clinical experiences. Armstrong et al. use the Quality and Safety Education for Nurses (QSEN) competencies to enhance a competency outcome performance assessment (COPA)-based curriculum. Reflective components of practice-based learning activities based on QSEN competencies engage students in evaluating: 1) level of comfort with being a change agent, 2) unit climate for change, and 3) delegation and communication skills.For eriences. Armstrg et al. use then skills.Implementation case studies have shown that students’ readiness to learn and ability to embrace concepts, processes, and outcomes measurement is enhanced by early teaching of systems and QI thinking within an undergraduate curriculum(-). Exposing students to real world experiences that underline the significance of these skills helps bridge the education-practice gap. A study with United Kingdom student nurses showed that they can successfully overcome their fears and meet the QI practicum challenge, if adequate support mechanisms are in place. Doing a practicum as a compulsory assignment in a large-scale cohort is achievable, if there is balance between motivation to learn and fear. Students develop teamwork and leadership skills, while clinical partners are motivated by the students. Adopting a blended learning approach, where face-to-face classroom and online teaching is combined with clinical experiences, along with incorporating IT through the use of electronic portfolios to demonstrate student accomplishments and document program and course outcomes, have shown promising results. Furthermore, use of patient simulators and the concierge model of simulation can be an effective strategy for deliberate practice of skills and standardized exposure to limited scenarios. Clinical simulation and debriefing are effective methods to incorporate the competencies as set forth by the Institute of Medicine (IOM) and the American Association of Colleges of Nursing (AACN); patient-centered care, interprofessional teams, evidence-based practice, clinical reasoning, patient safety, and practice across the life span . Subsequently, the EQI Toolkit has attempted to incorporate all of the above competencies.rriculum-35. Ein place. Doing and fear. Studenstudents. Adopti results,41. Fcenarios,33. CAs part of the evaluation process, all four international reviewers answered questions using the Cognitive Debriefing Form  addressi and PAHO documents. An overview of nursing and midwifery education presents the core competencies for educators(-), followed by an introduction of QI in nursing and midwifery education. The expert group adopted the Model for Improvement (MFI) as the operational model for educational QI due to its widespread use and relative simplicity for end users. The MFI is described as a systematic method to effectively identify weaknesses or gaps in educational structures, processes, and outcomes. Starting with organizing a team to tackle the improvements and assess the current state, the MFI progresses to a series of questions that lead to an aim statement of what is to be accomplished. Measures are presented along with a strategy to determine whether an improvement has been made, identification of changes that may be tested to accomplish improvements, and iterative PDSA cycles to evaluate changes. The last step is how to sustain the change(s) and perhaps scale it to other programs or schools. Both the MFI and the PDSA cycles are depicted in The toolkit includes an introduction to and definitions of UH, PHC, and IPCP concepts based upon WHO and PAHand PAHO documenducators-10, f“Universal Health and Primary Health Care in Nursing & Midwifery Education: A Self-Assessment QI Tool” is envisioned for use in conjunction with other available tools to assess the current state. Using ‘yes’ or ‘no’ responses to statements, Part one of the tool assesses items related to the school’s structure, including mission, objectives and philosophy, as well as resources, infrastructure, external relationships and policies. Part two covers 14 items evaluating students’ and graduates’ professional competencies addressed in the curriculum, followed by 8 items related to curriculum model, teaching and learning strategies. Part three includes 6 items to evaluate the school, program and faculty outcomes, followed by guidance for tallying total scores.To accomplish the MFI and PDSA steps, six tools were incorporated as appendices to the toolkit. First, an instrument was developed for schools of nursing and midwifery to assess program components representing areas of excellence, as well as opportunities for improvement, specific to UH and PHC. The “Prioritization Matrix” along with a completed example were included. Together, these items assist the improvement team to prioritize up to three items/areas requiring improvement, stemming from the results of the previously described self-assessment tool. The matrix guides the team in using weighed criteria to score improvement project options and use criteria-based evidence when choosing those to begin first. Priority criteria can be tailored for each school/program; cost, expertise, space, location, and organizational culture are some of the most frequently encountered.Second, a “PDSA Cycle Worksheet” consisting of four sections along with a completed example was provided. The first section, “Plan”, asks the improvement team to detail the plan for a test of change; the improvement to be implemented, who will be involved, what are the components of the change, when will the change begin and end, where the project will occur, and predictions regarding the change. The team is asked to define all measures , name their data source and frequency of measure, and assign responsibilities. The second section, “Do”, reminds the team to determine the number of participants, carry out the change or test and collect planned data, while documenting problems and unexpected observations. In the third section, “Study”, improvement team members are guided to analyze data and summarize lessons learned, then compare data to a priori predictions and reflect on what was learned. Last, in the “Act” section, the team is prompted to refine the change based on what was learned, and consider whether to adapt, adopt, or abandon the change.Third, a The fourth appendix includes two improvement exemplars for teaching UH and PHC competencies. The first one is a completed MFI/PDSA project illustrating an interprofessional team-based immersive simulation. It exemplifies language for problem statement, aim, measures, test of change , detailed project plan, and report of findings from the PDSA cycles. The second example offers an improvement project ready to be adapted and tested. Learning objectives, scenarios, questions to stimulate discussion, and prompts helping end users adapt the project to their courses are included.The last two appendices of the EQI Toolkit represent the group’s collective knowledge, experience, and multicultural awareness. An extensive list of freely available web-based QI resources for nursing and midwifery educators and administrators was compiled. Special emphasis was given to resources from the LAC region by searching through the LILACS database, a comprehensive index of scientific and technical literature from Latin America and the Caribbean. Also, a glossary of terms with definitions of main concepts covered in the EQI Toolkit and relevant references was included to enhance understanding and clarity.International reviewers were positive about using the QI process to evaluate their nursing programs, particularly if they didn’t currently use a standardized data collection and evaluation method. All reviewers suggested changes to make the toolkit more specific to the educational structure and culture of their region. Content was added to address the variation in nursing, nurse midwifery, and midwifery programs across LAC countries. Wording was changed to reflect cultural nuances and a PHC relevant case study was translated, culturally adapted, and applied to the PDSA model, addressing IPCP and nursing/midwifery competencies. During the last phase, a reconciliation list of QI resources was developed for final review and input with effort on including LAC specific resources in Spanish and/or Portuguese. Overall, feedback strengthened the usability of the toolkit.. Ultimately, the toolkit’s endorsement will be a measure of its comprehensiveness, relevance, and adaptability to different settings, institutions, and countries. Extension of its use to other types of programs, such as graduate, post-graduate, and interdisciplinary could signal greater adaptability and inclusiveness. Its potential for better educational outcomes and systematic QI is highly anticipated. Given that currently there is no mandatory accreditation for most nursing and midwifery programs in LAC countries, the EQI Toolkit could serve as a guide for institutions towards developing standards and procedures for ongoing, systematic internal quality control and improvement. Key higher education stakeholders can play an important role by providing strong incentives and building capacity at multiple levels . Examples include QI education and training, technical assistance, ongoing mentoring or coaching, and financial support towards QI infrastructure development and staff. Improvement science coursework could also be introduced to undergraduate students.The final EQI Toolkit is freely available through the UAB School of Nursing website. Ultima global campaign has recently teamed up with the Institute for Healthcare Improvement (IHI) Open School to offer several free online modules on QI, program management, and culture change(-). Although mostly targeting the healthcare environment, introductory and intermediate modules aim to build fundamental QI knowledge and skills in becoming a change agent. According to campaign leaders, nurses and midwives are well positioned to make innovative changes that not only improve their work practice environment, but also shift the universal health paradigm. After piloting and further refinement, LAC schools of nursing and midwifery could use this toolkit to improve capacity for preparing future professionals, according to their country and national health system needs. Long-term, the toolkit could perhaps, guide strategic planning initiatives and future directives for undergraduate programs and schools.Learning the science and application of improvement is becoming more pertinent to nurses and midwives around the globe. The Nursing Now global e change-44. AThe EQI Toolkit was developed using a systematic and iterative process. Upon identifying relevant evidence-based literature from the LAC region, online resources stemming from professional associations, organizations and institutes were compiled into an accompanying appendix. Completeness and accuracy of data were assessed through: 1) reaching consensus after several iterations, 2) choosing expert group members and reviewers based on diverse perspectives and technical expertise, and 3) indirectly asking reviewers to endorse the content and providing them the final draft before its release. Cognitive debriefing revealed some strong positive comments in support of the developed plan for QI. Nevertheless, certain limitations exist. Systematic literature review was limited to English language articles with full-text availability, published within the last 10 years. All included articles were appraised at a low level of evidence indicating a deficit in available rigorous evidence. Pilot testing of the EQI Toolkit was not within this project’s scope and is planned for the future. Translation and linguistic adaptation into Spanish and Portuguese would be required to broaden the toolkit’s reach, once the original English version has been sufficiently pilot tested and refined. Further cultural and organizational adaptation to each country and/or setting is needed to tailor and refine the toolkit.Strengthening UH and PHC through transformative education has been at the core of a PAHO/WHO “call for action” for preparing a competent global nursing and midwifery workforce. Building capacity in LAC countries, many with low ratios of nurses and midwives to population and great variation of professional entry-level education, is paramount. Based on a systematic review of available literature, and recognizing that substantial educational differences occur among programs, the EQI toolkit is poised to be adaptable to individual characteristics or circumstances within any nursing or midwifery undergraduate school program. The toolkit offers a dynamic model for a QI plan that could be adopted by educational programs in LAC countries to address weaknesses in UH and PHC within the framework of transformative education and IPCP. Inherent in this work is the acknowledgement that nurses and midwives have important leadership roles to play in health promotion, disease prevention, and reducing morbidity and mortality. A strength of this toolkit stems from its development in partnership with international stakeholders. Future activities will focus on how this educational intervention can be disseminated, evaluated and further improved for broader application.


The overall aim of this paper is to examine effectiveness of an evidence-based intervention in community settings, and the factors associated with effectiveness. Limited research in the area of HIV prevention has focused on evaluating intervention program effectiveness in real-world settings.Mujer Segura across 13 different clinics in 13 sites across Mexico. The overall design was a cluster randomized Type I design simultaneously testing intervention program effectiveness with an observational study of implementation factors. We aimed to examine the effectiveness of Mujer Segura at reducing HIV/STI incidence among FSW participants at each site, and to examine the client-, provider-, organization-, and structure-related factors associated with program effectiveness.We implemented an efficacious theory-based sexual risk reduction intervention for female sex workers (FSW) called We found lower HIV/STI incidence density in the intervention relative to the control group in 5 sites we labeled as “program effective sites,” but not in 8 sites we labeled as “program ineffective sites.” Using generalized estimating equations controlling for site and computed mean difference effect sizes, we examined statistically and practically significant differences, respectively, between the two groups of sites along various client-, provider-, organization-, and structure-related characteristics. Results indicated that client-level HIV/AIDS related knowledge, and proficiency and engagement in the organizational social context were associated with program effectiveness.Mujer Segura in 13 sites in Mexico. Results suggest that other measures may need to be included in future implementation studies than the ones included here. We were able to point to a few specific factors that should be targeted to increase effectiveness of similar evidence-based programs in low- and other middle-income countries like Mexico.Enormous resources are required to systematically and adequately test the role of multilevel factors on program effectiveness. We successfully implemented NCT01465607.ClinicalTrials.gov  Mujer Segura” (“Safe Woman”) to promote condom use and enhance safer sex negotiation skills and reduce HIV/STI risk among FSW in two U.S.-Mexico border cities. In a randomized controlled trial, we found that FSW randomized into Mujer Segura had a 40% lower cumulative incidence of sexually transmitted infections [Mujer Segura. Mexfam has clinic sites throughout Mexico, and we implemented the intervention program across 13 different Mexfam community clinics in 13 sites across Mexico. The general aim of the current paper is to examine the effectiveness of the intervention in the context of real-world community settings.Female sex workers (FSW) in Mexico have an elevated HIV prevalence compared to the general population . In a stfections . There wDespite a preponderance of studies in the HIV prevention literature that aim to develop and test the efficacy of behavioral interventions, less is known about whether efficacious interventions also demonstrate effectiveness in less controlled, real-world community settings. Even less is known about the factors in real-world community-based settings that may or may not affect effectiveness of evidence-based programs. The effectiveness of evidence-based programs within community settings may be affected by factors from different sources that occur within multiple system and organizational levels . The impClient-related factors include demographic characteristics , behavioral risk factors , as well as knowledge, beliefs, self-efficacy, and motivation to change behavior –11. Altheffectiveness of the intervention. We examined the effectiveness of Mujer Segura in each clinic , and examined the client-, provider-, organization-, and structure-related characteristics that might explain differences in program effectiveness. We hypothesize that provider- and organization-related factors known to be associated with successful implementation will also be associated with effectiveness in this study. These include more positive evidence-based practice attitudes or a more positive organizational social context , and higher emotional competence. Client- and structural- related factors have been studied less in the area of effectiveness; therefore we aim to explore how these factors might be related to effectiveness in this study.Implementation frameworks and models suggest that the achievement of effectiveness in evidence-based programs are the natural result of a successful implementation process. There often remains a gap of knowledge in the transition from efficacy in controlled trials to effectiveness in real-world settings . InsteadOur approach to studying factors related to program effectiveness focuses on characteristics that operate at multiple levels . However, we did not conduct multilevel modeling to analyze the data. Instead, while we still controlled for the nested structure of the data, we elected a more parsimonious approach and divided sites into those that were “program effective”  and “program ineffective,” and compared the two groups along different multi-layered characteristics.This study took place in 13 clinics in eight states in Mexico. Clinics were located in Veracruz, Iguala, Ixtaltepec, Revolución, Nezahualcoyotl. Naranjos, Tuctla, San Luis Potosí, San Luis de la Paz, Tepeji del Río, Guadalajara, Huajuapan, and Tlapa. The clinics primarily served women in regard to family planning, reproductive health, and childbirth.Mujer Segura, or a time-equivalent attention control condition. Thirteen different clinics in eight states in Mexico were enrolled and FSW were randomized within clinics with a target of recruiting 80 FSW per clinic. The selection process of the clinics is described in the protocol; generally, sites that had the capacity to implement the intervention were selected. We originally aimed to only include 12 clinics, but added a thirteenth after the study protocol was published because the first study site (Revolución) was slow to start due to drug-related violence in the area. The second aspect of the study, which was not randomized, involved the collection of data at the clinic sites concerning provider, organizational, and structural factors hypothesized to affect the effectiveness of the intervention. The implementation strategy was a “train the trainer” approach that was designed to build a network of HIV prevention services for FSW that would achieve sustained fidelity and provider competency [The overall design of the study was a cluster randomized (participants randomized within clinic) Type I design that simultaneously tested intervention program effectiveness with an observational study of implementation factors . Detailsmpetency .Mujer Segura or the control group using a computer-generated random order, and completed a 6-month follow-up assessment. In total, 1,088 FSW participated in this aspect of the study. A total of 558 were randomized into the experimental condition and 530 into the comparison condition. Staff at each site enrolled participants and assigned participants to conditions. Baseline and follow-up procedures were completed between June 8, 2011 to April 30, 2014.Eligibility criteria for the FSW were the same as those of the previous efficacy study. Participants were biologically female; at least 18 years of age; self-identifed as a FSW; report having traded sex for drugs, money, shelter, or other material benefit within the previous two months; had unprotected vaginal or anal sex with a client at least once during the previous two months; had no previous HIV-positive test result; and agreed to be tested for HIV and STIs at baseline and at 6-month follow-up. Female sex worker participants completed a baseline assessment, were randomized into Mujer Segura intervention, a brief (35 to 40 minute), single-session, individual intervention that combines principles of motivational interviewing (MI) [Those in the experimental condition received the ing (MI) , social ing (MI) , and theing (MI) . A detaiing (MI) . The couMujer Segura intervention. The comparison condition is time-equivalent and utilizes counseling materials provided by CENSIDA, the Mexican federal agency responsible for HIV and AIDS prevention [Because FSW are at high risk of STI and HIV infection, both interventions impart knowledge necessary for practicing safer sex. However, in the standard counseling condition, topics are covered in a lecture format without the use of MI or the role-plays, exercises, and problem-solving emphasized in the evention , that arAcross the 13 sites a total of 130 clinic staff completed a baseline battery of measures related to attitudes towards evidence-based practice, and organization attitudes and practices. Data were collected in-person, on-site at each clinic between January 11, 2011 to December 6, 2013. These measures include emotional competence, leadership, and organizational social context.All study procedures were approved by the Institutional Review Board of the University of California, San Diego (Project 090320) and by the Ethics Committee at Mexfam. All procedures were conducted in accordance with the Helsinki Declaration. Written informed consent was obtained from all participants.To evaluate program effectiveness, our primary outcome was the 6-months incidence density of HIV/any STI by site, calculated using person-time methods. More specifically, the incidence density was calculated by dividing the number of incident cases that occurred during the follow-up by the number of person time (years) spent at risk. For the women who did not acquire HIV or an STI during the follow-up, the time at risk was represented by the entire time period between the baseline- and their 6-month visit, whereas, for the women who acquired either HIV or an STI, the time at risk was represented by half of the time interval between the baseline and their 6-month visit.Our secondary outcome was behavioral, the number of unprotected sex acts with clients during the previous 30 days (reported at 6 months).FSW participants answered questions about their age, marital status, and age when they first started trading sex. We also asked about their alcohol use using the AUDIT-C scale; a three-item, reliable alcohol screening instrument that measures frequency and quantity of alcohol consumption using a 5-point scale for each item. Summary scores were used to create a dichotomous variable (hazardous versus non-hazardous drinking). Based on validation studies, a score of 3 or more for women was used to identify risky or hazardous drinking , 30. ParWe assessed HIV-related knowledge using the 18-item HIV knowledge questionnaire , with hiPersonnel from each site completed measures of organizational and provider factors. Specifically, they completed measures of evidence-based practice attitudes and emotional competence. For these two measures, we used the same response scale . We also assessed counselor fidelity to the intervention as a provider-related factor.Evidence-based practice attitudes. Personnel completed a measure of their attitudes toward the adoption of evidence-based practices using the Evidence-Based Practice Attitude Scale [de Scale ,32. ThisEmotional competence. Emotional competence is the learned capability based on emotional intelligence, or knowing and handling one’s own and others’ emotions that results in outstanding performance at work [ at work . The Emo at work ,34. The Intervention fidelity. FSW participants who were randomized into the experimental arm and completed the Mujer Segura intervention were asked to complete a checklist of all of the intervention tasks and elements that were or were not completed by their counselor. There was a total of 40 intervention tasks that each Mujer Segura counselor was to complete. We created a dichotomous fidelity variable of whether the participant reported that the counselor completed all intervention tasks. Participants in the control condition did not complete a measure of fidelity.Personnel from each site completed measures of evidence-based practice staffing and recognition, transformational leadership, and organizational social context. For these, we used the same response scale as the one used for the provider-related measures .Evidence-based practice staffing. Participants indicated their level of agreement with different items about whether the clinic selects staff that would support evidence-based programs [programs . There aEvidence-based practice recognition. Participants indicated their level of agreement with different items about whether the clinicians who use interventions would receive positive recognition [ognition . There aTransformational leadership. The Multifactor Leadership Questionnaire [ionnaire  was usedOrganizational social context. The organizational social context (OSC) measure includes eight subscales designed to capture organizational culture, climate, and employee morale [e morale . ParticiThe marginalization index was computed for each city in Mexico in 2010 by the Consejo Nacional de Población . A higheMujer Segura intervention from the program effective and program ineffective sites. For organization-related characteristics, we also compared responses from clinic personnel from program effective versus program ineffective sites (n = 130) on their responses to questions about their organization/clinic. For structure-related characteristics, we compared the 13 total program effective and program ineffective sites on population total, and health, education, income, and marginalization indices.We analyzed the data in steps. In the first step, for both the intervention and control groups we calculated our primary outcome, 6-months incidence density of HIV/any STI by site, using person-time methods. The thirteen sites were then individually categorized as a “program effective” or a “program ineffective” site, with a “program effective” site being defined as a site that yielded a lower incidence density in the intervention group as compared to the control group. The two site groups were then compared on client-, provider-, organization-, and structural-related characteristics by level of success. We used different data sources depending on the type of characteristic in question. For client-related characteristics, we compared FSW participants from program effective versus program ineffective sites  on their baseline characteristics. For provider-related characteristics, we compared clinic personnel from program effective versus program ineffective sites (n = 130) on their responses to provider-related questions. This was the case for all provider-related measures except intervention fidelity. Since the latter was completed by the FSW participants, we compared a total of 558 women who completed the We aggregated the different sources of data differently depending on the level/type of characteristic in question . For client-related characteristics, we aggregated data from female sex workers . For provider and organization characteristics, we aggregated data from clinic staff across the 13 sites (n = 130). For structure-related characteristics, we analyzed each clinic as its own unit (n = 13). Regardless of the type of characteristic, we divided data from program effective sites and compared them with data from program ineffective sites . To compare sites, we examined both statistical  and practical  significance. To analyze statistically significant differences between program effective and program ineffective sites, we modeled the data using generalized estimating equations (GEE). GEE allows us to adjust for site clustering, and appropriately model the distribution of the outcome. We conducted GEE to predict whether the particular unit of analysis  was part of a program effective or program ineffective site using a binomial distribution and logit link function. In the GEE models, we specified an exchangeable covariance structure. P-values from GEE model results are presented in the tables. In addition to examining statistically significant differences, we also computed effect sizes for differences between program effective and program ineffective sites. Effect sizes shed light on practical or clinical significance, rather than statistical significance. They also help with the analyses of the structure-related variables specifically, since we have an analytic sample of only 13 for these variables .d). For dichotomous predictor variables, we used sample sizes (by cell) and chi-square values to calculate the odds-ratio. We determined effect size magnitudes following recommendations by Cohen [We followed the procedures described by Lipsey and Wilson to calculate and interpret the magnitude of effect sizes ,42. For by Cohen  as follot = 3.73, p = 0.02). For the program effective sites combined, the HIV/any STI incidence in the control group was 2.8 times that of the intervention group. There was also a significant difference in the incident rate ratio between the control groups vs. the intervention groups among program ineffective sites. . For the program ineffective sites combined, the HIV/any STI incidence in the control group was 0.5 times that of the intervention group.To further show that program effective sites showed predicted changes in biological and behavioral outcomes, for each site group separately, we also evaluated the effect on the intervention on the 6-month HIV/STI incidence rate and the number of unprotected sex acts with clients during the previous 30 days (reported at 6 months). To evaluate the effect of intervention on the HIV/any STI incidence rate, we used mixed effects Poisson regression with empirical variance estimation and site level random intercepts. Results showed a significant difference in the incident rate ratio between the control groups vs. the intervention groups among program effective sites . For the program ineffective sites combined, there was no significant difference between the intervention and control groups in unprotected sex acts .To assess the effect of intervention on the number of unprotected sex acts, we used negative binomial regression with empirical variance estimation via generalized estimating equations (GEE). Site was used as a cluster variable with an exchangeable correlation structure . The number of unprotected sex acts at visit 1 was used as a covariate. For the program effective sites combined, at 6-months of follow-up, the 30 days rate of unprotected sex acts in the intervention group was 0.47 times the corresponding rate in the control group  among FSW in 13 clinics throughout Mexico. We found that in five sites, HIV/STI incidence density was lower in the intervention group as compared to the control group, and we categorized these sites as “program effective” in showing intervention program effectiveness. In eight sites that were categorized as “program ineffective” the incidence density was lower in the control group compared to the intervention group. We also examined different client-, provider-, organization-, and structure-related characteristics as potential correlates of Mujer Segura effectiveness. For client-related factors, we found that FSW in program effective sites reported greater HIV/AIDS knowledge than FSW in program ineffective sites. We did not find any statistically significant differences between the two site groups on provider-related factors . For organization-related factors, we found the program effective sites has staff that reported greater proficiency and engagement with clients than staff in program ineffective sites. For structure-related factors we were limited in statistical power (n = 13). However, effect size magnitudes suggest that regions with better health and education were more likely to have a program effective implementation site. Overall, across all of the variables studied here, we failed to reject the null hypothesis that program effective and ineffective sites were statistically significantly different from one another. We only found few client-, provider-, organization-, and structure-related factors that were associated with effectiveness of our evidence-based program.The present study examined the effectiveness of an efficacious HIV prevention intervention , and in this region (Mexico). The provider- and organization-related measures included in this study were written and developed in a higher income country- the U.S. For example, the emotional competence inventory was developed in the U.S. using a sample of people with advanced education and skills  [Overall, we examined many different client-, provider-, organization-, and structure-related variables and how they might be associated with neering) . It may Mujer Segura, including socio-political factors  and government funding [This study was conducted to examine client, provider, clinic, and system level predictors of effectiveness of an evidence-based HIV prevention intervention with FSW across Mexico. The present findings may lack generalizability to other FSW in other parts of the world, or to other types of interventions. The measure of intervention fidelity may be biased by the participants’ general attitudes towards their counselors. However, using the same fidelity measure we published data showing how fidelity is related to behavior change, suggesting good predictive validity for the measure . We did  funding ,46.To our knowledge, this is the first study to systematically examine implementation of a program of this kind in Mexico, and more research is needed to develop new measures to directly and systematically assess the role of providers, organizations, and structural contexts in affecting program effectiveness. Until then, our data help to generate hypotheses that could be tested in future implementation and effectiveness studies. Unfortunately, substantial resources would have to be spent to be able to have the statistical power  to detect structure-level differences in program effectiveness and implementation differences across the sites.S1 File(DOC)Click here for additional data file.S2 File(DOC)Click here for additional data file.


Organizational features of work often pose obstacles to workforce health, and a participatory change process may address those obstacles. In this research, an intervention program sought to integrate occupational safety and health (OSH) with health promotion (HP) in three skilled nursing facilities. Three facilities with pre-existing HP programs served as control sites. The intervention was evaluated after 3–4 years through focus groups, interviews, surveys, and researcher observations. We assessed process fidelity in the intervention sites and compared the two groups on the scope of topics covered (integration), program impact, and medium-term sustainability. The intervention met with initial success as workers readily accepted and operationalized the concept of OSH/HP integration in all three intervention facilities. Process fidelity was high at first but diminished over time. At follow-up, team members in two intervention sites reported higher employee engagement and more attention to organizational issues. Two of the three control facilities remained status quo, with little OSH/HP integration. The intervention had limited but positive impact on the work environment and health climate: staff awareness and participation in activities, and organizational factors such as decision-making, respect, communication, and sharing of opinions improved slightly in all intervention sites. Resources available to the teams, management support, and changing corporate priorities affected potential program sustainability. A number of chronic diseases are known to be associated with psychosocial work stressors, especially low job control, and other organizational factors such as night shift and overtime work ,2,3,4,5.A newer approach to workplace HP is that of enhancing its effectiveness by combining it with occupational safety and health (OSH) protections. This concept of integrated employee health programs has been put forward by a few researchers ,13, the Healthcare work is physically and psychologically demanding, exposing workers to many workplace stressors that affect their safety and health and which simultaneously may interfere with effective prevention measures ,22,23. WWe have previously proposed ,25 that This article reports our evaluation of process fidelity, extent of OSH/HP integration, health impact, and sustainability of the participatory intervention program. In line with the middle-range “partnership synergy” theory, we have relied on information collected before, during, and after the study to describe the context (institutions and workforce), evidence for posited mechanism of change , and short- and medium-term outcomes . The intervention was compared three to four years after initiation in the three participatory intervention program (PIP) sites to three control sites with non-participatory health promotion (NPHP) programs.This study, “Promoting Physical and Mental Health of Caregivers through Transdisciplinary Intervention,” was carried out through the Center for the Promotion of Health in the New England Workplace (CPH-NEW), a Total Worker Health® Center for Excellence of the National Institute of Occupational Safety and Health. At the time when this study began, “Total Worker Health” was defined by National Institute of Occupational Safety and Health as “a strategy integrating occupational safety and health protection with health promotion to prevent worker injury and illness and to advance worker health and well-being.” The study was approved by the University of Massachusetts Lowell Institutional Review Board (protocol # 06-1403).The project occurred within the context of a multi-year partnership with a large, for-profit long-term care company, which operated over 200 nursing facilities in 12 states within the eastern United States. In 2003–2006, the company had initiated a “safe resident handling” program in all of its skilled nursing facilities. Some sites also had HP activities as of 2006, depending on local initiative; a corporate-sponsored wellness program began in 2011.Within that context, this project component sought to improve the health and safety of nursing home workers by creating participatory teams of non-supervisory workers to address integrated workplace HP and OSH concerns within their own worksites . The teaBased on the researchers’ criteria, the company’s regional director for health and safety recommended five skilled nursing facilities that did not yet have active HP programs and whose administrators were expected to be receptive to the participatory intervention process. We selected three of these facilities using a priori criteria to judge which were most organizationally ready for PIP . AnotherIn the PIP centers, team members were recruited from the entire workforce from volunteers responding to posters and management promotion of the program. Each team started with employees from various departments  who met bi-weekly for one hour with two researchers. Initial team meetings involved identification of key issues in workplace health, psychosocial stress, and work organization. The intervention began with intensive orientation of the PIP team (2–4 meetings per site over 1–2 months) to worker health and well-being, and Total Worker Health as a comprehensive approach. PIP teams identified issues of importance to members and discussed possible solutions or projects to address these concerns. Team members sought the opinions of their co-workers for program goals and specific activities. PIP team members communicated with individuals at various levels of their facility with updates and available activities . Initially the researchers facilitated meetings, guided discussions, and assisted in framing presentations to the site administrator regarding a team’s proposed project. Active facilitation involved the wellness champion and a research assistant attending all bi-weekly team meetings over 2 to 3 years, followed by monthly team meetings over 1 to 2 years, then quarterly telephone check about the program process with the wellness champion per site. The researchers provided technical assistance on a variety of topics, such as seminars on ergonomics in skilled nursing facilities and a food preferences survey to assist in developing programs for healthier food provision. Meeting minutes and activity logs were maintained by the researchers and utilized for ongoing process evaluation. The goal was that participatory teams would move over time from co-governance to become independent, with support of the facility wellness champion. Thus, the study plan called for the researchers gradually to reduce our facilitation efforts over time. This was communicated to all participants at the beginning of the project. The phase-out period entailed 1–2 years of quarterly telephone check-ins with the wellness champion.A mixed-method  approach utilized qualitative and quantitative data to examine the process and impact of the participatory OSH/HP or NPHP program in each facility. Results from the qualitative and quantitative analyses were triangulated with the researchers’ direct experiences and knowledge of the organization to understand the process, impact, and sustainability of the participatory program.Quantitative data included a brief baseline (pre-intervention) survey of the wellness champions about HP program activities in all six centers. We also conducted employee surveys in the six centers at baseline 2008–2009) and around the fourth year (2012–2013) of the participatory intervention. A self-administered questionnaire collected information on worker chronic disease history, health beliefs and behaviors, and perception of the work environment: physical and psychological job demands, decision latitude, and social support from supervisors and coworkers –2009 and,34. Workn = 645) and post-intervention (n = 649) prevalences were compared by cross-tabulation with chi-square statistics and mean values by t-test for independent samples. Cumulative incidence of self-reported chronic diseases was computed from baseline to post-intervention within each group and compared between PIP and NPHP with Fisher exact tests due to small numbers. Within-person changes from baseline to follow-up (limited to workers responding to both surveys) were examined using stratified cross-tabulation and paired sample t-test. All analyses were done with SPSS 22.0 .Most analyses compared employees in the pooled intervention (PIP) group to the pooled control (NPHP) group (three centers per group). Baseline . Data types included: (1) focus groups with team members; (2) focus groups with other nursing home employees; (3) in-depth interviews with individual team members and wellness champions; (4) in-depth interviews with management (administrators and directors of nursing); and (5) in-depth interviews with supervisors (department heads and unit/office managers) .Other evaluation materials included researchers’ field notes on observed indoor spaces , outdoor spaces (employee picnic areas and gardens), and printed materials  devoted to HP, OSH, or related activities or information. Researcher experiences and observations were logged after each field visit and consulted for purposes of this evaluation. Content analysis was performed using NVivo 9.0 software on transcripts from interviews and focus groups. We compared the themes that emerged across and within the six sites, focusing on (1) integration of HP and OSH, (2) comparative effects of the PIP and NPHP programs, and (3) sustainability of PIP as a model.-Process: Fidelity, type and number of activities.-Integration: Extent to which programs and participants understood and adopted the approach of combined attention to HP  and OSH issues .-Impact: Evaluated both at the organizational level (characteristics of the work environment and organization); and at the level of individual staff members .-Sustainability: How long the program lasted, indications of future plans or activities.Metrics for comparison and evaluation included:In the six centers, a total of 47 interviews were conducted with management (administrators and directors of nursing), supervisors (department heads and unit/office managers), wellness champions, and individual PIP team members. In addition, qualitative data were obtained from three focus groups of PIP team members, three focus groups of employees engaged in wellness at NPHP sites, and eight focus groups with other employees at the six facilities conducted in 2011 and 2012.According to the baseline survey of wellness champions, and consistent with the criteria for their selection, the NPHP centers had well-developed programs at that time, whereas the PIP centers had emerging programs, i.e., 1–2 activities loosely organized by staff. p < 0.01)  .The participatory teams were active with the guidance of the researchers. In the intervention sites , the number of regularly attending PIP team members ranged from 4 to 8 of the 10 original members at each site. No NPHP control site  had an active team or wellness committee engaging front-line workers.At the start of this project, not all facilities had a wellness champion appointed. However, by three years after initiation of the program, all six centers had wellness champions, as required under the corporate-sponsored wellness program. Wellness champions were individuals appointed by site administrators to coordinate HP activities in addition to their regular duties. In the PIP centers, these were office staff in ancillary non-supervisory jobs like human resource payroll benefits, medical records manager, and data coordinator. Wellness champions in NPHP centers were all in supervisory positions . Interviews and focus groups showed that many staff in the PIP centers knew the identity of their wellness champions. In two of the three NPHP centers, focus group participants were unaware of their wellness champions. The three PIP teams met every two weeks for one hour. They were planned to involve only non-supervisory personnel, and they began as such. However, at two of the centers, (I-1 and I-3), they were later expanded by managerial decision to include supervisory and administrative employees. PIP teams were guided to utilize the program planning form and project proposals for approval from center administrators. The use of these forms and proposals became less consistent towards the end of the 5 years as some supervisors and managers became members of these participatory teams.In each of the PIP centers, the staff members who joined the teams readily voiced acceptance of the integration concept, i.e., that the obstacles to good health resided both in and outside the work environment. Issues related to both OSH and HP were identified in team meetings. These issues largely mirrored the results of the focus groups at the same centers. The discussions within each team, facilitated by the researchers, demonstrated their operational grasp of the connection between work organization, psychosocial stressors, and personal well-being. Many activities that they carried out represented on-going and systemic attempts to address these concerns in the work environment . These iAt intervention center I-3, when the PIP team spearheaded a gardening project, the original motivation was healthy eating. The team members then also used the garden project as a prototype for developing good proposals and presenting them to management for support and funding. Once it began, they discovered that the garden had many other benefits, including team-building, communications, exercise, stress relief, and a potential for fresh produce for residents as well as staff. In contrast, in the NPHP centers, there were no projects designed to address up-stream work organization, psychosocial stressors, ergonomics, or work environment factors. Activities tended to support individual behavior change, e.g., coping, relaxation, and exercise . Correspondingly, none of the interviewed wellness champions in these centers demonstrated an understanding of how features of the job or workplace might influence health behaviors, or any vision as to what integration of OSH and HP might entail. At the organizational level, as discussed above, a markedly larger number of activities was carried out in the PIP centers, compared with the control sites. The PIP teams had a number of positive impacts on their health environment at work. All three, independently, addressed lack of healthy food options as a priority and were able to obtain healthier food choices in vending machines. In one facility, the kitchen agreed to provide soups, salads, and sandwiches at reduced cost to employees. One team initiated the creation of a community garden.The post-intervention surveys demonstrated slightly more organizational changes in the PIP centers than the NPHP centers . These iFrom follow-up survey data, the PIP sites had slightly more employees participating in company exercise (18%) and nutrition programs (25%) than in the NPHP group. There was also notable participation in team-sponsored gardening (9%) and healthy back training (8%), and utilization of outdoor furniture niches set up by the teams for mental relaxation (14%). At the individual level, there was no within-person difference in self-reported health status  from pre-intervention to post-intervention in either group . There wp < 0.05) between groups for any of these metrics (data not shown).Self-efficacy for eating a healthy diet, avoiding fatty foods, and exercise worsened slightly in both groups over time. The NPHP group gained and the PIP group lost self-efficacy for losing weight, compared to their baseline ratings. Neither group had a change in self-efficacy for managing stress, avoiding smoking or alcohol, and there were no statistically significant differences (p = 0.001) than the NPHP group at baseline, while it increased slightly in the NPHP group relative to the PIP group  were mentioned as important in encouraging or discouraging team meetings and activities. Support from management (administrators and nursing directors) and from supervisors (department heads and unit/office managers) were examined separately.Overall, the NPHP centers demonstrated higher management support than the PIP centers. Administrators in PIP centers discussed HP as part of their everyday work and considered their wellness champions and teams as part of their organizational structure (except in I-2).Supervisor support was perceived to be higher in two PIP centers, I-1 and I-3, and low at I-2. At the NPHP center C-2, many employees described their supervisors as being supportive to HP by allowing flexibility in their staff schedules and even covering for them while they participated in activities. Other supervisors within this center were described to be supportive by participating in activities themselves and motivating their staff to participate.PIP teams were able to provide more activities for the staff members with the presence of leadership support. When leadership support was absent, management in these centers talked about being faced with other pressures, needing to make decisions in favor of other more urgent projects and programs.Management changes in two PIP centers led to combining the existing PIP teams with other, previously inactive committees . In both cases, the non-supervisory PIP teams were converted to administrator-directed committees with different agendas and priorities.Staff participation in programs and activities is obviously a key measure of program impact as well as likely sustainability. Perceived lack of staff participation in PIP teams and activities was evaluated from the PIP team member interviews and focus groups.All PIP team members stated in the focus groups that clinical staff had difficulty in getting time to attend team meetings or activities. Staff participation in team-sponsored activities was poor within center I-2. Researcher experience and logs showed that participation in PIP teams had been quite high at the start of the project, yet involvement of clinical staff diminished due to frustrations with their decision-making process and clinical responsibilities.Employee participation in all three PIP teams dropped to a low number of non-supervisory clinical staff at the end of year 3 of the intervention. Low participation of clinical staff in activities were attributed to staffing shortages, time constraints, and clinical care responsibilities.Resources mentioned in the interviews and focus groups included financial support from the corporation, in-house/in-kind personal effort, and outside support. Although wellness was a stated goal for both the PIP and NPHP centers, no funds were allocated in any of the facilities’ budgets for employee wellness except in the first year (2008), when the corporation provided $700 per year for each facility. After this line item was dropped, most facilities engaged in regular fund-raising for their programs and many of the key informants expressed frustration at the lack of funds for their programs.In the PIP centers, several projects and especially the higher cost projects simply could not be implemented without adequate resources. At one PIP site (I-2), much of their past activity had focused on fund-raising, which detracted from effort and time that could otherwise have been spent on health and wellness activities.Using existing in-house resources was an opportunity for the PIP teams to benefit from the knowledge and skills of individual staff members. At Center I-1, one of the nurses offered yoga classes and another offered massages; at I-3, the dietitian offered many in-house programs and services including a weight loss program, potlucks, and healthy recipes.Team members in the PIP centers identified researcher involvement and guidance as a key outside resource and a vital link to sustainability of the teams. At center I-2, most members believed that the PIP team would not continue without researcher support. The research team provided material support to all three PIP centers, including ergonomic training sessions to staff members, tools and seedlings for the garden, and an exercise class instructor.Among NPHP centers, the primary forms of outside support cited were discounts for gym membership, health information, and free health screenings by group health insurance companies. In Center C-2, respondents also cited outside support from community programs along with insurance company services.Communication was mentioned in the qualitative data as factors that are essential for sustaining a participatory health program.Communication between the PIP teams or wellness champions with management as well as with the employees was uneven in both the PIP and NPHP centers. Employees in the PIP centers reported having good communication except at the I-2 center, where both the employees and managers expressed lack of communication as one of the largest stressors and most critical barriers to successful program implementation. The administrator at center I-2 concurred about the lack of communication between the PIP team and management. There appeared to be a trend towards better communication in centers where there was focused attention on work organization issues. In centers where closed-door management meetings were opened to staff or where the wellness champion utilized several methods of communication , it was viewed as a smaller problem or not a problem at all.Despite the overall group differences, one NPHP center (C-2) exhibited some characteristics and activities similar to the two positive PIP centers from the qualitative data from this center. Even with no official team at this center, the few employees involved in planning the activities had positive management support and were successful in obtaining high staff participation. The administrator, supervisors, and employees appeared to see wellness as a part of their organizational culture. The employee focus group indicated that most of the employees at this center had very few complaints. Most people agreed that good communication existed between management and employees and between the people planning the wellness activities and the rest of the management and staff. While employees at other NPHP facilities appeared uncertain about the idea of integration, the facility manager at C-2  led the safety committee and demonstrated interest in wellness. This manager shared ideas with the researchers about opportunities for integrating OSH and HP within the center.This study evaluated a participatory, integrated OSH/HP intervention in three skilled nursing facilities compared to three others with more traditional, non-participatory HP programs. Mixed qualitative and quantitative data collected over a four-year follow-up period demonstrated program feasibility, good process fidelity while the researchers were actively involved, meaningfulness of the integration concept to worker representatives, and moderate program impact on some organizational conditions of work. Sustainability, however, suffered due to lack of resources and inconsistent manager support.Process fidelity was high initially in all three intervention sites; the program was introduced in a uniform manner by the researchers and proceeded as intended for the first two years or so. Members of the PIP teams were highly motivated and responsive to the organizing principles of worker priority-setting and a combined focus on both work and non-work obstacles to health. The center administrators permitted front-line staff to volunteer for the teams and assisted with the logistics of scheduling meetings, although use of paid work time for meetings was inconsistent. Involvement of non-supervisory clinical employees in the planning of workplace HP projects was high in the PIP centers at the start of the research project, although it diminished subsequently.Compared to the traditional wellness programs in the control sites, the participatory teams in the three PIP centers were far more likely to develop activities with a broad scope, encompassing elements of both OSH and HP. Over time, the teams in the PIP centers addressed work organization, psychosocial stressors, physical ergonomics, in addition to taking an organizational approach to HP goals such as improving the food environment at work. In contrast, the NPHP centers primarily supported individual behavior change, with minimal attention to psychosocial stressors or the work environment. Consistent with our findings, a previous intervention study reported higher blue-collar worker participation with OSH/HP interventions compared to standard HP interventions . In partThe number and variety of workplace HP activities initiated during the study period were higher in the PIP centers than the NPHP centers. Two of the three PIP centers indicated improvement in organizational factors. There were no larger corporate influences that would have produced these positive changes specifically in the three PIP centers, so it seems reasonable to consider them at least partially as outcomes of the integrated participatory program.There was no evidence of major change in chronic illness incidence or the perception of health status or behaviors following this participatory intervention, but the four-year follow-up period was too short for any such difference to be expected.The study hypothesis was that the PIP teams would have more impact than the NPHP programs. In a participatory approach, employees are actively engaged in problem identification, decision making, implementation, and evaluation of the program . This apOverall leadership support is widely recognized as crucial factors for a sustainable workforce health program of any type ,32,41 anUnfortunately, the initially high level of administrator supports and staff participation in project planning diminished somewhat over time. One of our goals was to incorporate the teams into other active committees with similar interests, in order to increase their long-term sustainability. In the two centers where this occurred, the teams were absorbed into committees without responsibility for employee health.Administrator changes also negatively impacted management support, financial resources, and time release for program participation—all identified as important for progress of the participatory program . ChallenAll six centers lacked financial resources to sustain even basic wellness programs, such as paying for instructors in yoga, meditation, or fitness. It did not appear that employee health  was perceived to generate a high enough return on investment to be sustained in this company.A major strength of this study is the detailed information obtained from various data sources and the triangulation of qualitative and quantitative data. Mixed methods research is valuable because it captures the information from various perspectives and can support qualitative and quantitative findings . This prOn the other hand, evaluation of the PIP’s impact was limited by the fact that the sites were not selected at random. The three PIP sites were volunteered by their administrators in response to a recruitment effort by the investigators through a key regional staff member. The three NPHP sites were also selected by the same regional representative, in response to the research team’s specified criteria, and then approached for management agreement to permit data collection. As a consequence, there were some anticipated baseline differences between the two groups, both in prior HP activity and possibly in administrator interest in and initiative toward workforce well-being. A further issue is that the study results are not expected to be generalizable, except to other nursing homes with similar management interest and willingness. Nevertheless, the results do demonstrate the feasibility of conducting a participatory change process in this sector, despite  tight staffing and scheduling, coupled with low union representation to protect job security for those who voice their opinions about root causes of health and safety problems.In theory, the gold standard for an intervention study is the randomized controlled trial. However, the benefits of randomization in reducing confounding are not realized except with a large sample. In this case, the intervention was carried out at the level of the entire facility (PIP). In practice, it was not logistically or economically feasible for us to enroll a large number of facilities for such an intervention. Even with some alternative designs, such as the stepped-wedge , there iThe other advantage of a randomized controlled trial is that with double-blinding, the possibility of information bias can be greatly reduced. However, blinding of participants and researchers is also infeasible for organizational-level interventions. In turn, randomization and blinding may have disadvantages for organizational-level interventions, such as limited capacity to assess multi-dimensional interventions or evaluate process, quality, or performance, and incompatibility with community trust, choice, and participation often needed for successful program design and evaluation ,45,46.Another limitation of this study is the difficulty to show the actual impact of the participatory program due to constant changes in the organization. For example, turnover in leadership in the study sites appeared to affect program success with administrator changes in the PIP sites during the study period. Similarly, employee turnover limited our ability to examine within-person changes over time, through reduced statistical power.Participatory programs can be challenging to implement and evaluate in a research context because key elements cannot be controlled by the investigators; for example, interventions are selected and designed by workers after the program is already underway, and interventions addressing higher-level organizational obstacles may provoke institutional resistance that might not have been visible or even present at the beginning of the project. These issues were known in advance and cannot be prevented even when they are anticipated. It will always be the case that many organizations and workplaces will refuse voluntary worker health improvement efforts, even when resources are offered to facilitate the program. The criteria for selection of participatory intervention sites have been revised on the basis of this experience, in an attempt to better inform decision-makers in advance about the expected process and its potential benefits and costs.The intervention program had some positive impacts on work organization in the intervention centers. The fundamental Total Worker Health concept of integrating OSH and HP was intuitive to many workers; they enthusiastically envisioned and sought to carry out integrated programming, and a number of activities improved the health climate in these workplaces. Active involvement of non-supervisory employees in program design and conduct appeared to benefit the work environment and employee morale and engagement. Actively engaged leadership was no less important: program intensity and success fluctuated noticeably with changes in management. Both managers and employees cited the importance for success of factors such as employee program ownership, empowerment, and skill-building .Unfortunately, our planned reduction in researcher facilitation efforts was followed by an erosion of the previously high level of staff participation in project planning. It was disappointing to observe the decline in company commitment to the participatory employee teams despite their demonstrated feasibility and robust worker engagement, in contrast to the company workplace health promotion program.Participatory OSH/HP is challenging in the long-term care sector due to highly demanding jobs and tight staffing. Managers and front-line workers have different perceptions of the long-term care environment , likely PIP team resources, breadth of worker participation, and management support were important preconditions for potential program sustainability. Future efforts should incorporate more robust organizational structures to enhance these factors for program success. Lessons from this study may guide other long-term care facilities to build a sustainable, integrated, and participatory program.