Several studies have demonstrated that cultural competence improves patient-provider communication, which promotes adherence to established care plans and improves patient satisfaction and health outcomes. However, there is very little data available regarding the costs associated with the development and implementation of cultural competence training, or the cost-effectiveness of these programs. To that end, this evaluation aims to describe costs, program effectiveness, and cost-effectiveness of a culturally tailored communication training program to improve cultural competence among multi-disciplinary care management teams.As part of a region-wide quality improvement initiative to reduce healthcare disparities among African American patients with uncontrolled hypertension, three multi-disciplinary care management teams were invited to participate in a two-part communication training program. A paired samples t-test was used to assess program effectiveness based on participant responses to a validated cultural competence self-assessment survey 2 weeks before and after the training program. A micro-costing approach was used to estimate programmatic costs for content development and delivery. Cost-effectiveness was then determined using the average cost-effectiveness ratio, and sensitivity analyses were conducted to assess the impact of participant mix on this result.n = 17) improved after training; however, only the culturally competent behaviors (CCB) subscale change was statistically significant (p = 0.02). Overall program costs were $5754.19. The average program cost per participant was $138.51, with an ACER of $337.83 per 1-unit increase in CCB score. Sensitivity analyses yielded a range of ACERs between $122.59 and $457.07, where all participants are support staff or nurses, respectively.All scores (Culturally tailored communication training increases how frequently participants demonstrate culturally competent behaviors and may be a cost-effective intervention for care management teams to address individual cultural competence. Detailed costs associated with cultural competence training are largely unavailable in the literature; as such, these data may serve as a financial framework for organizations considering the implementation of similar programs. In 2002, the Institute of Medicine released a groundbreaking report, Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care, that concluded racial and ethnic minorities in the United States receive lower quality of care and have higher rates of morbidity and mortality from chronic diseases than their white counterparts . Nearly Within the same groundbreaking report, healthcare provider training was identified as one of the most important tools to overcome disparities . More ofSeveral studies have tested the effectiveness of cultural competence training on intermediate outcomes, such as provider knowledge, attitudes, and behaviors, and long-term outcomes, like improvement in clinical measures; however, very few studies have reported detailed costs of cultural competence training , 7. In oAs part of a region-wide quality improvement initiative to reduce healthcare disparities among African American patients with uncontrolled hypertension, a two-part culturally tailored communication training program was developed and delivered to care management teams.Three care management teams, comprised of 14 registered nurses, 15 registered pharmacists, and 7 non-clinical support staff, were invited to participate in the training program and self-assessment. Prior to the initiation of the program, all participants were informed that their participation in the program and assessment was voluntary.The training was adapted from a series of resources and similar quality improvement initiative developed by, and with, Kaiser Permanente’s Equitable Care Health Outcomes team that incorporated an evidence-based communication model  and general awareness of African American culture and barriers within the healthcare system . The newr = 0.85, p = 0.002) and for each of the subscales, culturally competent behaviors  and cultural awareness and sensitivity  and acceptable reliability  [The primary effectiveness outcome was estimated using a paired samples t-test based on the change in self-assessed cultural competency from baseline on one index and two scales within the Cultural Competence Assessment (CCA) instrument before and after the training program. The CCA was developed to assess cultural competency in a multi-disciplinary setting, with adequate test-retest validity overall ( = 0.89) , 11. TheProgram costs were calculated using a micro-costing approach, where resource units are multiplied by unit costs. All participants were employees of Kaiser Foundation Health Plan at the time of training. Administrative time to develop and deliver the intervention was tracked throughout the program. Participant salaries and fringes were estimated using the most recent Bureau of Labor median occupation wages for the Atlanta-Sandy Springs-Roswell, GA metropolitan service area for the state of Georgia and employer costs for employee compensation, respectively , 13. ForThe cost-effectiveness analysis was from the employer perspective. Mean intervention costs (the costs associated with the training program) and mean effectiveness gained  were used to calculate the average cost-effectiveness ratio (ACER)Eq.. The ACESensitivity analysis was applied to establish how the costs and cost-effectiveness of the program might vary from the base-case for three participants mix scenarios: all nurses, all pharmacists, or all support staff.Institutional Review Board review was not sought for this work, as the authors determined the project did not meet the Federal definition for human subjects research, instead, falling within the scope of quality improvement. Quality improvement initiatives are exempt from Institutional Review Board review, by definition, as these initiatives do not aim to create new knowledge, per se, but rather use existing knowledge to improve health outcomes within a local system , 18.n = 34) completed the web-based module and attended the group session. There were no observed differences between respondents and non-respondents.Of those invited to participate in the training program, 92% (n = 36) of participants before the training and 61% (n = 22) after the training. Of the participants that completed the self-assessment before and after the training, 47% (n = 17) disclosed enough information to match responses for inferential analysis  and after training ; t (16)=1.74; p = 0.09. Similarly, no statistically significant difference was observed between CAS scores before  and after training ; t (16)=1.74; 0.46. However, there was a statistically significant difference between CCB scores before  and after training ; t (16)=1.74; p = 0.02.There was no statistically significant difference between overall cultural competence scores before , and rental space. Even so, the Office of Minority Health offers many educational resources at no charge through its website, Think Cultural Health, which organizations may use to improve cultural competence across many different healthcare roles [Because this was a new program and there were no active programs to which a comparison could be made, the cost-effectiveness analysis used for this program evaluation was the ACER. The use of an ACER to report the results of a CEA are likely less well known to the reader than the use of incremental cost effectiveness ratios (ICERs). However, well-known organizations have recommended the use of ACERs for similar purpose and the evaluation of de novo interventions, e.g. the World Health Organization recommends using ACERs for programs where resource allocation is limited by budgetary constraints and the Centers for Disease Control and Prevention has published an ACER-based CEA of West Nile vaccination , 21.The average cost of the program per participant was $138.51. The ACER of the training program was $337.83 per 1-unit increase in program effectiveness. Program effectiveness used in the calculation of the ACER was based solely on the culturally competent behavior (CCB) subscale, as this was the only statistically significant improvement observed in our analysis. Most cost-effectiveness analyses (CEAs) use quality-adjusted life years (QALYs) with commonly agreed upon thresholds upon which cost-effectiveness is compared. There are few thresholds available for CEAs based on natural units; instead, the costs of the program are often compared to the costs of an occurred or avoided event. There is no readily available data regarding healthcare costs associated with the frequency of culturally competent behaviors which leaves organizations to subjectively assess the worth of increasing the frequency of nurses, pharmacists, and non-clinical support staff from “often” to “somewhat often” or “very often” to “always”.When participant role and effectiveness by role was altered, the ACERs ranged from $122.59 (support staff only) to $457.07 (nurses only). These ACERs underscore the impact of balancing financial investment in the training program and the expected outcome of the program. For instance, pharmacists reported the largest improvement from baseline (0.83) yet their salaries tend to be roughly 65% higher than nurses and nearly triple that of support staff, which yielded an ACER one-half that of the nurses-only scenario. The lowest ACER, $122.59, represented the support-staff only scenario and may present the greatest return on the organization’s investment as some case studies have demonstrated the role that these frontline individuals can play in improved clinical and financial outcomes [There are several limitations of this evaluation. The small sample size significantly limits our ability to confidently demonstrate an effect on cultural competence because of the training. However, it is likely that other programs would implement this type of training in groups of similar size in accordance with standards set forth by accrediting bodies like The Joint Commission or professional associations like the American Nurses Association , 24. TheDespite these limitations, we believe this paper provides substantive contributions to the vast body of cultural competence training outcomes. The outcome of the program is congruent with other programs that aim to improve skills and behaviors. Moreover, this is the first cultural competence training evaluation that includes micro-costing and cost-effectiveness outcomes, which may serve as a financial framework for other organizations considering the implementation of a similar program.


Studies focusing on the Integrated Management of Childhood Illness (IMCI) program in the Philippines are limited, and perspectives of frontline health care workers (HCWs) are largely absent in relation to the introduction and current implementation of the program. Here, we describe the operational challenges and opportunities described by HCWs implementing IMCI in five regions of the Philippines. These perspectives can provide insights into how IMCI can be strengthened as the program matures, in the Philippines and beyond.n = 46) in five provinces , with full transcription and translation as necessary. In parallel, data collectors observed the status (availability and placement) of IMCI-related materials in facilities. All data were coded using NVivo 12 software and arranged along a Social Ecological Model.In-depth interviews (IDIs) were conducted with HCWs  facilities: 1) insufficient financial resources to fund program activities, 2) inadequate training, mentoring and supervision among and for providers, 3) fragmented leadership and governance, 4) substandard access to IMCI relevant written documents, and 5) professional hierarchies that challenge fidelity to IMCI protocols.Although the IMCI program was viewed by HCWs as holistic and as providing substantial benefits to the community, more viable implementation processes are needed to bolster acceptability in PHC facilities.The online version contains supplementary material available at 10.1186/s12913-021-06209-6. The Integrated Management of Childhood Illness (IMCI) program was developed by the World Health Organization (WHO), the United Nations International Children’s Emergency Fund (UNICEF) and other partners to reduce global child mortality and improve health care workers’ (HCWs) ability to provide holistic healthcare to manage childhood illnesses –4. SinceMulti-country evaluations underline the manner in which IMCI strengthens health systems , 12. TheWhile the IMCI program has developed evidence-based practice and contributed to global reductions in child mortality, its impact on reducing health inequity  is difficult to assess given the varied and uneven extent of implementation , 18. SimAlthough the benefits of IMCI are well established, less attention has been paid to how front-line HCWs perceive the strategy, where they see implementation gaps, and how they describe opportunities to improve the program within primary health care (PHC) facilities moving forward .The Philippines was among the first countries in the WHO-Western Pacific Region to adopt the IMCI program (in 1996) . An evalThe WHO evaluated the status of IMCI program implementation in 2013 and determined that the Philippines had effectively adopted key technical policies and guidelines, with coverage in 72% of districts . We are As the Philippines accelerates toward the attainment of Sustainable Developmental Goals (SDG) and Universal Health Coverage (UHC), it is critical to learn HCWs’ perspectives and experiences with this system-wide program. Such insights can inform strategic policy recommendations and bolster more inclusive IMCI implementation. This study aims to describe qualitatively the experiences and opinions of HCWs in implementing the IMCI program in five regions of the Philippines.This qualitative study was part of a broader, cross-sectional, mixed-methods research project examining the current state of IMCI program implementation in the Philippines. The Department of Health (DOH) – Disease Prevention and Control Bureau commissioned the Research Institute for Tropical Medicine (RITM) to assess the current status of the IMCI program in the country. We purposively selected five regional sites based on key characteristics . One province was selected per region: Ilocos Sur in Ilocos Region (Northern Luzon), Quezon Province in Calabarzon Region (Southern Luzon), Bohol in Central Visayas (Visayas), Davao del Sur in Davao Region (Mindanao) and the National Capital Region Fig. . This stAn inventory of all health facilities from the selected provinces was conducted over a five-week period (no listing of PHC facilities implementing the IMCI program was made publicly available prior). PHC facilities located in areas that are geographically isolated or in the midst of civil unrest were excluded during this preliminary phase.Research staff then visited PHC facilities to gather information pertaining to HCWs’ IMCI training and the program’s implementation status. IMCI-implementing PHC facilities were defined as those with at least one IMCI-trained professional  currently implementing IMCI. HCWs stationed in selected PHC facilities at the city or municipal level who had experience implementing the IMCI program were purposively selected. We excluded HCWs who were trained, but did not implement or discontinued implementing the IMCI program.A team of 6 data collectors, who each possess 4–10 years of qualitative research experience and have professional backgrounds in nursing, health and/or social sciences, collected data from June to December 2017. Since the study locales were in different regions, five teams were created comprised of research staff from the Department of Epidemiology and Biostatistics (DEBS) of the RITM.n = 46) in their respective PHC facilities, after obtaining written informed consent. A semi-structured interview guide facilitated interviews  to bolster familiarity with the content of the transcript, prior to the start of coding (done using NVivo 12) . As broad themes emerged, multi-level facilitators and barriers to IMCI implementation were identified and placed within the framework of the Social Ecological Model (SEM) . Themes Data analysis was guided by the tenets of constructivist grounded theory (CGT) following the analytical process outlined by Charmaz . The lean = 34; 73.9%); 58.7% were medical doctors while 41.3% were nurses  with a range of 1–20 years (mean: 9.87 years) of experience implementing IMCI. A majority (67.4%) of participants were involved not only in the implementation of IMCI, but also in program supervision and monitoring.IDIs with 46 participants across five regions are included in this analysis Table . A majorHCWs spoke of the perceived benefits of IMCI Table , describBarriers to implementation emphasized insufficiencies in the health system including: a low number of skilled HCWs, an insufficient number of IMCI refresher courses, and inadequate supervision and monitoring Table . HCWs alParticipants stated with certainty that IMCI brought about historic reductions in childhood morbidity and mortality Table  largely Participants spoke extensively about the damage that natural disasters caused on their health facility, and how this affected every aspect of health service provision including the existence of IMCI-related documents, equipment and recordings. Some participants said that typhoons and earthquakes damaged everything they needed for implementation  by being “washed-out” by floods  and “destroyed” by earthquakes  which they cited as among the reasons why IMCI “disappeared”  and why some PHC facilities were unable to cope and thus “poorly implemented” IMCI . Communities often became unreachable because of destroyed roads and bridges, leaving them “isolated” , when health services were needed most. When asked how they responded and reached out to communities amid disasters, participants often sighed and conveyed disappointment that the only option was to wait for the rain and floods to subside.Philippines-DOH guidelines require that specific documents related to different programs, such as IMCI,  be stored in health facilities. However, most IMCI documents were not present within the PHC facilities and/or were present but lacked ready access or proper filing. Our observations found that 2 out of 6 key WHO-recommended IMCI-specific documents  were available in sampled health offices across regional sites Table . Other IParticipants described how the unavailability of IMCI documents negatively affected implementation. Some participants mentioned that they did not receive any specific IMCI documents or some simply “cannot recall”  or “don’t know”  the status of documents. When asked if this affected implementation, participants expressed uncertainty. Without IMCI-specific documents, HCWs were not able to appreciate the fact that earlier guidelines on acute respiratory infections, control of diarrheal diseases and malnutrition have been integrated into one – the IMCI program. Further, some participants conveyed disappointment that local governments were not informed of the IMCI-specific department administrative orders (i.e. guidelines and standard procedures prescribed by the DOH), causing difficulties with budget allocation and synchronization to other child health programs.Some participants received documents that were not IMCI-specific, but involved processes and information used in IMCI assessment, such as immunization, deworming and feeding practices. When asked if they were familiar with these IMCI components, we observed and recorded that some participants laughed and seemed embarrassed, sharing that they had forgotten or had not fully read everything.Difficulties in budget allocation were identified as a major deterrent to effective IMCI implementation. The IMCI program in most sampled provinces did not entail an explicit financial plan with earmarked funding. Participants explained that the planning and budgeting for the IMCI program is centralized, and although they receive a budget , it is in the form of “lump sum money” . Absent an allocation of funds explicitly for IMCI, participants described the challenge of weighing the value of IMCI, viewed as quite costly, relative to other programs .IMCI coordinators at municipal levels described relying on the DOH Central and Regional levels for planning, budgeting and scheduling of IMCI training . Because of limited budgets for training, only few staff can be sent to training and precedence is often given to a select few  who can then cascade the training to other HCWs).When asked about what training they received, most participants mentioned that they were trained using the original training program - an 11-day, basic, face-to-face directed approach to establish expertise on illness management algorithms. Some participants said the last training had been conducted 2–5 years before, meaning they had not received any refresher course and/or new staff were not trained. Participants described a shortage of HCWs trained in the delivery of the IMCI program due to quick turnover of trained staff, who either retired or moved to other facilities or migrated abroad. Participants also described how, due to the expense and logistical difficulty of training new staff, it was hard to contend with an out-migration of IMCI-trained staff. Several participants lamented the long duration of IMCI training (11 days) and the strain it put on facilities when a staff member left for training. Most participants stated that they had not received refresher training. One region in this study conducted a refresher training for HCWs; the remaining regions used monthly meetings as a venue for informal IMCI updates.Because of the perceived effect of the basic training’s long duration, the DOH introduced the IMCI Computerized Adaptation and Training Tool (ICATT) nationally in 2005. ICATT uses a self-directed approach to provide a shorter version of the basic IMCI training program. When asked about their experiences with ICATT Table , most paOur records review confirmed that only a few PHC facilities keep records of trained IMCI staff; some participants said that they did not receive training certificates.Most medical doctors interviewed were aware that after IMCI training, a follow-up visit should be made to all HCWs, within 4 to 6 weeks of training completion. The monitoring visit was supposed to be done by supervisors from either the regional or provincial offices, to ensure that newly trained staff correctly implement the IMCI protocol. However, our data suggests minimal monitoring, with only one region receiving follow-up from regional or provincial offices. When participants who were meant to do monitoring and supervision were asked about this gap, they cited a lack of time, a lack of training on how to conduct formal supervisions, and/or an absence of a formal monitoring system or routine. Absent any structure on how or when to conduct formal monitoring, supervisors described devising their own monitoring mechanisms, which they adapted from other health programs.Every participant in this study described a lack of IMCI-specific data or reporting, which they felt hindered an ability to track IMCI’s progress and gaps. Participants described how all data is integrated within the existing Field Health Services Information System (FHSIS), which is the current Routine Health Information System (RHIS) that the DOH employs nationally. Some supervisors recalled difficulties of teasing out child health indicators from the RHIS as the system is managed regionally or nationally (thus beyond the reach of supervisors) and supervisors were not trained on how to extract relevant data. Further, most participants said they believed that the health system does not support the alignment of the IMCI program with other competing maternal and child health programs as each program has their own reporting, monitoring and documentation forms.Participants explained that they all receive essential IMCI medicines from the DOH central office. In addition, participants described that the local government units (LGUs) are authorized to procure drugs, but receive guidance from the DOH central office. While essential drugs were largely accessible, participants said they had problems with the procurement system, as it is a “very long and slow process” , and requires “a lot of documents to be submitted”  which causes drug stock-outs.When faced with drug stockouts, most participants said they had no choice but to write prescriptions and force caretakers to “buy the rest” . Participants described how caretakers often lack the necessary funds to buy medicine, and children are thus untreated or unable to complete a full course as recommended.In most sites, participants described a workaround to manage the inadequacy of medicines, which they called “priority allocation.” This term describes decisions made at the city or municipal health levels on the amount of medicines sent to specific facilities based on the total population and number of cases indicated in annual or monthly reports. This method determines the quantities and types of medicines and supplies required in lower levels. Priority is also given to municipalities with communities classified as GIDA. In this way, those who need the most were given proper allocation to manage childhood illnesses, which mitigated but did not solve the drug procurement issues described by HCWs.Participants described aspects of health facility structures or equipment, which impeded or delayed the uptake of IMCI services. Although most participants said they received adequate supplies, these were not for the IMCI program alone and had to be shared with other childhood programs. Also, there were some descriptions of substandard supplies and equipment such as broken thermometers and weighing scales, which in turn, added more costs to health facilities, which needed to procure new equipment or pay for repairs. Some HCWs also mentioned an unavailability of IMCI forms, which is a tool to facilitate the correct classification and thus appropriate management of illnesses. Initially the DOH Central Office provided IMCI forms, but the responsibility was later passed onto LGUs whereupon forms were no longer readily available or adequate in number. As a result, HCWs used personal funds to buy, prepare or photocopy forms.Some of the participants also said that the health facility itself is not conducive and not built to accommodate the IMCI program. Some expressed difficulties performing the consultation and counseling sessions, as they lack an “allocated corner to occupy” .HCWs said that because of the low community awareness of the IMCI program, they experienced some problems relating to treatment management. Caretakers sometimes questioned HCWs’ capabilities to manage childhood illnesses and expressed dissatisfaction with the support they received. Some HCWs described how caretakers had grown accustomed to receiving antibiotics and other medicines, so they were frustrated when they only received advice on increasing water intake or oral rehydration solution (ORESOL). HCWs also shared stories of parents whom they considered to be uncooperative because they did not follow treatment management protocols and/or attend clinic follow-ups.Participants described how caretakers and community members often prefer to see a doctor even if an IMCI-trained staff member, such as a nurses or midwife, would be equally well positioned to provide effective care.HCWs, specifically nurses, described difficulties with medical doctors because doctors’ clinical training did not always coalesce with algorithms of IMCI. Participants described how doctors would immediately prescribe antibiotics without following the IMCI algorithm as a means to save time. When probed on how they responded to these situations, one nurse said, “I’m just a nurse” ; others said they did not want to contradict the advice of doctors.Many participants said the IMCI process was “too laborious”  and placed undue pressure on HCWs who are responsible for all disease control programs, not singularly focused on IMCI. The IMCI assessment algorithm was described as complicated and time consuming; so too was the post-diagnosis consultation – “the exhausting part”  – which included a demonstration on how to prepare or administer medicines. The whole process could take 15 to 20 min per patient and HCWs described being bombarded with complaints from both patients and colleagues due to longer waiting times.Another frustrating reality for HCWs was the problem of reporting, as most participants said that there are some instances wherein it appears that they are only managing a small number of sick children but in reality, they are seeing more than the total reflected in the information system . Some participants said this registration approach created problems, as their municipal health officers sometimes scolded them for “few cases” . Participants described how this scolding prompted feelings of disappointment, undue criticism and frustration.This study is the first from the Philippines to qualitatively describe HCW perspectives on the key challenges to IMCI implementation in PHC facilities, 20 years after the strategy’s introduction. IMCI stresses three mutually dependent components: improving HCWs skills, health system support, and improving family and community practices , 35. HowThe key barriers in this study relating to the weakness in the IMCI program execution amid health-system constraints are consistent with several country evaluations , 36, 37,appearance of lower monthly patient counts  [Other studies further concur with our findings regarding demotivated staff, gaps in training and fragmented communication and supervision from central authorities , 38–41. alities) , 39. Ouralities) .Our findings also resonate with studies that describe HCW frustration with governments or donors shifting priorities away from IMCI in favor of vertical programs such as EPI, HIV and tuberculosis , 44, whiIMCI was originally framed as a strategy to empower communities , 52. OurThere is a need to ensure that provincial, district and municipal health management are supported and given adequate fiscal space for IMCI. Drawing on evidence from Nepal and the Democratic Republic of Congo, Doherty and colleagues , emphasiIn addition to improving care for children, district health systems empowered to deliver IMCI can be leveraged to improve the overall existing health system , 57 and Experiences from other LMICs including Ethiopia and South Africa provide insights into how the Philippines could address low training coverage including: improving pre-service training across all cadres with special focus on nursing and midwifery, strengthening the training of trainers and incorporating skills on supportive supervision and mentoring, instituting regular ICATT training with group discussion and demonstration of skills, and intensifying the follow-up for those trained every 2 years –64. InteDiversifying ways to reach untrained HCWs, and creating alternatives to intensive, costly, in-person IMCI courses and ICATT may prove valuable. Other countries, such as Tanzania and Burkina Faso, have used electronic IMCI (eIMCI) and digital IMCI (dIMCI), which entail the use of personal digital assistants, smart phones or tablets to assess and classify patients and to train HCWs remotely –67. The Our findings related to low community awareness and a desire among HCWs for communities to better understand the program and its tenets is echoed elsewhere . EvidencA study in Armenia, which involved employing health campaigns via trained community peer health educators, found a statistically significant impact on communities’ knowledge, practices and acceptance with respect to child health . In thisOur study demonstrates opportunities for more operational and implementation research to understand the adaptation of IMCI for health facilities and services at all levels of the health care system. In particular, policymakers’ perspectives on implementation could provide insights into differences across provinces, practice settings, and HCW cadres. This, in turn, could inform an adaptation of the IMCI program to bolster its relevance and responsiveness to the changing disease landscape , 58.This study has some limitations. We set out to describe the experiences of HCWs in implementing IMCI in the Philippines; however, the bulk of questions during IDIs were focused on challenges and less on the facilitators to IMCI implementation. Although data saturation was reached, which allowed us to address the aims and objectives relevant to this paper, future work could expand upon these research questions by examining, for example, whether there are different perspectives regarding IMCI based on recency of IMCI training, or comparing across types of health facilities.Experiences of HCWs in the Philippines in implementing IMCI showed the demotivating realities of a rather unsupportive health system and, 20 years in, weak or diminished program enthusiasm. Our findings suggest that HCWs in PHC facilities struggle to provide optimal IMCI services due to poor working conditions and the absence of IMCI institutionalization, notably including a lack of a specific budget allocation at district and PHC levels. These conditions affected how the IMCI program was prioritized and executed. We recommend several opportunities to improve the working conditions of HCWs and therefore the delivery and implementation of IMCI in the Philippines, emphasizing the importance of building capacity for local ownership of service delivery in communities, providing training, and equipping community health teams with technical support and monitoring. Further, our study demonstrates opportunities for more operational and implementation research to understand what works and does not work. Government leadership, along with community mobilization, is necessary to foster sustainable IMCI health care services to serve the needs of local communities.Additional file 1: Supplementary File 1. Semi-structured interview guide.


The Residency Match is becoming more competitive each year, and more than ever, medical students must prove themselves in the residency interview. Data from the 2018 National Residency Matching Program's Program Director Survey highlight the importance of the interview on Match performance. We developed a residency interview training program with the goal of preparing medical students for residency interviews, and we assessed the impact of the training on medical student confidence.Our residency interview training program includes (1) a short didactic session on the residency interview process, (2) an informational packet with commonly asked questions, (3) two 20-minute practice (mock) interviews, (4) 10 minutes of face-to-face feedback from interviewers, (5) a facilitated group debriefing, and (6) access to a participant's two videotaped interviews and a guided self-assessment. To evaluate the effectiveness of our program, we assessed student confidence with a pre- and postsurvey.We have been running our residency interview training program since 2014. Over the last 5 years, 73 fourth-year medical students have participated. When polled after completing their first authentic residency interview, students felt they had more knowledge of the interview process, better preparation, and more confidence in their skills to interview successfully; they also believed that the program improved their interview performance.Performance during the residency interview is the most important factor used by residency programs to rank applicants. Using our residency interview training program, medical students can improve their confidence prior to interviews. More specifically, for obstetrics and gynecology programs, 1,745 applicants competed for 1,336 residency positions, with an average of 1.31 applicants per position in 2018.2 Meanwhile, general surgery programs had 1,955 applicants for 1,319 resident slots, for an average of 1.48 applicants per position for the 2017–2018 residency application cycle.2 Thus, residency applicants are seeking avenues to become more competitive to attain a successful Match.The selection process for applicants to residency programs becomes increasingly competitive each year, as there is a steadily increasing applicant pool for a relatively stable number of residency positions. The 2018 National Resident Matching Program (NRMP) had a record-breaking total of 37,103 applicants for 30,232 first-year and 2,935 second-year residency positions.3 These data highlight the significance of the interview for applicant success in the Residency Match process.The application process consists of the standardized Electronic Residency Application Service (ERAS) application complete with United States Medical Licensing Examination step scores, medical school transcripts, a personal statement, faculty letters of recommendation, and a curriculum vitae, coupled with formal in-person interviews based on each program's selection criteria. This formal interview process arguably allows program personnel to best witness the intangible features of a student's personality and natural ability to communicate and connect with others. The NMRP surveyed program directors in all specialties across the United States for the 2017–2018 application cycle. Almost universally, program directors identified the three most important factors in ranking applicants as interactions with faculty, interpersonal skills, and interactions with house staff during the interview and program site visit.Because the interview portion is critical in a program's ranking algorithm, we developed a structured residency interview training program to prepare medical students for their authentic residency interviews. Our program adds to the literature on mock interviews by being more intentional, more formalized, and more structured. The goal of the training program is to prepare medical students for the residency interview by improving knowledge of the interview process, interview skills, and self-confidence in residency interview performance. Our program provides opportunities for knowledge acquisition, skill practice, feedback, reflection, and, finally, application in the authentic residency interviews. In the absence of an ethical way to capture medical student interview performance in authentic residency interviews, we use student self-confidence after the training program as a proxy outcome measure.We developed a structured residency interview training program that included (1) a short didactic session on the residency interview process , (2) an We aimed to run our residency training program near the end of September or early October in order to complete the training before authentic residency interviews began. First, a medical education coordinator identified a date when the majority of students could attend. Once the number of students participating was determined, the coordinator knew how many interviewers to recruit. The coordinator identified available interviewers and reserved the date, time, and space needed for the training program. Students submitted their ERAS application, a curriculum vitae, and a personal statement to the coordinator. The coordinator then created an interviewer packet that consisted of the following: the ERAS application, the curriculum vitae, and the personal statement for each of the students to be interviewed; a list of standardized, commonly asked interview questions ; and theStudents and interviewers were asked to arrive in professional attire. The didactic session was meant to mimic the group session given by the residency program director, except instead of talking about a specific residency program, the content was related to the residency interview process, interview skills, and interview dos and don'ts. A facilitator experienced with residency interviews such as a clerkship or program director presented this session. The session was approximately 30–40 minutes long and was a combination of didactics with guiding slides and a question-and-answer small-group segment. Students and interviewers then transitioned to the interview rooms. Five to 10 minutes were allowed for this transition. Students knocked on the door and were greeted by their interviewer and directed to a chair. Interviewers then initiated a 20-minute interview using the standard questions from After the practice interviews, interviewers and students met as a large group and participated in a facilitated debriefing using the resource in The facilitator and the interviewers were faculty who had experience with medical student, residency, and fellow interviews. This included faculty who held medical education leadership positions, such as a department chair, a vice-chair for education, a clerkship or program director, a clerkship or program associate director, or a fellowship director. If additional interviewers were necessary, faculty from other medical disciplines, fellows, or senior residents could be solicited.1 However, we believe our program would be generalizable to any residency applicant.Our sessions were done with fourth-year medical students who had submitted a residency application through ERAS. The training program occurred in late September/early October, before the authentic residency interviews began, and the students matched in March of the following year. Our training program was voluntary, not mandatory, and focused on general surgery and obstetrics and gynecology applicants due to the competitive nature of those specialties.To deliver our training program successfully, we used a medical education coordinator and a simulation technician from our simulation center. Our didactic and debrief sessions occurred in a large debrief room in our simulation center. The interviews took place in individual rooms within the simulation center. Each room was equipped with two chairs, one for each participant, and a video recorder. The simulation technician was responsible for recording the sessions, timing the interviews and feedback sessions, and alerting participants at the 2-minute warning and the end of the sessions.4 Our internal medical education experts felt the content in these three sources was representative of the residency interview.The didactic slide presentation , the infThe interview performance assessment or evaluation was created internally, by asking local program directors for a copy of the evaluation form they used for their residency interviews. We then created the form provided in 5 This script was specifically designed for new debriefers and could be easily adapted to multiple activities. This script had four domains  and consisted of the following five questions:1.How is everyone feeling? Or: How did that feel? (Reaction)2.Can someone summarize the experience? (Description)3.What aspects of the interview went well? 4.What aspects of the interview were challenging? 5.What is one takeaway that will help you in the future? What is one thing you will change to perform better in a future interview? (Application)There were several debriefing scripts in the literature. We based our debriefing script  on a scrFinally, the guided self-assessment or reflection  for stud•••••••p < .001), how to introduce and present oneself , what questions to ask , and ability to perform well in an authentic residency interview  After students completed their first authentic residency interview, their responses regarding the merits of the training program were highest. Survey results from our pilot participants after their first authentic residency interviews are shown in the We have been running our residency interview training program since 2014 and have had 73 students complete it. Pilot data from 29 students completing the residency interview training in 2015 demonstrated that medical students had limited prior experience with either medical school or job interviews. Likert score–based pre- and postsurveys were administered to all 29 students. Chi-square analysis of differences in mean scores between pre- and postsurveys demonstrated statistically significant increases in confidence in how to dress (pre = 4.50, post = 4.95, Residency interview performance contributes heavily to a candidate's overall rank, with the majority of program directors citing factors directly related to the interview day as having the greatest weight in the selection process. However, medical students are often underprepared for the gravity of the residency interview. We believe that residency interviewing is a professional activity with its own set of knowledge, skills, attitudes, and behaviors that can be taught. Using a residency interview program with the ability to practice and receive feedback, medical students can be better prepared, more confident, and perform better in the interview, potentially improving their rank by the program, with the ultimate objective of improving outcomes in the Match.Limitations of our program include no formal training for the interviewers, no simulation by interviewers of archetypal roles such as a distracted or hostile interviewer, and a heavy reliance on faculty time and simulation center resources. In addition, the videotaped interviews were not reviewed by interviewers, and students were not guaranteed a third interview after they had had time to review the entire program and reflect on and incorporate the feedback. However, we do not feel that the failure to train interviewers or have them simulate other roles is a major flaw. We would argue that most interviewers performing residency interviews have had no training, so our program's lack of training is actually typical of what students will see in authentic residency interviews. In addition, simulating other interviewer roles is not necessary. Due to the competitive environment, most programs are trying to impress applicants, and interviewers are generally engaged, positive, and polite, making simulation unnecessary.How much time it takes to run the program is dependent on the number of students and the number of interviewers. For example, if one has eight students and only four interviewers, then having each student do two 20-minute interviews with 10 minutes of feedback takes 2 hours. If one has eight students and eight interviewers, then it takes only 1 hour. If time is of the essence, there are numerous ways to shorten the session. First, the didactic slide presentation with the students and the facilitator on can be offered on a different date from the actual interviews. To save time, one can shorten the interview length, the feedback length, or both. If there are a lot of students and very few faculty, then recruiting others such as fourth-year students, residents, fellows, or faculty from other disciplines to be interviewers can be useful. As an example, in 2018, we ran the program with 120 rising fourth-year students. We did the didactic session with 120 students and one facilitator, then had students form groups of three to make 40 small groups. Each student in a small group took a turn as the interviewer, the interviewee, and, using a smart phone, the videographer. We had five faculty circulating the room to answer student questions. Finally, we have found other ways for students to review their interview videos and practice more interviews by asking them to seek out other mentors, advisors, or letter writers to perform these roles outside the structured training program. Students are encouraged to arrange a time to meet with their mentor, review the videos, receive additional feedback, and then practice another interview with the mentor either immediately following or at a later date and time.7 Koenigsfeld and colleagues demonstrated a higher Match rate among pharmacy students who completed their faculty-orchestrated mock residency interview exercise as compared to the American Society of Health-System Pharmacists national rate.6 Therefore, given the success of mock interviews among pharmacy students, the concept of interview programs can be translated for medical school students applying to residency programs. Residency interview training programs offer an unparalleled resource for medical students’ interview process preparation, allowing the opportunity for practice and individualized feedback on one of the most critical components of the ranking process: the interview.The tangible benefits of a residency interview program are difficult to quantify since an applicant's success in the Match process is based on a multitude of factors. However, evidence suggests that there are no negative consequences of such preparation and that student interview performance can only be improved. To our knowledge, the current literature has not yet examined the effect of residency interview programs on medical student Match outcomes. Studies have shown improved success in the Match process among pharmacy students.Didactic Slide Presentation.pptxInformational Packet for Students.docxQuestions for Facilitators.docxInterview Performance Evaluation Tool.docxDebriefing Script.docxGuided Self-Assessment.docxPre- and Posttraining Confidence Survey.docxAll appendices are peer reviewed as integral parts of the Original Publication.


Due to the lack of earthquake-resistant rural houses, small and medium earthquakes caused massive casualties in rural China. In 2004, China began implementing the Earthquake Rural Housing Safety Project Policy (ERHSPP) to reduce earthquake losses, mainly promoting the adoption of earthquake-resistant structures in rural residents’ self-built houses through subsidies, training of construction craftsmen, and provision of earthquake-resistant housing drawings. We conducted a field survey, collecting 1169 rural households in Pingliang, Gansu Province, and 1501 rural households in Yuxi, Yunnan Province, China. We studied Earthquake-Resistant Construction Behaviors (E-RCB) by the logistic and the ordered logistic regression models. Results show that government housing subsidy promotes E-RCB of rural households; E-RCB was affected by ERHSPP, positively correlated with economic status and housing earthquake damage; E-RCB was negatively correlated with structure danger, house age, and earthquake experience; and housing earthquake damage, not earthquake experience, strikingly promoted E-RCB in rural China. The results could provide suggestions in communication risk strategies for the government. We suggest the local government should promote local acceptable disaster propaganda, provide hierarchical housing subsidies, pay attention to housing seismic supervision, publish earthquake-resistant housing design drawings, and conduct more earthquake-resistant technical training for rural craftsmen. However, the lack of supervision and guidance during rural houses’ construction caused huge losses after the earthquake. From 2001 to 2005, 60 disastrous earthquakes occurred in mainland China, affecting approximately 7684 million people, 144,300 kmTo enhance the seismic performance of rural self-built houses, ERHSPP was implemented in Xinjiang in 2004 as a pilot, and it was officially promoted throughout China in 2007 in the Opinions on the Implementation of Earthquake Safety Project for Rural Residential Buildings . AccordiERHSPP has been successful in improving the earthquake resistance of rural houses. None of the newly built rural earthquake-resistant houses in Xinjiang province after 2004 were damaged after more than 60 earthquakes of magnitude ≥5 . TherefoPrior research on household disaster preparedness discussed psychological aspects and adaptive behaviors. We would like to explore the factors affecting actual behaviors. The Government housing subsidy in this article refers specifically to the special housing subsidies in ERHSPP, The Earthquake-Resistant Construction Behaviors (E-RCB) refer specifically to used seismic drawing, made seismic patands, and chosen qualified contractors when building the house. This paper adopts logistic and ordered logistic regression to quantitatively analyze E-RCB, ERHSPP effect, and other aspects to investigate:Did government housing subsidy promote rural household E-RCB or not?Are government project’s effect, house risk, economic level, and earthquake experience related to E-RCB or not?E-RCB in Yuxi and Pingliang, China, are conducted as the case study through questionnaire surveys. This article includes five parts: Existing research on household disaster preparedness discussed psychological aspects and adjustment behavior. Psychological research on household disaster preparedness discussed the factors that influence individual risk perceptions. Differences in individual perceptions of disaster risk are related to gender, age, education, socio-economic status, household income, housing economic value, the risk level of the residence, the number of children, individual responsibility for taking preventive measures, and psychosocial factors ,6,7,8. OIn terms of adjustment behaviors, studies discussed household and individual disaster preparedness and adjustment behaviors, mainly including the apportionment of economic losses such as earthquake insurance ,13,14,15Another possible research direction is to analyze the relationship between government project effect and household disaster preparedness. According to the practice of ERHSPP, this article defines “Earthquake-Resistant Construction Behaviors (E-RCB)” as the use of seismic drawing, seismic patands, and qualified contractors when rural households build their own houses. The hypotheses of E-RCB are as follows.Scholars have studied the impact on the government engineering projects, and whether the impact on residents’ behavior is positive or negative. There are different opinions.The government’s dissemination of disaster preparedness knowledge to encourage people to understand local disaster risk information is considered to be related to residents’ disaster preparedness behavior. The government’s disaster mitigation plan can promote residents to understand their location’s disaster risk ,24. In eResidents’ government trust is considered to be related to residents’ disaster preparedness. Han took Yushu earthquake survivors as samples, and pointed out that the two self-evaluation indicators of individual government trust and household disaster preparedness were significantly negatively correlated . AdditioSpecial government housing subsidies and similar economic subsidies are considered related to residents’ disaster preparedness behavior. Taylan found linking house maintenance credits with adoption of risk mitigation measures, motivating homeowners for taking earthquake preparedness measures in Turkey . On the Additionally, households may also passively rely on government actions. Wu’s research on rural communities in Kaohsiung City found that after the implementation of the disaster reduction plan, 50% of the communities passively rely on the assistance of the government and experts, and only about 1/6 of the communities have mobilized more materials for disaster reduction preparations with government policy support . Tian usTherefore, previous studies have different views on the impact of government engineering measures on residents’ behavior. This article provides the following hypotheses:Hypothesis 1a (H1a). Government project effect and E-RCB are positively correlated.Hypothesis 1b (H1b). Government project effect and E-RCB are negatively correlated.Hypothesis 1c (H1c). Government housing subsidy and E-RCB are positively correlated.Hypothesis 1d (H1d). Government housing subsidy and E-RCB are negatively correlated.The comparison between the new and old built rural houses can reflect the effect of ERHSPP. We considered house age to be related to the seismic performance of the house. According to the Second National Agricultural Census in 2006 , in ruraBy the Standard for Seismic Appraisal of Buildings (GB 50023–2009) , the expHypothesis 2 (H2). Structural danger and house age are negatively related to E-RCB.We considered the economic level represented by income directly related to the household earthquake preparedness behavior . EconomiHypothesis 3 (H3). Economic level and E-RCB are positively correlated.Considering that a catastrophe is a small probability event, it may be challenging to motivate households to build strong houses for earthquake preparedness, which is time-consuming and labor-intensive. Previous studies generally believed that the individual’s past disaster experiences such as directly experiencing large-scale destructive earthquakes and experiencing earthquake losses will help them decide how to respond to earthquakes. Compared with individuals who have not personally experienced earthquakes, individuals who have experienced earthquakes have taken more disaster preparedness measures ,45,46. BHypothesis 4 (H4). Earthquake experience and E-RCB are positively correlated.Wu has conducted field surveys on the affordability costs of the seismic fortification of rural houses in Pingliang, Gansu Province, in 2010, and Yuxi, Yunnan Province, in 2011 .The sample questionnaires collected valid sample information of 2670 rural households, among 1169 samples from Pingliang, Gansu Province, and 1501 samples from Yuxi, Yunnan Province. The sample questionnaires are the data source of this study. We used multi-stage stratified sampling, selecting villages with Earthquake Intensity Degree ≥ 7. Each family chose one person to form a list and stratified according to local population information sampling. Interviewers conducted face-to-face interviews due to the education level of the interviewees. This study uses Stata 15 for data analysis. Two survey sites are shown in 2 area. Yuxi and Pingliang City both belong to the earthquake-affected area.Both places have experienced major earthquakes in 1970 and 2008. The 7.7 Ms Tonghai earthquake on 15 January 1970, caused 15,621 deaths, and 338,456 houses collapsed. The entire area of Yuxi City caused heavy losses due to the 1970 Tonghai earthquake. The 8.0 Ms Wenchuan Earthquake on 12 May 2008 caused 69,227 deaths and severely damaged more than a 100,000 kmThis research mainly analyzes whether government project effect, house risk, economic level, and earthquake experience are related to E-RCB. The descriptive statistics of the variables are shown in The first dependent variable is Policy Subsidy, which reflects received the government housing subsidy when building this house with seismic structure. The questionnaire asked, “Have you received government subsidies for building/purchasing this house?”, 1 for “Yes” and 0 for “No”. 6.03% sample households received special housing subsidies. We only raised this question in Yuxi, and the total sample size for this variable is 1501.The second dependent variable is Earthquake-Resistant Construction Behaviors (E-RCB), which consists of three measures: used seismic drawing, made seismic patands, and chosen qualified contractors when building the house. 16.93% sample households completed all three measures, 9.78% sample households completed two measures, and 5.54% sample households completed one measure.Seismic Drawing: Used seismic drawing when building this house. The questionnaire asked “Is the construction drawings used when building this house?”, “Does the construction drawings consider seismic resistance?”, 1 for “Used” and “Considered”, 0 for “Used” but “Not considered”, and 0 for “Not used”. 15.88% sample households used seismic drawing.Seismic Patands: Made seismic patands when building this house. The questionnaire asked “Has the foundation of this house been treated?”, 1 for “Treated, there are ground ring beams” and 0 for “Simple treatment” and “No treatment”. 27.75% sample households made seismic patands.Qualified Contractors: Chosen qualified contractors with seismic construction skills when building this house. The questionnaire asked, “Who built this house?”, 1 for “Qualified construction team” and 0 for “Unqualified construction team” and “Friends and relatives I find myself”. 9.48% sample households chose qualified contractors.Policy Information: Knew the seismic fortification intensity of the area (or the seismic fortification standard to be achieved when building houses). The questionnaire from Pingliang asked “Do you know the seismic fortification intensity in this area?” and the questionnaire from Yuxi asked “Do you know the seismic fortification standards to be met when building houses in this area?”, 1 for “Known” and 0 for “Never heard of it”. 10.15% sample households knew the seismic fortification intensity.Government Trust: Recognition degree of “the local government’s earthquake countermeasures are adequate”. The questionnaire asked “How would you rate ‘the local government’s countermeasures to the earthquake are adequate’”; the options “Inconsistent”, “Not very consistent”, “Relatively consistent”, and “Consistent” are assigned a value of 0–3 in turn. 37.42% sample households are relatively consistent with this opinion.Structural Danger: The corresponding risk of the building structure can be divided into “Ring beam structure or steel frame”, “Brick–concrete or brick–wood”, and “Adobe” concerning Seismic Field Work Part 3: Survey Specifications (GB/T 18208.3-2011) . The hazHouse Age: The questionnaire asked, “When was this house built?” House Age is equal to the survey year minus the house construction year, and the result is in logarithm. The mean of the sample logarithm house age was 2.400.Annual Income: The questionnaire asked, “How much income did your family have in the past year?” The annual family income is RMB. 5000 yuan, 5000–8000 yuan, 8000–10,000 yuan, 10,000–20,000 yuan, 20,000–30,000 yuan, 30,000–50,000 yuan, 50,000–80,000 yuan, 80,000–100,000 yuan, 100,000–200,000 yuan, 200,000–500,000 yuan, and more than 500,000 yuan are assigned 1–11 points in turn. The mean of sample households’ annual income was within the range of 8000–20,000 yuan.Earthquake Experience: Experienced 1970 Tonghai or 2008 Wenchuan earthquake. Questionnaire of Pingliang asked “Have you encountered an earthquake disaster so far?”: 1 for “Have experienced an earthquake disaster” and 0 for “Have experienced other disaster “ and “Have not suffered a disaster”. Questionnaire of Yuxi asked “Have you heard of the 1970 Tonghai earthquake?”: 1 for “Experienced personally” and 0 for “Heard of” and “Never heard of”. Out of the sample households, 55.06% experienced the 2008 Wenchuan or 1970 Tonghai disastrous earthquake.Housing Damage: The house has previously collapsed due to an earthquake. The questionnaire asked “What degree of earthquake disaster did you suffer?”: 1 for “My house completely collapsed” and “My house partially collapsed” and 0 for “No house damage”. Out of the sample households, 30.19% experienced self-housing collapse due to an earthquake.To conduct hypothesis testing, this article regards the explanatory variable “Earthquake-Resistant Construction Behaviors (E-RCB) “ as an ordinal variable, and establishes an ordered logistic regression model with the expression:We aimed to explore whether the policy subsidy has an incentive effect on E-RCB and whether the household received the policy subsidy as a dependent variable. The use of seismic drawings that reflect technical services, the qualified construction team that reflects the training of construction craftsmen, and the seismic construction of self-built houses are used as independent variables. We only raised this question in Yuxi, and the total sample size for this variable is 1501. In This paper uses E-RCB as the dependent variable. We tested the interaction effects of the independent variables, and the results showed that there was no significant interaction effect between the independent variables, so we chose to use the following model. Model 1 is the basic model to examine the impact of government project effect on E-RCB. Model 2 adds house variables based on Model 1 to investigate the influence of house risk on E-RCB. Model 3 adds economic variables to Model 2 to examine the impact of the economic level on E-RCB. Model 4 adds experience variables based on model 3 to investigate the impact of earthquake experience on E-RCB. The results of Model 1–4 are shown in The results of Model 1 show that after controlling government trust, compared with households that did not know the local seismic fortification standards, households that know the standards enacted more E-RCB; the increase is about 94.3%  translated into the local language that is easy to understand .The Chinese government can provide policy subsidies, which are graded based on family income and provide more subsidies to poor families for housing special subsidies in rural earthquake-hazardous areas, helping families with financial difficulties build affordable earthquake-resistant houses. This discovery is similar to Wu’s conclusion, when policy subsidies are not graded based on family income and provide more subsidies to poor families. The result is often that families with higher economic affordability are more likely to receive policy subsidies, and poor families cannot use the policy, because they cannot afford out-of-pocket expenses after policy subsidy reduction .Timing of housing seismic supervision: The Chinese government may pay attention to the seismic fortification supervision in the process of building new houses in rural areas, and in the restoration and reconstruction after natural disasters, so that newly built rural houses meet the local seismic fortification standards.The local government should continue to publish unified, earthquake-resistant housing design drawings, which are suitable for local architectural styles in the local disaster reduction plan.The local government can conduct more earthquake-resistant technical training for rural craftsmen and encourage welcoming NGOs and community residents to participate in craftsman training and disaster education.