Use of Evidence-based Guidelines (EBGs) has been shown to improve and standardize care. After implementation and maturation of a guideline program, next steps include incorporating new evidence, sustaining adherence, minimizing measurement burden and fostering scaling of the program. We propose a framework for maintenance and dissemination of an EBG program.Using a program of 28 EBGs developed for use in a pediatric emergency department (ED) in 2010, we developed: a framework for iterative review and revision, a strategy to measure ongoing use in practice and an approach for minimizing repeated measurement sufficient to evaluate outcomes. Also, we created a process to spread the EBG program to the hospital's Department of Pediatrics.The framework for maintenance and spread of a program of EBGs resulted in an annual review of individual guidelines with 14 revisions warranted by new evidence, some leading to decreased medication utilization and hospitalization rates. We demonstrated adherence to key quality measures, and decreased the number of measures from 89 to 43, retiring 46 measures with stable peformance. We spread the process for program development to the hospital pediatric department resulting in 36 new EBGs.We developed a framework for maintenance and scale of a program of EBGs. Our key learning points were that regular incorporation of new evidence, assessment and feedback on performance and leadership with administrative support are necessary to maintain improvement. This framework may assure sustainability and inform other guideline programs. We offer processes to promote guideline dissemination within an academic hospital. EBGs are useful in emergency medicine given the breadth of presenting conditions, stressful environment and need for timely care.2 We recently reported on the implementation of an EBG program in a single institution.3 Other hospitals have instituted similar programs.8 Once established and embedded into care, the next challenge for an effective EBG program is the ongoing revision of the EBGs, the sustained use in practice, and development of effective and efficient strategies to minimize unnecessary measurement. There are well-documented strategies to sustain improvement of quality improvement (QI) initiatives, including eliminating causes of noncompliance,9 ongoing observations of desired behavior with immediate feedback,10 highly reliable interventions, and leadership support.12 Much of the literature relates to sustaining specific guidelines.15 We know less about how to sustain a program with many guidelines, including how to determine if the recommendations are embedded in practice and how to reduce the resources needed for improvement and measurement. Another consideration is the spread of guidelines. QI techniques have been used to increase the number of hospitals creating and adhering to guidelines.17 It has been difficult to ensure that hospitals implement reliable interventions to aid in sustaining the initiatives.17Evidence-based guidelines (EBGs), systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances,19 decreased resource utilization,20 and decreased the cost.21 It has been well accepted by clinicians.3 The evolution of the program has led to the development of processes for incorporating new evidence and sustaining guideline adherence while minimizing administrative burden. We report on our approach to overcome the challenges of maintaining, growing, and spreading a successful EBG program within the hospital.We introduced an EBG program into our hospital’s emergency department (ED) in 2010, and it has grown to include 28 individual EBGs (Table 1). This program has successfully standardized care,To anchor our work, we created a driver diagram Fig.  for main22) and modified as indicated. If new evidence led to substantial changes in the guideline, we consider and track new PICO questions closely. Using this process, we can sustain regular review and ensure modifications are incorporated as needed. New evidence is evaluated using Grading of Recommendations, Assessment, Development, and Evaluations criteria (gradeworkinggroup.org).23 Subsequently, the evidence is vetted with the ED physicians at faculty conference to gain awareness and consensus, and then revisions are incorporated into the guideline. We disseminate the revised EBG to clinicians through presentations at meetings, individual education, e-mails, posters, and bulletin boards. We maintain an intranet webpage for easy access and a direct link to condition-specific order sets in the electronic medical record (EMR). Our prior work presented data on successful uptake strategies, and note that paper copies of EBG were more useful for nurses and electronic resources more useful for physicians.3The evidence must be current and accurate to achieve the EBG program’s primary purpose of providing optimal care. In the beginning, we created a process whereby a dedicated physician and nurse champion developed each EBG  use for patients with suspected appendicitis. After approximately 25 periods of baseline data are gathered, these measures for implemented EBGs are tracked frequently. The nature of the condition and volume determines frequency . Data are generated electronically from the data warehouse  fed by the EMR . A local ED EBG leadership team comprised 2 data analysts with 3–5 years of healthcare experience, a QI consultant, and physician quality leader meet weekly to review data and statistical process control charts for a group of guidelines on a rotating basis. Trends and shifts25 Each year we incorporate selected EBGs into the review process. These are chosen based on the strength and importance of the guideline recommendation, such as antibiotic stewardship, resource utilization, and patient centeredness. Although not directly linked to monetary risk, individual physician performance is reviewed with the physician and compared with the performance of peers.27One such method to embed EBG adherence is the inclusion of physician performance in annual faculty reviews.3 These plans outline inclusion and exclusion criteria, data source, diagnostic codes, and detail on numerator and denominator, risk stratification as needed, display format, and frequency of review. Once performance stabilizes at a target level for a sustained period , we reduce the frequency of measurement , spot checks are performed every 6–12 months. If periodic audits reveal a decline in performance, measures return to a regular cycle of review to boost adherence. When performance is consistently stable at target, the measurement is retired. At this point, we are confident that the guideline is embedded into routine practice and considered standard care.3 The ED EBG leadership team was available as a resource and continues to consult as needed. The departmental EBG team, similar in make-up to the ED team, included a data analyst, QI consultants, pharmacist, and physician leader. They met regularly with subject matter experts in the department who served as local champions . Individual physician performance was measured either by chart review or database extraction. The departmental quality program supported this collaboration with a small amount of funding . We also developed a mobile app to facilitate access to guidelines.Spreading a successful EBG program is important for influencing the best practice. We collaborated with departmental quality leaders to share the strategy and tools for development and implementation of the EBG program.According to the policy at Boston Children’s Hospital, this work met criteria for operational improvement activities and not human subject research. Therefore, review and approval by the institutional review board were not required.26 Albuterol administration for infants with bronchiolitis decreased after new American Academy of Pediatrics guidelines recommending against its routine use were released in November 2014. The initial rate of albuterol use was 53%, which dropped to 43% within a month after new guideline implementation, but it was not until we formally revised the local EBG in November 2015 with new recommendations that the rate dropped to 18%. The anaphylaxis EBG was introduced in November of 2011 and led to initial avoidance of 140 hospital admissions by December 2014.19 After we incorporated new evidence27 in June of 2015, the hospitalization rate fell even further, avoiding an additional 32 hospital admissions by November 2017.Through active participation by guideline champions, supported by the quality administrative team, all 28 guidelines have been reviewed and/or revised at least annually, and 14 have undergone major revisions. Successful incorporation of new evidence has led to decreased resource utilization. One such example is reduced hospitalization rates for patients with idiopathic intussusception from 92.3% in 2010 to 32.4% in 2016 after new data on the safety of ED discharge after air contrast enema reduction was published.We developed a new framework for tailoring measurement and ensuring guideline establishment into practice Fig. . Through14 From its implementation in October 2012 through September 2017, we have safely avoided over 90 hospital admissions. Although we communicate success in program overviews, measurement occurs biannually, and there is no ongoing active improvement supporting the guideline as we consider it embedded into standard practice.Using our framework, we have decreased regularly monitored EBG performance measures from 89 to 43 while maintaining high performance and incorporating new guidelines and associated measures. Seventy-eight percent of EBGs have met goal performance. We have removed these EBGs from active monitoring. For example, the introduction of the minor head trauma EBG significantly decreased the hospitalization rates for patients with isolated nondepressed linear skull fracture.Many other successes have led to reduced measurement. The change to use magnetic resonance imaging (MRI) as the imaging modality for patients evaluated for ventricular shunt malfunction is another example. The percent compliance with the imaging recommendation increased from 44% to over 80% with the introduction of the EBG. After 17 months of high performance, we retired the measure. Subsequent annual spot checks ensured we still followed the recommendation.We also identified cases where performance declined after less frequent measurement . After a 6-month spot check identified the performance degradation, we reinitiated active monitoring.Supplemental DigitalContent 2, which provides list of departmental EBGs, http://links.lww.com/PQ9/A76). They range from straightforward and common conditions, such as management of acute otitis media, to more rare conditions, such as evaluation and treatment of Ehlers–Danlos Syndrome. All departmental EBGs are currently undergoing measurement and analysis in a process similar to the ED to determine their impact on care. Also, we have shared EBGs with all hospital providers through links to an online intranet site. This portal allows for easy access to all guidelines across the enterprise.We spread the process to non-ED settings in the Department of Pediatrics including ambulatory and inpatient units. Over 4–5 months, we shared our expertise and the tools for development and implementation of the EBG program with the departmental EBG leadership team. We supported the development of 36 new EBGs . The app is downloadable onto both iOS and Android mobile devices. It has been downloaded 281 times, with 101 monthly users. On average, the app has been used 9 times a day since go-live in November 2017. It includes step-by-step decision support mirroring the ED EBG algorithms and is updated with all EBG revisions.We present a practical process for sustaining and spreading a program of EBGs. The major lessons learned were as follows:A regular, structured process for incorporation of new evidence is critical to maintain the guidelines as “living documents” and thus the trust of those who use them.We achieved sustainment of EBGs in practice through regular assessment of performance, dedicated feedback to stakeholders, and strong leadership support.Consistent leadership and administrative support are needed to maintain a successful EBG program and to assess uptake for measurement reduction.The internal spread of the EBG program is achievable through sharing a package of tools for development, implementation and evaluation, practical support, and acknowledgement of challenges presented by a new context.28 Through annual, at a minimum, literature reviews by EBG champions and thoughtful vetting of new evidence with colleagues, we ensure that the documents remain fresh and reliable.For an EBG program to flourish, it is important that physicians trust the process and believe in its integrity.25As we learned in the EBG program development stage, strong leadership support is essential for continued success. Leadership-led annual faculty reviews of individual physicians emphasize the program’s value, especially when we provide peer comparison. Also, resources for data teams, QI experts, and dedicated QI physician time are critical to success. Without this, there would be limitations to the regular cycle of performance and feedback which we found fundamental to the sustainment of the program. We needed administrative support for data extraction, project management, QI, and regular review of performance. These resources can also support the development and maintenance of physician performance portfolios which we have noted to be an important component of faculty development.29 In our program, we sought sustained performance at the target to judge retirement eligibility.Periodic assessment of performance is fundamental to QI. Our framework for decreasing the frequency of assessments and ultimate retirement of measurement has led to efficiencies in administrative and clinician resources. Strategies for measuring retirement include achievement of a ceiling effect or sustained performance at or above target.30 There are several factors to consider when spreading to a new context, including those outlined in the Model for Understanding Success in Quality.31 These include the external and organizational environments, senior leadership support, the local QI culture, and the QI team with its skill and motivation. The context at the department level was in many ways similar to the ED concerning the environment and data sources. The QI expert teams were similarly engaged and trained. One substantial difference was that leadership in each division offered varying support and the EBG leadership team at the departmental level was tasked with navigating different quality and subject matter expert “microenvironments.” The leadership team overcame these contextual barriers by soliciting advice from the subject matter experts to determine the need for a top-down versus consensus approach. In this way, the local subject matter experts guided paths to buy-in.The spread of the program to other areas of the hospital was relatively seamless as shown by the development of a large number of regularly accessed guidelines.The interactive mobile app allows readily available access and assistance in spreading guideline adoption. Rotating residents report benefitting from the ready availability of standardized care guidelines on their phones and other devices which they also use for patients in other EDs.There are limitations to consider when developing and sustaining an EBG program. We had strong ED leadership support and institutional resources that may not be generalizable to other settings. We had well-developed technology, including the ability to easily access EMR data and create modifications to specific elements such as order sets.We propose a framework for maintaining a program of EBGs with strategies to reduce measurement. We demonstrate that it is possible to successfully spread an ED EBG program to other areas of a hospital. We believe adoption of this method of sustainment and spread of an EBG program are possible in similar settings.We acknowledge Taruna Banerjee, MPH, and the DoP Quality team for their assistance with the spread of the EBG program.The authors have no financial interest to declare in relation to the content of this article.


Moreover, when they experienced earthquake trauma, the adolescents with low OGM did not experience more depression, but the adolescents with average and high OGM experienced more depression than the adolescents with no earthquake trauma. This finding indicated that in a non-Western cultural context, adolescents’ propensity toward OGM made them vulnerable to depression after experiencing an earthquake trauma.Trauma has a profound impact on overgeneral autobiographical memory (OGM), which is a risk factor for depression. Violent earthquakes can cause tremendous trauma in survivors. We examined the relationship between earthquake trauma, OGM and depression in adolescent survivors of the Wenchuan earthquake in this study. OGM was assessed using the autobiographical memory test in a sample of adolescent participants who experienced the violent earthquakes in Wenchuan, China, in 2008 and control participants who had never experienced a destructive earthquake. Depression was measured using the Beck Depression Inventory-II in all participants. The results showed that compared with the adolescents with no earthquake trauma, the adolescents with earthquake trauma reported significantly more depression ( In the past decade, several devastating natural disasters, including the Wenchuan Earthquake in China at 2008, the Great East Japan Earthquake at 2011, the Typhoon Haiyan in the Philippines at 2013, and the Nepal earthquake at 2015, had struck the world. Besides human death and injury, such natural disasters also had a severe impact on the mental disorder of survivors. The risk of depression was highly increased after exposure to a natural disaster, and the post-traumatic stress disorder (PTSD) showed high prevalence rate among survivors .For survivors, such disasters usually became traumatic events, which could influence the onset of depression. Individuals who experienced a traumatic event, such as burn, traffic accidents, were more likely to be depressed. Substantial body of evidence indicated that exposure to traumatic events could result in disorders see . There wOn the other hand, individuals who had traumatic experiences exhibited more overgeneral autobiographical memories (OGMs) see , which wAutobiographical memory refers to one’s memories of personally experienced past events. It contributes to one’s sense of self, one’s ability to approach problems, and one’s ability to orient herself to be in a society . OvergenIn his trauma hypothesis, OGM of negative events might serve as a marker of depression risk. Adolescents with a current first episode of major depressive disorder retrieved more OGMs than did non-depressed controls , whereasPrevious studies in adolescents suggested that trauma exposure was associated with OGM  and OGM Furthermore, the cognitive vulnerability-stress model of depression suggested that an individual with cognitive vulnerability was more likely to become depressed than a non-vulnerable individual when she or he confronted a negative event and interpreted the event in a negative manner . Some stMage = 14.77, SDage = 0.56) in the earthquake trauma group (ET) and 46 participants  in the never-experienced earthquake (NEE) group, were recruited in 2010. The two groups matched on age, gender and educational level (both were at the second year of the middle school), and also lived in regions that were similar on economic development. There was no significant difference in age t(91) = 1.64, p = 0.10, and gender ratio χ2 = 0.10, p = 0.75 between the two groups.Ninety-three participants, including 47 participants  by their head teachers, who had taught these students for more than 1 year and knew the students well. Then students confirmed the information provided by their teachers. The students, who met at least three criteria in the EREQ, were included as the earthquake trauma group. Specifically, 17 participants met the three criteria (36%), 19 met the four criteria (40%), 8 met the five criteria (17%), and 3 met the six criteria (6%). Participants in the NEE group were from Changqing Middle School in Changqing County in Shandong Province, China, where no earthquake had occurred for 20 years. After the participants were chosen, they completed the BDI-II in their own classrooms. Next, the participants individually completed the AMT in a quiet cubicle. Finally, the participants were offered a gift.To identify the earthquake trauma group, 60 students from Beichuan Middle School in Beichuan County, the center and the most seriously damaged area of the Wenchuan Earthquakeyes/no choice.Earthquake-related experiences questionnaire developed by ourselves was used to access the earthquake-related trauma. It included six items measuring (1) serious casualties, (2) witnessing death, (3) touching a corpse, (4) being buried, (5) physical harm, and (6) family loss with a The BDI-II was a 21-item self-report questionnaire to measure current levels of depression . The ChiThe AMT  includedt-test was performed between the ET and NEE groups on depression. The ET group had significantly higher level of depression compared to the NEE group t(91) = 2.38, p = 0.02, d = 0.58.The Means and SDs of the depression and OGM proportions were shown in Table t-test was performed between the ET and NEE groups on OGM proportion. The ET group had significantly more OGMs than the NEE group in the total proportion t(91) = 2.66, p = 0.009, d = 0.48. The ET group had significantly more OGMs compared to the NEE group for the negative cues, t(91) = 2.46, p = 0.02, d = 0.49 and the neutral cues t(91) = 2.17, p = 0.03, d = 0.46. However, there was no significant difference between the ET and NEE groups for the positive cues, p = 0.40.To assess whether the two groups differed in terms of OGM, an independent sample r = 0.08, N = 93, p = 0.45); however, a significant correlation was found between the earthquake trauma and depression , and OGM .Using a Pearson’s product-moment correlation coefficient, we found no significant correlation between OGM and depression  = 3.59, p = 0.02, R2 = 0.10. Specifically, OGM showed no significant effect on depression, b = 0.08, t(89) = 0.92, p = 0.36, 95% confidence interval (CI) = , but the earthquake trauma had a significant effect on depression, b = -0.96, t(89) = -2.10, p = 0.04, 95% CI = . Further, the moderating effect of OGM on the relationship between earthquake trauma and depression was significant, b = -0.88, t(89) = -2.22, p = 0.03, 95% CI = . A simple slope analysis revealed that for participants with low OGM, there was no relationship between the earthquake trauma and depression, b = -0.06, t(89) = -0.03, p = 0.98, 95% CI = , but for participants with average OGM, b = -2.96, t(89) = -2.10, p = 0.04, 95% CI = , and high OGM, b = -5.86, t(89) = -3.27, p < 0.01, 95% CI = , the earthquake trauma made them experience more depression  designed by This study aimed to examine the relationship between earthquake trauma, OGM, and depression in adolescents who experienced a violent earthquake in a Chinese sample. First, it revealed that adolescents exposed to an earthquake trauma experienced more depression than adolescents with no history of experiencing a violent earthquake. This result was consistent with earlier studies that examined the prevalence of depression after exposure to other types of trauma , which influenced people’s depression  and OGM In summary, the current study revealed that adolescents exposed to earthquake trauma reported more depression and more OGM, and provided support for the moderating role of OGM on the effect of traumatic experiences on depression within an earthquake trauma sample in a non-Western cultural context. This finding highlighted that when suffering an earthquake trauma, people’s propensity toward OGM was a vulnerable factor for depression.QT wrote the article. HH and DZ analyzed the data. YM gave suggestions on writing. JZ collected the data. SL designed the study and modified the article.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Living Life to the Full 55+ (LLTTF) was an 8-week, 12-hour, community educational program based on CBT principles. The program aim was to teach participant skills and techniques to cope effectively with life stress. Participants self-referred to the program by responding to advertisements or were otherwise referred to the program by local community centres. Following the promising results of the pilot study, a follow-up empirical study was conducted to assess the impact of the program on clinical measures and other notable measures over time. Study participants (N = 514) were recruited at partner sites hosting the program. Demographic and other data was collected to assess the impact of the program. Measures collected to assess the impact of the program were the Beck Depression Inventory (BDI-II), Beck Anxiety Inventory (BAI), Warwick-Edinburgh Mental Well-being Scale (WEMWBS), and a brief measure of loneliness/social connectedness. Participant data was collected pre/post-intervention, and at post-course follow-up periods (3/6/9-months). Preliminary repeated-measures ANOVA analyses found statistically significant changes in depression, anxiety, mental well-being, and loneliness scores, from pre- to post-intervention. Participant’ depression, anxiety, and loneliness scores significantly reduced from pre- to post-course. Participants also experienced a significant improvement in mental well-being from pre- to post-course. Observed decreases in loneliness and depression did not significantly change from post-course to any of the follow-up periods. Results suggest the viability of a nonclinical program for addressing depression and loneliness in older individuals, and have implications for community-based strategies attempting to address mental health issues in older Canadians


Home visiting programs are implemented in high income countries to improve outcomes for families with young children. Significant resources are invested in such programs and high quality evaluations are important. In the context of research trials, implementation quality is often poorly reported and, when reported, is variable. This paper presents the quality of implementation of the right@home program, a sustained nurse home visiting intervention trialled in Australia, and delivered in a ‘real world’ context through usual child and family health services. right@home is structured around the core Maternal Early Childhood Sustained Home-visiting (MECSH) program, which is a salutogenic, child focused prevention model.At each visit right@home practitioners completed a checklist detailing the client unique identifier, date of contact and activities undertaken. These checklists were collated to provide data on intervention dose, retention to program completion at child age 2 years, and visit content, which were compared with the program schedule. Quality of family-provider relationship was measured using the Session Rating Scale. Exploratory factor analysis was conducted to identify clusters of activities and allow qualitative assessment of concordance between program aims and program delivery.Of 363 intervention families offered the program, 352 (97·0%) commenced the program and 304 (87·3%) completed the program to child age 2 years. 253 of 352 (71·9%) families who commenced the program received more than 75 percent of scheduled visits including at least one antenatal visit. Families rated the participant-practitioner relationship highly (mean 39.4/40). The factor analysis identified six antenatal and six postnatal components which were concordant with the program aims.The right@home program was delivered with higher adherence to program dose, schedule and content, and retention than usually reported in other home visiting research. Program compliance may have resulted from program design  that was consistent with family aims. The past two decades have seen increasing research evidence that sustained nurse home visiting (SNHV) can improve health and development outcomes for children and families experiencing adversity , 2, 3. AThere is, however, considerable variation in the quality of programs once they are delivered in the ‘real world’ as opposed to university research-based settings, with lack of attention to defining the elements of effective service delivery processes and measurement of program fidelity –7. ‘FideStudies in both research and real world environments have found that the retention rates and numbers of visits can fall below program ideals and expectations , 12. RecIn non-research implementation, reported retention and visit delivery elements of fidelity are even lower. Evaluation of the USA implementation of 35 various home visiting programs showed that retention at 12 months ranged from 3.9 to 73.0%, the proportion of families receiving the full dosage ranged from 5.3 to 26.4%, and the proportion receiving 80% of dosage ranged from 41.2 to 51.6% . A studyAs well as the retention in the program and delivering the expected number of visits, elements of fidelity include the nature of the relationship between the practitioner and parents, and delivery of program content. Effective home visiting services are relationship based, and depend on a positive relationship between the practitioner and parent/s in order to facilitate change , 10, 18.Research detailing the delivery of home visiting programs in real world settings is needed to unpack the “black box”  of SNHV, the right@home trial , a detaiThe right@home program is a SNHV intervention that was delivered to families from the antenatal period until their child turned two years old. The right@home program was trialled in seven localities in the Australian states of Victoria and Tasmania . NotablyThe right@home program was structured around the core Maternal Early Childhood Sustained Home-visiting (MECSH) model , with scNurses and social care practitioners were guided by a strengths-based approach and joint goal setting, based in the Family Partnership Model , which dpositive transition to parenting;mother, child and family health, development and wellbeing;maternal–infant bonding and attachment;positive parenting skills;mothers to be future orientated and aspirational for themselves, their child and family;mothers and families to enhance their coping and problem-solving skills, and ability to mobilize resources;supportive relationships in their family and community; andadditional support in response to family needs , 24, 28.Program implementation support, training, and fidelity monitoring was conducted by the MECSH Support Service at Western Sydney University in Liverpool NSW using the standard processes for the implementation of MECSH-based licenced programs  internatThe RCT recruited 722 pregnant women. Eligible participants were pregnant women attending the antenatal clinics in the trial areas from May 2013 to August 2014, with an expected due date before 1 October 2014. Participants needed to be less than 37 weeks gestation, have sufficient English proficiency to verbally answer interview questions, reside within the study travel boundaries, and have two or more of ten sociodemographic risk factors for adverse parent and/or child outcomes identified by risk factor screening , 36.The intervention arm of the right@home trial recruited 363 families, and 352 women commenced the intervention. Dates of families’ commencement on the program and children’s dates-of-birth were extracted from trial enrolment data . At the At the visits at child ages 3, 6, 12, 26, 52 and 104 weeks the nurse completed the Session Rating Scale (SRS) with the family. The SRS is a brief subjective measure of the quality of the working alliance between the nurse and family . The SRSA visit was considered valid if it occurred in the presence of the parent (e.g. a home visit), if it occurred over the phone and covered more than five topics, or if it was a group session and covered ten or more topics. Visits were considered invalid if: there was no date of visit, no topics were covered, mother failed to attend, the visit record was duplicated, or if the visit was a phone call or group session and did not cover the minimum number of topics. Some low activity phone calls were considered valid if, based on comments recorded by the nurse, it was clear that the phone call was therapeutic in nature.The date of each valid visit and child date-of-birth were used to determine the age of the child at the visit and then collated for each stage of the program: antenatal, and 0–6, 7–12, 13–26, 27–52, and 53–104 weeks postnatal. These data were collated at the end of the trial and descriptive statistics were used to determine family dose as compared to the program schedule, and retention in the program. Descriptive analyses were undertaken to describe the family session ratings and the proportion of visits that contained the listed activities. Exploratory factor analysis  with varimax rotation was undertaken to detect components of more common activities (occurred in more than one-third of visits) that co-occurred during the program, in order to determine concordance between program aims and foci of the delivered content. The antenatal and postnatal visit data met the statistical requirements for factor analysis . An inclusive factor loading of >0.3 was chosen .ISRCTN89962120 (21/Aug/2013), retrospectively registered.Trial registration number: The right@home trial was approved by the Human Research Ethics Committees in Australia of: The Royal Children’s Hospital, Victoria (HREC 32296); Peninsula Health, Victoria (HREC/13/PH/14); Ballarat Health Services, Victoria (HREC/13/BHSSJOG/9); Southern Health, Victoria (HREC 13084X); Northern Health, Victoria (HREC P03/13); and The University of Tasmania (HREC H0013113). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Written consent was obtained.Of 363 women randomized to the intervention group, 11 (3.0%) did not commence the intervention program; 352 families commenced the intervention (97.0%) and 304 of these families completed the right@home program at child age 2 years . The right@home program nurses delivered 9513 contacts, including 7992 valid visits. 95.3% of valid nurse visits were face-to-face home visits, ranging from 96.9% antenatal to 6 weeks postnatally to 94.3% after child-age 6 months.Program dose was measured for families who commenced the program, that is, who received at least one or more valid nurse visits. 56 families did not receive any antenatal nurse visits due to recruitment occurring too late in pregnancy for a visit to be arranged, and 175 families completed the program past the child’s second birthday, up to child age 133 weeks . 266 (75.6%) of families received the scheduled one or more visits by the social care practitioner. 288 families received more than 75% of the scheduled nurse visits (81.8% of program families and 91.8% of completing families), and for 251 (71.3% of program families) this included one or more antenatal visits.Fidelity was further monitored in terms of practitioner recorded delivery of program content. The proportion of valid antenatal (AN) and postnatal (PN) visits that contained each of the checklist activities is shown . The mosAs each visit contained multiple activities, factor analyses were conducted to identify which of the more common visit activities  clustered together in visits. The factor analysis resulted in six clusters for antenatal activities and six clusters for postnatal activities with eigenvalues greater than one, explaining a total of 50.3% and 47.8% of variance respectively. 21 = 3.6, p = 0.06), albeit the difference was not statistically significant.334 participants completed the SRS at one or more of the six scheduled times: a total of 1381 SRSs were completed. The mean score for their relationship with their nurse was 39.4 (SD 1.31) out of a possible 40. There was no evidence of differences in SRS scores at any time between those who completed the program (mean 39.5 SD 1.2) and those who did not complete the program (mean 38.9 SD 1.9). 32 (2.3%) of completed SRS rated the relationship less than nine (of concern). SRS scores less than 9 were somewhat more common early in the program  and McKPostnatally, the Promoting First Relationships program was an element in both the maternal-infant interaction and also family relationships components, suggesting that the program was promoting both mother-child and the child’s wider relationships, which may improve family engagement with the program . FurtherFamilies rated the relationship with their nurse very highly. As noted by Korfmacher (see page 465 in ), “serviThe fidelity to program retention and dose may have resulted from the timeliness and depth of the fidelity monitoring processes of external quarterly review and feedback from the MECSH Support Service team. Unlike Casillas et al.  who founNurses and families were aware that the program was part of a trial, albeit implemented in real world service, and they consented to participate in the research. This may have increased their commitment and engagement and so may not be replicable when taken to scale. In keeping with real world service, families were not provided with incentives to participate, and the strategies for supporting quality were those used in usual MECSH implementation suggesting that with appropriate technical, training and fidelity support, the quality should be replicable.In addition, the primary measure of the quality of the relationship between the nurse and the family, the SRS , is a suThe right@home program was implemented with fidelity. Families in the right@home trial intervention group were retained in the program for the expected duration, and received the program dose as scheduled in terms of the total number of visits overall and in each program stage. Antenatal and postnatal content actually delivered was consistent with the program aims. The retention of families in the program and the proportion of schedule dose received were much higher than achieved in comparable home visiting programs. The families also rated the relationship with their nurse provider consistently highly. This fidelity was particularly notable as the right@home program was delivered through usual child and family health services, rather than using nurses or systems especially established for the research. The structure and flexibility of the program and alignment with families’ capacity, together with the timeliness and depth of the quality monitoring processes, may have contributed to program fidelity that was superior to most previously evaluated home visiting programs.S1 File(DOCX)Click here for additional data file.


The overuse of diagnostic imaging for low back pain (LBP) in Australia results in unnecessary cost to the health system and, for patients, avoidable exposure to radiation. The 2013 NPS MedicineWise LBP program aimed to reduce unnecessary diagnostic imaging for non-specific acute LBP in the Australian primary care setting. The LBP program delivered referral pattern feedback, a decision support tool and patient information to 19,997 (60%) of registered Australian general practitioners (GPs). This study describes the findings from evaluation of the effectiveness of the 2013 LBP program at reducing X-ray and computed tomography (CT) scans of the lower back, and the financial costs and benefits of the program to the government funder.The effectiveness of the 2013 LBP program was evaluated using population-based time-series analysis of administrative claims data of Medicare Benefits Schedule (MBS) funded X-ray and CT scan services of the lower back. The CT scan referral trend of non-GP health professionals was used as an observational control group in a Bayesian structural time-series model. A retrospective cost–benefit analysis and cost-effectiveness analysis was conducted using program costs from organisational records and reimbursement data from the MBS.The 2013 NPS MedicineWise LBP program was associated with a statistically significant 10.85% relative reduction in the volume of CT scans of the lumbosacral region, equating to a cost reduction to the MBS of AUD$11,600,898. The best available estimate of program costs was AUD$141,154. Every dollar of funding spent on the 2013 LBP program saved AUD$82 of funding to the MBS for CT scan reimbursements. Therefore, from the perspective of the Australian Government Department of Health, the 2013 LBP program was cost saving. The program cost AUD$2.82 per CT scan averted in comparison to the scenario of no program. No association between the 2013 NPS MedicineWise LBP program and the volume of X-ray items on the MBS was observed..The 2013 NPS MedicineWise LBP program reduced CT scan referral by GPs, in line with the program’s messages and clinical guidelines. Reducing this low-value care produced savings to the health system that exceeded the costs of program implementation Low back pain (LBP) is a highly prevalent health problem. The 1-year prevalence of LBP is estimated to be 38% worldwide  Low backIn primary care in Australia, general practitioners (GPs) act as gatekeepers to various treatment pathways. Patients require GP referral to access care from specialist physicians and to receive government-subsidised imaging services outside of the hospital setting. In 2015–16, management of back complaints accounted for 3.1% of Australian general practice encounters.  the inveAlerting features (‘red flags’) for serious conditions associated with acute LBP have been published in Australian guidelines.  There isIn Australia, diagnostic imaging in the community is subsidised by the Australian Government via the Medicare Benefits Scheme (MBS). General practitioners can refer patients for MBS-subsidised computed tomography (CT) scans and X-rays, but not MBS-subsidised Magnetic Resonance Imaging (MRI) of the lower back. However, specialist physicians can refer patients for MBS-subsidised MRI of the back, or GPs can refer patients if they agree to pay the entire cost. A combination of referrer restrictions, specific clinical requirements for MRIs scans which can be MBS-subsidised and a licensing system unique to MRI machines contributes to a lower volume of MRI use compared to CT and x-ray modalities.  The use The use of CT scans represents significant diagnostic value but also potential risk. A positive association between exposure to CT scans and cancer risk has been reported in cohort studies of British  and AustThe inappropriate use of imaging for non-specific LBP is an important topic in the discussion about low value care in Australia. Through the Choosing Wisely Australia initiative, The Royal Australian and New Zealand College of Radiologists and the Australian Physiotherapy Association both include a recommendation to not request or perform imaging for patients with non-specific LBP with no indication of a serious cause. , 15 DiffNPS MedicineWise has been promoting the quality use of medicines in Australia since 1998. NPS MedicineWise is an independent organisation whose programs are funded by the Australian Government Department of Health. Since 2009, NPS MedicineWise has also sought to provide accurate, balanced, evidence-based information and services to health professionals and the community on the quality use of diagnostic tests. In 2013, NPS MedicineWise implemented the LBP program. The objective of the program was to educate consumers and GPs about the appropriate management and use of imaging tests for acute LBP. It built on previous NPS MedicineWise programs with the same objective, including one implemented in 2010. The anticipated outcome of the 2013 NPS MedicineWise LBP program was a reduction in unnecessary GP referrals for CT scans and X-rays of the lower back.In this study, we conducted an economic evaluation of the 2013 NPS MedicineWise LBP program. We aimed to identify the financial costs and benefits of the program to the Australian Government Department of Health and the program’s cost-effectiveness at reducing GP referrals for diagnostic imaging of the lower back.The program effectiveness study was a retrospective population-based cohort study using time-series analysis based on routinely collected practitioner level MBS data for the period May 2010 to February 2015, inclusive. The results informed the retrospective cost-effectiveness study. The reporting of this study is based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. The program was an educational feedback intervention which aimed to reduce inappropriate CT scans and X-rays of the lower back by improving GPs’ referral practice for acute LBP presentations. Consistent with Australian clinical guidelines,  the key In June 2013, the personalised MBS feedback report was sent to 19,997 registered practising GPs in Australia, 60% of the 33,337 Australian GPs.  The feedThe cover letter of the feedback report promoted free resources available from NPS MedicineWise to support GPs in managing LBP consultations, including managing patient expectations. The symptom self-management prescription pad was a pad of tear-off patient information sheets on how to manage acute LBP, including information about exercise, pain relief and imaging tests. The patient information sheets had space for the GP to write personalised advice.Back Pain Choices online decision support tool, based on a four step decision algorithm, assisted GPs in the diagnosis and management of LBP. A personalised information sheet for the patient could be printed out. The tool was developed in collaboration with the George Institute for Global Health and published online in 2012. [The in 2012. This study used the MBS national administrative database. The Australian government subsidises medical services that are listed on the MBS via Australia’s universal healthcare system, the Medicare Scheme. Listed imaging tests that occur outside of the public hospital setting are subsided by Medicare if an appropriate medical provider refers the patient. Provider level MBS data for CT scanning of the lumbosacral region (MBS item number: 56223) and X-ray of the spine lumbosacral and sacrococcygeal region (MBS item numbers: 58106 and 58109) were requested and provided by the DHS. The data contained the number of services and amount of benefit paid for each MBS item number grouped by individual referring provider and by month of service from May 2010 to February 2015. Since Australian GPs and GP registrars were the target population for the program and the cohort for this study, we requested that each referring provider in the data be classified by the DHS as either a GP or non-GP  according to a provider’s listed major specialty.The study evaluated the effectiveness of the NPS MedicineWise program at reducing the number of CT scans and X-rays of the low back referred by GPs. The MBS data was summarised as the total number of services and total amount of benefit paid each month for GPs and non-GPs. The total monthly volumes were used to measure the program’s effectiveness at changing utilisation and the total monthly amount of benefit reimbursed was used to analyse the monetary benefit associated with the program’s effect.A Bayesian structural time-series model (BSTM) , 26 was The BSTM method generalised the difference-in-difference approach to time-series analysis by forecasting the post-intervention period with a linear control predictor.  The undeProgram development and implementation costs were retrospectively calculated from NPS MedicineWise organisational records. Financial records were used to capture invoiced costs such as production, data, printing and distribution of the individualised MBS feedback intervention, the production and printing of the symptom self-management prescription pad, and out sourced development costs associated with the online decision aid tool. NPS MedicineWise staff costs were estimated from project management forecasting documents, timesheet data for equivalent tasks and staff salary data.Since similar products are regularly developed and implemented at NPS MedicineWise, resource data for five similar products were collected to estimate variation associated with the base case estimates. In the variation data based on actuals, the maximum value was 40% higher than the minimum value. For intervention components, which did not have data on similar products, a conservative estimate was applied of 50% higher or lower.A 25% increase was added to all human recourse costs to account for infrastructure and support services. All costs were adjusted to 2015 Australian dollars using the Consumer Price Index (CPI) published by the Australian Bureau of Statistics (ABS). The timeframe for the economic evaluation of the program was July 2013 to February 2015, 20 months post-program implementation. The effect of the program on GPs’ CT scan referral was discounted at an annual rate of 5%, calculated monthly after the first year.The Australian Government Department of Health funds both the MBS and the implementation of NPS MedicineWise quality use of diagnostics educational programs. From the perspective of this funder, the net benefit is the difference between the cost of NPS MedicineWise developing and delivering the program and the change in MBS expenditure on CT scans associated with the program. The monetary costs and benefits of the program from the perspective of the funder, the Australian Government Department of Health, were compared using the net benefit and cost-benefit ratio. The net benefit was calculated by subtracting the adjusted cost of implementing the program from the adjusted and discounted benefit of the program, defined as the change in reimbursement costs to the MBS associated with the intervention. The impact of the program on MBS services utilisation and expenditure was discounted at an annual rate of 5% after year one, calculated monthly. Incremental cost-effectiveness was estimated and compared to a hypothetical scenario of no program implementation.Sensitivity analysis for the estimates of cost-benefit ratio and incremental cost-effectiveness ratio (ICER) was conducted using estimates of variation in cost, benefit and outcomes derived from the 95% Bayesian credible intervals of MBS time-series analysis and the comparison of program costs to other similar programs delivered by NPS MedicineWise. Sensitivity analysis for the cost-benefit was estimated from the net benefit and cost-benefit ratio for the “best and worst” scenarios using the estimated maximum and minimum cost and benefit.For the cost-effectiveness analysis, univariate and probabilistic sensitivity analyses were conducted using TreeAge . UnivariBetween May 2010 and February 2015 (57 months), 1,119,796 CT scans of the lumbosacral region (MBS item number 56223) were referred by GPs, with a reimbursement cost of AUD$253 million. There was an upward trend in CT scan service volume .Following the intervention, the observed reimbursement of CT scans of the lumbosacral region referred by GPs was 11.05% lower than reimbursement without the intervention estimated by the counterfactual. This reduction was statistically significant. The intervention was associated with an estimated AUD$11,600,898 cost reduction between July 2013 and February 2015 .The BHTS method could not be used to analyse X-ray service volume (MBS item numbers 58106 and 58109) due to a lack of statistical stationarity and similarity between the GP and non-GP time series Figure. . ConventA summary of results is presented in Table Program costs were estimated to be between AUD$73,734 and AUD$195,960. A cost of AUD$141,154 was used as the base case, based on the best available estimates of costs. Based on disaggregation of the main components’ costs, the program cost AUD$7.05 per GP receiving the intervention. The online decision support tool, MBS feedback report and symptom self-management pad accounted for 41.0%, 40.6% and 4.2% of the program’s cost, respectively. Infrastructure and support services accounted for 14.2% of costs.The net benefit of the 2013 NPS MedicineWise LBP program to the Australian Government was estimated to be AUD$11.4 million  as a hypothetical comparator. The 2013 NPS MedicineWise LBP program cost approximately AUD$141,154 to implement. The CT scan service volume in the comparator scenario was defined as the modelled counterfactual estimate without the intervention, presented in Figure. The ICER was $2.82 per lumbosacral CT scan averted. Thus, from the perspective of the Department of Health, the AUD$2.82 spent to avert a lumbosacral CT scan was less than $231 average cost of the CT scan to the MBS, making the program dominant to the comparator of no program.Following a ‘best and worse’ case scenario sensitivity analysis, the cost-benefit ratio for the program’s effectiveness at reducing CT scans ranged from 1:5.57 (maximum estimation of costs and minimum estimation of benefit) to 1:299.07 (minimum estimation of costs and maximum estimation of benefit).Univariate sensitivity analysis of the ICER estimates gave an ICER range of AUD$1.47 to AUD$36.10 per CT scan averted. This range is lower than the AUD$231 willingness to pay threshold, which is the average MBS reimbursement for a lumbosacral CT scan. The ICER was most affected by introduction of the uncertainty in the estimate of the program’s impact. The 2013 NPS MedicineWise LBP program had an ICER less than AUD $231 in 98.3% of iterations in the probabilistic sensitivity analysis  as the widest-reaching intervention. The audit and feedback intervention was similar to one implemented by NPS MedicineWise two years earlier, adding support to the notion that repeated interventions are important to sustain behaviour change. While the 2013 NPS MedicineWise program included interventions with a lower intensity and burden on the GPs involved than other successful Australian programs that aimed to improve the management of LBP, such as the IMPLEMENT trial,  it’s strSeveral limitations should be considered when interpreting the results of our study. First, costs associated with program design and implementation were estimated retrospectively. However, audit and feedback interventions are regularly implemented at NPS MedicineWise for different therapeutic programs so multiple programs were available for our cost estimates. In addition, our sensitivity analysis demonstrated that the program was cost-saving even with variation in the cost estimate. Second, it was not possible to measure how appropriate observed changes in GP referral behaviour were as MBS data do not include reasons for test referral. We were unable to determine whether the reduction in GP referral for CT scans was associated with unintended consequences, such as missed or delayed diagnosis. As a result, we have assumed that the reduction was of unnecessary CT scans, consistent with an economic evaluation of a program promoting the same LBP guidelines  as our pThis study demonstrated the effectiveness of a large-scale program at improving clinical practice in line with evidence-based guidelines. The 2013 NPS MedicineWise LBP program was associated with a decrease in CT scans of the lower back referred by GPs. The findings of the economic evaluation suggest that addressing areas of low-value care in the primary care setting with targeted interventions can have a positive financial impact on the health system and reduce patient exposure to clinical interventions that may carry more health risks than benefits.


Policy makers and program managers are better enabled to draw relevant lessons from implementation research and program experience elsewhere when there is richer documentation on what was done and what key contextual factors may have influenced outcomes. Newly developed Program Reporting Standards from WHO provide helpful guidance on what is needed for optimally useful documentation. Policy makers and program managers are better enabled to draw relevant lessons from implementation research and program experience elsewhere when there is richer documentation on what was done and what key contextual factors may have influenced outcomes. Newly developed Program Reporting Standards from WHO provide helpful guidance on what is needed for optimally useful documentation. Koek.See related article by http://www.equator-network.org/). These standards have not only helped improve reporting of research results but also both raised the bar on quality of the research itself and strengthened guidelines based on evidence synthesis. Nevertheless, this development has been largely driven in response to felt needs of investigators and journal editors rather than those of program managers and policy makers.An important development over the past couple of decades has been the emergence of reporting standards for various types of research. In the early 1990s, journal editors and investigators agreed on standards for reporting randomized controlled trials, issuing the Standardized Reporting of Trials statement.Certainly, much is gained from translation of research evidence to practice. But achieving important gains in the complex, messy, real world of programs requires also that we draw evidence from practice.Achieving important gains in the complex, messy, real world of programs requires us to draw evidence from practice.Hales et al. (2016)providing there is sufficient information on context and how the interventions were delivered. In addition to studies published in the peer-reviewed literature, good gray literature documentation of such program experiences can be a valuable resource. But to date, although StaRI is a very helpful addition, there hasn't been similar attention to elaborating reporting standards for gray literature. As discussed in the article by Koek et al.Program Reporting Standards for Sexual, Reproductive, Maternal, Newborn, Child and Adolescent Health (PRS), which helps address this gap.For policy makers and program managers seeking to draw on what has been learned from programs in other settings that might be relevant to their own, published studies can be ahelpful source of such insights, Program managers seeking to draw on learnings from programs in other settings need sufficient information on context and how the intervention was delivered.Implementation design and logic modelContextThe time dimensionThe actors involved in deliveryResource requirementsThe planning, piloting, and scale-up processSpecific activitiesQuality assurance, monitoring, and evaluationCoverage and reach achievedAdaptations (fidelity vs. flexibility)SustainabilityStrengths and weaknessesLessons learned and implicationsKoek et al. acknowledge the value of current research reporting standards but they also point out that for program managers and decision makers existing standards have given insufficient attention to documentation needed on context and implementation details. The new PRS give appropriate attention to issues including:More systematic attention to such issues in gray literature reports can make them considerably richer, more useful learning documents for decision makers in other settings who are trying to draw lessons that could be relevant for their own programs.The new WHO Program Reporting Standards can help make program documentation richer and more useful for decision makers in other settings.At GHSP, we believe these new standards are also useful in specifying the kind of information that can make peer-reviewed implementation and operations research more valuable to program managers. In the Internet age, even hard-copy journals can offer their authors the option of online annexes, to provide richer detail on context and implementation. At the end of the day, we are better able to deliver effective programs and achieve improved population-level health outcomes if we can draw lessons from each other's program experiences.Global Health: Science and PracticeWe fully realize that the PRS represents something of an idealized model. Much useful evidence is developed that does not comprise every element in the standards. And the real world of program implementation can be messy and challenging. Still the elements in the PRS are to a very large extent what we look for in considering what papers to publish. We encourage colleagues in the global health community to use the PRS with vigor.–


Community pharmacists are accessible, knowledgeable, and capable of providing mental health promotion and care in communities. This may not be a role that is recognized by the public, and men in particular. Differences between men and women exist in help seeking practices. Headstrong–Taking Things Head-On is a men's mental health promotion program for community pharmacies that was designed to increase the capacity of community pharmacists in caring for men with lived experience of mental illness and addictions. The program's core components included signage in pharmacies, education and training for pharmacists, and a website for use with patients.We applied the Theoretical Framework of Acceptability as the coding scheme to pharmacists' qualitative interviews to examine the acceptability of Headstrong for pharmacists.Nine pharmacists consented to participate and all chose telephone interviews. With the exceptions of ethicality, affective attitude, and opportunity costs, all components from the TFA were coded in each of the nine transcripts. The most frequently coded constructs were perceived effectiveness of the intervention, burden, and self-efficacy. These were coded at least 20 times. The remaining categories ethicality, intervention coherence, affective attitude, and opportunity costs were coded between 11 to 17 times. Pharmacists' perceptions of the effectiveness of the program was mixed. The overall burden was perceived to be low, but opportunity costs appear to have limited the participation of some pharmacists in the program.Use of the Theoretical Framework of Acceptability as a coding scheme for qualitative data from community pharmacists in a men's mental health program was helpful for identifying issues with the program that may require redesign . Program design should consider how services are advertised in the pharmacy setting, how personal values of pharmacists influence intervention coherence, and whether minimizing the burden of an intervention negates issues related to opportunity costs. Headstrong is targeted to men in Nova Scotia, Canada, and was implemented in community pharmacies. Headstrong also has a standalone website (https://headstrong.life) that can be used by pharmacists and men with lived experience of mental illness and addictions. Headstrong focuses on six health topics: depression, anxiety, insomnia, alcohol use, tobacco use, and suicide. Overall, Headstrong is intended to build capacity in pharmacists and pharmacy teams so that they can help men access services, care, and community supports through the community pharmacy context. The core components of Headstrong included signage in the pharmacies to draw people to the pharmacy to ask about the program, an education program for pharmacists, and resources  for pharmacists to use with men. Signage included a 170 cm vertical mobile banner and a variety of shelf and counter signs. A training day for Headstrong was held to introduce and train pharmacists on the philosophy of the program and its resources. Additionally, pharmacists used an online learning management system that contained learning modules for each of the six areas of Headstrong and a module on motivational interviewing. The program's public website was designed and intended to support pharmacists and men in finding credible, curated resources for the content areas of Headstrong.Headstrong–Taking Things Head On  2.2.All pharmacists (n = 31) of the 23 participating pharmacies were invited to participate in the Headstrong interviews. Consent was required for participation as per ethical requirements. The research coordinator notified pharmacists of the opportunity to participate in an interview to discuss the program. A research coordinator offered to conduct interviews via telephone or face-to-face. Interviews were audio-recorded, transcribed verbatim, and cleaned and anonymized by a research coordinator. The interview guide included questions such as, “Tell me about what the Headstrong program is to you?” and “Tell me about your ability to continue with/sustain elements of the Headstrong program as time went on?” The interview guide is available as an Click here for additional data file.2.3.All interviews were conducted by the research coordinator who was completing a Masters of Public Administration at the time of the interviews and who had experience with qualitative research. The team leads (Murphy and Gardner) both have Doctor of Pharmacy degrees and are educators, researchers, and clinicians through our appointments with Dalhousie University. We identify as a woman (Murphy) and a man (Gardner). We have over 40 years in combined experience of clinical care and research in mental illness and addictions. We also believe in the importance of donating resources such as volunteering time and fundraising for non-governmental, community-based mental health organizations. A significant pillar in our program of research, in which we consistently use mixed methods, focuses on the design, implementation, and evaluation of programs, such as Headstrong and the Bloom Program As per Finlay 2.4.We used a directed content analysis technique with deductive coding Coding was conducted individually by authors using the TFA. Each author followed a process of reading the transcript aloud once followed by re-reading the transcript and using line-by-line coding. Following coding of each transcript, the authors met to discuss their coding. We attempted to use one code from the TFA to isolate the meaning of the text versus double-coding passages of text.2.5.Ethics was received for the study from the Dalhousie University Research Ethics Board, file number 2015–3728. Written informed consent was obtained prior to the interviews. The signed consent forms were received from the pharmacists in advance of the interview in one of three ways: 1) a postal mailing of the signed consent form; 2) a facsimile of the signed consent form; or 3) a signed copy obtained via a face-to-face meeting.3.3.1.Nine pharmacists consented to participate in the interviews. All participants chose telephone interviews. Each interview began with a review of the project and the consent process. Interviews took 20 minutes on average.3.2.Four of seven TFA constructs were coded in each of the nine transcripts . Coded i3.2.1.The anticipated versus the experienced effectiveness of Headstrong was mixed for pharmacists. All nine pharmacists discussed that they or a colleague on staff at their pharmacy had used at least one resource from Headstrong  during the program. For many, they discussed successes with incorporating the Headstrong resources into their regular care and patient consultations as another tool to help people. For example, pharmacist 8 indicated, “Well, for me the two biggest things were anxiety and the smoking cessation. …I certainly was comfortable and glad to point them in the direction of those tools for either of those.” Pharmacist 2 also commented, “I refer a lot to the apps for anxiety and depression, especially TruReach and things like that.”Several pharmacists indicated that the signage as a part of Headstrong did not produce the desired results and expressed disappointment that more people did not approach the pharmacists. Pharmacist 8 commented on the success with the resources but noted the issues with the signage, “I've referred patients to the website. …the postcards…and the brochure. We had the signage up. I'll be honest with you…we took the signage down probably a month ago. …Unfortunately it was in the way of some things, and management wanted it removed.” Similarly, pharmacist 3 stated, “My boss felt there was way too much actual stuff.” and that concern by management and staff was that “To the point that people just ignored it because there was just so much of it.” However, to contrast, pharmacist 1 indicated that other store staff were accepting of the signage and thought more could be of use and reflected on the placement, “we have signage located throughout the store.”3.2.2.The coding for burden predominantly captured discussions of minimal efforts being required to participate in Headstrong. At least four of the pharmacists discussed pre-existing involvement with another mental illness and addictions program. Several additional pharmacists described a passion for advocacy and the desire to help those with mental illness and addictions and being “committed to these folks” (pharmacist 3). Pharmacist 2 expressed the sentiment discussed by many, “Both pharmacists [at pharmacy] are pretty open and helpful when it comes to, you know, working with patients with mental illness. I don't think there was a big change. It's just more tools at our fingertips, I guess, when working with that patient population. That's the biggest change.” The concept of burden in a negative connotation was evident when pharmacists brought forward issues with competing demands in the workplace. Pharmacist 8 discussed, “Workload is always an issue…. It's a very busy pharmacy. So it's sometimes hard to find the time. But again, just the nature of the program, it doesn't take a lot of time. …It's difficult to engage other…coworkers to participate in it. You know, I think that was on their end though, not so much my end or the program's end.”3.2.3.Self-efficacy by the pharmacists was expressed when they described learning, being comfortable in recommending the resources, and discussing examples of successfully helping men. Pharmacist 5 discussed being able to change their “style of engagement” and “the information I share” with patients as a result of participation. The program also encouraged pharmacist 9 to do additional training in motivation and behavioural interviewing and described, “I definitely have found that people, especially men, are more forthcoming and that quiet pauses in between aren't something that I need to fill. …I let them think. …And it definitely has made a difference in my practice. And even my boss said to me, …‘Why do people tell you things?’.”3.2.4.Ethicality was coded in seven transcripts and was coded when pharmacists discussed values from their personal and professional roles in practice. Pharmacist 5 indicated, “I think it's great. I think it's important. I think it's probably lacking. I think it's a good way to help the healthcare system where there seems to be a lot of people that maybe don't get the help they need or don't have the time with other healthcare providers that they're seeking.” However, there was also tension for at least three pharmacists regarding the gendered nature of the program that may have impacted their use. For example, pharmacist 3 indicated, “Quite frankly because it's for men only, that always has pissed me off from day one. …There's no reason for it to be a gendered thing. And if it became a non-gendered thing, I think I'd be a lot more comfortable saying, ‘Hey, we have this thing Headstrong. It's really great. There's a lot of really great information here.’ I just hate that it's for men only, to be quite frank.” Others, as demonstrated by pharmacist 2, discussed professional duties  to patients, “I think it's our job. If there's something that you can do that helps your patients, I think it's your responsibility to do it. And these non-pharm things are as important, if not more, than the medications. So being aware that they exist and knowing about them, and being able to provide the information on them to your patients I think is part of our job and our duty.” Pharmacist 9 also discussed experiencing ethical dilemmas when considering resources that may help but may also create harms due to financial considerations with the more expensive apps on the website. They indicated, “And we just had a major shut down of [employer] where a lot of people were employed. …people are struggling. I don't want to put any more stress on people by saying, ‘Oh, yeah, try this app for $100.’ And they're like, ‘I can't even feed my kids this week, and that's why I need this app,’ to like de-stress or something.”3.2.5.All interviews had evidence of coding for intervention coherence. Pharmacists demonstrated clarity on the purpose of the program. However, in at least one example, the program may have been misinterpreted in term of the target demographic “I think a lot of the population that might benefit is very unstable.” (pharmacist 5). Pharmacist 9 described the purpose of the program and integrated their own personal example to demonstrate how it had worked, “So, the Headstrong program to me is just basically getting awareness out to men especially that it's okay that you talk about your feelings, and that it's not a bad thing, and that you don't have to keep it all in, and that there are resources for them, and that they're not alone. I get that a lot from patients. They don't want to tell anybody. Or, like they'll come in and they'll say that, you know, they've been suffering for 20 years and never told anyone. So, I think it's great that way.”3.2.6.The majority of participants discussed positive feelings about the program. However, there were negative feelings that manifested in several ways that were brought out by the program. As mentioned, when the gendered nature conflicted with core values around inclusivity in practice, this created negative emotions for several pharmacists.The activities that pharmacists engaged in during Headstrong were not part of a compensation model for community pharmacy. This brought to light issues with roles and compensation. Pharmacist 7 described, “I think there's a good spot for it [Headstrong program]. …Like the pharmacist is a very accessible person in the healthcare team. But as a pharmacist and as a manager who's trying to decide on staffing and that sort of thing, it's frustrating that the government doesn't support us or see us in that role even though we're doing that role. So, we're the most accessible healthcare professional, that you don't have to be triaged, and you don't have to have an appointment a lot of the times. And yet there's no compensation for that or no recognition in the payment model.”3.2.7.The opportunity costs expressed by pharmacists were often implicit versus directly stated and involved concerns around diverting time and money away from other tasks or removing human resources from services that generate revenue. Pharmacist 7 discussed concerns prior to attending the Headstrong training day as expressed to them from the pharmacy management, “At first they weren't really keen on it because it involved me being off-site for a day, and there was no fee for running the program here.” Pharmacist 9 indicated issues related to workload around workplace context issues and with other reimbursed interventions taking priority, “It was more difficult for us I think than maybe other stores because we were going through a renovation…. And with the flu shot season at the same time. So, I found it very hard. It's always hard during flu shot season to get any good interaction with patients because you're just so busy. …So it's hard, especially with flu shots, to actually use it. I wish it had been during a different time of year probably.”4.The Theoretical Framework of Acceptability was useful in determining several acceptability factors that may have influenced how pharmacists viewed and used the Headstrong program. These findings can be applied more generally for revising community pharmacy-based interventions and for improving their implementation. For example, when discussing Headstrong, the flexibility in using the resources for both men and women may not have been clear in our pharmacist participants as demonstrated by some comments from interviewees. This demonstrates that more work would be beneficial prior to intervention training in assessing participants' values, how these values align with respect to intervention coherence, and mechanisms to improve intervention coherence when values differ.Importantly, it was noted when examining perceived effectiveness through the coding process that the signage related to the program may have been problematic in several locations. Pharmacists did not perceive that the signage created more inquiries in men's mental health care, which was disappointing for some. Others found issues with the gendered approach to the program and the signage. These factors may have also influenced how pharmacists felt towards the program , its effectiveness, and potentially impacted intervention coherence. The presence of the signage as a physical barrier as pointed out by some pharmacists is also important to note. Community pharmacies in the Canadian context are primarily retail spaces with high use by the public. Signage may impact advertising and potential sales for other items, traffic flow, and create other hazards  in the pharmacy. Signage has been identified as an important factor as part of overall health literacy The concern for the gendered signage brought forward important points regarding program designs and implementation and also the preparedness of pharmacists to engage with diverse groups in pharmacy practice. There has been more work in recent years to provide education to pharmacists for transgender and gender diverse individuals In our study, pharmacists perceived minimal effort  when participating in Headstrong. This may have been influenced by the fact that more than half of the sample discussed participation in mental illness and addictions programs at their pharmacies or identified as having a passion for the population, potentially signifying a more motivated group. However, issues with opportunity costs were also brought forward, which inherently could negate perceptions of low effort. Participants discussed issues with time, staffing, and competing demands in practice that would not allow them to consistently add Headstrong into their practice. For example, influenza vaccinations, which are a reimbursable service and seasonally increase workload for pharmacists, co-occurred during Headstrong delivery and took priority. The issue of lack of compensation for the Headstrong program was also directly brought forward by a pharmacist in our study. These findings regarding issues with time and incentives concur with those of others. For example, pharmacists from an Internet-based questionnaire in the US with specialty training in psychiatry also found barriers including time and issues with compensation 4.1.To our knowledge, this is the first study to use the Theoretical Framework of Acceptability to examine pharmacists' acceptability of a health promotion intervention in community practice. The analysis produced useful findings to consider for future iterations of Headstrong's design and implementation.Headstrong pharmacists were a convenience sample of those who were interested in the program and wanted to participate. Only nine pharmacists from the 23 participating pharmacies consented to interviews therefore limiting the sample size and requiring caution for the interpretation of the results. This could indicate that volunteer bias is present in our overall sample of those wanting to offer Headstrong at their pharmacies and for those who wanted to participate in the interviews. We did not collect demographics from those who were interviewed given the small sample size of the program and the potential to identify those who participated in the interviews.5.Use of the Theoretical Framework of Acceptability in evaluating the experiences of community pharmacists in a men's mental health program helped to identify issues with the program useful for informing changes in its design . Pharmacists' perceptions of the effectiveness of the program was mixed and this may have potentially been impacted by public interest, values of participants, and opportunity costs in the face of competing services. Many pharmacists in the study had high self-efficacy for the service and were encouraged to change their practice and expand their professional development opportunities based on their involvement with the program. Design of programs should consider how services are promoted and advertised in the pharmacy setting, how personal values of pharmacists influence intervention coherence, and whether minimizing the burden of an intervention negates issues related to opportunity costs.


Aoerg19 in Aspergillus oryzae using bioinformatic analysis. In this study, we showed that AoErg19 function is conserved using phylogenetic analysis and yeast complementation assay. Quantitative reverse transcription–PCR (qRT-PCR) indicated that Aoerg19 expression changed in different growth stages and under different forms of abiotic stress. Subcellular localization analysis showed that AoErg19 was located in the vacuole. Overexpression of Aoerg19 decreased the ergosterol content in A. oryzae, which may due to the feedback-mediated downregulation of Aoerg8. Consistent with the decrease in ergosterol content, both Aoerg19 overexpression and RNAi strains of A. oryzae are sensitive to abiotic stressors, including ergosterol biosynthesis inhibitor, temperature, salt and ethanol. Thus, we have identified the function of AoErg19 in A. oryzae, which may assist in genetic modification of MVA and the ergosterol biosynthesis pathway.Mevalonate diphosphate decarboxylase  is a key enzyme of the mevalonic acid (MVA) pathway. In fungi, the MVA pathway functions as upstream of ergosterol biosynthesis, and MVD is also known as Erg19. Previously, we have identified  Mevalonate diphosphate decarboxylase  is a key enzyme of the mevalonic acid (MVA) pathway . It cataCandida albicans, human, Arabidopsis, and Ganoderma lucidum can complement the temperature-sensitive phenotype of the Saccharomyces cerevisiae mvd mutant , as most of the other genes in the ergosterol biosynthesis pathway have multiple paralogs  was obtained from the China Center of Industry Culture Collection  and used as the wild type strain. The A. oryzae 3.042 aaapyrG was constructed using Agrobacterium tumefaciens-mediated transformation in our laboratory (A. oryzae was cultured on (Czapek-Dox Agar) medium  at 30°C. Spores were harvested after 3 days of culture by scraping the agar surface with a sterile glass spreader under a laminar flow hood and suspending the spores in sterile water containing 0.05% Tween-80. The concentration of spores was determined using a hemocytometer. Escherichia coli DH5α . Total RNA was extracted from 0.5 g mycelia using a fungal RNA kit  according to the manufacturer’s instructions, with the addition of an RNase-free DNase I treatment (Omega). The quality and concentration of the DNA and RNA samples were determined using NanoDrop ND-1000 spectrophotometer .TM RT reagent kit . All quantitative reverse transcription–PCR (qRT–PCR) measurements were performed using a CFX96 real-time PCR detection system  with SYBR Premix Ex Taq , according to the manufacturer’s instructions. The house-keeping gene encoding histone H4 was used as a normalization control  mutant was purchased from EUROSCARF1 and the yeast strain BY4741 was the wild-type control. Plasmid pYES2-Aoerg19 contains Aoerg19 full-length coding sequence (CDS) under the control of the GAL1 promoter. To construct pYES2-Aoerg19, the CDS of Aoerg19 was amplified using PCR . The recombinant plasmid was sequenced. Then, the pYES2-Aoerg19 construct was transformed into the yeast mutant. The control and transformants were grown on YPD , and YPG  to assess the phenotypes.The Aoerg19 was fused to DsRed at the C- and N-terminal of AoErg19 using fusion PCR, and the primers are listed in XhoI and BamHI, followed by cloning of the fusion DNA fragments into pEX2B using a one-step cloning kit  to construct pEX2B-AoErg19-DsRed and pEX2B-DsRed-AoErg19 vectors. For the RNAi of Aoerg19, the 500 bp sequence of 5′ Aoerg19 mRNA was cloned to construct the hairpin structure, and the 79 bp intron of α-amylase was used as the loop of the hairpin structure by three times PCR using RNAi-forward R1, R2, and R3. The primer sequences for RNAi are listed in A. tumefaciens AGL1. Then, the vectors were transformed into A. oryzae 3.042 aaapyrG according to the procedure published by our laboratory , SRL amino acid, was fused to the C-terminal of GFP using PCR to visualize peroxisomes (ptrA (ptrA-GFP-PTS1 (ptrA-MTS-GFP. Aspergillus nidulans vacuolar carboxypeptidase Y (CPY) and A. oryzae vacuolar protein AoVam3 were used as vacuole maker by fusion the GFP at the C-terminal and N-terminal, respectively . A 488 nm and 554 argon ion laser line was used for excitation of GFP and DsRed, while 505–515 nm and 585–595 nm emission filters were used for simultaneously capturing GFP and DsRed fluorescence, respectively, using the Olympus FV1000MPE2 confocal microscope. Primer sequences used for plastid construction were provided in The constructed pEX2B-Aoerg19-DsRed and pEX2B-DsRed-Aoerg19 plasmids were transformed into oxisomes . Then, tes  was added to the 50 mg dry A. oryzae powder and mixed by vortexing for 1 min. The suspensions were then transferred to glass tubes and incubated at 85°C in a water bath for 1 h, after which the tubes were allowed to cool to ambient temperature. Three milliliters of n-heptane  was added to the tube, followed by vigorous vortexing for 3 min to extract ergosterol. The upper layer (the n-heptane layer) was isolated and stored at -20°C for 24 h prior to high-performance liquid chromatographic (HPLC) analysis. HPLC was conducted on Waters Alliance e2695 HPLC  using a UV detector set at 282 nm with a Zorbax SB-C18 column. Methanol/water  was used as the mobile phase, and the elution rate was 1.5 ml min-1. Ergosterol (Sigma-Aldrich) was used to obtain a calibration curve. The measurement of mevalonic-5-PP and isopentenyl-PP content was performed by using ELISA kits  according to the manufacturer’s instructions.Ergosterol extraction and quantification was performed according to 2. The IDs of the sequences are as follows: Zea mays-MVD1 (GI: 670375213), Zea mays-MVD2 (GI: 1142711841), Oryza sativa-MVD1(GI: 1443059899), Oryza sativa-MDV2 (GI: 1002245003), Danio rerio (GI: 55925435), Mus musculus (GI: 256985114), Homo sapiens (GI: 568815582), Arabidopsis thaliana (GI: 240254678), S. cerevisiae (GI: 330443715), A. niger (GI: 966767527), A. oryzae (GI: 391869300), A. flavus (GI: 238491808), E. coli (GI: 1205338453), Lactobacillus composti (GI: 737476048), and Staphylococcus aureus (GI: 739736044). Phylogenetic analysis was performed using the MEGA 5 software  is nearer than that of S. cerevisiae. However, MVD evolved two different paralogs in dicotyledons such as Zea mays and Oryza sativa  mutant is temperature-sensitive  and YPG  media at 30°C and 37°C to determine whether the temperature-sensitive phenotype of erg19 was restored. Results showed that the erg19 mutation was lethal at 37°C on YPD and YPG; Aoerg19/erg19 can restore the lethal phenotype of the yeast on YPG  in the ergosterol biosynthesis pathway. Results showed that with the exception of AohmgB, the expression of these genes was reduced in AoOE1#, and Aoerg8 showed the maximum reduction (AoErg8 is the feedback target in the Aoerg19 overexpression strains; of the expression of these genes was lowered in AoRNAi3#, with the exception of AohmgC (eAoerg19 mRNA levels in A. oryzae impair sporulation and decrease ergosterol content.Because revisiae . Hence, revisiae . Howeverrevisiae . We alsoecreased ; in OE secreased . To inveeduction , indicatf AohmgC . Taken tS. cerevisiae rendered it more sensitive to ergosterol biosynthesis inhibitor (erg19 increased sensitivity to NaCl stress (Aoerg19 overexpression and RNAi strains to different abiotic stresses, including ergosterol biosynthesis inhibitor, temperature, salt, and ethanol. The differences among the three colonies on the DPY media were not obvious at 37°C (Aoerg19 appear to increase sensitivity to different abiotic stress conditions.Ergosterol deficiency in nhibitor , and ovel stress . Therefo at 37°C . However at 37°C ; the hyp at 37°C . The AoO at 37°C . Interes at 37°C  which maS. cerevisiae (A. oryzae. Our previous study revealed that the ergosterol biosynthesis pathway is conserved between S. cerevisiae and A. oryzae by bioinformatics analysis. However, the ergosterol biosynthesis genes in A. oryzae are more complicated as there are multiple paralogs encoding the same biosynthetic enzymes (Aoerg19 does not only of significance in understanding MVA pathway but also in ergosterol biosynthesis pathway.The identification of genes involved in MVA pathway is important for genetic modification of MVA pathway, which has potential application for producing isoprenoids, an important feed-stock for commercial production of synthetic rubber and a high energy biofuel . Bioisoprevisiae . However enzymes . The resA. oryzae has not been identified. In this study, phylogenetic analysis and yeast complementation assay suggested that MVD function was conserved across fungal species (Aoerg19 overexpression and RNAi strains of A. oryzae showed delayed sporulation and sensitive to abiotic stress, indicating that ergosterol is involved in sporulation and stress response. Besides, we have noticed that under ethanol stress, the color of the overexpression colony turns red, which may be caused by the accumulation the AoErg19-DsRed protein, indicating that ethanol stress can also affect the accumulation of AoErg19. The subcellular localization of MVD in different species has been studied and it was reported to vary in plants and animals. In plants MVD was located in peroxisomes, whereas in animals it was mainly located in the cytosol (A. oryzae was located in the vacuole (The MVA pathway was conserved among different species . The fun species , 2. The  cytosol . However vacuole . The difS. cerevisiae resulted in squalene accumulation, but did not affect the content of ergosterol (S. cerevisiae (Ergosterol is a specific cell membrane component that is not only an important cell membrane component that affects fluidity and permeability, but also plays important roles in fungal growth, reproduction, and stress tolerance . Usuallygosterol . Other erevisiae . In filarevisiae . Little A. oryzae, the research about MVA or ergosterol pathway is of great significance because A. oryzae is the GRAS microorganism and has broadly applied in industry. Our previous studies showed that ergosterol biosynthesis in A. oryzae is more complicated than in S. cerevisiae (Aoerg19 resulting in the decreased ergosterol content by feedback regulation of Aoerg8 (A. oryzae strains which was possible the comprehensive results of the gene expression of Aoerg8 and Aoerg19 (A. oryzae more sensitive to abiotic stress (In revisiae . In thisf Aoerg8 . The int Aoerg19 . Consistc stress . We obsec stress . One expAoerg19. Overexpression of Aoerg19 decreased ergosterol content, suggesting Aoerg19 is the rate-limiting enzyme in the sterol biosynthesis pathway of A. oryzae. Further studies about the regulation network in ergosterol biosynthesis can be done to uncover the feedback regulation of MVA and ergosterol pathway. Thus, the gene function identification of Aoerg19 in A. oryzae may provide some useful information for the genetic modification of MVA pathway in A. oryzae.In summary, this study determined the function, expression pattern, and subcellular localization of A. oryzae transformation and manuscript revision.ZH and BZ designed the study and wrote the manuscript. YS and YN performed the main experiments. HH participated in the subcellular localization of AoErg19. BH and LM participated in the analysis of sterols. YT participated in the confocal microscopy. V-TT contributed to technical support of The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.