Recent trends in faculty development demonstrate a shift from short term to long-term programs; formal to informal learning in the workplace; individual to group settings; and from individual support to institutional support. The purpose of this study was to develop and evaluate a one-year Clinical Teaching Fellowship (CTF) program designed to equip early career medical practitioners and basic scientists with necessary skills to facilitate Team-based learning (TBL).The CTF program provided formal training, a choice of informal professional development activities, and practical co-teaching opportunities in TBL. Of the 40 registrants, 31 (78%) completed the program. Data were collected via questionnaire and focus group. Data were analysed using descriptive statistics and framework analysis.Participants considered the CTF program as relevant to their needs and useful to their career. Learning was enriched through the combination of training, practical teaching experience alongside senior clinical teachers, the multi-disciplinary context of training and co-teaching in TBLs; and the sense of community. Competing clinical responsibilities made it difficult to attend training and TBL teaching.The CTF program provided a longitudinal faculty development framework promoting preparation, practice and development of teaching skills. Securing institutional support to invest in the growth and development of early career teachers is essential to sustained innovation and excellence in teaching. Faculty development within healthcare education refers to the activities in which staff participate to improve their knowledge and skills as teachers, educators, leaders, managers, researchers, and scholars . Within Medical practitioners and basic scientists play an important role in educating the next generation of medical practitioners. A recent systematic review of faculty development programs within medicine found that formal, structured activities in group settings are most common . These pIn 2017, the first and last authors (AB and GF) developed and implemented a new “Clinical Teaching Fellowship” (CTF) program at the University of Sydney School of Medicine. The program was introduced to transfer theory to practice and achieve a sustainable, scalable faculty development program. It was designed to align with the introduction of Team-based learning (TBL), as a new method of teaching within Years 1 and 2 of the post-graduate medical program . With anTheories informing educational practice offer valuable lenses to analyse learning . Dornan Organisational support: to ensure the learning experience matches the desired outcomes, and provides opportunities for workplace practice.Pedagogic support: provided by teachers in the workplace setting, including mentors, role models and supervisors.Affective support: provided by a warm and inclusive environment.The ExBL model suggests that learning outcomes are acquired through provision of a supportive learning environment, and participation in authentic workplace activities. This model provides a transferrable blueprint for education that is applicable to the faculty development context . AccordiThe purpose of our study was to develop, implement and evaluate a longitudinal (1 year) Clinical Teaching Fellowship (CTF) program for junior medical staff and basic scientists. Further, to explore participants’ perceptions of the structure, processes and outcomes of the program, utilising the conceptual framework of ExBL.The CTF program was largely designed to prepare medical practitioners and basic scientists, for their new roles as skilled TBL facilitators; and potential future leaders in medical education. A coherent and flexible platform of activities was provided, that built upon each other to make use of various instructional formats, and to fit the personal needs of participants. The learning activities served as exemplars of modern learning methodologies.They were defined based upon the specific skills required by the medical program, and aligned with the educator’s interest.Mandatory, formal requirements to succeesfully complete the CTF program included attendance at two TBL teaching and leadership training (faculty development) sessions, observation of one TBL class; and co-facilitation of four TBL classes. Each TBL class was 2.5 h in duration. Most of the training was run in TBL format, providing a scalable model for future implementation. Optional, informal activities included participation in: the Clinical Teacher Training program ; careersEarly career medical staff  at teaching hospitals affiliated with the University of Sydney School of Medicine; and basic scientists  from associated medical research institutions were invited by email to participate in the CTF program. Our aim in recruitment was to be inclusive, rather than targeting those who had already taught in the medical program.the content of the CTF program, and the training providedTBL facilitation experiencenetworking opportunitiesintention to participate in future student teaching activities; andintention to participate in future CTF program activitiesQuantitative data were collected from participants by a post-program questionnaire, reflecting on participants’ perception of:Participants were asked to respond to 12 closed items, using a five-point Likert scale ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (5). Data were analysed using descriptive statistics.Qualitative data were collected via questionnaire and focus group. The questionnaire contained two opened ended questions: “What did you find to be the most useful aspects of the Clinical Teaching Fellowship program?” and “What suggestions would you make for improvement to the Clinical Teaching Fellowship program?”. Additionally, at the end of the final face-to-face session, all participants were invited to a focus group. The focus group was recorded and transcribed verbatim. Following consultation with all authors, the first author (AB), used framework analysis  to code The University of Sydney Human Research Ethics Committee approved the study: project number: 2018/685.In 2018, 30 medical practitioners and 10 basic scientists registered for the CTF program. Of the participants, 17 were male and 23 female. The medical practitioners included: 25 speciality trainees, three consultant physicians, and two consultant general practitioners. Participants were based across 10 metropolitan teaching hospitals and affiliated institutions. Thirty one out of 40 (78%) participants completed the CTF program. Of these 31 participants, six were basic scientists, and 25 were medical practitioners. In total, the 31 CTFs provided approximately 372 h of voluntary face-to-face teaching, an average of 12 h per participant.Following the graduation dinner, 23/31 (74%) of participants completed the questionnaire. Participant responses to the quantitative items on the questionnaire are shown in Fig. Four participants 13%) attended the focus group session: three medical practitioners; and one basic scientist. Combined qualitative data from the responses to open-ended questions on the questionnaire (completed by 23/31 (74%) of participants) and the focus group are presented in Tables 3% attendThe theme of ‘Organisational support’ is illustrated in Table The theme of ‘Pedagogic support’ is illustrated in Table The theme of ‘Affective support’ is illustrated in Table This study sought to explore participants’ perceptions of a new longitudinal Clinical Teaching Fellowship (CTF) program, specifically designed to support the professional development of participants’ skills in TBL facilitation, and their engagement in medical education. The vast majority of participants regarded the training and experience provided to be relevant to their needs and beneficial to their career. Participants felt their learning was enriched through a combination of formal training, practical teaching experience alongside senior clinical staff, the multi-disciplinary context of training and co-teaching in TBLs; and the ‘sense of community and mentorship’ provided through the CTF program. Most expressed a desire to continue teaching TBLs in future years, and an interest in remaining connected with the CTF program. Some participants felt there should be a greater emphasis placed on provision of feedback on their TBL teaching. Additionally, participants indicated that clinical responsibilities made it difficult to attend formal training and teaching sessions. These findings are expanded and discussed using the conceptual framework of ExBL.Organisational support ensures that the learners’ experiences are aligned with the program outcomes, and that there are opportunities for active participation . LearninPedagogic support refers to the support that is provided by the teachers in the workplace environment . The socThe CTF program emphasised the professional expectations of the basic scientists and medical practitioners as teachers. This focus on professional identity formation is relevant in contemporary faculty development programs, particularly for early-career staff who may hold a number of different roles. Steinert et al. (2019) state that clinicians and basic scientists often feel that they are undervalued by their institutions . They suAffective support is provided by a warm and inclusive environment, where participants feel they are treated as a member of one large community  and wherThis was a pilot study with a small sample size, and the opinions of participants may not be generalisable to all clinical educators in similar settings. We note that our response rate to the questionnaire was at 74%, and only 13% of participants took part in the focus group. Therefore, the responses may not be representative of the entire group of CTFs. There are challenges of aligning participant and institutional goals. However, the faculty development program is applicable to non-TBL settings. Implementation of the CTF program required institutional leadership support at a senior level.The CTF program demonstrated the feasibility and acceptability of a longitudinal faculty development framework within medical education, that is also applicable to non-TBL settings. The program allowed participants to prepare, practice and improve their teaching skills, within a supportive environment. In 2019, the number of CTF applications has continued to grow, with an increase in applications of 350% compared to the second year. Our findings suggest that through provision of opportunities for delivery of theory, choices in professional development activities, workplace practice in teaching with supportive co-teachers, and alumni engagement, a sustainable community of practice can be built. Securing institutional investment to support the growth and development of our volunteer teachers will be essential to ensure sustained innovation and excellence in clinical teaching.


Use of ultrasonography has revolutionized diagnosis of musculoskeletal disorders. Until now, few studies have investigated usefulness of a short-period workshop for musculoskeletal ultrasound (MSKUS) education. In this research, we attempted to explore (1) whether the physicians felt it useful to attend this type of courses for improving knowledge of sonoanatomy and scanning skills, (2) if the attendees’ perceived confidence in musculoskeletal diagnoses by using ultrasound increased following the program and (3) whether differences existed in perceived usefulness and confidence regrading different sessions of the course.The target participants of the courses were postgraduate physicians without limitation of their specialties. The attendees’ responses to questionnaires before and after the course were reviewed. The workshop contained didactic and practical sessions on 6 major joints in accordance with the scanning protocols of EURO-MUSCULUS/USPRM. The course usefulness and perceived confidence in MSKUS examination were evaluated using a 5-point Likert scale. Data relevant to participants’ pre-workshop confidence levels were also analyzed. If any participant attended the course for more than 1 time, only their first survey was used for analysis.The study included 156 participants. The average rating for the course usefulness ranged between 4 (useful) to 5 (very useful). There was no difference in perceived usefulness between the didactic and hands-on practical sessions. Participants’ perceived confidence significantly increased after the workshop but appeared to be lowest for evaluation on the hip joint. Previous experience in performing MSKUS (in years) was consistently associated with the level of pre-workshop confidence.A short period ultrasound workshop might be useful regarding making musculoskeletal diagnoses by using ultrasound based on an increase in post-workshop confidence in MSKUS examinations. The perceived confidence of hip scanning was lower than that of other joints after the course, indicating inadequacy of education in hip sonoanatomy and intensity of hand-on practice in the present program. An increase in the faculty-to-student ratio or length of practice in the hip section should be implemented in the future course.The online version of this article (10.1186/s12909-019-1769-6) contains supplementary material, which is available to authorized users. In recent years, the use of ultrasonography has revolutionized the diagnosis and management of musculoskeletal disorders for physicians . It has A structured curriculum comprising didactic and practical sessions has been applied for ultrasound education in many specialties. Bernard et al. found that a multimodal educational approach significantly improved the levels of confidence of medical students in scanning the head and neck regions, and that instructor demonstrations were rated as the most crucial component of the program . RecentlTherefore, through reviewing the pre- and post-workshop questionnaires of the attendees, the present research attempted to explore (1) whether the physicians felt it useful to attend a short-period ultrasound workshop for improving knowledge of sonoanatomy and scanning skills, (2) if the attendees’ perceived confidence in musculoskeletal diagnoses by using ultrasound improved following the program and (3) whether differences existed in perceived usefulness and confidence regrading different parts of the course.The present study employed a retrospective cross-sectional design. The target attendees of the 2-day course were postgraduate physicians. Most of the attendees were physiatrists. Other participants included orthopedic surgeons, family physicians, emergency medicine specialists, neurologists, anesthesiologists, radiologists, rheumatologists, pediatricians, neurosurgeons, and etc. The workshop has been held annually since 2015 by our department of physical medicine and rehabilitation, in collaboration with the medical ultrasound society. Registration for the workshop was open to physicians from all specialties. The study was approved by the institution’s review board of National Taiwan University Hospital (IRB NO. 201707033 RINB), and the need for informed consent was waived due to the retrospective nature of the research. The data were processed and analyzed anonymously.The workshop consisted of two parts: first, the didactic sessions elaborating fundamental knowledge were given, and then, relevant hands-on sessions focusing on scanning skills were conducted. Both parts included 6 sessions/stations arranged according to 6 joints  Figs.  and 2. EDuring the didactic part, ultrasound knobology, regional anatomy, standard postures during the examination, transducer placement, and ultrasonography findings of normal and abnormal target joints were addressed. The following structures were covered for each joint and scanned in accordance to the scanning protocols of EURO-MUSCULUS/USPRM: (1) for the shoulder: the biceps tendon, subscapularis tendon, acromioclavicular joint, supraspinatus tendon, infraspinatus tendon, and glenohumeral joint ; (2) forIn the hands-on sessions, the instructors demonstrated the procedure for scanning the pertinent joint and the participants then practiced under the supervision of the instructors Figs.  and 2.The questionnaires were designed to examine whether the course was helpful in making a musculoskeletal diagnosis based on different joints. The attendees were invited to respond to the questionnaires as a feedback to the course for future improvement. Prior to attending the lectures, all participants filled out a pre-workshop survey  and Bonferroni adjustments for the post-hoc tests. Chi-square test was employed to compare proportions between two or more groups. A paired t-test was used to compare the usefulness between the didactic and hands-on practical parts and the perceived confidence between the pre- and post-workshop evaluations. Factors related to pre-workshop perceived confidence levels were analyzed using a multiple linear regression model. All analyses were performed using Stata version 11 , and the level of statistical significance was set at Three courses had been held annually since 2015, with a total of 186 attendees. If any participant attended more than 1 workshop, only their first survey was used for analysis. After excluding 22 surveys with incomplete responses and 8 surveys of repeated attendance, 156 sets were included in the study.n = 44) of the study population. Regarding their workplaces, 61.6% (n = 96) of the attendees came from teaching hospitals, 19.2% (n = 30) from local hospitals, and 19.2% (n = 30) from private clinics. Their average number of working years was 8.7 (standard deviation: 9.2 years). In terms of specialty, physiatrists accounted for 55.1% of the participants (n = 86); other attendees included orthopedic surgeons (n = 30), family physicians (n = 8), emergency medicine specialists (n = 7), neurologists (n = 6), anesthesiologists (n = 6), radiologists (n = 5), rheumatologists (n = 2), pediatrician (n = 1), and neurosurgeon (n = 1), as well as several health practitioners who did not specify their specialties (n = 4). Among the analyzed surveys, 116 reported having experiences of performing MSKUS examinations earlier, and the mean duration of using it in clinical practice was 2.3 years (standard deviation: 3.8 years). Although 108 participants (69.2%) claimed they had good access to an ultrasound machine, 136 attendees (87.1%) had never attended a MSKUS workshop before.The mean age of the enrolled participants was 35.9 years (standard deviation: 8.4 years), and females accounted for 28.2% , peripheral nerve imaging , and ultrasonographic imaging of the spine . The proportions were significantly different (p < 0.001).When the participants’ choices for future courses were analyzed, the distribution of topics was as follows: interventional ultrasonography , it might sometimes require the use of a curvilinear transducer, for which musculoskeletal specialists are typically less experienced . Second,Our study revealed that the only factor that was consistently associated with the pre-workshop perceived confidence level was previous MSKUS experience, reflecting the number of years that participants had used MSKUS in their clinical practice. We believe that this observation also strengthens the point that the learning curve of MSKUS is never an abrupt rise, and that time spent in practice plays a vital role for the maturation of skills. Another interesting finding was the negative association between the confidence level and participants’ age for shoulder imaging. Since MSKUS had been included in the physiatry resident training programs in our country for approximately 10 years , 28, mosThe strength of our study was that we evaluated the perceived course usefulness and post-learning confidence based on different body regions, which had rarely been reported in previous literature. Our analysis helped identification of a gap regarding the current course arrangement. In the present program, the faculty-to-student ratio and length of hands-on practice were the same at each session. However, our data revealed inadequacy of education in hip sonoanatomy and intensity of hands-on practice. Therefore, we suggested increasing the number of instructors and duration of practice in the hip session in the next workshop. Concerning future courses, interventional ultrasonography seemed to be the most favorite topic. Different from diagnostic ultrasonography, interventional ultrasonography focuses on target selection and needling techniques. Herein, a successful interventional treatment is always based on an initial correct diagnosis, both of which can readily be done with the use of ultrasonographic imaging . LikewisThere are several limitations of our study. First, the Likert scale, which is considered ordinal, was used to represent the perceived confidence and usefulness, but was analyzed as interval or continuous variables. Until now, there is still no consensus regarding the best statistical method to investigate these data. Therefore, we have also given the distribution of participants’ responses in detail. Second, the post-workshop assessment was conducted immediately after the hands-on practical session (less than 36 h). The participants might have recalled what they had responded in the pre-workshop evaluation and thus erroneously estimated the improvement of their confidence after the course. Third, although the confidence increased after the course, it might not indicate that the course could improve the accuracy of making a musculoskeletal diagnosis. Therefore, it is important to conduct a validation test to compare the results of ultrasound examinations performed by trainees on patients with basic pathologies detectable by ultrasound to those conducted by experts. If the training course results in increased confidence but little skills, then the training would be inadequate and needs further modification . HoweverA short period ultrasound workshop incorporating the didactic and hand-practice parts might be useful regarding making a musculoskeletal diagnosis by using ultrasound, based on an increase in post-workshop confidence in MSKUS examinations. Our analysis revealed that the perceived confidence of hip scanning was lower than that of other joints after the course, indicating inadequacy of education in hip sonoanatomy and intensity of hands-on practice in the present training program. An increase in the faculty-to-student ratio or length of practice in the hip section should be considered in the future course.Additional file 1:Pre-Workshop Evaluation. (DOCX 20 kb)Additional file 2:Post-Workshop Evaluation (DOCX 24 kb)


Adverse drug reaction reporting in the UK is lower than expected based on epidemiological data. This study aims to explore (a) nonmedical prescribers’ (NMP) confidence in identifying and reporting ADRs, (b) NMP prescribing habits and engagement with the Yellow Card Scheme (YCS) and (c) NMP desire for future training in the identification and reporting of ADRs.A survey was distributed across NMP networks in the north‐west of England using Survey Monkey. Univariate analyses were performed to compare the features of reporters and nonreporters, Kruskal‐Wallis H tests for comparisons within multiple subgroups and Spearman's rank correlation coefficient for response associations between answers to ordered‐category questions.B = 0.30, P < 0.0005; 95% CI 1.01 to 1.05) and receiving additional training support about the YCS  were associated with an increased likelihood of submitting to the YCS. There was a positive linear relationship between confidence in identifying ADRs and likelihood of reporting to YCS. The most common reason given (n = 261) for never having reported to the YCS was “I have never seen an adverse drug reaction.” Training appears to give NMPs confidence in reporting ADRs, but there seems to be a gap in actually identifying ADRs given the comment that most had never seen an ADR.A total of 570 responses were available for analysis, an estimated response rate of 20%. Less than half  reported submitting a Yellow Card to the YCS, and the majority of those individuals have submitted five or less Yellow Cards; 28 responders reported more than five submissions. Being professionally qualified for more years (linear regression: Strategies for improving the translation of theoretical knowledge about ADRs into practical skills in identifying ADRs, and subsequently reporting them, will be important for improving pharmacovigilance practice. Epidemiological data from the UK suggest that ADRs are responsible for significant healthcare utilization accounting for 6.5% of hospital admissions.The Yellow Card Scheme (YCS) was introduced in the UK in 1964 following the thalidomide disaster.Nonmedical prescribing was introduced to allow healthcare professionals such as nurses, pharmacists, radiologists and physiotherapists to prescribe independently. Certification is gained through formal training, which generally consists of lectures and peer‐supervised practice, and subsequent assessment. The qualification is recognized and reported by each of the profession‐specific governing bodies . An independent nonmedical prescriber (NMP) theoretically can prescribe from the whole of the British National Formulary, but as with medical prescribers, NMP prescribing is usually dependent upon their training, clinical specialty and competence. Nonmedical prescribing was introduced to help improve flexible team working across the UK NHS, allow patients quicker access to the medicines they need and improve patient care without compromising patient safety.It is important that NMPs receive training on the identification and reporting of ADRs, initially within their course and subsequently once in clinical practice. A study by Stewart and colleagues showed that NMPs have a positive attitude towards ADR reporting, but many lacked sufficient knowledge to answer factual questions about the YCS.22.1https://www.surveymonkey.com/). The regional NMP leads have access to all NMPs that have registered with them, but this might not cover the entire NMP regional cohort. We therefore do not have a precise denominator for the number of individuals who received the survey, but estimates suggest that there are around 3000 NMPs in the NW region. No restrictions were placed on respondents, beyond the need to be a qualified NMP. The survey was opened online on 5 September 2016 and closed 9 weeks later. Whether reminders about the questionnaire were distributed was dependent on the local and regional NMP leads.A survey was developed by members of Liverpool Health Partners Yellow Card Working Group and Yellow Card Centre North West. Each complete draft of the survey was tested on blinded NMPs, and feedback was used to refine the questions included and the structure; there were four iterations. The survey was distributed via local and regional NMP leads in the NW of England using SurveyMonkey  and on the two different groups (YC reporters and nonreporters). Univariate analyses were performed to compare the features of reporters and nonreporters, using the Wilcoxon signed‐rank test for continuous variables and chi‐square for categorical variables. Comparisons within multiple subgroups, for example within answer patterns to ordered‐category questions, were performed using Kruskal‐Wallis H tests for continuous variables. Response associations between answers to the two ordered‐category questions were determined using Spearman's rank correlation coefficient (ρ). Analyses were performed using SPSS 22 software . The threshold for statistical significance was set at 0.05.33.1Table Of the 570 respondents, the majority were either nurses (68.1%) or community practitioners  (21.1%), with most completing their NMP training at institutes located in the north‐west of England (94.2%). The average duration between completion of professional qualification and NMP certification was 14 years , with 77.2% gaining their NMP certification within the last 10 years (ie from 2007 onwards). The majority of NMPs had exercised their right to prescribe with only 16 (2.8%) indicating that they had not prescribed. Almost 50% of NMPs were prescribing at least once daily.Not surprisingly, the majority of respondents (95%) had received training about the YCS, but this was delivered in different ways, with the majority (56%) stating that it was delivered as part of another session. Only a small proportion of the respondents (26.7%) stated that they were required to complete a Yellow Card as part of their training.χ2 = 19.3, P < 0.0005), with community practitioners having the lowest proportion  and “other” professions having the highest proportion . Further exploration of the data revealed that pharmacists were generally responsible for this trend . Being professionally qualified for more years  and receiving additional training support about ADRs and the YCS  were also associated with an increased likelihood of submitting ADR reports to the YCS. Responses to questions C, D, E and I were not statistically associated with submitting to the YCS.Only 219 of the respondents (38.4%) reported submitting a Yellow Card, with the majority of these individuals submitting five or less Yellow Cards; 28 (4.9%) stated they had submitted more than five Yellow cards. The reported profession had an impact on the likelihood of submitting to the YCS (For those who answered “no” to “Have you ever reported an adverse drug reaction on a Yellow Card?” we explored reasons preventing reporting to the YCS (question M). Thirty‐eight 10.8%) individuals stated that reporting was not applicable. Although we cannot be certain, we believe that this is mainly because these professionals were not engaged with prescribing or patient contact in their current role, such as managerial/administrative positions. The most common reason given by all individuals  who answered “no” to “Have you ever reported an adverse drug reaction on a Yellow Card?” was “I have never seen an adverse drug reaction.” Other reasons are summarized in Table 0.8% indiP = 0.145), but self‐reported confidence in the detection and reporting of ADRs immediately after completion of NMP training was significantly associated with current confidence in identifying ADRs (P < 0.0005) (ie higher confidence after NMP training resulted in more confidence at the time of the survey). There was a positive linear relationship between confidence in identifying ADRs and likelihood of ever reporting to the YCS as evidenced by the proportion (as a percentage) of individuals submitting a YC. There was also a significant association between confidence and the number of YCs submitted (P < 0.0005), but this did not follow an absolute linear trend. Over 80% of the cohort indicated that they would value future additional information/support/training on ADRs and/or the YCS.We attempted to gauge an understanding of identification of ADRs via Question J Table . The maj3.2The aim of this questionnaire was to gain an understanding of the training received by NMPs on pharmacovigilance, their confidence in identifying ADRs and their engagement with the YCS. The questionnaire attracted 570 valid responses from various healthcare professionals. Although we cannot calculate the response rate as we do not know the true NMP denominator in the north‐west of England, this is one the largest studies investigating the perceptions of healthcare professionals to ADR reporting. This gives us confidence that the results are broadly representative but should still be interpreted with caution.Most participants agreed that their NMP course had left them feeling “very confident” at being able to detect ADRs and a similar proportion reported they were confident or very confident at identifying ADRs at the time of completing the questionnaire, which is a positive outcome given the high rates. Indeed, those with self‐reported confidence in identifying ADRs were three times more likely to have submitted a Yellow Card. There was also a significant association between confidence and number of Yellow Cards submitted. However, when our results are taken as a whole, this confidence in identifying and reporting ADRs is inconsistent with the fact that only 38.4% of the respondents had submitted a Yellow Card. It is known that the majority of healthcare professionals  do not report ADRs: indeed, a systematic review of 37 papers demonstrated that the median underreporting rate of ADRs was 94% (interquartile range: 82%‐98%), with similar rates between medical prescribers and other healthcare professionals.The design of our study is similar to that of Stewart and colleagues, and many of the outcomes complement these previous findings.suspected ADR that is subsequently disproved may cause embarrassmentThe reasons why healthcare professionals, both medical and nonmedical, do not report ADRs have been extensively studied through attitudinal surveys. The most common reasons reported include (a) a lack of time to report,Training is critical in improving pharmacovigilance practice in prescribers and is known to be a positive predictor for future reporting.Our survey is of course a snapshot of NMPs and has inherent limitations. First, we lack an accurate denominator for the population under investigation and therefore are unable to demonstrate whether our sample is representative. Furthermore, these data may not represent other areas of the UK and profession‐specific interpretations are limited due to small samples from certain cohorts. However, our sample is relatively large for this type of study, and a crude estimate shows we captured data from around 20% of the NMP population in the north‐west of England. It is clear that overall, there needs to be a certain level of caution when interpreting the findings. Second, we also cannot exclude the possibility that the responses of nonparticipants may have been different to the responses received nor can we exclude the risks of recall bias and social desirability bias. Third, there is a lack of data regarding the completeness and value of Yellow Cards submitted by NMPs. Although this is a wider technical issue outside the control of this study, it does limit the impact of the findings. Finally, it would have been informative to have a measure of the volume of prescribing rather than just frequency. Such data would have enabled analysis of the association between prescribing volume and reported outcomes.4NMPs have an important role in drug safety and need to be encouraged to engage with formal pharmacovigilance systems such as the YCS. Training appears to give NMPs confidence to identify ADRs, but there seems to be a gap in actually identifying ADRs given that most reported that they had never seen an ADR. Strategies for improving the translation of theoretical knowledge about ADRs into practical skills in identifying ADRs, and subsequently reporting them, will be important. Minor modification of the current training courses to emphasize the importance of ADRs will be important in ensuring that future NMPs fully engage with spontaneous reporting systems such as the YCS. This survey has provided a baseline assessment to help direct future resource allocation in training and research provision. Further work is needed to add understanding to the gaps initially identified and subsequently explore interventions.The authors have stated explicitly that there are no conflict of interests in connection with this article.


Targeted chronic disease programs are vital to improving health outcomes for Indigenous people globally. In Australia it is not known where evaluated chronic disease programs for Aboriginal and Torres Strait Islander people have been implemented. This scoping review geographically examines where evaluated chronic disease programs for Aboriginal people have been implemented in the Australian primary health care setting. Secondary objectives include scoping programs for evidence of partnerships with Aboriginal organisations, and use of ethical protocols. By doing so, geographical gaps in the literature and variations in ethical approaches to conducting program evaluations are highlighted.The objectives, inclusion criteria and methods for this scoping review were specified in advance and documented in a published protocol. This scoping review was undertaken in accordance with the Joanna Briggs Institute (JBI) scoping review methodology. The search included 11 academic databases, clinical trial registries, and the grey literature.n = 40), 32.1% were implemented in Major Cities and 29.6% in Very Remote areas of Australia. A smaller proportion of programs were delivered in Inner Regional (12.3%), Outer Regional (18.5%) and Remote areas (7.4%) of Australia. Overall, 90% (n = 45) of the included programs collaborated with an Aboriginal organisation in the implementation and/or evaluation of the program. Variation in the use of ethical guidelines and protocols in the evaluation process was evident.The search resulted in 6894 citations, with 241 retrieved from the grey literature and targeted organisation websites. Title, abstract, and full-text screening was conducted by two independent reviewers, with 314 citations undergoing full review. Of these, 74 citations evaluating 50 programs met the inclusion criteria. Of the programs included in the geographical analysis  contains supplementary material, which is available to authorized users. It is well established that Indigenous people experience poorer health outcomes than non-Indigenous people globally . AustralTargeted chronic disease prevention and management programs delivered in the primary health care setting are imperative to alleviating the burden of disease and improving health outcomes for Indigenous people , 8. In AThe need for Aboriginal community-driven programs and governance of primary health care services, as supported by the international right to self-determination for Indigenous people, is becoming increasingly recognised as a key strategy to alleviating the burden of chronic disease –16. StroAlthough there has been a rhetorical shift from government initiated health programs to community-developed health programs for Aboriginal people in Australia , it is nThis study provides a systematic scoping review geographically examining the distribution of evaluated chronic disease prevention and management programs implemented for Australian Aboriginal people in the primary health care setting which includes community-health settings, general practice clinics and ACCHOs . This reA preliminary search was conducted in MEDLINE and CINAHL using keywords to develop a tailored search strategy for each information source. A combination of Boolean operators, truncations and Medical Subject Headings (MeSH) were used to develop database search strategies , Primary Health Care Research and Information Service (PHCRIS), ProQuest Dissertations and Theses Global, Trove and OAIster.Keywords were used to search the following information sources for unpublished studies, grey literature, and trials in order to avoid publication bias: Lowitja Institute, Indigenous Healthinfonet, National Aboriginal Community Controlled Health Organisation (NACCHO), Department of Health , informIT, Google, Cochrane Central Trials Register of Controlled Trials, ANZ Clinical Trials Registry, This review considered literature based on the following criteria Table .Table 1No restrictions were placed on the quality of evaluation or study design. As stated in the scoping review protocol, programs evaluated by any party to any level were included . LiteratFor consistency, the term ‘Aboriginal’ has been used throughout this review to refer to both Aboriginal and/or Torres Strait Islander people in Australia. This is due to brevity, and no disrespect is intended to any individual or group. The term ‘Indigenous’ has been reserved for the global context.Searches for published and unpublished literature were conducted by a health librarian. Titles and abstracts retrieved from the search were screened independently by two reviewers (HB and MJB). Conflicts were resolved through discussion with a third reviewer (CK). Full text review and data extraction was then conducted independently by two reviewers (HB and MJB) on selected articles. Reasons for exclusion were provided for articles that did not meet the review criteria. The reference lists of citations requiring full text review were also screened for additional citations in order to ensure that all possible literature was included.Extracted data were categorised under the following headings: author, year of publication, year of program implementation, location of program implementation/evaluation, evaluation methods, involvement of an ACCHO/other Aboriginal organisation and reference to Australian National Health and Medical Research Council’s (NHMRC) ‘Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research’ guideline, and other ethical protocols , 25. GeoCoordinates were then exported to ArcGIS® ArcMap™ and overlayed with the Remoteness Areas of Australia for analysis . To defiThe extracted data, synthesis of findings and review outcomes, were critically reviewed for culturally appropriateness by two Aboriginal researchers, as stated in the review protocol .Database searches yielded 14,366 citations. An additional 241 citations were retrieved from a search of the grey literature and targeted organisation websites. A total of 6894 title and abstracts were screened, with duplicates removed. The full texts of 314 citations were screened for relevance to the review criteria, identifying 74 pertinent records evaluating 50 chronic disease prevention and management programs , followed by ‘Sub-studies met inclusion criteria but already included in search, or sub-studies did not meet inclusion criteria’ (n = 50) and ‘not a primary health care delivered program’ (n = 30). Excluded records included 20 records which focused on evaluating national screening and vaccination programs. These were excluded as findings were based on national or state-wide aggregate data which would have been difficult to include in the geographical analysis.Reasons for excluding records were provided  included evaluated programs (n = 50) that addressed multiple chronic diseases (n = 16), a specific chronic disease  or risk factors for developing chronic disease  (Table n = 37) aimed to prevent and/or manage chronic disease using disease-specific screening, early intervention or treatment strategies, with the remaining programs (n = 13) applying general health promotion approaches to disease prevention, such as empowering participants to implement activities to improve their health.Citations meeting the review criteria , followed by a quantitative only  or qualitative only  approach. The methods of evaluation were not reported for one program [The data collection methods of program evaluations varied, with over half of the program evaluations using a mixed methods approach , 32.1% were implemented for Aboriginal people residing in Major Cities of Australia and 29.6% for Aboriginal people residing in Very Remote Australia. The remaining programs were implemented for Aboriginal people residing in the intermediate remoteness areas of Inner Regional Australia, Outer Regional Australia, and Remote Australia .Of the included programs in the geographical analysis (n = 81), 18 (22%) of these also had an evaluation undertaken on site. For studies with the site(s) of implementation and evaluation available, the mean distance between implementation and evaluation was 660 km . A visual representation of the distribution of included programs is provided , with the remaining studies declining in order of remoteness. Of the identifiable implementation locations  of the included programs collaborated with an Aboriginal organisation in the implementation and/or evaluation of the program.Of the 50 programs included in the review, 39 (78%) reported on the involvement of an ACCHO in the implementation or evaluation process , 74% (n = 55) were associated with a university or research institution, 9.5% (n = 7) with an ACCHO or other Aboriginal organisation, 9.5% (n = 7) with a non-Aboriginal health service or Non-Government Organisation (NGO) and 7% (n = 5) with both a university or research institution and an ACCHO.When examining the affiliation of the first author of the included citations (n = 52), particularly those published in a peer-reviewed journal, included a formal statement of ethical review and approval by a Human Research Ethics Committee (HREC) affiliated with a research institution or university. Only 23 citations (31%) discussed the use of other ethical protocols or a community-based ethical review process. These citations varied broadly in their descriptions of adhering to local cultural guidelines or consulting with an appointed Aboriginal advisory group. For example, Askew et al. (2016) [Of the 74 citations retrieved, seven explicitly referred to the NHMRC’s ‘Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research’ as a guideline underpinning the evaluation design and conduct . However. (2016)  describe. (2016)  describe. (2016) , 72 and . (2016) , 83.This review highlights the paucity of Aboriginal chronic disease program evaluations conducted in the primary health care setting across all geographical regions of Australia. Previous studies have acknowledged that only a small proportion of Aboriginal health programs have been subject to an evaluation process , 13. TheOverall, Major Cities and Very Remote areas of Australia displayed similar levels of chronic disease program implementation activity, with less activity noted for Inner and Outer Regional (IOR) areas and Remote areas of Australia. A greater focus on chronic disease programs for Very Remote Aboriginal people when compared to IOR or Remote Aboriginal people could be informed by national data which indicates that the burden of chronic disease in Aboriginal people increases with Remoteness . HoweverWhen considering evaluation activity, higher levels of evaluation were noted for Major Cities (71.6%) when compared to Very Remote areas (1.2%). This is despite the fact that national data indicates that less Aboriginal people reside in Major Cities compared to the total Australian population (37% compared to 73% respectively) . Furthern = 74), 74% (n = 55) of citations were associated with a university or research institution, with only 9.5% (n = 7) citations associated with an ACCHO or other Aboriginal organisation. A previous review of Aboriginal health programs in Australia also found that the majority of program evaluations (72%) were led by a research institution or university rather than an Aboriginal community organisation [n = 45) of included programs provided details of collaborating with an ACCHO or other Aboriginal organisation in the development or evaluation of the program. Strong support for the appropriateness of ACCHOs as a collaborating organisation for activities involving Aboriginal people is found in the literature [When examining first author affiliation for all included citations  provided a statement of formal review by an appointed committee (e.g. HREC). However, only 31% of the included citations (n = 23) provided some evidence of actions taken to adhere to Aboriginal community-based ethical protocols, or engagement with an Aboriginal advisory group in the design of the program or conduct of the evaluation. Indeed, a statement of formal ethical review does not provide sufficient detail describing how Aboriginal people were consulted and included in the evaluation process. Other Aboriginal program evaluation frameworks and models of Aboriginal health research should also be consulted, which are valuable in informing approaches to conducting program evaluations in partnership with Aboriginal people [Although the majority of programs partnered with an ACCHO or Aboriginal organisation, it is difficult to ascertain for all programs, the degree of community ownership and involvement in the evaluation process. This includes steps taken by evaluators to ensure the evaluation process was ethically and culturally appropriate for Aboriginal people . As repol people , 112. Prl people .It is important to acknowledge the limitations of this review. The selection criteria of the review influenced the geographical spread of studies retrieved. National screening and vaccination programs were excluded as program evaluations used national aggregate data. Geographical findings may also have been impacted by the exclusion of other programs which met the review criteria, but were excluded from the geographical analysis due to state-wide or national program implementation  programs developed for Aboriginal people. Programs should be implemented and evaluated in collaboration with partnering ACCHOs or other Aboriginal organisations, with an emphasis on accountability, sustainability, capacity-building, ownership and Aboriginal strengths. This includes equipping Aboriginal organizations with skills in conducting program evaluations. Evaluation reporting should be transparent in describing ethical approaches to conducting the program evaluation in partnership with Aboriginal communities. Furthermore, an evaluation process should be integrated into the design of Aboriginal health programs. Evaluation outcomes should be publicly available, ideally through the peer-reviewed literature, in order to build the evidence around the effectiveness of chronic disease programs for Indigenous peoples globally.A greater focus on the implementation and evaluation of chronic disease prevention and management programs for Aboriginal people in Australia is required, particularly for Aboriginal people residing in Inner and Outer Regional Areas of Australia. There is also a need to conduct evaluations of Social and Emotional Wellbeing (SEWB) programs across all geographical regions. This review highlights the need for more ethically rigorous approaches to Aboriginal health program evaluations which engage Aboriginal people in all stages of program design, implementation, evaluation and sustainability.Additional file 1:PRISMA 2009 Checklist. This file contains the PRISMA checklist and co-relating page numbers to the items for reporting. (DOCX 25 kb)Additional file 2:Electronic search results and terms. This file contains a table of search results and terms used to retrieve studies from databases (DOCX 43 kb)Additional file 3:Excluded Studies. This file contains a table of excluded studies and reasons for exclusion. (DOCX 68 kb)Additional file 4:Data extraction: evidence of partnerships and reference to ethical guidelines. This file contains a table of data extracted in relation to the secondary objectives of this review; scoping evidence of partnerships with Indigenous organizations and ethical approaches to undertaking a program evaluation. (DOCX 95 kb)


Background: Widespread implementation of rapid response (RR) systems positively impacts outcomes of clinically unstable hospitalized patients. Collaboration between bedside providers and specialized responding teams is crucial for effective functioning of RR system. Bedside, providers often harbor negative feelings about having to ‘call for help’ that could impact their active participation in RR.Objective: The objective of the study is to enhance active participation of bedside providers in RR by fostering self-determination through targeted education.Design: Needs assessment affirmed that bedside providers in our tertiary academic pediatric hospital felt loss of control over patient care, lack of competence, and disconnect from the RR team. We used the principles of autonomy, competence, and relatedness posited by the self-determination theory to guide the development, implementation, and evaluation of our educational program for bedside providers.Results: Forty-two bedside providers participated in our program. Participants reported significant improvement in RR-related clinical knowledge. More importantly, there was significant enhancement in individual perceptions of autonomy  competence , and relatedness  with RR (p < 0.01). The evaluation results for overall educational effectiveness showed a mean score of 4.69 ± 0.79. All scores were based on a 5-point Likert scale of 1: poor to 5: excellent. Educators noted good participant engagement. The program’s structure, evaluations, and data management were modified based on the feedback.Conclusions: We successfully developed and implemented targeted educational program for bedside providers based on self-determination theory. The evaluations showed improvement in bedside providers’ clinical RR knowledge and perceptions of autonomy, competence, and relatedness following the training. The Institute of Healthcare Improvement as a part of the ‘Save 100,000 Lives Campaign’ recommended widespread implementation of rapid response (RR) systems or medical emergency teams to propagate patient safety and improve clinical outcomes . NumerouNonetheless, the bedside providers (nurses and physicians) who are usually the initial responders (IRs) report certain unintended affective consequences of RR events. Whereas the IRs usually feel relieved when help arrives in the form of specialized responders (SRs), they also perceive at times a criticism of their ongoing management that led to the need for RR and a burden to explain why the RR system was activated ,12. As tAdverse feelings harbored by IRs related to RR events can compromise timely activation of response, delay initiation of proper care, and result in harm to the patient ,17. RR tThe Institutional Review Board of Baylor College of Medicine reviewed and approved the project. Detailed needs assessment and framework conceptualization preceded the program development.Several sources of data, including focus group interviews with IRs, surveys of nursing supervisors who guide triage during RR, feedback from SRs, and verbal feedback received by RR administrative committee members, informed the needs assessment.The thematic categories identified from the IRs’ perspectives were (1) need for more knowledge about correct use of the RR system, (2) desire for more knowledge and skills to be active participants during a RR event, (3) lack of clarity of the individual roles during a RR, (4) dissatisfaction with self about ‘missing something’ that led to the clinical deterioration, and (5) perceived loss of control over their own patients’ care plan during a RR. The thematic categories identified from the feedback from SRs were that the IRs needed improvement in (1) competency in initial management during a RR, (2) ability to use critical communication, and (3) motivation to participate as a team member in a RR.After performing a critical review of the literature and engaging in educational expert consultations, we determined that principles posited by the SDT would help address the three main affective consequences of IRs: (1) feelings of loss of control to continue management of their patients, (2) perceived lack of competence in an overwhelming situation, and (3) sense of disconnect from the RR team. The three basic psychological needs – autonomy, competence, and relatedness described in SDT as drivers to foster individuals’ intrinsic motivation and engagement in activities – were applied to our program development.We proposed that our training program based on SDT and targeted toward IRs would improve the competency and participation of the bedside providers in a RR event. We used SDT as a conceptual framework to guide the development, implementation, and evaluation of the educational activities of the RRT workshop program  and explThe process map used to guide the creation, development, and implementation of the educational program is depicted in The activities were structured into a 4-h workshop that included multiple instructional methods . In addition to large-group interactive didactics, the workshop had several small, case-based, group-learning activities through which groups of four to six participants rotated. The workshop organizer intentionally formed these multidisciplinary groups to include participants with diverse levels of experience in each group. The detailed aims and objectives of the individual educational activities are described in The workshop was advertised through general announcements to hospital-wide pediatric providers, and specific information was disseminated to leaders of the Department of Pediatrics and residency program. Nurses, residents, fellows, faculty, respiratory therapists, and advance practice providers from multiple disciplines within the hospital were invited to participate in the workshop.knowledge acquisition: we developed pre- and posttests with 20 questions each to assess the impact of the workshop training on the participants’ levels of RR-systems knowledge. Clinical case vignettes of commonly occurring pediatric RR events assessed high-order thinking rather than factual knowledge; (2) SDT components: we used a reflective self-assessment by participants to determine whether the workshop supported the three components of SDT as it intended. The pre- and posttests included items that address the components of autonomy , competence , and relatedness ; (3) instructional effectiveness: we used a standard teaching evaluation form recommended for all educational activities at our institution. This 5-point Likert scale questionnaire (ranging from poor to excellent) addressed various learning domains ; (4) group engagement and performance: the workshops’ educators and facilitators assessed the participants’ engagement in the learning activities and demonstration of knowledge and skills acquisition during the sessions using a learner-assessment questionnaire.The program evaluation included (1) After completion of the pilot sessions and obtaining preliminary data, we reviewed the feedback received from participants, facilitators, and workshop administrators to identify areas of the program that needed modification.p less than 0.05 was used. Statistical computations were carried out using SPSS version 24.0 .Descriptive and paired comparative analyses were performed and a significance level of During a period of 6 months, we piloted three sessions of the workshop. Forty-two IRs  participated in the sessions. The average pre-workshop experience level of the IRs in their usual clinical jobs was 1 year (range: 2 months–11 years). The results showed improvement in RR-related systems knowledge for surgical providers and clinical knowledge for both surgical providers and pediatric residents. There was a significant increase in self-reported scores of SDT components: ability to maintain control over the RR situation , capacity to perform competently the tasks needed during RR , and feeling of inclusion in the team  .10.1080/The overall workshop evaluation results are depicted in Several changes have been made to the structure of the workshop, the evaluations, and the organization and maintenance of program-related data, based on the preliminary evaluations and the observations made by educators during workshop sessions.Modifications made to the program include (1) the length of the didactic session was reduced; (2) group case discussions were structured to cover four main clinical domains: respiratory failure, shock, cardiac failure, and neurologic dysfunction, to address the four most common RR situations; (3) a case bank of clinical vignettes was developed with different scenarios that are applicable to daily practice of various groups of providers and represent the four main domains listed in modification number 2; and (4) the challenging communication station was converted from a discussion exercise to a low-fidelity simulation session to add realistic feeling to the communication challenges. Modifications made to the program evaluations are (1) case vignettes in the pre- and posttests were modified to test clinical knowledge of the four domains of respiratory failure, shock, cardiac failure, and neurologic dysfunction; (2) workshop evaluation forms were simplified (decreased from 25 to 10 questions) to avoid excessive burden of time commitment from the participants (decreased time spent from 15 min to about 5–7 min per participant); (3) five additional questions to assess changes in a participant’s self-efficacy with performance of essential RR tasks, pre- and post-training, were included in the workshop evaluation; and (4) an additional questionnaire was developed to test retention of enhancement in SDT components and self-efficacy at a time remote from the workshop. All questionnaires are being tested for content validity with experts and reliability for SDT components with internal consistency. Modifications made to the data organization are (1) development of a systematic way to record all the responses gathered to allow for optimum paired pre–post testing of each of the questionnaires and (2) creation of a database to record contributions of workshop educators and facilitators with respect to time commitment to provide them with deserved credit.RR systems are widely implemented across health-care settings and serve multifaceted functions of early detection of clinical instability, prompt response, efficient treatment, and appropriate triage to prevent cardiac arrests . The effWe took a novel approach to enhance our RR system by offering training targeted to the IRs, who perform the vital tasks of early detection of clinical instability, response activation, and participation in ongoing care. Additionally to our knowledge, our program is the first of its kind to employ a well-established meta-theory of Self Determination as a conceptual framework for RR education of IRs. SDT has been widely used in guiding educational strategies in health-care settings to achieve adult continued learning and motivation . ThroughThe workshop is unique in the aspect of building multidisciplinary collaborations at both the educators’ and participants’ levels. A variety of providers that included nurses, respiratory therapists, advance practice providers, and physicians committed their time to educate participants. The diversity of experience in the participant groups potentially allowed for realistic experience during simulation scenarios.Our program evaluations showed improvement in early educational outcomes. The improved self-assessment related to the components of SDT suggested that the program was on the right track in achieving what it intended to do in terms of supporting learners’ autonomy, competence, and relatedness. The overall positive workshop evaluations and notable engagement in activities suggest that the workshop’s structure and the information provided were considered beneficial and were well received. At the beginning of the program, some apprehension was expressed about voluntary involvement and commitment of participants in the workshop. However, we noted that participants were keen on wanting to enroll and engage in this workshop, which could have been due to their awareness of the need to expand RR-related knowledge.We plan to continue to enroll participants in the program and gather more data on the merit and worth of the program. We plan to test retention of knowledge and skills and application to real-world situations at a remote time point (6 months–1 year) from the initial training. We will also assess the need for other related educational activities such as refresher sessions for previous participants, multidisciplinary sessions with combined initial responders and SRs as participants, and condensed sessions that will offer specific components of the program for advanced level learners. The diversity and experience level of future participants is anticipated to continually increase. Additionally, the workshop has the potential to address certain RR-related unintended affective consequences that are barriers to effective RR utilization.We determined that certain aspects of program design were critically important in our ability to understand needs of providers that participate in RRs and develop a program that was beneficial and hence well received by participants. These included detailed needs assessment and consultation with educational experts that provided guidance on application of SDT to develop the program. Another important attribute of our engaging workshop was integration of various learning methodologies: didactics, group discussions, simulations, skill stations, etc. We also realized the importance of flexibility with program design, a kind of Educational ‘Plan-Do-Study-Act’ cycle. Our preliminary data have given us several opportunities to perform program modification as described in the “Program evolution” section.The workshop activities require significant commitment of time and effort from educators and participants, which could impact the sustainability of our program. Given the preliminary nature of the evaluation, we had not performed reliability and validity tests on the questionnaires, and we have included them as a modification for future comprehensive program evaluations. Additionally, our findings on educational outcomes reflect the local context of the program in a single center and may not be directly generalizable. Nonetheless, as we have systematically integrated a theoretical framework into our program design, implementation, and evaluation, we hope that our results can inform others involved in efforts to enhance RR education and quality improvement endeavors. We anticipate the need for ongoing modifications to program structure to address participants and context in diverse settings.In summary, we successfully developed and implemented a novel educational program guided by SDT to train bedside providers with regards to various aspects of pediatric RR. Our initial findings showed that participants had enhancement in perceived autonomy, competence, and relatedness with regard to RR-related tasks, in addition to improvement in participants’ knowledge and skill sets. We also were able to gather data to guide future modification and improvement of the program.


Workplace health programs (WHPs) may improve adult health but very little evidence exists on multi-level WHPs implemented at-scale and so the relationship between program implementation factors and outcomes of WHPs are poorly understood. This study evaluated Get Healthy at Work (GHaW), a state-wide government-funded WHP in Australia.A mixed-method design included a longitudinal quasi-experimental survey of businesses registered with GHaW and a comparison group of businesses surveyed over a 12-month period. Semi-structured interviews and focus groups with key contacts and employees of selected intervention group businesses and the service providers of the program were conducted to assess program adoption and adaptation.p = 0.045 timeXgroup effect) and that the workplace promoted healthy behaviours (p = 0.004 timeXgroup effect) improved significantly while the control group reported no change in work culture perceptions. Changes in perceptions about work productivity were not observed; however only one third of businesses registered for the program had adopted GHaW during the evaluation period. Qualitative results revealed a number of factors contributing to program adoption: which depended on program delivery , design features of the program, and organisational factors (primarily business size and previous experience of WHPs).Positive business-level changes in workplace culture were observed over time among GHaW businesses compared with the control group. Multilevel regression modelling revealed perceptions that employees were generally healthy  contains supplementary material, which is available to authorized users. Globally, non-communicable diseases (NCDs), including heart disease, stroke, cancer and diabetes, account for 71% of all deaths . In AustWorkplace health programs (WHPs) are coordinated and comprehensive health promotion strategies comprising of policies, environmental supports and activities in the workplace to engage workers in healthy behaviours and facilitate their wellbeing . WHPs diEvaluating the implementation of WHPs is central to understanding the benefits and the factors which facilitate or inhibit their effectiveness and sustainability . Most evemergence, and means that the specific outcomes and the way in which they occur may emerge during implementation process, and will be unknown a priori . Th. Th38]. A list of key contacts from non-government businesses registered for the program when it launched in 2014, were contacted by the authors by phone and email and invited to participate in an online survey  and program implementers (service providers). This allowed for emergent processes and interactions to be identified, as the nature of complex programs means that some element of uncertainty will continue even into the implementation stage.A theoretical framework for the qualitative evaluation was not imposed, but the approach relies in part on interactionism theory . Howevern = 11). The contact selected male and female employees to participate focus groups undertaken in the workplace (n = 3 groups of 4–5 participants). Focus groups did not include participants in a direct line management. Service providers engaged by the government to support implementation of the program were also interviewed about their experience (n = 9). Service providers were businesses qualified in workplace health services. They included staff from a mixture of backgrounds .A subset of key contacts from registered businesses  were randomly selected to participate in a face-to-face semi-structured interview at T3 at their workplace . Written consent was obtained from all participants. All interviews were conducted by researchers with interviewing experience .The primary exposure variable was a dichotomous indicator of registration into GHaW at baseline (intervention group) versus not registering (control group). Second, a graded measure of exposure (dose received) was based on progress through the program cycle Fig. , as regithe workplace as promoting healthy behaviour and health promotion and to what degree it was open to change on a five-point scale (from strongly disagree to strongly agree). Five-point agreement scales were also used to assess perceptions of work productivity .Primary short-term outcomes were assessed in terms of changes in perceptions of workplace culture and work productivity (“business impacts”) (See Table Possible mediating factors of WHP implementation (“implementation factors”) included organisational factors ), health promotion beliefs, support from senior leadership and satisfaction with the program. Business industry was determined by Australian and New Zealand Industry Classification as one of seven categories, and dichotomised into professional and service/industrial businesses . Health Interviews with key contacts (average duration 52 min) were used to collect data on how businesses implemented the program and explored their perceptions about program registration and implementation processes and perceived impacts of the program. Focus groups with employees (average duration 58 min) were conducted to understand what employees thought of the program after it was implemented, and explored awareness of GHaW, changes in work culture and experience of participating in the BHCs and any WHP interventions. Interviews with service providers (average duration 62 min) were used to understand business engagement and use of the program from the service provider perspective and explored provider impressions of the program and perceptions about how workplaces engaged with the various stages of implementation. These interviews also explored their own involvement and engagement with the program . To avoid contamination, these businesses were removed from the control group for data analysis. Scales were developed to assess work culture, health promotion beliefs and leadership items using principal factor analysis with orthogonal rotation; factor items with a loading > 0.3 were retained. Scales were developed around health beliefs  and leadership . Items pertaining to the workplace culture loaded across three factors (loading ~ 0.5 across multiple factors) and so were kept as single outcomes . The model included random intercepts for responses over time nested within key contacts. Fixed effects were included for group, time and group by time interaction. Business size was included as a covariate given baseline differences . A sensiImplementation factors were analysed as follows: business-level engagement in, and adoption of, the program was compared across the range of mediating factors. Program engagement was defined as being at least at stage 1 of the program cycle and adoption as completing at least stage 3. Binary univariate outcomes were modelled by logistic regression using Fisher’s exact test to reduce small sample biases. Frequencies are reported for program satisfaction related variables. Stata version 14  was used for statistical analyses.Interviews were transcribed verbatim and checked for accuracy. First cycle data coding was carried out independently by two authors (MC and EG). A deductive process was applied to investigate the implementation process described in the quantitative data . An induThe quantitative results are presented below, followed by the qualitative which provide potential explanations for the quantitative results.n = 94) and at 12 months   responding at T2 indicated having adopted the GHaW program. In terms of the program cycle of the 94 business contacts surveyed at T2, 35.1% reported they had developed a WHP action plan; 29.8% had conducted just the initial stage of brief health checks; and 35.1% indicated they had not yet started the program . As fewer businesses in the GHaW sample were retained at T3 , the results presented focus primarily on T2 findings.In 2014 when GHaW was launched there were 696,991 private and public businesses operating in NSW . At baseComparison of workplace culture perceptions between the GHaW and control groups are shown in Table p = 0.013) and that the workplace promotes healthy behaviours (p < 0.01). The control group did not change. Willingness to participate in workplace health promotion remained higher in the intervention than the control group over time but did not change significantly (p = 0.27), suggesting perceptions were sustained. The remaining workplace culture items did not change over the short-term in either group.Statistically significant improvements were observed over time in the intervention group for perceptions that people in the workplace were healthy  of businesses in the intervention sample who stated they had a GHaW program at T2 agreed that the program had an impact on increasing worker engagement/satisfaction. A third agreed that it had increased worker productivity . Few businesses indicated an impact on reducing sick leave or an increase in worker retention .In terms of perceptions of work productivity, nearly half (p > 0.1). Factors relating to readiness to change, such as supportive leadership, were also not significant (p = 0.60), although health beliefs  neared statistical significance (OR:2.26 (0.91–6.07); p = 0.081).Logistic regression analysis of organisational factors associated with whether registered workplaces started the program cycle (vs not starting) found neither business size, industry, nor mode of delivery for brief health check or WHP support to be statistically significant . Of these 36 businesses, 4 stated that the service provider had made no contact with them by T2, while 26 rated the service provider as extremely to moderately helpful. The online resources were rated by only 26 (27.7%) of businesses, of which 3 stated the resource was not useful. These results are an indication of usefulness however may not be representative of the whole sample. Most businesses which had adopted GHaW by T2 reported that the program proceeded as planned (26 of 32 businesses) and had its intended impact (29 of 33 businesses).In terms of satisfaction with the program: 63 business contacts (67%) rated the helpfulness of the brief health check. Of these, 52 (82.5%) stated it was extremely to moderately useful; only 11 (17.5%) said it was not useful. There was no significant difference between whether the business had used the online or service provider-assisted brief health check in terms of its usefulness. The helpfulness of the service provider was rated by almost all of those who used a service provider (n = 11), employee focus groups (n = 3) and service providers (n = 9) provide further insight on the implementation process. Key contacts were from ten businesses which had completed the brief health check and four which had also implemented a WHP. All businesses interviewed had requested service provider support .Two central overlapping factors appeared to underpin the implementation of the program across workplaces at the organisational level: business size and previous experience implementing WHPs. Business size and experience ran parallel in that smaller businesses had little previous experience of WHPs while larger businesses generally had previous experience developing WHPs, and therefore were more likely to select program elements. Smaller businesses may have felt ready to implement a WHP but were less familiar with the process of a WHP, or what it involved. They were therefore more in need of assistance: ““We noticed the guys buy a lot of chips and pies and things like that at lunch time. So what we were thinking of doing was trying to put a healthy eating program in place” . Yet further adoption of the program from the business’s end depended on leadership involvement or endorsement and alignment of the program with business priorities. Leadership actions, such as allowing WHP participation during work hours, and health beliefs, such as feeling a business has a role in employee health, had potential to drive the program. Two of the focus groups spoke of the difficulty getting things moving because of a lack of champion or push from senior level.Engagement with GHaW appeared predominantly driven by the business management’s motivation to improve employees’ health, irrespective of business size or WHP experience. “It’s probably not as top priority as for me as so many other things I’ve got to do at the moment, so that’s another issue I think” .Small businesses with competing priorities struggled more in this respect. Organisational priorities and WHP implementation did not always align. Service providers perceived smaller businesses were unlikely to reach the implementation stage because employee wellbeing was not a high priority. Some business contacts stated competing priorities as a reason for their slow progress through the program cycle: Organisational structures presented other challenges to implementation such as logistical challenges (if multi-site or if trading outside NSW) or administrative challenges (in the case of takeovers and mergers). For businesses with national or international management, WHP implementation at a local level could require a lengthy process of approvals and depend upon what is already in place nationally/ internationally. As observed by one service provider, “Every stage they [participating businesses] have to go through a level of approval before they’ll do it and it seems to be a long drawn out process, so no. We haven’t noticed that many get through”.It’s good to be seen as a company … as this organisation to be aligned with the government and supporting peoples’ health” (service provider). This was also important to one focus group: employees were favourable toward government involvement in workplace health, explaining it made sense given the amount of time people spend in the workplace and government’s position of influence over businesses’ approaches to employees’ health.A key reason for most businesses and service providers engaging with the program was the government sponsorship of the program. The Government was viewed as a trustworthy, reputable provider. “This is something we’ve had discussions about in different groups is small businesses ... struggling to survive most the time let alone thinking about putting more money into programmes... to support their workers” . Perspectives on the cost of running the program changed from being a barrier to an expense justified by the observed benefits, but for those businesses who stated they were unable to achieve the participation requirements in order to receive the incentive, this became a discouragement. “We planned to get to 50 so that we would get the report… But because we didn’t get to 50, we didn’t get the report...there was no tailored programme” .The financial incentive to support implementation of a WHP was a major attraction and factor in supporting small businesses implementing the program. “I guess, also, because they may have only used the program for the brief health checks. So they don’t see the big picture” . Completion of the brief health check was essential for moving on to the next stage however for many businesses, especially for multi-site business, the logistics of organising employees to complete this step was a challenge. While the online option provided flexibility, some businesses stated that staff computer access and literacy skills could be participation barriers.The steps within the GHaW program cycle were poorly understood, with a few business contacts perceiving the brief health check to be the entirety of the program. “Everyone’s done an online health check, they’ve come up with an individual risk factor, and yet we’re not necessarily targeting that because we have to do something for everybody” . Interviewed businesses that had implemented some WHP elements all listed both dietary and physical activity initiatives. This observation was also made by all service providers. “It’s been difficult getting companies to get on board to do smoking cessation programmes. …they see it as, I want to be offering something to the majority of my staff. ... I want to run a nutrition workshop. I want to get a better attendance, there’ll be more people engaged with it” (service provider). As such, WHP action plans appeared to be adopted piecemeal.Other program characteristics seemed to challenge implementation. For example, GHaW recommends that a single health priority is the WHP focus, but the majority of businesses interviewed wanted a more encompassing “wellbeing” program. “GHaW also recommends a mix of people, place and policy initiatives. Activity-based interventions (such as weekly exercise activity or gym equipment) appeared to attract low participation while policies  appeared to be more encompassing.I was still very open to it, but I think I was a bit more like, “There’s no urgency from that side, so there’s no urgency from my side”,…I think after I signed up for it, …somebody rang me …we’ve talked about, like why I signed up… I didn’t hear anything more about that” .Program delivery processes impeded implementation in a number of ways. Business contacts reported time delays between when businesses registered and when contacted by the service provider. This had implications on their enthusiasm for the program and momentum along the program cycle. “This would be easy now that they’ve gone online. But previously the sending of the questionnaires to a certain document management group who would scan the things and put it up online into the profile was tedious and made things difficult” (service provider). Communication between service providers and the GHaW staff was not always effective; further, service providers all reported difficulties when escalating issues and that this often involved two separate government departments. Communication was also unclear from the business’s perspective, not only about how the program worked, but also how the service provider was involved. “It was never very clear about how the system works, I really didn’t understand how they [service provider] plugged into Get Healthy at Work. Those sort of things that you had these separate providers and it wasn’t terribly clear” .The service providers’ delay in contacting businesses were reasoned as being due to initial administrative processes , and challenges with the website not being fully operational and delays in their accessing of the online portal.. “We’re just not even getting to the point where we’re delivering an action plan so we can offer those services. Because we’re not actually allowed to offer them until the action plan’s completed” (service provider). This appeared to affect their commitment, as a group, to the program.Service capacity was also discussed by the service providers as a complication. Some mentioned logistical issues of agreeing to contact a business located in a certain part of the state, only to find that the business was primarily based in another location that might be harder to reach. Many felt that the program ran a high operational cost, and so a balance between investment and revenue generation was felt to be unattained. “. This suggests that interaction with the GHaW program had a positive impact on the business contacts’ perceptions of health promotion in their workplace. Poor perceptions of health promotion in their workplace at baseline may have been an essential reason why this group of businesses first registered for the program in the first place –50. The Investigation of the implementation processes through the qualitative interviews with business contacts and service providers and the focus groups with employees revealed a number of factors contributing to the program outcomes observed; these include both business level factors and program and delivery factors. A recent synthesis of qualitative studies across all phases of WHP development identified six areas that facilitate or hinder WHP implementation. Three identified factors  related to the intervention, the implementer and organisational level; the other three factors included the participant level, the planning phase, and evaluation methodology . Our quaUnfamiliarity with WHPs in general was a major determining factor for how smaller businesses interacted with the GHaW program and in their initial struggle to connect their desire to change their workplaces with the ability to navigate the steps involved. Small businesses make up 97.5% of businesses in Australia  and are Government may play a central role in enabling prioritisation and incentivising health promotion in the workplace . We founProgram delivery factors were also identified as potential barriers to implementation. Technical issues and delays in the operation of the online portal, complicated administrative processes and unclear communication were reasons given for delays in businesses receiving service provider support in the early stages of the program. This may be a key reason for why less than one third of businesses sampled had developed a WHP after 12 months. Quality improvements to processes, for example automation of some of the service provider administrative tasks that were occurring over the course of the evaluation, and clearer communication channels are likely to mitigate many of these issues for future participants . In ordeComplex WHP programs like GHaW require multi-level evaluation methods to determine what worked, how it worked and what mechanisms enabled the program to work in this way . In one The complexity of WHP does not mean that the evaluation needs to be complex , howeverA major strength of the study was that it employed a mixed method approach, which integrated quantitative and qualitative data sources. The quasi-experimental longitudinal analysis allowed individual business responses assessed over time, adjusting for underlying trends in the control sample to account for baseline differences. Additionally, using qualitative data from different perspectives and points in the system  further enriched the analysis of the quasi-experiment. Moreover, evaluations embedded into program delivery can facilitate continuous improvement of programs to optimise program engagement and outcomes. Ongoing program evaluation, which includes feedback loops in WHP design for ongoing refinement and course-correction, is an important practice for an effective program . The evaOne of the major challenges of evaluating health promotion programs in practice is determining population impact. The reach of the GHaW program when it was first launched was limited if we were to consider just the initial sample in its first year as a proportion of all businesses in the State (which was less than 1%). A population dose of < 2% could be considered low impact, while a dose above 5% could be considered both significant and measureable . HoweverA limitation of the study was the rate of attrition in the evaluation sample; particularly at T3 amongst those who had reached the stage of adopting a WHP action plan. This attrition also highlights the challenges of evaluating WHPs in practice. Within the course of a year, businesses may expand or fold or change operating structures as revealed in the qualitative study and response rate of the survey. This can have a profound impact on evaluation. Tracking changes in the intervention group against a control sample, representative of the wider population of businesses, strengthens the study by accounting for secular trends which may be occurring over time  and consThe study relied on subjective measures of productivity and businesses’ interpretation of how far into the program cycle they progressed as a proxy for objective measures. Subjective responses were based on the opinions of the business contact who is likely to have a vested interest in the WHP and its impact. As noted earlier, this evaluation prioritised evaluating implementation at the business level over the employee level, assuming implementation at the business level leads to employee level impacts. Focus groups were used to develop an understanding of the impact as viewed at the employee level, however few businesses contacted felt they had the time or the logistical ability to enable employees to participate. The evidence obtained from these focus groups is therefore incomplete and should be treated with caution. This evaluation was conducted with businesses that were early adopters of the GHaW program and the findings therefore may not be representative of businesses in NSW. Nevertheless, the evaluation provides insights into the potential effectiveness of GHaW, as well as the complexities of implementing WHPs.This study sought to assess the implementation and short-term impacts of an at-scale state-wide WHP. Evaluation of the implementation process revealed some issues in program delivery and in the characteristics of the program that were not matched with what businesses want in a WHP, which may have contributed to why so few businesses had reached a stage of developing workplace action plans. The findings suggest businesses may benefit from earlier involvement by service providers, clearer communication and a supportive system to enable businesses to develop programs tailored to their needs and best practice.Additional file 1:Quantitative business key contact survey. (DOC 61 kb)Additional file 2:Qualitative data key stakeholder interview discussion guides. (DOC 40 kb)Additional file 3:Table S1. Quantitative survey items factor analysis and scale results. (DOC 52 kb)Additional file 4:Table S2. Qualitative research sample characteristics of participating businesses. (DOC 36 kb)Additional file 5:Table S3. Detailed qualitative data findings. (DOC 53 kb)