In 2011, Manitoba implemented a province-wide program of physician detailing and free sampling for generic atorvastatin to increase use of this generic statin. We examined the impact of this unique combined program of detailing and sampling for generic atorvastatin on the use and cost of statin medicines, market share of generic atorvastatin, the choice of starting statin for new users, and switching from a branded statin to generic atorvastatin.We conducted a retrospective study of Manitoba insurance claims data for all continuously enrolled patients who filled one or more prescriptions for a statin between 2008 and 2013. Data were linked to physician-level data on the number of detailing visits and sample provision. We used interrupted time series analyses to assess policy-related changes in the use and cost of statin medicines, market share of generic atorvastatin, the choice of starting statin for new users, and switching from a branded statin to generic atorvastatin.The detailing program reached 31% (651/2103) of physicians who prescribed a statin during the study period. Collectively, these physicians prescribed 61% of statins dispensed in the province. Free sample cards were provided to 61% (394/651) of the detailed physicians. The program did not change the level or trend in the overall statin use rate and the total cost of statins or increase the number of patients switching from another branded statin to generic atorvastatin. We found the program had a small impact on atorvastatin’s market share of new prescriptions, with a level increase of 2.6%.Though physician detailers were skilled at targeting high-prescribing physicians, a combined program of detailing visits and sample provision for generic atorvastatin did not lower overall statin costs or lead to switching from branded statins to the generic. The preceding introduction of generic atorvastatin appeared sufficient to modify prescribing patterns and decrease costs. Controlling prescription drug expenditures remains a top priority for both public and private drug programs. One important cost-saving opportunity is increasing the use of cheaper  generic alternatives . In CanaOne commonly used and effective method to change prescribing is “physician detailing”—the use of face-to-face visits by sales representatives to promote the prescribing of particular medicines , 6. MostWhile the impact of academic detailing in isolation has been well studied , there hIn 2011, Manitoba Health implemented an innovative program aimed at increasing the use of generic atorvastatin, a cholesterol-lowering statin medicine. At the time, atorvastatin was the top-selling drug in the world . Many otIn 2011, Manitoba was the fifth-most populous province in Canada, with a population of 1.2 million . ManitobThe province of Manitoba granted an exclusive listing on the public formulary to the generic manufacturer Ranbaxy for atorvastatin in June 2010 [We conducted a retrospective study of Manitoba insurance claims data for all continuously enrolled patients who filled one or more prescriptions for a statin between June 2008 and March 2013. This includes data from 2 years prior to generic atorvastatin being available on the Manitoba Pharmacare formulary until 21 months following the implementation of the generic detailing and sampling program. Based on prescriber information for each statin prescription, we linked this insurance claims data to physician-level information on the date of each detailing visit and the number of sample cards provided during each encounter.This study used administrative data that includes individual-level data on virtually all Manitobans contained in the Population Health Research Data Repository, which is housed at the Manitoba Centre for Health Policy . Four poSwitching: “Switchers” received a prescription for a statin after the receipt of two or more prescriptions for a different statin (either a different brand or a brand vs. generic) over the past year.Starting: “Starters” received a prescription for a statin after receiving no statin prescriptions in the previous year [ous year .To study the impact of the detailing and sampling program on particular groups, we constructed two sub-cohorts of patients according to the following definitions:We hypothesized that the program would increase the use of generic atorvastatin, decrease the use of other statins, and decrease total costs; that the policy would have a greater impact on new starters than those switching from a branded statin to generic atorvastatin; and that the policy would promote more switching from branded to generic atorvastatin than from branded rosuvastatin to generic atorvastatin.We used interrupted time series analysis to study longitudinal changes in drug utilization and costs, the statins which patients start on, and rates of switching . This meOur analyses used the following measures to determine the impact of the policy:Our analysis of drug utilization focused on the number of prescriptions. In this analysis, we examined whether the detailing and sampling program increased the overall level of statin prescribing in the province or led to changes between different types of statins. For this analysis, we classified drugs into six groups: (1) branded atorvastatin, (2) generic atorvastatin, (3) branded rosuvastatin, (4) generic rosuvastatin, (5) other branded statins, and (6) other generic statins. Branded drugs, such as Lipitor® in the case of atorvastatin and Crestor® in the case of rosuvastatin, are released by the company that originally developed and marketed the medicine. Generics are drugs of the same molecule that come to market after patent protection expires.We examined the impact of the detailing program on statin cost by studying the longitudinal change in overall statin costs within the province.Among switchers, we examined longitudinal changes in the proportion of patients that switched from (1) branded atorvastatin to generic atorvastatin and (2) branded rosuvastatin to generic atorvastatin.Amongst starters, we examined longitudinal changes in the proportion of patients initiated on each of the six types of statins.The detailing program reached 31% (651/2103) of physicians who prescribed a statin during the study period. Collectively, these detailed physicians prescribed 61% of statins dispensed in the province. Between June 2011 and February 2013, 5154 detailing visits were conducted, with the average physician being visited 7.9 times (approximately once every 3 months). The maximum number of visits to one physician was 30. Free sample cards were provided to 61% (394/651) of the detailed physicians. Of those physicians provided with sample cards, the median number of times sampled (visits during which sample cards were provided) was two and the maximum number of times sampled was 14.There were 152,020 statin users in Manitoba with continuous coverage between 2008 and 2013 included in the study. Of these users, 66,091 (43.4%) were new users and 69,876 (46.0%) switched statins over the study period. The age and sex of the study cohort, as well as the new starter and switcher cohorts, are presented in Table p < 0.01, Fig. p = 0.28, trend change of − 40 prescriptions per month, 95% CI − 574 to 494, p = 0.74) nor the detailing and sampling program  had a significant impact on overall statin prescribing.We observed that overall statin prescribing increased steadily by 486 prescriptions per month over the study period  before the introduction of generic atorvastatin , though we found no significant change in trend . We found no further changes in the level  or trend  of statin costs after the implementation of the detailing and sampling program.Total statin costs were rising at a rate of $34,884 per month  or trend  of atorvastatin’s share of new statin prescriptions. We did find, however, that the detailing and sampling program had a small impact on atorvastatin’s market share of new prescriptions, with an increase of 2.5% . Overall, this change translates to approximately 126 more new atorvastatin users in the year following the program than would have been expected based on existing trends. There was no significant impact on the trend of atorvastatin’s market share of new prescriptions following the detailing and sampling program .Prior to the introduction of generic atorvastatin, approximately 1560 patients were newly started on statins each month. As shown in Fig. As a growing number of commonly prescribed drugs lose patent protection, it is becoming increasingly important to promote the use of less expensive generic medicines. Despite being run by a commercial detailing firm and having successfully targeted high-prescribing physicians, we found that the use of physician detailing and sample provision in Manitoba did not meaningfully impact either generic statin use or overall statin costs. Importantly, we found the program did not promote switching established users from branded rosuvastatin, despite this being one of the main aims of the program. Finally, while we did find an impact on the choice of starting statin for new users, the size of this effect was very small.Our results run counter to a significant body of evidence in support of academic detailing , 28–33 aWhile this study was designed to exploit the natural experiment created by the implementation of the policy, there are several limitations to our work. As our study used administrative health data, we could not examine detailed clinical data that might be obtained through medical records. Similarly, we had limited measures regarding the appropriateness of prescribing, patient satisfaction, side effects, low-density lipoprotein levels, and health outcomes for individuals who were started on one statin versus another or were switched. However, much of the available clinical evidence suggests that these drugs have similar efficacy . As the In a strictly regulated environment that includes generic substitution, simply releasing a low-cost generic drug will influence new starters on the drug to be prescribed the generic. Switches on the other hand tend to be more refractory, and even well-designed and evidence-based programs to accelerate these pharmaceutical policies have no room to further increase starts and do not affect switching.


QKI-6) by binding to its 3′-UTR, and thus inhibited GSC malignant behavior. In addition, Wilms’ tumor 1-associating protein (WTAP) was identified as a downstream target of QKI-6. Overexpression of miR-29a in GSCs inhibited expression of WTAP and suppressed both phosphoinositide 3-kinase/AKT and extracellular signal-related kinase pathways by downregulating QKI-6, thereby inhibiting cell proliferation, migration, and invasion but promoting apoptosis. We have characterized a novel miR-29a/QKI-6/WTAP axis in GSCs, which may provide theoretical support for the treatment of glioblastoma with miR-29a agomirs.Glioblastoma is the most common type of malignant primary brain tumor and has high recurrence and lethality rates. Glioblastoma stem cells (GSCs), a subpopulation of glioblastoma cells, may promote rapid tumor recurrence and therapy resistance. Because altered microRNA (miR) expression in GSCs may lead to glioblastoma progression, we assessed the effects of miR-29a expression on the oncogenic behavior of GSCs. MiR-29a expression was lower in GSCs than non-GSCs, and overexpression of miR-29a in GSCs inhibited cell proliferation, migration and invasion, but promoted apoptosis. MiR-29a directly inhibited the expression of Quaking gene isoform 6 ( Glioblastoma, regarded as a grade IV astrocytoma, is the most common and malignant type of primary brain tumor. Some studies have demonstrated that glioblastomas contain a small subpopulation of cells with the ability to self-renew and to initiate brain tumors. These cells are considered to be glioblastoma stem cells (GSCs), which promote tumor progression, treatment resistance, and tumor recurrence –3. NovelMicroRNAs (miRNAs) are small, non-coding RNA molecules of approximately 19-22 nucleotides. Many recent studies have revealed that miRNAs regulate the behavior of GSCs and may have similar functions to oncogenes or tumor suppressors . For exaQuaking (QKI) gene was initially defined for its role in myelination in the central nervous system. Three major alternatively spliced mRNAs of QKI  are expressed, encoding the proteins QKI-5, QKI-6 and QKI-7, which differ only in their 30 C-terminal amino acids. The QKI proteins selectively bind to a short sequence, the QKI response element (QRE), which is frequently located in the 3′-untranslated regions (UTRs) of target mRNAs  motif in its 3′-UTR. The QRE includes a core ACCAAC sequence and a half-site CAAU, to which QKI-6 may bind and thus regulate WTAP expression. To determine whether WTAP participates in the tumor suppression pathway of miR-29a in GSCs, we evaluated WTAP expression in glioma tissues and GSCs. An IHC assay on tissue microarray chips (containing the same tissue samples used in the QKI-6 IHC assay) indicated that WTAP is localized throughout the nucleoplasm in glioma cells  to search for downstream targets of QKI-6. s Figure  and is oP<0.05, Figure WTAP directly, we used plasmid vectors to either knock down QKI-6, overexpress wild-type QKI-6 (pIRES2-QKI-6), or overexpress a mutant form of QKI-6 missing the WTAP binding sites (pIRES2-QKI-6-Mut). WTAP expression was lower in QKI-6-knockdown GSCs-U87, GSCs-U251 and GSCs-GTD than in their respective NC groups , and was much greater in the pIRES2-QKI-6 groups than in the pIRES2-QKI-6-Mut groups, but did not differ significantly between the QKI-6 mutant groups and the NC groups. These results demonstrate that QKI-6 can directly regulate WTAP expression.Overexpression of miR-29a significantly reduced WTAP expression in GSCs-U87, GSCs-U251 and GSCs-GTD, compared with their respective NCs . Furthermore, miR-29a overexpression combined with QKI-6 inhibition reduced WTAP expression to a greater extent than miR-29a overexpression alone . There were no significant differences between the control groups and the NC groups.To determine whether miR-29a inhibits the expression of WTAP by downregulating P<0.05). Furthermore, p-PI3K expression was obviously lower in the agomir-29a+pGPU6-shQKI-6 groups than in the agomir-29a groups . There were no significant differences between the control groups and NC groups. The analyses of the AKT and ERK pathways revealed similar changes in the GSCs.As shown in Figure VEGF-A  or high expression [2 (++) to 3 (+++)].Immunostained microarray chips were scored for the percentage of stained cells  and staining intensity (−ΔΔCt) method.Trizol reagent  was used to isolate total RNA from GSCs and non-GSCs, and a Nanodrop Spectrophotometer (ND100) was used to evaluate the RNA concentration and quality of each sample. The total RNA was then reverse-transcribed into cDNA with a Taqman MicroRNA Reverse Transcription Kit. Real-time PCR amplification was carried out using Taqman Universal Master Mix II with the TaqMan MicroRNA Assay of miR-29a and U6 on a 7500 Fast Real-Time PCR System . U6 was used as an endogenous control. Fold-changes were calculated through the relative quantification (2The miR-29a agomir (agomir-29a) and negative control (agomir-29a-NC) were synthesized by GenePharma , and were transfected into GSCs with Lipofectamine 3000 . The transfection efficacy was evaluated by quantitative real-time PCR, and 48 hours post-transfection was selected as the harvesting time for subsequent experiments, as the highest transfection efficacy was attained at this time point.QKI-6 gene (pGPU6-shQKI-6) or the negative control (pGPU6-shQKI-6-NC) (GenePharma). In other experiments, the QKI-6 gene or a mutant QKI-6 gene lacking the WTAP binding site (V157E) was cloned into the pIRES2-EGFP plasmid vector by SalI to allow overexpression of QKI-6 (pIRES2-QKI-6) or its mutant (pIRES2-QKI-6-Mut) . The same negative control (pIRES2-QKI-6-NC) was used for pIRES2-QKI-6 and pIRES2-QKI-6-Mut. Stable transfection was performed with Lipofectamine 3000 and P3000  per the manufacturer's instructions in 24-well plates with cells at about 80% confluency. The stably transfected cells were selected with culture medium containing 0.5 mg/mL G418 . G418-resistant cell clones were established after approximately four weeks. The transfection efficacy was determined by Western blot analysis.For the QKI-6 knockdown experiment, we transfected GSCs with plasmids containing a short hairpin RNA directed against the human 3 cells per well. Five replicate wells were used for each group. Three 96-well plates were incubated for 24, 48, and 72 hours, respectively; then, 10 μL CCK-8 was added to each well, and the plates were incubated for another 3 hours. The absorbance at 450 nm was measured on an ELX-800 spectrometer.Cell proliferation was assessed with a Cell Counting Kit-8 . The transfected GSC spheres were harvested by mechanical centrifugation, trypsinized into single cells, suspended in 100 μL GSC medium, and seeded in 96-well plates at a density of 3×10Apoptosis was assessed with the Annexin V-FITC/PI kit  with no light for transiently transfected cells, and with the Annexin V-PE/7-AAD kit  with green fluorescence for stably transfected cells. Cells were harvested and stained according to the manufacturer's instructions, and then were analyzed by flow cytometry. Data analysis was performed with CELLQuest 3.0 software. The upper right quadrant represents late apoptosis, while the lower right quadrant represents early apoptosis. The calculation of the total apoptotic rate included both the upper and lower right quadrants.5 cells were seeded in GSC medium in the upper chamber. Then, the lower chamber was loaded with 600 μL of medium containing 10% FBS. After incubation at 37 °C for 48 hours, the cells in the upper chamber were removed carefully with a cotton swab. The cells that had traversed the membrane were fixed and then stained with 20% Giemsa. For the cell invasion assay, the upper chamber was pre-coated with a 500 ng/μL Matrigel solution  and seeded with 1×105 cells, with the remainder of the procedure being similar to the migration assay. All chambers were photographed, and five randomly-selected fields were counted under a microscope.For the cell migration and invasion assay, a 24-well chamber with 8-μm pores was used. First, 1×10Western blotting was performed by standard methods. The PVDF membranes were incubated overnight at 4 °C with primary antibodies as follows: anti-QKI-6 , anti-WTAP , anti-PI3K, anti-p-PI3K, anti-AKT, anti-p-AKT, anti-ERK1/2, anti-p-ERK1/2 , and anti-GAPDH . The membranes were washed and then incubated with a horseradish peroxidase-conjugated secondary antibody  at room temperature for two hours. Finally, signals were detected with an enhanced chemiluminescence detection system. The relative integrated density values (IDVs) were measured with FluorChem 2.0 software and calculated with GAPDH as the internal control.QKI-6. The 3′-UTR fragment of the wild-type QKI-6 gene (QKI-6-3′-UTR-Wt) or the mutant gene with respect to the predicted miR-29a binding sites (QKI-6-3′-UTR-Mut) were subcloned into pmirGLO Dual-luciferase miRNA Target Expression Vectors (GenePharma). HEK 293T cells were seeded in a 24-well plate and co-transfected with QKI-6-3′-UTR-Wt (or QKI-6-3′-UTR-Mut) and agomir-29a by means of Lipofectamine 3000. After the cells were incubated for 48 hours at 37 °C, the luciferase reporter assay was performed with the Dual-Luciferase Reporter Assay System .We used bioinformatics databases  to search for the miR-29a binding site in in vivo xenograft assays, four-week old BALB/C athymic nude mice were purchased from the Shanghai Laboratory Animal Center and raised in laminar flow cabinets under specific pathogen-free conditions. The nude mice were divided into four groups according to the injection, as follows: control group (only GSCs), miR-29a (+) group (GSCs stably overexpressing miR-29a), pGPU6-shQKI-6 group (GSCs stably overexpressing the QKI-6 inhibition plasmid), and miR-29a (+) + pGPU6-shQKI-6 group (GSCs stably overexpressing miR-29a and the QKI-6 inhibition plasmid). Each group contained 15 mice. Mice were injected subcutaneously with 3×105 cells in the right axillary fossa. Tumor size was measured every five days until 60 days after injection, and was calculated by the formula: volume (mm3) = length×width2/2. For survival analysis in orthotopic inoculations, 3×105 cells were stereotactically implanted into the right striatum of nude mice. The number of surviving nude mice was recorded until 60 days post-injection, and survival was analyzed with the Kaplan–Meier survival curve. All the above procedures followed protocols approved by the Animal Care Committee of China Medical University.For the t-test or one-way ANOVA. Statistical significance was set at P<0.05.All statistical analyses were carried out with SPSS 17.0 statistical software, and the data are expressed as means ± standard deviations (SDs). Statistical significance was calculated by Student's


The number of community paramedic (CP) programs has expanded to mitigate the impact of increased patient usage on emergency services. However, it has not been determined to what extent emergency medical services (EMS) professionals would be willing to participate in this model of care. With this project, we sought to evaluate the perceptions of EMS professionals toward the concept of a CP program.U test, Kruskal-Wallis one-way analysis of variance, and Pearson correlation as appropriate. Multivariate logistic regression was performed to examine the impact of participant characteristics on their willingness to perform CP duties. Statistical significance was established at p ≤ 0.05.We used a cross-sectional study method to evaluate the perceptions of participating EMS professionals with regard to their understanding of and willingness to participate in a CP program. Approximately 350 licensed EMS professionals currently working for an EMS service that provides coverage to four states  were invited to participate in an electronic survey regarding their perceptions toward a CP program. We analyzed interval data using the Mann-Whitney Of the 350 EMS professionals receiving an invitation, 283 (81%) participated. Of those participants, 165 (70%) indicated that they understood what a CP program entails. One hundred thirty-five (58%) stated they were likely to attend additional education in order to become a CP, 152 (66%) were willing to perform CP duties, and 175 (75%) felt that their respective communities would be in favor of a local CP program. Using logistic regression with regard to willingness to perform CP duties, we found that females were more willing than males  and that those participants without any perceived time on shift to commit to CP duties were less willing than those who believed their work shifts could accommodate additional duties .The majority of EMS professionals in this study believe they understand CP programs and perceive that their communities want them to provide CP-level care. While fewer in number, most are willing to attend additional CP education and/or are willing to perform CP duties. The number of patients presenting to emergency departments (ED) in the United States has been steadily increasing over the past three decades.3The traditional model of care used by EMS systems in the U.S. has been to treat and transport patients to the ED for further assessment and care provided by physician and nursing staff. While each community’s respective CP program can be tailored to its individual needs, the CP model, in general, uses specially educated paramedics to treat minor injuries and manage chronic illnesses in the patient’s home and/or arrange care provided in the community, thereby limiting unnecessary transports to the ED.3Potential benefits of patients being assessed and treated in the prehospital environment include a reduction on the patient load in already strained EDs and a reduction in cost along the healthcare continuum. As the Affordable Care Act implementation continues to unfold, reducing the number of patients readmitted to the hospital within 30 days of discharge with diagnoses such as congestive heart failure, acute myocardial infarction, and pneumonia will also be financially advantageous for hospitals faced with potentially reduced levels of reimbursement for services.5Although the problem of ED crowding from increased patient visits has been identified, and some local CP programs have demonstrated a reduction in the number of patients transported to these EDs,What do we already know about this issue?Several studies have been published regarding community paramedic (CP) programs; we have not found any that address EMS professionals’ willingness to participate.What was the research question?What are EMS professionals’ attitudes about CP programs and are they willing to participate in them?What was the major finding of the study?Most EMS professionals stated they were willing to participate in a CP program.How does this improve population health?Meeting patients’ healthcare needs in their respective communities may reduce unnecessary ED visits and lessen the strain on crowded EDs.The authors used a cross-sectional survey  method i10EMS professionals  practicing in a hospital-based Advanced Life Support (ALS) EMS service headquartered in southwest Missouri were used as a sample of convenience. This EMS service delivers ALS ground coverage for 15 counties in four states  and consists of 9,871 square miles of service area. The coverage area is a mixture of rural, urban, and suburban areas with a total population of 686,462. The annual call volume is over 60,000 ambulance requests covered by 350 EMS professionals working out of 29 stations with 48 ambulances.To acquire a representative sample from the entire coverage area of the service, a request for voluntary participation by EMS professionals and a link to the electronic survey were sent to their respective hospital-issued electronic mail address by each regional manager. We collected demographic information  via the survey instrument. The survey was delivered in an electronic format using Qualtrics online software.11We asked participants to respond to a series of statements using a seven-point Likert-type scale. The survey was validated using an expert panel of four individuals with experience in emergency medical care. Three of the evaluators are EMS professionals with experience and licensure as paramedic-level instructors. Two have doctoral degrees, one of whom developed a CP education program at his university. The final evaluator is a practicing emergency physician with 19 years of experience and currently serves as the medical director for an ED, two EMS services, and an EMS education program. Following review and revision by the expert panel members, a pilot of the revised instrument was performed.The pilot testing sample consisted of a group of 16 currently licensed and practicing EMS professionals. We obtained a written letter of agreement to recruit participants from the service’s chief officer prior to initiation of the survey. These individuals were requested to provide additional feedback on ease of use, readability, and offer suggestions for further improvements. The pilot group was excluded from participating in the study.The Western Carolina University Institutional Review Board (IRB) approved this study. It also received approval from the IRB representing the hospital that oversees the EMS service which participated in this study. All study subjects granted their consent prior to participation in the study, and their responses were anonymous.All data were uploaded from Qualtrics into and analyzed with the Statistical Package for the Social Sciences (SPSS).U test to analyze the effects of gender as well as the current usage of a CP model concerning participants’ willingness to perform CP duties. A Kruskal-Wallis one-way analysis of variance was used to analyze the effects of participant characteristics  on their outcomes. We used the Pearson product-moment coefficient of correlation to determine if there was a relationship between factors  and participant outcomes. Multivariate logistic regression was also performed to examine the impact of participant characteristics  on the likelihood of participants’ willingness to perform CP duties.We used a Mann-Whitney The survey was opened on January 19, 2015, and closed on February 23, 2015. Members of the EMS service used for this research were notified of survey availability via an email to their work email account sent from their manager. Of the 350 EMS professionals receiving an invitation, 283 (81%) participated. Consent was given by 277 (98%) of the respondents and comprised the final data set for analysis. There were no sequential questions that forced participants to respond to a question before answering additional questions. Consequently, not all questions were answered by all respondents. See While the survey was conducted using a hospital-based EMS service, many of its part-time coworkers have primary EMS employment with different types of services. As expected, hospital-based EMS was the type of service listed as the primary work experience for a majority of the participants . Survey responses were representative of differing types of response settings and hours worked per shift .When questioned about the perceived number of hours per shift worked in which the participant could commit to a CP program, 73 (31%) indicated that they could not commit any time. A majority of the respondents  stated that they could spend one hour or more, with 51 (22%) indicating that they could commit more than four hours per shift to CP duties. Providers perceived that 47% (SD 1.7) of the patients they currently encounter in the field could potentially benefit from a CP program.When questioned as to their confidence in their understanding of what a CP program entails, 165 (70%) revealed feeling knowledgeable as to the requirements of a CP program. Eighteen (8%) selected a neutral response. Fifty-three (22%) indicated that they did not have a good understanding about CP programs. Regarding EMT and paramedic respondents’ perceived understanding of CP programs, positive responses were given by 49 (65%) and 111 (75%) respectively. A total of 135 (58%) survey participants indicated that they were likely, somewhat likely, or very likely to attend additional education to become a CP with 45 (19%) undecided. Fifty-six (23%) indicated that they were unlikely, somewhat unlikely, or very unlikely to participate in additional CP education.In response to whether or not they would perform CP duties with as much or more enthusiasm as they currently have for traditional, prehospital patient care, 152 (66%) survey participants indicated that they would. The number of undecided was 38 (16%). Forty-three (18%) gave a negative response. Regarding responses per licensure level, 44 (58%) EMTs and 102 (69%) paramedics indicated a willingness to participate in a CP program.When questioned if a CP program should be a significant responsibility for EMS in their community, 174 (74%) respondents gave a positive indication. Thirty-eight (16%) were neutral in their responses. The 24 (10%) other respondents perceived that a CP program should not be a significant responsibility for their community’s EMS agency.Three-fourths of those participating  felt that their respective community would be in favor of their service performing CP duties. Forty-eight (21%) were unsure of their community’s reaction to a local CP program. A minority of the respondents  felt that their community would not be in favor of CP duties being performed by their service. See Regarding the respondents’ type of community served and their willingness to complete CP duties, we found no statistical significance (p = 0.74). Participants who reported working for an EMS service that currently uses a CP model of patient care delivery did not have a statistically significant difference (p = 0.89). Additionally, reported rank (field provider of care vs. non-field provider of care [supervisor]) and the participants’ willingness to perform CP duties did not produce a statistical significance (p = 0.34). See Regarding participants’ willingness to perform CP duties, the results of the regression model correctly classified 79.2% of all the cases. Females were four times more likely than males to indicate a willingness to perform CP duties . The respondents perceiving that they had no spare time while on duty to commit to CP duties were less willing to accept the additional duties of a CP program than those who perceived they had any time on duty for these activities . See While some reports on the effectiveness of CP programs regarding the reduction of ED bed hours, unnecessary ambulance transports, and emergency services’ cost savings appear in the literature,Female participants were more than four times as likely as their male cohorts to express a willingness to participate in a CP program. Female willingness may be impacted, in part, by participants’ empathy levels. Compared to males, Williams et al. found that female paramedic students had higher empathy ratings toward all medical conditions queried.17We found no statistically significant differences in willingness to participate in a CP program with regard to EMS provider level, age, level of education, type of shift, community served, or current rank. EMTs and paramedics both expressed an interest in providing CP-level care to their communities. While a study by Simpson et al. found that younger EMS providers with a tertiary education had a higher level of support for evidence-based practice,Prior to implementing a CP program, EMS leaders should investigate potential barriers to successful implementation. While there is a paucity of research exploring the attitudes of EMS professionals regarding CP programs, the importance of employee acceptance to successful implementation of new programs has been reported among other public safety professions. For example, several reports examining the attitudes of law enforcement officers (LEOs) and firefighters toward the nontraditional role of providing care to patients in the prehospital environment exist in the literature.Over the past four decades, policing strategies have changed from an enforcement role to one of problem-solving.Green et al. found that LEOs expressed concerns with the added responsibilities inherent in preventing overdoses in a naloxone administration program.When attempting to implement an AED program, Husain et al. found that law enforcement leaders were challenged by LEOs who did not believe providing medical care was a part of their role in the community.Another obstacle faced when attempting to implement law enforcement and fire service AED and naloxone programs was hesitancy of the LEOs and firefighters to use the new equipment out of a concern for perceived new liability.Many obstacles to successful implementation of novel programs in emergency professions that can correlate to EMS providers’ implementation of CP programs have been identified. Fortunately, several items have been found to assist with the start of new programs. When LEOs and firefighters felt that their new roles could benefit the communities they served, they were more likely to have positive attitudes toward these new roles.All police chiefs and a majority of their respective officers surveyed by Papson et al. believed that the use of AEDs by police was not only appropriate, but also a valuable service to their communities.In addition to the benefits provided to the community via the new roles of LEOs and firefighters, a majority of public safety professionals surveyed agreed that providing EMS-related activities improved the public’s perception of the participating departments and their members.A positive attitude among LEOs and firefighters was found when they personally experienced the impact of EMS-related activities while performing their traditional public service duties.A significant barrier to implementing AED programs for public safety professionals has been negative attitudes related to a lack of education regarding the programs and their benefits.EMS leaders interested in pioneering a CP program in their area could use this information when making strategic plans for growth and expansion of their services in the community. This survey’s results indicate that EMS professionals perceive they understand CP programs, support providing CP services to their community, and are willing to participate in this model of care delivery.A significant limitation to this study was a lack of diversity among the subjects regarding the type of their primary EMS service. The service used for this study was hospital-based. While the service does employ part-time EMS professionals who have full-time employment at different types of EMS services, a majority of the respondents were hospital-based. The sample size was also relatively small and geographically non-diverse. A majority of the participants also had no current CP experience.The lack of racial and gender diversity among the participants are other limiting factors. Survey respondents identified themselves as White and as male, 94% and 75% respectively. While a 2013 study by Bentley et al. found that 85% of nationally certified EMS professionals identified themselves as nonminority and 74% as male,As with all cross-sectional studies, participants may be biased toward or against participation based on their feelings regarding the topic. In addition, several participants did not respond to all questions. The lack of complete responses combined with self-reporting could have led to biased results.While our results indicated that EMS professionals are generally receptive to participating in a CP model of care delivery, future studies are needed to confirm the findings in different regions, different types of EMS agencies, and among a more diverse group of EMS professionals.The authors sought to quantify the attitudes of EMS professionals toward a CP program and found that the majority of those surveyed believe they understand what a CP program entails, most are willing to attend additional education to offer CP services and are willing to serve in this new role. EMS administrators might reach buy-in from their employees if separate CP shifts or CP positions are offered instead of adding these new responsibilities to their employees’ current job duties. Results of this survey are limited by the predominate type of EMS service represented, as well as the geographical location and limited racial and gender diversity among the participants. Further studies are needed to assess the opinions of EMS professionals in differing types of EMS agencies, different geographical locations, and differing proportions of ethnicities and genders. These results will be important for EMS administrators and medical directors planning to develop and implement CP programs.


Documentation is a required component of a residency program, but can be difficult to collect and disseminate, resulting in minimal utilization by residents and faculty. The purpose of this work is to adapt a commercially‐available Web‐based medical residency management system to improve the learning experience by efficiently distributing program information, documenting resident activities, and providing frequent monitoring and timely feedback of resident progress. To distribute program information, program requirements and rotation readings were uploaded. An educational conference calendar was created with associated files and attendance records added. To document resident progress, requirements for over 37 different clinical procedures were added, for which the resident logged the total number of procedures performed. Progress reports were created and automatically distributed. To provide feedback to the resident, an extensive electronic evaluation system was created. Results are shown for the initial 21 months of program existence, consisting of a single resident for the first 12 months and two residents for the subsequent 9 months. The system recorded that 130 documents were uploaded and 100% of required documents were downloaded by the resident. In total, 385 educational conferences and meetings were offered, of which the residents attended 95%. The second‐year and first‐year residents logged 1030 and 522 clinical procedures, respectively. The residents submitted a total of 116 status reports detailing weekly activities, 100% of which were reviewed by faculty within an average of 11.3 days. A total of 65 evaluations of the residents were submitted. The residents reviewed 100% of respective evaluations within an average of 1.5 days. We have successfully incorporated a paperless, Web‐based management system in a medical physics residency program. A robust electronic documentation system has been implemented, which has played a central role in enhancing the training experience.PACS number: 01 I.www.campep.org). CAMPEP guidelines list the records required for their review of a program.Documentation is a required component of a residency program and is necessary for accreditation by the Commission on Accreditation of Medical Physics Educational Programs  designed for medical residencies, was adapted for use in a physics residency program. This system is a secure, Web‐based database, accessible for viewing and updating at any time through an Internet connection.A commercially available electronic residency management system  procedures multiple times. To efficiently document a running total of cases, procedures were added with minimum numbers of times they were required to be performed. Tracking of these procedures was performed on a daily basis and included information about the resident's involvement level  and involved faculty.To document the resident's didactic learning activities, the calendar and scheduling features were utilized. Meetings, course lectures, and educational conferences required by the program were added to the calendar, in which resident attendance was recorded. Files associated with presentations, such as journal club articles and presentation slides, were uploaded . The residents attended an average of 95% of educational conferences and meetings, exceeding the program's required attendance level. The system stored 28 files, including journal club articles and educational presentations, which were uploaded to their respective conferences.A total of 116 reports documenting the resident's weekly activities were submitted by the residents. An example of a submitted report is displayed in Fig. Requirements for 37 different types of clinical and QA procedures were added. At 21 months in the program, the second‐year resident logged a total of 1030 procedures, of which he observed 12, assisted in 64, and actively performed 954. At eight months in the program, the first‐year resident logged a total of 522 procedures, of which he observed 22, assisted in 25, and actively performed 475. The procedure logs provided clear expectations and a method for the resident to self‐pace and prioritize activities. Additionally they allowed faculty to quickly identify the resident's participation level for specific clinical tasks. Progress toward requirement completion is shown in Fig. At 21 months in the program, the second‐year resident had completed six rotations and was currently in his seventh rotation. During that time, 47 evaluations providing feedback to the resident were submitted by faculty (average of 2.4 per month). At eight months in the program, the first‐year resident had completed three rotations and was currently in his fourth rotation. During that time, 19 evaluations providing feedback to the resident were submitted by faculty (average of 2.1 per month). One hundred percent (100%) of these evaluations were reviewed by both residents within an average of 1.5 days (standard deviation 2.5 days).IV.Major goals of using an electronic residency management system are to provide efficient documentation methods and compile information that is well utilized. In our experience, automation tools, easy data entry methods, and instant display of feedback have contributed to the successful achievement of such goals. The large amount of feedback provided to residents via evaluations was made possible by the automatic distribution of electronic forms to the evaluators. Evaluators were automatically sent emails notifying them of new forms to be completed, as well as when evaluations were late. Rather than staff being tasked to distribute and collect paper forms, as well as remind faculty to complete forms on time, the electronic system automatically performed that work. The system was also set up to automatically email rotation goals and objectives at the start of each corresponding rotation to the residents and faculty, thus effortlessly providing notification of program expectations and rotation dates. Automated tools provided efficient monitoring of resident compliance — for example, by calculating the percentage of conferences the residents attended and recording when required readings were downloaded. Easy data entry methods also contributed to the successful utilization of procedure tracking by residents. Rather than requiring residents to keep a free text record, they simply clicked on the procedures performed that day. Since implementation, procedure tracking continues to be well used, which confirms its continued utility. Instant display of feedback is another advantage of electronic systems. Graphing features ; however, after 21 months in the program, the average increased to 11.3 days (standard deviation 11.7 days). There are several possible reasons for the increased time frame. These particular forms do not have notifications automatically emailed to the faculty when residents have submitted a form, thus reducing their visibility to faculty. Fatigue from reading weekly reports can occur, so a meeting where faculty verbally review activities with the resident is preferred, whenever possible. Finally, using a commercially available system can limit customization options. We currently still use paper checklists to document detailed rotation goals and objectives because we haven't yet found an optimal tool to electronically adapt the checklists to our desired use.www.e‐value.net) and Typhon Group systems  provide tools such as electronic evaluations and/or procedure tracking. A physics residency program need not rely on commercial products to create an electronic documentation system, as one can generate and customize their own internal system using commonly available programming or database tools. Therefore, despite the limited availability of the particular system in this study, the customization, utilization, and experience gained from our implementation of an electronic documentation system can be useful for other physics programs.It is important to note that the specific commercial system used in this study, MedHub, has limited availability to those institutions having multiple residency programs and is deployed through the institution's Graduate Medical Education (GME) office. Not all institutions use MedHub, and not all medical physics programs are structured within their institution's GME office. However, if the institution has deployed MedHub, the physics program itself need not be administered through GME, but simply needs permission from their GME office to gain access to MedHub, which was the case for our program. Despite this, potential uses of an electronic documentation system can expand outside of a single residency program and are not dependent on the specific system. Other electronic management systems designed for health‐care education, such as E‐Value  have been accredited within the last four years.This work has also provided a database of valuable information. Quantifiable data on resident progress is now available, allowing for more objective metrics for benchmarking competency levels. Specifically, the clinical procedure tracking allows the collection of the numbers of different procedures that residents perform. This can be analyzed across different programs, which provides data to potentially establish minimum numbers of procedures required. This system provides the information for a data‐driven approach in establishing guidelines, potentially standardizing and enhancing the quality of physics residency programs.V.A commercially available medical residency management system has been successfully adapted for use in a medical physics residency program. This has played a central role in enhancing the training experience by providing a vehicle for widespread communication of program requirements, documentation of resident activities, and frequent monitoring and feedback of training progress.A robust electronic system has been implemented in which hundreds of documents have been uploaded and viewed, clinical procedures logged, and educational conferences attended. The use of such a system can provide more robust, uniform documentation across different programs. The data collected by such system can provide information for establishing guidelines which can further standardize and enhance the quality of physics residency programs.The authors gratefully acknowledge our institution's Graduate Medical Education office, especially Dr. Carol Rumack, Gail Silber, and Ashley Walter, for providing our program access to the MedHub software. We thank Sandra Korn, our Program Coordinator, who has been instrumental in the success of this system. Finally and most importantly, we thank the current residents, Jerry George, Ph.D. and Joseph John Lucido, Ph.D., for their diligence and extensive use of this documentation system.Supplementary MaterialClick here for additional data file.


Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders’ needs. In this paper, we share what we learned from this program development process.Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders’ educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders’ needs/interests. A post-program survey evaluated program participants’ perceptions about the PSI educational program.Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed—three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive.Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders’ needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.The online version of this article (10.1186/s12913-018-2904-5) contains supplementary material, which is available to authorized users. Patient safety measures are increasingly used for public reporting and pay-for-performance –3. ImpliAmong the strategies that have been used to educate users and other stakeholders about the PSIs are the AHRQ Quality Indicators Toolkit , a modifThe present study was prompted by information within the VA that public reporting of VA PSI rates was imminent. Based on interviews with VA patient safety managers in a prior study , we knewWe conducted an implementation and evaluation study from 2011 to 2013. We began with a formative evaluation  involvinOur overall study population consisted of key VA stakeholders at 132 VA acute-care hospitals nationwide who were involved in quality/safety initiatives: middle managers  and senior managers .Within the overall population, we anticipated higher levels of participation from the middle manager group than the senior managers, based on our prior work . The midHowever, we did include senior managers because they are also key stakeholders: they are heavily involved in setting their organizations’ improvement priorities and reviewing performance reports with the patient safety, quality, and performance improvement managers for their hospital. Thus, when developing the PSI educational program, it was also important to understand their needs on learning about the PSIs.As a first step in the formative evaluation, we sought interviews with informants from eight of the 132 VA hospitals. We selected these sites based on their geographic diversity, variation in PSI rates, hospital size (number of Veterans served), and our knowledge about these facilities’ QI activities within the VA. Within each of these sites, we recruited 16 potential informants from our middle manager stakeholder group—all either Patient Safety Managers or Quality/Performance Improvement Managers—for the reasons described above, as hospital-level content experts on PSI-related learning needs at their facilities.We developed an interview guide based on our experience from a prior PSI study and a literature review , 26–28. Using the findings from the interviews, we developed a pre-program web-based survey that we then administered to all key VA stakeholders (described in the study population section above). The survey consisted of 56 closed-ended questions covering the a priori concepts plus the additional concepts that emerged from the data analysis of the interview guides as well as an additional literature review on organizational improvement . ConceptThe web-based survey was programmed and administered using Verint® Enterprise Feedback Management, a VA-approved software package that is widely used for survey development and administration. We used descriptive statistics to characterize our study sample and their responses to the survey questions, and qualitatively analyzed the open-ended responses to identify prevalent themes across respondents.Program development was informed by the formative evaluation survey results. It was also informed by a review of existing patient safety/quality educational materials from organizations including VA Health Services Research and Development’s Center for Information Dissemination and Education Resources (CIDER), AHRQ, and IHI , 32, 33.To learn about stakeholders’ perceptions of the program, a post-program evaluation survey was administered approximately 1 month after the PSI education program ended. This survey, consisting of 90 closed-ended questions and 12 open-ended questions, was administered to those VA stakeholders who had both (1) responded to the pre-program survey and (2) registered for the educational program. Response options were either a five-point Likert scale that ranged from 1 = strongly disagree to 5 = strongly agree, or yes/no. Additional file We used similar survey administration processes and data analyses for both the pre-program and post-program surveys. In addition, we matched the pre- and post-responses of individuals that participated in both surveys. Our analyses included comparison of pre- and post-program survey responses for the same 7 questions that appeared on both surveys.Some informants had heard about the PSIs at meetings held by national VA quality and safety offices, whereas other informants had not heard about the PSIs and were not aware that PSI rates were in reports that they received.Some were unclear on how to interpret PSIs.While some had received education on other performance measures from national VA offices, most reported that they had not received any formal education on the PSIs.Informants suggested ways they would like the PSI education to be delivered, such as web-based training and information placed on a VA SharePoint.Informants suggested making the educational program more relevant to the audience we were trying to educate and also more interactive.Informants believed that improvement priorities can be heavily driven by a hospital’s performance on quality/safety measures.Informants felt that improvement priorities are largely set by senior management according to internal hospital priorities; however, they can also be set according to mandates from the VA National and Regional Offices.We interviewed nine of the 16 potential informants: four Patient Safety Managers and five Quality/Performance Improvement Managers. This included at least one informant from each of the eight selected sites. Several themes that emerged from the interviews (below) helped guide our development of the formative evaluation pre-program survey and were also taken into consideration when developing the educational program:The interviews informed us about stakeholders’ potential PSI educational needs as well as aspects of organizational contextual factors that may drive hospital-level improvement priorities. Review of the interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Because we were able to identify helpful information through our a priori concepts and interview questions, we used similar concepts and several of the specific interview questions in the survey questionnaire. For example, in the interview, we asked informants “If you were to receive education on the PSIs, how would you like it to be delivered?” Some of the options that emerged from informants’ answers to this question included: having access to materials on a website, conducting presentations of case studies, conducting presentations via LiveMeeting (a widely used VA web-based meeting platform), and including specific information about the PSIs in a handout or PowerPoint. We then included several of those options as response items when developing the survey question:Web conferencing (e.g. LiveMeeting)Video conferencing Reports or journal articlesWritten case studiesVideo/audio materials Face-to-face conferenceQ&A sessions I would prefer to learn about the PSIs through:Table We administered the pre-program survey to 782 VA stakeholders at 94 out of the 132 VA hospitals (71%); 181 (23%) responded. Table The majority of respondents indicated that they were aware of the PSIs and had received reports containing PSI rates , yet their understanding of how rates were calculated and interpreted was considerably lower . Almost 50% of respondents reported having received education about the PSIs from VA Central Office. They were most interested in receiving the education via web-conferencing  (84%); education about the PSIs through written materials was of least interest. Respondents’ top two PSI-related educational interests were in monitoring trends in patient safety/quality (93%) and interpreting PSI rates (92%), although many of the other content areas also appeared to be of interest. Figures In addition, we learned more about whether survey respondents perceived the PSIs as an organizational priority and what factors might be barriers to using the PSIs for QI. As shown in Fig. We invited all 181 VA stakeholders who responded to the pre-program survey to participate in the educational program; 82 (45%) registered to participate. Guided by stakeholders’ input from the findings of the pre-program survey and the interviews, we developed and implemented a PSI educational program that was tailored to stakeholders’ needs in the following ways:We offered six cyberseminar sessions via LiveMeeting (a web-conferencing platform used throughout the VA) because survey respondents preferred this mode. Each session was 45 min and was offered once. On average, nine participants attended each of the cyberseminar sessions. A majority of cyberseminar participants were from our middle manager stakeholder group.In addition, by offering the program as cyberseminars, we were able to easily record and archive the sessions on the program’s SharePoint site so that stakeholders who were unable to attend the live cyberseminars, or who wanted to revisit the sessions, could readily access them. (Recordings can be made available upon request from the corresponding author.) This allowed us to tailor the program based on stakeholders’ input from the interview.To make the sessions more interactive, the cyberseminars were conducted in an interview format, similar to the style that AHRQ uses in its podcasts about the Quality Indicators and Toolkit, with a moderator interviewing a speaker. Cyberseminar speakers were study team members with extensive knowledge about the topic presented, or representatives from our VA national partners’ offices [VA National Center for Patient Safety (NCPS) and VA Inpatient Evaluation Center (IPEC)]. Using LiveMeeting also facilitated an interactive platform, as it allowed participants to ask questions at any time during the presentation and receive answers directly from the speakers at that time.We integrated snapshots of the VA’s PSI report into our cyberseminars so that participants could directly apply what they learned from our program because survey respondents indicated that they were interested in learning how to use the PSIs, and interview informants suggested making the education more relevant to participants.Cyberseminar session topics and content corresponded to survey respondents’ expressed needs and interests. For example, because respondents indicated that they did not understand how to interpret the PSI rates in reports, we developed two sessions substantially devoted to this . These sessions also focused on two of the three content areas that were of most interest to survey respondents (how to interpret PSI rates and specific definitions of selected PSIs). Table We also developed educational materials to complement what participants were learning in the cyberseminars: 1) an informational sheet, “Interpreting the AHRQ PSIs: A Basic Overview,” which was to be used, along with the cyberseminars, as a tool to help in interpreting and understanding the PSIs, as provided in Additional files Thirteen out of the 82 stakeholders who had registered to attend the cyberseminars responded to both the pre- and post-program surveys. In general, respondents’ answers between pre- and post-program survey questions reflected positive changes for “PSI knowledge,” although changes were somewhat negative for “PSI use.” Table Overall, survey respondents had positive perceptions of the PSI educational program. A majority of respondents agreed/strongly agreed that the sessions were useful (73%), were satisfied with what they learned (64%), and would recommend this program to their staff/colleagues (82%). Sessions 2 (“The PSIs and Your Facility’s Reports”) and 3 (“How to Interpret PSI Rates”) contributed most to respondents’ learning about the PSIs.Respondents noted several factors in the open-ended questions that contributed the most to their learning about the PSIs, including: the program’s structure  and content . Respondents noted that PSI data validity concerns were a barrier to using the PSIs for QI at their hospitals .We conducted this implementation and evaluation study with the goal of developing an educational program that was tailored to stakeholders’ needs. Ultimately, we hoped that our educational program would help VA stakeholders learn more about how to interpret and use the PSIs for QI. Our experiences in developing, implementing, and evaluating a PSI educational program yielded several valuable lessons.First, consistent with the literature , 34–36, Second, the formative evaluation enabled us to tailor the program to stakeholders’ needs and preferences. In particular, the formative evaluation influenced two key components of our educational program: the content areas for the cyberseminars and the mechanism by which the content should be delivered. Although the impact of the program was limited by a low participation rate, our post-program survey results showed that those respondents who did participate expressed positive perceptions of what they had learned from the program.Third, we learned that the timing of program implementation can be critical for success. When we began the study, we had anticipated that public reporting of VA PSI rates was imminent. However, as we moved forward with implementation, public reporting of VA PSI rates was postponed. Similar to what we found, prior studies show that hospitals’ responses to public reporting of performance measures may differ depending on hospitals’ perceptions of the need to improve on a given measure –9. PubliFourth, this study adds to the limited empirical evidence in the literature about the most useful strategies on how to educate stakeholders about the PSIs. Respondents preferred to obtain PSI education through web-based meetings . Consistent with the literature, we also learned that cyberseminars were relatively easy to implement, could be used to widely disseminate information, and appeared to be less resource-intensive than other options . Our finOur study has some limitations. As mentioned in the discussion above, although the program was developed through stakeholder input and initial interest in our PSI education program was high, actual attendance at our cyberseminar sessions was low. The response rate for our post-program survey was also low, potentially due to the timing of PSI reporting. In addition, although we intended the pre- and post-program surveys to take less than 20 min to complete, some respondents may have found the length of the surveys to be somewhat burdensome; thus, this may have impacted the response rate. Finally, while the U.S. is seeing growth in large integrated healthcare delivery systems that resemble the VA in some respects, there are, of course, organizational differences between the VA and those private sector systems.However, our study had strengths worth highlighting. A major strength of the study is that we engaged stakeholders in developing the cyberseminars within a national healthcare system through interviews and surveys. As we learned from this study, while there are challenges in developing and implementing programs within a national healthcare system, the lessons learned allowed us to gain a better understanding of how different strategies can be useful for disseminating information about quality and safety measures and helping stakeholders to effectively use these measures. Despite the small sample size at the end, the PSI educational program was developed through wider stakeholder input, and had initial interest from numerous stakeholders. In addition, although our study findings are within the context of the VA setting and are focused on the PSIs (one performance measure), we have attempted to present the lessons learned in this study in ways that facilitate their application to other health care settings, other program  development and implementation efforts, and other performance measures.Although this study focused on the AHRQ PSIs, our experiences yielded valuable lessons that others should consider when developing and implementing programs. Timing of implementation and alignment with organizational priorities are important elements for successful engagement of stakeholders in program development and implementation. However, we learned that one of the most critical elements of program development is the formative evaluation process, which we conducted through interviews and surveys. While the interviews provided us with an opportunity to engage with and get feedback from a smaller group of VA stakeholders , the pre-program survey allowed us to gather information more broadly across a range of VA stakeholders. Ultimately, through the formative evaluation process, we developed and implemented a PSI educational program that was tailored to stakeholders’ needs and interests by triangulating the information gathered from both methods.Additional file 1:Formative Evaluation: Interview Guide. This file provides the interview guide we used for the telephone interviews to obtain a general understanding of potential PSI educational needs and assess whether similar a priori concepts should inform the survey. (PDF 168 kb)Additional file 2:Formative Evaluation: Pre-program Survey. This file provides the survey that we administered to obtain stakeholders’ input on their educational needs related to the PSIs. (PDF 169 kb)Additional file 3:Post-program Evaluation Survey. This file provides the post-program survey that we administered to learn about stakeholders’ perceptions of the PSI Educational Program. (PDF 259 kb)Additional file 4:Informational Sheet, Interpreting the AHRQ PSIs: A Basic Overview. This file provides the informational sheet which PSI Educational Program participants could use to help them interpret and understand the PSIs. (PDF 87 kb)Additional file 5:PSI Educational Program Matrix. This file highlights information covered in each session of the PSI Educational Program and provides a list of materials referenced in each session. (PDF 213 kb)


Overall, participants’ affect and utility evaluations were positive for all programs, with the greatest satisfaction being reported in the tailored individual consultation and standardized group workshop conditions. Furthermore, mode of delivery and the physical presence of an expert health practitioner was more influential than the degree to which the information was tailored to the individual. Thus, the synergy in ratings between individually tailored consultations and standardized group workshops indicates that low-cost delivery health programs may be as appealing to employees as tailored, and comparatively high-cost, program options.In the design of workplace health promotion programs (WHPPs), employee perceptions represent an integral variable which is predicted to translate into rate of user engagement  and program loyalty. This study evaluated employee perceptions of three workplace health programs promoting nutritional consumption and physical activity. Programs included: (1) an individually tailored consultation with an exercise physiologist and dietitian; (2) a semi-tailored 12-week SMS health message program; and (3) a standardized group workshop delivered by an expert. Participating employees from a transport company completed program evaluation surveys rating the overall program, affect, and utility of: consultations ( Workplace health promotion programs (WHPPs) have become an increasingly popular means of promoting positive health behaviors in employees that are mutually beneficial to employers and employees ,2,3. SucScientific reviews have shown that the design of successful WHPPs is a complex process and programs are most effective when they are: (1) based on scientifically valid constructs ,20,21,22In practice, WHPPs are often designed and delivered by workplace health and safety (WH&S) professionals with limited specialized training in health intervention design and evaluation, or access to current scientific publications. As a result, practitioners tend to employ a “one-size-fits-all” approach , frequently with cost rather than engagement and outcomes being the primary driver of program selection ,24,29,37Of the research investigating participation and engagement, Crutzen et al. ,30 develGiven the important contribution of employee perceptions to the effectiveness of any WHPP, and the paucity of scientific literature that considers these constructs, further investigation is warranted to validate the measurement, application, and evaluation of employee perceptions of WHPPs. In particular, it is important to investigate whether any synergy exists between the often-selected “one-size-fits-all” standardized, cost-effective WHPP design and that of a more tailored but higher-cost approach recommended in the literature. At the time of publication, the authors were unable to identify any study that provided a comparison of employee perceptions of WHPPs categorized by the degree of program tailoring or mode of delivery, and which could be associated with the comparative cost of program implementation. Thus, the purpose of this study was to compare employee perceptions of the overall program offered, together with perceptions of program affect and utility across WHPPs that differed in their levels of tailoring and mode of delivery.(1)A fully tailored one-on-one health consultation with a registered dietitian and exercise physiologist;(2)A semi-tailored SMS health messages program with tailoring based on employees’ reported readiness to change exercise and eating behaviors; and(3)A standardized group health workshop presentation delivered by a registered dietitian and exercise physiologist.The three WHPPs included in the study were designed by a single research team with expertise in health behavior change, psychology, exercise physiology, and dietetics. All programs aimed to increase healthy lifestyle behaviors including physical activity and nutritional consumption. The programs comprised:It was hypothesized that the participant ratings of the overall program, as well as participant ratings of affect and utility, would be highest in the individual tailored condition, with a gradual decline as the rate of tailoring  diminished.A sample of 339 employees from an Australian transport company were recruited to participate in the study. Participants were aged between 19 and 64 years. Consistent with the organization’s workforce demographics, the majority of participants (62%) were male. Due to operational limitations of the project, including restricted access to employees and work time availability, the number of participants, and volume of program evaluations available, for each program varied considerably. For each program, employees who completed both the program and evaluations are described below.Nineteen employees  from three discrete worksites evaluated the tailored individual consultation delivered by a registered dietitian and exercise physiologist. In this program, participant age ranged between 25 and 64 years, with the majority (39%) in the 35 to 44 years cluster.Two hundred and thirty-four employees  evaluated the SMS health message program. Participating employees were aged between 19 and 64 years, with the majority (31%) in the 25 to 34 years group.Eighty-six employees  evaluated the group workshop program. Participants were aged between 19 and 64 years, with most (41%) in the 25 to 34 years cluster.The study employed a cross-sectional design. Employees’ perceptions of the overall program, affect, and utility for each WHPP were assessed by a combination of quantitative and qualitative items in an evaluation survey.Care Health Human Research Ethics Committee (# 2013.12.83) and informed consent was obtained from each participant. All three health programs were designed to promote positive health behaviors aimed at achieving healthy levels of physical activity and nutritional consumption. In all programs, the content was designed to increase participant’s knowledge and motivation, such that employees would be empowered to change or maintain their behavior consistent with the national guidelines [This study was approved by the by the Uniting idelines .Program trials were advertised by strategically located posters at worksites, promotional emails sent by site Managers to employee email addresses, and in-person promotion during site visits by the research team. During these visits, researchers invited all employees to voluntarily participate in a corporate health survey and trial the WHPP offered at their worksite.All three trials and evaluation surveys were completed between January and August 2016. In order to maintain the integrity of each trial and comply with the operational requirements of the organization, the specific WHPP offered differed according to geographic site location. Employees were permitted to participate in the WHPP during work hours. To control for fidelity in the delivery of the individually tailored consultation and the standardized group presentation, one expert consultant with qualifications in exercise physiology and dietetics delivered all consultations and workshops. The consultant also contributed to the refinement of the text message content for the SMS health messages program.Employees were invited to schedule an appointment with the dietitian and exercise physiologist during the trial period. Appointments ranged between 30 and 45 min in duration depending on the specific needs of the participant. The consultation was semi-structured, with the employee self-nominating the nutrition and/or physical activity issues to be discussed. Techniques used by the consultant included: motivational interviewing; goal setting; and the provision of expert nutrition and exercise planning assistance tailored to the individual’s health goals, risks, and current lifestyle behaviors.I’m already trying to achieve healthy exercise/eating habits’ or ‘I’m already achieving 30 min of moderate exercise on most days/I’m already eating healthy’, were categorized as ‘moderate to high readiness to change’. Notably, categorization of readiness to change physical activity and nutrition were assessed independently, with many participants being categorized as ‘low readiness to change’ for one behavior  and ‘moderate to high readiness to change’ for the other .Invited employees self-selected to participate in the SMS health message program. The health survey included independent measures of ‘readiness to change’ physical activity and nutritional consumption behaviors based on the readiness ruler used in motivational interviewing theory ,44. PartFitness is about being a better you. Have you challenged yourself today?’  or ‘Read the nutritional panel on your food packaging. For every 100 g, choose items with less than 3 g saturated fat and less than 400 mg sodium’ .A database of messages was developed by the research team and subjected to an extensive content validity validation process that included a modified Delphi technique with a panel of experts in the fields of health psychology, exercise physiology, and dietetics; and pilot trial with employees of a similar organization . ExampleParticipation in the trial required the provision of a mobile telephone number. Participating employees received three text messages per week for a period of 12 weeks with delivery times alternating between 9 a.m. and 4 p.m. local time, on a Monday, Wednesday and Friday.Employees were invited to participate in a 30-minute group workshop on nutrition. The presentation was delivered at five scheduled sessions across two office locations in order to maximize employee participation. The key objective of the workshop was to improve participant understanding of common food behavior patterns and develop positive short-term nutrition intake decisions for long-term health outcomes. Selected content included: the internal thought rationalization process; nutrition for optimal brain functioning; and strategies for improving concentration at work. Employees were provided with the opportunity to ask questions during the session regarding workshop content. No individual or tailored health advice was offered during the workshop sessions.The program evaluation survey was completed by willing participants immediately following the consultation and workshop programs. SMS health message program evaluation surveys were completed during a follow-up site visit at the completion of the 12-week trial.Participants’ readiness to change physical activity and nutritional consumption behaviors were measured as part of the baseline health survey on an 11-point scale, with two supplementary response options indicating active progress toward behavioral change and achievement of national physical activity and nutrition guidelines , refer tOverall, how would you rate the service?’ and was presented as an 11-point Likert scale with response options ranging from 0 (very poor) to 10 (excellent). The qualitative survey items were based on Walthouwer and colleagues’ [What did you like most about the service?’; ‘What did you like least about the service?’; and ‘Do you have any suggestions to improve the service?’.Employee perception of the overall program was assessed in the evaluation survey by a single quantitative item followed by a series of three open-response qualitative measures. The quantitative item of overall program rating asked, ‘leagues’  evaluatiMeasures of participants’ affect and utility of the trials were each assessed in the evaluation survey by eight items measured on a 5-point Likert scale ranging from one (strongly disagree) to five (strongly agree). These items shown in  were basData consisted of participants’ responses to both quantitative and qualitative items in the program evaluation survey. As a result of consistent observations of breaches of normality, all analyses of quantitative data were subjected to a bias-corrected and accelerated (BCa) bootstrap model with 1000 samples using IBM SPSS version 21 .D (19) = 0.334, p < 0.001; D (161) = 0.142, p < 0.001; D (123) = 0.215, p < 0.001, respectively. Levene’s test confirmed that variability in program ratings differed between the groups  = 25.81, p < 0.001) with variability in overall rating greatest in the semi-tailored SMS health message program. F  = 93.31, p < 0.001, η2 = 0.38. Post hoc Dunnett T3 comparisons revealed that the mean overall rating for the semi-tailored SMS health messages program was significantly lower than both the tailored individual consultation  and standardized group workshop, MD = −2.57, 95% CI = −2.95, −2.20, p < 0.001. However, no significant difference between the overall rating of the tailored individual consultation and standardized group workshop was detected, MD = 0.49, 95% CI = −0.06, 1.03, p = 0.093, n.s.Overall program ratings were high, with the greatest satisfaction being reported in the tailored individual consultation condition. A bootstrapped one-way ANOVA was performed to compare overall program ratings. The ANOVA test showed that differences in program ratings were significant, and the effect of program type on participant rating was large, A thematic analysis was conducted on the qualitative program experience question responses. The overall results of the thematic analysis are presented in liked most (n = 210) about the programs revealed that across all three programs, responses related to program content were common . For the consultation condition, responses that reflected the program tailoring were also common (31.6%) and reflected the personalized nature of the advice provided. Workshop participants also provided frequent responses related to delivery (44.2%) and overwhelmingly reflected the engaging presentation style of the presenter.Qualitative feedback on what participants n = 66) responded to the question regarding what they liked least about the program offered. Consultation participants’ responses predominantly aligned with the theme of self-awareness (80.0%) and reflected difficulty related to the process of behavior change . SMS program participant responses to this question varied considerably with the most common responses related to program content (25.9%) and tailoring (25.9%). Content theme responses reflected the generalized nature of message content that was referred to by one respondent as “common sense”; whereas tailoring theme responses reflected participants’ desire for a service more customized to their personal circumstances and health goals. Overwhelmingly, participants reported that what they liked least about the program related to delivery (85.3%), and requested the service to be provided on a regular basis and/or for a greater duration.Across all program conditions, few participants . For both programs, participants overwhelmingly requested an increase in the frequency and duration of the service. In the SMS program condition, the most common response related to the theme of program content (41.4%), and reflected participants’ desire for more detailed information and resources to supplement the information provided in the SMS messages.One hundred and one employees provided n = 8, Cronbach’s α = 0.93; n = 8, Cronbach’s α = 0.94, respectively. Furthermore, an analysis of the interrelatedness of each item to the overall scale showed that the removal of any singular item from either the affect or utility scales would not significantly improve the internal reliability of the scale.A reliability test of the affect and utility scales was conducted. Results of the reliability test revealed that the scales of affect and utility both held overall very high internal reliability, D (19) = 0.318, p < 0.001), semi-tailored (D (169) = 0.108, p < 0.001) and standardized conditions, D (140) = 0.118, p < 0.001. Levene’s test confirmed that variability in program ratings differed between the groups  = 34.09, p < 0.001) with the greatest variability shown in the semi-tailored SMS health message program rating of affect.Analyses of the affect ratings of each program revealed that participants consistently reported a strong affect in the individually tailored consultation and standardized group workshop trials. By contrast, participants in the semi-tailored SMS health messages program reported a positive, but slightly lower, affect. A Kolmogrov–Smirnov distribution analysis revealed that scores were significantly non-normal across the tailored  = 54.96, p < 0.001, η2 = 0.24. Post hoc Dunnett T3 comparisons identified significant differences between all three program trials. Specifically, the mean affect rating for tailored individual consultations was significantly higher than both the semi-tailored SMS program  and standardized group workshop, MD = 0.39, 95% CI = 0.18, 0.85, p = 0.002. Similarly, the mean affect rating was significantly greater for the standardized workshops when compared with the semi-tailored SMS program, MD = 0.79, 95% CI = 0.62, 0.93, p < 0.001.A bootstrapped one-way ANOVA of affect rating revealed significant differences, with program trials having a moderately large effect on perceived affect, D (19) = 0.246, p = 0.004), semi-tailored (D (169) = 0.122, p < 0.001) and standardized trial conditions, D (140) = 0.129, p < 0.001. Levene’s test confirmed that variability in program ratings differed between the groups  = 42.43, p < 0.001), with greatest variability of results observed in the semi-tailored SMS health message program trial.Overall, participants’ program utility ratings were high in the tailored individual consultation and standardized group workshop trials. Participants in the semi-tailored SMS health messages trial rated the program as marginally lower for utility. A Kolmogrov–Smirnov distribution analysis revealed that scores were significantly non-normal across the tailored  = 50.98, p < 0.001, η2 = 0.24. Post hoc Dunnett T3 comparisons showed that ratings of utility were significantly lower for the semi-tailored SMS health messages program when compared with both the individually tailored consultation  and standardized group workshop, MD = −0.84, 95% CI = −1.01, −0.68, p < 0.001. No significant difference was observed in the utility rating between the tailored individual consultation and standardized group workshop conditions, MD = 0.26, 95% CI = 0.39, 0.46, p = 0.084, n.s.A bootstrapped one-way ANOVA of utility rating revealed significant differences, with program trial having a moderately large effect on perceived program utility, The current study both confirms and extends upon contemporary literature highlighting the importance of employee perceptions in designing and evaluating WHPPs. Specifically, this study was the first to apply the evaluation constructs of affect and utility, as described by Crutzen et al. ,30, togeContrary to our prediction, all three programs received moderate-to-high overall ratings which may suggest that employees value efforts by their employer to support worker health and provide WHPP services in general. Notably, higher levels of satisfaction were reported for the WHPPs that offered the tailored individual consultation and standardized workshop conditions when compared to the semi-tailored SMS trial, suggesting that the physical presence of a highly regarded and engaging expert presenter may have positively influenced overall program ratings more than the degree of tailoring provided within the program.These hypotheses are further supported by the qualitative feedback in which the presentation style and manner of the exercise physiologist and dietitian was overwhelmingly reported as a positive aspect of the program by participants in the consultation and workshop trials. Similarly, SMS program participants were least satisfied with the content and tailoring of the program, despite the researcher’s efforts to match the level of messaging to their reported readiness to change health behaviors. This feedback may also reflect the unidirectional nature of SMS messaging and absence of the physical presence of an expert with whom participants could build a rapport. More generally, this finding may also suggest that personal communication, regardless of whether it be delivered via a private or a group setting, is more highly valued by employees in WHPPs than programs delivered through a virtual platform . liked most about the service. This reflects previous research which found that employees value WHPPs that promote health education and disseminate relevant information more highly than programs that approach health behavior change via simple give-aways, such as the provision of free fruit in lunch rooms, which may be perceived to be tokenistic [For all three trials, participants most frequently reported program content as what they kenistic .Within the workshop trial condition, employees initially reported 30 minutes as an appropriate session duration that could be completed during their lunch break. Many attendees later requested that longer sessions be offered on an ongoing basis due to the reported perception that additional information would have been appreciated. However, longer workshop session durations may limit participation by those who are unable to extend their lunch break due to work commitments. These requests for longer and ongoing support by participants is supported by the literature suggesting that the frequent contact improved participant outcomes and sustained health-promoting behaviors . SimilarMeasures of affect comprise the emotional responses and reactions of program participants and reflect how individuals feel about aspects of the program . On average, overall ratings of affect were high across all three programs. According to Crutzen et al. , such hiRatings of program utility represent cognitive responses to the program . In the present study, the trends observed for measures of affect were replicated in the utility data and are also incongruent with the study hypothesis. In particular, the average overall affect ratings were high across all three trials, which has been associated with elevated user loyalty to the program . OverallThis study provides practical contributions to the applied workplace health promotion literature. Based on the findings herein, the employees’ perception of any offered WHPP represents an integral variable which is predicted to translate into observable differences in rate of user engagement  and program loyalty. Therefore, the practice of WHPP development should be refined to include consideration of employee perceptions and preferences. Furthermore, the findings of this study are incongruent with the current scientific literature whereby, unlike the findings of previous studies, program tailoring was not a primary variable in determining the success of the WHPP in terms of user engagement and participation. However, it should be noted that the success of voluntary WHPPs is highly dependent upon rates of participation, and therefore employee perceptions. In this study it was observed that employee perceptions of the overall program, affect, and utility were of a similarly high level for participants in both the tailored and standardized study conditions, with significantly lower ratings in the semi-tailored condition. As the degree of tailoring typically influences program administration costs, this finding indicates that tailoring may play a less important role in attracting participation than previously reported. That is, standardized, more cost-effective programs such as standardized group workshops can be as attractive to employees as the more expensive tailored consultations, as long as the content of the offered WHPP is perceived as favourable by employees. Moreover, qualitative feedback from participating employees implied that mode of delivery, and in particular the physical presence of an expert health practitioner with whom they could build a rapport, was more important to employees than the degree to which the information provided was tailored to the individual. This result was demonstrated regardless of the extent to which the provided contact was individual- or group-based, and further supports the findings that program mode of delivery may also have a greater influence on user engagement and loyalty than program tailoring.There are a number of limitations to the current study that require comment, including: the cross-sectional design; the low response rate for the evaluation survey; sampling bias; replication of the consultation and workshop programs; and differences in program intensity. Firstly, while the cross-sectional design employed in the current study is an important first step toward understanding employee perceptions of different WHPPs, longitudinal studies are needed to investigate whether positive affect and utility ratings translate into participant engagement and loyalty, as predicted. Secondly, the low response rate for evaluation surveys, particularly in the tailored individual consultation condition, represents a major limitation to the current study. Arguably, this reflects the difficulty of obtaining feedback from employees following the conclusion of the free service, primarily due to time and operational constraints. However, regardless of the reason for this low rate of response, such an outcome reduces confidence in the accuracy of any follow-up findings. Thirdly, the recruitment strategy of seeking voluntary participation in WHPPs may have contributed to a selection bias whereby employees self-selecting into the programs disproportionally represent those who are already engaging in healthy lifestyle behaviors rather than those who require the most assistance with health behavior change but are least likely to volunteer for participation. Fourthly, the consultation and workshop programs received extensive positive feedback about the presenter and presentation style, this relates to the individual characteristics of the presenter in this case and evaluation results may vary where a different presenter was employed in a replication study. Finally, it is imperative to acknowledge that this study involved the comparison of WHPPs that differed in intensity . This notable difference in duration of engagement may have further influenced employee perceptions of the program to an extent greater than that provided in the evaluation survey results. Overall, the limitations of this study are consistent with other applied research within the workplace health promotion discipline. However, the authors acknowledge that the project may have been improved by quantifying: (1) the number of employees who declined to participate; (2) WHPP attrition rates; and (3) the rate of conversion from program completion to evaluation.Based on the findings in this study, it is recommended that future research investigate how to improve employee perceptions of WHPPs, particularly for engaging with those least likely to volunteer for participation. Such studies may include the measurement of what motivates employees to participate or which aspects of the program are most appealing to potential participants. Following this, employee feedback and preferences should be investigated and applied in the design and/or refinement of organization-specific WHPPs and evaluated longitudinally to confirm the previous findings that positive affect and utility ratings lead to increased participation  and loyaThis study compared employee perceptions of overall program rating, affect and utility across three WHPPs. Each offered program had differing levels of tailoring and mode of delivery. Overall, a trend was observed in the employee perceptions which revealed that participant ratings of the overall program, affect, and utility were similarly higher for the tailored individual consultation and standardized workshop condition when compared to the semi-tailored SMS trial. These findings suggest that the physical presence of a highly regarded expert may positively influence the overall program ratings of a standardised WHPP. Based on previous findings, higher ratings of participant perception are predicted to lead to higher levels of user engagement and loyalty within a WHPP. The synergy in ratings between the individually tailored consultation and standardized group workshop indicates that low-cost delivery health programs may, if delivered by a suitable expert, potentially be as appealing and beneficial to employees as comparatively high-cost options. This study was the first to apply the evaluation constructs of affect and utility to workplace health programs and highlights the importance of considering employee preferences in the design of such programs to ensure maximum effectiveness, including employee participation and loyalty.