Practice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness
Health administrative data provide enormous opportunities for health care organizations and systems to incorporate practice feedback as part of regular and sustainable quality improvement initiatives. However, despite decades of experience, efforts to improve practice with such feedback have stagnated (1) The latest Cochrane review (2) saw a positive but variable effect on performance across 140 randomized trials of feedback interventions (the modest effect size became stable in 2003 after only 30 trials) (1) Knowledge distributed across disciplines exists to inform more effective interventions, but these lessons have not been assembled and organized concisely for quality improvement professionals, information technology experts, educators, and others seeking to provide effective feedback. Over the past 10 years, our group has studied how to design better feedback interventions through systematic reviews (24) randomized, controlled trials (58) an international meeting of practice feedback developers and researchers (9) studies of the theory underlying feedback and behavior change (3, 10) and recent interviews of 28 experts in feedback theory from psychology (social, health, cognitive, and organizational) behavioral science, economics, management, and other related disciplines. Through discussions among members of the study team and by tapping its collective experience, we identified 15 suggestions that are likely to improve the effectiveness of feedback across a range of contexts and that are underutilized in the literature (2, 3, 1113) (Table) Examples, potential underlying mechanisms, and relevant citations are drawn from the interviews and the literature. Our discussion is limited to provision of feedback intended to encourage best practice in a specific clinical area (for example, reduce ordering of tests) rather than broader approaches (such as practice wide feedback and clinical decision support systems) Table. 15 Suggestions for Designers of Practice Feedback and Examples of Implementation Strategies In this article, we use the word suggestions to acknowledge the incomplete state of the literature, given that their specific mechanisms of effectiveness have seldom been explored in detail. We intend for this work to serve both as initial guidance and a call for more detailed study. The relative importance and feasibility of these suggestions for any specific context should be determined through systematic design, pilot testing, and assessment of practice change barriers and drivers (14, 15) This study was reviewed by the Ottawa Health Science Network Research Ethics Board and funded by the Canadian Institutes of Health Research. The funding source had no role in the design, conduct, or reporting of the study. 15 Suggestions for Designing Practice Feedback Interventions Nature of the Desired Action 1. Recommend actions that are consistent with established goals and priorities. Feedback that supports actions that are consistent with established goals and priorities is more likely to be effective (2, 16, 17) Considerable theoretical work has outlined various mechanisms related to how goals encourage behavior change, including facilitating priority setting, directing attention and effort, and establishing intention or commitment (16, 17) Intention, in turn, is the most important general predictor of behavior (18, 19) Goals that are explicit, specific, time bound, recipient defined, and challenging but attainable are more likely to engage these mechanisms (2, 16, 17) Intervention designers should ensure that feedback is consistent with recipients' goals and priorities. 2. Recommend actions that can improve and are under the recipient's control. Feedback should recommend actions that have room for improvement (2, 9) and over which the recipient has control (18, 20) Controllable actions may differ among recipients. For example, providing feedback about a hospital's overall performance may not be useful for individual physicians, who probably control only their immediate practice. However, such feedback may be actionable for hospital executives. Designers should consider whether feedback suggests actions that recipients can take. 3. Recommend specific actions. Feedback that recommends specific rather than general actions is more likely to be effective (21) For example, feedback that preventive care tests have been missed implies that the physician should develop strategies to avoid such lapses in the future, but offers no guidance about which strategies will be effective. In contrast, specific corrective actions, such as providing patient names, allowing immediate reexamination of case files, or providing reminders, are more likely to encourage practice change (7) In many cases, facilitating the development of specific implementation plans in which if X happens, I will do Y strengthens the link between intention and actual practice (2, 21) Designers should enable specific actions of feedback recipients. Nature of the Data Available for Feedback 4. Provide multiple instances of feedback. Feedback provided on several occasions is generally more effective than presenting it once (2) Multiple instances encourage a feedback loop (22) wherein the recipient can receive the initial feedback, make a change in the practice, and see whether the change has been effective (23, 24) They may improve memory for and attention to the feedback among recipients (25) They may also help with sustainability of the desired practices by allowing regular monitoring of useful outcomes (26) Designers should implement interventions that involve multiple instances of feedback. 5. Provide feedback as soon as possible and at a frequency informed by the number of new patient cases. Few studies inform the optimal intervals to provide feedback in health care settings (27, 28) Overly frequent feedback has been argued to be less effective due to increased cognitive load, alert fatigue (29) and discounting (30) However, infrequent feedback about common procedures may also be less effectivethe time lag may allow the recipients to discount the feedback as no longer relevant to their changing practice or may lead them to forget the lessons that it provided (31) The education literature has examined the relative merits of immediate versus delayed feedback (11) but has primarily focused on delays of seconds or minutes rather than the weeks or months that are common in practice. One practice based study (32) showed that immediate reminders were more effective than monthly feedback reports in terms of internal medicine specialists' adherence to preventive care protocols. In the absence of empirical work comparing feedback intervals in real world settings, designers should consider whether shorter intervals are more useful and whether the number of patient cases helps determine the frequency of feedback. 6. Provide individual rather than general data. Evidence from psychology shows that feedback data that are specific to an individual recipient are usually more effective than those that apply to a group (13, 28, 33) Although practice evidence is scant, our experts believed that feedback about one's own practice is usually more useful than feedback about one's team or unit, because group level feedback can be more easily discounted and person level data may be applied more readily to immediate corrective actions (such as reviewing patient charts and reexamining decision making) For similar reasons, feedback across small groups (such as units) may be preferable over larger groups (such as geographic regions) Even more specific feedback at the patient level can facilitate corrective actions in some cases (32) Designers should consider the specificity of available data and prioritize more specific rather than less specific feedback. 7. Choose comparators that reinforce desired behavior change. Although feedback without an explicit comparison is feasible (13, 34) practice feedback is most often given in the context of a comparator or benchmark. The comparator may be drawn from recipient performance (that is, how performance changes over time) formal guidance (that is, guideline recommended target rates) or a peer group (that is, mean performance of similar persons or organizations) Although relatively little evidence informs health feedback designers about which comparators should be chosen under which circumstances (27) using several comparators can create mixed messages for recipients. For example, if a physician's percentage of patients with diabetes receiving foot examinations has improved over time but is lower than the top 10% of practices, a summary message suggesting that improvement is needed might be inconsistent with the physician's interpretation that My numbers are improving, so I don't need to change. Without better evidence to inform comparator choice, designers should consider choosing comparators strategically with a preference for simple, clear comparisons that reinforce the desired behavior change. Feedback Display 8. Closely link the visual display and summary message. Feedback should include a verbal summary message (12) and can often be effectively supported by visual or graphical elements. Feedback is more effective if the summary message and visual display are linked both conceptually and visually. If a summary message indicates that the recipient's current practice is below a target rate but the visual display shows performance above some benchmark, the effectiveness of the feedback may be compromised, leading the recipient to discount, misinterpret, or ignore it. Placing summary messages and graphics on separate pages may also compromise effectiveness. Designers should link the displayed data and summary message through color, spatial proximity, or other common visual grouping techniques (35, 36) 9. Provide feedback in more than 1 way. Robust evidence suggests that feedback is more likely to be effec
Annals of Internal Medicine