Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> OBJECTIVE ::: To assess the effectiveness of housestaff physician reporting as a method for identifying adverse events on a medical service and to compare the physician reporting mechanism with a retrospective record review mechanism. ::: ::: ::: SETTING ::: Medical service of an urban, university-affiliated teaching hospital. ::: ::: ::: DESIGN ::: Concurrent physician reporting mechanism using the hospital electronic mail system compared with a retrospective record review using a screening mechanism followed by structured, implicit physician review of the record. ::: ::: ::: PATIENTS ::: All 3146 admissions to the medical service from 13 November 1990 to 14 March 1991. ::: ::: ::: RESULTS ::: The housestaff physician reporting method identified nearly the same number (89) of adverse events as did the record review (85). However, the two methods identified only 41 of the same patients (kappa = 0.52). No statistically significant clinical or socioeconomic differences occurred between the patients identified as having had an adverse event, using the two reporting methods (physician versus record review). The housestaff did report statistically more preventable adverse events (62.5% compared with 32%; P = 0.003). The physician reporting mechanism was also less costly (approximately $15,000 compared with $54,000). ::: ::: ::: CONCLUSION ::: An adverse event identification strategy based on physician self-referral uncovers as many adverse events as does a record review and is less costly. In addition, physician-identified events are more likely to be preventable and, thus, are targets for quality improvement. <s> BIB001 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Article-at-a-Glance Objectives The objectives of this study were 1) to determine the frequency with which adverse drug events result in an incident report (IR) in hospitalized patients; and 2) to determine if there were differences between quality assurance administrators, nurse leaders in quality assurance, and staff nurses as to whether an incident report should or would be filed for each adverse drug event. Study design All patients admitted to five patient care units (one medical intensive care unit, two surgical intensive care units, and two medical general care units) in one academic tertiary care hospital were studied between February and July 1993. The main outcome measures used were adverse drug events (ADEs) and IRs. Consensus voting was used by senior hospital administrators, nursing leaders, and staff nurses to determine whether an adverse drug event should have been reported and would have been reported. Results Of 54 adverse drug events identified by the study, only 3 patients (6%) had a corresponding incident report submitted to the hospital's quality assurance program or called into the pharmacy hotline. One additional ADE was identified by an IR, but not by the ADE study. Of the 55 ADEs, 15 were preventable, and 26 were serious or life-threatening, yet only 2 of the 26 led to an incident report. The three voting groups agreed that most ADEs justified an IR, but judged that in actual practice, an IR would infrequently have been filed. Conclusions Voluntary reporting identified only a small fraction of ADEs. Using IRs for quality assurance/quality improvement will lead to significant bias when assessing quality of care. <s> BIB002 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> OBJECTIVES ::: To assess incidence and preventability of adverse drug events (ADEs) and potential ADEs. To analyze preventable events to develop prevention strategies. ::: ::: ::: DESIGN ::: Prospective cohort study. ::: ::: ::: PARTICIPANTS ::: All 4031 adult admissions to a stratified random sample of 11 medical and surgical units in two tertiary care hospitals over a 6-month period. Units included two medical and three surgical intensive care units and four medical and two surgical general care units. ::: ::: ::: MAIN OUTCOME MEASURES ::: Adverse drug events and potential ADEs. ::: ::: ::: METHODS ::: Incidents were detected by stimulated self-report by nurses and pharmacists and by daily review of all charts by nurse investigators. Incidents were subsequently classified by two independent reviewers as to whether they represented ADEs or potential ADEs and as to severity and preventability. ::: ::: ::: RESULTS ::: Over 6 months, 247 ADEs and 194 potential ADEs were identified. Extrapolated event rates were 6.5 ADEs and 5.5 potential ADEs per 100 nonobstetrical admissions, for mean numbers per hospital per year of approximately 1900 ADEs and 1600 potential ADEs. Of all ADEs, 1% were fatal (none preventable), 12% life-threatening, 30% serious, and 57% significant. Twenty-eight percent were judged preventable. Of the life-threatening and serious ADEs, 42% were preventable, compared with 18% of significant ADEs. Errors resulting in preventable ADEs occurred most often at the stages of ordering (56%) and administration (34%); transcription (6%) and dispensing errors (4%) were less common. Errors were much more likely to be intercepted if the error occurred earlier in the process: 48% at the ordering stage vs 0% at the administration stage. ::: ::: ::: CONCLUSION ::: Adverse drug events were common and often preventable; serious ADEs were more likely to be preventable. Most resulted from errors at the ordering stage, but many also occurred at the administration stage. Prevention strategies should target both stages of the drug delivery process. <s> BIB003 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Background: Adverse drug events (ADEs) are both common and costly. Most hospitals identify ADEs using spontaneous reporting, but this approach lacks sensitivity; chart review identifies more events but is expensive. Computer-based approaches to ADE identification appear promising, but they have not been directly compared with chart review and they are not widely used. ::: ::: Objectives: To develop a computer-based ADE monitor, and to compare the rate and type of ADEs found with the monitor with those discovered by chart review and by stimulated voluntary report. ::: ::: Design: Prospective cohort study in one tertiary-care hospital. ::: ::: Participants: All patients admitted to nine medical and surgical units in a tertiary-care hospital over an eight-month period. ::: ::: Main Outcome Measure: Adverse drug events identified by the computer-based monitor, by chart review, and by stimulated voluntary report. ::: ::: Methods: A computer-based monitoring program identified alerts, which were situations suggesting that an ADE might be present (e.g., an order for an antidote such as naloxone). A trained reviewer then examined patients' hospital records to determine whether an ADE had occurred. The results of the computer-based monitoring strategy were compared with two other ADE detection strategies: intensive chart review and stimulated voluntary report by nurses and pharmacists. The monitor and the chart review strategies were independent, and the reviewers were blinded. ::: ::: Results: The computer monitoring strategy identified 2,620 alerts, of which 275 were determined to be ADEs. The chart review found 398 ADEs, whereas voluntary report detected 23. Of the 617 ADEs detected by at least one method, 76 ADEs were detected by both computer monitor and chart review. The computer monitor identified 45 percent; chart review, 65 percent; and voluntary report, 4 percent. The ADEs identified by computer monitor were more likely to be classified as “severe” than were those identified by chart review (51 versus 42 percent, p =.04). The positive predictive value of computer-generated alerts was 16 percent during the first eight weeks of the study; rule modifications increased this to 23 percent in the final eight weeks. The computer strategy required 11 person-hours per week to execute, whereas chart review required 55 person-hours per week and voluntary report strategy required 5. ::: ::: Conclusions: The computer-based monitor identified fewer ADEs than did chart review but many more ADEs than did stimulated voluntary report. The overlap among the ADEs identified using different methods was small, suggesting that the incidence of ADEs may be higher than previously reported and that different detection methods capture different events. The computer-based monitoring system represents an efficient approach for measuring ADE frequency and gauging the effectiveness of ADE prevention programs. <s> BIB004 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Reducing mishaps from medical management is central to efforts to improve quality and lower costs in health care. Nearly 100 000 patients are estimated to die preventable deaths annually in hospitals in the United States, with many more incurring injuries at an annual cost of $9 billion. Underreporting of adverse events is estimated to range from 50%-96% annually.1–3 This annual toll exceeds the combined number of deaths and injuries from motor and air crashes, suicides, falls, poisonings, and drownings.4 Many stakeholders in health care have begun to work together to resolve the moral, scientific, legal, and practical dilemmas of medical mishaps. To achieve this goal, an environment fostering a rich reporting culture must be created to capture accurate and detailed data about nuances of care. ::: ::: Outcomes in complex work depend on the integration of individual, team, technical, and organisational factors. 5 6 A continuum of cascade effects exists from apparently trivial incidents to near misses and full blown adverse events. 7 8 Consequently, the same patterns of causes of failure and their relations precede both adverse events and near misses. Only the presence or absence of recovery mechanisms determines the actual outcome.9 The National Research Council defines a safety “incident” as an event that, under slightly different circumstances, could have been an accident.10 Focusing on data for near misses may add noticeably more value to quality improvement than a sole focus on adverse events. ::: ::: Schemes for reporting near misses, “close calls,” or sentinel (“warning”) events have been institutionalised in aviation,w1 w2 nuclear power technology,w3 w4 petrochemical processing, steelw5 production,w6 military operations, and air transportation.w7-w11 In health care, efforts are now being made to create incident reporting systems for medical near misses 8 11–15 to supplement the limited … <s> BIB005 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Pilots and doctors operate in complex environments where teams interact with technology. In both domains, risk varies from low to high with threats coming from a variety of sources in the environment. Safety is paramount for both professions, but cost issues can influence the commitment of resources for safety efforts. Aircraft accidents are infrequent, highly visible, and often involve massive loss of life, resulting in exhaustive investigation into causal factors, public reports, and remedial action. Research by the National Aeronautics and Space Administration into aviation accidents has found that 70% involve human error.1 ::: ::: In contrast, medical adverse events happen to individual patients and seldom receive national publicity. More importantly, there is no standardised method of investigation, documentation, and dissemination. The US Institute of Medicine estimates that each year between 44 000 and 98 000 people die as a result of medical errors. When error is suspected, litigation and new regulations are threats in both medicine and aviation. ::: ::: #### Summary points ::: ::: In aviation, accidents are usually highly visible, and as a result aviation has developed standardised methods of investigating, documenting, and disseminating errors and their lessons ::: ::: Although operating theatres are not cockpits, medicine could learn from aviation ::: ::: Observation of flights in operation has identified failures of compliance, communication, procedures, proficiency, and decision making in contributing to errors ::: ::: Surveys in operating theatres have confirmed that pilots and doctors have common interpersonal problem areas and similarities in professional culture ::: ::: Accepting the inevitability of error and the importance of reliable data on error and its management will allow systematic efforts to reduce the frequency and severity of adverse events ::: ::: Error results from physiological and psychological limitations of humans.2 Causes of error include fatigue, workload, and fear as well as cognitive overload, poor interpersonal communications, imperfect information processing, and flawed decision making.3 In both aviation … <s> BIB006 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Errors in medicine are a major cause of harm to patients. Though there is little controversy among clinicians about the importance of accurate and reliable clinical data and the imperative of correct diagnosis, that commitment to exactitude dissolves when errors happen. Then, clinicians and managers may behave in a way that limits investigation. We often use the subjectivity and complexity of medicine to rationalise and justify error. ::: ::: Many factors explain this reluctance to investigate and to accept error. If we are to design effective systems to prevent errors from affecting patients, we must understand these factors. The following case study illustrates the concerns, fears, and practical problems that we faced in conducting an evaluation of misinterpreted prostate biopsies. ::: ::: #### Summary points ::: ::: Detection and prevention of errors are obvious goals for any organisation ::: ::: In reality, however, medicine's approach to error has been limited and inadequate ::: ::: Motivational factors, both real and perceived, that influence how errors in medicine are handled must be identified, discussed, and changed if the “patient safety movement” isto succeed ::: ::: Our experience with misread prostate biopsies illustrates the concerns, fears, and practical problems encountered when dealing with the discovery of medical errors ::: ::: In February 1999 a urologist at the Sturdy Memorial Hospital in Attleboro, Massachusetts, requested a retrospective review of a 1996 biopsy result because of the patient's clinical course and the results of a biopsy in 1999. The review revealed that the 1996 report was incorrect. The urologist and pathologist (neither of whom was responsible for the 1996 reading) implemented appropriate management for the affected patient. ::: ::: When they discovered a second misread prostate biopsy from the same period the urologist and pathologist became concerned that the frequency of these errors was higher than “expected.” Fears about malpractice suits and damaged reputations emerged. The potential ofa bigger problem meant risk to more … <s> BIB007 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Leading patient safety proposals promote the design and implementation of error prevention strategies that target systems used to deliver care and eschew individual blame. They also call for candor among practitioners about the causes and consequences of medical injury. Both goals collide with fundamental tenets of the medical malpractice system. Thus, the challenge of addressing error in medicine demands a thorough reconsideration of the legal mechanisms currently used to deal with harms in health care. In this article, we describe an alternative to litigation that does not predicate compensation on proof of practitioner fault, suggest how it might be operationalized, and argue that there is a pressing need to test its promise. We tackle traditional criticisms of "no-fault" compensation systems for medical injury-specifically, concerns about their cost and the presumption that eliminating liability will dilute incentives to deliver high-quality care. Our recent empirical work suggests that a model designed around avoidable or preventable injuries, as opposed to negligent ones, would not exceed the costs of current malpractice systems in the United States. Implementation of such a model promises to promote quality by harmonizing injury compensation with patient safety objectives, especially if it is linked to reforms that make institutions, rather than individuals, primarily answerable for injuries. <s> BIB008 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Abstract Intensive care is one of the largest and most expensive components of American health care. Studies suggest that errors and resulting adverse events are common in intensive care units (ICUs). The incidence may be as high as 2 errors per patient per day; 1 in 5 ICU patients may sustain a serious adverse event, and virtually all are exposed to serious risk for harm. Theories of error developed in aviation and other high-risk industries suggest that errors are likely to occur in all complex systems. Reporting of incidents, including both adverse events and near misses, is an essential component for improving safety. Voluntary, confidential reporting is likely to be more important than mandatory reporting. There have been a few efforts to apply such systems in medicine. In intensive care, the Australian Incident Monitoring System (AIMS)-ICU has been the most prominent.We have designed a Web-based ICU Safety Reporting System (ICUSRS). The goal is to identify high-risk situations and working conditions, to help change systems, and reduce the risk for error. The analysis and feedback of reports will inform the design of interventions to improve patient safety. The effort is aided substantially by collaboration with the 30 participating ICUs and important stakeholders including the Society of Critical Care Medicine, the American Society for Health-care Risk Management, the Food and Drug Administration Center for Devices and Radiological Health, the Foundation for Accountability, and the Leapfrog Group. A demonstration and evaluation of the system is underway, funded by the Agency for Healthcare Re-search and Quality. Copyright 2002, Elsevier Science (USA). All rights reserved. <s> BIB009 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Incident reporting <s> Systems for reporting adverse events can reduce medical errors by uncovering remediable problems in processes of care; however, current reporting systems are neither widely used nor highly effective. Reporting systems work best when they are confidential and easy to use, provide expert analysis of reports, and give timely feedback. <s> BIB010
A cornerstone of safety initiatives in other industries has been the reporting and analysis of errors BIB005 . The commercial aviation industry has been a pioneer in this area designing non-punitive, blame-free reporting systems BIB006  . Other industries, such as nuclear power BIB005 , have also follow this lead and established reporting systems, yet despite the successes of voluntary error reporting systems in other industries, voluntary reporting or incident reports in healthcare has been relatively unsuccessful. By incident reporting, we specifically mean the voluntary reporting of a medical event by a health care provider. Incident reports have traditionally been paper-based forms, however some online reporting systems exist BIB009 . Studies that have compared the rate of adverse event detection through incident reports to those detected by chart review have found that only 1.5% of adverse events BIB001 and only 6% of adverse drug events BIB004 are detected through incident reports. Nonetheless, incident reports still remain the mechanism that most institutions use to detect adverse events. There are several important reasons why incident reporting in its current state may not be the most effective way to detect adverse events BIB007 BIB010 , including interruption in workflow, perception that completing a form will not result any improvement, lack of knowledge that an adverse event had occurred, and fear of exposing oneself to litigation BIB010 BIB002 BIB008  . Incident reports remain an attractive source of information because they are generally readily available, it is often possible to obtain detailed descriptions of what went wrong, truly catastrophic errors or events can sometimes be found which can be used to make changes, and they represent the best source of potential adverse events, or near misses BIB003 . However, problems with underreporting greatly limit their utility for patient safety research.