Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> Types of medication errors are defined, error detection techniques are described, and the validity of several medication error studies is evaluated. A medication error is generally defined as a deviation from the physician's medication order as written on the patient's chart. In hospitals, medication errors occur at a rate of about one per patient per day. A dispensing error is one made by pharmacy staff when distributing medications to nursing units or directly to patients in an ambulatory-care pharmacy; the error rates for doses dispensed via the cart-filling process range from 0.87% to 2.9%. Categories of medication errors should be operationally defined before an investigation, and any allowable deviations from the physician's order should be clearly stated. Fourteen error category definitions are presented. Methods for detecting medication errors include anonymous self-reports (questionnaires), incident reports, the critical-incident technique (analyses of a large number of individual errors to identify common causal factors), and direct observation (including the disguised-observation and participant observer techniques). Observation is the best error detection method in terms of accuracy. Results of medication error studies were examined for validity and classified into one of four categories: (A) results should be accepted as reported, (B) results overestimate or underestimate the truth by a known amount, (C) results overestimate the truth by an unknown amount, and (D) results should not be accepted. All studies examined for validity used observation as the error detection technique. The following guidelines for observation-based medication error studies were established: The observer should follow the subject to the patient's bedside, the observer should witness patient consumption of each dose, the observer should not be familiar with patient drug regimens before observation, operational definitions must be used, and having an error validation committee can be advantageous. Future studies are needed that focus on the identification and testing of new error prevention methods that use the techniques described. <s> BIB001 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> Background: Adverse drug events (ADEs) are both common and costly. Most hospitals identify ADEs using spontaneous reporting, but this approach lacks sensitivity; chart review identifies more events but is expensive. Computer-based approaches to ADE identification appear promising, but they have not been directly compared with chart review and they are not widely used. ::: ::: Objectives: To develop a computer-based ADE monitor, and to compare the rate and type of ADEs found with the monitor with those discovered by chart review and by stimulated voluntary report. ::: ::: Design: Prospective cohort study in one tertiary-care hospital. ::: ::: Participants: All patients admitted to nine medical and surgical units in a tertiary-care hospital over an eight-month period. ::: ::: Main Outcome Measure: Adverse drug events identified by the computer-based monitor, by chart review, and by stimulated voluntary report. ::: ::: Methods: A computer-based monitoring program identified alerts, which were situations suggesting that an ADE might be present (e.g., an order for an antidote such as naloxone). A trained reviewer then examined patients' hospital records to determine whether an ADE had occurred. The results of the computer-based monitoring strategy were compared with two other ADE detection strategies: intensive chart review and stimulated voluntary report by nurses and pharmacists. The monitor and the chart review strategies were independent, and the reviewers were blinded. ::: ::: Results: The computer monitoring strategy identified 2,620 alerts, of which 275 were determined to be ADEs. The chart review found 398 ADEs, whereas voluntary report detected 23. Of the 617 ADEs detected by at least one method, 76 ADEs were detected by both computer monitor and chart review. The computer monitor identified 45 percent; chart review, 65 percent; and voluntary report, 4 percent. The ADEs identified by computer monitor were more likely to be classified as “severe” than were those identified by chart review (51 versus 42 percent, p =.04). The positive predictive value of computer-generated alerts was 16 percent during the first eight weeks of the study; rule modifications increased this to 23 percent in the final eight weeks. The computer strategy required 11 person-hours per week to execute, whereas chart review required 55 person-hours per week and voluntary report strategy required 5. ::: ::: Conclusions: The computer-based monitor identified fewer ADEs than did chart review but many more ADEs than did stimulated voluntary report. The overlap among the ADEs identified using different methods was small, suggesting that the incidence of ADEs may be higher than previously reported and that different detection methods capture different events. The computer-based monitoring system represents an efficient approach for measuring ADE frequency and gauging the effectiveness of ADE prevention programs. <s> BIB002 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> OBJECTIVE: Outpatient drug complications have not been well studied. We sought to assess the incidence and characteristics of outpatient drug complications, identify their clinical and nonclinical correlates, and evaluate their impact on patient satisfaction.DESIGN: Retrospective chart reviews and patient surveys.SETTING: Eleven Boston-area ambulatory clinics.PATIENTS: We randomly selected 2,248 outpatients, 20 to 75 years old.MEASUREMENTS AND MAIN RESULTS: Among 2,248 patients reporting prescription drug use, 394 (18%) reported a drug complication. In contrast, chart review revealed an adverse drug event in only 64 patients (3%). In univariate analyses, significant correlates of patient-reported drug complications were number of medical problems, number of medications, renal disease, failure to explain side effects before treatment, lower medication compliance, and primary language other than English or Spanish. In multivariate analysis, independent correlates were number of medical problems (odds ratio [OR] 1.17; 95% confidence interval [95% CI] 1.05 to 1.30), failure to explain side effects (OR 1.65; 95% CI, 1.16 to 2.35), and primary language other than English or Spanish (OR 1.40; 95% CI, 1.01 to 1.95). Patient satisfaction was lower among patients who reported drug complications (P < .0001). In addition, 48% of those reporting drug complications sought medical attention and 49% experienced worry or discomfort. On chart review, 3 (5%) of the patients with an adverse drug event required hospitalization and 8 (13%) had a documented previous reaction to the causative drug.CONCLUSIONS: Drug complications in the ambulatory setting were common, although most were not documented in the medical record. These complications increased use of the medical system and correlated with dissatisfaction with care. Our results indicate a need for better communication about potential side effects of medications, especially for patients with multiple medical problems. <s> BIB003 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> Background: Increasing data suggest that error in medicine is frequent and results in substantial harm. The recent Institute of Medicine report (LT Kohn, JM Corrigan, MS Donaldson, eds: To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press, 1999) described the magnitude of the problem, and the public interest in this issue, which was already large, has grown. ::: ::: Goal: The goal of this white paper is to describe how the frequency and consequences of errors in medical care can be reduced (although in some instances they are potentiated) by the use of information technology in the provision of care, and to make general and specific recommendations regarding error reduction through the use of information technology. ::: ::: Results: General recommendations are to implement clinical decision support judiciously; to consider consequent actions when designing systems; to test existing systems to ensure they actually catch errors that injure patients; to promote adoption of standards for data and systems; to develop systems that communicate with each other; to use systems in new ways; to measure and prevent adverse consequences; to make existing quality structures meaningful; and to improve regulation and remove disincentives for vendors to provide clinical decision support. Specific recommendations are to implement provider order entry systems, especially computerized prescribing; to implement bar-coding for medications, blood, devices, and patients; and to utilize modern electronic systems to communicate key pieces of asynchronous data such as markedly abnormal laboratory values. ::: ::: Conclusions: Appropriate increases in the use of information technology in health care—especially the introduction of clinical decision support and better linkages in and among systems, resulting in process simplification—could result in substantial improvement in patient safety. <s> BIB004 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> BACKGROUND ::: In response to the report by the Institute of Medicine on medical errors, national groups have recommended actions to reduce the occurrence of preventable medical errors. What is not known is the level of support for these proposed changes among practicing physicians and the public. ::: ::: ::: METHODS ::: We conducted parallel national surveys of 831 practicing physicians, who responded to mailed questionnaires, and 1207 members of the public, who were interviewed by telephone after selection with the use of random-digit dialing. Respondents were asked about the causes of and solutions to the problem of preventable medical errors and, on the basis of a clinical vignette, were asked what the consequences of an error should be. ::: ::: ::: RESULTS ::: Many physicians (35 percent) and members of the public (42 percent) reported errors in their own or a family member's care, but neither group viewed medical errors as one of the most important problems in health care today. A majority of both groups believed that the number of in-hospital deaths due to preventable errors is lower than that reported by the Institute of Medicine. Physicians and the public disagreed on many of the underlying causes of errors and on effective strategies for reducing errors. Neither group believed that moving patients to high-volume centers would be a very effective strategy. The public and many physicians supported the use of sanctions against individual health professionals perceived as responsible for serious errors. ::: ::: ::: CONCLUSIONS ::: Though substantial proportions of the public and practicing physicians report that they have had personal experience with medical errors, neither group has the sense of urgency expressed by many national organizations. To advance their agenda, national groups need to convince physicians, in particular, that the current proposals for reducing errors will be very effective. <s> BIB005 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> The validity and cost-effectiveness of three methods for detecting medication errors were examined. A stratified random sample of 36 hospitals and skilled-nursing facilities in Colorado and Georgia was selected. Medication administration errors were detected by registered nurses (R.N.s), licensed practical nurses (L.P.N.s), and pharmacy technicians from these facilities using three methods: incident report review, chart review, and direct observation. Each dose evaluated was compared with the prescriber's order. Deviations were considered errors. Efficiency was measured by the time spent evaluating each dose. A pharmacist performed an independent determination of errors to assess the accuracy of each data collector. Clinical significance was judged by a panel of physicians. Observers detected 300 of 457 pharmacist-confirmed errors made on 2556 doses (11.7% error rate) compared with 17 errors detected by chart reviewers (0.7% error rate), and 1 error detected by incident report review (0.04% error rate). All errors detected involved the same 2556 doses. All chart reviewers and 7 of 10 observers achieved at least good comparability with the pharmacist's results. The mean cost of error detection per dose was $4.82 for direct observation and $0.63 for chart review. The technician was the least expensive observer at $2.87 per dose evaluated. R.N.s were the least expensive chart reviewers at $0.50 per dose. Of 457 errors, 35 (8%) were deemed potentially clinically significant; 71% of these were detected by direct observation. Direct observation was more efficient and accurate than reviewing charts and incident reports in detecting medication errors. Pharmacy technicians were more efficient and accurate than R.N.s and L.P.N.s in collecting data about medication errors. <s> BIB006 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> BACKGROUND ::: Studies of hospitalized patients identify safety as a significant problem, but few data are available regarding injuries occurring after discharge. Patients may be vulnerable during this transition period. ::: ::: ::: OBJECTIVE ::: To describe the incidence, severity, preventability, and "ameliorability" of adverse events affecting patients after discharge from the hospital and to develop strategies for improving patient safety during this interval. ::: ::: ::: DESIGN ::: Prospective cohort study. ::: ::: ::: SETTING ::: A tertiary care academic hospital. ::: ::: ::: PATIENTS ::: 400 consecutive patients discharged home from the general medical service. ::: ::: ::: MEASUREMENTS ::: The three main outcomes were adverse events, defined as injuries occurring as a result of medical management; preventable adverse events, defined as adverse events judged to have been caused by an error; and ameliorable adverse events, defined as adverse events whose severity could have been decreased. Posthospital course was determined by performing a medical record review and a structured telephone interview approximately 3 weeks after each patient's discharge. Outcomes were determined by independent physician reviews. ::: ::: ::: RESULTS ::: Seventy-six patients had adverse events after discharge (19% [95% CI, 15% to 23%]). Of these, 23 had preventable adverse events (6% [CI, 4% to 9%]) and 24 had ameliorable adverse events (6% [CI, 4% to 9%]). Three percent of injuries were serious laboratory abnormalities, 65% were symptoms, 30% were symptoms associated with a nonpermanent disability, and 3% were permanent disabilities. Adverse drug events were the most common type of adverse event (66% [CI, 55% to 76%]), followed by procedure-related injuries (17% [CI, 8% to 26%]). Of the 25 adverse events resulting in at least a nonpermanent disability, 12 were preventable (48% [CI, 28% to 68%]) and 6 were ameliorable (24% [CI, 7% to 41%]). ::: ::: ::: CONCLUSION ::: Adverse events occurred frequently in the peridischarge period, and many could potentially have been prevented or ameliorated with simple strategies. <s> BIB007 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Patient interviews <s> Context: Although patient safety is a major problem, most health care organizations rely on spontaneous reporting, which detects only a small minority of adverse events. As a result, problems with safety have remained hidden. Chart review can detect adverse events in research settings, but it is too expensive for routine use. Information technology techniques can detect some adverse events in a timely and cost-effective way, in some cases early enough to prevent patient harm. ::: ::: Objective: To review methodologies of detecting adverse events using information technology, reports of studies that used these techniques to detect adverse events, and study results for specific types of adverse events. ::: ::: Design: Structured review. ::: ::: Methodology: English-language studies that reported using information technology to detect adverse events were identified using standard techniques. Only studies that contained original data were included. ::: ::: Main Outcome Measures: Adverse events, with specific focus on nosocomial infections, adverse drug events, and injurious falls. ::: ::: Results: Tools such as event monitoring and natural language processing can inexpensively detect certain types of adverse events in clinical databases. These approaches already work well for some types of adverse events, including adverse drug events and nosocomial infections, and are in routine use in a few hospitals. In addition, it appears likely that these techniques will be adaptable in ways that allow detection of a broad array of adverse events, especially as more medical information becomes computerized. ::: ::: Conclusion: Computerized detection of adverse events will soon be practical on a widespread basis. <s> BIB008
Much of the data concerning adverse events in the hospital setting has come from chart reviews. However, it has been more challenging to detect adverse events that occur in the outpatient setting using this methodology. Outpatient charts are generally less data intense than inpatient charts with patient visits occurring sporadically. As a result, many researchers have relied on patient interviews to detect adverse events in these settings BIB003 . Recent survey evidence suggests that patients can be good source for adverse event detection. Two recent patient surveys have indicated that 20-42% of patients have experienced and are aware of an error occurring with their medical care and that the majority of these errors resulted in serious consequences BIB005  . A recent study by Forster et al. BIB007 , utilized patient interviews to help construct case summaries for physician reviewers. Information sources in the preparation of the case summaries included patient interviews, signout notes, discharge summaries, and laboratory results. Adverse events were determined based on review of these case-summaries. Patient interviews offer a novel potential source for adverse event detection yet, similar to using observers, require a substantial resource commitment. Nonetheless, patient observations concerning their care likely uncovers adverse events that might not be detectable by other means and the approach warrants further study. In summary, manual adverse event and error detection systems offer the ability to detect a wide array of adverse events. All of the above systems are limited by either physician reluctance to use them or the resource requirements to maintain them, but they can be very useful in research, and direct observation can be used in clinical care to detect the medication error rate because the high rate of errors allows rapid assessment of the rate BIB006 BIB001 . To overcome these limitations, several electronic methods for detecting adverse events have been evaluated. Advances in information technology, the increase in hospitals with integrated information systems, and ambulatory electronic medical records have increasingly made it possible for institutions to automatically detect adverse events BIB008 . Institutions with computerized event-monitors in place have been able to detect a greater number of adverse drug events when compared to spontaneous reporting BIB002 . Even more exciting is that some of these methods can detect errors prior to the occurrence of an adverse event or before it becomes severe and thus prevent or ameliorate patient injury BIB004 . In the following section we review detection methods that combine automated techniques with manual review followed by fully automated systems.