Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> At the LDS Hospital in Salt Lake City, an interface was developed between the microbiology laboratory computer system and the HELP integrated central hospital computer system. The HELP system includes medical information from most clinical care support areas. The microbiology data are translated from the laboratory computer file structure to a hierarchical data structure on the HELP system. A knowledge base was created with the help of infectious disease experts, and became part of a Computerized Infectious Disease Monitoring system (CIDM). The knowledge base is automatically activated when specific microbiology data are entered into a patient's computer file (data driven), thus decisions are made automatically with no additional effort required of medical personnel. The CIDM was designed to inform infectious disease personnel when a patient has one of the following conditions: a hospital-acquired infection, an infection at a normally sterile body site, an infection due to a bacteria with an unusual antibiotic sensitivity pattern, an infection for which the patient is not receiving an antibiotic to which the offending bacteria is sensitive, an infection that could be treated with a less expensive antibiotic, an infection which is required by law to be reported to state and national health authorities, and those patients receiving prophylactic antibiotics longer than is medically indicated. All of the microbiology data are now extensively reviewed by nurses and physicians from terminals at nursing stations or intensive care units. The CIDM is currently being used for hospital-acquired infection surveillance at LDS Hospital. <s> BIB001 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> Surveillance of hospital-acquired infections and antibiotic use is required of US hospitals. The time and cost needed to actively perform this surveillance can be extensive. We developed a computerized infectious disease monitor that automatically generates four types of surveillance "alerts" for patients (1) with hospital-acquired infections, (2) not receiving antibiotics to which their pathogens are susceptible, (3) who could be receiving less expensive antibiotics, or (4) who are receiving prophylactic antibiotics too long. Surveillance personnel using computer screening for two months found more hospital-acquired infections when compared with our traditional surveillance methods, while requiring only 35% of the time. In addition, alerts from the computer identified 37 patients not receiving appropriate antibiotics, 31 patients who could have been receiving less expensive antibiotics, and 142 patients, during one month, receiving prolonged cephalosporin prophylaxis. Computer screening can help focus the activities and improve the efficiency of hospital surveillance personnel. ( JAMA 1986;256:1007-1011) <s> BIB002 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> OBJECTIVE ::: To develop a new method to improve the detection and characterization of adverse drug events (ADEs) in hospital patients. ::: ::: ::: DESIGN ::: Prospective study of all patients admitted to our hospital over an 18-month period. ::: ::: ::: SETTING ::: LDS Hospital, Salt Lake City, Utah, a 520-bed tertiary care center affiliated with the University of Utah School of Medicine, Salt Lake City. ::: ::: ::: PATIENTS ::: We developed a computerized ADE monitor, and computer programs were written using an integrated hospital information system to allow for multiple source detection of potential ADEs occurring in hospital patients. Signals of potential ADEs, both voluntary and automated, included sudden medication stop orders, antidote ordering, and certain abnormal laboratory values. Each day, a list of all potential ADEs from these sources was generated, and a pharmacist reviewed the medical records of all patients with possible ADEs for accuracy and causality. Verified ADEs were characterized as mild, moderate, or severe and as type A (dose-dependent or predictable) or type B (idiosyncratic or allergic) reactions, and causality was further measured using a standardized scoring method. ::: ::: ::: OUTCOME MEASURE ::: The number and characterization of ADEs detected. ::: ::: ::: RESULTS ::: Over 18 months, we monitored 36,653 hospitalized patients. There were 731 verified ADEs identified in 648 patients, 701 ADEs were characterized as moderate or severe, and 664 were classified as type A reactions. During this same period, only nine ADEs were identified using traditional detection methods. Physicians, pharmacists, and nurses voluntarily reported 92 of the 731 ADEs detected using this automated system. The other 631 ADEs were detected from automated signals, the most common of which were diphenhydramine hydrochloride and naloxone hydrochloride use, high serum drug levels, leukopenia, and the use of phytonadione and antidiarrheals. The most common symptoms and signs were pruritus, nausea and/or vomiting, rash, and confusion-lethargy. The most common drug classes involved were analgesics, anti-infectives, and cardiovascular agents. ::: ::: ::: CONCLUSION ::: We believe that screening for ADEs with a computerized hospital information system offers a potential method for improving the detection and characterization of these events in hospital patients. <s> BIB003 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> To improve the detection and characterization of adverse drug events (ADEs) in hospitalized patients, a computerized adverse drug event monitor was developed. Computer programs were written to allow for voluntary as well as automated detection of adverse drug events using the HELP hospital information system, a large integrated hospital database containing computerized patient medical records and a knowledge base allowing for automated medical decisions. Programs were created to allow simple computer entry of potential adverse drug events by physicians, pharmacists, and nurses. Automated detection of potential adverse drug events relied on signals such as sudden medication stop orders, "antidote" orders, and selected abnormal laboratory values. Each day a list of all potential adverse drug events from these sources was generated and a pharmacist reviewed the medical records and interviewed healthcare personnel associated with patients identified as having potential adverse drug events. This process allowed for characterization of the event, causality assessment, and follow-up of the resulting clinical course by the pharmacist. The permanent storage of these results in the computerized patient medical record permits their future retrieval to prevent adverse drug events during subsequent hospital care. The authors conclude that fully integrated hospital systems will permit the further development and evaluation of computer-assisted methods for the detection of adverse drug events in hospitalized patients. <s> BIB004 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> OBJECTIVES ::: To assess postoperative exposure to parenteral antibiotics and coded discharge diagnoses of infection as markers of nosocomial infection, postoperative morbidity, and potentially inappropriate antibiotic use after cesarean section. ::: ::: ::: DESIGN ::: Retrospective cohort study to compare automated markers with the criterion of record review. ::: ::: ::: SETTING ::: Tertiary care hospital. ::: ::: ::: PATIENTS ::: Women admitted to a large teaching hospital after April 15, 1987, and discharged before October 1, 1989, who underwent a nonrepeat, nonelective cesarean section and had received prophylaxis with a cephalosporin. ::: ::: ::: METHODS ::: Antibiotic exposure and discharge diagnosis codes were obtained from a large electronic hospital data base. A sample of charts was reviewed to determine the presence of infection, other postoperative complications, and postoperative antibiotic exposure. ::: ::: ::: RESULTS ::: A total of 2,197 women who had undergone a nonrepeat nonelective cesarean section were included in the study cohort. These women were assigned to 6 subgroups based on postoperative antibiotic exposure status and discharge codes suggesting endometritis, other postoperative infection, or no infection. Review of 457 records indicated that the overall infection rate was 9%. Eight percent of all the patients had a coded diagnosis for infection, and 16% received some parenteral antibiotics after the first postoperative day. Exposure to at least 2 days of parenteral postoperative antibiotics was the best marker by which to discriminate between infected and uninfected patients, with a sensitivity of 81%, a specificity of 95%, and a positive predictive value of 61% for detecting infection. The corresponding figures for coded diagnoses for infection had rates of 65%, 97%, and 74%, respectively. The combination of discharge codes and exposure to parenteral postoperative antibiotics resulted in a more accurate but less sensitive marker for nosocomial infections, with a positive predictive value of 94% and a sensitivity of 59%. The groups with discordant parenteral postoperative antibiotics exposure and discharge codes for infection were enriched for errors in coding, noninfectious morbidity, and unexplained antibiotic use. Less than 1% of the entire cohort had > or = to 2 days of parenteral postoperative antibiotics without any reason apparent in the medical record. ::: ::: ::: CONCLUSIONS ::: Parenteral postoperative antibiotic exposure determined from automated pharmacy records correlated with the results of the more labor-intensive manual review of medical records for the identification of nosocomial infection. In addition, information on antibiotic exposure combined with coded discharge diagnoses provided a rapid screen to identify subgroups of patients with higher rates of infectious and noninfectious morbidity, unexplained antibiotic use, and errors in discharge coding. Information derived from electronic data bases created for administrative purposes may be useful as a marker for infectious complications, inappropriate antibiotic prescribing, and other issues related to total quality hospital monitoring. <s> BIB005 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> Abstract ::: Hospital-acquired infections are responsible for an increase in patient mortality and costs. Their detection is essential to permit better infection control. We developed an expert system specifically to detect infections in pediatric patients. The expert system is implemented at LDS Hospital that has a level three newborn intensive care unit and well baby units. We describe how the knowledge base of the expert system was developed, implemented, and validated in a retrospective study. The results of the system were compared to manual reviewer results. The expert system had a sensitivity of 84.5% and specificity of 92.8% in detecting hospital-acquired infections when compared to a physician reviewer. The Cohen's kappa between the expert system and the physician reviewer was 0.62 (p < .001). <s> BIB006 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> Objective: To evaluate the automated detection of clinical conditions described in narrative reports. Design: Automated methods and human experts detected the presence or absence of six clinical conditions in 200 admission chest radiograph reports. Study Subjects: A computerized, general-purpose natural language processor, 6 internists; 6 radiologists; 6 lay persons; and 3 other computer methods. Main Outcome Measures: Intersubject disagreement was quantified by «distance» (the average number of clinical conditions per report on which two subjects disagreed) and by sensitivity and specificity with respect to the physicians. Results: Using a majority vote, physicians detected 101 conditions in the 200 reports (0.51 per report); the most common condition was acute bacterial pneumonia (prevalence, 0.14), and the least common was chronic obstructive pulmonary disease (prevalence, 0.03). Pairs of physicians disagreed on the presence of at least 1 condition for an average of 20% of reports. The average intersubject distance among physicians was 0.24 (95% CI, 0.19 to 0.29) out of a maximum possible distance of 6. No physician had a significantly greater distance than the average. The average distance of the natural language processor from the physicians was 0.26 (CI, 0.21 to 0.32; not significantly greater than the average among physicians). Lay persons and alternative computer methods had significantly greater distance from the physicians (all >0.5). The natural language processor had a sensitivity of 81 % (CI, 73% to 87%) and a specificity of 98% (CI, 97% to 99%); physicians had an average sensitivity of 85% and an average specificity of 98%. Conclusions: Physicians disagreed on the interpretation of narrative reports, but this was not caused by outlier physicians or a consistent difference in the way internists and radiologists read reports. The natural language processor was not distinguishable from the physicians and was superior to all other comparison subjects. Although the domain of this study was restricted (six clinical conditions in chest radiographs), natural language processing seems to have the potential to extract clinical information from narrative reports in a manner that will support automated decision-support and clinical research <s> BIB007 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> BACKGROUND: Methicillin-resistant Staphylococcus aureus (MRSA) is an escalating problem in hospitals worldwide. The hospital reservoir for MRSA includes recognized and unrecognized colonized or infected patients, as well as previously colonized or infected patients readmitted to the hospital. Early and appropriate infection control measures (ICM) are key elements to reduce MRSA transmission and to control the hospital reservoir. OBJECTIVE: To describe the role of an expert system applied to the control of MRSA at a large medical center (1,600 beds) with high endemic rates. METHODS: The University Hospital of Geneva has an extended hospital information system (HIS), DIOGENE, structured with an open distributed architecture. It includes administrative, medical, nursing, and laboratory applications with their relational databases. Among available patient databases, clinical microbiology laboratory and admission-discharge-transfer (ADT) databases are used to generate computer alerts. A laboratory alert (lab alert) is printed daily in the Infection Control Program (ICP) offices, listing all patients with cultures positive for MRSA detected within the preceding 24 hours. Patients might be either newly detected patients colonized or infected with MRSA, or previously recognized MRSA patients having surveillance cultures. The ICP nurses subsequently go to the ward or call the ward personnel to implement ICM. A second alert, the "readmission alert," detects readmission to the hospital of any patient previously colonized or infected with MRSA by periodic queries (q 1 min) to the ADT database. The readmission alert is printed in the ICP offices, but also forwarded with added guidelines to the emergency room. RESULTS: During the first 12 months of application (July 1994 to June 1995), the lab alert detected an average of 4.6 isolates per day, corresponding to 314 hospital admissions (248 patients); the use of this alert saved time for the ICP nurses by improving work organization. There were 438 readmission alerts (1.2 alerts per day) over the study period; of 347 patients screened immediately upon readmission, 114 (33%) were positive for MRSA carriage. Delayed recognition of readmitted MRSA carriers decreased significantly after the implantation of this alert; the proportion of MRSA patients recognized at the time of admission to the hospital increased from 13% in 1993 to 40% in 1995 (P <.001). CONCLUSIONS: Hospital information system-based alerts can play an important role in the surveillance and early prevention of MRSA transmission, and it can help to recognize patterns of colonization and transmission. <s> BIB008 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> BACKGROUND AND METHODS ::: Optimal decisions about the use of antibiotics and other antiinfective agents in critically ill patients require access to a large amount of complex information. We have developed a computerized decision-support program linked to computer-based patient records that can assist physicians in the use of antiinfective agents and improve the quality of care. This program presents epidemiologic information, along with detailed recommendations and warnings. The program recommends antiinfective regimens and courses of therapy for particular patients and provides immediate feedback. We prospectively studied the use of the computerized antiinfectives-management program for one year in a 12-bed intensive care unit. ::: ::: ::: RESULTS ::: During the intervention period, all 545 patients admitted were cared for with the aid of the antiinfectives-management program. Measures of processes and outcomes were compared with those for the 1136 patients admitted to the same unit during the two years before the intervention period. The use of the program led to significant reductions in orders for drugs to which the patients had reported allergies (35, vs. 146 during the preintervention period; P<0.01), excess drug dosages (87 vs. 405, P<0.01), and antibiotic-susceptibility mismatches (12 vs. 206, P<0.01). There were also marked reductions in the mean number of days of excessive drug dosage (2.7 vs. 5.9, P<0.002) and in adverse events caused by antiinfective agents (4 vs. 28, P<0.02). In analyses of patients who received antiinfective agents, those treated during the intervention period who always received the regimens recommended by the computer program (n=203) had significant reductions, as compared with those who did not always receive the recommended regimens (n= 195) and those in the preintervention cohort (n = 766), in the cost of antiinfective agents (adjusted mean, $102 vs. $427 and $340, respectively; P<0.001), in total hospital costs (adjusted mean, $26,315 vs. $44,865 and $35,283; P<0.001), and in the length of the hospital stay days (adjusted mean, 10.0 vs. 16.7 and 12.9; P<0.001). CONCLUSIONS; A computerized antiinfectives-management program can improve the quality of patient care and reduce costs. <s> BIB009 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> Background: Adverse drug events (ADEs) are both common and costly. Most hospitals identify ADEs using spontaneous reporting, but this approach lacks sensitivity; chart review identifies more events but is expensive. Computer-based approaches to ADE identification appear promising, but they have not been directly compared with chart review and they are not widely used. ::: ::: Objectives: To develop a computer-based ADE monitor, and to compare the rate and type of ADEs found with the monitor with those discovered by chart review and by stimulated voluntary report. ::: ::: Design: Prospective cohort study in one tertiary-care hospital. ::: ::: Participants: All patients admitted to nine medical and surgical units in a tertiary-care hospital over an eight-month period. ::: ::: Main Outcome Measure: Adverse drug events identified by the computer-based monitor, by chart review, and by stimulated voluntary report. ::: ::: Methods: A computer-based monitoring program identified alerts, which were situations suggesting that an ADE might be present (e.g., an order for an antidote such as naloxone). A trained reviewer then examined patients' hospital records to determine whether an ADE had occurred. The results of the computer-based monitoring strategy were compared with two other ADE detection strategies: intensive chart review and stimulated voluntary report by nurses and pharmacists. The monitor and the chart review strategies were independent, and the reviewers were blinded. ::: ::: Results: The computer monitoring strategy identified 2,620 alerts, of which 275 were determined to be ADEs. The chart review found 398 ADEs, whereas voluntary report detected 23. Of the 617 ADEs detected by at least one method, 76 ADEs were detected by both computer monitor and chart review. The computer monitor identified 45 percent; chart review, 65 percent; and voluntary report, 4 percent. The ADEs identified by computer monitor were more likely to be classified as “severe” than were those identified by chart review (51 versus 42 percent, p =.04). The positive predictive value of computer-generated alerts was 16 percent during the first eight weeks of the study; rule modifications increased this to 23 percent in the final eight weeks. The computer strategy required 11 person-hours per week to execute, whereas chart review required 55 person-hours per week and voluntary report strategy required 5. ::: ::: Conclusions: The computer-based monitor identified fewer ADEs than did chart review but many more ADEs than did stimulated voluntary report. The overlap among the ADEs identified using different methods was small, suggesting that the incidence of ADEs may be higher than previously reported and that different detection methods capture different events. The computer-based monitoring system represents an efficient approach for measuring ADE frequency and gauging the effectiveness of ADE prevention programs. <s> BIB010 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> CONTEXT ::: Adverse drug events (ADEs) are the most common type of iatrogenic injury occurring in hospitalized patients. Errors leading to ADEs are often due to restricted availability of information at the time of physician order writing. ::: ::: ::: OBJECTIVES ::: To develop, implement, and evaluate a computer alert system designed to correct errors that might lead to ADEs and to detect ADEs before maximum injury occurs. ::: ::: ::: DESIGN ::: Prospective case series. ::: ::: ::: SETTING ::: A 650-bed community teaching hospital in Phoenix, Ariz. ::: ::: ::: PATIENTS ::: Consecutive sample of 9306 nonobstetrical adult patients admitted during the last 6 months of 1997. ::: ::: ::: INTERVENTIONS ::: Thirty-seven drug-specific ADEs were targeted. Our hospital information system was programmed to generate alerts in clinical situations with increased risk for ADE-related injury. A clinical system was developed to ensure physician notification of alerts. ::: ::: ::: MAIN OUTCOME MEASURES ::: A true-positive alert was defined as one in which the physician wrote orders consistent with the alert recommendation after alert notification. ::: ::: ::: RESULTS ::: During the 6-month study period, the alert system fired 1116 times and 596 were true-positive alerts (positive predictive value of 53%). The alerts identified opportunities to prevent patient injury secondary to ADEs at a rate of 64 per 1000 admissions. A total of 265 (44%) of the 596 true-positive alerts were unrecognized by the physician prior to alert notification. ::: ::: ::: CONCLUSIONS ::: Clinicians can use hospital information systems to detect opportunities to prevent patient injury secondary to a broad range of ADEs. <s> BIB011 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> If accurate clinical information were available electronically, automated applications could be developed to use this information to improve patient care and lower costs. However, to be fully retrievable, clinical information must be structured or coded. Many online patient reports are not coded, but are recorded in natural-language text that cannot be reliably accessed. Natural language processing (NLP) can solve this problem by extracting and structuring text-based clinical information, making clinical data available for use. NLP systems are quite difficult to develop, as they require substantial amounts of knowledge, but progress has definitely been made. Some NLP systems have been developed and tested and have demonstrated promising performance in practical clinical applications; some of these systems have already been deployed. The authors provide background information about NLP, briefly describe some of the systems that have been recently developed, and discuss the future of NLP in medicine. <s> BIB012 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> OBJECTIVE ::: To evaluate the performance of a natural language processing system in extracting pneumonia-related concepts from chest x-ray reports. ::: ::: ::: METHODS ::: ::: ::: ::: DESIGN ::: Four physicians, three lay persons, a natural language processing system, and two keyword searches (designated AAKS and KS) detected the presence or absence of three pneumonia-related concepts and inferred the presence or absence of acute bacterial pneumonia from 292 chest x-ray reports. Gold standard: Majority vote of three independent physicians. Reliability of the gold standard was measured. ::: ::: ::: OUTCOME MEASURES ::: Recall, precision, specificity, and agreement (using Finn's R: statistic) with respect to the gold standard. Differences between the physicians and the other subjects were tested using the McNemar test for each pneumonia concept and for the disease inference of acute bacterial pneumonia. ::: ::: ::: RESULTS ::: Reliability of the reference standard ranged from 0.86 to 0.96. Recall, precision, specificity, and agreement (Finn R:) for the inference on acute bacterial pneumonia were, respectively, 0.94, 0.87, 0.91, and 0.84 for physicians; 0.95, 0.78, 0.85, and 0.75 for natural language processing system; 0.46, 0.89, 0.95, and 0.54 for lay persons; 0.79, 0.63, 0.71, and 0.49 for AAKS; and 0.87, 0.70, 0.77, and 0.62 for KS. The McNemar pairwise comparisons showed differences between one physician and the natural language processing system for the infiltrate concept and between another physician and the natural language processing system for the inference on acute bacterial pneumonia. The comparisons also showed that most physicians were significantly different from the other subjects in all pneumonia concepts and the disease inference. ::: ::: ::: CONCLUSION ::: In extracting pneumonia related concepts from chest x-ray reports, the performance of the natural language processing system was similar to that of physicians and better than that of lay persons and keyword searches. The encoded pneumonia information has the potential to support several pneumonia-related applications used in our institution. The applications include a decision support system called the antibiotic assistant, a computerized clinical protocol for pneumonia, and a quality assurance application in the radiology department. <s> BIB013 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> BACKGROUND ::: Adverse events are a leading cause of morbidity and mortality. Adverse Drug Events (ADEs) are frequent, under-reported, costly, and largely preventable. Computerized tools expose effectively ADEs and can reduce their impact. ::: ::: ::: METHODS ::: RADARx is a Veterans Administration (VA) VistA-compatible M software program that integrates computerized ADE screening, probability assessment, documentation, and reporting capabilities. Data dictionary mapping tools have enabled RADARx implementation at 3 sites. All RADARx alerts generated at VA Medical Center (VAMC) Nashville from 7/1/99 to 9/30/99 were evaluated. ::: ::: ::: RESULTS ::: Total ADEs documented using RADARx numbered 57. RADARx discovered 34 ADEs and 48 'potential' ADEs; 23 ADEs were found using traditional means. Overall, 11% of RADARx alerts were true positives. Pharmacist review of the 8-20 alerts generated daily costs between 10 and 30 minutes. <s> BIB014 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> Errors in health care facilities are common and often unrecognized. We have used our clinical event monitor to prevent and detect medication errors by scrutinizing electronic messages sent to it when any medication order is written in our facility. A growing collection of medication safety rules covering dose limit errors, laboratory monitoring, and other topics may be applied to each medication order message to provide an additional layer of protection beyond existing order checks, reminders, and alerts available within our computer-based record system. During a typical day the event monitor receives 4802 messages, of which 4719 pertain to medication orders. We have found the clinical event monitor to be a valuable tool for clinicians and quality management groups charged with improving medication safety. <s> BIB015 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> INTRODUCTION ::: In inpatients, computer monitors have been used to improve the detection of adverse drug events (ADEs). However, similar programs have not been available in outpatients. ::: ::: ::: OBJECTIVE ::: To describe an approach for detecting incidents suggesting that an ADE may have occurred in outpatients by adapting methods from inpatient computer monitoring and developing terminology searches of electronic medical records. ::: ::: ::: METHODS ::: One year of information from the outpatient electronic medical record (EMR) at one hospital and its clinics was reviewed. Altogether, 23064 patients and 88514 visits were identified. Patient demographics, medical problem lists, ICD-9 claims, patient allergies, medication history and all clinic visit notes were extracted and merged. We then searched for incidents suggesting that an ADE might be present using four methods: ICD-9 claims, new allergies, computer rules linking laboratory data to known medication exposures, and a medical terminology lexicon (M2D2). In this report, we describe how these search methods were developed to allow for ADE identification. ::: ::: ::: CONCLUSION ::: The ability to carry out such quality-related work is an example of the benefits of the outpatient EMR that may not be apparent to those institutions considering adopting it. <s> BIB016 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Integrated data source <s> OBJECTIVE ::: To evaluate the use of a computer program to identify adverse drug events (ADEs) in the ambulatory setting and to evaluate the relative contribution of four computer search methods for identifying ADEs, including diagnosis codes, allergy rules, computer event monitoring rules, and text searching. ::: ::: ::: DESIGN ::: Retrospective analysis of one year of data from an electronic medical record, including records for 23,064 patients with a primary care physician, of whom 15,665 actually came for care. ::: ::: ::: MEASUREMENT ::: Presence of an ADE; sensitivity and specificity of computer searches for ADE. ::: ::: ::: RESULTS ::: The computer program identified 25,056 incidents, which were associated with an estimated 864 (95 percent confidence interval [CI], 750-978) ADES. Thus, the ADE rate was 5.5 (CI, 5.2-5.9) per 100 patients coming for care. Furthermore, in 79 (CI, 68-89) ADEs, the patient required hospitalization, resulting in an estimated rate of 3.4 (CI, 2.7-4.3) admissions per 1,000 patients. The sensitivity of the search methods for identifying ADEs was estimated to be 58 (CI, 18-98) percent, and the estimated specificity was 88 (CI, 87-88) percent. The positive predictive value was 7.5 (CI, 6.5-8.5) percent, and the negative predictive value was 99.2 (CI, 95.5-99.98) percent. Compared with age and gender-matched controls with no positive screen, patients with ADEs had twice as many outpatient visits and were taking nearly three times as many drugs. Antihypertensives, ACE-inhibitors, antibiotics, and diuretics were associated with 56 (CI, 47-65) percent of ADES. Among ADEs, 23 (CI, 16-32) percent were life-threatening or serious, and 38 (CI, 29-47) percent were judged preventable. ::: ::: ::: CONCLUSION ::: Computerized search programs can detect ADEs, and free-text searches were especially useful. Adverse drug events were frequent, and admissions were not rare, although most hospitals today do not identify them. Thus, such detection programs demonstrate "value-added" for the electronic record and may be useful for directing and assessing the impact of quality improvement efforts. <s> BIB017
Using an electronic medical record, Honigman et al. BIB016 developed a tool that searched a patientÕs administrative data, medication history, and clinical narrative from the visit note to identify adverse drug events. The tool initially parsed the visit note into words and word combinations. These terms were then filtered through a medical dictionary database and joined into master concepts, which could be indicative of an adverse drug event. Drugs and drug classes were then linked to the reported adverse drug events. For example, if the term diarrhea was mentioned within the clinical narrative, this term was joined to the adverse drug event concept of ''diarrhea'' and linked to any medication the patient might be taking that could cause diarrhea, such as omeprazole. In a study comparing multiple detection methods (ICD-9-CM screening, allergy screening, event monitor, and text searching) the overall positive predictive value of this process was low (7.2%, 95% CI 6.8-7.5), but this methodology was estimated to have detected 91% of all adverse drug events recorded in the chart BIB016 . Using clinical narrative to detect adverse events has great potential as a screening tool. Advances in the searching algorithms and the use of natural language processors should greatly improve the test characteristics. Natural language processors extract concepts from free text narrative through using basic grammatical rules and knowledge about the domain BIB007 BIB012  . A range of natural language processing technologies exists. Some processors cover a broad range of medicine and produce comprehensive coded output  . To detect medical events, one applies queries or rules to the output to find cases with relevant evidence. If patient safety terms are missing from the processorÕs lexicon, it will be necessary to add them. Some natural language processors are designed to detect specific concepts. In this case, it is necessary to modify the processor itself (e.g., modify a Bayesian network BIB013 ). Natural language processors have achieved classification rates similar to that of expected manual reviewers, with receiver operating characteristic curve areas of 0.95-0.96 BIB007 BIB013  . These natural language processors outperformed keyword searches even when negation was explicitly handled BIB007 BIB013 . There is therefore enthusiasm that natural language processing may replace costly manual review of discharge summaries, operative notes, nursing notes, daily progress notes, and visit notes for detecting adverse events. Unfortunately, as of this writing, natural language processors are not broadly available, and it will take some time for commercial products to reach the performance of research systems. More often, nosocomial infection detection systems have relied on data from several sources. One approach has combined laboratory (microbiology) data and administrative data to generate alerts regarding possible nosocomial infections BIB005 BIB008 . Hirschhorn et al. BIB005 developed a computer program that captured the duration and timing of antibiotic exposure and the ICD-9-CM coded discharge diagnosis to detect nosocomial infections occurring after non-elective cesarean sections. Alerts generated when an individual had a discharge diagnosis code suggestive of infection and duration of antibiotic exposure greater than expected for prophylaxis purposes had a high positive predictive value for detecting a nosocomial infection (94%) but a poor sensitivity (59%). Major limitations of this detection system resulted from inaccuracies in the diagnostic codes, with 35% of the infections not being identified by ICD-9-CM codes. Other nosocomial infection surveillance systems have relied on a combination of microbiology data and an extensively developed rule-based expert system BIB001 BIB006 . These systems typically search for positive culture reports from microbiology reports. Based on information obtained from the culture report, the computers activates specific modules of its knowledge base to extract other information, such as admission dates and dates of recent hospitalizations, from the patientÕs electronic medical record. This information is used to generate a daily report for hospital infection control personnel. These systems have been developed to be both ''data driven,'' initiating the logic module anytime an alert signal in entered into the system, or ''time driven,'' initiating the logic module at a set time each day. Nosocomial infection surveillance systems coupled with decision support and electronic order entry can decrease inappropriate antibiotic utilization and reduce hospital expenditures BIB002 BIB009 . To increase the overall numbers of adverse drug events detected, many institutions have searched several data sources for signals. Some of these systems have incorporated an alert logic into the searching strategy. An early example of an event monitor, which utilized multiple data sources stems from work done at LDS Hospital in Utah BIB004 . The event monitor continuously searched patient records looking for laboratory abnormalities, such as eosinophil percent greater than 6 or a digoxin level greater than 2 ng/dL. The program also utilized screening algorithms to detect sudden stop orders, dose reduction orders, or drug antidotes. The event monitor was able to detect 731 adverse drug events over an 18-month period BIB003 . During this same time period traditional reporting means only detected nine events. The number of false negative reports generated by this system was not described. False positive reports were a problem and through iterative testing, certain screens that often produced false-positive alerts were dropped as signals BIB004 . Furthermore, logic was added to the event monitor to help further reduce false-positive reports. Rules were added to the knowledge base that would prevent the event monitor from generating an alert is patient had elevated liver function tests and an admission diagnosis reflective of liver disease. Jha et al. BIB010 , at Brigham and WomenÕs Hospital in Boston, used similar signals that were used at LDS Hospital for detection of drug events, and further expanded the rule-base. Rules consisted of Boolean combinations of signals detected through medication orders, laboratory results, and medication orders associated with changes in laboratory results. To reduce false positive alerts several rules were conditional on multiple criteria being met. An example would be if pharmacy records indicated a patient was taking a nephrotoxic medication while simultaneously laboratory data indicated the patientÕs serum creatinine had risen. (Table 2) The event monitor detected 275 adverse drug events compared to 398 detected through manual chart review and 23 detected through incident reports. In the first 8-weeks of the study the overall positive predictive value of the screening tool was 16%. Through removal of poorly performing screens and the addition on more rule combinations, this positive predictive value increased to 23%. Over the entire study 83% of the alerts were considered false positives. Despite this high level of false positive alerts, the computer strategy required only 11 person hours a week to execute as opposed to 55 person hours for manual chart review. Similar event monitors have been implemented in both community hospitals BIB011 as well as Veterans Affairs hospitals BIB014 BIB015 . Raschke et al. BIB011 developed an event-monitor which targeted 37 specific adverse events. Because this study focused on a pre-defined set of adverse drug events, an extensive alert logic could be developed. Not surprisingly the number of false positives alerts in this study were significantly less that most prior studies, with only 47% of alerts being judged as false positives. Drug-laboratory rules have also been used to detect adverse drug events occurring in outpatient care. As with several inpatient event monitors, Boolean combinations of new medication orders, laboratory results above or below a certain threshold, and changes in laboratory values over time were used as the alert logic Receiving diphenhydramine AND no diphenhydramine within last 7 days AND patient no on paclitaxel AND no blood transfusion in last 1 day a EMR, electronic medical record. BIB017 . Drug-laboratory rules for outpatient care were able to detect more adverse drug events than using ICD-9-CM codes or using patient allergy information but overall had very low positive predictive value of 3%. As with inpatient monitors, with iterative testing it is likely that the performance of drug-laboratory rules will improve.