Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> There are risks of adverse outcomes to patients in the course of health care management. A study was undertaken to measure these outcomes in California for 1974. Secondary benefits of the study include new insights into the types and sources of disabilities caused by health care management and the development of new methods for carrying out generic adverse-outcome monitoring. <s> BIB001 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> For nearly two decades, an atmosphere of crisis has enveloped our system of medical-malpractice litigation and liability insurance. Courts have expanded the limits of tort liability, an increasing number of patients have sought to redress their grievances in court, and juries have awarded ever-increasing damages for medical injuries. Legislatures have responded to the problems of a shrinking insurance market and rising insurance costs by restricting liability and, in some cases, limiting damages. The courts have reviewed some of these statutory restrictions and have struck down a number of them as unconstitutional. Mean-while, report after report has come from federal and . . . <s> BIB002 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> To evaluate a process for identifying adverse events through review of medical records, multiple reviews of 360 medical records from two teaching hospitals were performed. The data from these multiple reviews provided information about the validity and reliability of our two-phase review process. In <s> BIB003 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> BACKGROUND ::: As part of an interdisciplinary study of medical injury and malpractice litigation, we estimated the incidence of adverse events, defined as injuries caused by medical management, and of the subgroup of such injuries that resulted from negligent or substandard care. ::: ::: ::: METHODS ::: We reviewed 30,121 randomly selected records from 51 randomly selected acute care, nonpsychiatric hospitals in New York State in 1984. We then developed population estimates of injuries and computed rates according to the age and sex of the patients as well as the specialties of the physicians. ::: ::: ::: RESULTS ::: Adverse events occurred in 3.7 percent of the hospitalizations (95 percent confidence interval, 3.2 to 4.2), and 27.6 percent of the adverse events were due to negligence (95 percent confidence interval, 22.5 to 32.6). Although 70.5 percent of the adverse events gave rise to disability lasting less than six months, 2.6 percent caused permanently disabling injuries and 13.6 percent led to death. The percentage of adverse events attributable to negligence increased in the categories of more severe injuries (Wald test chi 2 = 21.04, P less than 0.0001). Using weighted totals, we estimated that among the 2,671,863 patients discharged from New York hospitals in 1984 there were 98,609 adverse events and 27,179 adverse events involving negligence. Rates of adverse events rose with age (P less than 0.0001). The percentage of adverse events due to negligence was markedly higher among the elderly (P less than 0.01). There were significant differences in rates of adverse events among categories of clinical specialties (P less than 0.0001), but no differences in the percentage due to negligence. ::: ::: ::: CONCLUSIONS ::: There is a substantial amount of injury to patients from medical management, and many injuries are the result of substandard care. <s> BIB004 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> OBJECTIVE ::: To determine whether adverse occurrence screening of inpatient discharge summaries by means of a limited number of criteria can provide an effective medical quality control system without requiring substantial additional resources. ::: ::: ::: DESIGN ::: The medical records department used 14 criteria to screen all inpatient discharge summaries shortly after discharge. Summaries that met particular criteria were sent for medical review. If an adverse patient event was found, appropriate action was determined by a committee of four doctors. ::: ::: ::: SETTING ::: 200 bed base hospital in country Victoria. ::: ::: ::: PATIENTS ::: All inpatients discharged over a two year period. ::: ::: ::: MAIN OUTCOME MEASURES ::: The incidence of negative patient outcomes and the cost of resources required to conduct the program. ::: ::: ::: RESULTS ::: A reduction in the incidence of some negative patient outcomes was achieved and the nature of postgraduate education meetings was changed. Minimal additional resources were required to run the program. ::: ::: ::: CONCLUSION ::: Limited adverse occurrence screening has provided a simple, inexpensive and effective medical quality control system in a medium-sized hospital. <s> BIB005 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> Objective : To 1) assess the degree of agreement among physicians on the cause of previously flagged adverse outcomes and 2) relate the findings to systems of quality assurance and performance assessment and proposals for no-fault compensation for medical injuries. Design : Observational study of 7533 pairs of structured implicit reviews (subjective opinions based on guidelines) of medical records done by 127 physicians working independently. Setting : Random sample of 51 inpatient facilities in New York State. Patients : Random sample of inpatient medical records from the selected facilities. Measurements : 1) Number of agreed-upon adverse events compared with the number of cases of extreme disagreement and 2) internally and indirectly standardized rates at which physician reviewers found adverse events (injuries to patients caused at least in part by medical management). Results : In 12.9% of cases (971 of 7533), the two physicians in a pair had extreme disagreement about the occurrence of an adverse event. These cases outnumbered those in which both reviewers found an adverse event (10% ; n = 757). Agreement was highest for wound infections and lowest for adverse events attributed to failure to diagnose or lack of therapy. The amount of experience the physicians had in reviewing records tended to increase the level of agreement. Even after standardization to the results of the entire sample, individual physicians' rates of finding at least slight evidence of an adverse event varied widely (range, 9.9% to 43.7%) (P < 0.001). Conclusions : Structured implicit reviews produced disagreement on the causes of adverse patient outcomes. If systems of quality assurance, performance audits, or no-fault patient compensation are to succeed, methods for overcoming the common tendency toward disagreement among experts must be developed. <s> BIB006 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> Background: Adverse drug events (ADEs) are both common and costly. Most hospitals identify ADEs using spontaneous reporting, but this approach lacks sensitivity; chart review identifies more events but is expensive. Computer-based approaches to ADE identification appear promising, but they have not been directly compared with chart review and they are not widely used. ::: ::: Objectives: To develop a computer-based ADE monitor, and to compare the rate and type of ADEs found with the monitor with those discovered by chart review and by stimulated voluntary report. ::: ::: Design: Prospective cohort study in one tertiary-care hospital. ::: ::: Participants: All patients admitted to nine medical and surgical units in a tertiary-care hospital over an eight-month period. ::: ::: Main Outcome Measure: Adverse drug events identified by the computer-based monitor, by chart review, and by stimulated voluntary report. ::: ::: Methods: A computer-based monitoring program identified alerts, which were situations suggesting that an ADE might be present (e.g., an order for an antidote such as naloxone). A trained reviewer then examined patients' hospital records to determine whether an ADE had occurred. The results of the computer-based monitoring strategy were compared with two other ADE detection strategies: intensive chart review and stimulated voluntary report by nurses and pharmacists. The monitor and the chart review strategies were independent, and the reviewers were blinded. ::: ::: Results: The computer monitoring strategy identified 2,620 alerts, of which 275 were determined to be ADEs. The chart review found 398 ADEs, whereas voluntary report detected 23. Of the 617 ADEs detected by at least one method, 76 ADEs were detected by both computer monitor and chart review. The computer monitor identified 45 percent; chart review, 65 percent; and voluntary report, 4 percent. The ADEs identified by computer monitor were more likely to be classified as “severe” than were those identified by chart review (51 versus 42 percent, p =.04). The positive predictive value of computer-generated alerts was 16 percent during the first eight weeks of the study; rule modifications increased this to 23 percent in the final eight weeks. The computer strategy required 11 person-hours per week to execute, whereas chart review required 55 person-hours per week and voluntary report strategy required 5. ::: ::: Conclusions: The computer-based monitor identified fewer ADEs than did chart review but many more ADEs than did stimulated voluntary report. The overlap among the ADEs identified using different methods was small, suggesting that the incidence of ADEs may be higher than previously reported and that different detection methods capture different events. The computer-based monitoring system represents an efficient approach for measuring ADE frequency and gauging the effectiveness of ADE prevention programs. <s> BIB007 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> BACKGROUND ::: The ongoing debate on the incidence and types of iatrogenic injuries in American hospitals has been informed primarily by the Harvard Medical Practice Study, which analyzed hospitalizations in New York in 1984. The generalizability of these findings is unknown and has been questioned by other studies. ::: ::: ::: OBJECTIVE ::: We used methods similar to the Harvard Medical Practice Study to estimate the incidence and types of adverse events and negligent adverse events in Utah and Colorado in 1992. ::: ::: ::: DESIGN AND SUBJECTS ::: We selected a representative sample of hospitals from Utah and Colorado and then randomly sampled 15,000 nonpsychiatric 1992 discharges. Each record was screened by a trained nurse-reviewer for 1 of 18 criteria associated with adverse events. If > or =1 criteria were present, the record was reviewed by a trained physician to determine whether an adverse event or negligent adverse event occurred and to classify the type of adverse event. ::: ::: ::: MEASURES ::: The measures were adverse events and negligent adverse events. ::: ::: ::: RESULTS ::: Adverse events occurred in 2.9+/-0.2% (mean+/-SD) of hospitalizations in each state. In Utah, 32.6+/-4% of adverse events were due to negligence; in Colorado, 27.4+/-2.4%. Death occurred in 6.6+/-1.2% of adverse events and 8.8+/-2.5% of negligent adverse events. Operative adverse events comprised 44.9% of all adverse events; 16.9% were negligent, and 16.6% resulted in permanent disability. Adverse drug events were the leading cause of nonoperative adverse events (19.3% of all adverse events; 35.1% were negligent, and 9.7% caused permanent disability). Most adverse events were attributed to surgeons (46.1%, 22.3% negligent) and internists (23.2%, 44.9% negligent). ::: ::: ::: CONCLUSIONS ::: The incidence and types of adverse events in Utah and Colorado in 1992 were similar to those in New York State in 1984. Iatrogenic injury continues to be a significant public health problem. Improving systems of surgical care and drug delivery could substantially reduce the burden of iatrogenic injury. <s> BIB008 </s> Detecting adverse events for patient safety research: a review of current methodologies <s> Chart review <s> Background: The data used by the U.S. Institute of Medicine to estimate deaths from medical errors come from a study that relied on nurse and physician reviews of medical records to detect the errors. Objective: To measure the reliability of medical record review for detecting adverse events and negligent adverse events. Design: Medical record review. Setting: Hospitalizations in Utah and Colorado in 1992. Measurements: After three independent reviews of 500 medical records, the following were measured: reliability and the effect of varying criteria for reviewer confidence in and reviewer agreement about the presence of adverse events. Results: For agreements in judgments of adverse events among the three sets of reviews, the K statistics ranged from 0.40 to 0.41 (95% CIs ranged from 0.30 to 0.51) for adverse events and from 0.19 to 0.23 (CIs, 0.05 to 0.37) for negligent adverse events. Rates for adverse events and for negligent adverse events varied substantially depending on the degree of agreement and the level of confidence that was required among reviewers. Conclusion: Estimates of adverse event rates from medical record review, including those reported by the Institute of Medicine in its 2000 report on medical errors, are highly sensitive to the degree of consensus and confidence among reviewers. <s> BIB009
Most of the epidemiological evidence describing adverse medical events comes from several large studies that used chart reviews to detect adverse events. The California Medical Insurance Feasibility Study, the Harvard Medical Practice Study, and the ColoradoUtah Study all described similar methodologies to detect adverse events BIB008 BIB001 BIB002 . Charts underwent a two-phase screening process. Initially a trained reviewer, usually a research nurse, examined charts for 18 screening criteria. (Table 1 ) Charts with one of these screening criteria then underwent physician review. Physician reviewers made judgments on whether they believed an adverse event had occurred based on the information contained within the chart. Reviewers then rated their confidence that the patientsÕ medical management resulted in the injury. When a physician reviewer was more than 50% confident that medical management resulted in the injury, an adverse event was considered to have occurred. Typically, either all or a fraction of the charts are reviewed independently by two physicians. If the physician reviewers disagree on whether an adverse event occurred, the physicians will try to come to a consensus or involve a third party. While this two-step chart review process has been used to provide much of the adverse event prevalence data, several problems exist with this methodology. First, the positive predictive value of the initial screening process was low (21%) BIB003 resulting in the physician reviewers evaluating a high number of false-positive charts. Second, agreement between physicians is generally poor with respect to causality. In one observational study, the number of cases in which there was ''extreme disagreement'' as to whether an adverse event occurred actually outnumbered the cases in which the physician agreed BIB006 . While the j-statistic for the physician agreement concerning an adverse event occurrence in the Harvard Medical Practice Study was moderate (j ¼ 0:61) BIB004 , other investigators have found lower rates of agreements (j ¼ 0:40, 95% confidence interval [CI] 0.30-0.51) BIB009 and have questioned this overall methodology  . But perhaps the greatest limitation to this methodology to detect adverse events is the overall resources required, which are great when compared to other screening modalities BIB007 . Some investigators have used fewer screening criteria to help reduce the overall resource burden of chart reviews BIB005 but in general, chart review remains an impractical means for routine adverse event detection because of the high costs.