A survey of automatic de-identification of longitudinal clinical narratives <s> Error Analysis <s> Display Omitted We present a method for automatic de-identification of clinical narratives.We propose and validate a two-pass tagging method to improve PHI entity recognition.We have shown that automated de-identification is comparable to human benchmark. A recent promise to access unstructured clinical data from electronic health records on large-scale has revitalized the interest in automated de-identification of clinical notes, which includes the identification of mentions of Protected Health Information (PHI). We describe the methods developed and evaluated as part of the i2b2/UTHealth 2014 challenge to identify PHI defined by 25 entity types in longitudinal clinical narratives. Our approach combines knowledge-driven (dictionaries and rules) and data-driven (machine learning) methods with a large range of features to address de-identification of specific named entities. In addition, we have devised a two-pass recognition approach that creates a patient-specific run-time dictionary from the PHI entities identified in the first step with high confidence, which is then used in the second pass to identify mentions that lack specific clues. The proposed method achieved the overall micro F1-measures of 91% on strict and 95% on token-level evaluation on the test dataset (514 narratives). Whilst most PHI entities can be reliably identified, particularly challenging were mentions of Organizations and Professions. Still, the overall results suggest that automated text mining methods can be used to reliably process clinical notes to identify personal information and thus providing a crucial step in large-scale de-identification of unstructured data for further clinical and epidemiological studies. <s> BIB001 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Error Analysis <s> Display Omitted The method used in our work is completely machine-learning-based.Pattern-matching techniques are used in the preprocessing module.Three groups of features are extracted to train the de-identifier model.Missing errors are the main component of the errors in system output. De-identification is a shared task of the 2014 i2b2/UTHealth challenge. The purpose of this task is to remove protected health information (PHI) from medical records. In this paper, we propose a novel de-identifier, WI-deId, based on conditional random fields (CRFs). A preprocessing module, which tokenizes the medical records using regular expressions and an off-the-shelf tokenizer, is introduced, and three groups of features are extracted to train the de-identifier model. The experiment shows that our system is effective in the de-identification of medical records, achieving a micro-F1 of 0.9232 at the i2b2 strict entity evaluation level. <s> BIB002 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Error Analysis <s> Display Omitted A model to automatically detect protected health information in clinical texts.Machine learning techniques combined with keywords and rule-based approaches.7 main PHI categories with 25 associated sub-categories are identified.Achieving an overall micro-averaged F-measure of 93.6%.The winner of 2014 i2b2 de-identification challenge task. This paper presents a natural language processing (NLP) system that was designed to participate in the 2014 i2b2 de-identification challenge. The challenge task aims to identify and classify seven main Protected Health Information (PHI) categories and 25 associated sub-categories. A hybrid model was proposed which combines machine learning techniques with keyword-based and rule-based approaches to deal with the complexity inherent in PHI categories. Our proposed approaches exploit a rich set of linguistic features, both syntactic and word surface-oriented, which are further enriched by task-specific features and regular expression template patterns to characterize the semantics of various PHI categories. Our system achieved promising accuracy on the challenge test data with an overall micro-averaged F-measure of 93.6%, which was the winner of this de-identification challenge. <s> BIB003 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Error Analysis <s> The CEGS N-GRID 2016 Shared Task 1 in Clinical Natural Language Processing focuses on the de-identification of psychiatric evaluation records. This paper describes two participating systems of our team, based on conditional random fields (CRFs) and long short-term memory networks (LSTMs). A pre-processing module was introduced for sentence detection and tokenization before de-identification. For CRFs, manually extracted rich features were utilized to train the model. For LSTMs, a character-level bi-directional LSTM network was applied to represent tokens and classify tags for each token, following which a decoding layer was stacked to decode the most probable protected health information (PHI) terms. The LSTM-based system attained an i2b2 strict micro-F_1 measure of 89.86%, which was higher than that of the CRF-based system. <s> BIB004 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Error Analysis <s> Abstract Clinical narratives (the text notes found in patients’ medical records) are important information sources for secondary use in research. However, in order to protect patient privacy, they must be de-identified prior to use. Manual de-identification is considered to be the gold standard approach but is tedious, expensive, slow, and impractical for use with large-scale clinical data. Automated or semi-automated de-identification using computer algorithms is a potentially promising alternative. The Informatics Institute of the University of Alabama at Birmingham is applying de-identification to clinical data drawn from the UAB hospital’s electronic medical records system before releasing them for research. We participated in a shared task challenge by the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) at the de-identification regular track to gain experience developing our own automatic de-identification tool. We focused on the popular and successful methods from previous challenges: rule-based, dictionary-matching, and machine-learning approaches. We also explored new techniques such as disambiguation rules, term ambiguity measurement, and used multi-pass sieve framework at a micro level. For the challenge’s primary measure (strict entity), our submissions achieved competitive results (f-measures: 87.3%, 87.1%, and 86.7%). For our preferred measure (binary token HIPAA), our submissions achieved superior results (f-measures: 93.7%, 93.6%, and 93%). With those encouraging results, we gain the confidence to improve and use the tool for the real de-identification task at the UAB Informatics Institute. <s> BIB005
Several of the de-identification systems developed during the competitions performed independent error analysis. Table 10 describes the most common types of errors that were defined by both 2014 and 2016 competition participants BIB001 ; BIB002 ; BIB003 ). In both 2014 and 2016 competitions, most of the label errors fall into inter-PHI ambiguity instances, such DOCTOR and PATIENT. It is challenging to distinguish between PATIENT and DOCTOR as they both fall under the same main category, NAME BIB002 ; BIB003 ). The 2016 competitions found sub-categories LOCATION and CONTACT had similar problems BIB004 ). The main issue with a label error is it will leave false placeholders and can cause confusion if it is a false positive ). If it is a false negative it is missed entirely. In the 2014 competition, it was observed that among the PHI categories DATE is responsible for one-third of the extent errors. BIB003 BIB003 ) reported that out of the 1161 errors 69.7% occurred due to missing errors. BIB002 BIB002 ) also reported missing errors to be 64.42%. In 2016 competitions, missing error occurs more often in subcategories with few instances of training data. For example, EMAIL, URL, MEDICAL RECORD; CON-TACT, PROFESSION and ORGANISATION had the highest missing rates BIB004 ). Missing errors are also more serious as they have the highest risk of revealing patient identity BIB002 ; ). In comparison to missing errors, the proportion of spurious errors is much lower BIB002 ). Spurious errors occurred when tokens had similar lexical or contextual features with real PHI entities -e.g., DATE, AGE BIB004 ). It falsely removes non-PHI information which may be important for clinical or research applications ). There were additional sources of error that were mentioned concerning the 2016 competition in comparison to that of 2014. These include, acronyms and misspelt words, unknown name entities and errors due to the corpora and annotation problems. Acronyms and misspelt words are caused by wrongly recognising and classifying PHIs. However, recognising and classifying acronyms and abbreviations is hard as they can be medical terms or non-medical terms. For example, NAME: NC, NG etc. and DATE: Su, Tu, We etc. BIB005 ). Corpora and annotation problems occur mainly due to merged words causing missing PHI, for example "Hancevillewith". Also, error is caused by false negative annotations BIB005 ). Also, the reliance on using dictionaries and ML approaches, where unseen terms and unfamiliar contexts cause difficulties in identifying person and location resulted in errors. ) were also interested in the following sources of the errors: abbreviations, ambiguities, data sparsity and debatable annotations. They found that abbreviations and ambiguities are the most challenging source of errors to address to improve the performance further. Abbreviations can be a source of confusion and error even for trained health professionals. For example, the dosing instruction "QD" can be interpreted as either "once a day" or "four times a day". Sparsity may be resolved by increasing the size of the training set and including more instances of difficult PHI types.