A survey of automatic de-identification of longitudinal clinical narratives <s> Evaluation <s> Display Omitted We present a method for automatic de-identification of clinical narratives.We propose and validate a two-pass tagging method to improve PHI entity recognition.We have shown that automated de-identification is comparable to human benchmark. A recent promise to access unstructured clinical data from electronic health records on large-scale has revitalized the interest in automated de-identification of clinical notes, which includes the identification of mentions of Protected Health Information (PHI). We describe the methods developed and evaluated as part of the i2b2/UTHealth 2014 challenge to identify PHI defined by 25 entity types in longitudinal clinical narratives. Our approach combines knowledge-driven (dictionaries and rules) and data-driven (machine learning) methods with a large range of features to address de-identification of specific named entities. In addition, we have devised a two-pass recognition approach that creates a patient-specific run-time dictionary from the PHI entities identified in the first step with high confidence, which is then used in the second pass to identify mentions that lack specific clues. The proposed method achieved the overall micro F1-measures of 91% on strict and 95% on token-level evaluation on the test dataset (514 narratives). Whilst most PHI entities can be reliably identified, particularly challenging were mentions of Organizations and Professions. Still, the overall results suggest that automated text mining methods can be used to reliably process clinical notes to identify personal information and thus providing a crucial step in large-scale de-identification of unstructured data for further clinical and epidemiological studies. <s> BIB001 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Evaluation <s> Display Omitted A model to automatically detect protected health information in clinical texts.Machine learning techniques combined with keywords and rule-based approaches.7 main PHI categories with 25 associated sub-categories are identified.Achieving an overall micro-averaged F-measure of 93.6%.The winner of 2014 i2b2 de-identification challenge task. This paper presents a natural language processing (NLP) system that was designed to participate in the 2014 i2b2 de-identification challenge. The challenge task aims to identify and classify seven main Protected Health Information (PHI) categories and 25 associated sub-categories. A hybrid model was proposed which combines machine learning techniques with keyword-based and rule-based approaches to deal with the complexity inherent in PHI categories. Our proposed approaches exploit a rich set of linguistic features, both syntactic and word surface-oriented, which are further enriched by task-specific features and regular expression template patterns to characterize the semantics of various PHI categories. Our system achieved promising accuracy on the challenge test data with an overall micro-averaged F-measure of 93.6%, which was the winner of this de-identification challenge. <s> BIB002
The NLP system, designed for the 2014 and 2016 competitions, contained seven main PHI categories and 25 associated sub-categories BIB001 ;  ; BIB002 ). The seven main categories were Name, Profession, Location, Contacts, IDs, Age and Dates BIB001 ). Example of the sub-categories includes patient name, doctor's name and username for NAME category (see Tubbs and  for full detail of these categories). The use of sub-categories was only introduced in the 2014 competition, as the 2006 competition just focused on the seven types of PHI. It was argued that HIPAA only provides a starting point for an effective deidentification and that expanding the HIPAA definitions is believed to be more effective in solving the problem ). This is the primary idea behind i2b2-PHI, where 25 subcategories were added to HIPAA-PHI to create i2b2-PHI categories ). In addition to the standard HIPAA-PHI, i2b2-PHI were also used in the competitions as an evaluation tool. De-identification performance is evaluated using Precision (P), Recall (R) and Fmeasure (F1), where, P = true positive rate true positive rate + f alse positive rate R = true positive rate true positive rate + f alse negative rate Evaluations of P, R and F1 were performed at the entity-level and token-level ; ; BIB002 ). For entity-level, an exact match of the gold standard entity with the predicted entity is required. Whereas, for token-level, each token is considered separately. One token of the golden standard entity and the predicted entity is needed to match. Evaluations were performed against the gold standard of i2b2 test data BIB002 ), using strict evaluation and relaxed evaluation. The strict metric involves exact string matching of the predicted entity against the gold standard. The relaxed metric, on the other hand, is approximate string matching with some leeway allowance at the end of a string. The overall performance of the system was measured using both micro and macro evaluation matrics BIB002 ). Macro evaluation calculates the scores for each document and then takes the average across the corpus, and micro evaluation calculates the scores for each annotation across the corpus. This study will focus mainly on the micro-averaged F-measure as this was the primary evaluation measure used for determining the performances of the systems.