A survey of automatic de-identification of longitudinal clinical narratives <s> The 2016 CEGS N-GRID Shared Task <s> Abstract Clinical narratives (the text notes found in patients’ medical records) are important information sources for secondary use in research. However, in order to protect patient privacy, they must be de-identified prior to use. Manual de-identification is considered to be the gold standard approach but is tedious, expensive, slow, and impractical for use with large-scale clinical data. Automated or semi-automated de-identification using computer algorithms is a potentially promising alternative. The Informatics Institute of the University of Alabama at Birmingham is applying de-identification to clinical data drawn from the UAB hospital’s electronic medical records system before releasing them for research. We participated in a shared task challenge by the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) at the de-identification regular track to gain experience developing our own automatic de-identification tool. We focused on the popular and successful methods from previous challenges: rule-based, dictionary-matching, and machine-learning approaches. We also explored new techniques such as disambiguation rules, term ambiguity measurement, and used multi-pass sieve framework at a micro level. For the challenge’s primary measure (strict entity), our submissions achieved competitive results (f-measures: 87.3%, 87.1%, and 86.7%). For our preferred measure (binary token HIPAA), our submissions achieved superior results (f-measures: 93.7%, 93.6%, and 93%). With those encouraging results, we gain the confidence to improve and use the tool for the real de-identification task at the UAB Informatics Institute. <s> BIB001 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> The 2016 CEGS N-GRID Shared Task <s> Abstract Background : Applications of natural language processing to mental health notes are not common given the sensitive nature of the associated narratives. The CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing (NLP) changed this scenario by providing the first set of neuropsychiatric notes to participants. This study summarizes our efforts and results in proposing a novel data use case for this dataset as part of the third track in this shared task. Objective : We explore the feasibility and effectiveness of predicting a set of common mental conditions a patient has based on the short textual description of patient’s history of present illness typically occurring in the beginning of a psychiatric initial evaluation note. Materials and methods : We clean and process the 1000 records made available through the N-GRID clinical NLP task into a key-value dictionary and build a dataset of 986 examples for which there is a narrative for history of present illness as well as Yes/No responses with regards to presence of specific mental conditions. We propose two independent deep neural network models: one based on convolutional neural networks (CNN) and another based on recurrent neural networks with hierarchical attention (ReHAN), the latter of which allows for interpretation of model decisions. We conduct experiments to compare these methods to each other and to baselines based on linear models and named entity recognition (NER). Results : Our CNN model with optimized thresholding of output probability estimates achieves best overall mean micro-F score of 63.144% for 11 common mental conditions with statistically significant gains ( p 0.05 ) over all other models. The ReHAN model with interpretable attention mechanism scored 61.904% mean micro-F1 score. Both models’ improvements over baseline models (support vector machines and NER) are statistically significant. The ReHAN model additionally aids in interpretation of the results by surfacing important words and sentences that lead to a particular prediction for each instance. Conclusions : Although the history of present illness is a short text segment averaging 300 words, it is a good predictor for a few conditions such as anxiety, depression, panic disorder, and attention deficit hyperactivity disorder. Proposed CNN and RNN models outperform baseline approaches and complement each other when evaluating on a per-label basis. <s> BIB002
The 2016 CEGS N-GRID shared task used psychiatric data, making it the first ever competition to release psychiatric intake records ; ). Psychiatric data much more challenging to de-identify, as they contain higher density of PHIs, and is one of the main reasons why mental health notes are not commonly used for research purposes BIB001 ; ; BIB002 ;  ). The 2016 competition included three tracks, of which the first track, track 1A and 1B, is the focus here. Track 1A was designed to evaluate existing systems from 2014 on the 2016 data without any further training or modification. It was designed to test how well the systems performed on, and adapted to, data that is different from the system's original training data ;  ). The best performing system on this data for track 1A was from the Harbin Institute of Technology Shenzhen Graduate School team, with an F-measure of 0.7985 ). It is important to note that the fifth best team scored 0.5266 and seventh team 0.1861; therefore the spread was vast. In general, most of the existing systems submitted to this competition did not perform well on a new dataset ). One potential reason for this is the nature of the dataset, as a psychiatric dataset has more personal information and more text. However, it is a good start at building models that can be tuned to new data ). Track 1B was designed as a traditional evaluation system, and the systems were trained and developed on the psychiatric dataset ). The best performing system for track 1B was from the Harbin Institute of Technology Shenzhen Graduate School again team with an F-measure of 0.9143 ). Seven of the top teams had an F-measure higher than 0.8 with the tenth best team having a score of 0.7175 ). Nearly all teams used hybrid systems for track 1B. Different PHI categories were detected by different systems (rules or machine learning) and were combined at the end to address all PHI categories. For example, some systems took the approach of using combinations of CRF addressing different PHI, and others combined CRF with bi-directional long-short-term memory systems ). The top-performing teams made use of neural networks ). Teams found PHI categories PROFESSION, LOCATION and ID to be the most difficult categories, and DATE and AGE were the easiest to detect ).