A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> To facilitate and survey studies in automatic de-identification, as a part of the i2b2 (Informatics for Integrating Biology to the Bedside) project, authors organized a Natural Language Processing (NLP) challenge on automatically removing private health information (PHI) from medical discharge records. This manuscript provides an overview of this de-identification challenge, describes the data and the annotation process, explains the evaluation metrics, discusses the nature of the systems that addressed the challenge, analyzes the results of received system runs, and identifies directions for future research. The de-indentification challenge data consisted of discharge summaries drawn from the Partners Healthcare system. Authors prepared this data for the challenge by replacing authentic PHI with synthesized surrogates. To focus the challenge on non-dictionary-based de-identification methods, the data was enriched with out-of-vocabulary PHI surrogates, i.e., made up names. The data also included some PHI surrogates that were ambiguous with medical non-PHI terms. A total of seven teams participated in the challenge. Each team submitted up to three system runs, for a total of sixteen submissions. The authors used precision, recall, and F-measure to evaluate the submitted system runs based on their token-level and instance-level performance on the ground truth. The systems with the best performance scored above 98% in F-measure for all categories of PHI. Most out-of-vocabulary PHI could be identified accurately. However, identifying ambiguous PHI proved challenging. The performance of systems on the test data set is encouraging. Future evaluations of these systems will involve larger data sets from more heterogeneous sources. <s> BIB001 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> BackgroundIn the United States, the Health Insurance Portability and Accountability Act (HIPAA) protects the confidentiality of patient data and requires the informed consent of the patient and approval of the Internal Review Board to use data for research purposes, but these requirements can be waived if data is de-identified. For clinical data to be considered de-identified, the HIPAA "Safe Harbor" technique requires 18 data elements (called PHI: Protected Health Information) to be removed. The de-identification of narrative text documents is often realized manually, and requires significant resources. Well aware of these issues, several authors have investigated automated de-identification of narrative text documents from the electronic health record, and a review of recent research in this domain is presented here.MethodsThis review focuses on recently published research (after 1995), and includes relevant publications from bibliographic queries in PubMed, conference proceedings, the ACM Digital Library, and interesting publications referenced in already included papers.ResultsThe literature search returned more than 200 publications. The majority focused only on structured data de-identification instead of narrative text, on image de-identification, or described manual de-identification, and were therefore excluded. Finally, 18 publications describing automated text de-identification were selected for detailed analysis of the architecture and methods used, the types of PHI detected and removed, the external resources used, and the types of clinical documents targeted. All text de-identification systems aimed to identify and remove person names, and many included other types of PHI. Most systems used only one or two specific clinical document types, and were mostly based on two different groups of methodologies: pattern matching and machine learning. Many systems combined both approaches for different types of PHI, but the majority relied only on pattern matching, rules, and dictionaries.ConclusionsIn general, methods based on dictionaries performed better with PHI that is rarely mentioned in clinical text, but are more difficult to generalize. Methods based on machine learning tend to perform better, especially with PHI that is not mentioned in the dictionaries used. Finally, the issues of anonymization, sufficient performance, and "over-scrubbing" are discussed in this publication. <s> BIB002 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> The revolution in health information technology has enabled the compilation and use of large data sets of health records for genomic and other research. Extensive collections of health records, especially those linked with biological specimens, are also extremely valuable for outcomes research, quality assurance, public health surveillance, and other beneficial purposes. The manipulation of large quantities of health information, however, creates substantial challenges for protecting the privacy of patients and research subjects. The strategy of choice for many health care providers and research institutions in dealing with this challenge has been to de-identify individual health information. <s> BIB003 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract Purpose Medical records must often be stripped of patient identifiers, or de-identified, before being shared. De-identification by humans is time-consuming, and existing software is limited in its generality. The open source MITRE Identification Scrubber Toolkit (MIST) provides an environment to support rapid tailoring of automated de-identification to different document types, using automatically learned classifiers to de-identify and protect sensitive information. Methods MIST was evaluated with four classes of patient records from the Vanderbilt University Medical Center: discharge summaries, laboratory reports, letters, and order summaries. We trained and tested MIST on each class of record separately, as well as on pooled sets of records. We measured precision, recall, F -measure and accuracy at the word level for the detection of patient identifiers as designated by the HIPAA Safe Harbor Rule. Results MIST was applied to medical records that differed in the amounts and types of protected health information (PHI): lab reports contained only two types of PHI (dates, names) compared to discharge summaries, which were much richer. Performance of the de-identification tool depended on record class; F -measure results were 0.996 for order summaries, 0.996 for discharge summaries, 0.943 for letters and 0.934 for laboratory reports. Experiments suggest the tool requires several hundred training exemplars to reach an F -measure of at least 0.9. Conclusions The MIST toolkit makes possible the rapid tailoring of automated de-identification to particular document types and supports the transition of the de-identification software to medical end users, avoiding the need for developers to have access to original medical records. We are making the MIST toolkit available under an open source license to encourage its application to diverse data sets at multiple institutions. <s> BIB004 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> BACKGROUND ::: De-identification and anonymization are strategies that are used to remove patient identifiers in electronic health record data. The use of these strategies in multicenter research studies is paramount in importance, given the need to share electronic health record data across multiple environments and institutions while safeguarding patient privacy. ::: ::: ::: METHODS ::: Systematic literature search using keywords of de-identify, deidentify, de-identification, deidentification, anonymize, anonymization, data scrubbing, and text scrubbing. Search was conducted up to June 30, 2011 and involved 6 different common literature databases. A total of 1798 prospective citations were identified, and 94 full-text articles met the criteria for review and the corresponding articles were obtained. Search results were supplemented by review of 26 additional full-text articles; a total of 120 full-text articles were reviewed. ::: ::: ::: RESULTS ::: A final sample of 45 articles met inclusion criteria for review and discussion. Articles were grouped into text, images, and biological sample categories. For text-based strategies, the approaches were segregated into heuristic, lexical, and pattern-based systems versus statistical learning-based systems. For images, approaches that de-identified photographic facial images and magnetic resonance image data were described. For biological samples, approaches that managed the identifiers linked with these samples were discussed, particularly with respect to meeting the anonymization requirements needed for Institutional Review Board exemption under the Common Rule. ::: ::: ::: CONCLUSIONS ::: Current de-identification strategies have their limitations, and statistical learning-based systems have distinct advantages over other approaches for the de-identification of free text. True anonymization is challenging, and further work is needed in the areas of de-identification of datasets and protection of genetic information. <s> BIB005 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract Objectives The aim of this study was to discover the public's attitude and views towards privacy in health care. This is a part of a larger project which aims to gain an insight into what kind of privacy is needed and develop technical measures to provide such privacy. Methods The study was a two-stage process which combined qualitative and quantitative research. Stage One of the study comprised arranging and facilitating focus groups while in Stage Two we conducted a social survey. Measurements We measured attitudes towards privacy, medical research and consent; privacy concern about sharing one's health information for research; privacy concern about the possibility that some specific information from medical records could be linked to the patient's name in a situation that was not related to medical treatment. Results The results of the study revealed both great support for medical research (98%), and concern about privacy of health information (66%). Participants prefer to be asked for their permission before their health information is used for any purpose other than medical treatment (92%), and they would like to know the organisation and details of the research before allowing the use of their health records (83%). Age, level of education, place of birth and employment status are most strongly associated with privacy concerns. The study showed that there are some particularly sensitive issues and there is a concern (42–60%) about any possibility of linking these kinds of data to the patient's name in a situation that is not related to medical treatment. Such issues include sexually transmitted diseases, abortions and infertility, family medical history/genetic disorders, mental illness, drug/alcohol related incidents, lists of previous operations/procedures/dates and current medications. Conclusions Participants believe they should be asked for permission before their health information is used for any purpose other than medical treatment. However, consent and privacy concerns are not necessary related. Assuring individuals that their personal health information is de-identified reduces their concern about the necessity of consent for releasing health information for research purposes, but many people are not aware that removing their names and other direct identifiers from medical records does not guarantee full privacy protection for their health information. Privacy concerns decrease as extra security measures are introduced to protect privacy. Therefore, instead of "tailoring concern" as proposed by Willison [1] we suggest improving privacy protection of personal information by introducing additional security measures in data publishing. <s> BIB006 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Objective (1) To evaluate a state-of-the-art natural language processing (NLP)-based approach to automatically de-identify a large set of diverse clinical notes. (2) To measure the impact of de-identification on the performance of information extraction algorithms on the de-identified documents. Material and methods A cross-sectional study that included 3503 stratified, randomly selected clinical notes (over 22 note types) from five million documents produced at one of the largest US pediatric hospitals. Sensitivity, precision, F value of two automated deidentification systems for removing all 18 HIPAA-defined protected health information elements were computed. Performance was assessed against a manually generated ‘gold standard’. Statistical significance was tested. The automated de-identification performance was also compared with that of two humans on a 10% subsample of the gold standard. The effect of deidentification on the performance of subsequent medication extraction was measured. Results The gold standard included 30815 protected health information elements and more than one million tokens. The most accurate NLP method had 91.92% sensitivity (R) and 95.08% precision (P) overall. The performance of the system was indistinguishable from that of human annotators (annotators’ performance was 92.15%(R)/93.95%(P) and 94.55%(R)/88.45%(P) overall while the best system obtained 92.91%(R)/95.73%(P) on same text). The impact of automated de-identification was minimal on the utility of the narrative notes for subsequent information extraction as measured by the sensitivity and precision of medication name extraction. Discussion and conclusion NLP-based deidentification shows excellent performance that rivals the performance of human annotators. Furthermore, unlike manual de-identification, the automated approach scales up to millions of documents quickly and inexpensively. This paper studied automated de-identification of clinical narrative text using natural language processing (NLP)-based methods. The specific aims were (1) to evaluate a state-of-the-art NLP-based approach to automatically de-identify a large set of diverse clinical notes for all HIPAA (Health Insurance Portability and Accountability Act)-defined protected health information (PHI) elements and (2) to measure the impact of de-identification on the performance of information extraction (IE) algorithms executed on the de-identified documents. In addition, we hope that our studydby contrasting the performance of human and automated de-identificationdwill shape policy expectations. <s> BIB007 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract Objective Evaluate the effectiveness and robustness of Anonym, a tool for de-identifying free-text health records based on conditional random fields classifiers informed by linguistic and lexical features, as well as features extracted by pattern matching techniques. De-identification of personal health information in electronic health records is essential for the sharing and secondary usage of clinical data. De-identification tools that adapt to different sources of clinical data are attractive as they would require minimal intervention to guarantee high effectiveness. Methods and materials The effectiveness and robustness of Anonym are evaluated across multiple datasets, including the widely adopted Integrating Biology and the Bedside (i2b2) dataset, used for evaluation in a de-identification challenge. The datasets used here vary in type of health records, source of data, and their quality, with one of the datasets containing optical character recognition errors. Results Anonym identifies and removes up to 96.6% of personal health identifiers (recall) with a precision of up to 98.2% on the i2b2 dataset, outperforming the best system proposed in the i2b2 challenge. The effectiveness of Anonym across datasets is found to depend on the amount of information available for training. Conclusion Findings show that Anonym compares to the best approach from the 2006 i2b2 shared task. It is easy to retrain Anonym with new datasets; if retrained, the system is robust to variations of training size, data type and quality in presence of sufficient training data. <s> BIB008 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract Purpose Medical free-text records enable to get rich information about the patients, but often need to be de-identified by removing the Protected Health Information (PHI), each time the identification of the patient is not mandatory. Pattern matching techniques require pre-defined dictionaries, and machine learning techniques require an extensive training set. Methods exist in French, but either bring weak results or are not freely available. The objective is to define and evaluate FASDIM, a Fast And Simple De-Identification Method for French medical free-text records. Methods FASDIM consists in removing all the words that are not present in the authorized word list, and in removing all the numbers except those that match a list of protection patterns. The corresponding lists are incremented in the course of the iterations of the method. For the evaluation, the workload is estimated in the course of records de-identification. The efficiency of the de-identification is assessed by independent medical experts on 508 discharge letters that are randomly selected and de-identified by FASDIM. Finally, the letters are encoded after and before de-identification according to 3 terminologies (ATC, ICD10, CCAM) and the codes are compared. Results The construction of the list of authorized words is progressive: 12h for the first 7000 letters, 16 additional hours for 20,000 additional letters. The Recall (proportion of removed Protected Health Information, PHI) is 98.1%, the Precision (proportion of PHI within the removed token) is 79.6% and the F -measure (harmonic mean) is 87.9%. In average 30.6 terminology codes are encoded per letter, and 99.02% of those codes are preserved despite the de-identification. Conclusion FASDIM gets good results in French and is freely available. It is easy to implement and does not require any predefined dictionary. <s> BIB009 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract As more and more electronic clinical information is becoming easier to access for secondary uses such as clinical research, approaches that enable faster and more collaborative research while protecting patient privacy and confidentiality are becoming more important. Clinical text de-identification offers such advantages but is typically a tedious manual process. Automated Natural Language Processing (NLP) methods can alleviate this process, but their impact on subsequent uses of the automatically de-identified clinical narratives has only barely been investigated. In the context of a larger project to develop and investigate automated text de-identification for Veterans Health Administration (VHA) clinical notes, we studied the impact of automated text de-identification on clinical information in a stepwise manner. Our approach started with a high-level assessment of clinical notes informativeness and formatting, and ended with a detailed study of the overlap of select clinical information types and Protected Health Information (PHI). To investigate the informativeness (i.e., document type information, select clinical data types, and interpretation or conclusion) of VHA clinical notes, we used five different existing text de-identification systems. The informativeness was only minimally altered by these systems while formatting was only modified by one system. To examine the impact of de-identification on clinical information extraction, we compared counts of SNOMED-CT concepts found by an open source information extraction application in the original (i.e., not de-identified) version of a corpus of VHA clinical notes, and in the same corpus after de-identification. Only about 1.2–3% less SNOMED-CT concepts were found in de-identified versions of our corpus, and many of these concepts were PHI that was erroneously identified as clinical information. To study this impact in more details and assess how generalizable our findings were, we examined the overlap between select clinical information annotated in the 2010 i2b2 NLP challenge corpus and automatic PHI annotations from our best-of-breed VHA clinical text de-identification system (nicknamed ‘BoB’). Overall, only 0.81% of the clinical information exactly overlapped with PHI, and 1.78% partly overlapped. We conclude that automated text de-identification’s impact on clinical information is small, but not negligible, and that improved clinical acronyms and eponyms disambiguation could significantly reduce this impact. <s> BIB010 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Preparing narrative medical records for use outside of their originating institutions requires that protected health information (PHI) be removed from the records. If researchers intend to use these records for natural language processing, then preparing the medical documents requires two steps: (1) identifying the PHI and (2) replacing the PHI with realistic surrogates. In this chapter we discuss the challenges associated with generating these realistic surrogates and describe the algorithms we used to prepare the 2014 i2b2/UTHealth shared task corpus for distribution and use in a natural language processing task focused on de-identification. <s> BIB011 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted We present a method for automatic de-identification of clinical narratives.We propose and validate a two-pass tagging method to improve PHI entity recognition.We have shown that automated de-identification is comparable to human benchmark. A recent promise to access unstructured clinical data from electronic health records on large-scale has revitalized the interest in automated de-identification of clinical notes, which includes the identification of mentions of Protected Health Information (PHI). We describe the methods developed and evaluated as part of the i2b2/UTHealth 2014 challenge to identify PHI defined by 25 entity types in longitudinal clinical narratives. Our approach combines knowledge-driven (dictionaries and rules) and data-driven (machine learning) methods with a large range of features to address de-identification of specific named entities. In addition, we have devised a two-pass recognition approach that creates a patient-specific run-time dictionary from the PHI entities identified in the first step with high confidence, which is then used in the second pass to identify mentions that lack specific clues. The proposed method achieved the overall micro F1-measures of 91% on strict and 95% on token-level evaluation on the test dataset (514 narratives). Whilst most PHI entities can be reliably identified, particularly challenging were mentions of Organizations and Professions. Still, the overall results suggest that automated text mining methods can be used to reliably process clinical notes to identify personal information and thus providing a crucial step in large-scale de-identification of unstructured data for further clinical and epidemiological studies. <s> BIB012 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted The method used in our work is completely machine-learning-based.Pattern-matching techniques are used in the preprocessing module.Three groups of features are extracted to train the de-identifier model.Missing errors are the main component of the errors in system output. De-identification is a shared task of the 2014 i2b2/UTHealth challenge. The purpose of this task is to remove protected health information (PHI) from medical records. In this paper, we propose a novel de-identifier, WI-deId, based on conditional random fields (CRFs). A preprocessing module, which tokenizes the medical records using regular expressions and an off-the-shelf tokenizer, is introduced, and three groups of features are extracted to train the de-identifier model. The experiment shows that our system is effective in the de-identification of medical records, achieving a micro-F1 of 0.9232 at the i2b2 strict entity evaluation level. <s> BIB013 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted We proposed a hybrid system to automatically de-identify electronic medical records.PHIs are identified by token-level and character-level conditional random fields.The character-level CRFs is used to avoid boundary errors caused by tokenization.Our system achieves an F-score of 91.24% on 2014 i2b2 corpus, which is top-ranked. De-identification, identifying and removing all protected health information (PHI) present in clinical data including electronic medical records (EMRs), is a critical step in making clinical data publicly available. The 2014 i2b2 (Center of Informatics for Integrating Biology and Bedside) clinical natural language processing (NLP) challenge sets up a track for de-identification (track 1). In this study, we propose a hybrid system based on both machine learning and rule approaches for the de-identification track. In our system, PHI instances are first identified by two (token-level and character-level) conditional random fields (CRFs) and a rule-based classifier, and then are merged by some rules. Experiments conducted on the i2b2 corpus show that our system submitted for the challenge achieves the highest micro F-scores of 94.64%, 91.24% and 91.63% under the "token", "strict" and "relaxed" criteria respectively, which is among top-ranked systems of the 2014 i2b2 challenge. After integrating some refined localization dictionaries, our system is further improved with F-scores of 94.83%, 91.57% and 91.95% under the "token", "strict" and "relaxed" criteria respectively. <s> BIB014 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted A model to automatically detect protected health information in clinical texts.Machine learning techniques combined with keywords and rule-based approaches.7 main PHI categories with 25 associated sub-categories are identified.Achieving an overall micro-averaged F-measure of 93.6%.The winner of 2014 i2b2 de-identification challenge task. This paper presents a natural language processing (NLP) system that was designed to participate in the 2014 i2b2 de-identification challenge. The challenge task aims to identify and classify seven main Protected Health Information (PHI) categories and 25 associated sub-categories. A hybrid model was proposed which combines machine learning techniques with keyword-based and rule-based approaches to deal with the complexity inherent in PHI categories. Our proposed approaches exploit a rich set of linguistic features, both syntactic and word surface-oriented, which are further enriched by task-specific features and regular expression template patterns to characterize the semantics of various PHI categories. Our system achieved promising accuracy on the challenge test data with an overall micro-averaged F-measure of 93.6%, which was the winner of this de-identification challenge. <s> BIB015 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted We used natural language processing (NLP) to extract heart disease risk factors.Components were adapted from two existing NLP applications.We used existing tools without feature engineering or re-training of models.Our system achieved an overall micro-averaged F1-measure of 87.47%.Adapting existing tools allowed for performance comparable to sophisticated systems. The 2014 i2b2 natural language processing shared task focused on identifying cardiovascular risk factors such as high blood pressure, high cholesterol levels, obesity and smoking status among other factors found in health records of diabetic patients. In addition, the task involved detecting medications, and time information associated with the extracted data. This paper presents the development and evaluation of a natural language processing (NLP) application conceived for this i2b2 shared task. For increased efficiency, the application main components were adapted from two existing NLP tools implemented in the Apache UIMA framework: Textractor (for dictionary-based lookup) and cTAKES (for preprocessing and smoking status detection). The application achieved a final (micro-averaged) F1-measure of 87.5% on the final evaluation test set. Our attempt was mostly based on existing tools adapted with minimal changes and allowed for satisfying performance with limited development efforts. <s> BIB016 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> The second track of the 2014 i2b2/UTHealth natural language processing shared task focused on identifying medical risk factors related to Coronary Artery Disease (CAD) in the narratives of longitudinal medical records of diabetic patients. The risk factors included hypertension, hyperlipidemia, obesity, smoking status, and family history, as well as diabetes and CAD, and indicators that suggest the presence of those diseases. In addition to identifying the risk factors, this track of the 2014 i2b2/UTHealth shared task studied the presence and progression of the risk factors in longitudinal medical records. Twenty teams participated in this track, and submitted 49 system runs for evaluation. Six of the top 10 teams achieved F1 scores over 0.90, and all 10 scored over 0.87. The most successful system used a combination of additional annotations, external lexicons, hand-written rules and Support Vector Machines. The results of this track indicate that identification of risk factors and their progression over time is well within the reach of automated systems. <s> BIB017 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted We introduce a novel use of non-parametric Bayesian HMM for de-identification.The paper gives a thorough discussion of the motivation of designing the model.Our model understands local context cues without significant feature engineering.The model offers competitive performance comparing to the state-of-the-art CRF model. For the 2014 i2b2/UTHealth de-identification challenge, we introduced a new non-parametric Bayesian hidden Markov model using a Dirichlet process (HMM-DP). The model intends to reduce task-specific feature engineering and to generalize well to new data. In the challenge we developed a variational method to learn the model and an efficient approximation algorithm for prediction. To accommodate out-of-vocabulary words, we designed a number of feature functions to model such words. The results show the model is capable of understanding local context cues to make correct predictions without manual feature engineering and performs as accurately as state-of-the-art conditional random field models in a number of categories. To incorporate long-range and cross-document context cues, we developed a skip-chain conditional random field model to align the results produced by HMM-DP, which further improved the performance. <s> BIB018 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted NLP task focused on identifying risk factors over time in clinical narratives.Corpus of 1304 longitudinal medical records for 296 patients."Light" annotation task for domain expert annotators.Gold standard created through voting.Corpus used for track 2 of 2014 i2b2/UTHealth NLP Shared Task. The 2014 i2b2/UTHealth natural language processing shared task featured a track focused on identifying risk factors for heart disease (specifically, Cardiac Artery Disease) in clinical narratives. For this track, we used a "light" annotation paradigm to annotate a set of 1304 longitudinal medical records describing 296 patients for risk factors and the times they were present. We designed the annotation task for this track with the goal of balancing annotation load and time with quality, so as to generate a gold standard corpus that can benefit a clinically-relevant task. We applied light annotation procedures and determined the gold standard using majority voting. On average, the agreement of annotators with the gold standard was above 0.95, indicating high reliability. The resulting document-level annotations generated for each record in each longitudinal EMR in this corpus provide information that can support studies of progression of heart disease risk factors in the included patients over time. These annotations were used in the Risk Factor track of the 2014 i2b2/UTHealth shared task. Participating systems achieved a mean micro-averaged F1 measure of 0.815 and a maximum F1 measure of 0.928 for identifying these risk factors in patient records. <s> BIB019 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Display Omitted New corpus of 1304 de-identified medical records.Longitudinal records represent 296 patients.Three patient cohorts: with Coronary Artery Disease (CAD), no CAD, develop CAD.Overview of novel uses for this corpus. The 2014 i2b2/UTHealth Natural Language Processing (NLP) shared task featured a new longitudinal corpus of 1304 records representing 296 diabetic patients. The corpus contains three cohorts: patients who have a diagnosis of coronary artery disease (CAD) in their first record, and continue to have it in subsequent records; patients who do not have a diagnosis of CAD in the first record, but develop it by the last record; patients who do not have a diagnosis of CAD in any record. This paper details the process used to select records for this corpus and provides an overview of novel research uses for this corpus. This corpus is the only annotated corpus of longitudinal clinical narratives currently available for research to the general research community. <s> BIB020 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> In recent years, sharing electronic medical records (EMRs) for more researchers outside the associated institutions is significant. For privacy preservation of the corresponding patients and the associated institutions, a de-identification task on the EMRs to be shared is a must. Although the deidentification task has been considered with positive research outcomes worldwide, especially those from the i2b2 (Informatics for Integrating Biology and the Bedside) shared tasks in 2006 and 2014, the task has not yet been a solved problem and still needs more investigation realistically. In this paper, we propose an automatic de-identification solution in a multilevel hybrid semi-supervised learning paradigm with a key focus on correctly identifying protected health information (PHI) in the EMRs. Similar to the existing works, our work defines a hybrid approach by combining a machine learning-based method with a conditional random fields model and a rule-based method in a post-processing phase to handle the PHI types with disambiguity. Nevertheless, our work is more general and practical. First, it considers the structure complexity of each EMR so that each section can be treated properly for more correct PHI identification up to its structure complexity: structured, semi-structured, or un-structured. Second, each EMR is then examined in our approach at three different levels of granularity such as a token level in the supervised learning phase, an entity level in the rule-based post-processing phase, and a section level along with the structure complexity in the semi-supervised learning phase. Many various detail levels will give our approach a deeper look at each EMR for more effectiveness. Third, our solution is conducted in a self-training manner so that it can get started with a small annotated data set in practice and get more effective with new EMRs over time. Evaluated with the i2b2 data set in comparison with the related works, our solution is effective with better F-measure values for the AGE, LOCATION, and PHONE PHI types and comparable for the other PHI types. <s> BIB021 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract Clinical narratives (the text notes found in patients’ medical records) are important information sources for secondary use in research. However, in order to protect patient privacy, they must be de-identified prior to use. Manual de-identification is considered to be the gold standard approach but is tedious, expensive, slow, and impractical for use with large-scale clinical data. Automated or semi-automated de-identification using computer algorithms is a potentially promising alternative. The Informatics Institute of the University of Alabama at Birmingham is applying de-identification to clinical data drawn from the UAB hospital’s electronic medical records system before releasing them for research. We participated in a shared task challenge by the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) at the de-identification regular track to gain experience developing our own automatic de-identification tool. We focused on the popular and successful methods from previous challenges: rule-based, dictionary-matching, and machine-learning approaches. We also explored new techniques such as disambiguation rules, term ambiguity measurement, and used multi-pass sieve framework at a micro level. For the challenge’s primary measure (strict entity), our submissions achieved competitive results (f-measures: 87.3%, 87.1%, and 86.7%). For our preferred measure (binary token HIPAA), our submissions achieved superior results (f-measures: 93.7%, 93.6%, and 93%). With those encouraging results, we gain the confidence to improve and use the tool for the real de-identification task at the UAB Informatics Institute. <s> BIB022 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Objective: Patient notes in electronic health records (EHRs) may contain critical information for medical investigations. However, the vast majority of medical investigators can only access de-identified notes, in order to protect the confidentiality of patients. In the United States, the Health Insurance Portability and Accountability Act (HIPAA) defines 18 types of protected health information (PHI) that needs to be removed to de-identify patient notes. Manual de-identification is impractical given the size of EHR databases, the limited number of researchers with access to the non-de-identified notes, and the frequent mistakes of human annotators. A reliable automated de-identification system would consequently be of high value. ::: Materials and Methods: We introduce the first de-identification system based on artificial neural networks (ANNs), which requires no handcrafted features or rules, unlike existing systems. We compare the performance of the system with state-of-the-art systems on two datasets: the i2b2 2014 de-identification challenge dataset, which is the largest publicly available de-identification dataset, and the MIMIC de-identification dataset, which we assembled and is twice as large as the i2b2 2014 dataset. ::: Results: Our ANN model outperforms the state-of-the-art systems. It yields an F1-score of 97.85 on the i2b2 2014 dataset, with a recall 97.38 and a precision of 97.32, and an F1-score of 99.23 on the MIMIC de-identification dataset, with a recall 99.25 and a precision of 99.06. ::: Conclusion: Our findings support the use of ANNs for de-identification of patient notes, as they show better performance than previously published systems while requiring no feature engineering. <s> BIB023 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Abstract De-identification, or identifying and removing protected health information (PHI) from clinical data, is a critical step in making clinical data available for clinical applications and research. This paper presents a natural language processing system for automatic de-identification of psychiatric notes, which was designed to participate in the 2016 CEGS N-GRID shared task Track 1. The system has a hybrid structure that combines machine leaning techniques and rule-based approaches. The rule-based components exploit the structure of the psychiatric notes as well as characteristic surface patterns of PHI mentions. The machine learning components utilize supervised learning with rich features. In addition, the system performance was boosted with integration of additional data to the training set through domain adaptation. The hybrid system showed overall micro-averaged F-score 90.74 on the test set, second-best among all the participants of the CEGS N-GRID task. <s> BIB024 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Introduction <s> Entity recognition is one of the most primary steps for text analysis and has long attracted considerable attention from researchers. In the clinical domain, various types of entities, such as clinical entities and protected health information (PHI), widely exist in clinical texts. Recognizing these entities has become a hot topic in clinical natural language processing (NLP), and a large number of traditional machine learning methods, such as support vector machine and conditional random field, have been deployed to recognize entities from clinical texts in the past few years. In recent years, recurrent neural network (RNN), one of deep learning methods that has shown great potential on many problems including named entity recognition, also has been gradually used for entity recognition from clinical texts. In this paper, we comprehensively investigate the performance of LSTM (long-short term memory), a representative variant of RNN, on clinical entity recognition and protected health information recognition. The LSTM model consists of three layers: input layer – generates representation of each word of a sentence; LSTM layer – outputs another word representation sequence that captures the context information of each word in this sentence; Inference layer – makes tagging decisions according to the output of LSTM layer, that is, outputting a label sequence. Experiments conducted on corpora of the 2010, 2012 and 2014 i2b2 NLP challenges show that LSTM achieves highest micro-average F1-scores of 85.81% on the 2010 i2b2 medical concept extraction, 92.29% on the 2012 i2b2 clinical event detection, and 94.37% on the 2014 i2b2 de-identification, which is considerably competitive with other state-of-the-art systems. LSTM that requires no hand-crafted feature has great potential on entity recognition from clinical texts. It outperforms traditional machine learning methods that suffer from fussy feature engineering. A possible future direction is how to integrate knowledge bases widely existing in the clinical domain into LSTM, which is a case of our future work. Moreover, how to use LSTM to recognize entities in specific formats is also another possible future direction. <s> BIB025
Health records are an extremely valuable but arguably underutilised source of health information for researchers. Such research is made possible by the widespread adaptation of electronic health records (EHR) in routine health care BIB021 ). These records create large and continuously updated datasets which can then be subjected to powerful analytic techniques. De-identification of electronic health records (EHR) is the process of identifying and CONTACT Vithya Yogarajan. Email: vyogaraj@waikato.ac.nz removing any information that will reveal patient identity from medical records. It is a method for protecting patient confidentiality and privacy before the use of EHR for research purposes. This avoids the need for individual patient consent before medical records (text or otherwise) are used in research. An automatic de-identification system allows researchers to use all the medical data sources available and utilise data-driven models to improve health outcomes. The de-identification of EHR consists of two stages. The first stage is to identify protected health information (PHI). The second stage is to replace the identified PHI with realistic surrogates. Figure 1 provides an example of EHR before and after deidentification. From this example it is evident PHI such as hospital name, patient name, date etc. are replaced with realistic surrogates. As illustrated in Figure 1 , after de-identification the structure and readability of the original text document should still be maintained BIB011 ). One of the significant difficulties in de-identifying the records is complying with the legal requirements. Rules and regulations are provided by the Health Insurance Portability Accountability Act (HIPAA) to de-identify EHR  ; ). In New Zealand, patient privacy is protected by the Health Information Privacy Code and the Health and Disability Consumers' Code of Rights (Commissioner (2009) ; Office of the Privacy Commissioner (2013)). Although there is no one standard set of rules and methods for de-identification, the HIPAA Privacy Rule is arguably the worldwide gold standard  ). HIPPA considers Expert Determination and Safe Harbor to be the two acceptable methods of de-identification. Expert Determination relies on a person with specialist knowledge and experience to ensure only a minimal, statistically acceptable level of risk of identifying an individual is present  ). Organisations such as the Institute of Medicine (IOM), the Health Information Trust Alliance (HITRUST), the Pharmaceutical Users Software Exchange (PhUSE) and the Council of Canadian Academics, have adapted expert determination standards for de-identification  ). The Safe Harbour method provides guidelines on 18 categories of PHI and requires all information that identifies the patient be to be removed before records are shared outside the clinical setting ;  ). These categories are presented in Table 1 . The primary challenge for de-identification of unstructured clinical data is that while identifying and replacing PHIs the integrity of data needs to be protected BIB005 ; BIB002 ; BIB001 ). This needs to be achieved in the presence of ambiguities (such as lexical overlap between PHI and non-PHI) and out-of-vocabulary words (such as misspelt and foreign words in PHIs) BIB001 ). For example, while the de-identifying name, it is essential to recognise the difference between the patient name, doctor's name and a spouse (or family member) name that may also be in the EHR. There is a need to be consistent while generating surrogates. This issue is even more complicated when the data is longitudinal of nature. Another major issue is that studies have shown that even with very few entities (such as gender, date of birth, ZIP code, and address over a time) reconstructions of identity is possible BIB003 ). For example, Governor Weld of Massachusetts had his health records identified from "anonymised" records of state employee hospital visits BIB003 ). Patients are concerned about the privacy and confidentiality of their health information if it is used for research, especially concerning sensitive items such as sexual history, genetic disease and mental health BIB006 ). However, these concerns decrease when demonstrable measures are taken to protect privacy BIB006 ). De-identification of EHR can be done manually or automatically. Manual deidentification is time consuming, tedious, costly, can only be done by a restricted set of individuals allowed access to the original patient notes, and is subject to human error BIB022 ; BIB012 ; BIB023 ; BIB013 ). There have been many attempts to develop an automatic system that will de-identify EHR with certainty. Several systems have been proposed, but they are not frequently used in practice. These include Anonym software BIB008 ), MITRE's MIST tool BIB004 ), CCHMC BIB007 ), CRATE (Cardinal (2017) ) and FASDIM BIB009 ). In addition, BIB002 BIB002 ) presents an overview of several other de-identification systems that are available. Techniques used for de-identification of EHR fit into three categories: rule-based, machine learning based and hybrid, which is a combination of both rule-based and machine learning-based. Rule-based techniques use dictionaries and hand-coded rules, and rely on patterns expressed as regular expressions that are defined and tuned by humans BIB022 ; ; BIB024 ). Rule-based systems also require an expert to provide knowledge of these rules. They do not need significant training data and are easily modifiable. Compared to machine learning approaches, rule-based approaches can distinguish some ambiguous instances, such as the difference between medical record numbers and phone numbers BIB014 ). However, rule-based methods have limitations, as they require additional data curation or annotation by domain experts ; BIB015 ). Also, with rule-based systems, the rules in one system are not easily adoptable by another system ). Machine learning approaches, on the other hand, can be trained to recognise PHI patterns automatically. The most popular machine learning techniques used are conditional random fields (CRF) BIB022 ; BIB012 ; BIB013 ; BIB014 ; BIB015 ), support vector machines BIB016 ; BIB017 ), hidden Markov models BIB018 ), and neural networks ). However, the main downside to machine learning techniques is that they require significantly large labelled datasets as well as feature engineering ; BIB015 ). Machine learning systems perform worse than rule-based systems on rare types of PHI due to a dearth of training data ). Hybrid systems, combining rules and machine learning, are currently the most popular and most effective systems BIB012 ; ; BIB014 ; ; ; BIB015 ). An initial impression is that de-identification of EHR is a relatively straightforward problem. The basic idea of de-identification is simple, and as mentioned above there are many tools and software programs available. However, in reality, it is an essential and currently unsolved problem in the medical informatics domain. The main reason is that the accuracy, precision, robustness and adaptability of the available open source tools and systems are mediocre BIB013 ; BIB002 ; BIB010 ; ; BIB019 ;  ;  ). Examples of issues that complicate developing a well-performing de-identification system that adapts to change include the nature, source, and type of dataset; the time frame of the dataset; the intensity of PHI in the dataset; the kind of PHI; free-form text vs formally defined texts. Unfortunately, most systems -BoB, MIT, MIST, HIDE, HMS, MeDScan misidentify clinical information as PHI BIB010 ). Ambiguous medical terms require more accurate de-identification techniques BIB010 ). Also, most of the open source software only target specific PHI and not all of them BIB002 ). It is argued that perfection cannot be achieved; however, 95% accuracy is considered the rule of thumb and an acceptable value for accuracy ; ). In the past five years, there have been more than 100 papers published relating to the de-identification problem. One of the main reasons for the recent development in the literature is the EHR de-identification competitions. They provide an excellent platform to achieve better results, and the data is open source BIB022 ; ; Uzuner (2015a, 2017) ). There have been three main competitions relating to de-identification of EHR: the 2006 Informatics for Integrating Biology and the Bedside (i2b2) competition BIB020 ;  ); the 2014 i2b2/UTHealth shared task BIB013 ; ; BIB019 ;  ); and the 2016 Centers of Excellence in Genomic Science (CEGS) and Neuropsychiatric Genome-Scale and RDOC Individualized Domain (N-GRID) shared task BIB022 BIB025 ; ; ). These competitions also included multiple tracks focusing on issues such as smoking status classification (in 2006); coronary artery disease (CAD) classification for diabetic patients (in 2014); classification of specific symptom severity based on the psychiatric evaluation (in 2016). Due to the abundance of literature in the recent years, this study restricts itself to the most recent work and provides an overview of the 2014 i2b2/UTHealth shared task, and 2016 CEGS N-GRID shared the task. This study primarily focuses on the trend changes noticed in the techniques used to develop de-identification systems. To the best of our knowledge, this paper is the most recent review paper since Meytre et al., (2010) BIB002 ). The rest of the paper is structured such that an overview of the 2014 and 2016 competitions is presented in section 2, resulting de-identification systems in section 3, error analysis in section 4 and finally discussions in section 5.