A survey of automatic de-identification of longitudinal clinical narratives <s> Summary of the competitions and de-identification systems <s> Preface. Part I: Overview Articles. 1. Learning to Learn: Introduction and Overview S. Thrun, L. Pratt. 2. A Survey of Connectionist Network Reuse Through Transfer L. Pratt, B. Jennings. 3. Transfer in Cognition A. Robins. Part II: Prediction. 4. Theoretical Models of Learning to Learn J. Baxter. 5. Multitask Learning R. Caruana. 6. Making a Low-Dimensional Representation Suitable for Diverse Tasks N. Intrator, S. Edelman. 7. The Canonical Distortion Measure for Vector Quantization and Function Approximation J. Baxter. 8. Lifelong Learning Algorithms S. Thrun. Part III: Relatedness. 9. The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness D.L. Silver, R.E. Mercer. 10. Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge S. Thrun, J. O'Sullivan. Part IV: Control. 11. CHILD: A First Step Towards Continual Learning M.B. Ring. 12. Reinforcement Learning with Self-Modifying Policies J. Schmidhuber, et al. 13. Creating Advice-Taking Reinforcement Learners R. Maclin, J.W. Shavlik. Contributing Authors. Index. <s> BIB001 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Summary of the competitions and de-identification systems <s> Display Omitted NLP task focused on identifying risk factors over time in clinical narratives.Corpus of 1304 longitudinal medical records for 296 patients."Light" annotation task for domain expert annotators.Gold standard created through voting.Corpus used for track 2 of 2014 i2b2/UTHealth NLP Shared Task. The 2014 i2b2/UTHealth natural language processing shared task featured a track focused on identifying risk factors for heart disease (specifically, Cardiac Artery Disease) in clinical narratives. For this track, we used a "light" annotation paradigm to annotate a set of 1304 longitudinal medical records describing 296 patients for risk factors and the times they were present. We designed the annotation task for this track with the goal of balancing annotation load and time with quality, so as to generate a gold standard corpus that can benefit a clinically-relevant task. We applied light annotation procedures and determined the gold standard using majority voting. On average, the agreement of annotators with the gold standard was above 0.95, indicating high reliability. The resulting document-level annotations generated for each record in each longitudinal EMR in this corpus provide information that can support studies of progression of heart disease risk factors in the included patients over time. These annotations were used in the Risk Factor track of the 2014 i2b2/UTHealth shared task. Participating systems achieved a mean micro-averaged F1 measure of 0.815 and a maximum F1 measure of 0.928 for identifying these risk factors in patient records. <s> BIB002 </s> A survey of automatic de-identification of longitudinal clinical narratives <s> Summary of the competitions and de-identification systems <s> Display Omitted A model to automatically detect protected health information in clinical texts.Machine learning techniques combined with keywords and rule-based approaches.7 main PHI categories with 25 associated sub-categories are identified.Achieving an overall micro-averaged F-measure of 93.6%.The winner of 2014 i2b2 de-identification challenge task. This paper presents a natural language processing (NLP) system that was designed to participate in the 2014 i2b2 de-identification challenge. The challenge task aims to identify and classify seven main Protected Health Information (PHI) categories and 25 associated sub-categories. A hybrid model was proposed which combines machine learning techniques with keyword-based and rule-based approaches to deal with the complexity inherent in PHI categories. Our proposed approaches exploit a rich set of linguistic features, both syntactic and word surface-oriented, which are further enriched by task-specific features and regular expression template patterns to characterize the semantics of various PHI categories. Our system achieved promising accuracy on the challenge test data with an overall micro-averaged F-measure of 93.6%, which was the winner of this de-identification challenge. <s> BIB003
De-identification competitions such as that of 2014 and 2016 provide excellent opportunities for researchers to further develop research in this area. One primary reason these competitions are so widely recognised is the open source nature of the data. This paper provided an overview of 2014 and 2016 competitions. Due to the difference in the dataset and the content of the data, both competitions offered exciting developments in this area of research. In general, teams found the de-identification of 2016 dataset more difficult than that of 2014. Also, HIPAA PHI categories were more straightforward to de-identify than that of i2b2 PHI (for 2014 data) or N-GRID PHI (for 2016 data). As explained earlier, i2b2 PHI and N-GRID PHI include additional and more complex categories to the HIPAA regulations. The de-identification competitions argued that HIPAA PHI is not enough for complete de-identification of the EHR, especially for a longitudinal dataset. An example would be the detection of the difference between the patient's name and doctor's name. Although the de-identification of NAME has high F-measure, the differentiation mentioned above has not been achieved successfully. This adds an extra layer of complexity. In the 2014 competition, top systems preferred hybrid approaches with CRF being the most used machine learning approach. Considering ten teams got very high F-measures BIB002 , it is evident that the gold standard had enough information presented. Lack of training examples may be one reason for the PROFESSION category performing so poorly among all systems presented in that competition ; BIB003 ). In the 2016 competition, as in 2014, most of the top systems were hybrid systems with CRF and/or neural networks systems combined with rule-based systems. In 2016, there was a noticeable shift in favouring deep learning techniques with hand corded rules for specific PHIs. The top-performing system in the 2016 competitions achieved a level of accuracy very close to that recorded by the least experienced annotator ). However, it is important to note that 95% rule of thumb for accuracy was not reached in either Task 1A or 1B ). Although there was a greater emphasis on robustness, being able to customise a de-identification system to the data results in much better outcome ). Regarding protecting privacy the three categories that are harder to predict in 2016 competitions are also the ones that have the potential to disclose patient identity ). In both 2014 and 2016 competitions, the best approach was hybrid systems. It is clear that hybrid systems have the potential of using the best of both rules-based and machine learning-based approaches. Categories such as date and contact obtained high F-measure in recently developed systems. Both of these categories also used rules/regular expressions. It is also important to point out that de-identification systems that were developed using the 2014 dataset, post competitions, favoured deep learning only systems, not hybrid systems. There was a noticeable shift towards using neural networks in 2016 competitions and systems developed after the 2014 competitions using 2014 data. Deep learning techniques allow learning representations that are tailored to the task at hand, in this case, de-identification. These representations have shown to be stronger than handengineered features  ). This shift is welcome, but there is still more scope to achieve perfection, adaptability and reproducibility. The results from both the 2014 and the 2016 competitions are very high for the top teams. However, it is important to note that there is a high variation in the frequency of occurrence of PHIs. For example, PHIs such as DATE, NAME and DOCTOR occur much more often than PHIs such as PROFESSION and ORGANISATION. These more frequently occurring PHIs are also considered to be the easier to find entities, compared to the less frequent PHIs. The higher F-measures seem to reflect this imbalance. Hence the results may be misleading, i.e. high average F-measure does not imply good deidentification performance on rare categories such as PROFESSION. It is also evident that compared to the systems presented in the 2014 competitions, 2016 competition systems are much more complicated. The resulting gain in accuracy seems rather small. This complexity makes it very hard to reproduce these systems and also make it hard for others to replicate their results for scientific purposes. It is also important to point out that an expert manually developed most of the rule-based components of the systems for the given data. However, the adaptability of these systems to new data remains an open question. This was partly evident from track 1A 2016 competition results, where only a couple of systems produced average results on a new dataset (in this case 2016). Looking at transfer learning BIB001 is, therefore, an opportunity. It might be worth exploring domains such as search engine logs and law data, in which de-identified data is less sensitive than it is in the medical field. Hence, it is possible to obtain large datasets suitable for transfer learning. More competitions such as that of the 2014 and the 2016 de-identification competitions will undoubtedly provide more opportunities for solving the de-identification problem. Emphasis on tracks such as that of track 3 in the 2014 competition and track 1A in the 2016 competition is also vital. The need for simpler systems that can be reproduced and used in different applications is crucial. One primary concern with these de-identification systems is how well they will perform with other datasets. Unfortunately, EHR differs in each institute, and sometimes even in each department ). Will these systems adapt to the change in EHR? It is important to note both data for both 2006 and 2014 were drawn from the same source, though 2014 data was much more extensive than for 2006 ). However, the 2016 data was very different from that of the other competitions. As track 1A from 2016 competition was explicitly designed to address this issue, it was clear that apart from a couple of average results, the existing systems did not perform well when the dataset was changed ). It is a good start for building models that can be tuned to new data, and with more competitions and new data, systems can be trained, and better accuracy might be achieved. These competitions have provided an excellent platform for developing and improving de-identification systems, but there is still a lot more work to be done to find a solution to the de-identification problem.