We present a weakly-supervised data augmentation approach to improve Named Entity Recognition (NER) in a challenging domain: extracting biomedical entities (e.g., proteins) from the scientific literature. First, we train a neural NER (NNER) model over a small seed of fully-labeled examples. Second, we use a reference set of entity names (e.g., proteins in UniProt) to identify entity mentions with high precision, but low recall, on an unlabeled corpus. Third, we use the NNER model to assign weak labels to the corpus. Finally, we retrain our NNER model iteratively over the augmented training set, including the seed, the reference-set examples, and the weakly-labeled examples, which results in refined labels. We show empirically that this augmented bootstrapping process significantly improves NER performance, and discuss the factors impacting the efficacy of the approach. The increasing wealth of available data fuels numerous machine learning applications. Unfortunately, much of this data is unlabeled, unstructured and noisy. Supervised learning achieves the best task performance, but obtaining training labels is expensive. Crowd-sourcing could provide labels at scale, but may not be feasible for acquiring high-quality labels in technical domains, such as biomedicine that requires expert annotators. In this paper, we explore augmented bootstrapping methods that leverage automatically assigned noisy labels obtained from a large unlabeled corpus. The biomedical literature is a high-impact domain with scarce annotations. Unlocking the knowledge in this data requires machine reading systems that automatically extract important concepts in the text, such as entities and their relations. A critical component of such systems is reliable Named Entity Recognition (NER), which aims to identify parts of the text that refer to a named entity (e.g., a protein). In line with advancements in many domains, most state-of-the-art NER approaches use a deep neural network model that relies on a large labeled training set, which is not usually available in biomedical domains. To address label scarcity, we propose a framework to train any effective neural NER model by leveraging partially labeled data. We do this by creating an augmented training set using a small fully-labeled seed set, and an unlabeled corpus set, which we weakly and automatically label, and then refine its labels via an iterative process. Our main contributions include: (1) An augmented bootstrapping approach combining information from a reference set with iterative refinements of soft labels to improve NER in a challenging domain (biomedicine) where labelling is expensive. (2) A detailed analysis in a controlled setting to study different aspects affecting performance. (3) An analysis of reference-based automated approaches to labeling data, showing that naive labeling decreases performance and how to overcome it. We proposed a method to improve NER with limited labeled data, which is often the case in technical domains, such as biomedicine. Our method combines bootstrapping and weakly-labeled data augmentation by using a small fully-labeled seed dataset and a large unlabeled corpus, automated labelling using a reference set, and an iterative label refinement process. Our experimental evaluation shows performance equivalent to systems trained with an order of magnitude more labeled data. In future work, we aim to explore additional augmentation methods over other challenging datasets. We plan to apply the findings of these controlled experiments to a much larger in-the-wild scenario where we use all the available labeled data as the seed and operate over a large corpus (e.g., all of PubMed, PubMed Central) to improve state-of-the-art NER performance.
Augmented bootstrapping approach combining information from a reference set with iterative refinements of soft labels to improve Name Entity Recognition from biomedical literature.
{"src": "scitldr-AIC_train", "paper_id": "S1ghJiRVd4"}