Traditional information retrieval treats named entity recognition as a pre-indexing corpus annotation task, allowing entity tags to be indexed and used during search. Named entity taggers themselves are typically trained on thousands or tens of thousands of examples labeled by humans. However, there is a long tail of named entities classes, and for these cases, labeled data may be impossible to find or justify financially. We propose exploring named entity recognition as a search task, where the named entity class of interest is a query, and entities of that class are the relevant "documents". What should that query look like? Can we even perform NER-style labeling with tens of labels? This study presents an exploration of CRF-based NER models with handcrafted features and of how we might transform them into search queries.
Some work explores more fine-grained representations of entities, namely a work by which explores a model for identifying named entities as a single step and classification into 147 different categories as a secondary step @cite_12 . More recently, Ling and Weld present , a system for fine-grained entity recognition of 112 tags, trained using data collected from Wikipedia links used a similar approach: a CRF for segmenting into B (beginning), I (intermediate), and O (outside) tags, and a multiclass classification task on top @cite_21 . assign more than 1000 types to noun-phrases to help classify out-of-knowledge-base entities @cite_19 . In order to be as general as possible, we do not consider filtering to noun-phrases or classified spans in this work, but that is a promising approach to increasing efficiency in future work.
@cite_19 @cite_21 @cite_12
11196620 1502473750 2402817887 2047477415
In many QA systems, fine-grained named entities are extracted by coarse-grained named entity recognizer and fine-grained named entity dictionary. In this paper, we describe a fine-grained Named Entity Recognition using Conditional Random Fields (CRFs) for question answering. We used CRFs to detect boundary of named entities and Maximum Entropy (ME) to classify named entity classes. Using the proposed approach, we could achieve an 83.2 precision, a 74.5 recall, and a 78.6 F1 for 147 fined-grained named entity types. Moreover, we reduced the training time to 27 without loss of performance compared to a baseline model. In the question answering, The QA system with passage retrieval and AIU archived about 26 improvement over QA with passage retrieval. The result demonstrated that our approach is effective for QA. We explore the use of Wikipedia as external knowledge to improve named entity recognition (NER). Our method retrieves the corresponding Wikipedia entry for each candidate word sequence and extracts a category label from the first sentence of the entry, which can be thought of as a definition part. These category labels are used as features in a CRF-based NE tagger. We demonstrate using the CoNLL 2003 dataset that the Wikipedia category labels extracted by such a simple method actually improve the accuracy of NER. This paper reports the highest results (95 in MUC and 92 in CoNLL metric) in the literature for Turkish named entity recognition; more specifically for the task of detecting person, location and organization entities in general news texts. We give an in depth analysis of the previous reported results and make comparisons with them whenever possible. We use conditional random fields (CRFs) as our statistical model. The paper presents initial explorations on the usage of rich morphological structure of the Turkish language as features to CRFs together with the use of some basic and generative gazetteers. Many named entities contain other named entities inside them. Despite this fact, the field of named entity recognition has almost entirely ignored nested named entity recognition, but due to technological, rather than ideological reasons. In this paper, we present a new technique for recognizing nested named entities, by using a discriminative constituency parser. To train the model, we transform each sentence into a tree, with constituents for each named entity (and no other syntactic structure). We present results on both newspaper and biomedical corpora which contain nested named entities. In three out of four sets of experiments, our model outperforms a standard semi-CRF on the more traditional top-level entities. At the same time, we improve the overall F-score by up to 30 over the flat model, which is unable to recover any nested entities.