In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8 of its average POS tagging accuracy when trained at 1.2 of the total available training data, i.e. 150 sentences per language.
There have been further previous research conducted on training neural networks for learning distributed word representations for various specific language analysis tasks. propose neural network architectures to four natural language processing tasks, i.e. POS tagging, named entity recognition, semantic role labeling and chunking. train word representations on large amounts of unannotated texts from Wikipedia, then update the pre-trained word representations for the individual tasks. Our approach is different in that we do not update our word representations for the different tasks and most importantly that we use successfully the features derived from sparse coding in a log-linear model instead of a neural network architecture. A final difference to @cite_6 is that we experiment with a much wider range of languages while they report results for English only.
@cite_6
2101609803
Distributed word representations have recently been proven to be an invaluable resource for NLP. These representations are normally learned using neural networks and capture syntactic and semantic information about words. Information about word morphology and shape is normally ignored when learning word representations. However, for tasks like part-of-speech tagging, intra-word information is extremely useful, specially when dealing with morphologically rich languages. In this paper, we propose a deep neural network that learns character-level representation of words and associate them with usual word representations to perform POS tagging. Using the proposed approach, while avoiding the use of any handcrafted feature, we produce state-of-the-art POS taggers for two languages: English, with 97.32 accuracy on the Penn Treebank WSJ corpus; and Portuguese, with 97.47 accuracy on the Mac-Morpho corpus, where the latter represents an error reduction of 12.2 on the best previous known result.