In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8 of its average POS tagging accuracy when trained at 1.2 of the total available training data, i.e. 150 sentences per language.
According to these findings, in order to avoid any hassles of tuning the hyperparameters of the word embedding model employed, we rather primarily use the publicly available pre-trained polyglot word embeddings of @cite_22 without any task specific modification for our experiments. A key thing to note is that polyglot word embeddings are not tailored for any specific language analysis task such as POS tagging or NER. These word embeddings are instead trained in a manner favoring the word analogy task introduced by . The polyglot project distributes word embeddings for more than 100 languages. also report results on POS tagging, however, word representations they apply for these experiments are different from the task-agnostic representations they made publicly available.
@cite_22
1574126082
Despite significant recent work, purely unsupervised techniques for part-of-speech (POS) tagging have not achieved useful accuracies required by many language processing tasks. Use of parallel text between resource-rich and resource-poor languages is one source of weak supervision that significantly improves accuracy. However, parallel text is not always available and techniques for using it require multiple complex algorithmic steps. In this paper we show that we can build POS-taggers exceeding state-of-the-art bilingual methods by using simple hidden Markov models and a freely available and naturally growing resource, the Wiktionary. Across eight languages for which we have labeled data to evaluate results, we achieve accuracy that significantly exceeds best unsupervised and parallel text methods. We achieve highest accuracy reported for several languages and show that our approach yields better out-of-domain taggers than those trained using fully supervised Penn Treebank.