Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Feature Selection from Typological Databases <s> We present a novel algorithm for multilingual dependency parsing that uses annotations from a diverse set of source languages to parse a new unannotated language. Our motivation is to broaden the advantages of multilingual learning to languages that exhibit significant differences from existing resource-rich languages. The algorithm learns which aspects of the source languages are relevant for the target language and ties model parameters accordingly. The model factorizes the process of generating a dependency tree into two steps: selection of syntactic dependents and their ordering. Being largely language-universal, the selection component is learned in a supervised fashion from all the training languages. In contrast, the ordering decisions are only influenced by languages with similar properties. We systematically model this cross-lingual sharing using typological features. In our experiments, the model consistently outperforms a state-of-the-art multi-lingual parser. The largest improvement is achieved on the non Indo-European languages yielding a gain of 14.4%. <s> BIB001 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Feature Selection from Typological Databases <s> We propose a simple cross-language parser adaptation strategy for discriminative parsers and apply it to easy-first transition-based dependency parsing (Goldberg and Elhadad, 2010). We evaluate our parsers on the Indo-European corpora in the CoNLL-X and CoNLL 2007 shared tasks. Using the remaining languages as source data we average under-fitted weights learned from each source language and apply the resulting linear classifier to the target language. Of course some source languages and some sentences in these languages are more relevant than others for the target language in question. We therefore explore improvements of our cross-language adaptation model involving source language and instance weighting, as well as unsupervised model selection. Overall our cross-language adaptation strategies provide better results than previous strategies for direct transfer, with near-linear time parsing and much faster training times than other approaches. <s> BIB002 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Feature Selection from Typological Databases <s> In this paper we explore the novel idea of building a single universal reordering model from English to a large number of target languages. To build this model we exploit typological features of word order for a large number of target languages together with source (English) syntactic features and we train this model on a single combined parallel corpus representing all (22) involved language pairs. We contribute experimental evidence for the usefulness of linguistically defined typological features for building such a model. When the universal reordering model is used for preordering followed by monotone translation (no reordering inside the decoder), our experiments show that this pipeline gives comparable or improved translation performance with a phrase-based baseline for a large number of language pairs (12 out of 22) from diverse language families. <s> BIB003 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Feature Selection from Typological Databases <s> This work examines the impact of crosslinguistic transfer on grammatical errors in English as Second Language (ESL) texts. Using a computational framework that formalizes the theory of Contrastive Analysis (CA), we demonstrate that language specific error distributions in ESL writing can be predicted from the typological properties of the native language and their relation to the typology of English. Our typology driven model enables to obtain accurate estimates of such distributions without access to any ESL data for the target languages. Furthermore, we present a strategy for adjusting our method to low-resource languages that lack typological documentation using a bootstrapping approach which approximates native language typology from ESL texts. Finally, we show that our framework is instrumental for linguistic inquiry seeking to identify first language factors that contribute to a wide range of difficulties in second language acquisition. <s> BIB004
The databases presented above can serve as a rich source of typological information for NLP. In this section, we survey the feature sets that have been extracted from these databases in typologically informed NLP studies. In § 5.4, we review in which ways and to what degree of success these features have been integrated in machine learning algorithms. Most NLP studies informed by typology only incorporated a subset of word order features from WALS (Dryer and Haspelmath 2013) . Most of these studies focused on the task of syntactic dependency parsing, where word order provides crucial guidance BIB001 , using the feature subsets shown in Figure 4 . As depicted in the figure, these studies utilized quite similar word order features. The feature set first established by Naseem, Barzilay, and Globerson (2012) served as inspiration for subsequent works. The main differences in these sets results from the practice of discarding features that are not discriminative, when they are identical for all the languages in the sample. Another group of studies used more comprehensive feature sets. The feature set of BIB003 included not only WALS word order features but also nominal categories (e.g., "Conjunctions and Universal Quantifiers") and nominal syntax (e.g., "Possessive Classification"). BIB004 considered all features from WALS associated with morphosyntax and pruned out the redundant ones, resulting in a total of 119 features. BIB002 utilized all the