Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Hand-Crafted Documentation in Typological Databases <s> LAPSyD, the Lyon-Albuquerque Phonological Systems Database, is an online phonological database equipped with powerful query, mapping and visualization tools. It stems from the UPSID and WALS databases, enhanced with newly validated data not only covering segmental inventories but also syllable structures, stress and tonal systems. In its current version it covers around 700 languages and it is accessible at http://www.lapsyd.ddl.ish-lyon.cnrs.fr. This paper provides a description of the data structure in LAPSyD and the features of the interface. Brief illustrations of the types of analysis that can be done with this tool are provided, exploiting the ability to cross-reference data on segments, other phonological properties and language location. <s> BIB001 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Hand-Crafted Documentation in Typological Databases <s> When Part-of-Speech annotated data is scarce, e.g. for under-resourced languages, one can turn to cross-lingual transfer and crawled dictionaries to collect partially supervised data. We cast this problem in the framework of ambiguous learning and show how to learn an accurate history-based model. Experiments on ten languages show significant improvements over prior state of the art performance. <s> BIB002
Typological databases are created manually by linguists. They contain taxonomies of typological features, their possible values, as well as the documentation of feature values for the world's languages. Major typological databases, listed in Table 1 , typically organize linguistic information in terms of universal features and language-specific values. For example, Figure 3 presents language-specific values for the feature number of grammatical genders for nouns on a world map. Note that each language is color-coded 568 Table 1 An overview of major publicly accessible databases of typological information. The databases are ordered by description level (and secondly by date of creation), along with their coverage. The table also provides feature examples: for each feature (in small capitals) we present two example languages with distinct feature values, and the total number of languages with each value in parenthesis (where applicable). according to its value. Further examples for each database can be found in the rightmost column of Table 1 . Some databases store information pertaining to multiple levels of linguistic description. These include WALS (Dryer and Haspelmath 2013) and the Atlas of Pidgin and Creole Language Structures (APiCS)  ). Among all presently available databases, WALS has been the most widely used in NLP. In this resource, which has 142 typological features in total, features 1-19 deal with phonology, 20-29 with morphology, 30-57 with nominal categories, 58-64 with nominal syntax, 65-80 with verbal categories, 81-97 and 143-144 with word order, 98-121 with simple clauses, 122-128 with complex sentences, 129-138 with the lexicon, and 139-142 with other properties. Other databases only cover features related to a specific level of linguistic description. For example, both Syntactic Structures of the World's Languages (SSWL)  ) and AUTOTYP  ) focus on syntax. SSWL features are manually crafted, whereas AUTOTYP features are derived automatically from primary liguistic data using scripts. The Valency Patterns Leipzig (ValPaL)  provides verbs as attributes and predicate-argument structures as their values (including both valency and morphosyntactic constraints). For example, in both Mandinka and Sliammon, the verb to laugh has a valency of 1; in other words, it requires only one mandatory argument, the subject. In Mandinka the subject precedes the verb, but there is no agreement requirement; in Sliammon, on the other hand, the word order does not matter, but the verb is required to morphologically agree with the subject. For phonology, the Phonetics Information Base and Lexicon (PHOIBLE) (Moran, McCloy, and Wright 2014) collates information on segments (binary phonetic features). In the Lyon-Albuquerque Phonological Systems Database (LAPSyD) BIB001 , attributes are articulatory traits, syllabic structures, or tonal systems. Finally, StressTyp2 (Goedemans, Heinz, and der Hulst 2014) deals with stress and accent patterns. For instance, in Koromfé each word's first syllable has to be stressed, but not in Cubeo. Other databases document various aspects of semantics. The World Loanword Database (WOLD) (Haspelmath and Tadmor 2009) documents loanwords by identifying the donor languages and the source words. The Automated Similarity Judgment Program (ASJP) BIB002 and the Intercontinental Dictionary Series (IDS) (Key and Comrie 2015) indicate how a meaning is lexicalized across languages: For example, the concept of WORLD is expressed as mir in Russian, and as arkiśos . i in Tocharian A. Although typological databases store abundant information on many languages, they suffer from shortcomings that limit their usefulness. Perhaps the most significant shortcoming of such resources is their limited coverage. In fact, feature values are missing for most languages in most databases. Other shortcomings are related to feature granularity. In particular, most databases fail to account for feature value variation within each language: They report only majority value rather than the full range of possible values and their corresponding frequencies. For example, the dominant adjectivenoun word order in Italian is adjective before noun; however, the opposite order is also attested. The latter information is often missing from typological databases. Further challenges are posed by restricted feature applicability and feature hierarchies. Firstly, some features apply, by definition, only to subsets of languages that share another feature value. For instance, WALS feature 113A documents "Symmetric and Asymmetric Standard Negation," whereas WALS feature 114A "Subtypes of Asymmetric Standard Negation." Although a special NA value is assigned for symmetric-negation languages in the latter, there are cases where languages without the prerequisite feature are simply omitted from the sample. Secondly, features can be partially redundant, and subsume other features. For instance, WALS feature 81A "Order of Subject, Object and Verb" encodes the same information as WALS feature 82A "Order of Subject and Verb" and 83A "Order of Object and Verb," with the addition of the order of subject and object.