Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> 1. In spite of the influential Saussurean dichotomy of synchronic and diachronic studies, it has in fact not been possible to keep these two main aspects of linguistics in hermetically sealed compartments. The aim of this paper is to explore the theoretical relations between phenomena in the two fields in the context of the current interest in universals. The examples cited are not intended to be conclusive and are meant only as illustrations. It is also not within the scope of this paper to present a detailed historical account. My indebtedness to others who have worked in this area should be evident; but three lines of previous investigation should not go unmentioned. The first of these is the work of Bonfante, Hoenigswald, Marchand, Chafe, and others on internal reconstruction, a method by which synchronic data regarding morphophonemic alternations lead to diachronic conclusions about earlier sound changes.1 The second is the emphasis of Jakobson on the importance of synchronic universals in testing the validity of reconstructed phonological systems.2 The third is the work of Martinet, which not only calls attention to the importance of synchronic structure in understanding sound change, but also contains numerous observations regarding factors in diachronic phonology which are implicitly equivalent in some instances to universals of change.3 <s> BIB001 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> The work reported in this monograph was begun in the winter of 1967 in a graduate seminar at Berkeley. Much has been discovered since 1969, the date of original publication, regarding the psychophysical and neurophysical determinants of universal, cross-linguistic constraints on the shape of basic color lexicons. <s> BIB002 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> Since its first publication, "Language Universals and Linguistic Typology" has become established as the leading introductory account of one of the most productive areas of linguistics-the analysis, comparison, and classification of the common features and forms of the organization of languages. Adopting an approach to the subject pioneered by Greenberg and others, Bernard Comrie is particularly concerned with syntactico-semantic universals, devoting chapters to word order, case making, relative clauses, and causative constructions. His book is informed throughout by the conviction that an exemplary account of universal properties of human language cannot restrict itself to purely formal aspects, nor focus on analysis of a single language. Rather, it must also consider language use, relate formal properties to testable claims about cognition and cognitive development, and treat data from a wide range of languages. This second edition has been revised and updated to take full account of new research in universals and typology in the past decade, and more generally to consider how the approach advocated here relates to recent advances in generative grammatical theory. <s> BIB003 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> Textile fibers composed of vinyl chloride homopolymers or of copolymers containing at least 80% by weight of vinyl chloride or mixtures thereof having an appearance (sheen) and a feel similar to those of animal hair are prepared by incorporating into the spinning solution from which the fibers are spun at least about 1% by weight of the polymer of a white or colorless salt havinG a density in excess of about 2 g/cc, extruding the fiber, and then removing at least a part of the salt from the surface of the resulting fiber with water or another solvent for the salt. <s> BIB004 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> 1. Introduction 2. Typological classification 3. Implicational universals 4. Markedness in typology 5. Grammatical hierarchies 6. Prototypes and the interaction of typological patterns 7. External motives and the typology of form-function relations 8. Diachronic typology 9. Linguistic explanation in the dynamic paradigm Notes References Map of languages cited Author index Language index Subject index. <s> BIB005 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> A method and apparatus is disclosed for determining the speed and direction and height/width dimensions of subject movement within a field of view. Two video frames are compared such that the dimensions of change within the two frames are determined, sorted and counted. One count is made of the vertically-occurring change; the other count is made of the horizontally-occurring change. The derived change dimensions and position is stored to be later compared with the change dimensions and position of a following comparison of two scans; the lateral displacement of one change position of a prior time with respect to the change position of the present time is determined and counted, the count representing the magnitude and direction of the subject movement that occurred. These determinors are 'dumped' into a computer which detects certain types of change and ignores others and which may be operated to determine the velocity of horizontal movement as well as changes in velocity of vertical movement. <s> BIB006 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> An evolutionary model of language change some theories of language change in an evolutionary framework a theory of language meaning in use form-function re-analysis interference, intraference and grammaticalization selection (propagation) of innovations in language change the descent of languages towards an evolutionary linguistics. <s> BIB007 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> It is argued that the principles needed to explain linguistic behavior are domain-general and based on the impact that specific experiences have on the mental organization and representation of language. This organizationmust be sensitive to both specific information and generalized patterns. In addition, knowledge of language is highly sensitive to frequency of use: frequently-used linguistic sequences become more frequent, more accessible and better integrated. The evidence adduced is mainly from phonology and morphology and addresses the issue of gradience and specificity found in postulated units, categories, and dichotomies such as regular and irregular, but the points apply to all levels of linguistic analysis including the syntactic, semantic, and discourse levels. Appropriate models for representing such phenomena are considered, including exemplar models and connectionist models, which are evolving to achieve a better fit with linguistic data. The major criticism of connectionist models often raised from within the combinatorial paradigm of much existing linguistic theory - that they do not capture 'free combination' to the extent that rule-based systems do, is regarded as a strength rather than a weakness. Recent connectionist models exhibit greater productivity and systematicity than earlier variants, but still show less uniformity of generalization than combinatorial models do. The remaining non-uniformity that the connectionist models show is appropriate, given that such non-uniformity is the rule in language structure and language behavior. <s> BIB008 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> Abstract 1. Introduction Structural categories of grammar (such as clitic, affix, compound, adjective, pronoun, dative, subject, passive, diphthong, coronal) have to be posited by linguists and by children during acquisition. This would be easier if they simply had to choose from a list of pre-established categories. However, existing proposals for what such a list might be are still heavily based on the Latin and English grammatical tradition. Thus, descriptive linguists still have no choice but to adopt the Boasian approach of positing special language-particular categories for each language. Theorists often resist it, but the crosslinguistic evidence is not converging on a smallish set of possibly innate categories. On the contrary, almost every newly described language presents us with some “crazy” new category that hardly fits existing taxonomies. Although there is thus no good evidence for pre-established categories, linguists still often engage in category-assignment controversies such as “Is the Tagalog ang-phrase a subject or a topic?”, “Is German er a pronoun or a determiner?”, “Are Mandarin Chinese property words adjectives or verbs?”, or “Is the Romanian definite article a clitic or a suffix?” <s> BIB009 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> Abstract This special issue of Cognitive Linguistics explores the linguistic encoding of events of cutting and breaking. In this article we first introduce the project on which it is based by motivating the selection of this conceptual domain, presenting the methods of data collection used by all the investigators, and characterizing the language sample. We then present a new approach to examining crosslinguistic similarities and differences in semantic categorization. Applying statistical modeling to the descriptions of cutting and breaking events elicited from speakers of all the languages, we show that although there is crosslinguistic variation in the number of distinctions made and in the placement of category boundaries, these differences take place within a strongly constrained semantic space: across languages, there is a surprising degree of consensus on the partitioning of events in this domain. In closing, we compare our statistical approach with more conventional semantic analyses, and show how an extensional semantic typological approach like the one illustrated here can help illuminate the intensional distinctions made by languages. <s> BIB010 </s> Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing <s> Overview of Linguistic Typology <s> Abstract A fundamental fact about grammatical structure is that it is highly variable both across languages and within languages. Typological analysis has drawn language universals from grammatical variation, in particular by using the semantic map model. But the semantic map model, while theoretically well-motivated in typology, is not mathematically well-defined or computationally tractable, making it impossible to use with large and highly variable crosslinguistic datasets. Multidimensional scaling (MDS), in particular the Optimal Classification nonparametric unfolding algorithm, offers a powerful, formalized tool that allows linguists to infer language universals from highly complex and large-scale datasets. We compare our approach to Haspelmath's semantic map analysis of indefinite pronouns, and reanalyze Dahl's (1985) large tense-aspect dataset. MDS works best with large datasets, demonstrating the centrality of grammatical variation in inferring language universals and the importance of examining as wide a range of grammatical behavior as possible both within and across languages. <s> BIB011
There is no consensus on the precise number of languages in the world. For example, Glottolog provides the estimate of 7,748  , whereas Ethnologue  refers to 7,097. 1 This is because defining what constitutes a 'language' is in part arbitrary. Mutual intelligibility, which is used as the main criterion for including different language variants under the same label, is gradient in nature. Moreover, social and political factors play a role in the definition of language. Linguistic Typology is the discipline that studies the variation among the world's languages through their systematic comparison BIB003 BIB005 . The comparison is challenging because linguistic categories cannot be predefined BIB009 . Rather, cross-linguistically significant categories emerge inductively from the comparison of known languages, and are progressively refined with the discovery of new languages. Crucially, the comparison needs to be based on functional criteria, rather than formal criteria. Typologists distinguish between constructions, abstract and universal functions, and strategies, the type of expressions adopted by each language to codify a specific construction  . For instance, the passive voice is considered a strategy that emphasizes the semantic role of patient: some languages lack this strategy and use other strategies to express the construction. For instance, Awtuw (Sepik family) simply allows for the subject to be omitted. The classification of the strategies in each language is grounded in typological documentation (Bickel 2007, page 248) . Documentation is empirical in nature and involves collecting texts or speech excerpts, and assessing the features of a language based on their analysis. The resulting information is stored in large databases (see § 4.1) of attribute-values (this pair is henceforth referred to as typological feature), where usually each attribute corresponds to a construction and each value to the most widespread strategy in a specific language. Analysis of cross-lingual patterns reveals that cross-lingual variation is bounded and far from random  ). Indeed, typological features can be interdependent: The presence of one feature may implicate another (in one direction or both). This interdependence is called restricted universal, as opposed to unrestricted universals, which specify properties shared unconditionally by all languages. Such typological universals (restricted or not) are rarely absolute (i.e., exceptionless); rather, they are tendencies  ), hence they are called "statistical." For example, consider this restricted universal: If a language (such as Hmong Njua, Hmong-Mien family) has prepositions, then genitive-like modifiers follow their head. If, instead, a language (such as Slavey, Na-Dené family) has postpositions, the order of heads and genitive-like modifiers is swapped. However, there are known exceptions: Norwegian (Indo-European) has prepositions but genitives precede their syntactic heads. 2 Moreover, some typological features are rare whereas others are highly frequent. Interestingly, this also means that some languages are intuitively more plausible than others. Implications and frequencies of features are important, as they unravel the deeper explanatory factors underlying the patterns of cross-linguistic variation BIB006 . Cross-lingual variation can be found at all levels of linguistic structure. The seminal works on Linguistic Typology were concerned with morphosyntax, mainly morphological systems   , page 128) and word order  ). This level of analysis deals with the form of meaningful elements (morphemes and words) and their combination, hence it is called structural typology. As an example, consider the alignment of the nominal case system (Dixon 1994) : Some languages like Nenets (Uralic) use the same case for subjects of both transitive and intransitive verbs, and a different one for objects (nominative-accusative alignment). Other languages like Lezgian (Northeast Caucasian) group together intransitive subjects and objects, and treat transitive subjects differently (ergative-absolutive alignment). On the other hand, semantic typology studies languages at the semantic and pragmatic levels. This area was pioneered by anthropologists interested in kinship (d 'Andrade 1995) and colors BIB002 , and was expanded by studies on lexical classes  . The main focus of semantic typology has been to categorize languages in terms of concepts  in the lexicon, in particular with respect to the 1) granularity, 2) division (boundary location), and 3) membership criteria (grouping and dissection). For instance, consider the event expressed by to open (something). It lacks a precise equivalent in languages such as Korean, where similar verbs overlap in meaning only in part  . For instance, ppaeda means 'to remove an object from tight fit' (used, e.g., for drawers) and pyeolchida means 'to spread out a flat thing' (used, e.g., for hands). Moreover, in most expressions, the English verb encodes the resulting state of the event, whereas an equivalent verb in another language such as Spanish (abrir) rather expresses the manner of the event  . Although variation in the categories is pervasive due to their partly arbitrary nature, it is constrained cross-lingually via shared cognitive constraints BIB010 . Similarities between languages do not always arise from language-internal dynamics but also from external factors. In particular, similarities can be inherited from a common ancestor (genealogical bias) or borrowed by contact with a neighbor (areal bias)  . Owing to genealogical inheritance, there are features that are widespread within a family but extremely rare elsewhere (e.g., the presence of click phonemes in the Khoisan languages). As an example of geographic percolation, most languages in the Balkan area (Albanian, Bulgarian, Macedonian, Romanian, Torlakian) have developed, even without a common ancestor, a definite article that is put after its noun simply because of their close proximity. Research in linguistic typology has sought to disentangle such factors and to integrate them into a single framework aimed at answering the question "what's where why?"  . Language can be viewed as a hybrid biological and cultural system. The two components co-evolved in a twin track, developing partly independently and partly via mutual interaction  . The causes of cross-lingual variation can therefore be studied from two complementary perspectives-from the perspective of functional theories or event-based theories  . The former theories involve cognitive and communicative principles (internal factors) and account for the origin of variation, whereas the latter ones emphasize the imitation of patterns found in other languages (external factors) and account for the propagation (or extinction) of typological features (Croft 1995 BIB007 . Examples of functional principles include factors associated with language use, such as the frequency or processing complexity of a pattern  . Patterns that are easy or widespread become integrated into the grammar (Haspelmath 1999, inter alia) . On the other hand, functional principles allow the speakers to draw similar inferences from similar contexts, leading to locally motivated pathways of diachronic change through the process known as grammaticalization BIB001  BIB004 . For instance, in the world's languages (including English) the future tense marker almost always originates from verbs expressing direction, duty, will, or attempt because they imply a future situation. The diachronic and gradual origin of the changes in language patterns and the statistical nature of the universals explain why languages do not behave monolithically. Each language can adopt several strategies for a given construction and partly inconsistent semantic categories. In other words, typological patterns tend to be gradient. For instance, the semantics of grammatical and lexical categories can be represented on continuous multi-dimensional maps BIB011 . BIB008 have noted how this gradience resembles the patterns learned by connectionist networks (and statistical machine learning algorithms in general). In particular, they argue that such architectures are sensitive to both local (contextual) information and general patterns, as well as to their frequency of use, similarly to natural languages. Typological documentation is limited by the fact that the evidence available for each language is highly unbalanced and many languages are not even recorded in a written form. 3 However, large typological databases such as WALS (Dryer and Haspelmath 2013) nevertheless have an impressive coverage (syntactic features for up to 1,519 languages). Where such information can be usefully integrated in machine learning, it can provide an alternative form of guidance to manual construction of resources that are now largely lacking for low resource languages. We discuss the existing typological databases and the integration of their features into NLP models in sections 4 and 5.