This paper presents the first attempt, up to our knowledge, to classify English writing styles on this scale with the challenge of classifying day to day language written by writers with different backgrounds covering various areas of topics.The paper proposes simple machine learning algorithms and simple to generate features to solve hard problems. Relying on the scale of the data available from large sources of knowledge like Wikipedia. We believe such sources of data are crucial to generate robust solutions for the web with high accuracy and easy to deploy in practice. The paper achieves 74 accuracy classifying native versus non native speakers writing styles. Moreover, the paper shows some interesting observations on the similarity between different languages measured by the similarity of their users English writing styles. This technique could be used to show some well known facts about languages as in grouping them into families, which our experiments support.
The first work related with native language identification is that of @cite_0 , in which they tried profiling anonymous authors with their native languages. Totally five different groups of English authors (whose native languages are Russian, Bulgarian, French, and Spanish) were picked from the first version of International Corpus of Learner English (ICLE) in their experiments. By applying a combined feature sets, including function words, character n-grams, part-of-speech bi-grams and spelling mistakes, they gained an accuracy of 65 Similar work was done by tsur2007using . They focused on the relationships between choice of words in second language writing and the frequency of native language syllables, also known as the phonology of native languages. estival2007author studied a wide range of lexical and document structure features in their native languages classification task. And zheng2003authorship , though they did not directly conduct related experiments on nationality detection, they provide some features of style markes that could be used in the task of judging one's native languages. Besides, gamon2004linguistic analysized the power of some general features under different frequency cutoffs. But none of these measured the usefulness of syntactic features under a general condition for the task of native language detection.
@cite_0
2251303571 2097227294 2151338697 2016630033
In this paper we present work on the task of Native Language Identification (NLI). We present an alternative corpus to the ICLE which has been used in most work up until now. We believe that our corpus, TOEFL11, is more suitable for the task of NLI and will allow researchers to better compare systems and results. We show that many of the features that have been commonly used in this task generalize to new and larger corpora. In addition, we examine possible ways of increasing current system performance (e.g., additional features and feature combination methods), and achieve overall state-of-the-art results (accuracy of 90.1 ) on the ICLE corpus using an ensemble classifier that includes previously examined features and a novel feature (n-gram language models). We also show that training on a large corpus and testing on a smaller one works well, but not vice versa. Finally, we show that system performance varies across proficiency scores. Language identification is a simple problem that becomes much more difficult when its usual assumptions are broken. In this paper we consider the task of classifying short segments of text in closely-related languages for the Discriminating Similar Languages shared task, which is broken into six subtasks, (A) Bosnian, Croatian, and Serbian, (B) Indonesian and Malay, (C) Czech and Slovak, (D) Brazilian and European Portuguese, (E) Argentinian and Peninsular Spanish, and (F) American and British English. We consider a number of different methods to boost classification performance, such as feature selection and data filtering, but we ultimately find that A series of progressive demasking and lexical decision experiments investigated how the recognition of target words exclusively belonging to one language is affected by the existence of orthographic neighbors from the same or the other language of bilingual participants. Increasing the number of orthographic neighbors in Dutch systematically slowed response times to English target words in Dutch English bilinguals, while an increase in target language neighbors consistently produced inhibitory effects for Dutch and facilitatory effects for English target words. Monolingual English speakers also showed facilitation due to English neighbors, but no effect of Dutch neighbors. The experiments provide evidence for parallel activation of words in an integrated Dutch English lexicon. An implemented version of such a model making these assumptions, the Bilingual Interactive Activation (BIA) model, is shown to account for the overall pattern of results. © 1998 Academic Press One of the striking features of bilingual language performance is the apparent ease with which the bilingual manages to keep interference from the non-target language at a minimal level. The fact remains, however, that interference from one language to the other does occur and is observable with respect to both language structure and linguistic processing. For example, in language production, interference from the first language can be noticed both at the phonological level (foreign accents) and at the sentence level (borrowed syntax), as well as in intrusions of words from the other language (accidental lexical borrowings). This paper describes a system and set of algorithms for automatically inducing stand-alone monolingual part-of-speech taggers, base noun-phrase bracketers, named-entity taggers and morphological analyzers for an arbitrary foreign language. Case studies include French, Chinese, Czech and Spanish.Existing text analysis tools for English are applied to bilingual text corpora and their output projected onto the second language via statistically derived word alignments. Simple direct annotation projection is quite noisy, however, even with optimal alignments. Thus this paper presents noise-robust tagger, bracketer and lemmatizer training procedures capable of accurate system bootstrapping from noisy and incomplete initial projections.Performance of the induced stand-alone part-of-speech tagger applied to French achieves 96 core part-of-speech (POS) tag accuracy, and the corresponding induced noun-phrase bracketer exceeds 91 F-measure. The induced morphological analyzer achieves over 99 lemmatization accuracy on the complete French verbal system.This achievement is particularly noteworthy in that it required absolutely no hand-annotated training data in the given language, and virtually no language-specific knowledge or resources beyond raw text. Performance also significantly exceeds that obtained by direct annotation projection.