In this work, we explore the ability of artificial neural networks to judge the grammatical acceptability of a sentence. Machine learning research of this kind is well placed to answer important open questions about the role of prior linguistic bias in language acquisition by providing a test for the Poverty of the Stimulus Argument. In service of this goal, we introduce the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical by expert linguists. We train several recurrent neural networks to do binary acceptability classification. These models set a baseline for the task. Error-analysis testing the models on specific grammatical phenomena reveals that they learn some systematic grammatical generalizations like subject-verb-object word order without any grammatical supervision. We find that neural sequence models show promise on the acceptability classification task. However, human-like performance across a wide range of grammatical constructions remains far off.
It is possible to construct an acceptability corpus automatically if a method is found that can produce (roughly) unacceptable sentences. One approach is to programmatically generate sentences that are unlikely to be acceptable. distort real sentences by, for example, deleting words, inserting words, or altering verbal inflection. use round-trip machine-translation from English into various languages and back. A second family of approaches take sentences from essays written by non-native speakers @cite_19 . A third takes advantage of linguistics examples: and build datasets of 133 and 552 examples from syntax textbooks. CoLA scales up this line of work.
@cite_19
2251930319
Automated methods for identifying whether sentences are grammatical have various potential applications (e.g., machine translation, automated essay scoring, computer-assisted language learning). In this work, we construct a statistical model of grammaticality using various linguistic features (e.g., misspelling counts, parser outputs, n-gram language model scores). We also present a new publicly available dataset of learner sentences judged for grammaticality on an ordinal scale. In evaluations, we compare our system to the one from Post (2011) and find that our approach yields state-of-the-art performance.