Incremental Prediction of Sentence final Verbs: Humans versus Machines
Verb prediction is important in human sentence processing and, practically, in simultaneous machine translation. In verb final languages, speakers select the final verb before it is uttered, and listeners predict it before it is uttered. Simultaneous interpreters must do the same to translate in real time. Motivated by the problem of SOV SVO simultaneous machine translation, we provide a study of incremental verb prediction in verb final languages. As a basis of comparison, we examine incremental verb prediction with human participants in a multiple choice setting using crowdsourcing to gain insight into incremental human performance in a constrained setting. We then examine a computational approach to incremental verb prediction using discriminative classification with shallow features. Both humans and machines predict verbs more accurately as more of a sentence becomes available, and case markers when available help humans and sometimes machines predict final verbs. 1 The Importance of Verb Prediction Humans predict future linguistic input before it is observed (Kutas et al. 2011) This predictability has been formalized in information theory (Shannon, 1948) the more predictable a word is, the lower the entropy and has explained various linguistic phenomena, such as garden path ambiguity (Den and Inoue, 1997; Hale, 2001) Such instances of linguistic prediction are fundamental to statistical NLP. Auto complete from search engines has made next word prediction one of best known NLP applications. Long distance word prediction, such as verb prediction in SOV languages (Levy and Keller, 2013; Momma et al. 2015; Chow et al. 2015) is important in simultaneous machine translation from subject object verb (SOV) languages to subjectverb object (SVO) languages. In SVO languages such as English, for example, the main verb phrase usually comes after the first noun phrase the main subject in a sentence, while in verb final languages such as Japanese or German, it comes very last. Human simultaneous translators must make predictions about the unspoken final verb to incrementally translate the sentence. Minimizing interpretation delay thus requires making constant predictions and deciding when to trust those predictions and commit to translating in real time. Such prediction can also aid machines. Matsubara et al. (2000) use pattern matching rules; Grissom II et al. (2014) use a statistical n gram approach; and Oda et al. (2015) extend the idea of using prediction by predicting entire syntactic constituents for English Japanese translation. These systems require fast, accurate verb prediction to further improve simultaneous translation systems. We focus on verb prediction in verb final languages such as Japanese with this motivation in mind. In Section 2, we present what is, to our knowledge, the first study of humans' ability to incrementally predict the verbs in Japanese. We use these human data as a yardstick to which to compare computational incremental verb prediction. Incorporating some of the key insights from our human study into a discriminative model namely, the importance of case markers Section 3 presents a better incremental verb classifier than existing verb prediction schemes. Having established both human and computer performance on this challenging and interesting task, Section 4 reviews our work's relationship to other studies in NLP and linguistics.
CoNLL