Recent work on word ordering has argued that syntactic structure is important, or even required, for effectively recovering the order of a sentence. We find that, in fact, an n-gram language model with a simple heuristic gives strong results on this task. Furthermore, we show that a long short-term memory (LSTM) language model is even more effective at recovering order, with our basic model outperforming a state-of-the-art syntactic model by 11.5 BLEU points. Additional data and larger beams yield further gains, at the expense of training and search time.
Recent approaches to linearization have been based on reconstructing the syntactic structure to produce the word order. Let @math represent all projective dependency parse trees over @math words. The objective is to find @math where @math is now over both the syntactic structure and the linearization. The current state of the art on the Penn Treebank (PTB) @cite_11 , without external data, of uses a transition-based parser with beam search to construct a sentence and a parse tree. The scoring function is a linear model @math and is trained with an early update structured perceptron to match both a given order and syntactic tree. The feature function @math includes features on the syntactic tree. This work improves upon past work which used best-first search over a similar objective @cite_12 .
@cite_12 @cite_11
129942754 1632114991
Machine-produced text often lacks grammaticality and fluency. This paper studies grammaticality improvement using a syntax-based algorithm based on ccg. The goal of the search problem is to find an optimal parse tree among all that can be constructed through selection and ordering of the input words. The search problem, which is significantly harder than parsing, is solved by guided learning for best-first search. In a standard word ordering task, our system gives a BLEU score of 40.1, higher than the previous result of 33.7 achieved by a dependency-based system. Abstract : As a result of this grant, the researchers have now published oil CDROM a corpus of over 4 million words of running text annotated with part-of- speech (POS) tags, with over 3 million words of that material assigned skelet al grammatical structure. This material now includes a fully hand-parsed version of the classic Brown corpus. About one half of the papers at the ACL Workshop on Using Large Text Corpora this past summer were based on the materials generated by this grant.