In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences. Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms.   Instead, it directly outputs phrases in a sequential order and can decode in linear time. Our experiments show that NPMT achieves superior performances on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese machine translation tasks compared with strong NMT baselines. We also observe that our method produces meaningful phrases in output languages. A word can be considered as a basic unit in languages. However, in many cases, we often need a phrase to express a concrete meaning. For example, consider understanding the following sentence, "machine learning is a field of computer science". It may become easier to comprehend if we segment it as " [machine learning] [is] [a field of] [computer science]", where the words in the bracket '[]' are regarded as "phrases". These phrases have their own meanings, and can often be reused in other contexts.The goal of this paper is to explore the use of phrase structures aforementioned for neural networkbased machine translation systems BID22 BID0 . To this end, we develop a neural machine translation method that explicitly models phrases in target language sequences. Traditional phrase-based statistical machine translation (SMT) approaches have been shown to consistently outperform word-based ones (Koehn et al., 2003; Koehn, 2009; BID15 . However, modern neural machine translation (NMT) methods BID22 BID0 do not have an explicit treatment on phrases, but they still work surprisingly well and have been deployed to industrial systems BID31 BID28 . The proposed Neural Phrase-based Machine Translation (NPMT) method tries to explore the advantages from both kingdoms. It builds upon Sleep-WAke Networks (SWAN), a segmentation-based sequence modeling technique described in BID25 , where segments (or phrases) are automatically discovered given the data. However, SWAN requires monotonic alignments between inputs and outputs. This is often not an appropriate assumption in many language pairs. To mitigate this issue, we introduce a new layer to perform (soft) local reordering on input sequences. Experimental results show that NPMT outperforms attention-based NMT baselines in terms of the BLEU score BID19 on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese translation tasks. We believe our method is one step towards the full integration of the advantages from neural machine translation and phrase-based SMT. This paper is organized as follows. Section 2 presents the neural phrase-based machine translation model. Section 3 demonstrates the usefulness of our approach on several language pairs. We conclude our work with some discussions in Section 4.
Neural phrase-based machine translation with linear decoding time
{"src": "scitldr-AIC_train", "paper_id": "HktJec1RZ"}