In this work, we propose simple yet effective approaches to exploiting source-side global sentence-level context for NMT models.
We use encoder representations to represent the source-side context, which are summarized into a sentential context vector. The source-side context vector is fed to the decoder, so that translation at each step is conditioned on the whole source-side context.
Specifically, we propose two types of sentential context: 1) the shallow one that only exploits the top encoder layer, and 2) the deep one that aggregates the sentence representations of all the encoder layers. The deep sentential context can be viewed as a more comprehensive global sentence representation, since different types of syntax and semantic information are encoded in different encoder layers [1]}, [2]}, [3]}.
