There also exist some works that propose new architectures to integrate the retrieval information. Under the RNN-based framework, [1]} and [2]} use the gating and attention mechanism to incorporate the retrieved target sentences. When Transformer [3]} becomes the backbone of NMT, some works also use additional transformer encoders to encode retrieved target sentences, and integrate them through attention mechanism [4]}, [5]}. [6]} represent the retrieved target sentences in a different data structure, i.e., a graph structure, and integrate it through attention mechanism. [7]} propose a light-weight method to encode the retrieved target sentences and leverage the alignment information to filter out irrelevant information. Different from previous works that rely on bilingual memories, [8]} propose a framework that can retrieve the most similar target sentence in a monolingual dataset, using a source sentence as query.
