Then, we introduce notable methods about retrieval-augmented generation, which are organized with respect to different tasks. Specifically, on the dialogue response generation task, exemplar/template retrieval as an intermediate step has been shown beneficial to informative response generation [1]}, [2]}, [3]}, [4]}. In addition, there has been growing
interest in knowledge-grounded generation exploring different forms of knowledge such as knowledge bases and external documents [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]}.
On the machine translation task, we summarize the early work on how the retrieved sentences (called translation memory) are used to improve statistical machine translation (SMT) [12]} models [13]}, [14]} and in particular, we intensively highlight several popular methods to integrating translation memory to NMT models [15]}, [16]}, [17]}, [18]}.
We also review the applications of retrieval-augmented generation in other generation tasks such as abstractive summarization [19]}, code generation [20]}, paraphrase [21]}, [22]}, and knowledge-intensive generation [23]}.
Finally, we also point out some promising directions on retrieval-augmented generation to push forward the future research.
