Earlier studies [1]}, [2]}, [3]} alleviate the above problems by putting hard constraints on the data (e.g., discarding data with low similarity of the retrieved response and the target response), which, however, greatly reduces the amount of usable data. cai-etal-2019-retrieval employ a random mechanism for generating the skeletons used for training, which extract skeletons from the corresponding responses with some deliberate disturbance. paranjape2021hindsight propose to model the retriever after the posterior distribution of retrieval given the input and the target output and train it jointly with the standard retriever and the generator by maximizing the evidence lower bound (ELBo) in expectation over retrieval.
