We experiment with Autoencoder (AE) as the self-supervised data augmentation objective. For this, we use target data as the input and train the model to reconstruct it. The intuition behind it is that we want to teach the model to reproduce English words in the query as it is. AE techniques have been experimented with NMT [1]}, however, authors have used an additional target-to-source model to auto-encode the source sentence. In our work, we only use a single source-to-target model for the AE objective.
