Fine-tuning a pre-trained GPT-2 model for query rewriting requires large amounts of gold query rewrites \(\hat{Q}^*\)  as training data. However, the TREC CAsT datasets consists of only a few hundred manually generated query rewrites, raising a concern that such a small amount of training data is likely to lead to overfitting when used to fine-tune deep neural networks like GPT-2. A common approach is to use the crowd sourced query rewrites in CANARD [1]} as additional training data.
It has been reported that the neural query writer based on fine-tuned GPT-2 is simple and effective. It leads to one of the best performers in TREC CAsT 2019 [2]} and later becomes a standard solution adopted by many participants in TREC CAsT 2020 [3]}. The neural query rewriter is also easy to be combined with other conversational query understanding methods  [4]} to achieve better IR results.
