Both GPT-2\(_{finetune}\)  and DialoGPT\(_{finetune}\)  are trained on 4 Nvidia V100 machines until there is no progress in validation loss. We adopt Adam algorithm [1]} in optimization with a learning rate 1e-5 as in [2]}. For all the three models, we use the vocabulary in OpenAI GPT-2 and DialoGPT that consists of \(50,257\)  entries. The latest 128 tokens in a context are kept as input.
<TABLE><TABLE>