The results in Table REF  show state-of-the-art relevance performance with some compromise on the response length. Here, we explore the possibility of generating longer and more diverse responses with the trained models and estimate the effect on the relevance scores. For this experiment, we chose the larger DLGNet-345M models of both datasets and tried two sampling techniques, i.e., top_k [1]} and top_p nucleus [2]}, [3]} sampling strategies on the validation sets.
The trajectory of the evaluation metrics with increasing top_k and top_p values are shown Figs. REF  and REF  respectively.
With top_k sampling, increasing the top_k value increases the response length at the expense of relevance metrics like BLEU for both datasets, as expected. However, the response length increase is more significant on the Ubuntu dataset than the Movie dataset. It is also surprising that the ROGUE-2 score for Ubuntu increases with increasing top_k value, which is the reverse of the case for the Movie dataset. Also, Fig. REF  shows that it is more advantageous to trade off relevance for diversity on the Ubuntu dataset compare to the Movie dataset. This is probably due to the size and closed-domain nature of the Ubuntu dataset, which makes it more difficult to learn with the maximum likelihood estimation only.
