dialogpt was trained on Reddit conversations of 147M exchanges [1]}.
It is an autoregressive lm based on gpt-2.
Its second version (RetGen) is a hybrid retrieval-augmented/grounded version.
In single-turn conversations, it achieved sota in human evaluation and performance that is close to human in open-domain dialogues, besides achieving sota in automatic evaluation.
The large model has 762M parameters with 36 Transformer layers; the medium model has 345M parameters with 24 layers; the small model has 117M parameters with 12 layers.
A multiturn conversation session is framed as a long text in the model and the generation as language modeling.
The model is easily adaptable to new dialogue datasets with few samples.
The RetGen version of the model jointly trains a grounded generator and document retriever [2]}.
