Some of the ingredients for the success of BlenderBot, as identified by roller2020recipes, are empathy and personality, consistent persona, displaying knowledge, and engagingness.
Three different parameter models are built for the variants: 90M, 2.7B, and 9.4B.
The variants, which are all based on the Transformer, involve the latter three approaches: retrieval, generative, and a retrieve-and-refine combination of the earlier two.
The generative architecture is a seq2seq model and uses Byte-Level bpe for tokenization.
Human evaluation of multi-turn conversations, using ACUTE-Eval method, shows that its best model outperforms the previous sota on engagingness and humanness by using the Blended Skill Talk (BST) dataset [1]}.
They observed that models may give different results when different decoding algorithms are used, though the models may report the same perplexity in automatic metric.
The more recent version of the set of models learns to generate an online search query from the internet based on the context and conditions on the results to generate a response, thereby employing the latest relevant information [2]}, [3]}.
