Meena is presented by adiwardana2020towards.
It is a multi-turn open-domain conversational ai seq2seq model that was trained end-to-end [1]}.
The underlying architecture of this seq2seq model is the Evolved Transformer (ET).
It has 2.6B parameters and includes 1 ET encoder stack and 13 ET decoder stacks.
Manual coordinate-descent search was used to determine the hyperparameters of the best Meena model.
The data it was trained on is a filtered public domain corpus of social media conversations containing 40B tokens.
Perplexity was used to automatically evaluate the model.
It was also evaluated in multi-turn conversations using the human evaluation metric: Sensibleness and Specificity Average (SSA).
This combines two essential aspects of a human-like chatbot: being specific and making sense.
