Next-Utterance-Classification (NUC, see Figure 1 ) is a task, which is straightforward to evaluate, designed for training and validation of dialogue systems. They are evaluated using the metric of Recall@k, which we define in this section. In NUC, a model or user, when presented with the context of a conversation and a (usually small) pre-defined list of responses, must select the most appropriate response from this list. This list includes the actual next response of the conversation, which is the desired prediction of the model. The other entries, which act as false positives, are sampled from elsewhere in the corpus. Note that no assumptions are made regarding the number of utterances in the context: these can be fixed or sampled from arbitrary distributions. Performance on this task is easy to assess by measuring the success rate of picking the correct next response; more specifically, we measure Recall@k (R@k), which is the percentage of correct responses (i.e. the actual response of the conversation) that are found in the top k responses with the highest rankings according to the model. This task has gained some popularity recently for evaluating dialogue systems<cite> (Lowe et al., 2015a</cite>; Kadlec et al., 2015) .