Human evaluation of conversations is typically in the format where humans rate the quality of machine utterances given context, without actually taking part in the conversation, as in <cite>(Das et al. 2017b</cite> ) and (Li et al. 2016) .