The first are generative models, which are usually trained with cross-entropy to generate responses word-by-word conditioned on a dialogue context [Ritter et al., 2011 , Vinyals and Le, 2015 , Sordoni et al., 2015 , Shang et al., 2015<cite> , Li et al., 2016a</cite> , Serban et al., 2016b .