In this research, we address the novel task of response generation in search-based multimodal dialogue by learning from the recently released <cite>Multimodal Dialogue</cite> (<cite>MMD</cite>) dataset <cite>(Saha et al., 2017)</cite> .