We then focus on a new case study of hierarchical deep reinforcement learning for video captioning<cite> (Wang et al., 2018b)</cite> , discussing the techniques of leveraging hierarchies in DRL for NLP generation problems.