Memory Augmented Recurrent Neural Networks for Dense Video Captioning
Dense video captioning is a challenging computer vision task that involves effectively understanding long video sequences. In this work, we address this problem by augmenting recurrent neural network architectures with external memory. We propose a dense captioning model that incorporates external memory augmentation both to encode video and densely caption it. We demonstrate that recurrent video encoder and dense captioner networks augmented with external memory can be used to effectively encode frames based on the content of the entire video, as well as for generating dense captions better than recurrent networks without external memory augmentation. We conduct experiments on the ActivityNet Captions and YouCook II datasets to demonstrate the potential of external memory augmentation.