Multi region Two Stream R CNN for Action Detection
We propose a multi region two stream R CNN model for action detection in realistic videos. We start from frame level action detection based on faster R CNN [1] and make three contributions: (1) we show that a motion region proposal network generates high quality proposals which are complementary to those of an appearance region proposal network; (2) we show that stacking optical flow over several frames significantly improves frame level action detection; and (3) we embed a multi region scheme in the faster R CNN model, which adds complementary information on body parts. We then link frame level detections with the Viterbi algorithm, and temporally localize an action with the maximum subarray method. Experimental results on the UCF Sports, J HMDB and UCF101 action detection datasets show that our approach outperforms the state of the art with a significant margin in both frame mAP and video mAP.
ECCV