Action Duration Prediction for Segment Level Alignment of Weakly Labeled Videos
This paper focuses on weakly supervised action alignment, where only the ordered sequence of video level actions is available for training. We propose a novel Duration Network1, which captures a short temporal window of the video and learns to predict the remaining duration of a given action at any point in time with a level of granularity based on the type of that action. Further, we introduce a Segment Level Beam Search to obtain the best alignment, that maximizes our posterior probability. Segment Level Beam Search efficiently aligns actions by considering only a selected set of frames that have more confident predictions. The experimental results show that our alignments for long videos are more robust than existing models. Moreover, the proposed method achieves state of the art results in certain cases on the popular Breakfast and Hollywood Extended datasets.
2021 IEEE Winter Conference on Applications of Computer Vision (WACV)