Action Localization with Class-agnostic Attention. STPN [1]} proposed to learn attention through class-agnostic features but has a low performance as cross entropy loss alone does not train accurate attention signals. BG modeling [2]} used a clustering loss to separate action from background. BG modeling [2]} and BaSNet [4]} force all background frames to belong to one specific class which is not desirable as they do not share any common semantics. RPN [5]}, and Huang et al. [6]} increase inter-class separateness by pushing action (or sub-action) features to their prototypes. Huang et al. [6]} outperforms RPN [5]} by modeling the relations between sub-actions of each action. DGAM [9]} addressed the action-context confusion through imposing different attentions on different features with a generative model. EM-MIL [10]} employed Expectation-Maximization to capture complete action instances and outperformed DGAM [9]} on THUMOS14 dataset.
