
Class-agnostic attention learning with prototypes: Prototypical network which was introduced in [1]} for classification task, represents each class as a prototype and matches each instance with a prototype with highest similarity. During training, the semantically-related prototypes are pushed closer than unrelated prototypes. Huang et al. in RPN [2]} proposed a prototype learning scheme for action localization. For temporal position \(t\)  and action class \(c\) , the similarity score \(s_{t,c}\)  between feature \(x_t\)  and prototype \(p_c\)  is computed and similarity vector \(s_t\)  consists of \(s_{t,c}\)  for all classes. Then the similarity vector \(s_t\)  is fused with attention score \(\lambda _t\)  into a video-level score \(\hat{s}\) :
\(s_{t,c} = - \left\Vert x_t - p_c\right\Vert ^2_2 \ \ , \ \ \hat{s} = \sum _{t=1}^{T} \lambda _t s_t.\) 
