Many methods [1]}, [2]}, [3]}, [4]} localize the actions by applying thresholds on attention scores. The disadvantage of thresholding is that the snippets are treated independently and their temporal relations are neglected. Also, thresholding may not be robust to noises in class activation maps. Shou et al. [5]} in AutoLoc directly predict the temporal boundary of each action instance. A localization branch is designed to directly predict the action boundaries (inner boundaries). The outer boundaries are also obtained by inflating the inner boundaries. Knowing that a video includes action class \(c\) , an outer-inner-contrastive (OIC) loss is applied on the activation scores of action \(c\) . The OIC loss computes the average activation in the outer area minus the average activation in the inner area to encourage high activations inside and penalize high activations outside because a complete action clip should look different from its neighbours. Liu et al. [6]} proposed CleanNet to exploit temporal contrast for action localization. A contrast score is generated by summing up action, starting and ending scores for each action proposal. The action localization is trained by maximizing the average contrast score of the proposals, which penalizes fragmented short proposals and promotes completeness and continuity in action proposals.
