There has been an increasing interest to look beyond a single action label for video understanding [1]}. Even for action recognition, it is important for video models to look at multiple elements in a video, such as objects, scenes, people, and their interactions [2]}, [3]}, [4]}. One prominent approach to achieve this is through multi-task learning (MTL), where video models are not only trained for recognizing actions (the primary task of interest) but also other auxiliary tasks such as recognizing objects and scenes. The assumption is that encouraging the model to do well on various auxiliary tasks improves the generality of the learned feature representation, which in turn improves the performance and generalization of action recognition.
<FIGURE>