More complex approaches to condition modalities have been explored, for example, the attention mechanism. Attention models were first proposed for object detection [1]} and then used for other applications such as natural language processing with the self-attention mechanism [2]}. Attention has been applied to video classification under the form of temporal and/or spatial attention [3]}, [4]}, [5]}. However, these models did not include an interaction between modality paths. Only Tian et al. proposes to realize the interaction between paths by implementing visual attention guided by audio to condition vision [6]}.
