In recent years, only few works exploited the information present in the audio signal. The concatenation is used in [1]}, [2]} to fuse the visual and audio paths to exploit the information from the two modalities. Long et al. went further by testing different levels of fusion in the network [3]}. These networks integrated the visual and audio information with hard fusion without exploiting a possible interaction between visual and audio paths. These works did not study more complex fusion techniques such as multimodal compact bilinear pooling (MCB) [4]} or dual multimodal residual fusion (DMR) [5]}. Furthermore, they did not take into account the different learning dynamics of the different modalities.
