Inspired by the success of image classification, convolutional neural networks (CNN) have also been applied in visual event recognition. Several methods have been proposed to take advantage of the temporal information. The usage of recurrent neural networks (RNN) on top of 2D convolutional layers is investigated in [1]}, [2]}, [3]} to take into account long-term dependencies. Li et al. went further by proposing a convolutional long-short term memory (LSTM) [4]}. Another approach was to extend 2D convolution kernels to 3D convolution kernels to learn spatio-temporal features [5]}, [6]}, [7]}. In addition, to reduce complexity, the 3D convolution is decomposed into two convolutions: a spatial 2D convolution and a temporal 1D convolution [8]}, [9]}. Another strategy was to capture fine low-level motion by calculating optical flow [10]}, [11]}, [12]}. However, all these techniques do not exploit an important part of the video classification: the acoustic information.
