Event recognition and detection have attracted wide attention in the last decade. In general, event video recognition system usually consists of three stages, i.e., feature extraction, feature aggregation/pooling and training/recognition. As in event retrieval, the first two stages aim at building discriminative video representations. Previous methods focused mostly on designing better video features or representations for the classifier, such as hand-crafted visual features [1]}, [2]}, motion features [3]}, [4]}, audio features [5]}, and mid-level concept/attribute features [6]}, [7]}. Recently, the advancements of CNN [8]}, [9]} lead to promising results in event recognition task [10]}, [11]}, [12]}. The video representations are usually constructed by direct aggregation of the frame-level CNN features. Due to limited amount of training data, video representations are typically fed to classifiers such as the Support Vector Machine (SVM)[13]} or multi-feature fusion framework [10]}, [12]}, [16]}, [17]}.
