Existing event recognition algorithms rely on a compact and discriminative video representation, which is typically direct average of the frame-level descriptors [1]}, [2]}, [3]}, [4]}. Such sum-aggregation strategy disregards the strong temporal correlations among consecutive frames, which may over-emphasize certain long or recurrent shots in the video. Therefore, irrelevant and repetitive shots in the video might dominate the obtained video representation. For instance, in the event “Dominique Strauss-Kahn arrested" as shown in Figure REF , many video frames showing the news anchor are irrelevant, but they are visually similar, therefore the simple averaging aggregation strategy for video representation may over-emphasize such irrelevant frames. To mitigate this problem, we propose the feature alignment procedure to regularize the influence of frame features.
