A major challenge in event retrieval and event recognition is the construction of appropriate video representations, which should ideally be both discriminative for efficient disambiguation and compact for computational efficiency. Conventionally, a video representation is a fixed-length per-video global feature vector extracted from many frame-level appearance features [1]}, [2]}, [3]}, [4]}, [5]}. In a typical event recognition task, this video representation is fed into a linear classifier [5]} or a neural network [7]}, [4]} for classification. However, this procedure is generally incompatible with event recounting, as tracing the decision back to individual frame locations is impossible due to the irreversible video representation (global feature) extraction. Therefore, most existing methods perform event recounting as an extra post-processing step [2]}, [10]}, [11]}.
