1. In a computing environment, a system comprising at least one processor, and an interest point detection component configured to detect possible points of interest in video data, wherein the possible points of interest correspond to action motions in the video data, a hierarchical motion filter configured to process the possible points of interest into a set of interest points corresponding to motion in video data, wherein the hierarchical motion filter comprises a global filtering and a local filtering component, from which feature descriptors representing regions corresponding to those interest points are generated, each feature descriptor including a spatial component and a temporal component, wherein the spatial component is first obtained separately from the temporal component, wherein the spatial component and the temporal component are subsequently combined to detect possible points of interest, wherein the spatial component and the temporal component detect the possible points of interest in the video data with a most recent motion association with the possible points of interest, and wherein the hierarchical motion filter distinguishes the action motions from background motions in the video data.