Modeling short-term temporal dependencies.
Other research has investigated the middle ground between late aggregation (of frame features) and early temporal processing (to get optical flow), by modeling short-term dependencies. This includes differencing of intermediate features [1]} and combining Sobel filtering with feature differencing [2]}. Other works [3]}, [4]} develop a differentiable network that performs TV-L1 [5]}, a popular optical flow extraction technique. The work of [6]} instead uses a set of fixed filters for extracting motion features, thereby greatly reducing the number of parameters. DMC-Nets [7]} leverage motion vectors in the compressed video to synthesize discriminative motion cues for two-stream action recognition at low computational cost compared to raw flow extraction.
