Reducing computations by selective processing. [1]} and [2]} develop a conditional computing approach that decides whether to compute fine resolution features in a data dependent manner, to avoid additional compute overhead. AdaFrame [3]} trains a policy gradient network to determine which frame to process next, thereby skipping processing of non-informative frames in a video. SCSampler [4]} develops a light-weight model for identifying the most salient clip for recognizing the action present in a video. AdaFuse [5]} learns a decision policy to reduce temporal redundancy by removing feature channels that are uninformative and reuse the feature channels from past frames for efficient inference. A reinforcement policy based learning approach for selecting the optimum input frame resolution is developed in [6]}. [7]} follows the approach of knowledge distillation to train a student network that processes a smaller number of frames using a teacher network trained with a larger number of frames. All these approaches are based on existing spatio-temporal feature extractors or architectures that require large memory requirements due to the model size. Even though these approaches reduce the computations in inference time, majority of them suffer from heavy computations during the training stage.
