In recent years, with the fast development of the deep learning based compression techniques [1]}, [2]}, [3]}, several studies contribute some new compression schemes that could directly support downstream intelligent tasks without decoding all the compressed bitstream [4]}. Torfason et al. , [5]} use a neural network to generate a compressed bitstream as input for supporting the downstream tasks directly, such as classification and segmentation, which bypasses decoding of the compressed representation into RGB space, and thus reducing computational cost. Similar ideas can be found in the video-based schemes, CoViAR [6]} and DMC-Net [7]}, they directly leverage the motion vectors and residuals that are both readily available in the compressed video to represent motion at no cost to support the downstream action recognition task. However, these schemes are still task-specific, or said, designed for a limited range of applications, they cannot meet the higher and more general requirements for flexibility and efficiency. Because they did not consider the intrinsic semantics contained in the compressed bitstream, and can not leverage different structural bitstream for different tasks.
