Based on the separation of the RD performance, more efforts should be paid to promote the unified auto-encoder network for the quality of the reconstruction and improve the spatiotemporal entropy network for less compressed bitstream.
To further enhance the performance of the proposed framework, an interesting topic is to use multiple latents for the spatiotemporal entropy network, such as the ConvLSTMs [1]} for temporal relationships, or B-frame compression.
To make up the gap with traditional video codecs, such as H.264 and H.265, the input of learned video compression may be consistent with the traditional codecs, exploring the training methods in YUV color space.
To be adaptive to the different video contents, some works need to be conducted to explore finetuning the Encoder in the proposed framework.
