In order to not explicitly learn sequence information, video colorization [1]} was proposed for self-supervised learning of videos. In this approach, the objective was to colorize input gray scale videos using a pre-colored reference frame. Unlike colorizing a single image, the temporal coherence in a video needed to be preserved, hence an loss function was designed to keep track of correlated pixels in consecutive frames.
