To solve the temporal incoherence problem, most video stylization models track the motion of pixels along time in the original video and use this information to stylize videos. This leads to the need for optical flow [1]}. Many algorithms have been proposed to estimate optical flow in videos, including (slow) iterative algorithms [1]}, [3]} and (fast) deep learning models [4]}, [5]}.
Most algorithms are developed under the assumption of brightness constancy—a pixel's intensity or color does not change when it flows from one frame to another [3]}. This assumption alone leads to an ill-posed problem (one constraint for two unknowns at each pixel), and different algorithms use different prior terms for regularization (see [3]} for a review).
