Video frame interpolation, aiming to synthesize intermediate frames between existing ones of a video, is a longstanding problem. The recent mainstream VFI methods are the motion-based methods, which first estimate the bidirectional optical flows and then use them to warp the input frames.
Liu et al. [1]} introduced a deep network that produced 3D optical flow vectors and warps input frames by trilinear sampling. Recent works have explored a few strategies for improving the performance of such methods. These strategies include utilizing additional contextual information to interpolate high-quality results [2]}, developing unsupervised techniques by cycle consistency [3]}, detecting the occlusion by exploring the depth information [4]}, forward warping input frames using softmax splatting [5]}, leveraging the transformer [6]} to model the long-term dependencies, exploring more diverse information using many-to-many flow [7]}, [8]}, and constructing efficient architectures for speeding up the interpolation [9]}.
Although significant progress has been made by these motion-based VFI approaches, their basic assumption that the motions between neighboring frames are uniform linearity, seriously limits their performance in complex scenarios.
Recently, a more accurate quadratic model was proposed in [10]} to replace the naive linear model for estimating motion. Nevertheless, the complexities and irregularities of real-world motions cannot be completely captured by a simple mathematical model [11]}.
In this paper, we employ a pyramid deformable network to compensate for the defects of the motion-based approaches.
