Optical flow is one of the most fundamental quantities in computer vision and robotics which measures the 2D motion of each pixel between consecutive frames. Contrary to high-level computer vision applications like instance segmentation, ground truth cannot be reliably obtained from real-world data with human annotation. Recent advances in optical flow methods – PWC-Net [1]}, RAFT [2]} and GMFlow [3]} all rely on synthetic data like MPI Sintel [4]}, FlyingChairs [5]} and FlyingThings3D [6]} for pre-training. Such datasets use synthetic chairs and lack photorealism and realistic 3D motion. Recent advances in dataset generation – AutoFlow [7]} learns to render hyperparameters for generating synthetic flow datasets reducing the End Point Error (EPE). AutoFlow lacks 3D motion and photorealism as it utilizes a simple 2D layered model. Having photo-realistic data, Kubric [8]} tends to outperform AutoFlow, but only on Sintel Clean samples. This is the rendered output that contains shading but no image degradation whereas the Final pass includes motion blur, defocus blur and atmospheric effects. Since Kubric lacks both volumetric features (like fog, mist, and rain) and advanced camera features (such as depth of field and motion blur), AutoFlow performs better even without photorealism as images are generated with motion blur and fog models as well as data augmentation for visual effects.
