It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning.
Since occlusion is a consequence of depth and motion, it is inevitable to model occlusion in order to accurately estimate flow. Most existing methods jointly estimate optical flow and occlusion. Based on the methodology, we divide them into three major groups. The first group treats occlusion as outliers and predict target pixels in the occluded regions as a constant value or through interpolation @cite_37 @cite_29 @cite_10 @cite_32 . The second group deals with occlusion by exploiting the symmetric property of optical flow and ignoring the loss penalty on predicted occluded regions @cite_50 @cite_31 @cite_20 . The last group builds more sophisticated frameworks such as modeling depth or a layered representation of objects to reason about occlusion @cite_11 @cite_42 @cite_22 @cite_43 . Our model is similar to the second group, such that we do not take account the difference where the occlusion happens into the loss function. To the best of our knowledge, we are the first to incorporate such kind of method with a neural network in an end-to-end trainable fashion. This helps our model to obtain more robust flow estimation around the occlusion boundary @cite_7 @cite_5 .
@cite_37 @cite_22 @cite_7 @cite_29 @cite_42 @cite_32 @cite_43 @cite_50 @cite_5 @cite_31 @cite_10 @cite_20 @cite_11
2770424797 2962864875 2007012942 2887479417
It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning. It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning. Most motion estimation algorithms (optical flow, layered models) cannot handle large amount of occlusion in textureless regions, as motion is often initialized with no occlusion assumption despite that occlusion may be included in the final objective. To handle such situations, we propose a local layering model where motion and occlusion relationships are inferred jointly. In particular, the uncertainties of occlusion relationships are retained so that motion is inferred by considering all the possibilities of local occlusion relationships. In addition, the local layering model handles articulated objects with self-occlusion. We demonstrate that the local layering model can handle motion and occlusion well for both challenging synthetic and real sequences. Occlusions play an important role in disparity and optical flow estimation, since matching costs are not available in occluded areas and occlusions indicate depth or motion boundaries. Moreover, occlusions are relevant for motion segmentation and scene flow estimation. In this paper, we present an efficient learning-based approach to estimate occlusion areas jointly with disparities or optical flow. The estimated occlusions and motion boundaries clearly improve over the state-of-the-art. Moreover, we present networks with state-of-the-art performance on the popular KITTI benchmark and good generic performance. Making use of the estimated occlusions, we also show improved results on motion segmentation and scene flow estimation.