We present a novel method for simultaneous learning of depth, egomotion, object motion, and camera intrinsics from monocular videos, using only consistency across neighboring video frames as supervision signal. Similarly to prior work, our method learns by applying differentiable warping to frames and comparing the result to adjacent ones, but it provides several improvements: We address occlusions geometrically and differentiably, directly using the depth maps as predicted during training. We introduce randomized layer normalization, a novel powerful regularizer, and we account for object motion relative to the scene. To the best of our knowledge, our work is the first to learn the camera intrinsic parameters, including lens distortion, from video in an unsupervised manner, thereby allowing us to extract accurate depth and motion from arbitrary videos of unknown origin at scale. We evaluate our results on the Cityscapes, KITTI and EuRoC datasets, establishing new state of the art on depth prediction and odometry, and demonstrate qualitatively that depth prediction can be learned from a collection of YouTube videos.
Calibration of wide field-of-view cameras is a fundamental step for numerous visual media production applications, such as 3D reconstruction, image undistortion, augmented reality and camera motion estimation. However, existing calibration methods require multiple images of a calibration pattern (typically a checkerboard), assume the presence of lines, require manual interaction and or need an image sequence. In contrast, we present a novel fully automatic deep learning-based approach that overcomes all these limitations and works with a single image of general scenes. Our approach builds upon the recent developments in deep Convolutional Neural Networks (CNN): our network automatically estimates the intrinsic parameters of the camera (focal length and distortion parameter) from a single input image. In order to train the CNN, we leverage the great amount of omnidirectional images available on the Internet to automatically generate a large-scale dataset composed of millions of wide field-of-view images with ground truth intrinsic parameters. Experiments successfully demonstrated the quality of our results, both quantitatively and qualitatively. The focal length information of an image is indispensable for many computer vision tasks. In general, focal length can be obtained via camera calibration using specific planner patterns. However, for images taken by an unknown device, focal length can only be estimated based on the image itself. Currently, most of the single-image focal length estimation methods make use of predefined geometric cues (such as vanishing points or parallel lines) to infer focal length, which constrains their applications mainly on manmade scenes. The machine learning algorithms have demonstrated great performance in many computer vision tasks, but these methods are seldom used in the focal length estimation task, partially due to the shortage of labeled images for training the model. To bridge this gap, we first introduce a large-scale dataset FocaLens, which is especially designed for single-image focal length estimation. Taking advantage of the FocaLens dataset, we also propose a new focal length estimation model, which exploits the multiscale detection architecture to encode object distributions in images to assist focal length estimation. Additionally, an online focal transformation approach is proposed to further promote the model’s generalization ability. Experimental results demonstrate that the proposed model trained on FocaLens can not only achieve state-of-the-art results on the scenes with distinct geometric cues but also obtain comparable results on the scenes even without distinct geometric cues. 