Commodity RGB-D sensors capture color images along with dense pixel-wise depth information in real-time. Typical RGB-D sensors are provided with a factory calibration and exhibit erratic depth readings due to coarse calibration values, ageing and thermal influence effects. This limits their applicability in computer vision and robotics. We propose a novel method to accurately calibrate depth considering spatial and thermal influences jointly. Our work is based on Gaussian Process Regression in a four dimensional Cartesian and thermal domain. We propose to leverage modern GPUs for dense depth map correction in real-time. For reproducibility we make our dataset and source code publicly available.
The availability of modern commodity RGB-D sensors raised the interest in methods for accurate calibration thereof. Typical approaches @cite_8 @cite_10 use a calibration target together with intrinsic and extrinsic parameter estimation methods, such as the one proposed by Zhang @cite_5 . Since one usually has no access to the internals of the depth generation process, these attempts are insufficient for accurate depth correction, because they merely correct for lens effects. In contrast, the work of @cite_16 assumes planarity in the observed data. They propose to correct depth by computing a pixel-wise mean residual depth image over all calibration poses. @cite_17 exploit co-planarity in the structure of the calibration target and propose to model the observed depth as a linear function of the true depth. In the work of @cite_14 a second order polynomial per pixel is proposed to compensate for depth artefacts. The closest work compared to ours, in terms of applied methods, is by Amamra et. al @cite_6 , who use a Gaussian Process to predict absolute depth from spatial locations. More recently online depth calibration methods try to compensate depth errors using a visual SLAM system @cite_7 .
@cite_14 @cite_7 @cite_8 @cite_6 @cite_5 @cite_16 @cite_10 @cite_17
1976617541 2775132987 2120484897 2167667767 1965386961 2097107088
Highlights? A calibration procedure for depth and color cameras. ? RGB, IR and D cameras joint calibration. ? Spatially variant depth correction model for calibrated RGB-D devices. ? A novel stereoscopic rendering technique by taking into account eyes' position. ? A stereoscopic Augmented Reality system for natural Human-machine interaction. A Human-machine interaction system requires precise information about the user's body position, in order to allow a natural 3D interaction in stereoscopic augmented reality environments, where real and virtual objects should coherently coexist. The diffusion of RGB-D sensors seems to provide an effective solution to such a problem. Nevertheless, the interaction with stereoscopic 3D environments, in particular in peripersonal space, requires a higher degree of precision. To this end, a reliable calibration of such sensors and an accurate estimation of the relative pose of different RGB-D and visualization devices are crucial. Here, robust and straightforward procedures to calibrate a RGB-D camera, to improve the accuracy of its 3D measurements, and to co-register different calibrated devices are proposed. Quantitative measures validate the proposed approach. Moreover, calibrated devices have been used in an augmented reality system, based on a dynamic stereoscopic rendering technique that needs accurate information about the observer's eyes position. Modern consumer RGB-D cameras are affordable and provide dense depth estimates at high frame rates. Hence, they are popular for building dense environment representations. Yet, the sensors often do not provide accurate depth estimates since the factory calibration exhibits a static deformation. We present a novel approach to online depth calibration that uses a visual SLAM system as reference for the measured depth. A sparse map is generated and the visual information is used to correct the static deformation of the measured depth while missing data is extrapolated using a small number of thin plate splines (TPS). The corrected depth can then be used to improve the accuracy of the sparse RGB-D map and the 3D environment reconstruction. As more data becomes available, the depth calibration is updated on the fly. Our method does not rely on a planar geometry like walls or a one-to-one-pixel correspondence between color and depth camera. Our approach is evaluated in real-world scenarios and against ground truth data. Comparison against two popular self-calibration methods is performed. Furthermore, we show clear visual improvement on aggregated point clouds with our method. In this work, we present a novel method to accurately calibrate active depth sensors such as the Microsoft Kinect. Our approach is based on the Gaussian Process Regression (GPR). It is applied after the standard calibration and it is particularly useful for the aging depth cameras. The latter, were proven to suffer from inaccuracies that cannot be fixed with the standard pinhole camera calibration procedure. Experimental results show the weaknesses of this naive calibration and the corrective of effect our algorithm. We further justify the possibility to extend the same approach to any other type of cameras with similar characteristics. We propose a flexible technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique and very good results have been obtained. Compared with classical techniques which use expensive equipment such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one more step from laboratory environments to real world use. We analyze Kinect as a 3D measuring device, experimentally investigate depth measurement resolution and error properties and make a quantitative comparison of Kinect accuracy with stereo reconstruction from SLR cameras and a 3D-TOF camera. We propose Kinect geometrical model and its calibration procedure providing an accurate calibration of Kinect 3D measurement and Kinect cameras. We demonstrate the functionality of Kinect calibration by integrating it into an SfM pipeline where 3D measurements from a moving Kinect are transformed into a common coordinate system by computing relative poses from matches in color camera. Commodity depth cameras have created many interesting new applications in the research community recently. These applications often require the calibration information between the color and the depth cameras. Traditional checkerboard based calibration schemes fail to work well for the depth camera, since its corner features cannot be reliably detected in the depth image. In this paper, we present a maximum likelihood solution for the joint depth and color calibration based on two principles. First, in the depth image, points on the checker-board shall be co-planar, and the plane is known from color camera calibration. Second, additional point correspondences between the depth and color images may be manually specified or automatically established to help improve calibration accuracy. Uncertainty in depth values has been taken into account systematically. The proposed algorithm is reliable and accurate, as demonstrated by extensive experimental results on simulated and real-world examples.