FIELD

The present disclosure relates to a method and/or process for sensor calibration. In particular, the embodiments of the present disclosure relate to the calibration of visual and distance/depth sensors.

BACKGROUND

RGB-D sensors, or sensors that combine RGB color information with depth information, are devices that can provide color and depth information from a scene at the same time. RGB-D sensors have become popular in many applications for depth measurement and 3D modeling. The two major principles of RGB-D sensors are time of flight (ToF) and structured light (SL). Other sensors widely used in industrial measurement and manufacturing are displacement sensors such as laser displacement sensors (LDS). The displacement sensors provide non-contact, high accuracy, and high measurement speed. Combining an RGB-D camera with a displacement sensor is one of the most flexible and cost-effective approach to obtaining colorized 3D point clouds with accurate depth measurement. The limitation of a displacement sensor is that it can only measure the distance to a point.

Several methods exist for calibration of a 2D laser scanner with respect to a camera, calibration for stereo vision sensors, calibration of intrinsic and extrinsic parameters of an RGB-D sensor, and calibration of a 3D range sensor and a camera. But there lacks an effective method to calibrate an RGB-D camera and a displacement sensor.

What is needed is a method to compute the temporal and spatial relations between an RGB-D camera and a displacement sensor, including translation and rotation.

SUMMARY

The above and other needs are met by a method of calibrating an RGB-D device. It is an object of the invention of the preset disclosure to provide a calibration method that finds a relative configuration of an imaging sensor and a displacement sensor, including relative position and attitude.

In a first aspect, a method of calibrating an RGB-D device having an RGB-D camera and a displacement sensor includes: providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target, wherein the plurality of visual markers have known locations on the calibration target; positioning the calibration target within a field of view of the RGB-D device such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target; moving the calibration target a known distance in relation to a location of the RGB-D device; receiving output data from the RGB-D device including data from the RGB-D camera and displacement sensor; align data from the RGB-D camera and data from the displacement sensor; calibrate a relative configuration of the RGB-D device based on aligned data output from the RGB-D device and known position information of the calibration target.

In one embodiment, the method further includes rotating the calibration target a known amount in relation to an orientation of the RGB-D device and further calibrating a relative configuration of the RGB-D device based on the known change of rotation of the calibration target.

In another embodiment, the method includes calibrating intrinsic camera parameters and lens distortion of the RGB-D device based on known locations of the visual markers on the calibration target.

In yet another embodiment, aligning data from the RGB-D camera and data from the displacement sensor includes interpolating measurements of one of the RGB-D camera and displacement sensor.

In one embodiment, the plurality of visual markers on the calibration target are arranged in an array or known positions. In another embodiment, the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.

In a second aspect, a method of calibrating an RGB-D device having an RGB-D camera and a displacement sensor includes: providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target, wherein the plurality of visual markers has known locations on the calibration target; positioning the calibration target within a field of view of the RGB-D device such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target; moving and rotating the calibration target a known distance in relation to a location of the RGB-D device; receiving output data from the RGB-D device including data from the RGB-D camera and displacement sensor; align data from the RGB-D camera and data from the displacement sensor; calibrating intrinsic camera parameters and lens distortion of the RGB-D device based on known locations of the visual markers on the calibration target; calibrate a relative configuration of the RGB-D device based on aligned data output from the RGB-D device and known position information of the calibration target.

In one embodiment, aligning data from the RGB-D camera and data from the displacement sensor includes interpolating measurements of one of the RGB-D camera and displacement sensor.

In another embodiment, the plurality of visual markers on the calibration target are arranged in an array or known positions. In yet another embodiment, the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.

In a third aspect, a method of calibrating an RGB-D device having an RGB-D camera and a displacement sensor includes: providing a calibration target, the calibration target including a plurality of visual markers shown in a surface of the calibration target, wherein the plurality of visual markers has known locations on the calibration target; positioning the calibration target within a field of view of the RGB-D device such that the RGB-D camera and displacement sensor capture data corresponding to the calibration target; locating one or more corners of the calibration target with the RGB-D device; locating a projected measuring point of the displacement sensor projected onto the calibration target; rotating the calibration target by a defined rotation amount and recording output data from the RGB-D device corresponding to rotation of the calibration target; and solving an optimization problem of parameters of the RGB-D device to determine a relative relationship of the RGB-D device.

In one embodiment, the method further includes moving and rotating the calibration target and receiving data from the RGB-D device corresponding to additional movement and rotation of the RGB-D device.

In another embodiment, the plurality of visual markers on the calibration target are arranged in an array or known positions. In yet another embodiment, the plurality of visual markers are shaped having a corner for ready identification of the visual markers by the RGB-D device.

