We present and evaluate an approach for human-in-the-loop specification of shape reconstruction with annotations for basic robot-object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to specify shape and delineate a simple task. The goal is to explore reducing the complexity of CAD-like interfaces so that novice users can quickly recover an object's shape and describe a manipulation task that is then carried out by a robot. The object modeling and interaction annotation capabilities are tested with a user study and compared against results obtained using existing approaches. The approach has been analyzed using a variety of shape comparison, grasping, and manipulation metrics, and tested with the PR2 robot platform, where it was shown to be successful.
Off-the-shelf sensors like the Microsoft Kinect give easy access to these input streams, and have made the 3D scanning of environments and objects ubiquitous. Several algorithms have been developed and refined to allow the integration of depth sensing onto unified models of the scenes they capture @cite_30 . The Kinect Fusion algorithm @cite_39 , and its many variants @cite_6 , allow the tracking of a scene and integration of captured depth-frames into a coherent volumetric representation. Some variants have even accomplished deformation and part-tracking @cite_90 . One important aspect of the models that these approaches create is their sensitivity to the sensor and the context. The Kinect has a resolution of about 2 mm and has trouble capturing thin surfaces, transparent or shiny objects and requires careful scanning or large amounts of computation for dealing with noise. While various techniques exist to deal with these problems, when these appear in conjunction to context-dependent constraints, human annotations become highly valuable. For an in-depth review of automatic and semi-automatic methods of reconstruction from point clouds, see @cite_123 .
@cite_30 @cite_90 @cite_6 @cite_39 @cite_123
244217497 2098883970 1669953666 2743552164
The availability of commodity depth sensors such as Kinect has enabled development of methods which can densely reconstruct arbitrary scenes. While the results of these methods are accurate and visually appealing, they are quite often incomplete. This is either due to the fact that only part of the space was visible during the data capture process or due to the surfaces being occluded by other objects in the scene. In this paper, we address the problem of completing and refining such reconstructions. We propose a method for scene completion that can infer the layout of the complete room and the full extent of partially occluded objects. We propose a new probabilistic model, Contour Completion Random Fields, that allows us to complete the boundaries of occluded surfaces. We evaluate our method on synthetic and real world reconstructions of 3D scenes and show that it quantitatively and qualitatively outperforms standard methods. We created a large dataset of partial and complete reconstructions which we will make available to the community as a benchmark for the scene completion task. Finally, we demonstrate the practical utility of our algorithm via an augmented-reality application where objects interact with the completed reconstructions inferred by our method. Recent proliferation of a cheap but quality depth sensor, the Microsoft Kinect, has brought the need for a challenging category-level 3D object detection dataset to the fore. We review current 3D datasets and find them lacking in variation of scenes, categories, instances, and viewpoints. Here we present our dataset of color and depth image pairs, gathered in real domestic and office environments. It currently includes over 50 classes, with more images added continuously by a crowd-sourced collection effort. We establish baseline performance in a PASCAL VOC-style detection task, and suggest two ways that inferred world size of the object may be used to improve detection. The dataset and annotations can be downloaded at http: www.kinectdata.com. We present a novel application of the Kinect™, an input device designed for the Microsoft® Xbox 360® video game system. The device can be used by Earth scientists as a low-cost, high-resolution, short-range 3D 4D camera imaging system producing data similar to a terrestrial light detection and ranging (LiDAR) sensor. The Kinect contains a structured light emitter, an infrared camera (the combination of these two produce a distance image), a visual wavelength camera, a three-axis accelerometer, and four microphones. The cost is US $100, frame rate is 30 Hz, spatial and depth resolutions are mm to cm depending on range, and the optimal operating range is 0.5 to 5 m. The resolution of the distance measurements decreases with distance and is ≤1 mm at 0.5 m and 75 mm at 5 m. We illustrate data collection and basic data analysis routines in three experiments designed to demonstrate the breadth and utility of this new sensor in domains of glaciology, stream bathymetry, and geomorphology, although the device is applicable to a number of other Earth science fields. Copyright © 2012 John Wiley & Sons, Ltd. Commercial RGB-D cameras provide the possibility of fast, accurate, and cost-effective 3-D scanning solution in a single package. These economical depth cameras provide several advantages over conventional depth sensors, such as sonars and lidars, in specific usage scenarios. In this paper, we analyze the performance of Kinect v2 time-of-flight camera while operating fully submerged underwater in a customized waterproof housing. Camera calibration has been performed for Kinect’s RGB and NIR cameras, and the effect of calibration on the generated 3-D mesh is discussed in detail. To overcome the effect of refraction of light due to the sensor housing and water, we propose a time-of-flight correction method and a fast, accurate and intuitive refraction correction method that can be applied to the acquired depth images, during 3-D mesh generation. Experimental results show that the Kinect v2 can acquire point cloud data up to 650 mm. The reconstruction results have been analyzed qualitatively and quantitatively, and confirm that the 3-D reconstruction of submerged objects at small distances is possible without the requirement of any external NIR light source. The proposed algorithms successfully generated 3-D mesh with a mean error of ±6 mm at a frame rate of nearly 10 fps. We acquired a large data set of RGB, IR and depth data from a submerged Kinect v2. The data set covers a large variety of objects scanned underwater and is publicly available for further use, along with the Kinect waterproof housing design and correction filter codes. The research is aimed toward small-scale research activities and economical solution for 3-D scanning underwater. Applications such as coral reef mapping and underwater SLAM in shallow waters for ROV’s can be a viable application area that can benefit from results achieved.