Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> This paper proposes a novel 3D scene interpretation approach for robots in mobile manipulation scenarios using a set of 3D point features (Fast Point Feature Histograms) and probabilistic graphical methods (Conditional Random Fields). Our system uses real time stereo with textured light to obtain dense depth maps in the robot's manipulators working space. For the purposes of manipulation, we want to interpret the planar supporting surfaces of the scene, recognize and segment the object classes into their primitive parts in 6 degrees of freedom (6DOF) so that the robot knows what it is attempting to use and where it may be handled. The scene interpretation algorithm uses a two-layer classification scheme: i) we estimate Fast Point Feature Histograms (FPFH) as local 3D point features to segment the objects of interest into geometric primitives; and ii) we learn and categorize object classes using a novel Global Fast Point Feature Histogram (GFPFH) scheme which uses the previously estimated primitives at each point. To show the validity of our approach, we analyze the proposed system for the problem of recognizing the object class of 20 objects in 500 table settings scenarios. Our algorithm identifies the planar surfaces, decomposes the scene and objects into geometric primitives with 98.27% accuracy and uses the geometric primitives to identify the object's class with an accuracy of 96.69%. <s> BIB001 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> In our recent work [1], [2], we proposed Point Feature Histograms (PFH) as robust multi-dimensional features which describe the local geometry around a point p for 3D point cloud datasets. In this paper, we modify their mathematical expressions and perform a rigorous analysis on their robustness and complexity for the problem of 3D registration for overlapping point cloud views. More concretely, we present several optimizations that reduce their computation times drastically by either caching previously computed values or by revising their theoretical formulations. The latter results in a new type of local features, called Fast Point Feature Histograms (FPFH), which retain most of the discriminative power of the PFH. Moreover, we propose an algorithm for the online computation of FPFH features for realtime applications. To validate our results we demonstrate their efficiency for 3D registration and propose a new sample consensus based method for bringing two datasets into the convergence basin of a local non-linear optimizer: SAC-IA (SAmple Consensus Initial Alignment). <s> BIB002 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> We present the Viewpoint Feature Histogram (VFH), a descriptor for 3D point cloud data that encodes geometry and viewpoint. We demonstrate experimentally on a set of 60 objects captured with stereo cameras that VFH can be used as a distinctive signature, allowing simultaneous recognition of the object and its pose. The pose is accurate enough for robot manipulation, and the computational cost is low enough for real time operation. VFH was designed to be robust to large surface noise and missing depth information in order to work reliably on stereo data. <s> BIB003 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> Object recognition and especially object class recognition is and will be a key capability in home robotics when robots have to tackle manipulation tasks and grasp new objects or just have to search for objects. The goal is to have a robot classify 'never before seen objects' at first occurrence in a single view in a fast and robust manner. The classification task can be seen as a matching problem, finding the most appropriate 3D model and view with respect to a given depth image. We introduce a single-view shape model based classification approach using RGB-D sensors and a novel matching procedure for depth image to 3D model matching leading inherently to object classification. Utilizing the inter-view similarity of the 3D models for enhanced matching, the average precision of our descriptors is increased of up to 15% resulting in high classification accuracy. The presented adaptation of 3D shape descriptors to 2.5D data enables us to calculate the features in real time, directly from the 3D points of the sensor, without any calculation of normals or generating a mesh from it which is typical of state-of-art methods. Furthermore, we introduce a semi-automatic, user-centric approach to utilize the Internet for acquiring the required training data in the form of 3D models which significantly reduces the time for teaching new categories. <s> BIB004 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> This paper presents a novel method for alignment of geometrically similar 3D models. It is based on the prior that both models have at least one common tangent plane on which both can stand stably and when standing on it the models are partially aligned. To determine the final rotation around the stable plane's normal, needed for a complete alignment, we adapt an image alignment technique based on the log-polar transformation. Because the set of stable planes of a model is small enough, alignment is efficiently approached as a global optimization problem that finds the common stable plane providing the best alignment according to a similarity measure. As the method does not rely on any kind of global symmetry features, we show it can be used to register incomplete stereo point clouds of objects located on a stable plane (table, ground, etc.) with the corresponding similar 3D models. We evaluate the 3D-alignment method by comparing it to the well-known CPCA and show a significant improvement when aligning 120 models belonging to 12 different classes. <s> BIB005 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> In this work we address joint object category and instance recognition in the context of RGB-D (depth) cameras. Motivated by local distance learning, where a novel view of an object is compared to individual views of previously seen objects, we define a view-to-object distance where a novel view is compared simultaneously to all views of a previous object. This novel distance is based on a weighted combination of feature differences between views. We show, through jointly learning per-view weights, that this measure leads to superior classification performance on object category and instance recognition. More importantly, the proposed distance allows us to find a sparse solution via Group-Lasso regularization, where a small subset of representative views of an object is identified and used, with the rest discarded. This significantly reduces computational cost without compromising recognition accuracy. We evaluate the proposed technique, Instance Distance Learning (IDL), on the RGB-D Object Dataset, which consists of 300 object instances in 51 everyday categories and about 250,000 views of objects with both RGB color and depth. We empirically compare IDL to several alternative state-of-the-art approaches and also validate the use of visual and shape cues and their combination. <s> BIB006 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> Humanoid robots should be able to grasp and handle objects in the environment, even if the objects are seen for the first time. A plausible solution to this problem is to categorize these objects into existing classes with associated actions and functional knowledge. So far, efforts on visual object categorization using humanoid robots have either been focused on appearance-based methods or have been restricted to object recognition without generalization capabilities. In this work, a shape model-based approach using stereo vision and machine learning for object categorization is introduced. The state-of-the-art features for shape matching and shape retrieval were evaluated and selectively transfered into the visual categorization. Visual sensing from different vantage points allows the reconstruction of 3D mesh models of the objects found in the scene by exploiting knowledge about the environment for model-based segmentation and registration. These reconstructed 3D mesh models were used for shape feature extraction for categorization and provide sufficient information for grasping and manipulation. Finally, the visual categorization was successfully performed with a variety of features and classifiers allowing proper categorization of unknown objects even when object appearance and shape substantially differ from the training set. Experimental evaluation with the humanoid robot ARMAR-IIIa is presented. <s> BIB007 </s> Data-Driven Grasp Synthesis - A Survey <s> Object Features <s> In this paper we address the problem of grasp generation and grasp transfer between objects using categorical knowledge. The system is built upon an i) active scene segmentation module, able of generating object hypotheses and segmenting them from the background in real time, ii) object categorization system using integration of 2D and 3D cues, and iii) probabilistic grasp reasoning system. Individual object hypotheses are first generated, categorized and then used as the input to a grasp generation and transfer system that encodes task, object and action properties. The experimental evaluation compares individual 2D and 3D categorization approaches with the integrated system, and it demonstrates the usefulness of the categorization in task-based grasping and grasp transfer. <s> BIB008
Grasp Synthesis robust object categorization the approach by Madry et al. BIB008 is integrated with the 3D-based categorization system by Wohlkinger and Vincze BIB004 . The pose of the categorized object is estimated with the approach presented by Aldoma and Vincze BIB005 . Given this, the inferred grasp configuration can be checked for reachability and executed by the robot. Recently, we have seen an increasing amount of new approaches towards pure 3D descriptors of objects for categorization. Although, the following methods look promising, it has not been shown yet that they provide a suitable base for generalizing grasps over an object category. Rusu et al. BIB003 BIB001 provide extensions of BIB002 for either recognizing or categorizing objects and estimating their pose relative to the viewpoint. While in BIB001 quantitative results on real data are presented, BIB003 uses simulated object point clouds only. Lai et al. BIB006 perform object category and instance recognition. The authors learn an instance distance using the database presented in  . A combination of 3D and 2D features is used. However, the focus of this work is not grasping. Gonzalez-Aguirre et al. BIB007 present a shape-based object categorization system. A point cloud of an object is reconstructed by fusing partial views. Different descriptors (capturing global and local object shape) in combination with standard machine learning techniques are studied. Their performance is evaluated on real data.