Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds. <s> BIB001 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In order for humanoid robots to enter human-centred environments, it is indispensable to equip them with manipulative, perceptive and communicative skills necessary for real-time interaction with the environment and humans. The goal of our work is to provide reliable and highly integrated humanoid platforms which on the one hand allow the implementation and tests of various research activities and on the other hand the realization of service tasks in a household scenario. In this paper, we present a new humanoid robot currently being developed for applications in human-centred environments. In addition, we present an integrated grasping and manipulation system consisting of a motion planner for the generation of collision-free paths and a vision system for the recognition and localization of a subset of household objects as well as a grasp analysis component which provides the most feasible grasp configurations for each object. <s> BIB002 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In this paper, we investigate the problem of 3D object categorization of objects typically present in kitchen environments, from data acquired using a composite sensor. Our framework combines different sensing modalities and defines descriptive features in various spaces for the purpose of learning good object models. By fusing the 3D information acquired from a composite sensor that includes a color stereo camera, a time-of-flight (TOF) camera, and a thermal camera, we augment 3D depth data with color and temperature information which helps disambiguate the object categorization process. We make use of statistical relational learning methods (Markov Logic Networks and Bayesian Logic Networks) to capture complex interactions between the different feature spaces. To show the effectiveness of our approach, we analyze and validate the proposed system for the problem of recognizing objects in table settings scenarios. <s> BIB003 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In this paper we present a method for building models for grasping from a single 3D snapshot of a scene composed of objects of daily use in human living environments. We employ fast shape estimation, probabilistic model fitting and verification methods capable of dealing with different kinds of symmetries, and combine these with a triangular mesh of the parts that have no other representation to model previously unseen objects of arbitrary shape. Our approach is enhanced by the information given by the geometric clues about different parts of objects which serve as prior information for the selection of the appropriate reconstruction method. While we designed our system for grasping based on single view 3D data, its generality allows us to also use the combination of multiple views. We present two application scenarios that require complete geometric models: grasp planning and locating objects in camera images. <s> BIB004 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In this article we describe an object perception system for autonomous robots performing everyday manipulation tasks in kitchen environments. The perception system gains its strengths by exploiting that the robots are to perform the same kinds of tasks with the same objects over and over again. It does so by learning the object representations necessary for the recognition and reconstruction in the context of pick-and-place tasks. The system employs a library of specialized perception routines that solve different, well-defined perceptual sub-tasks and can be combined into composite perceptual activities including the construction of an object model database, multimodal object classification, and object model reconstruction for grasping. We evaluate the effectiveness of our methods, and give examples of application scenarios using our personal robotic assistants acting in a human living environment. <s> BIB005 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> A major challenge in modeling with BNs is learning the structure from both discrete and multivariate continuous data. A common approach in such situations is to discretize continuous data before structure learning. However efficient methods to discretize high-dimensional variables are largely lacking. This paper presents a novel method specifically aiming at discretization of high-dimensional, high-correlated data. The method consists of two integrated steps: non-linear dimensionality reduction using sparse Gaussian process latent variable models, and discretization by application of a mixture model. The model is fully probabilistic and capable to facilitate structure learning from discretized data, while at the same time retain the continuous representation. We evaluate the effectiveness of the method in the domain of robot grasping. Compared with traditional discretization schemes, our model excels both in task classification and prediction of hand grasp configurations. Further, being a fully probabilistic model it handles uncertainty in the data and can easily be integrated into other frameworks in a principled manner. <s> BIB006 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> We design an example based planning framework to generate semantic grasps, stable grasps that are functionally suitable for specific object manipulation tasks. We propose to use partial object geometry, tactile contacts, and hand kinematic data as proxies to encode semantic constraints, which are task-related constraints. We introduce a semantic affordance map, which relates local geometry to a set of predefined semantic grasps that are appropriate to different tasks. Using this map, the pose of a robotic hand can be estimated so that the hand is adjusted to achieve the ideal approach direction required by a particular task. A grasp planner is then used to generate a set of final grasps which have appropriate stability, tactile contacts, and hand kinematics along this approach direction. We show experiments planning semantic grasps on everyday objects and executing these grasps with a physical robot. <s> BIB007 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> We present a method for transferring grasps between objects of the same functional category. This transfer is intended to preserve the functionality of a grasp constructed for one of the objects, thus enabling the analogous action to be performed on a novel object for which no grasp has been specified. Manipulation knowledge is hence generalized from a single example to a class of objects with a significant amount of shape variability. The transfer is achieved through warping the surface geometry of the source object onto the target object, and along with it the contact points of a grasp. The warped contacts are locally replanned, if necessary, to ensure grasp stability, and a suitable grasp pose is computed. We present extensive results of experiments with a database of four-finger grasps, designed to systematically cover variations on grasping the mugs of the Princeton Shape Benchmark. <s> BIB008 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In this paper we address the problem of grasp generation and grasp transfer between objects using categorical knowledge. The system is built upon an i) active scene segmentation module, able of generating object hypotheses and segmenting them from the background in real time, ii) object categorization system using integration of 2D and 3D cues, and iii) probabilistic grasp reasoning system. Individual object hypotheses are first generated, categorized and then used as the input to a grasp generation and transfer system that encodes task, object and action properties. The experimental evaluation compares individual 2D and 3D categorization approaches with the integrated system, and it demonstrates the usefulness of the categorization in task-based grasping and grasp transfer. <s> BIB009 </s> Data-Driven Grasp Synthesis - A Survey <s> D. Category-based Grasp Synthesis <s> In this paper, we present an approach towards autonomous grasping of objects according to their category and a given task. Recent advances in the field of object segmentation and categorization as well as task-based grasp inference have been leveraged by integrating them into one pipeline. This allows us to transfer task-specific grasp experience between objects of the same category. The effectiveness of the approach is demonstrated on the humanoid robot ARMAR-IIIa. <s> BIB010
Most of the previously discussed approaches link low-level information of the object to a grasp. Given that a novel object is similar in shape or appearance to a previously encountered one, then it is assumed that they can also be grasped in a similar way. However, objects might be similar on a different level. Objects in a household environment that share the same functional category might have a vastly different shape or appearance. However they still can be grasped in the same way. In Section III-B1, we have already mentioned the work by Dang and Allen BIB007 , Hillenbrand and Roa BIB008 in which task-specific grasps are synthesized for objects of the same category. The authors assume that the category is known apriori. In the following, we review methods that generalize grasps to familiar objects by first determining their category. Marton et al. BIB003 use different 3D sensors and a thermo camera for performing object categorization. Features of the segmented point cloud and the segmented image region are extracted to train a Bayesian Logic Network for classifying object hypotheses as either boxes, plates, bottles, glasses, mugs or silverware. In BIB003 no grasping results are shown. A modified approach is presented in BIB005 . A layered 3D object descriptor is used for categorization and an approach based on the Scale-invariant feature transform (SIFT) BIB001 is applied for view based object recognition. To increase robustness of the categorization, the examination methods are run iteratively on the object hypotheses. A list of potential matching objects are kept and reused for verification in the next iteration. Objects for which no matching model can be found in the database are labeled as novel. Given that an object has been recognized, associated grasp hypotheses can be reused. These have been generated using the technique presented in BIB004 . Song et al. BIB006 treat object category as one variable in the Bayesian network. Madry et al. BIB009 demonstrate how the category of an object can be robustly detected given multi-modal visual descriptors of an object hypothesis. This information is fed into the Bayesian Network together with the desired task. A full hand configuration can then be inferred that obeys the task constraints. Bohg et al. BIB010 demonstrate this approach on the humanoid robot ARMAR III BIB002 . For Object-Grasp Represen.