Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> Abstract This paper addresses the problem of designing a practical system able to grasp real objects with a three-fingered robot hand. A general approach for synthesizing two- and three-finger grasps on planar unknown objects is presented. Visual perception is used to reduce the uncertainty and to obtain relevant information about the objects. We focus on non-modeled planar extruded objects, which can represent many real-world objects. In addition, particular mechanical constraints of the robot hand are considered. First, a vision processing module that extracts from object images the relevant information for the grasp synthesis is developed. This is completed with a set of algorithms for synthesizing two- and three-finger grasps taking into account force-closure and contact stability conditions, with a low computational effort. Finally, a procedure for constraining these results to the kinematics of the particular hand, is also developed. In addition, a set of heuristic metrics for assessing the quality of the computed grasps is described. All these components are integrated in a complete system. Experimental results using the Barrett hand are shown and discussed. <s> BIB001 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> This paper describes the development of a novel vision-based grasping system for unknown objects based on range images. We realize a synthesis of the calculated grasp points with a 3D model of a hand prosthesis, which we are using as gripper. We locally find grasp point candidates based on the shape of the object and validate the globally by checking collisions between the gripper and surrounding objects and the table top. Our approach integrates a robust object segmentation and grasp point detection for every object on a table in front of a 7-DOF robot arm. The algorithm analyzes the top surface of every object and outputs the generated grasp points and the required gripper pose to grasp the desired object. Additionally we can calculate the optimal opening angle of the gripper. The first experimental results show that the presented automated grasping system is able to generate successful grasp points for a wide range of different objects. <s> BIB002 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> Robotic grasping in an open environment requires both object-specific as well as general grasping skills. When the objects are previously known it is possible to employ techniques that exploit object models, like geometrical grasping simulators. On the other hand, a competent system will also be able to deal with unmodeled objects using general solutions. In this paper we present an integrated system for autonomous rigid-object pick-up tasks in domestic environments, focusing on the gripping of unmodeled objects and exploiting sensor feedback from the robot hand to monitor the grasp. We describe the perception system based on time-of-flight range data, the grasp pose optimization algorithm and the grasp execution. The performance and robustness of the system is validated by experiments including pick-up tasks on many different common kitchen items. <s> BIB003 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> Robotic grasping in unstructured environments requires the ability to select grasps for unknown objects and execute them while dealing with uncertainty due to sensor noise or calibration errors. In this work, we propose a simple but robust approach to grasp selection for unknown objects, and a reactive adjustment approach to deal with uncertainty in object location and shape. The grasp selection method uses 3D sensor data directly to determine a ranked set of grasps for objects in a scene, using heuristics based on both the overall shape of the object and its local features. The reactive grasping approach uses tactile feedback from fingertip sensors to execute a compliant robust grasp. We present experimental results to validate our approach by grasping a wide range of unknown objects. Our results show that reactive grasping can correct for a fair amount of uncertainty in the measured position or shape of the objects, and that our grasp selection approach is successful in grasping objects with a variety of shapes. <s> BIB004 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> We present a complete software architecture for reliable grasping of household objects. Our work combines aspects such as scene interpretation from 3D range data, grasp planning, motion planning, and grasp failure identification and recovery using tactile sensors. We build upon, and add several new contributions to the significant prior work in these areas. A salient feature of our work is the tight coupling between perception (both visual and tactile) and manipulation, aiming to address the uncertainty due to sensor and execution errors. This integration effort has revealed new challenges, some of which can be addressed through system and software engineering, and some of which present opportunities for future research. Our approach is aimed at typical indoor environments, and is validated by long running experiments where the PR2 robotic platform was able to consistently grasp a large variety of known and unknown objects. The set of tools and algorithms for object grasping presented here have been integrated into the open-source Robot Operating System (ROS). <s> BIB005 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> In this paper we present a method for building models for grasping from a single 3D snapshot of a scene composed of objects of daily use in human living environments. We employ fast shape estimation, probabilistic model fitting and verification methods capable of dealing with different kinds of symmetries, and combine these with a triangular mesh of the parts that have no other representation to model previously unseen objects of arbitrary shape. Our approach is enhanced by the information given by the geometric clues about different parts of objects which serve as prior information for the selection of the appropriate reconstruction method. While we designed our system for grasping based on single view 3D data, its generality allows us to also use the combination of multiple views. We present two application scenarios that require complete geometric models: grasp planning and locating objects in camera images. <s> BIB006 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> In this paper, we describe efficient methods for solving everyday mobile manipulation tasks that require object pick-up. In order to achieve real-time performance in complex environments, we focus our approach on fast yet robust solutions. For 3D perception of objects on planar surfaces, we develop scene segmentation methods that process Microsoft Kinect depth images in real-time at high frame rates. We efficiently plan feasible, collision-free grasps on the segmented objects directly from the perceived point clouds to achieve fast execution times. We evaluate our approaches quantitatively in lab experiments and also report on the successful integration of our methods in public demonstrations at RoboCup German Open 2011 and RoboCup 2011 in Istanbul, Turkey. Mobile manipulation tasks in domestic environments require a vast set of perception and action capabilities. The robot not only requires localization, mapping, path planning, and obsta- cle avoidance abilities to safely navigate through the environ- ment. It also needs to integrate object detection, recognition, and manipulation. A typical requirement for a service robot is not just to achieve the task, but to perform it in reasonable time. While much research has been invested into the general solution of complex perception and motion planning problems, only few work has been focused on methods that solve the tasks efficiently in order to allow for continuous task execution without interruptions. In this paper, we present fast methods to flexibly grasp objects from planar surfaces. To achieve fast performance, we combine real-time object perception with efficient grasp planning and motion control. For real-time perception, we combine rapid normal estimation using integral images with efficient segmentation techniques. We segment the scene into the support plane of interest and the objects thereon. Our perception algorithm processes depth images of a Microsoft Kinect in real-time at a frame rate of approx. 16 Hz. From the raw object point clouds our grasp planning method derives fea- sible, collision-free grasps within about 100 milliseconds. We consider grasps on objects from either the side or from above. The planned grasps are then executed using parametrized motion primitives. We integrate our approaches into a system that we publicly evaluate at RoboCup competitions. We further conduct experiments in our lab to demonstrate the robustness and efficiency of our approaches. This paper is organized as follows: after a brief system overview in Sec. III, we detail our approaches to real-time <s> BIB007 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> Grasp planning based on perceived sensor data of an object can be performed in different ways, depending on the chosen semantic interpretation of the sensed data. For example, if the object can be recognized and a complete 3D model is available, a different planning tool can be selected compared to the situation in which only the raw sensed data, such as a single point cloud, is available. Instead of choosing between these options, we present a framework that combines them, aiming to find consensus on how the object should be grasped by using the information from each object representation according to their confidence levels. We show that this method is robust to common errors in perception, such as incorrect object recognition, while also taking into account potential grasp execution errors due to imperfect robot calibration. We illustrate this method on the PR2 robot by grasping objects common in human environments. <s> BIB008 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> In this article we describe an object perception system for autonomous robots performing everyday manipulation tasks in kitchen environments. The perception system gains its strengths by exploiting that the robots are to perform the same kinds of tasks with the same objects over and over again. It does so by learning the object representations necessary for the recognition and reconstruction in the context of pick-and-place tasks. The system employs a library of specialized perception routines that solve different, well-defined perceptual sub-tasks and can be combined into composite perceptual activities including the construction of an object model database, multimodal object classification, and object model reconstruction for grasping. We evaluate the effectiveness of our methods, and give examples of application scenarios using our personal robotic assistants acting in a human living environment. <s> BIB009 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> We study visual servoing in a framework of detection and grasping of unknown objects. Classically, visual servoing has been used for applications where the object to be servoed on is known to the robot prior to the task execution. In addition, most of the methods concentrate on aligning the robot hand with the object without grasping it. In our work, visual servoing techniques are used as building blocks in a system capable of detecting and grasping unknown objects in natural scenes. We show how different visual servoing techniques facilitate a complete grasping cycle. <s> BIB010 </s> Data-Driven Grasp Synthesis - A Survey <s> C. From Global Shape to Grasp Hypothesis <s> This paper explores how Cloud Computing can facilitate grasping with shape uncertainty. We consider the most common robot gripper: a pair of thin parallel jaws, and a class of objects that can be modeled as extruded polygons. We model a conservative class of push-grasps that can enhance object alignment. The grasp planning algorithm takes as input an approximate object outline and Gaussian uncertainty around each vertex and center of mass. We define a grasp quality metric based on a lower bound on the probability of achieving force closure. We present a highly-parallelizable algorithm to compute this metric using Monte Carlo sampling. The algorithm uses Coulomb frictional grasp mechanics and a fast geometric test for conservative conditions for force closure. We run the algorithm on a set of sample shapes and compare the grasps with those from a planner that does not model shape uncertainty. We report computation times with single and multi-core computers and sensitivity analysis on algorithm parameters. We also describe physical grasp experiments using the Willow Garage PR2 robot. <s> BIB011
Other approaches use the global shape of an object to infer one good grasp hypothesis. Morales et al. BIB001 extracted the 2D silhouette of an unknown object from an image and computed two and three-fingered grasps taking into account the kinematics constraints of the hand. Richtsfeld and Vincze BIB002 use a segmented point cloud from a stereo camera. They search for a suitable grasp with a simple gripper based on the shift of the top plane of an object into its center of mass. A set of heuristics is used for selecting promising fingertip positions. Maldonado et al. BIB003 model the object as a 3D Gaussian. For choosing a grasp configuration, it optimizes a criterion in which the distance between palm and object is minimized while the distance between fingertips and the object is maximized. The simplified model of the hand and optimization variables are shown in Fig. 29a . Stückler et al. BIB007 generate grasp hypotheses based on eigenvectors of the object's footprints on the table. Footprints refer to the 3D object point cloud projected onto the supporting surface. A similar approach has been taken by BIB010 . Kehoe et al. BIB011 assume an overhead view of the object and approximate its shape with an extruded polygon. The goal is to synthesize a zero-slip push grasp with a parallel jaw gripper given uncertainty about the precise object shape and the position of its center of mass. For this purpose, perturbations of the initial shape and position of the centroid are sampled. For an example of this, see Fig. 29b . For each of these samples, the same grasp candidate is evaluated. Its quality depends on how often it resulted in force closure under the assumed model of object shape uncertainty. V. HYBRID APPROACHES There are a few data-driven grasp synthesis methods that cannot clearly be classified as using only one kind of prior knowledge. One of these approaches has been proposed in Brook et al. BIB008 with an extension in  . Different grasp planners provide grasp hypotheses which are integrated to reach a consensus on how to grasp a segmented point cloud. The authors show results using the planner presented in BIB004 for unknown objects in combination with grasp hypotheses generated through fitting known objects to point cloud clusters as described in BIB005 . Fig. 30 shows the grasp hypotheses for a segmented point cloud based on the input from these different planners. Another example for a hybrid approach is the work by Marton et al. BIB009 . A set of very simple shape primitives like boxes, cylinders and more general rotational objects are considered. They are reconstructed from segmented point clouds by analysis of their footprints. Parameters such as circle radius and the side lengths of rectangles are varied; curve parameters are estimated to reconstruct more complex rotationally symmetric objects. Given these reconstructions, a look-up is made in a database of already encountered objects for re-using successful grasp hypotheses. In case no similar object is found, new grasp hypotheses are generated using the technique presented in BIB006 . For object hypotheses that cannot be represented by the simple shape primitives mentioned above, a surface is reconstructed through triangulation. Grasp hypotheses are generated using the planner presented in BIB004 .