Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> Humanoid robots must be capable of interacting with the world using their hands. A variety of humanlike robot hands have been constructed, but it remains difficult to control these hands in a dexterous way. One challenge is grasp synthesis, where we wish to place the hand and control its shape to successfully grasp a given object. In this paper, we present a datadriven approach to grasp synthesis that treats grasping as a shape matching problem. We begin with a database of grasp examples. Given a model of a new object to be grasped (the query), shape features of the object are compared to shape features of hand poses in these examples in order to identify candidate grasps. For effective retrieval, we develop a novel shape matching algorithm that can accommodate the sparse shape information associated with hand pose and that considers relative placements of contact points and normals, which are important for grasp function. We illustrate our approach with examples using a model of the human hand <s> BIB001 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> We describe a process in which the segmentation of objects as well as the extraction of the object shape becomes realized through active exploration of a robot vision system. In the exploration process, two behavioral modules that link robot actions to the visual and haptic perception of objects interact. First, by making use of an object independent grasping mechanism, physical control over potential objects can be gained. Having evaluated the initial grasping mechanism as being successful, a second behavior extracts the object shape by making use of prediction based on the motion induced by the robot. This also leads to the concept of an "object" as a set of features that change predictably over different frames. The system is equipped with a certain degree of generic prior knowledge about the world in terms of a sophisticated visual feature extraction process in an early cognitive vision system, knowledge about its own embodiment as well as knowledge about geometric relationships such as rigid body motion. This prior knowledge allows the extraction of representations that are semantically richer compared to many other approaches. <s> BIB002 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> Robotic grasping in unstructured environments requires the ability to select grasps for unknown objects and execute them while dealing with uncertainty due to sensor noise or calibration errors. In this work, we propose a simple but robust approach to grasp selection for unknown objects, and a reactive adjustment approach to deal with uncertainty in object location and shape. The grasp selection method uses 3D sensor data directly to determine a ranked set of grasps for objects in a scene, using heuristics based on both the overall shape of the object and its local features. The reactive grasping approach uses tactile feedback from fingertip sensors to execute a compliant robust grasp. We present experimental results to validate our approach by grasping a wide range of unknown objects. Our results show that reactive grasping can correct for a fair amount of uncertainty in the measured position or shape of the objects, and that our grasp selection approach is successful in grasping objects with a variety of shapes. <s> BIB003 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> We present a novel vision-based grasp point detection algorithm that can reliably detect the corners of a piece of cloth, using only geometric cues that are robust to variation in texture. Furthermore, we demonstrate the effectiveness of our algorithm in the context of folding a towel using a general-purpose two-armed mobile robotic platform without the use of specialized end-effectors or tools. The robot begins by picking up a randomly dropped towel from a table, goes through a sequence of vision-based re-grasps and manipulations—partially in the air, partially on the table—and finally stacks the folded towel in a target location. The reliability and robustness of our algorithm enables for the first time a robot with general purpose manipulators to reliably and fully-autonomously fold previously unseen towels, demonstrating success on all 50 out of 50 single-towel trials as well as on a pile of 5 towels. <s> BIB004 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> We present a complete software architecture for reliable grasping of household objects. Our work combines aspects such as scene interpretation from 3D range data, grasp planning, motion planning, and grasp failure identification and recovery using tactile sensors. We build upon, and add several new contributions to the significant prior work in these areas. A salient feature of our work is the tight coupling between perception (both visual and tactile) and manipulation, aiming to address the uncertainty due to sensor and execution errors. This integration effort has revealed new challenges, some of which can be addressed through system and software engineering, and some of which present opportunities for future research. Our approach is aimed at typical indoor environments, and is validated by long running experiments where the PR2 robotic platform was able to consistently grasp a large variety of known and unknown objects. The set of tools and algorithms for object grasping presented here have been integrated into the open-source Robot Operating System (ROS). <s> BIB005 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> Robotic grasping in an open environment requires both object-specific as well as general grasping skills. When the objects are previously known it is possible to employ techniques that exploit object models, like geometrical grasping simulators. On the other hand, a competent system will also be able to deal with unmodeled objects using general solutions. In this paper we present an integrated system for autonomous rigid-object pick-up tasks in domestic environments, focusing on the gripping of unmodeled objects and exploiting sensor feedback from the robot hand to monitor the grasp. We describe the perception system based on time-of-flight range data, the grasp pose optimization algorithm and the grasp execution. The performance and robustness of the system is validated by experiments including pick-up tasks on many different common kitchen items. <s> BIB006 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> Grasping unknown objects based on real-world visual input is a challenging problem. In this paper, we present an Early Cognitive Vision system that builds a hierarchical representation based on edge and texture information, which is a sparse but powerful description of the scene. Based on this representation we generate edge-based and surface-based grasps. The results show that the method generates successful grasps, that the edge and surface information are complementary, and that the method can deal with more complex scenes. We furthermore present a benchmark for visual-based grasping. <s> BIB007 </s> Data-Driven Grasp Synthesis - A Survey <s> B. From Low-Level Features to Grasp Hypotheses <s> This paper explores how Cloud Computing can facilitate grasping with shape uncertainty. We consider the most common robot gripper: a pair of thin parallel jaws, and a class of objects that can be modeled as extruded polygons. We model a conservative class of push-grasps that can enhance object alignment. The grasp planning algorithm takes as input an approximate object outline and Gaussian uncertainty around each vertex and center of mass. We define a grasp quality metric based on a lower bound on the probability of achieving force closure. We present a highly-parallelizable algorithm to compute this metric using Monte Carlo sampling. The algorithm uses Coulomb frictional grasp mechanics and a fast geometric test for conservative conditions for force closure. We run the algorithm on a set of sample shapes and compare the grasps with those from a planner that does not model shape uncertainty. We report computation times with single and multi-core computers and sensitivity analysis on algorithm parameters. We also describe physical grasp experiments using the Willow Garage PR2 robot. <s> BIB008
A common approach is to map low-level 2D or 3D visual features to a predefined set of grasp postures and then rank them dependent on a set criteria. Kraft et al. BIB002 use a stereo camera to extract a representation of the scene. Instead of a raw point cloud, they process it further to obtain a sparser model consisting of local multi-modal contour descriptors. Four elementary grasping actions are associated to specific constellations of these features. With the help of heuristics, the large number of resulting grasp hypotheses is reduced. Popović et al. BIB007 present an extension of this system that uses local surfaces and their interrelations to propose and filter two and three-fingered grasp hypotheses. The feasibility of the approach is evaluated in a mixed real-world and simulated environment. The object representation and the evaluation in simulation is visualized in Fig. 27a . Figure 27 : Generating and ranking grasp hypotheses from local object features. 27a) Generation of grasp candidates from local surface features and evaluation in simulation BIB007 . 27b) Generated grasp hypotheses on point cloud clusters and execution results BIB003 . 27c) Top) Grasping a towel from the table. Bottom) Re-grasping a towel for unfolding BIB004 . Hsiao et al. BIB003 employ several heuristics for generating grasp hypotheses dependent on the shape of the segmented point cloud. These can be grasps from the top, from the side or applied to high points of the objects. The generated hypotheses are then ranked using a weighted list of features such as for example number of points within the gripper or distance between the fingertip and the center of the segment. Some examples for grasp hypotheses generated in this way are shown in Fig. 27b . This method is integrated into the grasping pipeline proposed in BIB005 for the segmented point cloud clusters that did not get recognized as a specific object. The main idea presented by Klingbeil et al. [117] is to search for a pattern in the scene that is similar to the 2D cross section of the robotic gripper interior. This is visualized in Fig. 28 . The idea is similar to the work by Li and Pollard BIB001 as shown in Fig. 18 . However, in this work the authors do not rely on the availability of a complete 3D object model. A depth image serves as the input to the method and is sampled to find a set of grasp hypotheses. These are ranked according to an objective function that takes pairs of these grasp hypotheses and their local structure into account. Maitin-Shepard et al. BIB004 propose a method for grasping and folding towels that can vary in size and are arranged in unpredictable configurations. Different from the approaches discussed above, the objects are deformable. The authors propose a border detection methods that relies on depth discontinuities and then fit corners to border points. These then serve as grasping points. Examples for grasping a towel are shown in Fig. 27c . Although this approach is applicable to a family of deformable objects, it does not detect grasping points by comparing to previously encountered grasping points. Instead it directly links local structure to a grasp. For this reason, we consider it as an approach towards grasping unknown objects. (a) (b) Figure 29 : Mapping global object shape to grasps. 29a) Simplified hand model and grasp parameters to be optimized BIB006 . 29b) Planar object shape uncertainty model Left) Vertices and center of mass with Gaussian position uncertainty (σ = 1). Right) 100 samples of perturbed object models BIB008 .