We present and evaluate an approach for human-in-the-loop specification of shape reconstruction with annotations for basic robot-object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to specify shape and delineate a simple task. The goal is to explore reducing the complexity of CAD-like interfaces so that novice users can quickly recover an object's shape and describe a manipulation task that is then carried out by a robot. The object modeling and interaction annotation capabilities are tested with a user study and compared against results obtained using existing approaches. The approach has been analyzed using a variety of shape comparison, grasping, and manipulation metrics, and tested with the PR2 robot platform, where it was shown to be successful.
Significant effort has been placed on obtaining a taxonomy of types of grasps and to judge their physical qualities @cite_84 @cite_93 @cite_111 . These studies draw inspiration from early analysis of human grasping @cite_61 or robot hands @cite_35 . One widely used approach for judging grasp quality is that of the metric developed by @cite_27 . It uses the grasp contact points to construct a convex hull over the set of wrenches that can be applied with the given grasp. This is called the Grasp Wrench Space (GWS). Two measurements that can be used are the volume of the computed convex hull and the radius of the maximum sphere contained in it, which we call the epsilon-distance, or @math -dist. These indicate the robustness of a grasp and the versatility of the wrenches applicable for the given grasp. One popular tool used in the literature to evaluate grasp quality and generate grasp candidates is GraspIt! @cite_66 . It uses the GWS volume and @math metrics to rank grasps that have been generated using a variety of grasp-planning techniques including the above mentioned .
@cite_61 @cite_35 @cite_84 @cite_27 @cite_93 @cite_111 @cite_66
2211217408 2144573888 1794703952 2282481780 1510186039
Grasp quality metrics which analyze the contact wrench space are commonly used to synthesize and analyze preplanned grasps. Preplanned grasping approaches rely on the robustness of stored solutions. Analyzing the robustness of such solutions for large databases of preplanned grasps is a limiting factor for the applicability of data driven approaches to grasping. In this work, we will focus on the stability of the widely used grasp wrench space epsilon quality metric over a large range of poses in simulation. We examine a large number of grasps from the Columbia Grasp Database for the Barrett hand. We find that in most cases the grasp with the most robust force closure with respect to pose error for a particular object is not the grasp with the highest epsilon quality. We demonstrate that grasps can be reranked by an estimate of the stability of their epsilon quality. We find that the grasps ranked best by this method are successful more often in physical experiments than grasps ranked best by the epsilon quality. This paper formalizes qualitative test of 3D frictional form-closure grasps of n robotic fingers as a problem of linear programming (LP). It is well-known that a sufficient and necessary condition for form-closure grasps is that the origin of the wrench space lies inside the convex hull of primitive contact wrenches. We demonstrate that the problem of querying whether the origin lies inside the convex hull is equivalent to a ray-shooting problem, which is dual to a LP problem based on the duality between convex hulls and convex polytopes. Furthermore, this paper addresses a problem of minimizing the L sub 1 norm of the grasp forces balancing an external wrench, which can be also transformed to a ray-shooting problem. We have implemented the algorithms and confirmed their real-time efficiency for qualitative test and grasp force optimization. Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98 @math , 82 @math , and 58 @math respectively, improving to 81 @math in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at this http URL . A robotic grasping simulator, called Graspit!, is presented as versatile tool for the grasping community. The focus of the grasp analysis has been on force-closure grasps, which are useful for pick-and-place type tasks. This work discusses the different types of world elements and the general robot definition, and presented the robot library. The paper also describes the user interface of Graspit! and present the collision detection and contact determination system. The grasp analysis and visualization method were also presented that allow a user to evaluate a grasp and compute optimal grasping forces. A brief overview of the dynamic simulation system was provided. Current analytical models of grasping and manipulation with robotic hands contain simplifications and assumptions that limit their application to manufacturing environments. To evaluate these models, a study was undertaken of the grasps used by machinists in a small batch manufacturing operation. Based on the study, a taxonomy of grasps was constructed. An expert system was also developed to clarify the issues involved in human grasp choice. Comparisons of the grasp taxonomy, the expert system, and grasp-quality measures derived from the analytic models reveal that the analytic measures are useful for describing grasps in manufacturing tasks despite the limitations in the models. In addition, the grasp taxonomy provides insights for the design of versatile robotic hands for manufacturing. > The correct grasp of objects is a key aspect for the right fulfillment of a given task. In robotics, the development of grippers more and more complex and versatile, such as mechanical hands, augments the necessity for algorithms to automatically determine grasps, and in a parallel way, arises the need to quantify the quality of the generated grasps in order to optimize them. This work reviews the quality measures proposed in the grasp literature to quantify the grasp quality. We classify the quality measures into two groups according to the main aspect evaluated by the measure: the location of the contact points on the object or the hand configuration. We also review the approaches that combine different quality measures from the two previous groups to obtain a global quality measure. We present a novel and simple experimental method called physical human interactive guidance to study human-planned grasping. Instead of studying how the human uses his her own biological hand or how a human teleoperates a robot hand in a grasping task, the method involves a human interacting physically with a robot arm and hand, carefully moving and guiding the robot into the grasping pose, while the robot's configuration is recorded. Analysis of the grasps from this simple method has produced two interesting results. First, the grasps produced by this method perform better than grasps generated through a state-of-the-art automated grasp planner. Second, this method when combined with a detailed statistical analysis using a variety of grasp measures (physics-based heuristics considered critical for a good grasp) offered insights into how the human grasping method is similar or different from automated grasping synthesis techniques. Specifically, data from the physical human interactive guidance method showed that the human-planned grasping method provides grasps that are similar to grasps from a state-of-the-art automated grasp planner, but differed in one key aspect. The robot wrists were aligned with the object's principal axes in the human-planned grasps (termed low skewness in this paper), while the automated grasps used arbitrary wrist orientation. Preliminary tests show that grasps with low skewness were significantly more robust than grasps with high skewness (77-93 ). We conclude with a detailed discussion of how the physical human interactive guidance method relates to existing methods to extract the human principles for physical interaction. Planning realizable and stable grasps on 3D objects is crucial for many robotics applications, but grasp planners often ignore the relative sizes of the robotic hand and the object being grasped or do not account for physical joint and positioning limitations. We present a grasp planner that can consider the full range of parameters of a real hand and an arbitrary object, including physical and material properties as well as environmental obstacles and forces, and produce an output grasp that can be immediately executed. We do this by decomposing a 3D model into a superquadric 'decomposition tree' which we use to prune the intractably large space of possible grasps into a subspace that is likely to contain many good grasps. This subspace can be sampled and evaluated in GraspIt!, our 3D grasping simulator, to find a set of highly stable grasps, all of which are physically realizable. We show grasp results on various models using a Barrett hand. We present a new approach for modelling grasping using an integrated space of grasps and shapes. In particular, we introduce an infinite dimensional space, the Grasp Moduli Space, which represents shapes and grasps in a continuous manner. We define a metric on this space allowing us to formalize ‘nearby’ grasp shape configurations and we discuss continuous deformations of such configurations. We work in particular with surfaces with cylindrical coordinates and analyse the stability of a popular L1 grasp quality measure Ql under continuous deformations of shapes and grasps. We experimentally determine bounds on the maximal change of Ql in a small neighbourhood around stable grasps with grasp quality above a threshold. In the case of surfaces of revolution, we determine stable grasps which correspond to grasps used by humans and develop an efficient algorithm for generating those grasps in the case of three contact points. We show that sufficiently stable grasps stay stable under small deformations. For larger deformations, we develop a gradient-based method that can transfer stable grasps between different surfaces. Additionally, we show in experiments that our gradient method can be used to find stable grasps on arbitrary surfaces with cylindrical coordinates by deforming such surfaces towards a corresponding ‘canonical’ surface of revolution. Humanoid robots must be capable of interacting with the world using their hands. A variety of humanlike robot hands have been constructed, but it remains difficult to control these hands in a dexterous way. One challenge is grasp synthesis, where we wish to place the hand and control its shape to successfully grasp a given object. In this paper, we present a datadriven approach to grasp synthesis that treats grasping as a shape matching problem. We begin with a database of grasp examples. Given a model of a new object to be grasped (the query), shape features of the object are compared to shape features of hand poses in these examples in order to identify candidate grasps. For effective retrieval, we develop a novel shape matching algorithm that can accommodate the sparse shape information associated with hand pose and that considers relative placements of contact points and normals, which are important for grasp function. We illustrate our approach with examples using a model of the human hand Automatic grasp planning for robotic hands is a difficult problem because of the huge number of possible hand configurations. However, humans simplify the problem by choosing an appropriate prehensile posture appropriate for the object and task to be performed. By modeling an object as a set of shape primitives, such as spheres, cylinders, cones and boxes, we can use a set of rules to generate a set of grasp starting positions and pregrasp shapes that can then be tested on the object model. Each grasp is tested and evaluated within our grasping simulator "GraspIt!", and the best grasps are presented to the user. The simulator can also plan grasps in a complex environment involving obstacles and the reachability constraints of a robot arm. Describes a synergistic integration of a grasping simulator and a real-time visual tracking system, that work in concert to (1) find an object's pose, (2) plan grasps and movement trajectories, and (3) visually monitor task execution. Starting with a CAD model of an object to be grasped, the system can find the object's pose through vision which then synchronizes the state of the robot workcell with an online, model-based grasp planning and visualization system we have developed called GraspIt. GraspIt can then plan a stable grasp for the object, and direct the robotic hand system to perform the grasp. It can also generate trajectories for the movement of the grasped object, which are used by the visual control system to monitor the task and compare the actual grasp and trajectory with the planned ones. We present experimental results using typical grasping tasks. Robotic grasping has attracted considerable interest, but it still remains a challenging task. The data-driven approach is a promising solution to the robotic grasping problem; this approach leverages a grasp dataset and generalizes grasps for various objects. However, these methods often depend on the quality of the given datasets, which are not trivial to obtain with sufficient quality. Although reinforcement learning approaches have been recently used to achieve autonomous collection of grasp datasets, the existing algorithms are often limited to specific grasp types. In this paper, we present a framework for hierarchical reinforcement learning of grasping policies. In our framework, the lower-level hierarchy learns multiple grasp types, and the upper-level hierarchy learns a policy to select from the learned grasp types according to a point cloud of a new object. Through experiments, we validate that our approach learns grasping by constructing the grasp dataset autonomously. The experimental results show that our approach learns multiple grasping policies and generalizes the learned grasps by using local point cloud information. Typical tasks of future service robots involve grasping and manipulating a large variety of objects differing in size and shape. Generating stable grasps on 3D objects is considered to be a hard problem, since many parameters such as hand kinematics, object geometry, material properties and forces have to be taken into account. This results in a high-dimensional space of possible grasps that cannot be searched exhaustively. We believe that the key to find stable grasps in an efficient manner is to use a special representation of the object geometry that can be easily analyzed. In this paper, we present a novel grasp planning method that evaluates local symmetry properties of objects to generate only candidate grasps that are likely to be of good quality. We achieve this by computing the medial axis which represents a 3D object as a union of balls. We analyze the symmetry information contained in the medial axis and use a set of heuristics to generate geometrically and kinematically reasonable candidate grasps. These candidate grasps are tested for force-closure. We present the algorithm and show experimental results on various object models using an anthropomorphic hand of a humanoid robot in simulation. We present an example-based planning framework to generate semantic grasps, stable grasps that are functionally suitable for specific object manipulation tasks. We propose to use partial object geometry, tactile contacts, and hand kinematic data as proxies to encode task-related constraints, which we call semantic constraints. We introduce a semantic affordance map, which relates local geometry to a set of predefined semantic grasps that are appropriate to different tasks. Using this map, the pose of a robot hand with respect to the object can be estimated so that the hand is adjusted to achieve the ideal approach direction required by a particular task. A grasp planner is then used to search along this approach direction and generate a set of final grasps which have appropriate stability, tactile contacts, and hand kinematics. We show experiments planning semantic grasps on everyday objects and applying these grasps with a physical robot. This paper presents a novel robot grasping planning approach that extracts grasp strategies (grasp type, and thumb placement and direction) from human demonstration and integrates them into the grasp planning procedure to generate a feasible grasp concerning the target object geometry and manipulation task. Our study results show that the grasp strategies of grasp type and thumb placement not only represent important human grasp intentions, but also provide meaningful constraints on hand posture and wrist position, which highly reduce both the feasible workspace of a robotic hand and the search space of the grasp planning. This approach has been thoroughly evaluated both in simulation and with a real robotic system for multiple daily living representative objects. We have also demonstrated the robustness of our approach in the experiment with different levels of perception uncertainties. This paper presents an egocentric vision (ego-vision) system for hand grasp analysis in unstructured environments. Our goal is to automatically recognize hand grasp types and to discover the visual structures of hand grasps using a wearable camera. In the proposed system, free hand–object interactions are recorded from a first-person viewing perspective. State-of-the-art computer vision techniques are used to detect hands and extract hand-based features. A new feature representation that incorporates hand tracking information is also proposed. Then, grasp classifiers are trained to discriminate among different grasp types from a predefined grasp taxonomy. Based on the trained grasp classifiers, visual structures of hand grasps are learned using an iterative grasp clustering method. In experiments, grasp recognition performance in both laboratory and real-world scenarios is evaluated. The best classification accuracy our system achieves is @math and @math , respectively. System generality to different tasks and users is also verified by the experiments. Analysis in a real-world scenario shows that it is possible to automatically learn intuitive visual grasp structures that are consistent with expert-designed grasp taxonomies. How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. By dexterous grasping we simply mean grasping by any hand with more than two fingers, such that the robot has some choice about where to place each finger. Such grasps typically fail in one of two ways, either unmodeled objects in the scene cause collisions or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anticipated actions. Our approach has three stages. First, we take a single view and generate candidate grasps from the resulting partial object reconstruction. Second, we drive the active vision approach to maximise surface reconstruction quality around the planned contact points. During this phase, the anticipated grasp is continually refined. Third, we direct gaze to improve the safety of the planned reach to grasp trajectory. We show, on a dexterous manipulator with a camera on the wrist, that our approach (80.4 success rate) outperforms a randomised algorithm (64.3 success rate). The ability to grasp large objects with both hands enables bimanual robot systems to fully employ their capabilities in human-centered environments. Hence, algorithms are needed to precompute bimanual grasping configurations that can be used online to efficiently create whole body grasps. In this work we present a bimanual grasp planner that can be used to build a set of grasps together with manipulability information for a given object. For efficient grasp planning precomputed reachability information and a beneficial object representation, based on medial axis descriptions, are used. Since bimanual grasps may suffer from low manipulability, caused by a closed kinematic chain, we show how the manipulability of a bimanual grasp can be used as a quality measure. Therefore, manipulability clusters are introduced as an efficient way to approximatively describe the manipulability of a given bimanual grasp. The proposed approach is evaluated with a reference implementation, based on Simox [1], for the humanoid robot ARMAR-III [2]. Since the presented algorithms are robot-independent, there are no limitations for using this planner on other robot systems. In this paper, we analyze and compare existing human grasp taxonomies and synthesize them into a single new taxonomy (dubbed “The GRASP Taxonomy” after the GRASP project funded by the European Commission). We consider only static and stable grasps performed by one hand. The goal is to extract the largest set of different grasps that were referenced in the literature and arrange them in a systematic way. The taxonomy provides a common terminology to define human hand configurations and is important in many domains such as human–computer interaction and tangible user interfaces where an understanding of the human is basis for a proper interface. Overall, 33 different grasp types are found and arranged into the GRASP taxonomy. Within the taxonomy, grasps are arranged according to 1) opposition type, 2) the virtual finger assignments, 3) type in terms of power, precision, or intermediate grasp, and 4) the position of the thumb. The resulting taxonomy incorporates all grasps found in the reviewed taxonomies that complied with the grasp definition. We also show that due to the nature of the classification, the 33 grasp types might be reduced to a set of 17 more general grasps if only the hand configuration is considered without the object shape size.