Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review <s> A Brief History of Robotic Grasping <s> This paper describes a general purpose, representation independent method for the accurate and computationally efficient registration of 3-D shapes including free-form curves and surfaces. The method handles the full six-degrees of freedom and is based on the iterative closest point (ICP) algorithm, which requires only a procedure to find the closest point on a geometric entity to a given point. The ICP algorithm always converges monotonically to the nearest local minimum of a mean-square distance metric, and experience shows that the rate of convergence is rapid during the first few iterations. Therefore, given an adequate set of initial rotations and translations for a particular class of objects with a certain level of 'shape complexity', one can globally minimize the mean-square distance metric over all six degrees of freedom by testing each initial registration. For examples, a given 'model' shape and a sensed 'data' shape that represents a major portion of the model shape can be registered in minutes by testing one initial translation and a relatively small set of rotations to allow for the given level of model complexity. One important application of this method is to register sensed data from unfixtured rigid objects with an ideal geometric model prior to shape inspection. The described method is also useful for deciding fundamental issues such as the congruence (shape equivalence) of different geometric representations as well as for estimating the motion between point sets where the correspondences are not known. Experimental results show the capabilities of the registration algorithm on point sets, curves, and surfaces. <s> BIB001 </s> Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review <s> A Brief History of Robotic Grasping <s> In this paper, we survey the field of robotic grasping and the work that has been done in this area over the last two decades, with a slight bias toward the development of the theoretical framework and analytical results in this area. <s> BIB002 </s> Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review <s> A Brief History of Robotic Grasping <s> We review the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps. We divide the approaches into three groups based on whether they synthesize grasps for known, familiar, or unknown objects. This structure allows us to identify common object representations and perceptual processes that facilitate the employed data-driven grasp synthesis technique. In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation. In the case of familiar objects, the techniques use some form of a similarity matching to a set of previously encountered objects. Finally, for the approaches dealing with unknown objects, the core part is the extraction of specific features that are indicative of good grasps. Our survey provides an overview of the different methodologies and discusses open problems in the area of robot grasping. We also draw a parallel to the classical approaches that rely on analytic formulations. <s> BIB003
Robotic grasping has long been researched in literature. In early years, the robotic arm was not endowed with the perception ability, and robotic grasping was conducted by manually controlling the mechanical arms and hands. Later on, with the assistance of tactile sensors such as data gloves, a robot can grasp by coping the behaviour of human hands. Both of these methods still rely on human labor and are with few intelligence. With the development of optical sensors, robots have been able to grasp automatically through vision-based perception system. Traditionally, a vision-based robotic grasping system is composed of a series of components, including target object localization, object pose estimation, grasp detection and grasp planning  . Object localization finds the location of the target object in the input data. Object pose estimation estimates the rotation as well as the translation of the target object with respect to a reference. Grasp detection computes the grasp configuration with regard to the target object. This configuration can be represented as a seven dimensional variable, including a grasping point, grasping orientation, and gripper opening width  . Grasp planning refers to the path planning process which is required to safely grab the object and maintain the closed gripper contacts to hold and lift the object from its resting surface BIB002 . Early methods assume that the object to grasp is placed in a clean environment with simple background and thus simplifies the object localization task, while in relatively complex environments their capabilities are quite limited. Some object detection methods utilized machine learning methods to train classifiers based on hand-crafted 2D descriptors. However, these classifiers show limited performance since the limitations of hand-crafted descriptors. In recent years, deep learning has begun to dominant the image-related tasks such as object detection and segmentation. Besides, the training data ranges from RGB images to depth images, and deep learning networks with 2D or 3D inputs are proposed, which highly improves the performance of object localization and extremely prompts the development of robotic grasping. With the location of the target object, the grasp detection can be conducted. In early years, analytical methods were utilized which directly analyze the geometric structure of the input data, and find the points suitable to grasp according to force closure or shape closure. Sahbani et al.  presented an overview of 3D object grasping algorithms, where analytical approaches are introduced in detail. However, analytical methods have many problems such as time consuming, difficult to compute the force. Later, with the emergence of large numbers of 3D models, datadriven methods could be analyzed to transfer grasps in the 3D model database to the target object. Bohg et al. BIB003 reviewed data-driven grasp methods and divided the approaches into three groups based on whether the grasp configuration is computed for known, familiar or unknown objects. Generally, the 6D pose of the target object is essential to accomplish this task. Both RGB image-based and depth image-based methods could achieve accurate pose estimations. However, these methods such as partial registration methods BIB001 are susceptible to sensor noise or incomplete data. The poses could also be estimated directly or indirectly from the input data through deep learning methods in order to get resistance to the sensor noise or incomplete data. There also exist deep learning-based methods where the 6D poses are not needed to conduct grasp detection. The grasp configuration could be regressed directly or indirectly through deep convolutional networks. Caldera et al.  reviewed the deep learning-based methods in robotic grasping detection. They discussed how each element of the deep learning approach improves the overall performance of robotic grasping detection. Besides, the supervised learning methods, reinforcement learning have also been utilized to directly accomplish specific tasks like toy assembly and pour water, which are closely related to grasping.