Abstract Interaction in virtual reality (VR) environments (e.g. grasping and manipulating virtual objects) is essential to ensure a pleasant and immersive experience. In this work, we propose a visually realistic, flexible and robust grasping system that enables real-time interactions in virtual environments. Resulting grasps are visually realistic because hand is automatically fitted to the object shape from a position and orientation determined by the user using the VR handheld controllers (e.g. Oculus Touch motion controllers). Our approach is flexible because it can be adapted to different hand meshes (e.g. human or robotic hands) and it is also easily customizable. Moreover, it enables interaction with different objects regardless their geometries. In order to validate our proposal, an exhaustive qualitative and quantitative performance analysis has been carried out. On one hand, qualitative evaluation was used in the assessment of abstract aspects, such as motor control, finger movement realism, and interaction realism. On the other hand, for the quantitative evaluation a novel metric has been proposed to visually analyze the performed grips. Performance analysis results indicate that previous experience with our grasping system is not a prerequisite for an enjoyable, natural and intuitive VR interaction experience.
Grasping action is the most basic component of any interaction and it is composed of three major components @cite_21 . The first one is related to the process of approaching the arm and hand to the target object, considering the overall body movement. The second component focuses on the hand and body pre-shaping before the grasping action. Finally, the last component fits the hand to the geometry of the object by closing each of the fingers until contact is established.
@cite_21
1999329153
Animated human characters in everyday scenarios must interact with the environment using their hands. Captured human motion can provide a database of realistic examples. However, examples involving contact are difficult to edit and retarget; realism can suffer when a grasp does not appear secure or when an apparent impact does not disturb the hand or the object. Physically based simulations can preserve plausibility through simulating interaction forces. However, such physical models must be driven by a controller, and creating effective controllers for new motion tasks remains a challenge. In this paper, we present a controller for physically based grasping that draws from motion capture data. Our controller explicitly includes passive and active components to uphold compliant yet controllable motion, and it adds compensation for movement of the arm and for gravity to make the behavior of passive and active components less dependent on the dynamics of arm motion. Given a set of motion capture grasp examples, our system solves for all but a small set of parameters for this controller automatically. We demonstrate results for tasks including grasping and two-hand interaction and show that a controller derived from a single motion capture example can be used to form grasps of different object geometries. Subjects were asked to shape the right hand as if to grasp and use a large number of familiar objects. The chosen objects typically are held with a variety of grips, including “precision” and “power” grips. Static hand posture was measured by recording the angular position of 15 joint angles of the fingers and of the thumb. Although subjects adopted distinct hand shapes for the various objects, the joint angles of the digits did not vary independently. Principal components analysis showed that the first two components could account for .80 of the variance, implying a substantial reduction from the 15 degrees of freedom that were recorded. However, even though they were small, higher-order (more than three) principal components did not represent random variability but instead provided additional information about the object. These results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments. Because the postural synergies did not coincide with grip taxonomies, the results suggest that hand posture may be regulated independently from the control of the contact forces that are used to grasp an object. This paper explores the problem of autonomous, in-hand regrasping-the problem of moving from an initial grasp on an object to a desired grasp using the dexterity of a robot's fingers. We propose a planner for this problem which alternates between finger gaiting, and in-grasp manipulation. Finger gaiting enables the robot to move a single finger to a new contact location on the object, while the remaining fingers stably hold the object. In-grasp manipulation moves the object to a new pose relative to the robot's palm, while maintaining the contact locations between the hand and object. Given the object's geometry (as a mesh), the hand's kinematic structure, and the initial and desired grasps, we plan a sequence of finger gaits and object reposing actions to reach the desired grasp without dropping the object. We propose an optimization based approach and report in-hand regrasping plans for 5 objects over 5 in-hand regrasp goals each. The plans generated by our planner are collision free and guarantee kinematic feasibility. This paper proposes a planning framework to deal with the problem of computing the motion of a robot with dual arm hand, during an object pick-and-place task. We consider the situation where the start and goal configurations of the object constrain the robot to grasp the object with one hand, to give it to the other hand, before placing it in its final configuration. To realize such a task, the proposed framework treats the grasp computation, for one or two multi-fingered hands, of an arbitrarily-shaped object, the exchange configuration and finally the motion of the robot arms and body. In order to improve the planner performance, a context-independent grasp list is computed offline for each hand and for the given object as well as computed offline roadmap that will be adapted according to the environment composition. Simulation results show the planner performance on a complex scenario. Traditionally, grasp and arm motion planning are considered as separate tasks. This paper presents an integrated approach that only requires the initial configuration of the robotic arm and the pose of the target object to simultaneously plan a good hand pose and arm trajectory to grasp the object. The planner exploits the concept of independent contact regions to look for the best possible grasp. The goal poses for the end effector are obtained using two different methods: one that biases a sampling approach towards favorable regions using principal component analysis, and another one that considers the capabilities of the robotic arm to decide the most promising hand poses. The proposed method is evaluated using different scenarios for the humanoid robot Spacejustin. An important challenge in robotics is to achieve robust performance in object grasping and manipulation, dealing with noise and uncertainty. This paper presents an approach for addressing the performance of dexterous grasping under shape uncertainty. In our approach, the uncertainty in object shape is parametrized and incorporated as a constraint into grasp planning. The proposed approach is used to plan feasible hand configurations for realizing planned contacts using different robotic hands. A compliant finger closing scheme is devised by exploiting both the object shape uncertainty and tactile sensing at fingertips. Experimental evaluation demonstrates that our method improves the performance of dexterous grasping under shape uncertainty. We considered object shape uncertainty in grasp planning and control.We proposed a probabilistic model to solve hand inverse kinematics.Our grasp planning approach is hand interchangeable.We presented a compliant uncertainty-aware controller for finger closing during grasp execution. We address the problem of grasping everyday objects that are small relative to an anthropomorphic hand, such as pens, screwdrivers, cellphones, and hammers from their natural poses on a support surface, e.g., a table top. In such conditions, state of the art grasp generation techniques fail to provide robust, achievable solutions due to either ignoring or trying to avoid contact with the support surface. In contrast, we show that contact with support surfaces is critical for grasping small objects. This also conforms with our anecdotal observations of human grasping behaviors. We develop a simple closed-loop hybrid controller that mimics this interactive, contact-rich strategy by a position-force, pre-grasp and landing strategy for finger placement. The approach uses a compliant control of the hand during the grasp and release of objects in order to preserve safety. We conducted extensive grasping experiments on a variety of small objects with similar shape and size. The results demonstrate that our approach is robust to localization uncertainties and applies to many everyday objects. In this paper we focus on the concept of low-dimensional posture subspaces for artificial hands. We begin by discussing the applicability of a hand configuration subspace to the problem of automated grasp synthesis; our results show that low-dimensional optimization can be instrumental in deriving effective pre-grasp shapes for a number of complex robotic hands. We then show that the computational advantages of using a reduced dimensionality framework enable it to serve as an interface between the human and automated components of an interactive grasping system. We present an on-line grasp planner that allows a human operator to perform dexterous grasping tasks using an artificial hand. In order to achieve the computational rates required for effective user interaction, grasp planning is performed in a hand posture subspace of highly reduced dimensionality. The system also uses real-time input provided by the operator, further simplifying the search for stable grasps to the point where solutions can be found at interactive rates. We demonstrate our approach on a number of different hand models and target objects, in both real and virtual environments. A grasped object is defined to be in equilibrium if the sum of all forces and moments acting on a body equals zero. An equilibrium grasp may be stable or unstable. Force closed grasps are a well-known subset of equilibrium grasps, and they are known to be stable. However, not all stable grasps are force closed, including many common and easily obtainable grasps. In this paper, we classify the categories of equilibrium grasps and establish a general framework for the determination of the stability of a grasp. In order to analyze the stability of grasps with multiple contacts, we first model the compliance at each contact. We develop expressions for the changes in contact forces as a function of the rigid body relative motion between the fingers and the grasped object. The stability of a grasp is shown to depend on the local curvature of the contacting bodies, as well as the magnitude and arrangement of the contact forces. We then derive results providing simple criteria to determine the stability of a grasped object, including the special but important limiting case of rigid bodies where the contact compliance is zero. This paper presents a method to transfer functional grasps among objects of the same category through contact warping and local replanning. The method transfers implicit knowledge that enables an action on a class of objects for which no explicit grasp or task information has been given in advance. Contact points on the source object are warped based on global and local shape similarities to the target object. These warped contacts are then used to define a hand posture that reaches close to them, while at the same time provides the desired functionality on the object. The approach is tested on different sets of objects with a success rate of 87.5 , and large benefits are shown when compared to a naive technique that only transfers a suitable hand pose to the novel object. This overview presents computational algorithms for generating 3D object grasps with autonomous multi-fingered robotic hands. Robotic grasping has been an active research subject for decades, and a great deal of effort has been spent on grasp synthesis algorithms. Existing papers focus on reviewing the mechanics of grasping and the finger-object contact interactions Bicchi and Kumar (2000) [12] or robot hand design and their control Al- (1993) [70]. Robot grasp synthesis algorithms have been reviewed in Shimoga (1996) [71], but since then an important progress has been made toward applying learning techniques to the grasping problem. This overview focuses on analytical as well as empirical grasp synthesis approaches. We present a method for transferring grasps between objects of the same functional category. This transfer is intended to preserve the functionality of a grasp constructed for one of the objects, thus enabling the analogous action to be performed on a novel object for which no grasp has been specified. Manipulation knowledge is hence generalized from a single example to a class of objects with a significant amount of shape variability. The transfer is achieved through warping the surface geometry of the source object onto the target object, and along with it the contact points of a grasp. The warped contacts are locally replanned, if necessary, to ensure grasp stability, and a suitable grasp pose is computed. We present extensive results of experiments with a database of four-finger grasps, designed to systematically cover variations on grasping the mugs of the Princeton Shape Benchmark. Many manipulation planning problems involve several related sub-problems, such as the selection of grasping points on an object, choice of hand posture, and determination of the arm's configuration and evolving trajectory. Traditionally, these planning sub-problems have been handled separately, potentially leading to sub-optimal, or even infeasible, combinations of the individually determined solutions. This paper formulates the combined problem of grasp contact selection, grasp force optimization, and manipulator arm hand trajectory planning as a problem in optimal control. That is, the locally optimal trajectory for the manipulator, hand mechanism, and contact locations are determined during the pre-grasping, grasping, and subsequent object transport phase. Additionally, a barrier function approach allows for non-feasible grasps to be optimized, enlarging the region of convergence for the algorithm. A simulation of a simple planar object manipulation task is used to illustrate and validate the approach. How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. By dexterous grasping we simply mean grasping by any hand with more than two fingers, such that the robot has some choice about where to place each finger. Such grasps typically fail in one of two ways, either unmodeled objects in the scene cause collisions or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anticipated actions. Our approach has three stages. First, we take a single view and generate candidate grasps from the resulting partial object reconstruction. Second, we drive the active vision approach to maximise surface reconstruction quality around the planned contact points. During this phase, the anticipated grasp is continually refined. Third, we direct gaze to improve the safety of the planned reach to grasp trajectory. We show, on a dexterous manipulator with a camera on the wrist, that our approach (80.4 success rate) outperforms a randomised algorithm (64.3 success rate). We design an example based planning framework to generate semantic grasps, stable grasps that are functionally suitable for specific object manipulation tasks. We propose to use partial object geometry, tactile contacts, and hand kinematic data as proxies to encode semantic constraints, which are task-related constraints. We introduce a semantic affordance map, which relates local geometry to a set of predefined semantic grasps that are appropriate to different tasks. Using this map, the pose of a robotic hand can be estimated so that the hand is adjusted to achieve the ideal approach direction required by a particular task. A grasp planner is then used to generate a set of final grasps which have appropriate stability, tactile contacts, and hand kinematics along this approach direction. We show experiments planning semantic grasps on everyday objects and executing these grasps with a physical robot. We present an example-based planning framework to generate semantic grasps, stable grasps that are functionally suitable for specific object manipulation tasks. We propose to use partial object geometry, tactile contacts, and hand kinematic data as proxies to encode task-related constraints, which we call semantic constraints. We introduce a semantic affordance map, which relates local geometry to a set of predefined semantic grasps that are appropriate to different tasks. Using this map, the pose of a robot hand with respect to the object can be estimated so that the hand is adjusted to achieve the ideal approach direction required by a particular task. A grasp planner is then used to search along this approach direction and generate a set of final grasps which have appropriate stability, tactile contacts, and hand kinematics. We show experiments planning semantic grasps on everyday objects and applying these grasps with a physical robot. Humanoid robots must be capable of interacting with the world using their hands. A variety of humanlike robot hands have been constructed, but it remains difficult to control these hands in a dexterous way. One challenge is grasp synthesis, where we wish to place the hand and control its shape to successfully grasp a given object. In this paper, we present a datadriven approach to grasp synthesis that treats grasping as a shape matching problem. We begin with a database of grasp examples. Given a model of a new object to be grasped (the query), shape features of the object are compared to shape features of hand poses in these examples in order to identify candidate grasps. For effective retrieval, we develop a novel shape matching algorithm that can accommodate the sparse shape information associated with hand pose and that considers relative placements of contact points and normals, which are important for grasp function. We illustrate our approach with examples using a model of the human hand The ability to grasp unknown objects is an important skill for personal robots, which has been addressed by many present and past research projects, but still remains an open problem. A crucial aspect of grasping is choosing an appropriate grasp configuration, i.e. the 6d pose of the hand relative to the object and its finger configuration. Finding feasible grasp configurations for novel objects, however, is challenging because of the huge variety in shape and size of these objects. Moreover, possible configurations also depend on the specific kinematics of the robotic arm and hand in use. In this paper, we introduce a new grasp selection algorithm able to find object grasp poses based on previously demonstrated grasps. Assuming that objects with similar shapes can be grasped in a similar way, we associate to each demonstrated grasp a grasp template. The template is a local shape descriptor for a possible grasp pose and is constructed using 3d information from depth sensors. For each new object to grasp, the algorithm then finds the best grasp candidate in the library of templates. The grasp selection is also able to improve over time using the information of previous grasp attempts to adapt the ranking of the templates. We tested the algorithm on two different platforms, the Willow Garage PR2 and the Barrett WAM arm which have very different hands. Our results show that the algorithm is able to find good grasp configurations for a large set of objects from a relatively small set of demonstrations, and does indeed improve its performance over time. Our goal is to automate the understanding of natural hand-object manipulation by developing computer visionbased techniques. Our hypothesis is that it is necessary to model the grasp types of hands and the attributes of manipulated objects in order to accurately recognize manipulation actions. Specifically, we focus on recognizing hand grasp types, object attributes and actions from a single image within an unified model. First, we explore the contextual relationship between grasp types and object attributes, and show how that context can be used to boost the recognition of both grasp types and object attributes. Second, we propose to model actions with grasp types and object attributes based on the hypothesis that grasp types and object attributes contain complementary information for characterizing different actions. Our proposed action model outperforms traditional appearance-based models which are not designed to take into account semantic constraints such as grasp types or object attributes. Experiment results on public egocentric activities datasets strongly support our hypothesis. This paper introduces the iRobot-Harvard-Yale (iHY) Hand, an underactuated hand driven by five actuators that is capable of performing a wide range of grasping and in-hand repositioning tasks. This hand was designed to address the need for a durable, inexpensive, moderately dexterous hand suitable for use on mobile robots. The primary focus of this paper will be on the novel simplified design of the iHY Hand, which was developed by choosing a set of target tasks around which the hand was optimized. Particular emphasis is placed on the development of underactuated fingers that are capable of both firm power grasps and low-stiffness fingertip grasps using only the compliant mechanics of the fingers. Experimental results demonstrate successful grasping of a wide range of target objects, the stability of fingertip grasping, and the ability to adjust the force exerted on grasped objects using high-impedance actuators and underactuated fingers.