We present and evaluate an approach for human-in-the-loop specification of shape reconstruction with annotations for basic robot-object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to specify shape and delineate a simple task. The goal is to explore reducing the complexity of CAD-like interfaces so that novice users can quickly recover an object's shape and describe a manipulation task that is then carried out by a robot. The object modeling and interaction annotation capabilities are tested with a user study and compared against results obtained using existing approaches. The approach has been analyzed using a variety of shape comparison, grasping, and manipulation metrics, and tested with the PR2 robot platform, where it was shown to be successful.
One popular tool for motion planning is Moveit! @cite_113 . This tool can be used within the larger Robot Operating System (ROS) to perform automatic motion planning with collision avoidance. This may also be used in conjunction with a manipulation interface that humans can use to indicate motions. In the work by , @cite_16 , this interface was used to analyze manipulation strategies for indicating grasps and grasp approach methodologies. They compared continuous teleoperation against three other strategies with varying degrees of autonomy. They showed than a combination of manual annotation and automatic processing resulted in the greatest success rate. This is supported by the work done by Hertkorn @cite_129 , where the importance of shared workload is thoroughly analyzed in the context of grasping and manipulation.
@cite_129 @cite_16 @cite_113
2538518392 2752209622
We add to a manipulator's capabilities a new primitive motion which we term a push-grasp. While significant progress has been made in robotic grasping of objects and geometric path planning for manipulation, such work treats the world and the object being grasped as immovable, often declaring failure when simple motions of the object could produce success. We analyze the mechanics of push-grasping and present a quasi-static tool that can be used both for analysis and simulation. We utilize this analysis to derive a fast, feasible motion planning algorithm that produces stable pushgrasp plans for dexterous hands in the presence of object pose uncertainty and high clutter. We demonstrate our algorithm extensively in simulation and on HERB, a personal robotics platform developed at Intel Labs Pittsburgh. Studies of human manipulation strategies suggest that pre-grasp object manipulation, such as rotation or sliding of the object to be grasped, can improve task performance by increasing both the task success rate and the quality of load-supporting postures. In previous demonstrations, pre-grasp object rotation by a robot manipulator was limited to manually-programmed actions. We present a method for automating the planning of pre-grasp rotation for object transport tasks. Our technique optimizes the grasp acquisition point by selecting a target object pose that can be grasped by high-payload manipulator configurations. Careful selection of the transition states leads to successful transport plans for tasks that are otherwise infeasible. In addition, optimization of the grasp acquisition posture also indirectly improves the transport plan quality, as measured by the safety margin of the manipulator payload limits. We present a complete software architecture for reliable grasping of household objects. Our work combines aspects such as scene interpretation from 3D range data, grasp planning, motion planning, and grasp failure identification and recovery using tactile sensors. We build upon, and add several new contributions to the significant prior work in these areas. A salient feature of our work is the tight coupling between perception (both visual and tactile) and manipulation, aiming to address the uncertainty due to sensor and execution errors. This integration effort has revealed new challenges, some of which can be addressed through system and software engineering, and some of which present opportunities for future research. Our approach is aimed at typical indoor environments, and is validated by long running experiments where the PR2 robotic platform was able to consistently grasp a large variety of known and unknown objects. The set of tools and algorithms for object grasping presented here have been integrated into the open-source Robot Operating System (ROS). The class of problems that involve finding where to place or how to move a solid object in the presence of obstacles is discussed. The solution to this class of problems is essential to the automatic planning of manipulator transfer movements, i.e., the motions to grasp a part and place it at some destination. For example, planning transfer movements requires the ability to plan paths for the manipulator that avoid collisions with objects in the workspace and the ability to choose safe grasp points on objects. The approach to these problems described here is based on a method of computing an explicit representation of the manipulator configurations that would bring about a collision. We present a mobile manipulation platform operated by a motor-impaired person using input from a head-tracker, single-button mouse. The platform is used to perform varied and unscripted manipulation tasks in a real home, combining navigation, perception and manipulation. The operator can make use of a wide range of interaction methods and tools, from direct tele-operation of the gripper or mobile base to autonomous sub-modules performing collision-free base navigation or arm motion planning. We describe the complete set of tools that enable the execution of complex tasks, and share the lessons learned from testing them in a real user's home. In the context of grasping, we show how the use of autonomous sub-modules improves performance in complex, cluttered environments, and compare the results to those obtained by novice, able-bodied users operating the same system. We propose a planning method for grasping in cluttered environments, where the robot can make simultaneous contact with multiple objects, manipulating them in a deliberate and controlled fashion. This enables the robot to reach for and grasp the target while simultaneously contacting and moving aside obstacles in order to clear a desired path. We use a physicsbased analysis of pushing to compute the motion of each object in the scene in response to a set of possible robot motions. In order to make the problem computationally tractable, we enable multiple simultaneous robot-object interactions, which we pre-compute and cache, but avoid object-object interactions. Tests on large sets of simulated scenes show that our planner produces more successful grasps in more complex scenes than versions that avoid any interaction with surrounding clutter. Validation on a real robot shows that our grasp evaluation method accurately predicts the outcome of a grasp, and that our approach, in conjunction with state-of-the-art object recognition tools, is applicable in reallife scenes that are highly cluttered and constrained. Learning from demonstration (LfD) is a common technique applied to many problems in robotics, such as populating grasp databases, training for reinforcement learning of high-level skill sets and bootstrapping motion planners. While such approaches are generally highly valued, they rely on the often time-consuming process of gathering user demonstrations, and hence it becomes difficult to attain a sizeable dataset. In this paper, we present a tool capable of recording large numbers of high-dimensional demonstrations of mobile manipulation tasks provided by non-experts in the field. Our tool accomplishes this via a web interface that requires no additional software to be installed beyond a web browser, as well as a scalable architecture that is capable of supporting 10 concurrent demonstrators on a single server. Our architecture employs a lightweight simulation environment to reduce unnecessary computations and improve performance. Furthermore, we show how our tool can be used to gather a large set of demonstrations of a mobile manipulation task by leveraging existing crowdsource platforms. The data set collected has been made available to the robotics community. We also present experiments in which we apply demonstrations collected through our infrastructure to teach a robot how to grasp, to teach a robot how to perform dexterous manipulation tasks such as scooping and to accelerate motion planning for full-body manipulation tasks. We present an approach that enables robots to learn motion primitives that are robust towards state estimation uncertainties. During reaching and preshaping, the robot learns to use fine manipulation strategies to maneuver the object into a pose at which closing the hand to perform the grasp is more likely to succeed. In contrast, common assumptions in grasp planning and motion planning for reaching are that these tasks can be performed independently, and that the robot has perfect knowledge of the pose of the objects in the environment. ''Shared Grasping'' describes a new approach to shared autonomy for grasping and manipulation tasks to be used in a telepresence system. It is realized with two assistance functionalities, which simplify the grasping, and a user interface, which merges haptic feedback and visual assistance. User studies confirm a reduction in operator workload and an increase in task performance while maintaining a high immersion of the operator into the remote environment when using the assistance functions. Humanoid robotics hardware and control techniques have advanced rapidly during the last five years. Presently, several companies have announced the commercial availability of various humanoid robot prototypes. In order to improve the autonomy and overall functionality of these robots, reliable sensors, safety mechanisms, and general integrated software tools and techniques are needed. We believe that the development of practical motion planning algorithms and obstacle avoidance software for humanoid robots represents an important enabling technology. This paper gives an overview of some of our recent efforts to develop motion planning methods for humanoid robots for application tasks involving navigation, object grasping and manipulation, footstep placement, and dynamically-stable full-body motions. We show experimental results obtained by implementations running within a simulation environment as well as on actual humanoid robot hardware. This paper presents a novel robot grasping planning approach that extracts grasp strategies (grasp type, and thumb placement and direction) from human demonstration and integrates them into the grasp planning procedure to generate a feasible grasp concerning the target object geometry and manipulation task. Our study results show that the grasp strategies of grasp type and thumb placement not only represent important human grasp intentions, but also provide meaningful constraints on hand posture and wrist position, which highly reduce both the feasible workspace of a robotic hand and the search space of the grasp planning. This approach has been thoroughly evaluated both in simulation and with a real robotic system for multiple daily living representative objects. We have also demonstrated the robustness of our approach in the experiment with different levels of perception uncertainties. A simple and efficient randomized algorithm is presented for solving single-query path planning problems in high-dimensional configuration spaces. The method works by incrementally building two rapidly-exploring random trees (RRTs) rooted at the start and the goal configurations. The trees each explore space around them and also advance towards each other through, the use of a simple greedy heuristic. Although originally designed to plan motions for a human arm (modeled as a 7-DOF kinematic chain) for the automatic graphic animation of collision-free grasping and manipulation tasks, the algorithm has been successfully applied to a variety of path planning problems. Computed examples include generating collision-free motions for rigid objects in 2D and 3D, and collision-free manipulation motions for a 6-DOF PUMA arm in a 3D workspace. Some basic theoretical analysis is also presented. In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execute the desired grasp, our method enables closed-loop vision-based control, whereby the robot continuously updates its grasp strategy based on the most recent observations to optimize long-horizon grasp success. To that end, we introduce QT-Opt, a scalable self-supervised vision-based reinforcement learning framework that can leverage over 580k real-world grasp attempts to train a deep neural network Q-function with over 1.2M parameters to perform closed-loop, real-world grasping that generalizes to 96 grasp success on unseen objects. Aside from attaining a very high success rate, our method exhibits behaviors that are quite distinct from more standard grasping systems: using only RGB vision-based perception from an over-the-shoulder camera, our method automatically learns regrasping strategies, probes objects to find the most effective grasps, learns to reposition objects and perform other non-prehensile pre-grasp manipulations, and responds dynamically to disturbances and perturbations. Robust remote teleoperation of high-DOF manipulators is of critical importance across a wide range of robotics applications. Contemporary robot manipulation interfaces primarily utilize a free-positioning pose specification approach to independently control each axis of translation and orientation in free space. In this work, we present two novel interfaces, constrained positioning and point-and-click, which incorporate scene information, including points-of-interest and local surface geometry, into the grasp specification process. We also present results of a user study evaluation comparing the effects of increased use of scene information in grasp pose specification algorithms for general object manipulation. The results of our study show that constrained positioning and point-and-click significantly outperform the widely used free positioning approach by significantly reducing the number of grasping errors and the number of user interactions required to specify poses. Furthermore, the point-and-click interface significantly increased the number of tasks users were able to complete. Grasping an object in a cluttered, unorganized environment is challenging because of unavoidable contacts and interactions between the robot and multiple immovable (static) and movable (dynamic) obstacles in the environment. Planning an approach trajectory for grasping in such situations can benefit from physics-based simulations that describe the dynamics of the interaction between the robot manipulator and the environment. In this work, we present a physics-based trajectory optimization approach for planning grasp approach trajectories. We present novel cost objectives and identify failure modes relevant to grasping in cluttered environments. Our approach uses rollouts of physics-based simulations to compute the gradient of the objective and of the dynamics. Our approach naturally generates behaviors such as choosing to push objects that are less likely to topple over, recognizing and avoiding situations which might cause a cascade of objects to fall over, and adjusting the manipulator trajectory to push objects aside in a direction orthogonal to the grasping direction. We present results in simulation for grasping in a variety of cluttered environments with varying levels of density of obstacles in the environment. Our experiments in simulation indicate that our approach outperforms a baseline approach that considers multiple straight-line trajectories modified to account for static obstacles by an aggregate success rate of 14 with varying degrees of object clutter. Planning motions to grasp an object in cluttered and uncertain environments is a challenging task, particularly when a collision-free trajectory does not exist and objects obstructing the way are required to be carefully grasped and moved out. This letter takes a different approach and proposes to address this problem by using a randomized physics-based motion planner that permits robot-object and object-object interactions. The main idea is to avoid an explicit high-level reasoning of the task by providing the motion planner with a physics engine to evaluate possible complex multibody dynamical interactions. The approach is able to solve the problem in complex scenarios, also considering uncertainty in the objects' pose and in the contact dynamics. The work enhances the state validity checker, the control sampler, and the tree exploration strategy of a kinodynamic motion planner called KPIECE. The enhanced algorithm, called p-KPIECE, has been validated in simulation and with real experiments. The results have been compared with an ontological physics-based motion planner and with task and motion planning approaches, resulting in a significant improvement in terms of planning time, success rate, and quality of the solution path. The term "hyper-redundant" refers to robotic manipulators and mobile robots with a very large, possibly infinite, number of actuatable degrees of freedom. These robots are analogous in morphology and operation to snakes, worms, elephant trunks, and tentacles. This thesis presents a novel kinematic framework for hyper-redundant manipulator motion planning and task implementation. The basis of this formulation is the use of a "backbone reference set" which captures the essential macroscopic geometric features of hyper-redundant robots. In the analytical part of this work, the backbone representation is developed and used to solve problems in obstacle avoidance, locomotion, grasping, and "optimal" end effector placement. The latter part of this thesis deals with the design and implementation of a thirty-degree-of-freedom planar hyper-redundant manipulator which is used to demonstrate these novel kinematic and motion planning techniques. Design issues such as robustness with respect to mechanical failure, and design for easy assembly and repair are also addressed. The analytical and design concepts are combined to illustrate tasks for which hyper-redundant robotic mechanisms are well suited. The ability to grasp large objects with both hands enables bimanual robot systems to fully employ their capabilities in human-centered environments. Hence, algorithms are needed to precompute bimanual grasping configurations that can be used online to efficiently create whole body grasps. In this work we present a bimanual grasp planner that can be used to build a set of grasps together with manipulability information for a given object. For efficient grasp planning precomputed reachability information and a beneficial object representation, based on medial axis descriptions, are used. Since bimanual grasps may suffer from low manipulability, caused by a closed kinematic chain, we show how the manipulability of a bimanual grasp can be used as a quality measure. Therefore, manipulability clusters are introduced as an efficient way to approximatively describe the manipulability of a given bimanual grasp. The proposed approach is evaluated with a reference implementation, based on Simox [1], for the humanoid robot ARMAR-III [2]. Since the presented algorithms are robot-independent, there are no limitations for using this planner on other robot systems. We describe the software components of a robotics system designed to autonomously grasp objects and perform dexterous manipulation tasks with only high-level supervision. The system is centered on the tight integration of several core functionalities, including perception, planning and control, with the logical structuring of tasks driven by a Behavior Tree architecture. The advantage of the implementation is to reduce the execution time while integrating advanced algorithms for autonomous manipulation. We describe our approach to 3-D perception, real-time planning, force compliant motions, and audio processing. Performance results for object grasping and complex manipulation tasks of in-house tests and of an independent evaluation team are presented. This paper deals with motion planning for robots manipulating movable objects among obstacles. We propose a general manipulation planning approach capable of addressing continuous sets for modeling both the possible grasps and the stable placements of the movable object, rather than discrete sets generally assumed by the previous approaches. The proposed algorithm relies on a topological property that characterizes the existence of solutions in the subspace of configurations where the robot grasps the object placed at a stable position. It allows us to devise a manipulation planner that captures in a probabilistic roadmap the connectivity of sub-dimensional manifolds of the composite configuration space. Experiments conducted with the planner in simulated environments demonstrate its efficacy to solve complex manipulation problems.