Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors amid challenging cases of clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after only a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. Qualitative results (videos), code, pre-trained models, and simulation environments are available at this http URL
Non-prehensile manipulation. Planning non-prehensile motions, such as pushing, is a fundamental problem that dates back to the early days of robotic manipulation. The literature in this area is vast, emerging early from classical solutions that explicitly model the dynamics of pushing with frictional forces @cite_14 @cite_35 . While inspiring, many of these methods rely on modeling assumptions that do not hold in practice @cite_1 @cite_37 . For example, non-uniform friction distributions across object surfaces and the variability of friction are only some of the factors that can lead to erroneous predictions of friction-modeling pushing solutions in real-world settings. While recent methods have explored data-driven algorithms for learning the dynamics of pushing @cite_5 @cite_17 @cite_20 , many of these works have largely focused on the execution of stable pushes for one object at a time. Modeling the larger-scale consequences of pushing in the face of severe clutter and friction variation continues to be a complex problem; effectively using these models to discover optimal policies in real settings -- even more so.
@cite_35 @cite_37 @cite_14 @cite_1 @cite_5 @cite_20 @cite_17
2029342956 2962867039 2963030226 1608593143 2963892386
Abstract We present two geometric descriptions of the net frictional force and moment between a rigid body and a planar surface on which it slides. The limit surface, from classical plasticity theory, is the surface in load space which bounds the set of all possible frictional forces and moments that can be sustained by the frictional interface. Zhukovskii's moment function (N.E. Zhukovskii, Collected Works, Vol. 1, Gostekhizdat, Moscow, 1948, pp. 339–354) is the net frictional moment about the body's instantaneous center of rotation as a function of its location. Both of these descriptions implicitly contain the full relation between slip motion and frictional load. While Zhukovskii's moment function applies only to ordinary isotropic Coulomb friction, the limit surface applies to a wider class of friction laws that includes, for example, contact mediated by massless rigid wheels. Both the limit surface and the moment function can be used to deduce results concerning the motion of sliding rigid bodies. This paper presents a data-driven approach to model planar pushing interaction to predict both the most likely outcome of a push and its expected variability. The learned models rely on a variation of Gaussian processes with input-dependent noise called Variational Heteroscedastic Gaussian processes (VHGP) [1] that capture the mean and variance of a stochastic function. We show that we can learn accurate models that outperform analytical models after less than 100 samples and saturate in performance with less than 1000 samples. We validate the results against a collected dataset of repeated trajectories, and use the learned models to study questions such as the nature of the variability in pushing, and the validity of the quasi-static assumption. Pushing is a motion primitive useful to handle objects that are too large, too heavy, or too cluttered to be grasped. It is at the core of much of robotic manipulation, in particular when physical interaction is involved. It seems reasonable then to wish for robots to understand how pushed objects move. We describe an unsupervised on-line method for learning of manipulative actions that allows a robot to push an object connected to it with a rotational point con- tact to a desired point in image-space. By observing tile results of its actions on tile object's orientation in image- space, the system forms a predictive forward empirical model. This acquired model is used on-line for manipula- tion planning and control as it improves. Rather than ex- plicitly inverting the forward model to achieve trajectory control, a stochastic action selection technique (Moore, 1990) is used to select the most informative and promising actions, thereby integrating active perception and learn- ing by combining on-line improvement, task-directed ex- ploration, and model exploitation. Simulation and exper- imental results of the approach are presented. We propose a polynomial force-motion model for planar sliding. The set of generalized friction loads is the 1-sublevel set of a polynomial whose gradient directions correspond to generalized velocities. Additionally, the polynomial is confined to be convex even-degree homogeneous in order to obey the maximum work inequality, symmetry, shape invariance in scale, and fast invertibility. We present a simple and statistically-efficient model identification procedure using a sum-of-squares convex relaxation. Simulation and robotic experiments validate the accuracy and efficiency of our approach. We also show practical applications of our model including stable pushing of objects and free sliding dynamic simulations.