End-to-end learning for autonomous navigation has received substantial attention recently as a promising method for reducing modeling error. However, its data complexity, especially around generalization to unseen environments, is high. We introduce a novel image-based autonomous navigation technique that leverages in policy structure using the Riemannian Motion Policy (RMP) framework for deep learning of vehicular control. We design a deep neural network to predict control point RMPs of the vehicle from visual images, from which the optimal control commands can be computed analytically. We show that our network trained in the Gibson environment can be used for indoor obstacle avoidance and navigation on a real RC car, and our RMP representation generalizes better to unseen environments than predicting local geometry or predicting control commands directly.
 What is it that makes movement around obstacles hard? The answer seems clear: obstacles contort the geometry of the workspace and make it difficult to leverage what we consider easy and intuitive straight-line Cartesian geometry. But is Cartesian motion actually easy? It's certainly well-understood and has numerous applications. But beneath the details of linear algebra and pseudoinverses, lies a non-trivial Riemannian metric driving the solution. Cartesian motion is easy only because the pseudoinverse, our powerhouse tool, correctly represents how Euclidean workspace geometry pulls back into the configuration space. In light of that observation, it reasons that motion through a field of obstacles could be just as easy as long as we correctly account for how those obstacles warp the geometry of the space. This paper explores extending our geometric model of the robot beyond the notion of a Cartesian workspace space to fully model and leverage how geometry changes in the presence of obstacles. Intuitively, impenetrable obstacles form topological holes and geodesics curve around them accordingly. We formalize this intuition and develop a general motion optimization framework called Riemannian Motion Optimization (RieMO) to efficiently find motions using our geometric models. Our experiments demonstrate that, for many problems, obstacle avoidance can be much more natural when placed within the right geometric context. This paper introduces a new mathematical object for modular motion generation called the Riemannian Motion Policy (RMP). An RMP is an acceleration field (dynamical system) coupled with a corresponding Riemannian metric defining directions of importance at each point, typically defined on a nonlinear task space. We derive operators for transforming RMPs through the network of nonlinear maps defining their task spaces and combining multiple RMPs residing at a given task space, and show that these operators optimally preserve the geometric defined by the metrics in a way strongly analogous to the covariant transformations of natural gradients. This framework enables us to bridge the literature by fusing motion policies together across differing motion generation paradigms, from Dynamical Systems and Dynamic Movement Primitives to Optimal Control and Model Predictive Control. RMPs are easy to implement and manipulate, simplify controller design, clarify a number of open questions around how to effectively combine existing techniques, and their properties of geometric consistency, for the first time, make feasible the generic application of a single smooth and reactive motion generation system across a range of robots with zero re-tuning.