High Fidelity Grasping in Virtual Reality using a Glove based System
This paper presents a design that jointly provides hand pose sensing, hand localization, and haptic feedback to facilitate real time stable grasps in Virtual Reality (VR) The design is based on an easy to replicate glove based system that can reliably perform (i) a high fidelity hand pose sensing in real time through a network of 15 IMUs, and (ii) the hand localization using a Vive Tracker. The supported physics based simulation in VR is capable of detecting collisions and contact points for virtual object manipulation, which drives the collision event to trigger the physical vibration motors on the glove to signal the user, providing a better realism inside virtual environments. A caging based approach using collision geometry is integrated to determine whether a grasp is stable. In the experiment, we showcase successful grasps of virtual objects with large geometry variations. Comparing to the popular LeapMotion sensor, we demonstrate the proposed glove based design yields a higher success rate in various tasks in VR. We hope such a glove based system can simplify the data collection of human manipulations with VR.
2019 International Conference on Robotics and Automation (ICRA)