We propose a new approach for interaction in Virtual Reality (VR) using mobile robots as proxies for haptic feedback. This approach allows VR users to have the experience of sharing and manipulating tangible physical objects with remote collaborators. Because participants do not directly observe the robotic proxies, the mapping between them and the virtual objects is not required to be direct. In this paper, we describe our implementation, various scenarios for interaction, and a preliminary user study.
Haptic feedback has been mentioned frequently in VR training, especially in the medical field. The sense of touch is the earliest developed in human embryology and is believed to be essential for practice @cite_5 @cite_0 . Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon @cite_13 . Haptic feedback has potential benefits not only in training, but in other interactions as well. Reality based interaction was proposed for post-WIMP interfaces @cite_4 . Tangible interactions are observable with both visual and haptic modalities that could help people utilize basic knowledge about the behavior of our physical world @cite_6 .
@cite_4 @cite_6 @cite_0 @cite_5 @cite_13
2009180946 2476739747 2057244156 1489557789
Purpose of Review Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon while minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks. The efficient control of our body and successful interaction with the environment are possible through the integration of multisensory information. Brain–computer interface (BCI) may allow people with sensorimotor disorders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and-place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot’s perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous asynchronous auditory feedback. Healthy participants achieved better placing accuracy when listening to human footstep sounds relative to a computer-generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feedback conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to understand how the contribution of residual sensory channels could improve the reliability of BCI systems. Explains the need for haptics (feeling of touch) in medical simulation systems. Describes a variety of laparoscopic training systems and other surgical simulators. Highlights the Reachin Technologies AB Application Programming Interface (API) which is a software tool that significantly speeds up the development of surgical simulators. Copyright © 2004 Robotic Publications Ltd. One of the most disconcertingly unnatural properties of most virtual environments (VEs) is the ability of the user to pass through objects. I hypothesize that passive haptics, augmenting a high-fidelity visual virtual environment with low-fidelity physical objects, will markedly improve both sense of presence and spatial knowledge training transfer. The low-fidelity physical models can be constructed from cheap, easy-to-assemble materials such as styrofoam, plywood, and particle board. The first study investigated the effects of augmenting a visual-cliff environment with a slight physical ledge on participants' sense of presence. I found when participants experienced passive haptics in the VE, they exhibited significantly more behaviors associated with pit avoidance than when experiencing the non-augmented VE. Changes in heart rate and skin conductivity were significantly higher than when they experienced the VE without passive haptics. The second study investigated passive haptics' effects on performance of a real-world navigation task after training in a virtual environment. Half of the participants trained on maze navigation in a VE augmented with a styrofoam physical model, while half trained in a non-augmented VE but were given visual and audio contact cues. The task was to gain as much information as possible about the layout of the environment. Participants knew before the VE session that their training would be tested by navigating an identical real maze environment while blindfolded. Significant differences in the time to complete the blindfolded navigation task and significant differences in the number of collisions with objects were found between the participants trained in an augmented VE and the participants trained in a non-augmented VE. 11 of 15 participants trained without passive haptics bumped into the next-to-last obstacle encountered in the testing session and turned the wrong direction to navigate around it: only 2 of 15 participants trained with passive haptics made the same navigation error. On the other hand, the assessment of the participants' cognitive maps of the virtual environment did not find significant differences between groups as measured by sketch maps and object dimension estimation.