Head Mounted Displays (HMDs) allow users to experience virtual reality with a great level of immersion. However, even simple physical tasks like drinking a beverage can be difficult and awkward while in a virtual reality experience. We explore mixed reality renderings that selectively incorporate the physical world into the virtual world for interactions with physical objects. We conducted a user study comparing four rendering techniques that balances immersion in a virtual world with ease of interaction with the physical world. Finally, we discuss the pros and cons of each approach, suggesting guidelines for future rendering techniques that bring physical objects into virtual reality.
Since its inception, a large body of VR work has explored incorporating virtual models of physical objects into virtual experiences. For instance, a low-fidelity model of a user's hands could be captured using data input gloves @cite_8 . Previous work has also explored approaches for directly merging the virtual world with the physical world @cite_0 . Augmented Reality (AR) overlays virtual objects into the physical world, and has a rich history of use in mobile phones and HMDs (e.g., @cite_6 ). In contrast, our work is aligned with Augmented Virtuality (AV), where virtual reality is enhanced with parts of the physical world, grounding the experience in the virtual world. Previous work in AV has focused on collaborative applications including displaying real world video on virtual office windows @cite_9 or displaying group communication around a virtual table @cite_5 . More recent work has explored physical depth based renderings of a user's hands in VR for productivity applications @cite_4 . In contrast, we focus on peripheral physical interactions, exploring a design space of rendering techniques that selectively show aspects of the physical world, reinforcing immersion while minimizing distraction.
@cite_4 @cite_8 @cite_9 @cite_6 @cite_0 @cite_5
2142028894 2081752675 2028086564 2058352122
In video see-through augmented reality (AR), virtual objects are overlaid over digital video images. One particular problem of this image mixing process is that the visual appearance of the computer graphics differs strongly from the real background image. The reason for this is that typical AR systems use fast but simple real-time rendering techniques for displaying virtual objects. In this paper, methods for reducing the impact of three effects which make virtual and real objects easily distinguishable are presented. The first effect is camera image noise, which is contained in the data delivered by the image sensor used for capturing the real scene. The second effect considered is edge aliasing, which makes distinguishing virtual objects from real objects simple. Finally, we consider motion blur, which is caused by the temporal integration of color intensities in the image sensor during fast movements of the camera or observed objects. In this paper, we present a system for generating a realistic simulation of image noise based on a new camera calibration step. Additionally, a rendering algorithm is introduced, which performs a smooth blending between the camera image and virtual objects at their boundary in order to reduce aliasing. Lastly, a rendering method is presented, which produces motion blur according to the current camera movement. The implementation of the new rendering techniques utilizes the programmability of modern graphics processing units (GPUs) and delivers real-time frame rates. Over the past few years, virtual reality has experienced a resurgence. Fueled by a proliferation of consumer-level head-mounted display and motion tracking devices, an unprecedented quantity of immersive experiences and content are available for both desktop and mobile platforms. However, natural locomotion in immersive virtual environments remains a significant challenge. Many of the VR applications available to date require seated use or limit body movement within a small area, instead relying a gamepad or mouse keyboard for movement within the virtual world. Lacking support for natural walking, these virtual reality experiences do not fully replicate the physical and perceptual cues from the real world, and often fall short in maintaining the illusion that the user has been transported to another place. We present a virtual reality demonstration that supports infinite walking within a confined physical space. This is achieved using redirected walking, a class of techniques that introduce subtle discrepancies between physical and virtual motions [ 2001]. When employed properly, redirected walking can be stunningly effective. Previous research has made users believe that they walked in a straight line when they actually traveled in a wide circle, or that they walked between waypoints in a long virtual hallway when in fact they went back and forth between the same two points in the real world. While perceptually compelling, redirected walking is challenging to employ effectively in an unconstrained scenario because users' movements may often be unpredictable. Therefore, our recent research has focused on dynamic planning and optimization of redirected walking techniques, enabling the system to intelligently apply redirection as users explore virtual environments of arbitrary size and shape [ 2014b] [ 2014a]. In this Emerging Technologies exhibit, attendees will explore a large-scale, outdoor immersive virtual environment in a head-mounted display (see Figure 1). The demonstration will support natural walking within a physical area of at least 6x6m, using a wide-area motion tracking system provided by PhaseSpace Inc. The virtual reality scenario will instruct users to scout the environment while stopping to take panoramic photos at various locations in the virtual world. As users explore the environment, our automated planning algorithm will dynamically apply redirection to optimally steer them away from the physical boundaries of the exhibit, thus enabling the experience of limitless walking in a potentially infinite virtual world (see Figure 2). We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users' sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users' sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user's awareness of the proximity and presence of others. This paper presents a technique for natural, fingertip-based interaction with virtual objects in Augmented Reality (AR) environments. We use image processing software and finger- and hand-based fiducial markers to track gestures from the user, stencil buffering to enable the user to see their fingers at all times, and fingertip-based haptic feedback devices to enable the user to feel virtual objects. Unlike previous AR interfaces, this approach allows users to interact with virtual content using natural hand gestures. The paper describes how these techniques were applied in an urban planning interface, and also presents preliminary informal usability results.