In this paper, we propose a new interface for virtual reality headset: a touchpad in front of the headset. To demonstrate the feasibility of the front touch interface, we built a prototype device, explored VR UI design space expansion, and performed various user studies. We started with preliminary tests to see how intuitively and accurately people can interact with the front touchpad. Then, we further experimented various user interfaces such as a binary selection, a typical menu layout, and a keyboard. Two-Finger and Drag-n-Tap were also explored to find the appropriate selection technique. As a low-cost, light-weight, and in low power budget technology, a touch sensor can make an ideal interface for mobile headset. Also, front touch area can be large enough to allow wide range of interaction types such as multi-finger interactions. With this novel front touch interface, we paved a way to new virtual reality interaction methods.
There are a plethora of research and industry products on virtual reality devices and interactions. However, an ability to provide precise and intuitive input signal is still identified as an issue among the VR users @cite_23 . In this section, we address how prior studies approached to solve selection tasks, especially with the menu interface, and text-entry.
@cite_23
2095128833 2028086564 2533480359 2085882361 2159923417 2005864097 2104901536 2039239232 2022891348 2154868891 1592369715 2015247397 1553093163 182695033 2011442434 2888940161 2750802720 2081752675 2556634673 2051495970
Virtual reality applications are emerging into various regions of research and entertainment. Although visual and acoustic capabilities are already quite impressive, a wide range of users still criticizes the user interface. Frequently complex and very sensitive input devices are being used, although simple gestures would be preferred. While gesture recognition systems are quite common, see Nintendo's Wii mote, a CAVE has further challenges, as the person can be located in any random position and the gestures are not being performed related to a common fixpoint. Applying an infrared tracking system it is possible to reliably locate the hand and compute 3D trajectories. These are then further analyzed with a Long Short-Term Memory approach, which is able to model sequences of variable length with a higher reliability than HMMs. We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users' sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users' sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user's awareness of the proximity and presence of others. Virtual reality systems often co-locate the display and input (motor) spaces. Many input devices, such as the mouse, use indirect input mappings, and are disjoint from the display space. A study of visual motor co-location was conducted to determine if there is any benefit to working directly “in” a virtual environment. Using a fish-tank VR setup, participants performed a 3D object movement task. This required moving an object from the centre of the environment to target regions, using a tracked pen, in both co-located and disjoint display input conditions. Results were analyzed in the context of Fitts' Law, which models rapid aimed movements. Ultimately, no significant differences were found between co-located and disjoint conditions. However, when analyzing object movement in specific directions, the co-located condition was somewhat better than the disjoint one. In particular, movement into the scene was faster when the display and input device were co-located rather than disjoint. We report results from a formal user study of interactive 3D rotation using the mouse-driven Virtual Sphere and Arcball techniques, as well as multidimensional input techniques based on magnetic orientation sensors. MultidimensionaI input is often assumed to allow users to work quickly, but at the cost of precision, due to the instability of the hand moving in the open air. We show that, at least for the orientation matching task used in this experiment, users can take advantage of the integrated degrees of freedom provided by multidimensional input without necessarily sacrificing precision: using multidimensional input, users completed the experimental task up to 36 faster without any statistically detectable loss of accuracy. We also report detailed observations of common usability problems when first encountering the techniques. Our observations suggest some design issues for 3D input devices. For example, the physical form-factors of the 3D input device significantly influenced user acceptance of otherwise identical input sensors. The device should afford some tactile cues, so the user can feel its orientation without looking at it. In the absence of such cues, some test users were unsure of how to use the device. Virtual reality (VR) technology holds promise as a virtual prototyping (VP) tool for mechanical assembly; however, several developmental challenges still need to be addressed before VP applications can successfully be integrated into the product realization process. This paper describes the development of System for Haptic Assembly and Realistic Prototyping (SHARP), a portable virtual assembly system. SHARP uses physics-based modeling for simulating realistic part-to-part and hand-to-part interactions in virtual environments. A dual-handed haptic interface for a realistic part interaction using the PHANToM® haptic devices is presented. The capability of creating subassemblies enhances the application’s ability to handle a wide variety of assembly scenarios at the part level as well as at the subassembly level. Swept volumes are implemented for addressing maintainability issues, and a network module is added for communicating with different VR systems at dispersed geographic locations. Support for various types of VR systems allows an easy integration of SHARP into the product realization process, resulting in faster product development, faster identification of assembly and design issues, and a more efficient and less costly product design process. An increase number of gesture sensors and AR(augmented reality) displays are emerging these years. There are more and more study working on spatial interaction. In this work, we focus on evaluating the feasibility of hand-based interaction designed for AR glasses to do basic manipulation on 3D virtual objects in AR applications with current interaction technology. We design a user study with 10 volunteers to discuss whether and how to use natural user interfaces like gestures, with the help of 3D menus, to finish the tasks of manipulating AR scenes. The interface including 3D menus presents the advantages of usability and merits of intuition; However, it also exposes some issues like ergonomic discomfort and limitation like imprecise input. So our results indicate the kind of interactive device and technology are valuable in natural interaction without extra knowledge, though they are limited in serious applications now. Presents a novel class of virtual reality input devices that combine pop-through buttons with 6-DOF trackers. Compared to similar devices that use conventional buttons, pop-through devices double the number of potential discrete interaction modes, since each button has two activation states corresponding to light and firm pressure. This additional state per button provides a foundation to address a range of shortcomings with conventional virtual environment (VE) input devices that includes reducing the physical dexterity required to perform interactions, reducing the cognitive complexity of some compound tasks and enabling the design of less obtrusive devices without sacrificing expressive power. Specifically, we present two novel input devices: the FingerSleeve was designed to be minimally obtrusive physically, whereas the TriggerGun was designed to be physically similar to, yet more functional than a conventional hand-held trigger device. Further, we present a set of novel navigation and interaction techniques that leverage the capabilities of our pop-through button devices to improve interaction quality and provide insight into harnessing the potential of pop-through buttons for other tasks. Finally, we discuss how we incorporated one of our devices into a real application. There are currently no solutions for enabling direct face-to-face interaction between virtual reality (VR) users wearing head-mounted displays (HMDs). The main challenge is that the headset obstructs a significant portion of a user's face, preventing effective facial capture with traditional techniques. To advance virtual reality as a next-generation communication platform, we develop a novel HMD that enables 3D facial performance-driven animation in real-time. Our wearable system uses ultra-thin flexible electronic materials that are mounted on the foam liner of the headset to measure surface strain signals corresponding to upper face expressions. These strain signals are combined with a head-mounted RGB-D camera to enhance the tracking in the mouth region and to account for inaccurate HMD placement. To map the input signals to a 3D face model, we perform a single-instance offline training session for each person. For reusable and accurate online operation, we propose a short calibration step to readjust the Gaussian mixture distribution of the mapping before each use. The resulting animations are visually on par with cutting-edge depth sensor-driven facial performance capture systems and hence, are suitable for social interactions in virtual worlds. Virtual Reality (VR) technology holds promise as a virtual prototyping tool for mechanical assembly; however, several developmental challenges still need to be addressed before virtual prototyping applications can successfully be integrated into the product realization process. This paper describes the development of SHARP (System for Haptic Assembly & Realistic Prototyping), a portable VR interface for virtual assembly. SHARP uses physically-based modeling for simulating realistic part-to-part and hand-to-part interactions in virtual environments. A dual handed haptic interface for realistic part interaction using the PHANToM® haptic devices is presented. The capability of creating subassemblies enhances the application’s ability to handle a wide variety of assembly scenarios. Swept volumes are implemented for addressing maintainability issues and a network module is added for communicating with different VR systems at dispersed geographic locations. Support for various types of VR systems allows an easy integration of SHARP into the product realization process resulting in faster product development, faster identification of assembly and design issues and a more efficient and less costly product design process. Virtual Reality (VR) has been constantly evolving since its early days, and is now a fundamental technology in different application areas. User evaluation is a crucial step in the design and development of VR systems that do respond to users' needs, as well as for identifying applications that indeed gain from the use of such technology. Yet, there is not much work reported concerning usability evaluation and validation of VR systems, when compared with the traditional desktop setup. The paper presents a user study performed, as a first step, for the evaluation of a low-cost VR system using a Head-Mounted Display (HMD). That system was compared to a traditional desktop setup through an experiment that assessed user performance, when carrying out navigation tasks in a game scenario for a short period. The results show that, although users were generally satisfied with the VR system, and found the HMD interaction intuitive and natural, most performed better with the desktop setup. The use of virtual reality (VR) in interactive design and manufacture has been researched extensively but its practical application in industry is still very much in its infancy. Indeed one would have expected that, after some 30 years of research, commercial applications of interactive design or manufacturing planning and analysis would be widespread throughout the product design domain. Similarly, investigations into virtual environments (VE) for assembly and disassembly tasks have been carried out for many years. Given the availability of moderately-priced high performance computing technology, many of these virtual manufacturing interfaces which only stimulate the visual senses – have made actual physical contact during product development an increasingly rare occurrence. Symbolic input, including text and numeric input, can be an important user task in applications of virtual environments (VEs). However, very little research has been performed to support this task in immersive VEs. This paper presents the results of an empirical evaluation of four text input techniques for immersive VEs. The techniques include the Pinch Keyboard (a typing emulation technique using pinch gloves), a one-hand chord keyboard, a soft keyboard using a pen & tablet, and speech. The experiment measured both task performance and usability characteristics of the four techniques. Results indicate that the speech technique is the fastest, while the pen & tablet keyboard produces the fewest errors. However, no single technique exhibited high levels of performance, usability and user satisfaction. The growing power of computing devices allows the representation of three-dimensional interactive virtual worlds. Interfaces with such a world must profit from our experience in the interaction with the real world. This paper corrects the early taxonomy of interaction devices and actions introduced by Foley for screen based interactive systems by adapting it to real world and to virtual reality systems. Basing on the taxonomy derived, the paper presents a model for a Virtual Reality system based on Systems Theory. The model is capable of including both traditional event-based interaction input devices, as well as continuous input devices. It is strongly device oriented, and allows to model mathematically all currently possible input devices for Virtual Reality. The model has been used for the implementation of a general input device library serving as an abstraction layer to a Virtual Reality system. With the embedding of EEG (electro-encephalography) sensors in wireless headsets and other consumer electronics, authenticating users based on their brainwave signals has become a realistic possibility. We undertake an experimental study of the usability and performance of user authentication using consumer-grade EEG sensor technology. By choosing custom tasks and custom acceptance thresholds for each subject, we can achieve 99 authentication accuracy using single-channel EEG signals, which is on par with previous research employing multi-channel EEG signals using clinical-grade devices. In addition to the usability improvement offered by the single-channel dry-contact EEG sensor, we also study the usability of different classes of mental tasks. We find that subjects have little difficulty recalling chosen “pass-thoughts” (e.g., their previously selected song to sing in their mind). They also have different preferences for tasks based on the perceived difficulty and enjoyability of the tasks. These results can inform the design of authentication systems that guide users in choosing tasks that are both usable and secure. Smart glasses, such as Google Glass, provide always-available displays not offered by console and mobile gaming devices, and could potentially offer a pervasive gaming experience. However, research on input for games on smart glasses has been constrained by the available sensors to date. To help inform design directions, this paper explores user-defined game input for smart glasses beyond the capabilities of current sensors, and focuses on the interaction in public settings. We conducted a user-defined input study with 24 participants, each performing 17 common game control tasks using 3 classes of interaction and 2 form factors of smart glasses, for a total of 2448 trials. Results show that users significantly preferred non-touch and non-handheld interaction over using handheld input devices, such as in-air gestures. Also, for touch input without handheld devices, users preferred interacting with their palms over wearable devices (51 vs 20 ). In addition, users preferred interactions that are less noticeable due to concerns with social acceptance, and preferred in-air gestures in front of the torso rather than in front of the face (63 vs 37 ). In near future, we expect the Virtual Reality (VR) headsets to be wireless and to be demanding data-rate up to Tbps, thus pushing RF technology to the limit. In this work, we explore the possibility of using steerable Free Space Optics (FSO) to create a wireless link between a wall-mounted static transceiver and a VR headset in motion with its user. We describe our architecture, develop a link emulator using off-the-shelf optical devices, analyze the response time of the system and its effect on tolerable motion of the VR headset. We describe an in-band feedback mechanism to track the laser beam between the transceivers in motion. We lay out additional challenges to tackle, so as to unleash the full potential of FSO communications for wearable devices like VR headset. All existing high-quality virtual reality (VR) systems (e.g., HTC Vive and Oculus Rift) are tethered, requiring an HDMI cable to connect the head mounted display (HMD) to a PC for rendering rich graphic contents. Such a tethered design not only limits user mobility but also imposes hazards to users. To get rid of the cable, "cable replacement" solutions have been proposed but without any programmability at the HMD side. In this paper, we explore how to build a programmable wireless high-quality VR system using commodity hardware. With programmability at both the PC side and the HMD side, our system provides extensibility and flexibility for exploring various new ideas and software-based techniques in high-quality VR. We present our system design, describe challenges, explore possible solutions to cut the wire, and compare the performance of different approaches for transmitting the high-volume graphics data over a wireless link. We share our experience and report preliminary findings. Experimental results show that building a wireless high-quality VR system is very challenging, and needs extensive effort on both the software and hardware sides in order to meet the performance requirements. Over the past few years, virtual reality has experienced a resurgence. Fueled by a proliferation of consumer-level head-mounted display and motion tracking devices, an unprecedented quantity of immersive experiences and content are available for both desktop and mobile platforms. However, natural locomotion in immersive virtual environments remains a significant challenge. Many of the VR applications available to date require seated use or limit body movement within a small area, instead relying a gamepad or mouse keyboard for movement within the virtual world. Lacking support for natural walking, these virtual reality experiences do not fully replicate the physical and perceptual cues from the real world, and often fall short in maintaining the illusion that the user has been transported to another place. We present a virtual reality demonstration that supports infinite walking within a confined physical space. This is achieved using redirected walking, a class of techniques that introduce subtle discrepancies between physical and virtual motions [ 2001]. When employed properly, redirected walking can be stunningly effective. Previous research has made users believe that they walked in a straight line when they actually traveled in a wide circle, or that they walked between waypoints in a long virtual hallway when in fact they went back and forth between the same two points in the real world. While perceptually compelling, redirected walking is challenging to employ effectively in an unconstrained scenario because users' movements may often be unpredictable. Therefore, our recent research has focused on dynamic planning and optimization of redirected walking techniques, enabling the system to intelligently apply redirection as users explore virtual environments of arbitrary size and shape [ 2014b] [ 2014a]. In this Emerging Technologies exhibit, attendees will explore a large-scale, outdoor immersive virtual environment in a head-mounted display (see Figure 1). The demonstration will support natural walking within a physical area of at least 6x6m, using a wide-area motion tracking system provided by PhaseSpace Inc. The virtual reality scenario will instruct users to scout the environment while stopping to take panoramic photos at various locations in the virtual world. As users explore the environment, our automated planning algorithm will dynamically apply redirection to optimally steer them away from the physical boundaries of the exhibit, thus enabling the experience of limitless walking in a potentially infinite virtual world (see Figure 2). Significant challenges currently prohibit expressive interaction in virtual reality (VR). Occlusions introduced by head-mounted displays (HMDs) make existing facial tracking techniques intractable, and even state-of-the-art techniques used for real-time facial tracking in unconstrained environments fail to capture subtle details of the user's facial expressions that are essential for compelling speech animation. We introduce a novel system for HMD users to control a digital avatar in real-time while producing plausible speech animation and emotional expressions. Using a monocular camera attached to an HMD, we record multiple subjects performing various facial expressions and speaking several phonetically-balanced sentences. These images are used with artist-generated animation data corresponding to these sequences to train a convolutional neural network (CNN) to regress images of a user's mouth region to the parameters that control a digital avatar. To make training this system more tractable, we use audio-based alignment techniques to map images of multiple users making the same utterance to the corresponding animation parameters. We demonstrate that this approach is also feasible for tracking the expressions around the user's eye region with an internal infrared (IR) camera, thereby enabling full facial tracking. This system requires no user-specific calibration, uses easily obtainable consumer hardware, and produces high-quality animations of speech and emotional expressions. Finally, we demonstrate the quality of our system on a variety of subjects and evaluate its performance against state-of-the-art real-time facial tracking techniques. The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.