The increasing prevalence of Virtual Reality technologies as a platform for gaming and video playback warrants research into how to best apply the current state of the art to challenges in data visualization. Many current VR systems are noncollaborative, while data analysis and visualization is often a multi-person process. Our goal in this paper is to address the technical and user experience challenges that arise when creating VR environments for collaborative data visualization. We focus on the integration of multiple tracking systems and the new interaction paradigms that this integration can enable, along with visual design considerations that apply specifically to collaborative network visualization in virtual reality. We demonstrate a system for collaborative interaction with large 3D layouts of Twitter friend follow networks. The system is built by combining a 'Holojam' architecture (multiple GearVR Headsets within an OptiTrack motion capture stage) and Perception Neuron motion suits, to offer an untethered, full-room multi-person visualization experience.
Work by Donalek et. al. @cite_5 uses uses Unity3D and Oculus VR headsets to visualize astronomical datasets, but has key differences in terms of capabilities and constraints. The rendering in @cite_5 occurs on a personal computer as opposed to a mobile device, and therefore is less resource bound than in our context. Furthermore, the underlying system that we build upon is designed for collaborative full-room VR, allowing users to interact with a social network in much the same way they would a real object.
@cite_5
2116958202
Background: Digital slides obtained from whole slide imaging (WSI) platforms are typically viewed in two dimensions using desktop personal computer monitors or more recently on mobile devices. To the best of our knowledge, we are not aware of any studies viewing digital pathology slides in a virtual reality (VR) environment. VR technology enables users to be artificially immersed in and interact with a computer-simulated world. Oculus Rift is among the world's first consumer-targeted VR headsets, intended primarily for enhanced gaming. Our aim was to explore the use of the Oculus Rift for examining digital pathology slides in a VR environment. Methods: An Oculus Rift Development Kit 2 (DK2) was connected to a 64-bit computer running Virtual Desktop software. Glass slides from twenty randomly selected lymph node cases (ten with benign and ten malignant diagnoses) were digitized using a WSI scanner. Three pathologists reviewed these digital slides on a 27-inch 5K display and with the Oculus Rift after a 2-week washout period. Recorded endpoints included concordance of final diagnoses and time required to examine slides. The pathologists also rated their ease of navigation, image quality, and diagnostic confidence for both modalities. Results: There was 90 diagnostic concordance when reviewing WSI using a 5K display and Oculus Rift. The time required to examine digital pathology slides on the 5K display averaged 39 s (range 10-120 s), compared to 62 s with the Oculus Rift (range 15-270 s). All pathologists confirmed that digital pathology slides were easily viewable in a VR environment. The ratings for image quality and diagnostic confidence were higher when using the 5K display. Conclusion: Using the Oculus Rift DK2 to view and navigate pathology whole slide images in a virtual environment is feasible for diagnostic purposes. However, image resolution using the Oculus Rift device was limited. Interactive VR technologies such as the Oculus Rift are novel tools that may be of use in digital pathology. In near future, we expect the Virtual Reality (VR) headsets to be wireless and to be demanding data-rate up to Tbps, thus pushing RF technology to the limit. In this work, we explore the possibility of using steerable Free Space Optics (FSO) to create a wireless link between a wall-mounted static transceiver and a VR headset in motion with its user. We describe our architecture, develop a link emulator using off-the-shelf optical devices, analyze the response time of the system and its effect on tolerable motion of the VR headset. We describe an in-band feedback mechanism to track the laser beam between the transceivers in motion. We lay out additional challenges to tackle, so as to unleash the full potential of FSO communications for wearable devices like VR headset. We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users' sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users' sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user's awareness of the proximity and presence of others. We introduce NeuroCave, a novel immersive visualization system that facilitates the visual inspection of structural and functional connectome datasets. The representation of the human connectome as a graph enables neuroscientists to apply network-theoretic approaches in order to explore its complex characteristics. With NeuroCave, brain researchers can interact with the connectome— either in a standard desktop environment or while wearing portable VR headsets (such as Oculus Rift, Samsung Gear, or Google Daydream VR platforms)— in any coordinate system or topological space, as well as cluster brain regions into different modules on demand. Furthermore, a default side-by-side layout enables simultaneous, synchronized manipulation in 3D, utilizing modern GPU hardware architecture, and facilitates comparison tasks across different subjects or diagnostic groups or longitudinally within the same subject. Visual clutter is mitigated using a state-of-the-art edge bundling technique and through an interactive lay... All existing high-quality virtual reality (VR) systems (e.g., HTC Vive and Oculus Rift) are tethered, requiring an HDMI cable to connect the head mounted display (HMD) to a PC for rendering rich graphic contents. Such a tethered design not only limits user mobility but also imposes hazards to users. To get rid of the cable, "cable replacement" solutions have been proposed but without any programmability at the HMD side. In this paper, we explore how to build a programmable wireless high-quality VR system using commodity hardware. With programmability at both the PC side and the HMD side, our system provides extensibility and flexibility for exploring various new ideas and software-based techniques in high-quality VR. We present our system design, describe challenges, explore possible solutions to cut the wire, and compare the performance of different approaches for transmitting the high-volume graphics data over a wireless link. We share our experience and report preliminary findings. Experimental results show that building a wireless high-quality VR system is very challenging, and needs extensive effort on both the software and hardware sides in order to meet the performance requirements. This paper presents a framework which allows urban planners to navigate and interact with large datasets fused with social feeds in real-time, enhanced by a virtual reality (VR) capability, which further promotes the knowledge discovery process and allows to interact with urban data in natural yet immersive way. A challenge in urban planning is making decisions based on datasets which are many times ambiguous, together with effective use of newly available yet unstructured sources of information like social media. Providing expert users with novel ways of representing knowledge can be beneficial for decision making. Game engines have evolved into capable testbeds for novel visualization and interaction techniques. We therefore explore the possibility of using a modern game engine as a platform for knowledge representation in urban planning and how it can be used to model ambiguity. We also investigate how urban planners can benefit from immersion when it comes to data exploration and knowledge discovery. We apply the concept of using primitives to publicly available transportation datasets and social feeds of New York city, we discuss a gesture-based VR extension of our framework and lastly, we conclude the paper with feedback from expert users in urban planning and with an outlook of future challenges. Personal media collections are often viewed and managed along the social dimension, the places we spend time at and the people we see, thus tools for extracting and using this information are required. We present novel algorithms for identifying socially significant places termed social spheres unobtrusively from GPS traces of daily life, and label them as one of Home, Work, or Other, with quantitative evaluation of 9 months taken from 5 users. We extract locational co-presence of these users and formulate a novel measure of social tie strength based on frequency of interaction, and the nature of spheres it occurs within. Comparative user studies of a multimedia browser designed to demonstrate the utility of social metadata indicate the usefulness of a simple interface allowing navigation and filtering in these terms. We note the application of social context is potentially much broader than personal media management, including context-aware device behaviour, life logs, social networks, and location-aware information services. Preface 1. Introduction 2. Some context for Context Analysis: a view of the origins of structural studies of face-to-face interaction 3. Some functions of gaze direction in two-person conversation 4. Movement co-ordination in social interaction 5. Some functions of the face in a kissing round 6. A description of some human greetings 7. Spatial organisation in social encounters: the F-formation system 8. Behavioural foundations for the process of frame-attunement in face-to-face interaction List of films cited References Index. We demonstrate a low-cost telerobotic system that leverages commercial virtual reality (VR) technology and integrates it with existing robotics control infrastructure. The system runs on a commercial gaming engine using off-the-shelf VR hardware and can be deployed on multiple network architectures. The system is based on the homunculus model of mind wherein we embed the user in a VR control room. The control room allows for multiple sensor displays, and dynamic mapping between the user and robot. This dynamic mapping allows for selective engagement between the user and the robot. We compared our system with state-of-the-art automation algorithms and standard VR-based telepresence systems by performing a user study. The study showed that new users were faster and more accurate than the automation or a direct telepresence system. We also demonstrate that our system can be used for pick and place, assembly, and manufacturing tasks. There are currently no solutions for enabling direct face-to-face interaction between virtual reality (VR) users wearing head-mounted displays (HMDs). The main challenge is that the headset obstructs a significant portion of a user's face, preventing effective facial capture with traditional techniques. To advance virtual reality as a next-generation communication platform, we develop a novel HMD that enables 3D facial performance-driven animation in real-time. Our wearable system uses ultra-thin flexible electronic materials that are mounted on the foam liner of the headset to measure surface strain signals corresponding to upper face expressions. These strain signals are combined with a head-mounted RGB-D camera to enhance the tracking in the mouth region and to account for inaccurate HMD placement. To map the input signals to a 3D face model, we perform a single-instance offline training session for each person. For reusable and accurate online operation, we propose a short calibration step to readjust the Gaussian mixture distribution of the mapping before each use. The resulting animations are visually on par with cutting-edge depth sensor-driven facial performance capture systems and hence, are suitable for social interactions in virtual worlds. Virtual Reality (VR) has the potential to support motor learning in ways exceeding beyond the possibilities provided by real world environments. New feedback mechanisms can be implemented that support motor learning during the performance of the trainee and afterwards as a performance review. As a consequence, VR environments excel in controlled evaluations, which has been proven in many other application scenarios. However, in the context of motor learning of complex tasks, including full-body movements, questions regarding the main technical parameters of such a system, in particular that of the required maximum latency, have not been addressed in depth. To fill this gap, we propose a set of requirements towards VR systems for motor learning, with a special focus on motion capturing and rendering. We then assess and evaluate state-of-the-art techniques and technologies for motion capturing and rendering, in order to provide data on latencies for different setups. We focus on the end-to-end latency of the overall system, and present an evaluation of an exemplary system that has been developed to meet these requirements. Currently, more and more 360-degree videos (or 360 videos for short) are being provided via the Internet. This kind of videos can render a virtual reality (VR) environment via a head-mounted display (HMD). However, understanding the quality of experience (QoE) of 360 videos is a big challenge because user experience in VR is a very complex phenomenon. In this paper, the QoE of 360 videos is considered in terms of four aspects, namely perceptual quality, presence, acceptability, and cybersickness. Subjective tests are designed to investigate the influences of important factors including encoding parameters, content characteristics, and device types on QoE aspects. In addition, a comparison of perceptual quality and acceptability between VR and non-VR rendering modes, which refer to watching 360 videos with and without using an HMD respectively, is also made in this study. To the best of our knowledge, this is the first study that covers these four QoE aspects and a large number of influence factors. Tracing neurons in large-scale microscopy data is crucial to establishing a wiring diagram of the brain, which is needed to understand how neural circuits in the brain process information and generate behavior. Automatic techniques often fail for large and complex datasets, and connectomics researchers may spend weeks or months manually tracing neurons using 2D image stacks. We present a design study of a new virtual reality (VR) system, developed in collaboration with trained neuroanatomists, to trace neurons in microscope scans of the visual cortex of primates. We hypothesize that using consumer-grade VR technology to interact with neurons directly in 3D will help neuroscientists better resolve complex cases and enable them to trace neurons faster and with less physical and mental strain. We discuss both the design process and technical challenges in developing an interactive system to navigate and manipulate terabyte-sized image volumes in VR. Using a number of different datasets, we demonstrate that, compared to widely used commercial software, consumer-grade VR presents a promising alternative for scientists. Virtual Reality (VR) technology holds promise as a virtual prototyping tool for mechanical assembly; however, several developmental challenges still need to be addressed before virtual prototyping applications can successfully be integrated into the product realization process. This paper describes the development of SHARP (System for Haptic Assembly & Realistic Prototyping), a portable VR interface for virtual assembly. SHARP uses physically-based modeling for simulating realistic part-to-part and hand-to-part interactions in virtual environments. A dual handed haptic interface for realistic part interaction using the PHANToM® haptic devices is presented. The capability of creating subassemblies enhances the application’s ability to handle a wide variety of assembly scenarios. Swept volumes are implemented for addressing maintainability issues and a network module is added for communicating with different VR systems at dispersed geographic locations. Support for various types of VR systems allows an easy integration of SHARP into the product realization process resulting in faster product development, faster identification of assembly and design issues and a more efficient and less costly product design process. Recently, virtual reality (VR) becomes more and more popular and provides users an immersive experience with a head-mounted display (HMD). However, in some applications, users have to interact with physical objects while immersed in VR. With a non-see-through HMD, it is difficult to perceive visual information from the real world. Users must recall the spatial layout of the real surroundings and grope around to find the physical objects. After locating the objects, it is still inconvenient to use them without any visual feedback, which would detract the immersive experience. This paper introduces a computer-based system that is designed to record a surgical procedure with multiple depth cameras and reconstruct in three dimensions the dynamic geometry of the actions and events that occur during the procedure. The resulting 3D-plus-time data takes the form of dynamic, textured geometry and can be immersively examined at a later time; equipped with a Virtual Reality headset such as Oculus Rift DK2, a user can walk around the reconstruction of the procedure room while controlling playback of the recorded surgical procedure with simple VCR-like controls (play, pause, rewind, fast forward). The reconstruction can be annotated in space and time to provide more information of the scene to users. We expect such a system to be useful in applications such as training of medical students and nurses. Although smartphones are increasingly becoming more and more powerful, enabling pervasiveness is severely hindered by the resource limitations of mobile devices. The combination of social interactions and mobile devices in the form of ‘crowd computing’ has the potential to surpass these limitations. In this paper, we introduce Honeybee; a crowd computing framework for mobile devices. Honeybee enables mobile devices to share work, utilize local resources and human collaboration in the mobile context. It employs ‘work stealing’ to effectively load balance tasks across nodes that are a priori unknown. We describe the design of Honeybee, and report initial experimental data from applications implemented using Honeybee. We propose FaceVR, a novel image-based method that enables video teleconferencing in VR based on self-reenactment. State-of-the-art face tracking methods in the VR context are focused on the animation of rigged 3d avatars. While they achieve good tracking performance the results look cartoonish and not real. In contrast to these model-based approaches, FaceVR enables VR teleconferencing using an image-based technique that results in nearly photo-realistic outputs. The key component of FaceVR is a robust algorithm to perform real-time facial motion capture of an actor who is wearing a head-mounted display (HMD), as well as a new data-driven approach for eye tracking from monocular videos. Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances. For instance, we can alter facial expressions or change gaze directions in the prerecorded target video. In a live setup, we apply these newly-introduced algorithmic components. Objective: To examine the accuracy and effectiveness of a novel virtual reality (VR) simulation tool in capturing Parkinson’s disease psychosis (PDP) Background: Despite its frequency, there remains confusion regarding diagnosing PDP. This may reflect knowledge gaps of its spectrum (i.e., illusions, hallucinations, delusions), overlapping non-motor issues (i.e., vivid dreams, dementia), failure of patients caregivers and healthcare providers (HCP) to discuss PDP, or an inability to “see what patients see” in visual hallucinations. With VR technology, one can simulate experiences not possible in “real life,” and these simulations may provide useful educational tools. Design methods: We developed a novel VR simulation of PDP using Oculus rift technology and descriptions by patients, caregivers, and HCP. The simulation, which included two hallucinatory scenarios, was presented at the 2015 AAN and Movement Disorder Society meetings. Viewers at the latter meeting completed an IRB-approved questionnaire, using a 5-point Likert-scale (“very unlikely” to “very likely”), to rate the simulation’s accuracy of capturing PDP symptoms and impact, effectiveness as a teaching tool, influence on medical practice, and side effects. Results: Questionnaires were completed by 502 602 (83.4[percnt]) viewers (56.8[percnt] Movement Disorder specialists, 40[percnt] Allied HCP Other). Both VR scenarios accurately (“likely very likely”) captured PDP symptoms reported by patients and caregivers (76.4[percnt], 67.6[percnt], respectively) and their impact on patients and caregivers (78.4[percnt], 66.2[percnt], respectively). The scenarios were “likely very likely” to be effective teaching tools for HCP (87.5[percnt]) and caregivers (90.8[percnt]), though slightly less for patients (76.1[percnt]). About 50[percnt] of respondents would “likely very likely” change their medical practice. Few experienced difficulties using the simulation, though mild motion sickness was occasionally reported. Conclusions: Using virtual technology, viewers were able to experience PDP first-hand. The PDP VR simulation has potential as an educational tool for not only HCP, but also caregivers and patients. Greater awareness of PDP may help reduce diagnostic confusion and associated stigma. Disclosure: Dr. Goldman has received personal compensation for activities with Acadia, Teva and Pfizer as a consultant and Advisory Board Member. Dr. Goldman has received research support from NIH, Michael J. Fox Foundation, Rush University Medical Center, Parkinson9 Dr. Stebbins has nothing to disclose. Dr. Fredericks has nothing to disclose. Dr. Upchurch has nothing to disclose. Small, mobile devices without user interfaces, such as Bluetooth headsets, often need to communicate securely over wireless networks. Active attacks can only be prevented by authenticating wireless communication, which is problematic when devices do not have any a priori information about each other. We introduce a new method for device-to-device authentication by shaking devices together. This paper describes two protocols for combining cryptographic authentication techniques with known methods of accelerometer data analysis to the effect of generating authenticated, secret keys. The protocols differ in their design, one being more conservative from a security point of view, while the other allows more dynamic interactions. Three experiments are used to optimize and validate our proposed authentication method.