MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple users can not only see and hear but also interact with each other, grasp and manipulate objects, walk around in space, and get tactile feedback. MS2 allows walking in physical space by tracking each user’s skeleton in realtime and allows users to feel by employing passive haptics i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. To enable these elements in VR, MS2 creates a correspondence in spatial layout and object placement by building the virtual world on top of a 3D scan of the real world. Through the association between the real and virtual world, users are able to walk freely while wearing a head-mounted device, avoid obstacles like walls and furniture, and interact with people and objects. Most current virtual reality (VR) environments are designed for a single user experience where interactions with virtual objects are mediated by hand-held input devices or hand gestures. Additionally, users are only shown a representation of their hands in VR floating in front of the camera as seen from a first person perspective. We believe, representing each user as a fullbody avatar that is controlled by natural movements of the person in the real world (see Figure 1d), can greatly enhance believability and a user’s sense immersion in VR.
In the real world, people have a physical presence in their spatial environment. We are surrounded by objects and landscape that facilitate our sense of orientation and spatial understanding @cite_6 . For example, the horizon gives us a sense of directional information, occlusions give us relative distance cues while atmospheric color, fog, lighting, and shadow provide depth cues @cite_9 . People also develop skills to manipulate objects in their environment, such as picking up, positioning, altering, and arranging objects @cite_6 . Our VR system uses these skills along with people's innate sense of motion and proprioception for interacting naturally with the virtual world, grasping and manipulating objects in both the real and virtual environments.
@cite_9 @cite_6
1561256858 1997447598
Foreword. Preface. I. FOUNDATIONS OF 3D USER INTERFACES . 1. Introduction to 3D User Interfaces. What Are 3D User Interfaces? Why 3D User Interfaces? Terminology. Application Areas. Conclusion. 2. 3D User Interfaces: History and Roadmap. History of 3D UIs. Roadmap to 3D UIs. Scope of This Book. Conclusion. II. Hardware Technologies for 3D User Interfaces. 3. 3D User Interface Output Hardware. Introduction. Visual Displays. Auditory Displays. Haptic Displays. Design Guidelines: Choosing Output Devices for 3D User Interfaces. Conclusion. 4. 3D User Interface Input Hardware. Introduction. Desktop Input Devices. Tracking Devices. 3D Mice. Special-Purpose Input Devices. Direct Human Input. Home-Brewed Input Devices. Choosing Input Devices for 3D Interfaces. III. 3D INTERACTION TECHNIQUES. 5. Selection and Manipulation. Introduction. 3D Manipulation Tasks. Manipulation Techniques and Input Devices. Interaction Techniques for 3D Manipulation. Design Guidelines. 6. Travel. Introduction. 3D Travel Tasks. Travel Techniques. Design Guidelines. 7. Wayfinding. Introduction. Theoretical Foundations. User-Centered Wayfinding Support. Environment-Centered Wayfinding Support. Evaluating Wayfinding Aids. Design Guidelines. Conclusion. 8. System Control. Introduction. Classification. Graphical Menus. Voice Commands. Gestural Commands. Tools. Multimodal System Control Techniques. Design Guidelines. Case Study: Mixing System Control Methods. 8.10. Conclusion. 9. Symbolic Input. Introduction. Symbolic Input Tasks. Symbolic Input Techniques. Design Guidelines. Beyond Text and Number Entry. IV. DESIGNING AND DEVELOPING 3D USER INTERFACES. 10. Strategies for Designing and Developing 3D User Interfaces. Introduction. Designing for Humans. Inventing 3D User Interfaces. Design Guidelines. 11. Evaluation of 3D User Interfaces. Introduction. Background. Evaluation Metrics for 3D Interfaces. Distinctive Characteristics of 3D Interface Evaluation. Classification of 3D Evaluation Methods. Two Multimethod Approaches. Guidelines for 3D Interface Evaluation. V. THE FUTURE OF 3D USER INTERFACES. 12. Beyond Virtual: 3D User Interfaces for the Real World. Introduction. AR Interfaces as 3D Data Browsers. 3D Augmented Reality Interfaces. Augmented Surfaces and Tangible Interfaces. Tangible AR Interfaces. Agents in AR. Transitional AR-VR Interfaces. Conclusion. 13. The Future of 3D User Interfaces. Questions about 3D UI Technology. Questions about 3D Interaction Techniques. Questions about 3D UI Design and Development. Questions about 3D UI Evaluation. Million-Dollar Questions. Appendix A: Quick Reference Guide to 3D User Interface Mathematics. Scalars. Vectors. Points. Matrices. Quaternions. Bibliography. Index. We are in the midst of an explosion of emerging human-computer interaction techniques that redefine our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Based on this concept of RBI, we provide a framework that can be used to understand, compare, and relate current paths of recent HCI research as well as to analyze specific interaction designs. We believe that viewing interaction through the lens of RBI provides insights for design and uncovers gaps or opportunities for future research.