MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple users can not only see and hear but also interact with each other, grasp and manipulate objects, walk around in space, and get tactile feedback. MS2 allows walking in physical space by tracking each user’s skeleton in realtime and allows users to feel by employing passive haptics i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. To enable these elements in VR, MS2 creates a correspondence in spatial layout and object placement by building the virtual world on top of a 3D scan of the real world. Through the association between the real and virtual world, users are able to walk freely while wearing a head-mounted device, avoid obstacles like walls and furniture, and interact with people and objects. Most current virtual reality (VR) environments are designed for a single user experience where interactions with virtual objects are mediated by hand-held input devices or hand gestures. Additionally, users are only shown a representation of their hands in VR floating in front of the camera as seen from a first person perspective. We believe, representing each user as a fullbody avatar that is controlled by natural movements of the person in the real world (see Figure 1d), can greatly enhance believability and a user’s sense immersion in VR.
As humans, we are generally aware of the presence of others and since childhood are taught skills for social interaction. These include verbal and non-verbal communication, the ability to exchange physical objects, and the ability to work with others to collaborate on a task @cite_6 . Our system uses social awareness and skills by representing users's presence as full-body avatars and by making the avatars' actions visible. Additionally the environment allows for multiple co-located users to interact with each other both in the real and virtual worlds and collaborate on tasks.
@cite_6
2011348940 800808052 2788093588 1539839126
Abstract In this article we propose an account of the early development of children's knowledge about the mind and report two studies designed to test a part of it. According to this “connections-representations” account, young children begin their discovery of the mental world by learning that they and other people have internal experiences or mental states that connect them cognitively to external objects and events-experiences such as seeing them, hearing them, and wanting them. Later, they realize that the same object can be seriously (other than in pretense) mentally represented in different, seemingly contradictory ways: for example, as A in appearance, B in reality, C according to their perceptual or conceptual perspective, and D according to another person's. The results of both studies confirmed the prediction that 3-year-olds would perform well on appearance-reality and perceptual perspective-taking tasks requiring only an understanding of cognitive connections, but perform poorly on tasks requiring an understanding of seemingly-contrary-to-fact mental representations. To illustrate, children of this age had little difficulty determining that they could hear but could not see a noise-making object located on the experimenter's side of a barrier, and that the experimenter could see it (connections-level tasks). In contrast, they were largely unable to say, for instance, that a toy bear held behind a large elephant mask and emitting a cat sound looked like an elephant, sounded like a cat, and really was a toy bear (representations-level tasks)—even though the experimenter had actually told them previously what it looked like, sounded like, and really was. The article concludes with speculations about the possible origins of connections and representations knowledge and observations about the significance of these acquisitions for the child's development. The understanding of mutual people interaction is a key component for recognizing people social behavior, but it strongly relies on a personal point of view resulting difficult to be a-priori modeled. We propose the adoption of the unique head mounted cameras first person perspective (ego-vision) to promptly detect people interaction in different social contexts. The proposal relies on a complete and reliable system that extracts people's head pose combining landmarks and shape descriptors in a temporal smoothed HMM framework. Finally, interactions are detected through supervised clustering on mutual head orientation and people distances exploiting a structural learning framework that specifically adjusts the clustering measure according to a peculiar scenario. Our solution provides the flexibility to capture the interactions disregarding the number of individuals involved and their level of acquaintance in context with a variable degree of social involvement. The proposed system shows competitive performances on both publicly available ego-vision datasets and ad hoc benchmarks built with real life situations. HighlightsA head pose estimation method designed to work in ego-vision scenarios is provided.We define a 3D people localization method that works without any camera calibration.We estimate social groups with supervised correlation clustering and structural SVM.A tracking state-of-the-art evaluation applied to first person videos is provided. Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically naturalistic simulated environment in which an agent can move and interact with objects it sees, we propose a "world-model" network that learns to predict the dynamic consequences of the agent's actions. Simultaneously, we train a separate explicit "self-model" that allows the agent to track the error map of its own world-model, and then uses the self-model to adversarially challenge the developing world-model. We demonstrate that this policy causes the agent to explore novel and informative interactions with its environment, leading to the generation of a spectrum of complex behaviors, including ego-motion prediction, object attention, and object gathering. Moreover, the world-model that the agent learns supports improved performance on object dynamics prediction, detection, localization and recognition tasks. Taken together, our results are initial steps toward creating flexible autonomous agents that self-supervise in complex novel physical environments. Understanding social context is an important skill for robots that share a space with humans. In this paper, we address the problem of recognizing gender, a key piece of information when interacting with people and understanding human social relations and rules. Unlike previous work which typically considered faces or frontal body views in image data, we address the problem of recognizing gender in RGB-D data from side and back views as well. We present a large, gender-balanced, annotated, multi-perspective RGB-D dataset with full-body views of over a hundred different persons captured with both the Kinect v1 and Kinect v2 sensor. We then learn and compare several classifiers on the Kinect v2 data using a HOG baseline, two state-of-the-art deep-learning methods, and a recent tessellation-based learning approach. Originally developed for person detection in 3D data, the latter is able to learn the best selection, location and scale of a set of simple point cloud features. We show that for gender recognition, it outperforms the other approaches for both standing and walking people while being very efficient to compute with classification rates up to 150 Hz.