We have designed and implemented an application running inside Second Life that supports user annotation of graphical objects and graphical visualization of concept ontologies, thus providing a formal, machine-accessible description of objects. As a result, we offer a platform that combines the graphical knowledge representation that is expected from a MUVE artifact with the semantic structure given by the Resource Framework Description (RDF) representation of information.
Since its appearance Second Life attracted a lot of attention from users and several research efforts have aimed at coupling virtual environment with semantic web applications running on web servers external to SL. Karakatsiotis et @cite_18 @cite_0 developed a robotic avatar (perhaps better termed a bot) which follows a virtual visitor in her tour of the virtual museum. Whenever the visitor touches an exhibit the bot produces a textual description developed in NaturalOwl, a Natural Language Generation engine developed in Java. A more scalable version of this work is described in @cite_10 .
@cite_0 @cite_18 @cite_10
2128335999 2061309618 2123564449 2896182651
We describe initial work on building a virtual gallery, within Second Life, which can automatically tailor itself to an individual visitor, responding to their abilities, interests, preferences or history of interaction. The description of an object in the virtual world can be personalised to suit the beginner or the expert, varying how it is said—via the choice of language (such as English or Greek), the words, or the complexity of sentences—as well as what is said—by taking into account what else has been seen or described already. The guide delivering the descriptions can remain disembodied, or be embodied as a robotic avatar. This paper reports on a field trial with interactive humanoid robots at a science museum where visitors are supposed to study and develop an interest in science. In the trial, each visitor wore an RFID tag while looking around the museum's exhibits. Information obtained from the RFID tags was used to direct the robots' interaction with the visitors. The robots autonomously interacted with visitors via gestures and utterances resembling the free play of children [1]. In addition, they performed exhibit-guiding by moving around several exhibits and explaining the exhibits based on sensor information. The robots were highly evaluated by visitors during the two-month trial. Moreover, we conducted an experiment in the field trial to compare the detailed effects of exhibit-guiding and free-play interaction under three operating conditions. This revealed that the combination of the free-play interaction and exhibit-guiding positively affected visitors' experiences at the science museum. This paper describes the software architecture of an autonomous tour-guideltutor robot. This robot was recently deployed in the "Deutsches Museum Bonn," were it guided hundreds of visitors through the museum during a six-day deployment period. The robot's control software integrates low-level probabilistic reasoning with high-level problem solving embedded in first order logic. A collection of software innovations, described in this paper, enabled the robot to navigate at high speeds through dense crowds, while reliably avoiding collisions with obstacles--some of which could not even be perceived. Also described in this paper is a user interface tailored towards non-expert users, which was essential for the robot's success in the museum. Based on these experiences, this paper argues that time is ripe for the development of AI-based commercial service robots that assist people in everyday life. Data-driven algorithms have surpassed traditional techniques in almost every aspect in robotic vision problems. Such algorithms need vast amounts of quality data to be able to work properly after their training process. Gathering and annotating that sheer amount of data in the real world is a time-consuming and error-prone task. Those problems limit scale and quality. Synthetic data generation has become increasingly popular since it is faster to generate and automatic to annotate. However, most of the current datasets and environments lack realism, interactions, and details from the real world. UnrealROX is an environment built over Unreal Engine 4 which aims to reduce that reality gap by leveraging hyperrealistic indoor scenes that are explored by robot agents which also interact with objects in a visually realistic manner in that simulated world. Photorealistic scenes and robots are rendered by Unreal Engine into a virtual reality headset which captures gaze so that a human operator can move the robot and use controllers for the robotic hands; scene information is dumped on a per-frame basis so that it can be reproduced offline to generate raw data and ground truth annotations. This virtual reality environment enables robotic vision researchers to generate realistic and visually plausible data with full ground truth for a wide variety of problems such as class and instance semantic segmentation, object detection, depth estimation, visual grasping, and navigation.