Building large-scale, globally consistent maps is a challenging problem, made more difficult in environments with limited access, sparse features, or when using data collected by novice users. For such scenarios, where state-of-the-art mapping algorithms produce globally inconsistent maps, we introduce a systematic approach to incorporating sparse human corrections, which we term Human-in-the-Loop Simultaneous Localization and Mapping (HitL-SLAM). Given an initial factor graph for pose graph SLAM, HitL-SLAM accepts approximate, potentially erroneous, and rank-deficient human input, infers the intended correction via expectation maximization (EM), back-propagates the extracted corrections over the pose graph, and finally jointly optimizes the factor graph including the human inputs as human correction factor terms, to yield globally consistent large-scale maps. We thus contribute an EM formulation for inferring potentially rank-deficient human corrections to mapping, and human correction factor extensions to the factor graphs for pose graph SLAM that result in a principled approach to joint optimization of the pose graph while simultaneously accounting for multiple forms of human correction. We present empirical results showing the effectiveness of HitL-SLAM at generating globally accurate and consistent maps even when given poor initial estimates of the map.
Rescue robotics has been suggested by a recent DARPA NSF study as an application domain for the research in human-robot integration (HRI). This paper provides a short tutorial on how robots are currently used in urban search and rescue (USAR) and discusses the HRI issues encountered over the past eight years. A domain theory of the search activity is formulated. The domain theory consists of two parts: 1) a workflow model identifying the major tasks, actions, and roles in robot-assisted search (e.g., a workflow model) and 2) a general information flow model of how data from the robot is fused by various team members into information and knowledge. The information flow model also captures the types of situation awareness needed by each agent in the rescue robot system. The article presents a synopsis of the major HRI issues in reducing the number of humans it takes to control a robot, maintaining performance with geographically distributed teams with intermittent communications, and encouraging acceptance within the existing social structure.  This work establishes an architecture for Urban Search and Rescue and a methodology for mixing real-world and simulation-based testing. A sensor suite and sensor fusion algorithm for robust victim detection permits aggregation of sensor readings from various sensors on multiple robots. We have embarked on a research program focusing on the enabling technologies of effective USAR robotic rescue devices. The program is also researching system-level design, evaluation, and refinement of USAR rescue architectures that include teams of sensor-laden robots and human rescuers. In this paper, we present highlights from our research, which include our multiagent system (MAS) infrastructure, our simulation environment, and our approach to sensor fusion and interface design for effective robotic control. Abstract : Advanced command and control (C2) systems such as the U.S. Army's Future Combat Systems (FCS) will increasingly use more flexible, reconfigurable components, including numerous robotic (unmanned) air and ground vehicles. Human operators will be involved in supervisory control of unmanned vehicles (UVs) with the need for occasional manual intervention. This paper discusses the design of automation support in C2 systems with multiple UVs. Following a model of effective human-automation interaction design, the authors propose that operators can best be supported by high-level automation of information acquisition and analysis functions. Automation of decision-making functions, on the other hand, should be set at a moderate level, unless 100 percent reliability can be assured. The use of adaptive automation support technologies also is discussed. They present a framework for adaptive and adaptable processes as methods that can enhance human-system performance while avoiding some of the common pitfalls of "static" automation such as over-reliance, skill degradation, and reduced situational awareness. Adaptive automation invocation processes are based on critical mission events, operator modeling, and real-time operator performance and physiological assessment, or hybrid combinations of these methods. They describe the results of human-in-the-loop experiments involving human operator supervision of multiple UVs under multi-task conditions in simulations of reconnaissance missions. The results support the use of adaptive automation to enhance human-system performance in supervision of multiple UVs, balance operator workload, and enhance situational awareness. Implications for the design and fielding of adaptive automation architectures for C2 systems involving UVs are discussed. In this paper, we suggest a new method of human augmented mapping for indoor environments using only a stereo camera. Through user's help, a robot with a stereo camera can investigate the environment without failure and even more efficiently. Moreover, the user can share the information about the environment with the robot and add semantic information to the environmental map. We employ PCA features for visual landmarks and a hybrid map for map representation. Particularly, we define two types of nodes, U R-nodes and divide the map building into three processes, User's Guidance, Robot's Map Revision, and Robot's Map Completion. We implemented a human augmented mapping system with a stereo camera and demonstrated it in rectangular-shaped corridors. From the comparison with a manually-built map, we showed the feasibility of the environmental map generated by our proposed method. Current applications of mobile robots in urban search and rescue (USAR) environments require a human operator in the loop to help guide the robot remotely. Although human operation can be effective, the unknown cluttered nature of the environments make robot navigation and victim identification highly challenging. Operators can become stressed and fatigued very quickly due to a loss of situational awareness, leading to the robots getting stuck and not being able to find victims in the scene during this time-sensitive operation. In addition, current autonomous robots are not capable of traversing these complex unpredictable environments. To address this challenge, a balance between the level of autonomy of the robot and the amount of human control over the robot needs to be addressed. In this paper, we present a unique control architecture for semi-autonomous navigation of a robotic platform utilizing sensory information provided by a novel real-time 3D mapping sensor. The control system provides the robot with the ability to learn and make decisions regarding which rescue tasks should be carried out at a given time and whether an autonomous robot or a human controlled robot can perform these tasks more efficiently without compromising the safety of the victims, rescue workers and the rescue robot. Preliminary experiments were conducted to evaluate the performance of the proposed collaborative control approach for a USAR robot in an unknown cluttered environment. The MAGIC 2010 robot competition showed how well multi-robot teams can work with human teams in urban search.