We describe a shared control methodology that can, without knowledge of the task, be used to improve a human's control of a dynamic system, be used as a training mechanism, and be used in conjunction with Imitation Learning to generate autonomous policies that recreate novel behaviors. Our algorithm introduces autonomy that assists the human partner by enforcing safety and stability constraints. The autonomous agent has no a priori knowledge of the desired task and therefore only adds control information when there is concern for the safety of the system. We evaluate the efficacy of our approach with a human subjects study consisting of 20 participants. We find that our shared control algorithm significantly improves the rate at which users are able to successfully execute novel behaviors. Experimental results suggest that the benefits of our safety-aware shared control algorithm also extend to the human partner's understanding of the system and their control skill. Finally, we demonstrate how a combination of our safety-aware shared control algorithm and Imitation Learning can be used to autonomously recreate the demonstrated behaviors.
For this reason, the most closely related shared control paradigms are safety, or context, aware. In this area, researchers have explored the impact of autonomous obstacle avoidance on teleoperated robots in search and rescue @cite_5 . Additionally, safety is a particular concern when the human and robot partner are co-located, such as with autonomous vehicles @cite_24 . Co-located partners are also common in assistive robotics where researchers have developed environment-aware smart walkers @cite_8 to help visually-impaired people avoid dynamic obstacles.
@cite_24 @cite_5 @cite_8
1985574396 1568126255 2061118406
Collaboration between heterogeneous robots can greatly improve the overall robot system by obtaining capabili­ties that each single robot is unable to achieve. In this paper, we present a collaborative robot system designed for search and rescue missions in an unknown environment with obstacles. The system consists of an aerial robot and a ground robot. An extended Kalman filter (EKF) is used for robot pose estimation, and an online trajectory generation algorithm is implemented for dynamic obstacle avoidance of the ground robot. The aerial robot first surveys an area of interests and sources a number of targets. The ground robot is then guided by the aerial robot to reach the target location while at the same time avoids obstacles along the way using a laser range finder. The system is entirely autonomous, achieves maximum efficiency and releases the human operator from all low-level types of operations. A centralized EKF is implemented with the flexibility of easily being modified into a distributed EKF. A major goal in robot motion planning and control is to be able to specify a task in a high-level, expressive language and have the robot(s) to automatically convert the specification into a set of low-level primitives, such as feedback controllers and communication protocols, to accomplish the task. The robots can vary from manipulator arms used in manufacturing or surgery, to autonomous vehicles used in search and rescue or in planetary exploration, and to smart wheel chairs for disabled people. They are subject to mechanical constraints (e.g., a carlike robot cannot move sideways,an airplane cannot stop in place) and have limited computation, sensing, and communication capabilities. The environments can be cluttered with possibly moving and shape-changing obstacles and can con tain dynamic (moving, appearing, or disappearing) targets. One of the major challenges in this area is the development of a computationally efficient frame work accommodating both the robot constraints and the complexity of the environment, while, at the same time, allowing for a large spectrum of task specifications. This paper presents a new adaptive servo-level shared control scheme for a mobile assistive robot that aims at assisting senior and disabled people to transport heavy objects in a complex environment with obstacles. Several technical problems and challenges related to the assistive robotic system and the shared controller are addressed. Specifically, a nonlinear tracking controller is developed for the robot to follow the user. An obstacle avoidance controller is developed based on the deformable virtual zone principle for the robot to avoid obstacles. The adaptive servo-level shared controller utilizes the tracking controller and the obstacle avoidance controller’s outputs to generate a new shared control output to command the robot. Experiments show that the user can guide the movement of the robot safely and smoothly in the complex environment with the developed controllers. Current applications of mobile robots in urban search and rescue (USAR) environments require a human operator in the loop to help guide the robot remotely. Although human operation can be effective, the unknown cluttered nature of the environments make robot navigation and victim identification highly challenging. Operators can become stressed and fatigued very quickly due to a loss of situational awareness, leading to the robots getting stuck and not being able to find victims in the scene during this time-sensitive operation. In addition, current autonomous robots are not capable of traversing these complex unpredictable environments. To address this challenge, a balance between the level of autonomy of the robot and the amount of human control over the robot needs to be addressed. In this paper, we present a unique control architecture for semi-autonomous navigation of a robotic platform utilizing sensory information provided by a novel real-time 3D mapping sensor. The control system provides the robot with the ability to learn and make decisions regarding which rescue tasks should be carried out at a given time and whether an autonomous robot or a human controlled robot can perform these tasks more efficiently without compromising the safety of the victims, rescue workers and the rescue robot. Preliminary experiments were conducted to evaluate the performance of the proposed collaborative control approach for a USAR robot in an unknown cluttered environment. Autonomous driving is likely to be the heart of urban transportation in the future. Autonomous vehicles have the potential to increase the safety of passengers and also to make road trips shorter and more enjoyable. As the first steps toward these goals, many car manufacturers are investing in designing and equipping their vehicles with advanced driver-assist systems. Road intersections are considered to be serious bottlenecks of urban transportation, as more than 44 of all reported crashes in U.S. occur within intersection areas which in turn lead to 8,500 fatalities and approximately 1 million injuries every year. Furthermore, the impact of road intersections on traffic delays leads to enormous waste of human and natural resources. In this paper, we therefore focus on intersection management in Intelligent Transportation Systems (ITS) research. In the future, when dealing with autonomous vehicles, it is critical to address safety and throughput concerns that arise from autonomous driving through intersections and roundabouts. Our goal is to provide vehicles with a safe and efficient passage method through intersections and roundabouts. We have been investigating vehicle-to-vehicle (V2V) communications as a part of co-operative driving in the context of autonomous driving. We have designed and developed efficient and reliable intersection protocols to avoid vehicle collisions at intersections and increase traffic throughput. In this paper, we introduce new V2V intersection protocols to achieve the above goals. We show that, in addition to intersections, these protocols are also applicable to vehicle crossings at roundabouts. Additionally, we study the effects of position inaccuracy of commonly-used GPS devices on some of our V2V intersection protocols and suggest required modifications to guarantee their safety and efficiency despite these impairments. Our simulation results show that we are able to avoid collisions and also increase the throughput of the intersections up to 87.82 compared to common traffic-light signalized intersections. Robots that solve complex tasks in environments too dangerous for humans to enter are desperately needed, e.g. for search and rescue applications. We describe our mobile manipulation robot Momaro, with which we participated successfully in the DARPA Robotics Challenge. It features a unique locomotion design with four legs ending in steerable wheels, which allows it both to drive omnidirectionally and to step over obstacles or climb. Furthermore, we present advanced communication and teleoperation approaches, which include immersive 3D visualization, and 6D tracking of operator head and arm motions. The proposed system is evaluated in the DARPA Robotics Challenge, the DLR SpaceBot Camp 2015, and lab experiments. We also discuss the lessons learned from the competitions and present initial steps towards autonomous operator assistance functions. Autonomous mobile robots are required to operate in partially known and unstructured environments. It is imperative to guarantee safety of such systems for their successful deployment. Current state of the art does not fully exploit the sensor and dynamic capabilities of a robot. Also, given the non-holonomic systems with non-linear dynamic constraints, it becomes computationally infeasible to find an optimal solution if the full dynamics are to be exploited online. In this paper we present an online algorithm to guarantee the safety of the robot through an emergency maneuver library. The maneuvers in the emergency maneuver library are optimized such that the probability of finding an emergency maneuver that lies in the known obstacle free space is maximized. We prove that the related trajectory set diversity problem is monotonic and sub-modular which enables one to develop an efficient trajectory set generation algorithm with bounded sub-optimality. We generate an off-line computed trajectory set that exploits the full dynamics of the robot and the known obstacle-free region. We test and validate the algorithm on a full-size autonomous helicopter flying up to speeds of 56m s in partially-known environments. We present results from 4 months of flight testing where the helicopter has been avoiding trees, performing autonomous landing, avoiding mountains while being guaranteed safe. Despite the stable walking capabilities of modern biped humanoid robots, their ability to autonomously and safely navigate obstacle-filled, unpredictable environments has so far been limited. We present an approach to autonomous humanoid walking that combines vision-based sensing with a footstep planner, allowing the robot to navigate toward a desired goal position while avoiding obstacles. An environment map including the robot, goal, and obstacle locations is built in real-time from vision. The footstep planner then computes an optimal sequence of footstep locations within a time-limited planning horizon. Footstep plans are reused and only partially recomputed as the environment changes during the walking sequence. In our experiments, combining real-time vision with plan reuse has allowed a Honda ASIMO humanoid robot to autonomously traverse dynamic environments containing unpredictably moving obstacles Tele-operated systems allow humans to extend their physical capabilities and enable them to intervene in hazardous operations or where their presence is not possible. However, the operation of such systems over long periods has proved to be difficult and stressful. Consequently, means to facilitate their use are the subject of much study and experimental work. In this paper, we propose a collaborative-shared control strategy that combines the operator abilities with robotic-based tasks to render these systems more flexible and robust. In our method, the collaborative control component is responsible for allowing operator intervention when the robot is facing complex situations, whilst the shared control component provides an automatic control mechanism to assist and to monitor-correct irrational operator actions. The paper demonstrates how collaborative and shared control strategies work together to facilitate the teleoperated control of a mobile platform in a cluttered environment. The experimental results include applications to surveillance and to search & rescue operations. In addition, a key component in the form of a hybrid obstacle avoidance module is introduced that allows the robot to be guided on a task basis by the operator at a distance. In this thesis we present a technique for the composition of robot control laws in dynamical environments. We propose a challenging robotic task, called Dynamical Pick and Place, in which a robot equipped with merely a soft paddle must capture and contain a ball, safely negotiate it past obstacles, and bring it to rest at a desired location. We develop a composition technique for local controllers that provides a formal guarantee of the stability of the switching behavior required in this task, and provide descriptive statistics of a working implementation. Our robotic system displays unusually dexterous behavior in the face of significant system noise, and recovers gracefully from large unexpected perturbations caused by the experimenters. Our approach to controller composition makes use of the funnel as a metaphor for asymptotic stability, is motivated by the pre-image backchaining techniques developed by Lozano-Perez, Mason and Taylor, and extends their ideas from quasi-static environments to systems with full dynamics. We introduce the concepts of "dynamical obstacle avoidance" and "dynamical safety" for systems with only intermittent control of their environment, and show that it is important not only that the system avoid obstacles directly, but also that the system will never reach an obstacle before getting another chance to effect control. The Dynamical Pick and Place problem addressed by this thesis is a difficult control problem, but an easy planning problem. The system we develop provides a way to engage more powerful AI planning tools without sacrificing access to the stability arguments of dynamical systems theory. We introduce a hybrid algorithm for the self-semantic location and autonomous navigation of robots using entropy-based vision and visual topological maps. In visual topological maps the visual landmarks are considered as leave points for guiding the robot to reach a target point (robot homing) in indoor environments. These visual landmarks are defined from images of relevant objects or characteristic scenes in the environment. The entropy of an image is directly related to the presence of a unique object or the presence of several different objects inside it: the lower the entropy the higher the probability of containing a single object inside it and, conversely, the higher the entropy the higher the probability of containing several objects inside it. Consequently, we propose the use of the entropy of images captured by the robot not only for the landmark searching and detection but also for obstacle avoidance. If the detected object corresponds to a landmark, the robot uses the suggestions stored in the visual topological map to reach the next landmark or to finish the mission. Otherwise, the robot considers the object as an obstacle and starts a collision avoidance maneuver. In order to validate the proposal we have defined an experimental framework in which the visual bug algorithm is used by an Unmanned Aerial Vehicle (UAV) in typical indoor navigation tasks. Tracked vehicles are frequently used as search-and-rescue robots for exploring disaster areas. To enhance their ability to traverse rough terrain, some of these robots are equipped with swingable subtracks. However, manual control of such subtracks also increases the operator's workload, particularly in teleoperation with limited camera views. To eliminate this trade-off, we have developed a shared autonomy system using an autonomous controller for subtracks that is based on continuous three-dimensional terrain scanning. Using this system, the operator has only to specify the direction of travel to the robot, following which the robot traverses rough terrain using autonomously generated subtrack motions. In our system, real-time terrain slices near the robot are obtained using two or three LIDAR (laser imaging detection and ranging) sensors, and these terrain slices are integrated to generate three-dimensional terrain information. In this paper, we introduce an autonomous controller for subtracks and validate the reliability of a shared autonomy system on actual rough terrains through experimental results. © 2011 Wiley Periodicals, Inc. Operators' intelligent and skillful decisions are necessary for the teleoperation of a mobile robot when there are many scattered obstacles. Among the sensors used for environment recognition, the camera is the most popular and powerful. However, there are several limitations in the camera-based teleoperation of a mobile robot. For example, shadowed and curved areas cannot be viewed using a narrow view-angle camera, especially in an environment with bad illumination and several obstacles. Therefore, it is necessary to have other sensory information for reliable teleoperations. In this study, 16 ultrasonic sensors are attached around a mobile robot in a ring pattern to measure the distances to the obstacles and a collision vector is introduced as a new tool for obstacle avoidance, which is defined as the normal vector from an obstacle to the mobile robot. Based on this collision vector, a virtual reflection force is generated to avoid the obstacles and then the reflection force is transferred to the operator who is holding the joystick used to control the mobile robot. Based on this reflection force, the operator can control the mobile robot more smoothly and safely. For this bidirectional teleoperation, a master joystick system using a two-axis hall sensor was designed to eliminate the nonlinear region, which exists in a general joystick with two motors and potentiometers. The effectiveness of the collision vector and force-reflection joystick is verified by comparing two vision-based teleoperation experiments, with and without force reflection. Safe autonomous navigation of microair vehicles in cluttered dynamic environments is challenging due to the uncertainties arising from robot localization, sensing, and motion disturbances. This letter presents a probabilistic collision avoidance method for navigation among other robots and moving obstacles, such as humans. The approach explicitly considers the collision probability between each robot and obstacle and formulates a chance constrained nonlinear model predictive control problem (CCNMPC). A tight bound for approximation of collision probability is developed, which makes the CCNMPC formulation tractable and solvable in real time. For multirobot coordination, we describe three approaches, one distributed without communication (constant velocity assumption), one distributed with communication (of previous plans), and one centralized (sequential planning). We evaluate the proposed method in experiments with two quadrotors sharing the space with two humans and verify the multirobot coordination strategy in simulation with up to sixteen quadrotors. This paper presents a motion control design for a walking-assistant robot by combining passive-compliant behavior and active obstacle avoidance. The proposed method recognizes the intended motion for a user using gait movement and the applied force of a hand. A passive-compliant motion control is developed that allows the user to safely walk with the robot. The obstacle avoidance controller actively controls the velocity to guide the user in a direction that avoids collisions. A shared-control scheme combines active obstacle avoidance and passive-compliant motion commands. The experimental results verify the effectiveness of the proposed method and show that the robot allows a user to walk safely in a complex environment. A questionnaire survey of elderly users shows that the proposed design for a walking-assistant robot gives satisfactory performance. Accurate localization of a vehicle is a fundamental challenge and one of the most important tasks of mobile robots. For autonomous navigation, motion tracking, and obstacle detection and avoidance, a robot must maintain knowledge of its position over time. Vision-based odometry is a robust technique utilized for this purpose. It allows a vehicle to localize itself robustly by using only a stream of images captured by a camera attached to the vehicle. This paper presents a review of state-of-the-art visual odometry (VO) and its types, approaches, applications, and challenges. VO is compared with the most common localization sensors and techniques, such as inertial navigation systems, global positioning systems, and laser sensors. Several areas for future research are also highlighted. For many robotics applications (such as robotic search and rescue), information about the environment must be gathered by a team of robots and returned to a single, specific location. Coordination of robots and sharing of information is vital, and when environments have severe communication limitations, approaches must be robust to communication drop-out and failure. The difficulties are compounded in dynamic environments, where paths previously believed to be free can suddenly become blocked. In this paper, we introduce a novel way of calculating rendezvous points for robots to meet and share information. Using role-based exploration, some robots continuously explore the environment while others ferry information back and forth to a central command centre. Optimal rendezvous point selection leads to more efficient exploration, and allows robots to replan when one of them has unexpected obstacles in its path. We address the problem of learning vision-based, collision-avoiding, and target-selecting controllers in 3D, specifically in underwater environments densely populated with coral reefs. Using a highly maneuverable, dynamic, six-legged (or flippered) vehicle to swim underwater, we exploit real time visual feedback to make close-range navigation decisions that would be hard to achieve with other sensors. Our approach uses computer vision as the sole mechanism for both collision avoidance and visual target selection. In particular, we seek to swim close to the reef to make observations while avoiding both collisions and barren, coral-deprived regions. To carry out path selection while avoiding collisions, we use monocular image data processed in real time. The proposed system uses a convolutional neural network that takes an image from a forward-facing camera as input and predicts unscaled and relative path changes. The network is trained to encode our desired obstacle-avoidance and reef-exploration objectives via supervised learning from human-labeled data. The predictions from the network are transformed into absolute path changes via a combination of a temporally-smoothed proportional controller for heading targets and a low-level motor controller. This system enables safe and autonomous coral reef navigation in underwater environments. We validate our approach using an untethered and fully autonomous robot swimming through coral reef in the open ocean. Our robot successfully traverses 1000 m of the ocean floor collision-free while collecting close-up footage of coral reefs. Robotic systems have been widely used in many areas to assist human beings. Mobile manipulators are among the most popular choices. This paper investigates human assistance systems using a mobile manipulator, for example, to guide the blind and to transport objects. Distinct from existing systems, an integrated dynamic model and controller of the mobile manipulator are designed. Singularity, manipulability and safety are all considered in the system design. Furthermore, two human assistance modes - Robot-Human mode and Teleoperator- Robot-Human mode - are designed and analysed. The Teleoperator-Robot-Human mode can integrate human intelligence into the assistance system to further enhance the system efficiency and safety. The experimental results implemented on a mobile manipulator demonstrated the effectiveness of the designed systems. "Robotic Mapping and Exploration" is an important contribution in the area of simultaneous localization and mapping (SLAM) for autonomous robots, which has been receiving a great deal of attention by the research community in the latest few years. The contents are focused on the autonomous mapping learning problem. Solutions include uncertainty-driven exploration, active loop closing, coordination of multiple robots, learning and incorporating background knowledge, and dealing with dynamic environments. Results are accompanied by a rich set of experiments, revealing a promising outlook toward the application to a wide range of mobile robots and field settings, such as search and rescue, transportation tasks, or automated vacuum cleaning.