We describe a shared control methodology that can, without knowledge of the task, be used to improve a human's control of a dynamic system, be used as a training mechanism, and be used in conjunction with Imitation Learning to generate autonomous policies that recreate novel behaviors. Our algorithm introduces autonomy that assists the human partner by enforcing safety and stability constraints. The autonomous agent has no a priori knowledge of the desired task and therefore only adds control information when there is concern for the safety of the system. We evaluate the efficacy of our approach with a human subjects study consisting of 20 participants. We find that our shared control algorithm significantly improves the rate at which users are able to successfully execute novel behaviors. Experimental results suggest that the benefits of our safety-aware shared control algorithm also extend to the human partner's understanding of the system and their control skill. Finally, we demonstrate how a combination of our safety-aware shared control algorithm and Imitation Learning can be used to autonomously recreate the demonstrated behaviors.
For this reason, the most closely related shared control paradigms are safety, or context, aware. In this area, researchers have explored the impact of autonomous obstacle avoidance on teleoperated robots in search and rescue @cite_5 . Additionally, safety is a particular concern when the human and robot partner are co-located, such as with autonomous vehicles @cite_24 . Co-located partners are also common in assistive robotics where researchers have developed environment-aware smart walkers @cite_8 to help visually-impaired people avoid dynamic obstacles.
@cite_24 @cite_5 @cite_8
2909324603 2588104504 1568126255 2140542425 2116113712 1964918420 2588802774 2513313098 6827933 2061118406 861352110 638212880 2056600097 2555647367 1995782858 2071361688 615006660 2103120971 2594336441 2153348469
Safe autonomous navigation of microair vehicles in cluttered dynamic environments is challenging due to the uncertainties arising from robot localization, sensing, and motion disturbances. This letter presents a probabilistic collision avoidance method for navigation among other robots and moving obstacles, such as humans. The approach explicitly considers the collision probability between each robot and obstacle and formulates a chance constrained nonlinear model predictive control problem (CCNMPC). A tight bound for approximation of collision probability is developed, which makes the CCNMPC formulation tractable and solvable in real time. For multirobot coordination, we describe three approaches, one distributed without communication (constant velocity assumption), one distributed with communication (of previous plans), and one centralized (sequential planning). We evaluate the proposed method in experiments with two quadrotors sharing the space with two humans and verify the multirobot coordination strategy in simulation with up to sixteen quadrotors. Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain---Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operators capabilities and their perception of control authority. Additionally, a custom servo controller design allow for safe interactions of the robotic arm with the environment. We present experimental results demonstrating significant performance improvement using our shared-control assistance framework on adapted rehabilitation benchmarks with two subjects at various timepoints relative to their implantation with intracortical BCIs. Our results indicate that shared assistance mitigates perceived user difficulty in using a seven-degree of freedom robotic arm as a prosthetic and enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with objects previously unknown to the system in densely cluttered environments. Tele-operated systems allow humans to extend their physical capabilities and enable them to intervene in hazardous operations or where their presence is not possible. However, the operation of such systems over long periods has proved to be difficult and stressful. Consequently, means to facilitate their use are the subject of much study and experimental work. In this paper, we propose a collaborative-shared control strategy that combines the operator abilities with robotic-based tasks to render these systems more flexible and robust. In our method, the collaborative control component is responsible for allowing operator intervention when the robot is facing complex situations, whilst the shared control component provides an automatic control mechanism to assist and to monitor-correct irrational operator actions. The paper demonstrates how collaborative and shared control strategies work together to facilitate the teleoperated control of a mobile platform in a cluttered environment. The experimental results include applications to surveillance and to search & rescue operations. In addition, a key component in the form of a hybrid obstacle avoidance module is introduced that allows the robot to be guided on a task basis by the operator at a distance. We consider the problem of navigating a mobile robot through dense human crowds. We begin by exploring a fundamental impediment to classical motion planning algorithms called the âfreezing robot problemâ: once the environment surpasses a certain level of dynamic complexity, the planner decides that all forward paths are unsafe, and the robot freezes in place or performs unnecessary maneuvers to avoid collisions. We argue that this problem can be avoided if the robot anticipates human cooperation, and accordingly we develop interacting Gaussian processes, a prediction density that captures cooperative collision avoidance, and a âmultiple goalâ extension that models the goal-driven nature of human decision making. We validate this model with an empirical study of robot navigation in dense human crowds 488 runs, specifically testing how cooperation models effect navigation performance. The multiple goal interacting Gaussian processes algorithm performs comparably with human teleoperators in crowd densities nearing 0.8 humans m2, while a state-of-the-art non-cooperative planner exhibits unsafe behavior more than three times as often as the multiple goal extension, and twice as often as the basic interacting Gaussian process approach. Furthermore, a reactive planner based on the widely used dynamic window approach proves insufficient for crowd densities above 0.55 people m2. We also show that our non-cooperative planner or our reactive planner capture the salient characteristics of nearly any dynamic navigation algorithm. Based on these experimental results and theoretical observations, we conclude that a cooperation model is critical for safe and efficient robot navigation in dense human crowds. In this paper a real-time collision avoidance approach is presented for safe human-robot coexistence. The main contribution is a fast method to evaluate distances between the robot and possibly moving obstacles (including humans), based on the concept of depth space. The distances are used to generate repulsive vectors that are used to control the robot while executing a generic motion task. The repulsive vectors can also take advantage of an estimation of the obstacle velocity. In order to preserve the execution of a Cartesian task with a redundant manipulator, a simple collision avoidance algorithm has been implemented where different reaction behaviors are set up for the end-effector and for other control points along the robot structure. The complete collision avoidance framework, from perception of the environment to joint-level robot control, is presented for a 7-dof KUKA Light-Weight-Robot IV using the Microsoft Kinect sensor. Experimental results are reported for dynamic environments with obstacles and a human. Current applications of mobile robots in urban search and rescue (USAR) environments require a human operator in the loop to help guide the robot remotely. Although human operation can be effective, the unknown cluttered nature of the environments make robot navigation and victim identification highly challenging. Operators can become stressed and fatigued very quickly due to a loss of situational awareness, leading to the robots getting stuck and not being able to find victims in the scene during this time-sensitive operation. In addition, current autonomous robots are not capable of traversing these complex unpredictable environments. To address this challenge, a balance between the level of autonomy of the robot and the amount of human control over the robot needs to be addressed. In this paper, we present a unique control architecture for semi-autonomous navigation of a robotic platform utilizing sensory information provided by a novel real-time 3D mapping sensor. The control system provides the robot with the ability to learn and make decisions regarding which rescue tasks should be carried out at a given time and whether an autonomous robot or a human controlled robot can perform these tasks more efficiently without compromising the safety of the victims, rescue workers and the rescue robot. Preliminary experiments were conducted to evaluate the performance of the proposed collaborative control approach for a USAR robot in an unknown cluttered environment. This paper presents safety barrier certificates that ensure scalable and provably collision-free behaviors in multirobot systems by modifying the nominal controllers to formally satisfy safety constraints. This is achieved by minimizing the difference between the actual and the nominal controllers subject to safety constraints. The resulting computation of the safety controllers is done through a quadratic programming problem that can be solved in real-time and in this paper, we describe a series of problems of increasing complexity. Starting with a centralized formulation, where the safety controller is computed across all agents simultaneously, we show how one can achieve a natural decentralization whereby individual robots only have to remain safe relative to nearby robots. Conservativeness and existence of solutions as well as deadlock-avoidance are then addressed using a mixture of relaxed control barrier functions, hybrid braking controllers, and consistent perturbations. The resulting control strategy is verified experimentally on a collection of wheeled mobile robots whose nominal controllers are explicitly designed to make the robots collide. This paper studies how an operator with limited situational awareness can collaborate with a swarm of simulated robots. The robots are distributed in an environment with wall obstructions. They aggregate autonomously but are unable to form a single cluster due to the obstructions. The operator lacks the bird’s-eye perspective, but can interact with one robot at a time, and influence the behavior of other nearby robots. We conducted a series of experiments. They show that untrained participants had marginal influence on the performance of the swarm. Expert participants succeeded in aggregating 85 of the robots while untrained participants, with bird’s-eye view, succeeded in aggregating 90 . This demonstrates that the controls are sufficient for operators to aid the autonomous robots in the completion of the task and that lack of situational awareness is the main difficulty. An analysis of behavioral differences reveals that trained operators learned to gain superior situational awareness. This paper describes a multi-robot collision avoidance system based on the velocity obstacle paradigm. In contrast to previous approaches, we alleviate the strong requirement for perfect sensing (i.e. global positioning) using Adaptive Monte-Carlo Localization on a per-agent level. While such methods as Optimal Reciprocal Collision Avoidance guarantee local collision-free motion for a large number of robots, given perfect knowledge of positions and speeds, a realistic implementation requires further extensions to deal with inaccurate localization and message passing delays. The presented algorithm bounds the error introduced by localization and combines the computation for collision-free motion with localization uncertainty. We provide an open source implementation using the Robot Operating System (ROS). The system is tested and evaluated with up to eight robots in simulation and on four differential drive robots in a real-world situation. This paper describes the use of a Bayesian network to provide context-aware shared control of a robot mobility aid for the frail blind. The robot mobility aid, PAM-AID, is a “smart walker” that aims to assist the frail and elderly blind to walk safely indoors. The Bayesian network combines user input with high-level information derived from the sensors to provide a context-aware estimate of the user’s current navigation goals. This context-aware action selection mechanism facilitates the use of a very simple, low bandwidth user interface, which is critical for the elderly user group. The PAM-AID systems have been evaluated through a series of field trails involving over 30 potential users. All currently used mobile robot platforms are able to navigate safely through their environment, avoiding static and dynamic obstacles. However, in human populated environments mere obstacle avoidance is not sufficient to make humans feel comfortable and safe around robots. To this end, a large community is currently producing human-aware navigation approaches to create a more socially acceptable robot behaviour. Amajorbuilding block for all Human-Robot Spatial Interaction is the ability of detecting and tracking humans in the vicinity of the robot. We present a fully integrated people perception framework, designed to run in real-time on a mobile robot. This framework employs detectors based on laser and RGB-D data and a tracking approach able to fuse multiple detectors using different versions of data association and Kalman filtering. The resulting trajectories are transformed into Qualitative Spatial Relations based on a Qualitative Trajectory Calculus, to learn and classify different encounters using a Hidden Markov Model based representation. We present this perception pipeline, which is fully implemented into the Robot Operating System (ROS), in a small proof of concept experiment. All components are readily available for download, and free to use under the MIT license, to researchers in all fields, especially focussing on social interaction learning by providing different kinds of output, i.e. Qualitative Relations and trajectories. Directing robot attention to recognise activities and to anticipate events like goal-directed actions is a crucial skill for human-robot interaction. Unfortunately, issues like intrinsic time constraints, the spatially distributed nature of the entailed information sources, and the existence of a multitude of unobservable states affecting the system, like latent intentions, have long rendered achievement of such skills a rather elusive goal. The problem tests the limits of current attention control systems. It requires an integrated solution for tracking, exploration and recognition, which traditionally have been seen as separate problems in active vision. We propose a probabilistic generative framework based on information gain maximisation and a mixture of Kalman Filters that uses predictions in both recognition and attention-control. This framework can efficiently use the observations of one element in a dynamic environment to provide information on other elements, and consequently enables guided exploration. Interestingly, the sensors control policy, directly derived from first principles, represents the intuitive trade-off between finding the most discriminative clues and maintaining overall awareness. Experiments on a simulated humanoid robot observing a human executing goal-oriented actions demonstrated improvement on recognition time and precision over baseline systems. This work extends the framework of bilateral shared control of mobile robots with the aim of increasing the robot autonomy and decreasing the operator commitment. We consider persistent autonomous behaviors where a cyclic motion must be executed by the robot. The human operator is in charge of modifying online some geometric properties of the desired path. This is then autonomously processed by the robot in order to produce an actual path guaranteeing: i) tracking feasibility, ii) collision avoidance with obstacles, iii) closeness to the desired path set by the human operator, and iv) proximity to some points of interest. A force feedback is implemented to inform the human operator of the global deformation of the path rather than using the classical mismatch between desired and executed motion commands. Physically-based simulations, with human hardware-in-the-loop and a quadrotor UAV as robotic platform, demonstrate the feasibility of the method. Trained human co-workers can often easily predict each other's intentions based on prior experience. When collaborating with a robot coworker, however, intentions are hard or impossible to infer. This difficulty of mental introspection makes human-robot collaboration challenging and can lead to dangerous misunderstandings. In this paper, we present a novel, object-aware projection technique that allows robots to visualize task information and intentions on physical objects in the environment. The approach uses modern object tracking methods in order to display information at specific spatial locations taking into account the pose and shape of surrounding objects. As a result, a human co-worker can be informed in a timely manner about the safety of the workspace, the site of next robot manipulation tasks, and next subtasks to perform. A preliminary usability study compares the approach to collaboration approaches based on monitors and printed text. The study indicates that, on average, the user effectiveness and satisfaction is higher with the projection based approach. When humans and mobile robots share the same space, one of their challenges is to navigate around each other and manage their mutual navigational intents. While humans have developed excellent skills in inferring their counterpart's intentions via a number of implicit and non-verbal cues, making navigation also in crowds an ease, this kind of effective and efficient communication often falls short in human-robot encounters. In this paper, two alternative approaches to convey navigational intent of a mobile robot to humans in a shared environment are proposed and analysed. The first is utilising anthropomorphic features of the mobile robot to realise an implicit joint attention using gaze to represent the direction of navigational intent. In the second approach, a more technical design adopting the semantics of car's turn indicators, has been implemented. The paper compares both approaches with each other and against a control behaviour without any communication of intent. Both approaches show statistically significant differences in comparison to the control behaviour. However, the second approach using indicators has shown as being more effective in conveying the intent and also has a higher positive impact on the comfort of the humans encountering the robot. In this paper we present an approach that aids the human operator of unmanned aerial vehicles by automatically performing collision avoidance with obstacles in the environment so that the operator can focus on the global direction of motion of the vehicle. As opposed to systems that override operator control as a last resort in order to avoid collisions (such as those found in modern automobiles), our approach is designed such that the operator can rely on the automatic collision avoidance, enabling intuitive and safe operator control of vehicles that may otherwise be difficult to control. Our approach continually extrapolates the future flight path of the vehicle given the current operator control input. If an imminent collision is predicted our algorithm will override the operator’s control input with the nearest control input that will actually let the vehicle avoid collisions with obstacles. This ensures safe flight while simultaneously maintaining the intent of the human operator as closely as possible. We successfully implemented our approach on a physical quadrotor system in a laboratory environment. In all experiments the human operator failed to crash the vehicle into floors, walls, ceilings, or obstacles, even when deliberately attempting to do so. We consider the cooperative control of a team of robots to estimate the position of a moving target using onboard sensing. In this setting, robots are required to estimate their positions using relative onboard sensing while concurrently tracking the target. Our probabilistic localization and control method takes into account the motion and sensing capabilities of the individual robots to minimize the expected future uncertainty of the target position. Two measures of uncertainty are extensively evaluated and compared: mutual information and the trace of the extended Kalman filter covariance. Our approach reasons about multiple possible sensing topologies and incorporates an efficient topology switching technique to generate locally optimal controls in polynomial time complexity. Simulations illustrate the performance of our approach and prove its flexibility in finding suitable sensing topologies depending on the limited sensing capabilities of the robots and the movements of the target. Furthermore, we demonstrate the applicability of our method in various experiments with single and multiple quadrotor robots tracking a ground vehicle in an indoor environment. This paper presents a unique real-time obstacle avoidance approach for manipulators and mobile robots based on the artificial potential field concept. Collision avoidance, tradi tionally considered a high level planning problem, can be effectively distributed between different levels of control, al lowing real-time robot operations in a complex environment. This method has been extended to moving obstacles by using a time-varying artificial patential field. We have applied this obstacle avoidance scheme to robot arm mechanisms and have used a new approach to the general problem of real-time manipulator control. We reformulated the manipulator con trol problem as direct control of manipulator motion in oper ational space—the space in which the task is originally described—rather than as control of the task's corresponding joint space motion obtained only after geometric and kine matic transformation. Outside the obstacles' regions of influ ence, we caused the end effector to move in a straight line with an... Shared expectations and mutual understanding are critical facets of teamwork. Achieving these in human-robot collaborative contexts can be especially challenging, as humans and robots are unlikely to share a common language to convey intentions, plans, or justifications. Even in cases where human co-workers can inspect a robot's control code, and particularly when statistical methods are used to encode control policies, there is no guarantee that meaningful insights into a robot's behavior can be derived or that a human will be able to efficiently isolate the behaviors relevant to the interaction. We present a series of algorithms and an accompanying system that enables robots to autonomously synthesize policy descriptions and respond to both general and targeted queries by human collaborators. We demonstrate applicability to a variety of robot controller types including those that utilize conditional logic, tabular reinforcement learning, and deep reinforcement learning, synthesizing informative policy descriptions for collaborators and facilitating fault diagnosis by non-experts. For many robotics applications (such as robotic search and rescue), information about the environment must be gathered by a team of robots and returned to a single, specific location. Coordination of robots and sharing of information is vital, and when environments have severe communication limitations, approaches must be robust to communication drop-out and failure. The difficulties are compounded in dynamic environments, where paths previously believed to be free can suddenly become blocked. In this paper, we introduce a novel way of calculating rendezvous points for robots to meet and share information. Using role-based exploration, some robots continuously explore the environment while others ferry information back and forth to a central command centre. Optimal rendezvous point selection leads to more efficient exploration, and allows robots to replan when one of them has unexpected obstacles in its path.