Learning from human demonstrations can facilitate automation but is risky because the execution of the learned policy might lead to collisions and other failures. Adding explicit constraints to avoid unsafe states is generally not possible when the state representations are complex. Furthermore, enforcing these constraints during execution of the learned policy can be challenging in environments where dynamics are difficult to model such as push mechanics in grasping. In this paper, we propose Derivative-Free Recovery (DFR), a two-phase method for generating robust policies from demonstrations in robotic manipulation tasks where the system comes to rest at each time step. In the first phase, we use support estimation of supervisor demonstrations and treat the support as implicit constraints on states. We also propose a time-varying modification for sequential tasks. In the second phase, we use this support estimate to derive a switching policy that employs the learned policy in the interior of the support and switches to a recovery policy to steer the robot away from the boundary of the support if it drifts too close. We present additional conditions, which linearly bound the difference in state at each time step by the magnitude of control, allowing us to prove that the robot will not violate the constraints using the recovery policy. A simulated pushing task in MuJoCo suggests that DFR can reduce collisions by 83 . On a physical line tracking task using a da Vinci Surgical Robot and a moving Stewart platform, DFR reduced collisions by 84 .
In surgical robotics, Yip and Camarillo @cite_21 studied model-free control of continuum manipulators in constrained environments where the constraints are initially unknown. The authors proposed a combined position and force controller which actively estimates Jacobians. Continuum manipulators in surgical environments are in general designed to conform" to obstacles constraints. In this paper, we consider manipulators in general constrained environments where the manipulator may not have direct force feedback from interacting with constraints.
@cite_21
2318613964 2313922182 2605447266 2396317032
Continuum manipulators are designed to operate in constrained environments that are often unknown or unsensed, relying on body compliance to conform to obstacles. The interaction mechanics between the compliant body and unknown environment present significant challenges for traditional robot control techniques based on modeling these interactions exactly. In this letter, we describe a hybrid position force control method that uses a model-less approach. Model-less control is a recently-described approach to control that learns the continuum manipulator Jacobian and adapts to constraints in the environment in a safe manner. When interacting with the environment using the end-effector, these tip-constraints can affect the manipulator Jacobian estimates to become ill-conditioned. This letter addresses this issue by defining a hybrid control approach for simultaneously controlling end-effector position and forces while still maintaining the minimalist approach of model-less control. Under this method, continuum manipulators can safely and effectively interact with the environment, even when these interactions are arbitrary and unknown constraints. Flexible manipulators such as tendon-driven serpentine manipulators perform better than traditional rigid ones in minimally invasive surgical tasks, including navigation in confined space through key-hole like incisions. However, due to the inherent nonlinearities and model uncertainties, motion control of such manipulators becomes extremely challenging. In this work, a hybrid framework combining Programming by Demonstration (PbD) and reinforcement learning is proposed to solve this problem. Gaussian Mixture Models (GMM), Gaussian Mixture Regression (GMR) and linear regression are used to learn the inverse kinematic model of the manipulator from human demonstrations. The learned model is used as nominal model to calculate the output end-effector trajectories of the manipulator. Two surgical tasks are performed to demonstrate the effectiveness of reinforcement learning: tube insertion and circle following. Gaussian noise is introduced to the standard model and the disturbed models are fed to the manipulator to calculate the actuator input with respect to the task specific end-effector trajectories. An expectation maximization (E-M) based reinforcement learning algorithm is used to update the disturbed model with returns from rollouts. Simulation results have verified that the disturbed model can be converged to the standard one and the tracking accuracy is enhanced. In order to get natural and intuitive physical interaction in the pose adjustment of the minimally invasive surgery manipulator, a hybrid variable admittance model based on Fuzzy Sarsa(λ)-learning is proposed in this paper. The proposed model provides continuous variable virtual damping to the admittance controller to respond to human intentions, and it effectively enhances the comfort level during the task execution by modifying the generated virtual damping dynamically. A fuzzy partition defined over the state space is used to capture the characteristics of the operator in physical human-robot interaction. For the purpose of maximizing the performance index in the long run, according to the identification of the current state input, the virtual damping compensations are determined by a trained strategy which can be learned through the experience generated from interaction with humans, and the influence caused by humans and the changing dynamics in the robot are also considered in the learning process. To evaluate the performance of the proposed model, some comparative experiments in joint space are conducted on our experimental minimally invasive surgical manipulator. This paper presents a Robust Constrained Learning-based Nonlinear Model Predictive Control RC-LB-NMPC algorithm for path-tracking in off-road terrain. For mobile robots, constraints may represent solid obstacles or localization limits. As a result, constraint satisfaction is required for safety. Constraint satisfaction is typically guaranteed through the use of accurate, a priori models or robust control. However, accurate models are generally not available for off-road operation. Furthermore, robust controllers are often conservative, since model uncertainty is not updated online. In this work our goal is to use learning to generate low-uncertainty, non-parametric models in situ. Based on these models, the predictive controller computes both linear and angular velocities in real-time, such that the robot drives at or near its capabilities while respecting path and localization constraints. Localization for the controller is provided by an on-board, vision-based mapping and navigation system enabling operation in large-scale, off-road environments. The paper presents experimental results, including over 5 km of travel by a 900 kg skid-steered robot at speeds of up to 2.0 m s. The result is a robust, learning controller that provides safe, conservative control during initial trials when model uncertainty is high and converges to high-performance, optimal control during later trials when model uncertainty is reduced with experience.