Reinforcement learning (RL) methods have demonstrated their efficiency in simulation environments. However, many applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet safety specifications in the presence of physical constraints. This paper introduces a concept, called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-scale systems. The certificate is realized through a stochastic tube that safely connects the current system state with a terminal set of states, that is known to be safe. A novel formulation in terms of a convex receding horizon problem allows a recursively feasible real-time computation of such probabilistic tubes, despite the presence of possibly unbounded disturbances. A design procedure for PMPSC relying on bayesian inference and recent advances in probabilistic set invariance is presented. Using a numerical car simulation, the method and its design procedure are illustrated by enhancing a simple RL algorithm with safety certificates.
Closely related to the approach proposed in this paper, the concept of a safety framework for learning-based control emerged from robust reachability analysis, robust invariance, as well as classical Lyapunov-based methods @cite_25 @cite_27 @cite_13 @cite_24 . The concept consists of a safe set in the state space and a safety controller as originally proposed in @cite_15 for the case of perfectly known system dynamics in the context of safety barrier functions. While the system state is contained in the safe set, any feasible input (including learning-based controllers) can be applied to the system. However, if such an input would cause the system to leave the safe set, the safety controller interferes. Since this strategy is compatible with any learning-based control algorithm, it serves as a universal safety certification concept. Previously proposed concepts are limited to a robust treatment of the uncertainty, in order to provide rigorous safety guarantees. This potentially results in a conservative system behavior, or even ill-posedness of the overall safety requirement e.g. in case of frequently considered Gaussian distributed additive system noise which has unbounded support.
@cite_24 @cite_27 @cite_15 @cite_13 @cite_25
2766730210 1972149633 2962789054 2127192854
Abstract Learning in interacting dynamical systems can lead to instabilities and violations of critical safety constraints, which is limiting its application to constrained system networks. This paper introduces two safety frameworks that can be applied together with any learning method for ensuring constraint satisfaction in a network of uncertain systems, which are coupled in the dynamics and in the state constraints. The proposed techniques make use of a safe set to modify control inputs that may compromise system safety, while accepting safe inputs from the learning procedure. Two different safe sets for distributed systems are proposed by extending recent results for structured invariant sets. The sets differ in their dynamical allocation to local sets and provide different trade-offs between required communication and achieved set size. The proposed algorithms are proven to keep the system in the safe set at all times and their effectiveness and behavior is illustrated in a numerical example. Abstract This paper presents a new safety feedback design for nonlinear systems based on barrier certificates and the idea of control Lyapunov functions. In contrast to existing methods, this approach ensures safety independently of abstract high-level tasks that might be unknown or change over time. Leaving as much freedom as possible to the safe system, the authors believe that the flexibility of this approach is very promising. The design is validated using an illustrative example. The control of complex systems faces a trade-off between high performance and safety guarantees, which in particular restricts the application of learning-based methods to safety-critical systems. A recently proposed framework to address this issue is the use of a safety controller, which guarantees to keep the system within a safe region of the state space. This paper introduces efficient techniques for the synthesis of a safe set and control law, which offer improved scalability properties by relying on approximations based on convex optimization problems. The first proposed method requires only an approximate linear system model and Lipschitz continuity of the unknown nonlinear dynamics. The second method extends the results by showing how a Gaussian process prior on the unknown system dynamics can be used in order to reduce conservatism of the resulting safe set. We demonstrate the results with numerical examples, including an autonomous convoy of vehicles. For some time now machine learning methods have been widely used in perception for autonomous robots. While there have been many results describing the performance of machine learning techniques with regards to their accuracy or convergence rates, relatively little work has been done on developing theoretical performance guarantees about their stability and robustness. As a result, many machine learning techniques are still limited to being used in situations where safety and robustness are not critical for success. One way to overcome this difficulty is by using reachability analysis, which can be used to compute regions of the state space, known as reachable sets, from which the system can be guaranteed to remain safe over some time horizon regardless of the disturbances. In this paper we show how reachability analysis can be combined with machine learning in a scenario in which an aerial robot is attempting to learn the dynamics of a ground vehicle using a camera with a limited field of view. The resulting simulation data shows that by combining these two paradigms, one can create robotic systems that feature the best qualities of each, namely high performance and guaranteed safety.