Validation is a key challenge in the search for safe autonomy. Simulations are often either too simple to provide robust validation, or too complex to tractably compute. Therefore, approximate validation methods are needed to tractably find failures without unsafe simplifications. This paper presents the theory behind one such black-box approach: adaptive stress testing (AST). We also provide three examples of validation problems formulated to work with AST. An open question when robots operate autonomously in uncertain, real-world environments is how to tractably validate that the agent will act safely. Autonomous robotic systems may be expected to interact with a number of other actors, including humans, while handling uncertainty in perception, prediction and control. Consequently, scenarios are often too high-dimensional to tractably simulate in an exhaustive manner. As such, a common approach is to simplify the scenario by constraining the number of non-agent actors and the range of actions they can take. However, simulating simplified scenarios may compromise safety by eliminating the complexity needed to find rare, but important failures. Instead, approximate validation methods are needed to elicit agent failures while maintaining the full complexity of the simulation.One possible approach to approximate validation is adaptive stress testing (AST) BID6 . In AST, the validation problem is cast as a Markov decision process (MDP). A specific reward function structure is then used with reinforcement learning algorithms in order to identify the most-likely failure of a system in a scenario. Knowing the most-likely failure is useful for two reasons: 1) all other failures are at most as-likely, so it provides a bound on the likelihood of failures, and 2) it uncovers possible failure modes of an autonomous system so they can be addressed. AST is not a silver bullet: it requires accurate models of all actors in the scenario and is susceptible to local convergence. However, it allows failures to be identified tractably in simulation for complicated autonomous systems acting in high-dimensional spaces. This paper briefly presents the latest methodology for using AST and includes example validation scenarios formulated as AST problems. This paper presents the latest formulation of adaptive stress testing, and examples of how it can be applied. AST is an approach to validation that can tractably find failures in autonomous systems in simulation without reducing scenario complexity. Autonomous systems are difficult to validate because they interact with many other actors in high-dimensional spaces according to complicated policies. However, validation is essential for producing autonomous systems that are safe, robust, and reliable.
A formulation for a black-box, reinforcement learning method to find the most-likely failure of a system acting in complex scenarios.
{"src": "scitldr-AIC_train", "paper_id": "rJgoNK-oaE"}