While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the @math evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of miles in order to statistically validate performance claims. We implement a simulation framework that can test an entire modern autonomous driving system, including, in particular, systems that employ deep-learning perception and control algorithms. Using adaptive importance-sampling methods to accelerate rare-event probability evaluation, we estimate the probability of an accident under a base distribution governing standard traffic behavior. We demonstrate our framework on a highway scenario, accelerating system evaluation by @math - @math times over naive Monte Carlo sampling methods and @math - @math times (where @math is the number of processors) over real-world testing.
Industrial cyber-physical systems are hybrid systems with strict safety requirements. Despite not having a formal semantics, most of these systems are modeled using Stateflow Simulink for mainly two reasons: (1) it is easier to model, test, and simulate using these tools, and (2) dynamics of these systems are not supported by most other tools. Furthermore, with the ever growing complexity of cyber-physical systems, grows the gap between what can be modeled using an automatic formal verification tool and models of industrial cyber-physical systems. In this paper, we present a simple formal model for self-deriving cars. While after some simplification, safety of this system has already been proven manually, to the best of our knowledge, no automatic formal verification tool supports its dynamics. We hope this serves as a challenge problem for formal verification tools targeting industrial applications. This paper proposes an approach to automatically generating test cases for testing motion controllers of autonomous vehicular systems. Test scenarios may consist of single or multiple vehicles under test at the same time. Tests are performed in simulation environments. The approach is based on using a robustness metric for evaluating simulation outcomes as a cost function. Initial states and inputs are updated by stochastic optimization methods between the tests for achieving smaller robustness values. The test generation framework has been implemented in the toolbox S-TaLiRo. The proposed framework's ability to generate interesting test cases is demonstrated by a case study. Euro NCAP has released its updated rating scheme for 2013-2016 that outlines, amongst other technologies, the implementation of Autonomous Emergency Braking (AEB) technologies within the overall rating scheme. Three types of AEB technologies will be included in the rating scheme, starting with low speed car-to-car AEB City and higher speed car-to-car AEB Inter-Urban in 2014, followed two years later by AEB Pedestrian. In 2011 the Primary Safety Technical Working Group (PNCAP TWG) started working on AEB protocols, where Euro NCAP members have contributed to the development of the Test and Assessment protocols. They have been developed in a relatively short time, by finding the commonalities and discussing the differences between different initiatives from industry, insurers and others that were the main source of input to the working group. Recently, both AEB City and AEB Inter-Urban protocols were finalized. The test protocol details a series of tests, following an incremental speed approach for systems with AEB and Forward Collision Warning (FCW) functionality, and specifies in detail the target vehicle to ensure the highest level of reproducibility and repeatability. The assessment protocols identify the scoring principle and relative weight of each scenario for inclusion in the overall rating scheme. This paper describes both protocols.