Vehicle safety depends on (a) the range of identified hazards and (b) the operational situations for which mitigations of these hazards are acceptably decreasing risk. Moreover, with an increasing degree of autonomy, risk ownership is likely to increase for vendors towards regulatory certification. Hence, highly automated vehicles have to be equipped with verified controllers capable of reliably identifying and mitigating hazards in all possible operational situations. To this end, available methods for the design and verification of automated vehicle controllers have to be supported by models for hazard analysis and mitigation.
Wei et al. @cite_0 describe an autonomous driving platform, capable of bringing vehicles to a safe state and stop, activating a fail-operational mode on critical failure, and a limp-home mode on less critical failure. These are mitigation strategies we can assess in our framework. Their work elaborates on designing a specific class of architectures. Additionally, we provide an approach to systematically evaluate risks and, consequently, derive an architecture design.
@cite_0
2107656445
Reinforcement learning has steadily improved and outperform human in lots of traditional games since the resurgence of deep neural network. However, these success is not easy to be copied to autonomous driving because the state spaces in real world are extreme complex and action spaces are continuous and fine control is required. Moreover, the autonomous driving vehicles must also keep functional safety under the complex environments. To deal with these challenges, we first adopt the deep deterministic policy gradient (DDPG) algorithm, which has the capacity to handle complex state and action spaces in continuous domain. We then choose The Open Racing Car Simulator (TORCS) as our environment to avoid physical damage. Meanwhile, we select a set of appropriate sensor information from TORCS and design our own rewarder. In order to fit DDPG algorithm to TORCS, we design our network architecture for both actor and critic inside DDPG paradigm. To demonstrate the effectiveness of our model, We evaluate on different modes in TORCS and show both quantitative and qualitative results. Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societ al impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, despite large interest and a number of industry players working in the autonomous domain, there is still more to be done in order to develop a system capable of operating at a level comparable to best human drivers. One reason for this is high uncertainty of traffic behavior and large number of situations that an SDV may encounter on the roads, making it very difficult to create a fully generalizable system. To ensure safe and efficient operations, an autonomous vehicle is required to account for this uncertainty and to anticipate a multitude of possible behaviors of traffic actors in its surrounding. In this work, we address this critical problem and present a method to predict multiple possible trajectories of actors while also estimating their probabilities. The method encodes each actor's surrounding context into a raster image, used as input by deep convolutional networks to automatically derive relevant features for the task. Following extensive offline evaluation and comparison to state-of-the-art baselines, as well as closed course tests, the method was successfully deployed to a fleet of SDVs. Lessons from safety-critical anomalies during operation provide important information for constructing safer systems. To assist anomaly analysis, this research develops an integrated Failure Mode and Effect Analysis (FMEA) model to analyze causal scenarios and a Three-Frame Mode model to analyze the working mode inconsistencies of failure cases. The models are used to analyze 180 digital Instrumentation and Control (IC therefore, recovery should be emphasized and planned. The primary focus of autonomous driving research is to improve driving accuracy. While great progress has been made, state-of-the-art algorithms still fail at times. Such failures may have catastrophic consequences. It therefore is im- portant that automated cars foresee problems ahead as early as possible. This is also of paramount importance if the driver will be asked to take over. We conjecture that failures do not occur randomly. For instance, driving models may fail more likely at places with heavy traffic, at complex intersections, and or under adverse weather illumination conditions. This work presents a method to learn to predict the occurrence of these failures, i.e., to assess how difficult a scene is to a given driving model and to possibly give the human driver an early headsup. A camera- based driving model is developed and trained over real driving datasets. The discrepancies between the model's predictions and the human 'ground-truth' maneuvers were then recorded, to yield the 'failure' scores. Experimental results show that the failure score can indeed be learned and predicted. Thus, our prediction method is able to improve the overall safety of an automated driving model by alerting the human driver timely, leading to better human-vehicle collaborative driving. Robotic systems are complex and critical: they are inherently hybrid, combining both hardware and software; they typically exhibit both cyber-physical attributes and autonomous capabilities; and are required to be at least safe and often ethical. While for many engineered systems testing, either through real deployment or via simulation, is deemed sufficient the uniquely challenging elements of robotic systems, together with the crucial dependence on sophisticated software control and decision-making, requires a stronger form of verification. The increasing deployment of robotic systems in safety-critical scenarios exacerbates this still further and leads us towards the use of formal methods to ensure the correctness of, and provide sufficient evidence for the certification of, robotic systems. There have been many approaches that have used some variety of formal specification or formal verification in autonomous robotics, but there is no resource that collates this activity in to one place. This paper systematically surveys the state-of-the art in specification formalisms and tools for verifying robotic systems. Specifically, it describes the challenges arising from autonomy and software architectures, avoiding low-level hardware control and is subsequently identifies approaches for the specification and verification of robotic systems, while avoiding more general approaches. Assuring safety of autonomous vehicles requires that the vehicle control system can perceive the situation in the environment and react to actions of other entities. One approach to vehicle safety assurance is based on the assumption that hazardous sequences of events should be identified during hazard analysis and then some means of hazard avoidance and mitigation, like barriers, should be designed and implemented. Another approach is to design a system which is able to dynamically examine the risk associated with possible actions and then select the safest action to carry it out. Dynamic risk assessment requires maintaining the situation awareness and prediction of possible future situations. We analyse how these two approaches can be applied for autonomous vehicles and what strategies can be used for safety argumentation. This book takes a look at fully automated, autonomous vehicles and discusses many open questions: How can autonomous vehicles be integrated into the current transportation system with diverse users and human drivers? Where do automated vehicles fall under current legal frameworks? What risks are associated with automation and how will society respond to these risks? How will the marketplace react to automated vehicles and what changes may be necessary for companies? Experts from Germany and the United States define key societ al, engineering, and mobility issues related to the automation of vehicles. They discuss the decisions programmers of automated vehicles must make to enable vehicles to perceive their environment, interact with other road users, and choose actions that may have ethical consequences. The authors further identify expectations and concerns that will form the basis for individual and societ al acceptance of autonomous driving. While the safety benefits of such vehicles are tremendous, the authors demonstrate that these benefits will only be achieved if vehicles have an appropriate safety concept at the heart of their design. Realizing the potential of automated vehicles to reorganize traffic and transform mobility of people and goods requires similar care in the design of vehicles and networks. By covering all ofthese topics, the book aims to provide a current, comprehensive, and scientifically sound treatment of the emerging field of autonomous driving". As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical. This motivates the need to build systems that have situational awareness to assess how quali ed they are at that moment to make a decision. We call this self-evaluating capability as introspection. In this paper, we take a small step in this direction and propose a generic framework for introspective behavior in perception systems. Our goal is to learn a model to reliably predict failures in a given system, with respect to a task, directly from input sensor data. We present this in the context of vision-based autonomous MAV flight in outdoor natural environments, and show that it effectively handles uncertain situations. Context: Self-Driving cars are getting more and more attention with public demonstration from all important automotive OEMs but also from companies, which do not have a long history in the automotive industry. Fostered by large international competitions in the last decade, several automotive OEMs have already announced to bring this technology to the market around 2020. Objective: International competitions like the 2007 DARPA Urban Challenge did not focus on efficient usage of resources to realize the self-driving vehicular functionality. Since the automotive industry is very cost-sensitive, realizing reliable and robust self- driving functionality is challenging when expensive and sophisticated sensors mounted very visibly on the vehicle’s roof for example cannot be used. Therefore, the goal for this study is to investigate how architectural design decisions of recent self-driving vehicular technology consider resource-efficiency. Method: In a multiple case study, the architectural design decisions derived for resource- constrained self-driving miniature cars for the international competition CaroloCup are compared with architectural designs from recent real-scale self-driving cars. Results: Scaling down available resources for realizing self-driving vehicular technol- ogy puts additional constraints on the architectural design; especially reusability of software components in platform-independent algorithmic concepts are prevailing. Conclusion: Software frameworks like the robotic operating system (ROS) enable fast prototypical solutions; however, architectural support for resource-constrained devices is limited. Here, architectural design drivers as realized in AUTOSAR are more suitable. Autonomous driving in urban environments is potentially dangerous since a malfunction of vehicle guidance systems can lead to severe situations for passengers inside the autonomous vehicle and other road users. Therefore both, monitoring the current system operation state by a surveillance system, which is able to detect failures of software and hardware modules, and a safety system, which reacts on these failures immediately, is necessary. This book presents approaches to address key challenges based on a vehicle level view and with a special emphasis on Drive-by-Wire systems. The design and testing of modern vehicle electronics are becoming more and more demanding due to increasing interdependencies among components and the safety criticality of tasks. The development towards Drive-by-Wire functionalities in vehicles with multiple actuators for vehicle control further increases the challenge. The book explicitly takes into account the interactions between components and aims at bridging the gap between the need to generate additional customer benefits and the effort to achieve functional safety. The book follows a twofold approach: on the one side, it presents a toolchain to support efficient further development of novel functionalities for Drive-by-Wire vehicles. The toolchain comprises appropriate software tools and scaled and full-scale experimental vehicles. On the other side, development towards functionally safe and flexible Drive-by-Wire vehicles is addressed by proposing a top-down designed architecture for vehicle electronics that is enabled by suitable mechanisms. The resulting goal achievement with regard to functional safety is evaluated based on a novel hierarchical approach Autonomous driving is likely to be the heart of urban transportation in the future. Autonomous vehicles have the potential to increase the safety of passengers and also to make road trips shorter and more enjoyable. As the first steps toward these goals, many car manufacturers are investing in designing and equipping their vehicles with advanced driver-assist systems. Road intersections are considered to be serious bottlenecks of urban transportation, as more than 44 of all reported crashes in U.S. occur within intersection areas which in turn lead to 8,500 fatalities and approximately 1 million injuries every year. Furthermore, the impact of road intersections on traffic delays leads to enormous waste of human and natural resources. In this paper, we therefore focus on intersection management in Intelligent Transportation Systems (ITS) research. In the future, when dealing with autonomous vehicles, it is critical to address safety and throughput concerns that arise from autonomous driving through intersections and roundabouts. Our goal is to provide vehicles with a safe and efficient passage method through intersections and roundabouts. We have been investigating vehicle-to-vehicle (V2V) communications as a part of co-operative driving in the context of autonomous driving. We have designed and developed efficient and reliable intersection protocols to avoid vehicle collisions at intersections and increase traffic throughput. In this paper, we introduce new V2V intersection protocols to achieve the above goals. We show that, in addition to intersections, these protocols are also applicable to vehicle crossings at roundabouts. Additionally, we study the effects of position inaccuracy of commonly-used GPS devices on some of our V2V intersection protocols and suggest required modifications to guarantee their safety and efficiency despite these impairments. Our simulation results show that we are able to avoid collisions and also increase the throughput of the intersections up to 87.82 compared to common traffic-light signalized intersections. A unit-modular robot is a robot that is composed of modules that are all identical. In this thesis we study the design and control of unit-modular dynamically reconfigurable robots. This is based upon the design and construction of a robot called Polypod. We further choose statically stable locomotion as the task domain to evaluate the design and control strategy. The result is the creation of a number of unique locomotion modes. The exciting aspect about a modular robot like Polypod is that it does not only describe one robot, but also presents the building blocks from which many different types of robots can be formed. Dynamic reconfigurability adds a new dimension to the capabilities of the robot. To gain insight into these capabilities in the domain of locomotion, we first build a general, functional taxonomy of locomotion modes. We show that Polypod is capable of generating all classes of statically stable locomotion, a feature unique to Polypod. Next, we propose methods to evaluate vehicles under different operating conditions such as different terrain conditions. We then evaluate and compare each mode of locomotion on Polypod within each class. This study leads to interesting insights into the general characteristics of the corresponding classes of locomotion. Finally, since more modules are expected to increase robot capability, it is important to examine the limit to the number of modules that can be put together in a useful form. We answer this question by investigating the issues of structural stability, actuator strength, computation and control requirements. Modern automated systems evolve both continuously and discretely, and hence require estimation techniques that go well beyond the capability of a typical Kalman filter. Multiple model (MM) estimation schemes track these system evolutions by applying a bank of filters, one for each discrete system mode. Modern systems, however, are often composed of many interconnected components that exhibit rich behaviors, due to complex, system-wide interactions. Modeling these systems leads to complex stochastic hybrid models that capture the large number of operational and failure modes. This large number of modes makes a typical MM estimation approach infeasible for online estimation. This paper analyzes the shortcomings of MM estimation, and then introduces an alternative hybrid estimation scheme that can efficiently estimate complex systems with large number of modes. It utilizes search techniques from the toolkit of model-based reasoning in order to focus the estimation on the set of most likely modes, without missing symptoms that might be hidden amongst the system noise. In addition, we present a novel approach to hybrid estimation in the presence of unknown behavioral modes. This leads to an overall hybrid estimation scheme for complex systems that robustly copes with unforeseen situations in a degraded, but fail-safe manner. The authors propose a Bayesian trajectory prediction and criticality assessment system that allows to reason about imminent collisions of a vehicle several seconds in advance. The authors first infer a distribution of high-level, abstract driving maneuvers such as lane changes, turns, road followings, etc. of all vehicles within the driving scene by modeling the domain in a Bayesian network with both causal and diagnostic evidences. This is followed by maneuver-based, long-term trajectory predictions, which themselves contain random components due to the immanent uncertainty of how drivers execute specific maneuvers. Taking all uncertain predictions of all maneuvers of every vehicle into account, the probability of the ego vehicle colliding at least once within a time span is evaluated via Monte-Carlo simulations and given as a function of the prediction horizon. This serves as the basis for calculating a novel criticality measure, the Time-To-Critical-Collision-Probability (TTCCP) - a generalization of the common Time-To-Collision (TTC) in arbitrary, uncertain, multi-object driving environments and valid for longer prediction horizons. The system is applicable from highly-structured to completely non-structured environments and additionally allows the prediction of vehicles not behaving according to a specific maneuver class. It is expected that autonomous vehicles capable of driving without human supervision will be released to market within the next decade. For user acceptance, such vehicles should not only be safe and reliable, they should also provide a comfortable user experience. However, individual perception of comfort may vary considerably among users. Whereas some users might prefer sporty driving with high accelerations, others might prefer a more relaxed style. Typically, a large number of parameters such as acceleration profiles, distances to other cars, speed during lane changes, etc., characterize a human driver's style. Manual tuning of these parameters may be a tedious and error-prone task. Therefore, we propose a learning from demonstration approach that allows the user to simply demonstrate the desired style by driving the car manually. We model the individual style in terms of a cost function and use feature-based inverse reinforcement learning to find the model parameters that fit the observed style best. Once the model has been learned, it can be used to efficiently compute trajectories for the vehicle in autonomous mode. We show that our approach is capable of learning cost functions and reproducing different driving styles using data from real drivers. Uncertainty complicates early requirements and architecture decisions and may expose a software project to significant risk. Yet software architects lack support for evaluating uncertainty, its impact on risk, and the value of reducing uncertainty before making critical decisions. We propose to apply decision analysis and multi-objective optimisation techniques to provide such support. We present a systematic method allowing software architects to describe uncertainty about the impact of alternatives on stakeholders' goals; to calculate the consequences of uncertainty through Monte-Carlo simulation; to shortlist candidate architectures based on expected costs, benefits and risks; and to assess the value of obtaining additional information before deciding. We demonstrate our method on the design of a system for coordinating emergency response teams. Our approach highlights the need for requirements engineering and software cost estimation methods to disclose uncertainty instead of hiding it. Threat assessment during semiautonomous driving is used to determine when correcting a driver's input is required. Since current semiautonomous systems perform threat assessment by predicting a vehicle's future state while treating the driver's input as a disturbance, autonomous controller intervention is limited to a restricted regime. Improving vehicle safety demands threat assessment that occurs over longer prediction horizons wherein a driver cannot be treated as a malicious agent. In this paper, we describe a real-time semiautonomous system that utilizes empirical observations of a driver's pose to inform an autonomous controller that corrects a driver's input when possible in a safe manner. We measure the performance of our system using several metrics that evaluate the informativeness of the prediction and the utility of the intervention procedure. A multisubject driving experiment illustrates the usefulness, with respect to these metrics, of incorporating the driver's pose while designing a semiautonomous system. Automatic methods for generating state-of-the-art neural network architectures without human experts have generated significant attention recently. This is because of the potential to remove human experts from the design loop which can reduce costs and decrease time to model deployment. Neural architecture search (NAS) techniques have improved significantly in their computational efficiency since the original NAS was proposed. This reduction in computation is enabled via weight sharing such as in Efficient Neural Architecture Search (ENAS). However, recently a body of work confirms our discovery that ENAS does not do significantly better than random search with weight sharing, contradicting the initial claims of the authors. We provide an explanation for this phenomenon by investigating the interpretability of the ENAS controller's hidden state. We are interested in seeing if the controller embeddings are predictive of any properties of the final architecture - for example, graph properties like the number of connections, or validation performance. We find models sampled from identical controller hidden states have no correlation in various graph similarity metrics. This failure mode implies the RNN controller does not condition on past architecture choices. Importantly, we may need to condition on past choices if certain connection patterns prevent vanishing or exploding gradients. Lastly, we propose a solution to this failure mode by forcing the controller's hidden state to encode pasts decisions by training it with a memory buffer of previously sampled architectures. Doing this improves hidden state interpretability by increasing the correlation controller hidden states and graph similarity metrics. Recent advancements in the perception for autonomous driving are driven by deep learning. In order to achieve the robust and accurate scene understanding, autonomous vehicles are usually equipped with different sensors (e.g. cameras, LiDARs, Radars), and multiple sensing modalities can be fused to exploit their complementary properties. In this context, many methods have been proposed for deep multi-modal perception problems. However, there is no general guideline for network architecture design, and questions of "what to fuse", "when to fuse", and "how to fuse" remain open. This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. To this end, we first provide an overview of on-board sensors on test vehicles, open datasets and the background information of object detection and semantic segmentation for the autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions. In the appendix, we provide tables that summarize topics and methods. We also provide an interactive online platform to navigate each reference: this https URL.