A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The overall objective of the Strategic Computing (SC) Program of the Defense Advanced Research Projects Agency (DARPA) is to develop and demonstrate a new generation of machine intelligence technology which can form the basis for more capable military systems in the future and also maintain a position of world leadership for the US in computer technology. Begun in 1983, SC represents a focused research strategy for accelerating the evolution of new technology and its rapid prototyping in realistic military contexts. Among the very ambitious demonstration prototypes being developed within the SC Program are: 1) the Pilot's Associate which will aid the pilot in route planning, aerial target prioritization, evasion of missile threats, and aircraft emergency safety procedures during flight; 2) two battle management projects one for the for the Army, which is just getting started, called the AirLand Battle Management program (ALBM) which will use knowledge-based systems technology to assist in the generation and evaluation of tactical options and plans at the Corps level; 3) the other more established program for the Navy is the Fleet Command Center Battle Management Program (FCCBIVIP) at Pearl Harbor. The FCCBMP is employing knowledge-based systems and natural language technology in a evolutionary testbed situated in an operational command center to demonstrate and evaluate intelligent decision-aids which can assist in the evaluation of fleet readiness and explore alternatives during contingencies; and 4) the Autonomous Land Vehicle (ALV) which integrates in a major robotic testbed the technologies for dynamic image understanding, knowledge-based route planning with replanning during execution, hosted on new advanced parallel architectures. The goal of the Strategic Computing computer vision technology base (SCVision) is to develop generic technology that will enable the construction of complete, robust, high performance image understanding systems to support a wide range of DoD applications. Possible applications include autonomous vehicle navigation, photointerpretation, smart weapons, and robotic manipulation. This paper provides an overview of the technical and program management plans being used in evolving this critical national technology. <s> BIB001 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The objectives of this paper are to discuss how vehicle automation technology can be used to benefit car drivers and also to propose a concept of an autonomous highway vehicle which improves highway driving safety. <s> BIB002 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> From the Publisher: ::: Increasingly, microcomputers are being used in applications where their correct operation is vital to ensure the safety of the public and the environment: from anti-lock braking systems in automobiles, to fly-by-wire aircraft, to shut-down systems at nuclear power plants. It is, therefore, vital that engineers are aware of the safety implications of the systems they develop. This book is an introduction to the field of safety-critical computer systems, and is written for any engineer who uses microcomputers within real-time embedded systems. It assumes no prior knowledge of safety, or of any specific computer hardware or programming language. This book covers all phases of the life of a safety-critical system from its conception and specification, through to its certification, installation, service and decommissioning; provides information on how to assess the safety implications of projects, and determine the measures necessary to develop systems to meet safety needs; gives a thorough grounding in the techniques available to investigate the safety aspects of computer-based systems and the methods that may be used to enhance their dependability; and uses case studies and worked examples from a wide range of industrial sectors including the nuclear, aircraft, automotive and consumer products industries. This text is intended for both engineering and computer science students, and for practising engineers within computer-related industries. The approach taken is equally suited to engineers who consider computers from a hardware, software or systems viewpoint. <s> BIB003 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Partial table of contents: THE HUMAN FACTORS FUNCTION. System Design and Evaluation (S. Czaja). THE HUMAN FACTORS FUNDAMENTALS. Perceptual Motor Skills and Human Motion Analysis (D. Regan). Biomechanics of the Human Body (W. Marras). JOB DESIGN. Allocation of Functions (J. Sharit). Task Analysis (H. Luczak). EQUIPMENT, WORKPLACE, AND ENVIRONMENTAL DESIGN. Biomechanical Aspects of Workplace Design (D. Chaffin). Illumination (P. Boyce). DESIGN FOR HEALTH AND SAFETY. Occupational Risk Management (B. Zimolong). PERFORMANCE MODELING. Cognitive Modeling (R. Eberts). EVALUATION. Human Factors Audits (C. Drury). HUMAN--COMPUTER INTERACTION. Software--User Interface Design (Y. Liu). SELECTED APPLICATIONS OF HUMAN FACTORS. Human Factors in Process Control (N. Moray). Indexes. <s> BIB004 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The final purpose of Automated Vehicle Guidance Systems (AVGSs) is to obtain fully automatic driven vehicles to optimize transport systems, minimizing delays, increasing safety and comfort. In order to achieve these goals, lots of Artificial Intelligence techniques must be improved and merged. In this article we focus on learning and simulating the Human-Level decisions involved in driving a racing car. To achieve this, we have studied the convenience of using Neuroevolution of Augmenting Topologies (NEAT). To experiment and obtain comparative results we have also developed an online videogame prototype called Screaming Racers, which is used as test-bed environment. <s> BIB005 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The navigation of mobile robots or unmanned autonomous vehicles (UAVs) in an environment full of obstacles has a significant impact on its safety. If the robot maneuvers too close to an obstacle, it increases the probability of an accident. Preventing this is crucial in dynamic environments, where the obstacles, such as other UAVs, are moving. This kind of safe navigation is needed in any autonomous movement application but it is of a vital importance in applications such as automated transportation of nuclear or chemical waste. This paper presents the Maximum Margin Search using a Convex Hull construction (MMS-CH), an algorithm for a fast construction of a maximum margin between sets of obstacles and its maintenance as the input data are dynamically altered. This calculation of the safest path is inspired by the Support Vector Machines (SVM). It utilizes the convex hull construction to preprocess the input data and uses the boundaries of the hulls to search for the optimal margin. The MMS-CH algorithm takes advantage of the elementary geometrical properties of the 2-dimensional Euclidean space resulting in 1) significant reduction of the problem complexity by eliminating irrelevant data; 2) computationally less expensive approach to maximum margin calculation than standard SVM-based techniques; and 3) inexpensive recomputation of the solution suitable for real time dynamic applications. <s> BIB006 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Increasing demands on safety and energy efficiency will require higher levels of automation in transportation systems. This involves dealing with safety-critical distributed coordination. In this paper we demonstrate how a Satisfiability Modulo Theories (SMT) solver can be used to prove correctness of a vehicular coordination problem. We formalise a recent distributed coordination protocol and validate our approach using an intersection collision avoidance (ICA) case study. The system model captures continuous time and space, and an unbounded number of vehicles and messages. The safety of the case study is automatically verified using the Z3 theorem prover. <s> BIB007 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Recently several artificial intelligence labs have suggested the use of fully equipped vehicles with the capability of sensing the surrounding environment to enhance roadway safety. As a result, it is anticipated in the future that many vehicles will be autonomous and thus there is a need to optimize the movement of these vehicles. This paper presents a new tool for optimizing the movements of autonomous vehicles through intersections: iCACC. The main concept of the proposed tool is to control vehicle trajectories using Cooperative Adaptive Cruise Control (CACC) systems to avoid collisions and minimize intersection delay. Simulations were executed to compare conventional signal control with iCACC considering two measures of effectiveness - delay and fuel consumption. Savings in delay and fuel consumption in the range of 91 and 82 percent relative to conventional signal control were demonstrated, respectively. <s> BIB008 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Abstract The intelligent control of autonomous vehicles is one of the most important challenges that intelligent transport systems face today. The application of artificial intelligence techniques to the automatic management of vehicle actuators enables the different Advanced Driver Assistance Systems (ADAS) or even autonomous driving systems, to perform a low level management in a very similar way to that of human drivers by improving safety and comfort. In this paper, we present a control schema to manage these low level vehicle actuators (steering throttle and brake) based on fuzzy logic, an artificial intelligence technique that is able to mimic human procedural behavior, in this case, when performing the driving task. This automatic low level control system has been defined, implemented and tested in a Citroen C3 testbed vehicle, whose actuators have been automated and can receive control signals from an onboard computer where the soft computing-based control system is running. <s> BIB009 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Automation in transportation (rail, air, road, etc.) is becoming increasingly complex and interconnected. Ensuring that these sophisticated non-deterministic software systems can be trusted and remain resilient is a community concern. As technology evolves, systems are moving increasingly towards autonomy where the “machine” is intelligent: perceiving, deciding, learning, etc. often without human engagement. Our current mechanisms and policies for oversight and certification of these systems to ensure they operate robustly in safety-critical situations are not keeping pace with technology advancements. How is an autonomous system different than an advanced automatic system? How is trust different for an autonomous system? What are different perspectives on trust? Is it appropriate to apply the techniques used to establish trust in a today’s transportation systems to establishing trust in an autonomous system? This paper will examine these questions and propose a framework for discussing autonomy assurance and trust in transportation applications. We will explore further with two examples: 1) the notion of a self-driving taxi-cab; and 2) the evolution of a two-pilot flight deck, to single-pilot operations. <s> BIB010 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Highly automated driving on highways requires a complex artificial intelligence that makes optimal decisions based on ongoing measurements. Notably no attempt has been performed to evaluate the impacts of such a sophisticated system on traffic. Another important point is the impact of continuous increase in the number of highly automated vehicles on future traffic safety and traffic flow. This work introduces a novel framework to evaluate these impacts in a developed microscopic traffic simulation environment. This framework is used on the one hand to ensure the functionality of the driving strategy in the early development stage. On the other hand, the impacts of increasing automation rates, up to hundred percent, on traffic safety and traffic flow is evaluated. <s> BIB011 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Forecasts of automation in driving suggest that wide spread market penetration of fully autonomous vehicles will be decades away and that before such vehicles will gain acceptance by all stake holders, there will be a need for driving assistance in key driving tasks, supplemented by automated active safety capability. This paper advances a Bayesian Artificial Intelligence model for the design of real time transition from assisted driving to automated driving under conditions of high probability of a collision if no action is taken to avoid the collision. Systems can be designed to feature collision warnings as well as automated active safety capabilities. In addition to the high level architecture of the Bayesian transition model, example scenarios illustrate the function of the real-time transition model. <s> BIB012 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The historical roles of drivers, vehicle manufacturers, federal and state regulators, and law enforcement agencies in automotive safety is well understood. However, the increasing deployment of driving automation technologies to support various comfort, convenience, efficiency, productivity, mobility, and possibly safety features has the potential to alter this understanding. In order to facilitate clarity in discussing the topic of driving automation with other stakeholders and to clarify the level(s) of automation on which the agency is currently focusing its efforts, the National Highway Traffic Safety Administration (NHTSA) released a Preliminary Statement of Policy (SOP) concerning Automated Vehicles that included its automation levels. In this paper, the authors present key factors for consideration in each automation level which are based upon Society of Automotive Engineers (SAE) J3016. These factors focus on adding more specificity with regard to the distribution of the driving tasks between the driver and the automation system. The result of this effort has led to a refinement of an understanding of the automation levels based on the nature of the vehicle control aspect provided by the feature, the nature of the environmental sensing and response, the fallback strategy employed, and the feature’s scope of operation. <s> BIB013 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Traffic engineers, vehicle manufacturers, technology groups, and government agencies are anticipating and preparing for the emergence of fully automated vehicles into the American transportation system. This new technology has the potential to revolutionize many aspects of transportation, particularly safety. However, fully automated vehicles may not create the truly crash-free environment predicted. One particular problem is crash assignment, especially between automated vehicles and nonautomated vehicles. Although some researchers have indicated that automated vehicles will need to be programmed with some sort of ethical system in order to make decisions on how to crash, few, if any, studies have been conducted on how particular ethical theories will actually make crash decisions and how these ethical paradigms will affect automated vehicle programming. The integration of three ethical theories—utilitarianism, respect for persons, and virtue ethics—with vehicle automation is examined, and a simple progr... <s> BIB014 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> In Intelligent Transportation System, Advance Driver Assistance Systems (ADAS) plays a vital role. In ADAS, many research works are done in the area of traffic sign recognition, Forward Collision Warning, Automotive navigation system, Lane departure warning system but an another important area to look through is speed bumps detection. The recognition of speed bump is a safety to a human and a vehicle. Early research in speed bump detection is done with the help of sensors, accelerometer and GPS. In this paper, a novel method is presented to achieve speed bump detection and recognition either to alert or to interact directly with the vehicle. Detection of speed bump is recognized with a help of image processing concepts. This methodology is effortless and simple to implement without the investment of special sensors, hardware, Smartphone and GPS. This procedure suits very well for the roads constructed with proper marking, and can be used in self-driving car. <s> BIB015 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Greater production and improved safety in the mining industry can be enhanced by the use of automated vehicles. This paper presents results in applying Learning from Demonstration (LfD) to a laboratory semi-automated mine inspection robot following a path through a simulated mine. Three methods, Gaussian Mixture Model (GMM), Continuous Hidden Markov Model (CHMM), and Discrete Hidden Markov Model (DHMM) were used to implement the LfD and a comparison of the implementation results is presented. The results from the different models were then used to implement a novel, optimised path decomposition technique that may be suitable for possible robot use within an underground mine. <s> BIB016 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The constant growth of the number of vehicles in today’s world demands improvements in the safety and efficiency of roads and road use. This can be in part satisfied by the implementation of autonomous driving systems because of their greater precision than human drivers in controlling a vehicle. As result, the capacity of the roads would be increased by reducing the spacing between vehicles. Moreover, greener driving modes could be applied so that the fuel consumption, and therefore carbon emissions, would be reduced. This paper presents the results obtained by the AUTOPIA program during a public demonstration performed in June 2012. This driverless experiment consisted of a 100-kilometre route around Madrid (Spain), including both urban and motorway environments. A first vehicle – acting as leader and manually driven – transmitted its relevant information – i.e., position and speed – through an 802.11p communication link to a second vehicle, which tracked the leader’s trajectory and speed while maintaining a safe distance. The results were encouraging, and showed the viability of the AUTOPIA approach. ::: First published online: 28 Jan 2015 <s> BIB017 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Today research is going on within different essential functions need to bring automatic vehicles to the roads. However, there will be manual driven vehicles for many years before it is fully automated vehicles on roads. In complex situations, automated vehicles will need human assistance for long time. So, for road safety driver monitoring is even more important in the context of autonomous vehicle to keep the driver alert and awake. But, limited effort has been done in total integration between automatic vehicle and human driver. Therefore, human drivers need to be monitored and able to take over control within short notice. This papers provides an overview on autonomous vehicles and un-obstructive driver monitoring approaches that can be implemented in future autonomous vehicles to monitor driver e.g., to diagnose and predict stress, fatigue etc. in semi-automated vehicles. <s> BIB018 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> RobIL program is a collaborative effort of several leading Israeli academic institutions and industries in the field of robotics. The current objective of the program is to develop an Autonomous Off-Road Unmanned Ground Vehicle (UGV). The intention is to deal through this project with some of the most urgent challenges in the field of autonomous vehicles. One of these challenges, is the lack of efficient Safety Performance Verification technique, as the existing tools for hardware and software reliability and safety engineering do not provide a comprehensive solution regarding algorithms that are based on Artificial Intelligent (AI) and Machine Learning. In order to deal with this gap a novel methodology that is based on statistical testing in simulated environment, is presented. In this work the RobIL UGV project with special emphasis on the performance and safety verification methodology is presented. <s> BIB019 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Abstract: Growing research progress in Internet of Things (IoT), automated/connected cars, Artificial Intelligence and person's data acquisition (Quantified Self) will help to reduce behavioral uncertainties in transport and unequivocally influence future transport landscapes. This vision paper argues that by capitalizing advances in data collection and methodologies from emerging research disciplines, we could make the driver amenable to a knowable and monitorable entity, which will improve road safety. We present an interdisciplinary framework, inspired by the Safe system, to extract knowledge from the large amount of available data during driving. The limitation of our approach is discussed. <s> BIB020 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The future will see autonomous machines acting in the same environment as humans, in areas as diverse as driving, assistive technology, and health care. Think of self-driving cars, companion robots, and medical diagnosis support systems. We also believe that humans and machines will often need to work together and agree on common decisions. Thus hybrid collective decision making systems will be in great need. ::: ::: In this scenario, both machines and collective decision making systems should follow some form of moral values and ethical principles (appropriate to where they will act but always aligned to humans'), as well as safety constraints. In fact, humans would accept and trust more machines that behave as ethically as other humans in the same environment. Also, these principles would make it easier for machines to determine their actions and explain their behavior in terms understandable by humans. Moreover, often machines and humans will need to make decisions together, either through consensus or by reaching a compromise. This would be facilitated by shared moral values and ethical principles. <s> BIB021 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Nowadays, many attentions have been devoted to autonomous vehicles because the automation of driving technology has a large number of benefits, such as the minimization of risks, the improvement of mobility and ease of drivers. Among many technologies for autonomous driving, road environmental recognition is one of the key issues. In this paper, we present the test results of various object detection algorithms using single monocular camera for autonomous vehicle in real driving conditions. The vision recognition system tested in this paper has three main recognition parts: pedestrian detection, traffic sign and traffic light recognition. We use Histogram of Gradients (HOG) features and detect the pedestrians by Support Vector Machine (SVM). Also features of traffic signs are extracted by Principal Components Analysis (PCA) and canny edge detection is used for traffic lights. These two signals are classified by Neural Network (NN). Algorithms that we tested are implemented in General-Purpose computing on Graphics Processing Units (GPGPU). We show the effectiveness of these methods in real-time applications for autonomous driving. <s> BIB022 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Detecting changing traffic conditions is of primal importance for the safety of autonomous cars navigating in urban environments. Among the traffic situations that require more attention and careful planning, road junctions are the most significant. This work presents an empirical study of the application of well known machine learning techniques to create a robust method for road junction detection. Features are extracted from 3D pointclouds corresponding to single frames of data collected by a laser rangefinder. Three well known classifiers-support vector machines, adaptive boosting and artificial neural networks-are used to classify them into “junctions” or “roads”. The best performing classifier is used in the next stage, where structured classifiers-hidden Markov models and conditional random fields-are used to incorporate contextual information, in an attempt to improve the performance of the method. We tested and compared these approaches on datasets from two different 3D laser scanners, and in two different countries, Germany and Brazil. <s> BIB023 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Abstract Advanced driver assistance systems (ADASs) enhance transportation safety and mobility, and reduce impacts on the environment and economical costs, through decreasing driver errors. One of the main features of ADASs is cruise control system that maintains the driver's desired speed without intervention from the driver. Adaptive cruise control (ACC) systems adjust the vehicle's speed to maintain a safe following distance to the vehicle in front. Adding vehicle-to-vehicle and vehicle-to-infrastructure communications (V2X) to ACC systems, result in cooperative adaptive cruise control (CACC) systems, where each vehicle has trajectory data of all other vehicles in the same lane. Although CACC systems offer advantages over ACC systems in increasing throughput and average speed, they are more vulnerable to cyber-security attacks. This is due to V2X communications that increase the attack surface from one vehicle to multiple vehicles. In this paper, we inject common types of attack on the application layer of connected vehicles to show their vulnerability in comparison to autonomous vehicles. We also proposed a decision support system that eliminates risk of inaccurate information. The microscopic work simulates a CACC system with a bi-objective PID controller and a fuzzy detector. A case study is illustrated in detail to verify the system functionality. <s> BIB024 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> PreCrash problem of Intelligent Control of autonomous vehicles robot is a very complex problem, especially vehicle pre-crash scenariws and at points of intersections in real-time environmenta. This Paper presents a novel architecture of Intelligent adaptive control for autonomous vehicle agent that depends on Artificial Intelligence Techniques that applies case-based reasoning techniques, where Parallel CBR Engines are implemented for different scenarios' of PreCrash problem and sub-problems of intersection safety and collision avoidance, in the higher level of the controller and A∗ path planner for path planning and at lower-levels it also uses some features of autonomous vehicle dynamics. Moreover, the planner is enhanced by combination of Case-Based Planner. All modules are presented and discussed. Experimental results are conducted in the framework of Webots autonomous vehicle tool and overall results are good for the CBR Engine for Adaptive control and also for the hybrid Case-Based Planner, A∗ and D∗ motion planner along with conclusion and future work. <s> BIB025 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Autonomous Vehicle (AV) or self-driving vehicle technology promises to provide many economical and societal benefits and impacts. Safety is on the top of these benefits. Trajectory or path planning is one of the essential and critical tasks in operating the autonomous vehicle. In this paper we are tackling the problem of trajectory planning for fully-autonomous vehicles. Our use cases are designed for autonomous vehicles in a cloud based connected vehicle environment. This paper presents a method for selecting safe-optimal trajectory in autonomous vehicles. Selecting the safe trajectory in our work mainly based on using Big Data mining and analysis of real-life accidents data and real-time connected vehicles' data. The decision of selecting this trajectory is done automatically without any human intervention. The human touches in this scenario could be only at defining and prioritizing the driving preferences and concerns at the beginning of the planned trip. Safety always overrides the ranked user preferences listed in this work. The output of this work is a safe trajectory that represented by the position, ETA, distance, and the estimated fuel consumption for the entire trip. <s> BIB026 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Developers of autonomous systems face distinct challenges in conforming to established methods of validating safety. It is well known that testing alone is insufficient to assure safety, because testing long enough to establish ultra-dependability is generally impractical. Thatfis why software safety standards emphasize high quality development processes. Testing then validates process execution rather than directly validating dependability. Two significant challenges arise in applying traditional safety processes to autonomous vehicles. First, simply gathering a complete set of system requirements is difficult because of the sheer number of combinations of possible scenarios and faults. Second, autonomy systems commonly use machine learning (ML) in a way that makes the requirements and design of the system opaque. After training, usually we know what an ML component will do for an input it has seen, but generally not what it will do for at least some other inputs until we try them. Both of these issues make it difficult to trace requirements and designs to testing as is required for executing a safety validation process. In other words, we are building systems that can not be validated due to incomplete or even unknown requirements and designs. Adaptation makes the problem even worse by making the system that must be validated a moving target. In the general case, it is impractical to validate all the possible adaptation states of an autonomy system using traditional safety design processes. An approach that can help with the requirements, design, and adaptation problems is basing a safety argument not on correctness of the autonomy functionality itself, but rather on conformance to a set of safety envelopes. Each safety envelope describes a boundary within the operational state space of the autonomy system. A system operating within a "safe" envelope knows that it is safe and can operate with full autonomy. A system operating within an "unsafe" envelope knows that it is unsafe, and must invoke a failsafe action. Multiple partial specifications can be used as an envelope set, with the intersection of safe envelopes permitting full autonomy, and the union of unsafe envelopes provoking validated, and potentially complex, failsafe responses. Envelope mechanisms can be implemented using traditional software engineering techniques, reducing the problems with requirements, design, and adaptation that would otherwise impede safety validation. Rather than attempting to prove that autonomy will always work correctly (which is still a valuable goal to improve availability), the envelope approach measures the behavior of one or more autonomous components to determine if the result is safe. While this is not necessarily an easy thing to do, there is reason to believe that checking autonomy behaviors for safety is easier than implementing perfect, optimized autonomy actions. This envelope approach might be used to detect faults during development and to trigger failsafes in fleet vehicles. Inevitably there will be tension between simplicity of the envelope definitions and permissiveness, with more permissive envelope definitions likely being more complex. Operating in the gap areas between "safe" and "unsafe" requires human supervision, because the autonomy system can not be sure it is safe. One way to look at the progression from partial to full autonomy is that, over time, systems can increase permissiveness by defining and growing "safe" envelopes, shrinking "unsafe" envelopes, and eliminating any gap areas. <s> BIB027 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The sole idea of the autonomous driving is to reduce the number of accidents which will be caused by human errors. There will be a phase where autonomous vehicles will have to take part with the human driven vehicles. In this phase autonomous vehicle and vehicle drivers must be careful in driving maneuvers, since it is impossible to just have autonomous vehicles on the road while the transition to fully autonomous driving takes place. To make the system more reliable and robust, authors feel that it must be substantially improved and to achieve this 3 methodologies have been proposed. The use cases are derived from the camera placed in the suitable positions of every vehicle. This creates several use cases with different driving behaviors, which truly reflects the conditions which the autonomous car needs to undergo in series. Authors propose several architectures to address the safety related issues. Since, the number of use cases are so random that it becomes practically impossible to manage data, so we propose to have a secured cloud storage. The cloud provides a secure and reliable data management system to the machine learning algorithm computation. The data management till now is effectively done via trained unique use cases. This make a big data which is encapsulation of different unique use cases under different driving conditions. So we conclude that the usage of ML and cloud based data management is the way forward for handling autonomous vehicle reliably. This may not be the complete solution, but nevertheless we are one step closer towards 2025. This concept can be implemented only in co-operation of the OEMs and Public authorities. <s> BIB028 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Mobility is a basic necessity of contemporary society and it is a key factor in global economic development. The basic requirements for the transport of people and goods are: safety and duration of travel, but also a number of additional criteria are very important: energy saving, pollution, passenger comfort. Due to advances in hardware and software, automation has penetrated massively in transport systems both on infrastructure and on vehicles, but man is still the key element in vehicle driving. However, the classic concept of 'human-in-the-loop' in terms of 'hands on' in driving the cars is competing aside from the self-driving startups working towards so-called 'Level 4 autonomy', which is defined as "a self-driving system that does not requires human intervention in most scenarios". In this paper, a conceptual synthesis of the autonomous vehicle issue is made in connection with the artificial intelligence paradigm. It presents a classification of the tasks that take place during the driving of the vehicle and its modeling from the perspective of traditional control engineering and artificial intelligence. The issue of autonomous vehicle management is addressed on three levels: navigation, movement in traffic, respectively effective maneuver and vehicle dynamics control. Each level is then described in terms of specific tasks, such as: route selection, planning and reconfiguration, recognition of traffic signs and reaction to signaling and traffic events, as well as control of effective speed, distance and direction. The approach will lead to a better understanding of the way technology is moving when talking about autonomous cars, smart/intelligent cars or intelligent transport systems. Keywords: self-driving vehicle, artificial intelligence, deep learning, intelligent transport systems. <s> BIB029 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Adrian Weller acknowledges support by the Alan Turing Institute under the EPSRC grant EP/N510129/1, and by the Leverhulme Trust via the CFI. <s> BIB030 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Autonomous vehicles pose new challenges to their testing, which is required for safety certification. While Autonomous vehicles will use training sets as specification for machine learning algorithms, traditional validation depends on the system’s requirements and design.The presented approach uses training sets which are observations of traffic situations as system specification. It aims at deriving test-cases which incorporate the continuous behavior of other traffic participants. Hence, relevant scenarios are mined by analyzing and categorizing behaviors. By using abstract descriptions of the behaviors we discuss how test-cases can be compared to each other, so that similar test-cases are avoided in the test-suite. We demonstrate our approach using a combination of an overtake assistant and an adaptive cruise control. <s> BIB031 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Ever increasing autonomy of machines and the need to interact with them creates challenges to ensure safe operation. Recent technical and commercial interest in increasing autonomy of vehicles has led to the integration of more sensors and actuators inside the vehicle, making them more like robots. For interaction with semi-autonomous cars, the use of these sensors could help to create new safety mechanisms. This work explores the concept of using motion tracking (i.e. skeletal tracking) data gathered from the driver whilst driving to learn to classify the manoeuvre being performed. A kernel-based classifier is trained with empirically selected features based on data gathered from a Kinect V2 sensor in a controlled environment. This method shows that skeletal tracking data can be used in a driving scenario to classify manoeuvres and sets a background for further work. <s> BIB032 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Traffic safety is an important problem for autonomous vehicles. The development of Traffic Sign Recognition (TSR) dedicated to reducing the number of fatalities and the severity of road accidents is an important and an active research area. Recently, most TSR approaches of machine learning and image processing have achieved advanced performance in traditional natural scenes. However, there exists a limitation on the accuracy in road sign recognition and on time consuming. This paper proposes a real-time algorithm for shape classification of traffic signs and their recognition to provide a driver alert system. The proposed algorithm is mainly composed of two phases: shape classification and content classification. This algorithm takes as input a list of Bounding Boxes generated in a previous work, and will classify them. The traffic sign's shape is classified by an artificial neural network (ANN). Traffic signs are classified according to their shape characteristics, as triangular, squared and circular shapes. Combining color and shape information, traffic signs are classified into one of the following classes: danger, information, obligation or prohibition. The classified circular and triangular shapes are passed on to the second ANN in the third phase. These identify the pictogram of the road sign. The output of the second artificial neural network allows the full classification of the road sign. The algorithm proposed is evaluated on a dataset of road signs of a Tunisian database sign. <s> BIB033 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Today, In the content of road vehicles, intelligent systems and autonomous vehicles, one of the important problem that should be solved is Road Terrain Classification that improves driving safety and comfort passengers. There are many studies in this area that improved the accuracy of classification. An improved classification method using color feature extraction is proposed in this paper. Color Feature of images is used to classify The Road Terrain Type and then a Neural Network (NN) is used to classify the Color Features extracted from images. Proposed idea is to identify road by processing digital images taken from the roads with a camera installed on a car. Asphalt, Grass, Dirt and Rocky are four types of terrain that identified in this study. <s> BIB034 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Understanding intent and relevance of surrounding agents from video is an essential task for many applications in robotics and computer vision. The modeling and evaluation of contextual, spatio-temporal situation awareness is particularly important in the domain of intelligent vehicles, where a robot is required to smoothly navigate in a complex environment while also interacting with humans. In this paper, we address these issues by studying the task of on-road object importance ranking from video. First, human-centric object importance annotations are employed in order to analyze the relevance of a variety of multi-modal cues for the importance prediction task. A deep convolutional neural network model is used for capturing video-based contextual spatial and temporal cues of scene type, driving task, and object properties related to intent. Second, the proposed importance annotations are used for producing novel analysis of error types in image-based object detectors. Specifically, we demonstrate how cost-sensitive training, informed by the object importance annotations, results in improved detection performance on objects of higher importance. This insight is essential for an application where navigation mistakes are safety-critical, and the quality of automation and human-robot interaction is key. HighlightsWe study a notion of object relevance, as measured in a spatio-temporal context of driving a vehicle.Various spatio-temporal object and scene cues are analyzed for the task of object importance classification.Human-centric metrics are employed for evaluating object detection and studying data bias.Importance-guided training of object detectors is proposed, showing significant improvement over an importance-agnostic baseline. <s> BIB035 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> In the field of self-driving technology, the stability and comfort of the intelligent vehicle are the focus of attention. The paper applies cognitive psychology theory to the research of driving behavior and analyzes the behavior mechanism about the driver’s operation. Through applying the theory of hierarchical analysis, we take the safety and comfort of intelligent vehicle as the breakthrough point. And then we took the data of human drivers’ perception behavior as the training set and did regression analysis using the method of regression analysis of machine learning according to the charts of the vehicle speed and the visual field, the vehicle speed and the gaze point as well as the vehicle speed and the dynamic vision. At last we established linear and nonlinear regression models (including the logarithmic model) for the training set. The change in thinking is the first novelty of this paper. Last but not least important, we verified the accuracy of the model through the comparison of different regression analysis. Eventually, it turned out that using logarithmic relationship to express the relationship between the vehicle speed and the visual field, the vehicle speed and the gaze point as well as the vehicle speed and the dynamic vision is better than other models. In the aspect of application, we adopted the technology of multi-sensor fusion and transformed the acquired data from radar, navigation and image to log-polar coordinates, which makes us greatly simplify information when dealing with massive data problems from different sensors. This approach can not only reduce the complexity of the server’s processing but also drives the development of intelligent vehicle in information computing. We also make this model applied in the intelligent driver’s cognitive interactive debugging program, which can better explain and understand the intelligent driving behavior and improved the safety of intelligent vehicle to some extent. <s> BIB036 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> Researchers and automotive industry experts are in favour of accelerating the development of “cognitive vehicle” features, which will integrate intelligent technology and human factors for providing non-distracting interface for safety, efficiency and environmental sustainability in driving. In addition, infotainment capability is a desirable feature of the vehicle of the future, provided that it does not add to driver distraction. Further, these features are expected to be a stepping-stone to fully autonomous driving. This paper describes advances in driver-vehicle interface and presents highlights of research in design. Specifically, the design features of the cognitive vehicle are presented and measures needed to address major issues are noted. The application of Bayesian Artificial Intelligence is described in the design of driving assistance system, and design considerations are advanced in order to overcome issues in in-vehicle telematics systems. Finally, conclusions are advanced based on coverage of research material in the paper. <s> BIB037 </s> A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety <s> IV. SLR FINDINGS ORIENTED BY AN AV SYSTEM MODEL <s> The inflection point in the development of some core technologies enabled the Autonomous Vehicles (AV). The unprecedented growth rate in Artificial Intelligence (AI) and Machine Learning (ML) capabilities, focusing only on AVs, is expected to shift the transportation paradigm and bring relevant benefits to the society, such as accidents reduction. However, recent AVs accidents resulted in life losses. This paper presents a viewpoint discussion based on findings from a preliminary exploratory literature review. It was identified an important misalignment between AI and Safety research communities regarding the impact of AI on the safety risks in AV. This paper promotes this discussion, raises concerns on the potential consequences and suggests research topics to reduce the differences between AI and system safety mindsets. <s> BIB038
In the previous section, the state of the art in the literature about AI on AV safety was identified and investigated by means of a SLR. Six research questions oriented the literature identification, in which studies that include keywords related to safety, AI and AV were considered. The resulting studies were investigated and mapped into 6 categories: Impact (increase or decrease safety risks), Topics (sensors and perception; navigation and control; fault prevention; conceptual model and framework; human factor; etc.), Techniques (general AI/ML; ANN; SVM; etc.), Problem (AV validation; road detection; collision avoidance; etc.), Findings, and Future Studies. These results considered the AV as a system, but its specific components and functions in an architectural point-of-view were not considered. For deepening the understanding about the state of the art of AI on AV safety it is necessary to show how the presented works are applied/fitted on AV in an architectural point-of-view. In other words, which of AV modules/components and functions are already being developed and which one could be more explored. In order to achieve this goal, it is going to be considered the AV architecture proposed in BIB013 . An automotive manufacturer consortium (CAMP-AVR) BIB013 proposed a high-level architecture considering the main system components demanded for the vehicle movement control, to be used in the deployment of future Dynamic Driving Tasks (DDT). Figure 4 (left) illustrates the model considering a traditional vehicle (i.e. human operation with no automation deployed), and Figure 4 (right) illustrates the introduction of some level of machine automation (hybrid) in Sensors, Controller and Actuators elements. While the diagram considering the human operation can solely be mapped to the SAE Automation Level 0 (no automation), the hybrid one encompassing machine automation with human-in-the-loop can be mapped to the SAE Automation Levels 1 to 4 (semiautonomous) BIB013 . In this context, a modified version of the semi-autonomous model is proposed here ( Figure 5 ) including the system boundary. Also, the human related components were grouped as one single component (human-in-the-loop), which interacts with Machine Actuators, Machine Control, Machine Perception and Environment. A single component represents a more realistic approach facing the complexity added by the human in the system and allows the examination of the user actions and interactions as suggested by BIB003 . Also, it supports a necessary human-centered and holistic view BIB004 to better support the complexity of the human behavior and its interaction to the system. It avoids the misconceptions of the too logical designs from some engineering designs and helps to consider and accept human behavior the way it is, not the way engineers would wish it to be  . In fact, this is a necessary upgrade considering the original model is derived by the classical view from the automation engineering for industrial applications, where the environment was under control of the system designer, the human interactions had a considerable narrower scope, and its potential impact to the whole system were much lower, when compared to its application to the semi-autonomous vehicles. As a result, the proposed DDT version ( Figure 5 ) can be used to map the selected scientific literature. Therefore, it can provide a concise perspective on how the field literature covers those main components and which the uncovered areas are. Also, it can provide a good overview on the predominance of the papers valence (increase or decrease) on safety. Table IV shows how CT.1 (Impact) and CT.2 (Topic) codes are mapped to the components of the modified semiautonomous system model, as well as the relationship between CT.1 and CT.2. Most of the papers are related to machine perception, followed by papers related to a broad system view. Then, the next largest group of papers is related to the machine control component. The remaining papers are related to the human-in-the-loop aspects. An interesting aspect is that only the studies with a broad system aspect were found to have both CT.1 codes (increase and decrease system safety). Basically, the studies focused on distinct components solely understand AI can increase the safety risk. Therefore, there is a lack of studies with a critical mindset that explore the potential negative impacts of AI on the individual components. Finally, no papers were found related to the vehicle, machine actuators or environment. Table V shows how the wide range of AI techniques (CT.3 code) is mapped to the components of the modified semiautonomous system model. The AI techniques are grouped around their scope: system-oriented (32%, 19) and componentoriented (68%, 40). When a paper uses a combination of techniques, for example, ANN and SVM, it results into a unit added to the total number of papers using ANN and a unit added to the total number of papers using SVM. In this context, most of the studies (63%, 12) related to system-wide scope referred to general AI/MI. Most of the studies (20%, 11) related to machine perception used ANNs. In fact, ANN, SVM and HMM (Hidden Markov Model) account for 48% of the studies related to machine perception. Fuzzy logic (18%, 3) is the most widely used technique in the machine control-related papers. Fuzzy Logic, SVM, Optimization Heuristics and Ramer-DouglasPeucker or Ramert Douglas algorithm account for 53% of the studies related to machine control. Finally, Bayesian Artificial Intelligence techniques are used in most of the studies (29%) related to human-in-the-loop. Many different combinations of ANNs with other techniques were found (7 papers). As shown in Table VII most of those papers are related to Sensors and Perception (3 papers) as well as Navigation and Control (2 papers). Also, papers related to Conceptual Model and Framework and Fault Prevention employed hybrid approach (2 papers). The papers that used a combination of models associated to Hidden Markov Based Models were related to Navigation and Control as well as Sensors and Perception. The paper that used Hough Transformation combined to other models is related to Navigation and Control. The paper that employed a combination of techniques to propose a Novel Image Recognition Technique is related to Sensors and Perception. The paper using Regression-Based Models combined to other techniques is related to AV Navigation and Control. Finally, all the papers employing SVM combined to other techniques were related to the topic Sensors and Perception. The same grouping strategy applied to Table V (system-oriented and componentoriented) can be applied here to evaluate the problems (CT.4). System-level problems include 16 papers: AV Validation BIB027 , Machine-learning-based systems validation to the ultradependable levels required for AV  , Human and Machine Driver Co-existence BIB028 , Coexistence Human Machine Controller  , Driving Car Tasks Classification BIB029 , Lack of efficient Safety Performance Verification technique when AI/ML is used BIB019 , Crash assignment, especially between automated vehicles and non-automated vehicles BIB014 , Reduce the uncertainty of a driver behavior prediction model BIB020 , Investigate three underexplored themes for AV research: safety, interpretability, and compliance BIB030 , How vehicle autonomy technology can be used to benefit car drivers and also to propose a concept of an autonomous highway vehicle which improves highway driving safety BIB002 , AV decisions in complex dilemmas as a social agent  , Hybrid (humans and machines) collective decision-making systems BIB021 , Autonomy assurance and trust in Automated Transportation Systems BIB010 , AV Test BIB031 and, Evaluate the impacts of the number of highly automated vehicles on future traffic safety and traffic flow BIB011 .  , Road environmental recognition and various object detection in real driving conditions BIB022 , Obstacle clustering and tracking [23] . For an autonomous behavior, each truck must be able to follow the vehicle ahead. Due to that, each vehicle must be able to recognize the leading vehicle [22] , Speed bump detection BIB015 , providing road safety to connected drivers and connected autonomous vehicles  , how to "automate" manual annotation for images to train visual perception for AVs BIB032 , Road Sign Classification in Real-time BIB033 , Road Terrain detection BIB034 , Spatio-temporal situation awareness BIB035 , Pedestrian detection and movement direction recognition [26] , Pedestrian Trajectory Prediction  , Road junction detection BIB023 , Cyber Attack in V2X BIB024 , Learn from Demonstration BIB016 , Early detection of faults or malfunction  , Road and Obstacle Detection [24] and Enhance Image Understanding BIB001 . The problems related to Machine Control were found in 17 papers (22%). Those problems include: Pre-Crash problem of Intelligent Control of autonomous vehicles robot BIB025 , Safeoptimal trajectory selection for autonomous vehicle BIB026 , Driverless car 100 km experiment BIB017 , Robot maneuvers too close to an obstacle, which increases the probability of an accident. Preventing this is crucial in dynamic environments, where the obstacles, such as other UAVs, are moving BIB006 , Learning and simulation of the Human-Level decisions involved in driving a racing car BIB005 , Control intersection crossing (all way stop) and optimizing it  , How to prove the correctness of an algorithm for Vehicle Coordination BIB007 , Path tracking  , Drivers maneuver classification BIB032 , AVs intersections crossing optimization BIB008 and Manage low level vehicle actuators (steering throttle and brake) BIB009 . Finally, the problems related to Human-in-the-loop new DDT component ( Figure 5 ) are present in 7 papers (10%). Those problems include: Selective Attention Mechanism BIB036 , Developing remote controlled car with some automation to deal with traffic light detection, obstacle avoidance system and lane detection system to be driven from anywhere over a secured internet connection  , Collision avoidance when no action is taken by driver to avoid the collision BIB012 , Human drivers monitoring system to ensure they will be able to take over control within short notice BIB018 and, Design of driving assistance system BIB037 . This seems to be an attention-point; this problem category can be considered one serious challenge to semi-autonomous vehicles (SAE Level 1 to Level4). Therefore, more research is needed into this topic because only 6 papers were found. V. FINAL REMARKS Machine Perception has more studies with practical results. Considering the other components, few studies with practical results from real deployments were found. Most of the papers presented preliminary results. In fact, some papers start with a promise and finish with more promises. Considering only 24% of the total papers considered in this study were published by journals, it is possible to conclude the field is not mature yet. Some similar issues were studied in more than one paper about Machine Perception, and distinct techniques were applied to address them (for example, ANN and SVM applied to similar issues as well issues as well distinct techniques applied to the topic cyber-attack). Considering some of those techniques have different working mechanisms, that fact can be an important finding for the safety of autonomous cars as regards the need of redundant components. Similar issues being addressed by different techniques were not identified for Machine Control. The papers related to Navigation and Control also reported positive and promising results, although the level of maturity of the achievements are clearly much lower than the sensors and perception as well as far from what would be expected for an autonomous vehicle considering the potential hazardous situations it may face. In fact, most of the results presented are preliminary. Only few of the studies related to system described practical results from real deployments. The papers proposing conceptual models and frameworks bring important contributions, but they are mostly not tested in real set-ups. There is thus a lack of reported results from models and frameworks that could build the foundation of AVs safety. Also, few human-in-the-loop studies had practical results from real deployments. However, they seem to be one of the most important topics seeing that there will be more semiautonomous cars than fully autonomous ones for a while, and they will co-exist. The human factor will thus be an important variable in the system to be considered not only as the impacted side of the safety, but as one of the sources of interactions influencing the safety levels. The topic requires multidisciplinary studies involving fields beyond engineering and computer science, such as neurosciences. This shows the field is not mature yet. The amplitude and range of the reported future researches in the papers reviewed suggest that there is an empty space for new research into this field. For example, only few studies were found about the three topics positioning AI as a potential source of negative impact on safety -Fault Forecasting, Ethics and Policies, and Dependability and Trust. When combined to the other findings reported by the present study, it confirms the impressions formed during an exploratory research of the literature BIB038 . It reinforces the perception that the field of AI and AV is not heavily influenced by the safety engineering culture yet. In fact, the studies published about this current topic seem to be more driven by computation-related domains, with no tradition regarding safety culture, than other fields that are much more connected to safety in critical systems BIB038 . Additional research is necessary for most of the studies reviewed. They need to be extended to be tested in simulated or real set-ups, new and broader scenarios, with new and more data, and consider experimental designs whereby the results from the proposed approach are compared to benchmarks and alternative techniques. Many AI techniques have achieved impressive results; however, it is still arguable whether the error rates are suitable for real deployments in AVs under the light of a (missing) hazard analysis. Therefore, additional studies with improvements in those techniques are required. Finally, a stronger influence of safety engineering on most of the studies would benefit the field. A research agenda must consider a serious safety agenda for future studies, at system-level, component-level and AI technique-level. In fact, there are some topics related to safety concerns over AVs, which are critical-path to the development of the field. Some of the suggested topics are related to the challenges with validating machine-learning-based systems to the ultra-dependable levels required for AVs; wider and deeper studies about human-machine collaboration in the context of AVs; autonomy assurance and trust in AVs; ethical and moral decisions in the context of AVs; among other topics, from Validating machine-learning-based systems to the ultradependable levels required for AVs; and autonomy assurance and trust in AVs seem to be the holy grail towards a fully autonomous AV -SAE level 5 . They are also key topics for the Safety Certification of non-deterministic control systems. In those contexts, there are many gaps to be filled by future researches, such as AVs software testing, Fault Injection Testing for AI on AVs, Failure Modes and Effects Analysis (FMEA) for AI on AVs, AI safeguards for AVs, AI safety envelopes for AVs, AI redundancy for AVs (many possible approaches, such as a hybrid connectionist and symbolic architecture using causal inference), explainable AI for AVs, AI fault forecasting. Finally, studies on V2X communication can help autonomy assurance by providing channels for hardware and software redundancy. Human-machine collaboration in the context of AVs is another key topic with special impact on the semi-autonomous vehicles (SAE levels 1 to 4). Investigations on the best way humans and AVs can interact during normal operations and facing hazardous situations are needed to meet the adequate safety requirements the semi-autonomous vehicles must have. Those studies must consider hybrid collective decision-making systems to enable humans and machines to work together and to agree on common decisions, as well as how to deal with the lack of agreement in some situations. Also, there is another important discussion arising in the context of human-machine collaboration that must be investigated. On the one hand, there are reports about advanced driver assistant technologies that failed (such as Tesla Autopilot) and the driver was not able to react in time to avoid the accident. They ended-up in life losses and property damages. On the other hand, there are reports about situations in which the advanced driver assistant technologies saved the drivers' life by automatically taking the driver suffering a heart attack to the hospital; fully controlling the car with a drunk driver sleeping; and using a defensive lane change  maneuver to avoid being hit by a truck changing its lane. Some players in the industry are pushing the automation evolution steps towards full automation by requiring the human driver to be a backup to the automated driver. Other players in the industry believe the automated driver must be a backup to the human driver. It looks like the second approach can be a smoother and safer path towards a SAE level-5 automation. Immersive environments for training and testing AVs represent another research trend. As the underlying technologies supporting AVs development evolve, higher automation-levels become possible. Considering the potential hazards until the AVs are well trained and fine-tuned, the immersive technologies are becoming an important tool to support the development, training and tests of fully autonomous machines. Another broad topic requiring further research is related to ethical and moral decisions in AVs. Some studies only mention issues related to moral dilemmas while others provide some simple experiments involving simulated environments and/or human interviews. However, they misinterpret important concepts and bring the discussions around the decisions AVs must make when life losses are involved, besides the moral and ethical perceptions from the human perspective. All of them miss important points such as statistical considerations and the societal result. In other words, the discussions are not deep enough as regards situations such as whether an AV should hit an old man or a child, while a true safe machine control should consider all the probabilities involved and select the one that minimizes the chances of life losses instead of just picking an option. For example, the system must consider small signals, such as which of the potential victims is paying attention to the approaching AV and what would their potential reaction be and effectiveness of it based on the age and other metrics, as well, considering the multiple scenarios, and the configuration of each, such as speed, region of the car hitting which region of each victim, the potential damages and the severity of the damages considering the estimated weight and overall physical condition, to decide based on the minimization of chances of life losses. This approach will result into higher safety levels for society. Finally, only 1 paper about autonomous truck was found. Considering some specificities of autonomous truck and its risks, at least a few more studies about the topic could be expected. The 