Although traditional control theory and supervised learning applied to path planning of autonomous driving have been investigated since the last century [1]}, [2]}, [3]}, reinforcement learning (RL) has recently demonstrated powerful performance, with a basis of enabling a vehicle to learn optimal control policies by interacting with the environment, without the need for prior knowledge or a large amount of training data [4]}. Furthermore, deep reinforcement learning (DRL) models have achieved some notable successes [5]}. For example, AlphaGo [6]} defeated a human professional in the board game Go in 2015, which was the first time a computer Go program had defeated a professional human player in history. The deep Q network (DQN) [7]}, an off-policy method that adds a target network for stability and prioritised experience replay for efficient sampling, demonstrated high efficiency in playing 2600 Atari video games and outperformed humans just by learning from raw image pixels. Furthermore, advantage actor-critic (A2C) [8]}, as another off-policy method that uses advantage estimates to calculate the value proposition for each action state pair, is a lightweight framework that uses the synchronised gradient to update that keeps the training more cohesive and potentially makes convergence faster. In addition, proximal policy optimisation (PPO) [9]} is an on-policy method that provides better convergence with simplicity to implement and tune in most benchmark RL testing environments and was applied to traffic oscillations [10]} and hierarchical guidance competitions [11]}, [12]}. These DRL research initiatives have stimulated more recent research areas on the application of DRL in autonomous driving and provided promising solutions in predictive perception, path and motion planning, driving strategies, and low-level controller design [5]}. However, they paid too much attention to evaluating the reward coverage returns and ignored the crucial safety factors in autonomous driving.
