Making the right decision in traffic is a challenging task that is highly dependent on individual preferences as well as the surrounding environment. Therefore it is hard to model solely based on expert knowledge. In this work we use Deep Reinforcement Learning to learn maneuver decisions based on a compact semantic state representation. This ensures a consistent model of the environment across scenarios as well as a behavior adaptation function, enabling on-line changes of desired behaviors without re-training. The input for the neural network is a simulated object list similar to that of Radar or Lidar sensors, superimposed by a relational semantic scene description. The state as well as the reward are extended by a behavior adaptation function and a parameterization respectively. With little expert knowledge and a set of mid-level actions, it can be seen that the agent is capable to adhere to traffic rules and learns to drive safely in a variety of situations.
Initially most behavior planners were handcrafted state machines, made up by a variety of modules to handle different tasks of driving. During the DARPA Urban Challenge Boss (CMU) for example used five different modules to conduct on road driving. The responsibilities of the modules ranged from lane selection, merge planning to distance keeping @cite_10 . Other participants such as Odin (Virginia Tech) or Talos (MIT) developed very similar behavior generators @cite_1 @cite_27 .
@cite_1 @cite_27 @cite_10
2121806728 2127005930 1919962963 1893920468
Boss is an autonomous vehicle that uses on-board sensors (global positioning system, lasers, radars, and cameras) to track other vehicles, detect static obstacles, and localize itself relative to a road model. A three-layer planning system combines mission, behavioral, and motion planning to drive in urban environments. The mission planning layer considers which street to take to achieve a mission goal. The behavioral layer determines when to change lanes and precedence at intersections and performs error recovery maneuvers. The motion planning layer selects actions to avoid obstacles while making progress toward local goals. The system was developed from the ground up to address the requirements of the DARPA Urban Challenge using a spiral system development process with a heavy emphasis on regular, regressive system testing. During the National Qualification Event and the 85-km Urban Challenge Final Event, Boss demonstrated some of its capabilities, qualifying first and winning the challenge. © 2008 Wiley Periodicals, Inc. The DARPA Urban Challenge required robotic vehicles to travel more than 90 km through an urban environment without human intervention and included situations such as stop intersections, traffic merges, parking, and roadblocks. Team VictorTango separated the problem into three parts: base vehicle, perception, and planning. A Ford Escape outfitted with a custom drive-by-wire system and computers formed the basis for Odin. Perception used laser scanners, global positioning system, and a priori knowledge to identify obstacles, cars, and roads. Planning relied on a hybrid deliberative-reactive architecture to analyze the situation, select the appropriate behavior, and plan a safe path. All vehicle modules communicated using the JAUS (Joint Architecture for Unmanned Systems) standard. The performance of these components in the Urban Challenge is discussed and successes noted. The result of VictorTango's work was successful completion of the Urban Challenge and a third-place finish. © 2008 Wiley Periodicals, Inc. Recently, automated driving has more and more been transformed from an exciting vision into hands on reality by prototypes. While drivers are used to assistance and maybe even automation for driving within a lane, it is exciting to dare a step ahead: Deciding and executing tactical maneuvers like lane changes in automated vehicles without any human interaction. In this paper, we present our approach for tactical behavior planning for lane changes. We present a way to tackle perception uncertainties and how to achieve provident, prediction-based behavior planning. For this, we introduce a novel framework to plan in high-dimensional, mixed-integer state spaces in real-time. Our approach is evaluated not only in simulation, but also in real traffic. The implementation has recently been demonstrated to the public in the Audi A7 piloted driving concept vehicle, driving from Stanford to the Consumer Electronics Show (CES) 2015 in Las Vegas. Automated driving within a lane is a fascinating experience already. However, more exiting but also technically more challenging is to dare the next step of automating tactical behavior decisions for lane changes, as well. In this paper, we present our approach for situation assessment in tactical behavior planning for lane changes, whether lane changes are beneficial and or possible. We present a way to tackle perception uncertainties and how to monitor the system's abilities and current skills. This is achieved by a dynamic Bayesian network and an unscented variance transform. Our approach is evaluated not only in a simulation, but also in real traffic. Our implementation has recently been demonstrated to the public in the Audi A7 piloted driving concept vehicle, driving 550 miles from Stanford to Las Vegas to the Consumer Electronics Show (CES) 2015.