Recent advances in combining deep learning and Reinforcement Learning have shown a promising path for designing new control agents that can learn optimal policies for challenging control tasks. These new methods address the main limitations of conventional Reinforcement Learning methods such as customized feature engineering and small action state space dimension requirements. In this paper, we leverage one of the state-of-the-art Reinforcement Learning methods, known as Trust Region Policy Optimization, to tackle intersection management for autonomous vehicles. We show that using this method, we can perform fine-grained acceleration control of autonomous vehicles in a grid street plan to achieve a global design objective.
A decentralized optimal control formulation is proposed in @cite_1 in which the acceleration deceleration of the vehicles are minimized subject to collision avoidance constraints.
@cite_1
2904365072 2167485994 2170942374 2085865324
In earlier work, we established a decentralized optimal control framework for coordinating online connected and automated vehicles (CAVs) in specific transportation segments, e.g., urban intersections, merging roadways, roundabouts, and speed reduction zones. In this paper, we address coordination of CAVs in a corridor with multiple such scenarios and derive a closed-form analytical solution that includes interior boundary conditions. We evaluate the effectiveness of the solution through simulation in VISSIM. The proposed approach reduces significantly both fuel consumption and travel time for the CAVs compared to the baseline scenario where traditional human-driven vehicles without control are considered. This paper describes decentralized control laws for the coordination of multiple vehicles performing spatially distributed tasks. The control laws are based on a gradient descent scheme applied to a class of decentralized utility functions that encode optimal coverage and sensing policies. These utility functions are studied in geographical optimization problems and they arise naturally in vector quantization and in sensor allocation tasks. The approach exploits the computational geometry of spatial structures such as Voronoi diagrams. This article describes the use of predictive control for the decentralized cooperative control of unmanned aerial vehicles in an unfamiliar three-dimensional environment. It is assumed that each vehicle is equipped with an autopilot and a trajectory control unit. The autopilot insures the stability of the vehicle. The setpoints of the autopilot are calculated by the trajectory control unit, thus forming a cascade control structure. The trajectory control unit relies on a predictive control algorithm to calculate the optimal commands (autopilot setpoints) such that the vehicle will reach fixed targets at known positions while avoiding static obstacles that are detected en route. The advantage in using predictive control is that it offers great flexibility in the objective function to optimize while respecting constraints such as command limits, limits on the displacement that a vehicle can carry out, and the constraints that allow obstacle avoidance. The principle proposed in order to avoid static obstacles (that are assumed ellipsoid) is to verify that the predicted vehicle trajectory does not intersect these obstacles. With the objective to increase performance, cooperation between vehicles must also be privileged. Thus, if some vehicles are within the communication range, they can share the position and shape of the obstacles they have detected. Simulations illustrate the method and higlight the benefits of cooperation. We consider the problem of synthesising the least restrictive controller for collision avoidance of multiple vehicles at an intersection. The largest set of states for which there exists a control that avoids collisions is known as the maximal controlled invariant set. Exploiting results from the scheduling literature we prove that, for a general model of vehicle dynamics at an intersection, the problem of checking membership in the maximal controlled invariant set is NP-hard. We then describe an algorithm that solves this problem approximately and with provable error bounds. The approximate solution is used to design a supervisor for collision avoidance whose complexity scales polynomially with the number of vehicles. The supervisor is based on a hybrid algorithm that employs a dynamic model of the vehicles and periodically solves a scheduling problem.