Intelligent Transportation Systems (ITS) have attracted the attention of researchers and the general public alike as a means to alleviate traffic congestion. Recently, the maturity of wireless technology has enabled a cost-efficient way to achieve ITS by detecting vehicles using Vehicle to Infrastructure (V2I) communications. Traditional ITS algorithms, in most cases, assume that every vehicle is observed, such as by a camera or a loop detector, but a V2I implementation would detect only those vehicles with wireless communications capability. We examine a family of transportation systems, which we will refer to as Partially Detected Intelligent Transportation Systems'. An algorithm that can act well under a small detection rate is highly desirable due to gradual penetration rates of the underlying wireless technologies such as Dedicated Short Range Communications (DSRC) technology. Artificial Intelligence (AI) techniques for Reinforcement Learning (RL) are suitable tools for finding such an algorithm due to utilizing varied inputs and not requiring explicit analytic understanding or modeling of the underlying system dynamics. In this paper, we report a RL algorithm for partially observable ITS based on DSRC. The performance of this system is studied under different car flows, detection rates, and topologies of the road network. Our system is able to efficiently reduce the average waiting time of vehicles at an intersection, even with a low detection rate.
Bingham proposed RL for parameter search of a fuzzy-neural traffic signal controller for a single intersection @cite_49 , while adapted RL on the fuzzy-neural system in a cooperative scheme, achieving adaptive control for a large area @cite_33 . These algorithms are based on RL, but the major goal of RL is parameter tuning of the fuzzy-neural system. proposed the first true adaptive traffic signal which learns to control the traffic signal dynamically based on a Cerebellar Model Articulation Controller (CMAC), as a Q-estimation network @cite_35 . Silva, et.al. and Oliveira et.al. then proposed a context-detector (CD) in conjunction with RL to further improve the performance under non-stationary traffic situations @cite_52 @cite_44 . Several researchers have focused on multi-agent reinforcement learning for implementing it on a large scale @cite_74 @cite_3 @cite_26 @cite_27 .
@cite_35 @cite_26 @cite_33 @cite_52 @cite_3 @cite_44 @cite_27 @cite_49 @cite_74
2126226307 1986659999 2548134372 2078895652
This research develops and evaluates a new multi-agent adaptive traffic signal control system based on swarm intelligence and the neural-fuzzy actor-critic reinforcement learning (NFACRL) method. The proposed method combines the better attributes of swarm intelligence and the NFACRL method. Two scenarios are used to evaluate the method and the new NFACRL-Swarm method is compared with its NFACRL counterpart. First, the proposed control model is applied to isolated intersection signal adaptive control to evaluate its learning performance. Then, the control system is implemented in signal control coordination in a typical arterial. In the isolated intersection, the proposed hybrid method outperforms its previous counterpart by improving the learning speed and is shown to be insensitive to reward function parameters. In the network, by introducing a coordination scheme inspired by swarm intelligence, the proposed method improves the performance by up to 12 and has a faster learning speed. Abstract A fuzzy traffic signal controller uses simple “if–then” rules which involve linguistic concepts such as medium or long , presented as membership functions. In neurofuzzy traffic signal control, a neural network adjusts the fuzzy controller by fine-tuning the form and location of the membership functions. The learning algorithm of the neural network is reinforcement learning, which gives credit for successful system behavior and punishes for poor behavior; those actions that led to success tend to be chosen more often in the future. The objective of the learning is to minimize the vehicular delay caused by the signal control policy. In simulation experiments, the learning algorithm is found successful at constant traffic volumes: the new membership functions produce smaller vehicular delay than the initial membership functions. Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82 , average queue length by 66 and average travel time by 20 . In many urban areas where traffic congestion does not have the peak pattern, conventional traffic signal timing methods does not result in an efficient control. One alternative is to let traffic signal controllers learn how to adjust the lights based on the traffic situation. However this creates a classical non-stationary environment since each controller is adapting to the changes caused by other controllers. In multi-agent learning this is likely to be inefficient and computationally challenging, i.e., the efficiency decreases with the increase in the number of agents (controllers). In this paper, we model a relatively large traffic network as a multi-agent system and use techniques from multi-agent reinforcement learning. In particular, Q-learning is employed, where the average queue length in approaching links is used to estimate states. A parametric representation of the action space has made the method extendable to different types of intersection. The simulation results demonstrate that the proposed Q-learning outperformed the fixed time method under different traffic demands.