Increased stress, fuel consumption, air pollution, accidents and delays are some of the consequences of traffic congestion usually incurring in tremendous economic impacts, which society aims to remedy in order to leverage a sustainable development. Recently, unconventional means for modeling and controlling such complex traffic systems relying on multi-agent systems have arisen. This paper contributes to the understanding of such complex and highly dynamic systems by proposing an open-source tool-chain to implement multi-agent-based solutions in traffic and transportation. The proposed approach relies on two very popular tools in both domains, with focus on traffic light control. This tool-chain consists in combining JADE (Java Agent DEvelopment Framework), for the implementation of multi-agent systems, with SUMO (Simulation of Urban MObility), for the microscopic simulation of traffic interactions. TraSMAPI (Traffic Simulation Manager Application Programming Interface) is used to combine JADE and SUMO allowing communication between them. A demonstration of the concept is presented to illustrate the main features of this tool-chain, using Q-Learning as the reinforcement learning method for each traffic light agent in a simulated network. Results demonstrate the feasibility of the proposed framework as a practical means to experiment with different agent-based designs of intelligent transportation solutions.
To date, there are not many solutions for traffic that make full use of the intelligent agent concept. However, the multi-agent system approach has become recognized as a convenient approach for modelling and simulating complex systems @cite_19 . Also, it has grown enormously not only applied to traffic but also to transportation in general terms @cite_20 .
@cite_19 @cite_20
2028223672 2740264533
Advances in autonomous vehicles and intelligent transportation systems indicate a rapidly approaching future in which intelligent vehicles will automatically handle the process of driving. However, increasing the efficiency of today's transportation infrastructure will require intelligent traffic control mechanisms that work hand in hand with intelligent vehicles. To this end, Dresner and Stone proposed a new intersection control mechanism called Autonomous Intersection Management (AIM) and showed in simulation that by studying the problem from a multiagent perspective, intersection control can be made more efficient than existing control mechanisms such as traffic signals and stop signs. We extend their study beyond the case of an individual intersection and examine the unique implications and abilities afforded by using AIM-based agents to control a network of interconnected intersections. We examine different navigation policies by which autonomous vehicles can dynamically alter their planned paths, observe an instance of Braess' paradox, and explore the new possibility of dynamically reversing the flow of traffic along lanes in response to minute-by-minute traffic conditions. Studying this multiagent system in simulation, we quantify the substantial improvements in efficiency imparted by these agent-based traffic control methods. The operation of large dynamic systems such as urban traffic networks remains a challenge in control engineering to a great extent due to their sheer size, intrinsic complexity, and nonlinear behavior. Recently, control engineers have looked for unconventional means for modeling and control of complex dynamic systems, in particular the technology of multi-agent systems whose appeal stems from their composite nature, flexibility, and scalability. This paper contributes to this evolving technology by proposing a framework for multi-agent control of linear dynamic systems, which decomposes a centralized model predictive control problem into a network of coupled, but small sub-problems that are solved by the distributed agents. Theoretical results ensure convergence of the distributed iterations to a globally optimal solution. The framework is applied to the signaling split control of traffic networks. Experiments conducted with simulation software indicate that the multi-agent framework attains performance comparable to conventional control. The main advantages of the multi-agent framework are its graceful extension and localized reconfiguration, which require adjustments only in the control strategies of the agents in the vicinity. Considering that the transport sector is responsible for an increasingly important share of current environmental problems, we look at Intelligent Transportation Systems (ITS) as a feasible means of helping in solving this issue. In particular, we evaluate the impact in terms of Carbon Dioxide (CO2)emissions of Virtual Traffic Light (VTL), which is a recently proposed infrastructureless traffic control system solely based on Vehicle-to-Vehicle (V2V) communication. Our evaluation uses a real-city scenario in a complex simulation framework, involving microscopic traffic, wireless communication, and emission models. Compared with an approximation of the physical traffic light system deployed in the city, our results show a significant reduction on CO2 emissions when using VTLs, reaching nearly 20 under high-density traffic. Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82 , average queue length by 66 and average travel time by 20 . One way to cope with the increasing demand in transportation networks is to integrate standard solutions with more intelligent measures. This problem has been approached from different sides such as the study of the assignment of the demand in the network, and the investigation of the effects of control measures. However, given that most of these approaches are complex and deal with different levels of abstraction of the original problem, there has been few attempts to address both demand and control in a single tool. This paper presents an overview of ITSUMO, a microscopic traffic simulator whose implementation uses agent technologies with a bottom-up philosophy in mind. We give an overview of the system and some details of its modules (simulation kernel, data management, control, driver and routing, and visualization), followed by an example of its use. I. I NTRODUCTION AND MOTIVATION The ability to exert real-time, adaptive control of transportation processes is the core of many intelligent transportation systems decision support tools. Reinforcement learning, an artificial intelligence approach undergoing development in the machine- learning community, offers key advantages in this regard. The ability of a control agent to learn relationships between control actions and their effect on the environment while pursuing a goal is a distinct improvement over prespecified models of the environment. Prespecified models are a prerequisite of conventional control methods and their accuracy limits the performance of control agents. This paper contains an introduction to Q-learning, a simple yet powerful reinforcement learning algorithm, and presents a case study involving application to traffic signal control. Encouraging results of the application to an isolated traffic signal, particularly under variable traffic conditions, are presented. A broader research effort is outlined, including extension to linear and networked signal systems and integration with dynamic route guidance. The research objective involves optimal control of heavily congested traffic across a two-dimensional road network—a challenging task for conventional traffic signal control methodologies. Traffic congestion is one of the leading causes of lost productivity and decreased standard of living in urban settings. Recent advances in artificial intelligence suggest vehicle navigation by autonomous agents will be possible in the near future. In this paper, we propose a reservation-based system for alleviating traffic congestion, specifically at intersections, and under the assumption that the cars are controlled by agents. First, we describe a custom simulator that we have created to measure the different delays associated with conducting traffic through an intersection. Second, we specify a precise metric for evaluating the quality of traffic control at an intersection. Using this simulator and this metric, we show that our reservation-based system can perform two to three hundred times better than traffic lights. As a result, it can smoothly handle much heavier traffic conditions. We show that our system very closely approximates an overpass, which is the optimal solution for the problem with which we are dealing. In many urban areas where traffic congestion does not have the peak pattern, conventional traffic signal timing methods does not result in an efficient control. One alternative is to let traffic signal controllers learn how to adjust the lights based on the traffic situation. However this creates a classical non-stationary environment since each controller is adapting to the changes caused by other controllers. In multi-agent learning this is likely to be inefficient and computationally challenging, i.e., the efficiency decreases with the increase in the number of agents (controllers). In this paper, we model a relatively large traffic network as a multi-agent system and use techniques from multi-agent reinforcement learning. In particular, Q-learning is employed, where the average queue length in approaching links is used to estimate states. A parametric representation of the action space has made the method extendable to different types of intersection. The simulation results demonstrate that the proposed Q-learning outperformed the fixed time method under different traffic demands. This paper presents the application of fuzzy-neuro-evolutionary hybrid system with online reinforcement learning for intelligent road traffic management and control. Taking a step away from the conventional traffic control system, the hybrid system presents different methodologies in knowledge acquisition, decisionmaking, learning and goal formulation with the use of a three-layered hierarchical, distributed agent architecture. Distributed and hierarchical fuzzy knowledge acquisition allows different levels of perception to be derived for the same traffic situation by the intelligent agents. Agents' perceptions can be changed with the use of online reinforcement learning. Initial experimental results show that the implementation of the hybrid agents in the traffic network generally yields better network performance when compared to a network without the agents. The probability of a traffic network evolving into pathological states with oversaturation is also reduced with the implementation of the agents. Reinforcement Learning was originally developed for Markov Decision Processes (MDPs). It allows a single agent to learn a policy that maximizes a possibly delayed reward signal in a stochastic stationary environment. It guarantees convergence to the optimal policy, provided that the agent can sufficiently experiment and the environment in which it is operating is Markovian. However, when multiple agents apply reinforcement learning in a shared environment, this might be beyond the MDP model. In such systems, the optimal policy of an agent depends not only on the environment, but on the policies of the other agents as well. These situations arise naturally in a variety of domains, such as: robotics, telecommunications, economics, distributed control, auctions, traffic light control, etc. In these domains multi-agent learning is used, either because of the complexity of the domain or because control is inherently decentralized. In such systems it is important that agents are capable of discovering good solutions to the problem at hand either by coordinating with other learners or by competing with them. This chapter focuses on the application reinforcement learning techniques in multi-agent systems. We describe a basic learning framework based on the economic research into game theory, and illustrate the additional complexity that arises in such systems. We also described a representative selection of algorithms for the different areas of multi-agent reinforcement learning research. Abstract Documents in computer-readable form can be used to provide information about other documents, i.e. those they cite. To do this efficiently requires procedures for computer recognition of citing statements. This is not easy, especially for multi-sentence citing statements. Computer recognition procedures have been developed which are accurate to the following extent: 73 of the words in statements selected by computer procedures as being citing statements are words which are correctly attributable to the corresponding documents. The retrieval effectiveness of computer-recognized citing statements was tested in the following way. First, for eight retrieval requests in inorganiic chemistry, average recall by search of Chemical Abstracts Service indexing and Chemical Abstracts abstract text words was found to be 50 . Words from citing statements referring to the papers to be retrieved were then added to the index terms and abstract words as additional access points, and searching was repeated. Average recall increased to 70 . Only words from citing statements published within a year of the cited papers were used. The retrieval effect of citing statement words alone (published within a year) without index or abstract terms was the following: average recall was 40 . When just the words of the titles of the cited papers were added to those citing statement words, average recall increased to 50 . Although drivers obtain road information through radio broadcasting or specific in-car equipment, there is still a wide gap between the synchronization of information and the actual conditions on the road. In the absence of adequate information, drivers often react to conditions with inefficient behaviors that do not contribute to their own driving goals, but increase traffic complication. Therefore, this study applies the features of information exchanged between ''Multi-Agents'' and mutual communication and collaboration mechanisms to intelligent transportation systems (ITS). If drivers could achieve distributed communication, share their driving information, and submit their own reasoned driving advice to others, many traffic situations will improve effectively. Additionally, the efficiency of the computing processes could have improved through distributed communication. At the same time, this paper proposes an architecture design, including vehicle components, OBU (On-Board Unit) devices and roadside device components (Roadside Unit) with hybrid architecture, which is intended to establish intelligent diversified road services to provide information support and applications. Purpose – The purpose of this paper is to suggest an alternative to the widely used Times Cited criterion for analysing citation networks. The approach involves taking account of the natures of the papers that cite a given paper, so as to differentiate between papers that attract the same number of citations.Design methodology approach – ArticleRank is an algorithm that has been derived from Google's PageRank algorithm to measure the influence of journal articles. ArticleRank is applied to two datasets – a citation network based on an early paper on webometrics, and a self‐citation network based on the 19 most cited papers in the Journal of Documentation – using citation data taken from the Web of Knowledge database.Findings – ArticleRank values provide a different ranking of a set of papers from that provided by the corresponding Times Cited values, and overcomes the inability of the latter to differentiate between papers with the same numbers of citations. The difference in rankings between Times Cited ... In this paper we introduce a multi-faceted trust model of use for the application of ad-hoc vehicular networks (VANETs) – scenarios where agents representing drivers exchange information with other drivers regarding road and traffic conditions. We argue that there is a need to model trust in various dimensions and that combining these elements effectively can assist agents in making transportation decisions. We then introduce two new elements to our proposed model: i) distinguishing direct and indirect reports that are shared ii) employing a penalty for misleading reports, to promote honesty. We demonstrate how these two elements together serve to increase the value of the trust model, through a series of experiments of simulated traffic. In brief, we present a framework to facilitate the effective sharing of information in VANET environments between agents representing the vehicles. In order to improve the veracity and reliability of a traffic model built, or to extract important and valuable information from collected traffic data, the technique of outlier mining has been introduced into the traffic engineering domain for detecting and analyzing the outliers in traffic data sets. Three typical outlier algorithms, respectively the statistics-based approach, the distance-based approach, and the density-based local outlier approach, are described with respect to the principle, the characteristics and the time complexity of the algorithms. A comparison among the three algorithms is made through application to intelligent transportation systems (ITS). Two traffic data sets with different dimensions have been used in our experiments carried out, one is travel time data, and the other is traffic flow data. We conducted a number of experiments to recognize outliers hidden in the data sets before building the travel time prediction model and the traffic flow foundation diagram. In addition, some artificial generated outliers are introduced into the traffic flow data to see how well the different algorithms detect them. Three strategies-based on ensemble learning, partition and average LOF have been proposed to develop a better outlier recognizer. The experimental results reveal that these methods of outlier mining are feasible and valid to detect outliers in traffic data sets, and have a good potential for use in the domain of traffic engineering. The comparison and analysis presented in this paper are expected to provide some insights to practitioners who plan to use outlier mining for ITS data. Artificial intelligence research is ushering in a new era of sophisticated, mass-market transportation technology. While computers can already fly a passenger jet better than a trained human pilot, people are still faced with the dangerous yet tedious task of driving automobiles. Intelligent Transportation Systems (ITS) is the field that focuses on integrating information technology with vehicles and transportation infrastructure to make transportation safer, cheaper, and more efficient. Recent advances in ITS point to a future in which vehicles themselves handle the vast majority of the driving task. Once autonomous vehicles become popular, autonomous interactions amongst multiple vehicles will be possible. Current methods of vehicle coordination, which are all designed to work with human drivers, will be outdated. The bottleneck for roadway efficiency will no longer be the drivers, but rather the mechanism by which those drivers' actions are coordinated. While open-road driving is a well-studied and more-or-less-solved problem, urban traffic scenarios, especially intersections, are much more challenging. We believe current methods for controlling traffic, specifically at intersections, will not be able to take advantage of the increased sensitivity and precision of autonomous vehicles as compared to human drivers. In this article, we suggest an alternative mechanism for coordinating the movement of autonomous vehicles through intersections. Drivers and intersections in this mechanism are treated as autonomous agents in a multiagent system. In this multiagent system, intersections use a new reservation-based approach built around a detailed communication protocol, which we also present. We demonstrate in simulation that our new mechanism has the potential to significantly outperform current intersection control technology--traffic lights and stop signs. Because our mechanism can emulate a traffic light or stop sign, it subsumes the most popular current methods of intersection control. This article also presents two extensions to the mechanism. The first extension allows the system to control human-driven vehicles in addition to autonomous vehicles. The second gives priority to emergency vehicles without significant cost to civilian vehicles. The mechanism, including both extensions, is implemented and tested in simulation, and we present experimental results that strongly attest to the efficacy of this approach. Urban simulations are an important tool for analyzing many policy questions relating to the usage of public space, roads, and communal transportation; they can be used to predict the long-term impact of new construction projects, traffic restrictions, and zoning laws. However, it is unwise to rely upon predictions from a single model since each technique possesses different strengths and weaknesses and can be highly sensitive to the choice of parameters and initial conditions. In this article, we describe a hybrid approach for combining agent-based and stochastic simulations (Markov chain Monte Carlo, MCMC) to improve the accuracy and reduce the variance of long-term predictions. In our proposed approach, the agent-based model is used to bootstrap the proposal distribution for the MCMC estimator. To demonstrate the applicability of our modeling technique, this article presents a case study describing the usage of our hybrid simulation method for forecasting transportation patterns and parking lot utilization on a large university campus. A comparison of our simulation results against an independently collected dataset reveals that our hybrid approach accurately predicts parking lot usage and performs significantly better than other comparable modeling techniques. Developing novel architectures for combining the predictions of agent-based models can produce insights that are different than simply selecting the best model. Over the last fifteen years, researchers in the Basic and Applied Simulation Science group at Los Alamos National Laboratory have worked with various collaborators on the modeling, simulation, and development of associated decision support tools for understanding large socio-technical systems. The extremely detailed multiscale computer simulations allow individual agents (e.g., people, cars, digital devices) to interact among themselves, as well as with the environment and the networked infrastructure. Such simulations are helpful to policy makers and infrastructure planners who need to answer specific questions. Additionally, our formal results show that simulation-based methods are both necessary and sufficient for understanding the dynamics of such complex systems [4–5]. The mathematical and computational theory views simulations as certain kinds of discrete dynamical systems and provides formal methods for the design, specification, and analysis of such simulations [3–5,11]. Unlike physical systems, socio-technical systems are affected not only by physical laws but also by human behavior, regulatory agencies, and government and private enterprise. The simulation of such systems thus presents novel challenges to researchers. Urban transportation systems constitute a canonical example of the types and levels of interactions that characterize these systems: Traffic rules in distant parts of a city can have an important bearing on traffic congestion downtown, and seemingly “reasonable” Short-term traffic forecast is one of the essential issues in intelligent transportation system. Accurate forecast result enables commuters make appropriate travel modes, travel routes, and departure time, which is meaningful in traffic management. To promote the forecast accuracy, a feasible way is to develop a more effective approach for traffic data analysis. The availability of abundant traffic data and computation power emerge in recent years, which motivates us to improve the accuracy of short-term traffic forecast via deep learning approaches. A novel traffic forecast model based on long short-term memory (LSTM) network is proposed. Different from conventional forecast models, the proposed LSTM network considers temporal-spatial correlation in traffic system via a two-dimensional network which is composed of many memory units. A comparison with other representative forecast models validates that the proposed LSTM network can achieve a better performance. Recently, localization has become an indispensable technique for wireless applications. In view of the limitation of global position system (GPS) in certain environments, alternative approaches are in demand. In this paper, we consider a cooperative localization approach named sum-product algorithm over a wireless network (SPAWN). Although SPAWN theoretically facilitates cooperative localization, it has several practical limitations. Specifically, SPAWN results in high computational complexity and increased network traffic. The main complexity of SPAWN lies in the selection of agents anchors involved in the cooperative localization. To this end, we formulate the agent anchor selection problem into a network formation game. Together with a practical limit on the number of agents anchors used for cooperative localization, our proposed approach can markedly reduce the computational complexity and the resultant network traffic. Simulations show that these advantages come with a slight degradation in the localization mean squared error (MSE) performance.