Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers).The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-h traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.     

System performance and user response under real-time information in a congested traffic corridor

A modelling framework is developed to analyze the effect of in-vehicle real time information strategies on the performance of a congested traffic communing corridor. The framework consists of a special-purpose simulation component and a user decisions component that determines users' responses to the supplied information. The user decisions component is microscopic and determines individual commuters' route switching, at any node of the network, as a function of the supplied information. The traffic simulation component moves vehicles in bundles or macroparticles at the prevailing local speeds, as determined by macroscopic traffic relations. The framework allows the investigation of system performance under alternative behavioral response mechanisms, as well as under different information strategies. Results are presented for simulation experiments in a commuting corridor with a special network structure that simplifies the network computations. The results illustrate the effect of the fraction of users equipped with in-vehicle navigation systems on overall system performance. In addition, alternative assumptions on user response reflecting varying degrees of optimizing behavior are explored. The modelling framework is shown to provide a useful approach for addressing key questions of interest in the design of real time in-vehicle information system.     

A micro-simulation model system of departure time using a perception updating model under travel time uncertainty

Existing microscopic traffic models have often neglected departure time change as a possible response to congestion. In addition, they lack a formal model of how travellers base their daily travel decisions on the accumulated experience gathered from repetitively travelling through the transport network. This paper proposes an approach to account for these shortcomings. A micro-simulation approach is applied, in which individuals base their consecutive departure time decisions on a mental model. The mental model is the outcome of a continuous process of perception updating according to principles of reinforcement learning. Individuals’ daily travel decisions are linked to the traffic simulator SIAS-PARAMICS to create a simulation system in which both individual decision-making and system performance (and interactions between these two levels) are adequately represented. The model is applied in a case study that supports the feasibility of this approach.     

Classification modeling approach for vehicle dynamics model‐integrated traffic simulation assessing surrogate safety

To assess safety impacts of untried traffic control strategies, an earlier study developed a vehicle dynamics model‐integrated (i.e., VISSIM‐CarSim‐SSAM) simulation approach and evaluated its performance using surrogate safety measures. Although the study found that the integrated simulation approach was a superior alternative to existing approaches in assessing surrogate safety, the computation time required for the implementation of the integrated simulation approach prevents it from using it in practice. Thus, this study developed and evaluated two types of models that could replace the integrated simulation approach with much faster computation time, feasible for real‐time implementation. The two models are as follows: (i) a statistical model (i.e., logit model) and (ii) a nonparametric approach (i.e., artificial neural network). The logit model and the neural network model were developed and trained on the basis of three simulation data sets obtained from the VISSIM‐CarSim‐SSAM integrated simulation approach, and their performances were compared in terms of the prediction accuracy. These two models were evaluated using six new simulation data sets. The results indicated that the neural network approach showing 97.7% prediction accuracy was superior to the logit model with 85.9% prediction accuracy. In addition, the correlation analysis results between the traffic conflicts obtained from the neural network approach and the actual traffic crash data collected in the field indicated a statistically significant relationship (i.e., 0.68 correlation coefficient) between them. This correlation strength is higher than that of the VISSIM only (i.e., the state of practice) simulation approach. The study results indicated that the neural network approach is not only a time‐efficient way to implementing the VISSIM‐CarSim‐SSAM integrated simulation but also a superior alternative in assessing surrogate safety. Copyright © 2014 John Wiley & Sons, Ltd.     

An analysis of internal transit systems requirements for central cities

The deficient performance of existing downtown transit distribution systems may significantly affect the overall performance of the urban transportation system in many metropolitan areas. This paper has two principal objectives. First, it describes an approach entitled Performance Requirements Analysis for developing a set of requirements or standards that a downtown transit system should satisfy and structuring these requirements so as to generate a small set of alternative generic systems for detailed evaluation. Second, it provides a preliminary assessment of performance requirements for distribution systems in 19 of the largest cities in the United States.     

Exploring the effects of cooperative adaptive cruise control on highway traffic flow using microscopic traffic simulation

This paper examines the impact of having cooperative adaptive cruise control (CACC) embedded vehicles on traffic flow characteristics of a multilane highway system. The study identifies how CACC vehicles affect the dynamics of traffic flow on a complex network and reduce traffic congestion resulting from the acceleration/deceleration of the operating vehicles. An agent-based microscopic traffic simulation model (Flexible Agent-based Simulator of Traffic) is designed specifically to examine the impact of these intelligent vehicles on traffic flow. The flow rate of cars, the travel time spent, and other metrics indicating the evolution of traffic congestion throughout the lifecycle of the model are analyzed. Different CACC penetration levels are studied. The results indicate a better traffic flow performance and higher capacity in the case of CACC penetration compared to the scenario without CACC-embedded vehicles.     

Nonlinear gating control for urban road traffic network using the network fundamental diagram

This work proposes a nonlinear model predictive controller for the urban gating problem. The system model is formalized based on a research on existing models of the network fundamental diagram and the perimeter control systems. For the existing models, modifications are suggested: additional state variables are allocated to describe the queue dynamics at the network gates. Using the extended model, a nonlinear model predictive controller is designed offering a ‘non‐greedy’ policy compared with previous, ‘greedy’ gating control designs. The greedy and non‐greedy nonlinear model predictive control (NMPC) controllers are compared with a greedy linear feedback proportional‐integral‐derivative (PID) controller in different traffic situations. The proposed non‐greedy NMPC controller outperforms the other two approaches in terms of travel distance performance and queue lengths. The performance results justify the consideration of queue lengths in dynamic modeling, and the use of NMPC approach for controller design. Copyright © 2014 John Wiley & Sons, Ltd.     

On the distribution of urban road space for multimodal congested networks

Transport systems in real cities are complex with many modes of transport sharing and competing for limited road space. This work intends to understand how space distributions for modes and interactions among modes affect network traffic performance. While the connection between performance of transport systems and general land allocation is the subject of extensive research, space allocation for interacting modes of transport is an open research question. Quantifying the impact of road space distribution on the performance of a congested multimodal transport system with a dynamic aggregated model remains a challenge. In this paper, a multimodal macroscopic fundamental diagram (MFD) is developed to represent the traffic dynamics of a multimodal transport system. Optimization is performed with the objective of minimizing the total passenger hours traveled (PHT) to serve the total demand by redistributing road space among modes. Pricing strategies are also investigated to provide a higher demand shift to more efficient modes. We find by an application to a bi-modal two-region city that (i) the proposed model captures the operational characteristics of each mode, and (ii) optimal dynamic space distribution strategies can be developed. In practice, the approach can serve as a physical dynamic model to inform space distribution strategies for policy makers with different goals of mobility.     

Behavior-consistent real-time traffic routing under information provision

The problem addressed here involves a controller seeking to enhance traffic network performance via real-time routing information provision to drivers while explicitly accounting for drivers’ likely reactions towards the information. A fuzzy control modeling approach is used to determine the associated behavior-consistent information-based network control strategies. Experiments are performed to compare the effectiveness of the behavior-consistent approach with traditional dynamic traffic assignment based approaches for deployment. The results show the importance of incorporating driver behavior realistically in the determination of the information strategies. Significant differences in terms of system travel time savings and compliance to the information strategies can be obtained when the behavior-consistent approach is compared to the traditional approaches. The behavior-consistent approach can provide more robust performance compared to the standard user or system optimal information strategies. Subject to a meaningful estimation of driver behavior, it can ensure system performance improvement. By contrast, approaches that do not seek to simultaneously achieve the objectives of the drivers and the controller can potentially deteriorate system performance because the controller may over-recommend or under-recommend some routes, or recommend routes that are not considered by the drivers.     

Construction of traffic state vector using mutual information for short-term traffic flow prediction

Short-term traffic flow prediction is an integral part in most of Intelligent Transportation Systems (ITS) research and applications. Many researchers have already developed various methods that predict the future traffic condition from the historical database. Nevertheless, there has not been sufficient effort made to study how to identify and utilize the different factors that affect the traffic flow. In order to improve the performance of short-term traffic flow prediction, it is necessary to consider sufficient information related to the road section to be predicted. In this paper, we propose a method of constructing traffic state vectors by using mutual information (MI). First, the variables with different time delays are generated from the historical traffic time series, and the spatio-temporal correlations between the road sections in urban road network are evaluated by the MI. Then, the variables with the highest correlation related to the target traffic flow are selected by using a greedy search algorithm to construct the traffic state vector. The K-Nearest Neighbor (KNN) model is adapted for the application of the proposed state vector. Experimental results on real-world traffic data show that the proposed method of constructing traffic state vector provides good prediction accuracy in short-term traffic prediction.     

Integration of signal timing estimation model and dynamic traffic assignment in feedback loops: system design and case study

This paper presents an integrated framework for effective coupling of a signal timing estimation model and dynamic traffic assignment (DTA) in feedback loops. There are many challenges in effectively integrating signal timing tools with DTA software systems, such as data availability, exchange format, and system coupling. In this research, a tight coupling between a DTA model with various queue‐based simulation models and a quick estimation method Excel‐based signal control tool is achieved and tested. The presented framework design offers an automated solution for providing realistic signal timing parameters and intersection movement capacity allocation, especially for future year scenarios. The framework was used to design an open‐source data hub for multi‐resolution modeling in analysis, modeling and simulation applications, in which a typical regional planning model can be quickly converted to microscopic traffic simulation and signal optimization models. The coupling design and feedback loops are first demonstrated on a simple network, and we examine the theoretically important questions on the number of iterations required for reaching stable solutions in feedback loops. As shown in our experiment, the current coupled application becomes stable after about 30 iterations, when the capacity and signal timing parameters can quickly converge, while DTA's route switching model predominately determines and typically requires more iterations to reach a stable condition. A real‐world work zone case study illustrates how this application can be used to assess impacts of road construction or traffic incident events that disrupt normal traffic operations and cause route switching on multiple analysis levels. Copyright © 2014 John Wiley & Sons, Ltd.     

System dynamics of urban traffic based on its parking-related-states

The urban parking and the urban traffic systems are essential components of the overall urban transportation structure. The short-term interactions between these two systems can be highly significant and influential to their individual performance. The urban parking system, for example, can affect the searching-for-parking traffic, influencing not only overall travel speeds in the network (traffic performance), but also total driven distance (environmental conditions). In turn, the traffic performance can also affect the time drivers spend searching for parking, and ultimately, parking usage. In this study, we propose a methodology to model macroscopically such interactions and evaluate their effects on urban congestion.The model is built on a matrix describing how, over time, vehicles in an urban area transition from one parking-related state to another. With this model it is possible to estimate, based on the traffic and parking demand as well as the parking supply, the amount of vehicles searching for parking, the amount of vehicles driving on the network but not searching for parking, and the amount of vehicles parked at any given time. More importantly, it is also possible to estimate the total (or average) time spent and distance driven within each of these states. Based on that, the model can be used to design and evaluate different parking policies, to improve (or optimize) the performance of both systems.A simple numerical example is provided to show possible applications of this type. Parking policies such as increasing parking supply or shortening the maximum parking duration allowed (i.e., time controls) are tested, and their effects on traffic are estimated. The preliminary results show that time control policies can alleviate the parking-caused traffic issues without the need for providing additional parking facilities. Results also show that parking policies that intend to reduce traffic delay may, at the same time, increase the driven distance and cause negative externalities. Hence, caution must be exercised and multiple traffic metrics should be evaluated before selecting these policies.Overall, this paper shows how the system dynamics of urban traffic, based on its parking-related-states, can be used to efficiently evaluate the urban traffic and parking systems macroscopically. The proposed model can be used to estimate both, how parking availability can affect traffic performance (e.g., average time searching for parking, number of cars searching for parking); and how different traffic conditions (e.g., travel speed, density in the system) can affect drivers ability to find parking. Moreover, the proposed model can be used to study multiple strategies or scenarios for traffic operations and control, transportation planning, land use planning, or parking management and operations.     

A novel forecasting approach inspired by human memory: The example of short-term traffic volume forecasting

Short-term traffic volume forecasting represents a critical need for Intelligent Transportation Systems. This paper develops a novel forecasting approach inspired by human memory, called the spinning network (SPN). The approach is then used for short-term traffic volume forecasting, utilizing a data set compiled from real-world traffic volume data obtained from the Hampton Roads traffic operations center in Virginia. To assess the accuracy of the SPN approach, its performance is compared to two other approaches, namely a back propagation neural network and a nearest neighbor approach. The transferability of the SPN approach and its ability to forecast for longer time periods into the future is also assessed. The results of the performance testing conducted in this paper demonstrates the superior predictive accuracy and drastically lower computational requirements of the SPN compared to either the neural network or the nearest neighbor approach. The tests also confirm the ability of the SPN to predict traffic volumes for longer time periods into the future, as well as the transferability of the approach to other sites.     

Dynamics of heterogeneity in urban networks: aggregated traffic modeling and hierarchical control

Real traffic data and simulation analysis reveal that for some urban networks a well-defined Macroscopic Fundamental Diagram (MFD) exists, which provides a unimodal and low-scatter relationship between the network vehicle density and outflow. Recent studies demonstrate that link density heterogeneity plays a significant role in the shape and scatter level of MFD and can cause hysteresis loops that influence the network performance. Evidently, a more homogeneous network in terms of link density can result in higher network outflow, which implies a network performance improvement. In this article, we introduce two aggregated models, region- and subregion-based MFDs, to study the dynamics of heterogeneity and how they can affect the accuracy scatter and hysteresis of a multi-subregion MFD model. We also introduce a hierarchical perimeter flow control problem by integrating the MFD heterogeneous modeling. The perimeter flow controllers operate on the border between urban regions, and manipulate the percentages of flows that transfer between the regions such that the network delay is minimized and the distribution of congestion is more homogeneous. The first level of the hierarchical control problem can be solved by a model predictive control approach, where the prediction model is the aggregated parsimonious region-based MFD and the plant (reality) is formulated by the subregion-based MFDs, which is a more detailed model. At the lower level, a feedback controller of the hierarchical structure, tries to maximize the outflow of critical regions, by increasing their homogeneity. With inputs that can be observed with existing monitoring techniques and without the need for detailed traffic state information, the proposed framework succeeds to increase network flows and decrease the hysteresis loop of the MFD. Comparison with existing perimeter controllers without considering the more advanced heterogeneity modeling of MFD highlights the importance of such approach for traffic modeling and control.     

Dynamic traffic metering in urban street networks: Formulation and solution algorithm

Traffic metering offers great potential to reduce congestion and enhance network performance in oversaturated urban street networks. This paper presents an optimization program for dynamic traffic metering in urban street networks based on the Cell Transmission Model (CTM). We have formulated the problem as a Mixed-Integer Linear Program (MILP) capable of metering traffic at network gates with given signal timing parameters at signalized intersections. Due to the complexities of the MILP model, we have developed a novel and efficient solution approach that solves the problem by converting the MILP to a linear program and several CTM simulation runs. The solution algorithm is applied to two case studies under different conditions. The proposed solution technique finds solutions that have a maximum gap of 1% of the true optimal solution and guarantee the maximum throughput by keeping some vehicles at network gates and only allowing enough vehicles to enter the network to prevent gridlocks. This is confirmed by comparing the case studies with and without traffic metering. The results in an adapted real-world case study network show that traffic metering can increase network throughput by 4.9–38.9% and enhance network performance.     

Cooperative traffic control of a mixed network with two urban regions and a freeway

Currently most optimization methods for urban transport networks (i) are suited for networks with simplified dynamics that are far from real-sized networks or (ii) apply decentralized control, which is not appropriate for heterogeneously loaded networks or (iii) investigate good-quality solutions through micro-simulation models and scenario analysis, which make the problem intractable in real time. In principle, traffic management decisions for different sub-systems of a transport network (urban, freeway) are controlled by operational rules that are network specific and independent from one traffic authority to another. In this paper, the macroscopic traffic modeling and control of a large-scale mixed transportation network consisting of a freeway and an urban network is tackled. The urban network is partitioned into two regions, each one with a well-defined Macroscopic Fundamental Diagram (MFD), i.e. a unimodal and low-scatter relationship between region density and outflow. The freeway is regarded as one alternative commuting route which has one on-ramp and one off-ramp within each urban region. The urban and freeway flow dynamics are formulated with the tool of MFD and asymmetric cell transmission model, respectively. Perimeter controllers on the border of the urban regions operating to manipulate the perimeter interflow between the two regions, and controllers at the on-ramps for ramp metering are considered to control the flow distribution in the mixed network. The optimal traffic control problem is solved by a Model Predictive Control (MPC) approach in order to minimize total delay in the entire network. Several control policies with different levels of urban-freeway control coordination are introduced and tested to scrutinize the characteristics of the proposed controllers. Numerical results demonstrate how different levels of coordination improve the performance once compared with independent control for freeway and urban network. The approach presented in this paper can be extended to implement efficient real-world control strategies for large-scale mixed traffic networks.     

A microscopic analysis of speed deviation impacts on lane-changing behavior

Driving behavior models that capture drivers’ tactical maneuvering decisions in different traffic conditions are essential to microscopic traffic simulation systems. This paper focuses on a parameter that has a great impact on road users’ aggressive overtaking maneuvers and directly affects lane-changing models (an integral part of microscopic traffic simulation models), namely, speed deviation. The objective of this research is to investigate the impacts of speed deviation in terms of performance measures (delay time, network mean speed, and travel time duration) and the number of lane-change maneuvers using the Aimsun traffic simulator. Following calibration of the model for a section of urban highway in Tehran, this paper explores the sensitivity of lane-changing maneuvers during different speed deviations by conducting two types of test. Simulation results show that, by decreasing speed deviation, the number of lane changes reduces remarkably and so network safety increases, thus reducing travel time due to an increase in network mean speed.     

Real-time estimation of turning movement proportions from partial counts on urban networks

Modern traffic signal control systems require reliable estimates of turning flows in real time to formulate effective control actions, and accommodate disturbances in traffic demand without deteriorating the system performance. The more accurate the estimation is, the more effective the control plan is. Most of the previous research works assumed that a full set of detector counts is available and employed the least-squares methods to produce unbiased estimates of the turning movement proportions. However, in practice, such a dense detector configuration is expensive to install and maintain. Also, the least-squares estimates are not feasible when the travel time between inflows and outflows is significant, or when intervening traffic conditions change the travel time. This study proposes a nonlinear least-square (NLS) approach and a quasi maximum likelihood (QML) approach to recursively estimate turning movement proportions in a network of intersections where only a partial set of detector counts are available. Using large population approximation technique, a class of nonlinear, discrete-time traffic flow models are transformed into a linear state–space model tractable for on-line applications. The quality of estimates is demonstrated by implementing the proposed algorithms with simulation and real data. As a comparison, the NLS estimator shows less bias but with higher variance than the QML estimator. The QML estimator outperforms the NLS estimator in terms of total mean square error, due to an increase in bias being traded for a decrease in variance.     

Real-time traffic network state estimation and prediction with decision support capabilities: Application to integrated corridor management

This paper presents a real-time traffic network state estimation and prediction system with built-in decision support capabilities for traffic network management. The system provides traffic network managers with the capabilities to estimate the current network conditions, predict congestion dynamics, and generate efficient traffic management schemes for recurrent and non-recurrent congestion situations. The system adopts a closed-loop rolling horizon framework in which network state estimation and prediction modules are integrated with a traffic network manager module to generate efficient proactive traffic management schemes. The traffic network manger adopts a meta-heuristic search mechanism to construct the schemes by integrating a wide variety of control strategies. The system is applied in the context of Integrated Corridor Management (ICM), which is envisioned to provide a system approach for managing congested urban corridors. A simulation-based case study is presented for the US-75 corridor in Dallas, Texas. The results show the ability of the system to improve the overall network performance during hypothetical incident scenarios.