Adaptive traffic signal control using approximate dynamic programming

This paper presents a study on an adaptive traffic signal controller for real-time operation. The controller aims for three operational objectives: dynamic allocation of green time, automatic adjustment to control parameters, and fast revision of signal plans. The control algorithm is built on approximate dynamic programming (ADP). This approach substantially reduces computational burden by using an approximation to the value function of the dynamic programming and reinforcement learning to update the approximation. We investigate temporal-difference learning and perturbation learning as specific learning techniques for the ADP approach. We find in computer simulation that the ADP controllers achieve substantial reduction in vehicle delays in comparison with optimised fixed-time plans. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised, which can be achieved conveniently using the ADP approach.     

A systems approach to area traffic control

Since the first pilot scheme for area‐traffic control was introduced in the city of Montreal (1959–60), computer control of traffic in urban areas through the adaptation of existing traffic‐signal systems has been provided to an increasing extent. This area of work may pose problems for the professional traffic engineer whose background in computer technology and general digital electronics may be limited.

In considering the engineering implications of such schemes a systems approach is important and is adopted here. Three existing and representative schemes are briefly mentioned in order to outline basic features. A more detailed examination of the various system elements follows with mention of data collection and transmission, and the role of the control computer.

The paper continues with a reconsideration of the three representative schemes in the light of the detailed treatment of system components. It concludes with a tentative assessment of the present position of area traffic control schemes and some suggestions as to the future development.     

A multiple transfer function model for air traffic control systems

This paper considers the problem of modeling dynamic fluctuations in aircraft concentration within a group of air traffic control sectors. Using simultaneous time series recorded for each of the sectors, a multiple transfer function noise model is constructed. The modeling procedure demonstrates a data-dependent approach to ATC systems analysis which does not rely on describing the movement of individual aircraft.     

An integrated framework for embedding large-scale dynamic traffic assignment models in advanced traveler information systems

In recent years, increasing attention has been drawn to the development of various applications of intelligent transportation systems (ITS), which are credited with the amelioration of traffic conditions in urban and regional environments. Advanced traveler information systems (ATIS) constitute an important element of ITS by providing potential travelers with information on the network's current performance both en-route and pre-trip. In order to tackle the complexity of such systems, derived from the difficulty of providing real-time estimations of current as well as forecasts of future traffic conditions, a series of models and algorithms have been initiated. This paper proposes the development of an integrated framework for real-time ATIS and presents its application on a large-scale network, that of Thessaloniki, Greece, concluding with a discussion on development and implementation challenges as well as on the advantages and limitations of such an effort.     

A phenomenological model for dynamic traffic flow in networks

A macroscopic model for dynamic traffic flow is presented. The main goal of the model is the real time simulation of large freeway networks with multiple sources and sinks. First, we introduce the model in its discrete formulation and consider some of its properties. It turns out, that our non-hydrodynamical ansatz for the flows results in a very advantageous behavior of the model. Next the fitting conditions at junctions of a traffic network are discussed. In the following sections we carry out a continuous approximation of our discrete model in order to derive stationary solutions and to consider the stability of the homogeneous one. It turns out, that for certain conditions unstable traffic flow occurs. In a subsequent section, we compare the stability of the discrete model and the corresponding continuous approximation. This confirms in retrospection the close similarities of both model versions. Finally we compare the results of our model with the results of another macroscopic model, that was recently suggested by Kerner and Konhäuser [Phys. Rev. E 48, 2335–2338 (1993)].     

Route swapping in dynamic traffic networks

A dynamic traffic assignment (DTA) model typically consists of a traffic performance model and a route choice model. The traffic performance model describes how traffic propagates (over time) along routes connecting origin-destination (OD) pairs, examples being the cell transmission model, the vertical queueing model and the travel time model. This is implemented in a dynamic network loading (DNL) algorithm, which uses the given route inflows to compute the link inflows (and hence link costs), which are then used to compute the route travel times (and hence route costs). A route swap process specifies the route inflows for tomorrow (at the next iteration) based on the route inflows today (at the current iteration). A dynamic user equilibrium (DUE), where each traveller on the network cannot reduce his or her cost of travel by switching to another route, can be sought by iterating between the DNL algorithm and the route swap process. The route swap process itself takes up very little computational time (although route set generation can be very computationally intensive for large networks). However, the choice of route swap process dramatically affects convergence and the speed of convergence. The paper details several route swap processes and considers whether they lead to a convergent system, assuming that the route cost vector is a monotone function of the route inflow vector.     

Development and evaluation of a knowledge-based system for traffic congestion management and control

This paper describes a real-time knowledge-based system (KBS) for decision support to Traffic Operation Center personnel in the selection of integrated traffic control plans after the occurrence of non-recurring congestion, on freeway and arterial networks. The uniqueness of the system, called TCM, lies in its ability to cooperate with the operator, by handling different sources of input data and inferred knowledge, and providing an explanation of its reasoning process. A data fusion algorithm for the analysis of congestion allows to represent and interpret different types of data, with various levels of reliability and uncertainty, to provide a clear assessment of traffic conditions. An efficient algorithm for the selection of control plans determines alternative traffic control responses. These are proposed to an operator, along with an explanation of the reasoning process that led to their development and an estimation of their expected effect on traffic. The validation of the system, which is one of only few examples of validation of a KBS in transportation, demonstrates the validity of the approach. The evaluation results, in a simulated environment demonstrate the ability of TCM to reduce congestion, through the formulation of traffic diversion and control schemes.     

A new approach to time-of-day control based on a dynamic freeway traffic model

This paper presents a new approach to time-of-day control. While time-of-day control strategies presented up-to-now are only optimal under steady-state conditions, the control algorithm derived in this paper takes into account the evolution of traffic flow according to the time delay between a volume change at a ramp and its subsequent disturbance at a freeway point downstream. The new control strategy is based on the solution of a linear programming optimization problem and makes freeway volume hold the capacity constraints for the total time of control operation. In order to reduce the computational effort a simplified version of the new algorithm is also discussed. Simulation results obtained by use of two different traffic flow models show that control derived through the new algorithm can avoid congestion and ensure operation with peak performance even if a steady-state condition is never attained.     

Multiagent architectures for intelligent traffic management systems

This paper reports our experiences with agent-based architectures for intelligent traffic management systems. We describe and compare integrated TRYS and TRYS autonomous agents, two multiagent systems that perform decision support for real-time traffic management in the urban motorway network around Barcelona. Both systems draw upon traffic management agents that use similar knowledge-based reasoning techniques in order to deal with local traffic problems. Still, the former achieves agent coordination based on a traditional centralized mechanism, while in the latter coordination emerges upon the lateral interaction of autonomous traffic management agents. We evaluate the potentials and drawbacks of both multiagent architectures for the domain, and develop some conclusions respecting the general applicability of multiagent architectures for intelligent traffic management.     

Optimal offsets for traffic signal systems in urban networks

The paper proposes a binary integer programming model for the computation of optimal traffic signal offsets for an urban road network. The basic theoretical assumptions for the computation of delay on the network are those employed by the main models developed during the last few years. The set of input data coincides with that needed for the Combination Method and its extensions. The model is solved through a branch-and-backtrack method and allows the obtaining of optimal offsets for condensable or uncondensable networks without introducing any special assumption on delay-offset functions, contrary to what occurs within other mathematical programming formulations of the problem. A reduced memory dimension is required by the developed algorithm, which promptly supplies during the computation better and better sub-optimal solutions, very interesting in view of the possible application of the method to real-time control problems. The tests performed show that the method can be applied to networks of practical size.     

Multiple equilibria in a dynamic traffic network

This study provides an example in which the dynamic user equilibrium (DUE) assignment of a congested road network with bottlenecks is non-unique. In previous studies, the uniqueness of DUE assignments with the bottleneck model has been shown in limited cases such as single-origin and single-destination networks. Consequently, it is still an important issue whether or not uniqueness is a general property of DUE assignments. The present study describes a network in which multiple patterns of link travel time are found, thus providing a negative answer to this question. The network has a loopy structure with multiple bottlenecks and multiple origin-destination (OD) pairs. Given a certain demand pattern of departure times for vehicles leaving their origins, a non-convex set of equilibria with a non-unique pattern of link travel times is shown to exist.     

A hierarchical control system for freeway traffic

A hierarchical control system consisting of three control layers is developed for the freeway traffic control problem. A simplified optimization problem for the overall freeway system is solved on-line in an optimization layer. Optimization results are used as reference values for an inferior decentralized direct control layer. Prediction of slowly varying variables like on-ramp demands and origin-destination rates as well as of particular model parameters are provided by a supremal adaption layer. The overall control structure is shown to be robust even in the case of strong unexpected disturbances like incidents.     

Lane‐based optimization of traffic equilibrium settings for area traffic control

Area traffic control is an important element in Intelligent Transportation System (ITS). This paper extends the lane‐based optimization method to a traffic equilibrium network, which improves the operational performance of signal‐controlled network. We formulate a decomposition approach to simultaneously optimize the lane markings and signal settings for a signal‐controlled network that comprises two levels of optimization. At the junction level, the lane markings, control sequence, and other aspects of the signal settings are optimized for individual junctions, whereas at the network level, the group‐based signal settings are optimized to take into account the re‐routing characteristics of travelers and signal coordination effects that are based on a TRANSYT traffic model, which is a well‐known procedure for evaluating the performance of signal‐controlled networks. We use a numerical example to demonstrate the effectiveness of the proposed methodology.     

Macroscopic traffic models for vehicle-follower automated transportation systems

The problem of distributing and routing vehicles in a large automated transportation network may be approached through the design of on-line control algorithms, particularly when the network contains many origin-destination pairs and alternate routes. To develop such algorithms, it is necessary to obtain models that accurately represent the dynamic behavior of vehicles on the guideway network. In this paper, models based on density, flow and average velocity variables are derived for the vehicle-follower longitudinal control scheme. Models suitable for use in analysis and simulation work are developed for links, merges, diverges, and stations. The proposed models are shown to compare favorably with simulation results that use explicit modeling of vehicle dynamic modeling of vehicle dynamic interaction.     

A dynamic traffic assignment model for a continuum transportation system

A predictive continuum dynamic user-optimal (PDUO-C) model is formulated in this study to investigate the dynamic characteristics of traffic flow and the corresponding route-choice behavior of travelers within a region with a dense urban road network. The modeled region is arbitrary in shape with a single central business district (CBD) and travelers continuously distributed over the region. Within this region, the road network is represented as a continuum and travelers patronize a two-dimensional continuum transportation system to travel to the CBD. The PDUO-C model is solved by a promising solution algorithm that includes elements of the finite volume method (FVM), the finite element method (FEM), and the explicit total variation diminishing Runge-Kutta (TVD-RK) time-stepping method. A numerical example is given to demonstrate the utility of the proposed model and the effectiveness of the solution algorithm in solving this PDUO-C problem.     

A dynamic traffic assignment model for the assessment of moving bottlenecks

Moving bottlenecks in highway traffic are defined as a situation in which a slow-moving vehicle, be it a truck hauling heavy equipment or an oversized vehicle, or a long convey, disrupts the continuous flow of the general traffic. The effect of moving bottlenecks on traffic flow is an important factor in the evaluation of network performance. This effect, though, cannot be assessed properly by existing transportation tools, especially when the bottleneck travels relatively long distances in the network.This paper develops a dynamic traffic assignment (DTA) model that can evaluate the effects of moving bottlenecks on network performance in terms of both travel times and traveling paths. The model assumes that the characteristics of the moving bottleneck, such as traveling path, physical dimensions, and desired speed, are predefined and, therefore, suitable for planned conveys.The DTA model is based on a mesoscopic simulation network-loading procedure with unique features that allow assessing the special dynamic characteristics of a moving bottleneck. By permitting traffic density and speed to vary along a link, the simulation can capture the queue caused by the moving bottleneck while preserving the causality principles of traffic dynamics.     

Machine learning for multi-jurisdictional optimal traffic corridor control

Urban traffic corridors are often controlled by more than one agency. Typically in North America, a state of provincial transportation department controls freeways while another agency at the municipal or city level controls the nearby arterials. While the different segments of the corridor fall under different jurisdictions, traffic and users know no boundaries and expect seamless service. Common lack of coordination amongst those authorities due to lack of means for information exchange and/or possible bureaucratic ‘institutional grid-lock’ could hinder the full potential of technically-possible integrated control. Such institutional gridlock and related lack of timely coordination amongst the different agencies involved can have a direct impact on traffic gridlock. One potential solution to this problem is through integrated automatic control under intelligent transportation systems (ITS). Advancements in ITS and communication technology have the potential to considerably reduce delay and congestion through an array of network-wide traffic control and management strategies that can seamlessly cross-jurisdictional boundaries. Perhaps two of the most promising such control tools for freeway corridors are traffic-responsive ramp metering and/or dynamic traffic diversion possibly using variable message signs (VMS). Technically, the use of these control methods separately might limit their potential usefulness. Therefore, integrated corridor control using ramp metering and VMS diversion simultaneously might be synergetic and beneficial. Motivated by the above problem and potential solution approach, the aim of the research presented in this paper is to develop a self-learning adaptive integrated freeway-arterial corridor control for both recurring and non-recurring congestion. The paper introduces the use of reinforcement learning, an Artificial Intelligence method for machine learning, to provide optimal control using ramp metering and VMS routing in an integrated agent for a freeway-arterial corridor. Reinforcement learning is an approach whereby the control agent directly learns optimal strategies via feedback reward signals from its environment. A simple but powerful reinforcement learning method known as Q-learning is used. Results from an elaborate simulation study on a key corridor in Toronto are very encouraging and discussed in the paper.     

A splitting rate model of traffic re-routeing and traffic control

The paper presents an idealised dynamical model of day-to-day or within-day re-routeing using splitting rates at nodes, or node-exit flows, rather than route-flows. It is shown that under certain conditions the dynamical model gives rise to a sequence of link flow vectors which converges to a set of approximate Wardrop equilibria. A special dynamical signal green-time re-allocation model is added; the combination is also shown (in outline) to converge to the set of approximate consistent equilibria under certain conditions. Finally the paper uses model network results to illustrate a method of designing fixed time signal timings to meet different scenarios.     

Vehicle routing in reverse logistics for recycling end-of-life consumer electronic goods in South Korea

This paper presents a vehicle routing approach for the transport of end-of-life consumer electronic goods for recycling in South Korea. The objective is to minimize the distance of transportation of end-of-life goods collected by local authorities and major manufacturers’ distribution centers to four regional recycling centers located. A vehicle routing problem is constructed for each regional center, and a Tabu search is applied to solve it. Computational results using field data show that the method outperforms existing approaches to reverse logistics.