Adaptive traffic signal control with equilibrium constraints under stochastic demand

This study develops a methodology to model transportation network design with signal settings in the presence of demand uncertainty. It is assumed that the total travel demand consists of commuters and infrequent travellers. The commuter travel demand is deterministic, whereas the demand of infrequent travellers is stochastic. Variations in demand contribute to travel time uncertainty and affect commuters’ route choice behaviour. In this paper, we first introduce an equilibrium flow model that takes account of uncertain demand. A two-stage stochastic program is then proposed to formulate the network signal design under demand uncertainty. The optimal control policy derived under the two-stage stochastic program is able to (1) optimize the steady-state network performance in the long run, and (2) respond to short-term demand variations. In the first stage, a base signal control plan with a buffer against variability is introduced to control the equilibrium flow pattern and the resulting steady-state performance. In the second stage, after realizations of the random demand, recourse decisions of adaptive signal settings are determined to address the occasional demand overflows, so as to avoid transient congestion. The overall objective is to minimize the expected total travel time. To solve the two-stage stochastic program, a concept of service reliability associated with the control buffer is introduced. A reliability-based gradient projection algorithm is then developed. Numerical examples are performed to illustrate the properties of the proposed control method as well as its capability of optimizing steady-state performance while adaptively responding to changing traffic flows. Comparison results show that the proposed method exhibits advantages over the traditional mean-value approach in improving network expected total travel times.     

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers).The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-h traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.     

A bi-level model of dynamic traffic signal control with continuum approximation

This paper proposes a bi-level model for traffic network signal control, which is formulated as a dynamic Stackelberg game and solved as a mathematical program with equilibrium constraints (MPEC). The lower-level problem is a dynamic user equilibrium (DUE) with embedded dynamic network loading (DNL) sub-problem based on the LWR model (Lighthill and Whitham, 1955; Richards, 1956). The upper-level decision variables are (time-varying) signal green splits with the objective of minimizing network-wide travel cost. Unlike most existing literature which mainly use an on-and-off (binary) representation of the signal controls, we employ a continuum signal model recently proposed and analyzed in Han et al. (2014), which aims at describing and predicting the aggregate behavior that exists at signalized intersections without relying on distinct signal phases. Advantages of this continuum signal model include fewer integer variables, less restrictive constraints on the time steps, and higher decision resolution. It simplifies the modeling representation of large-scale urban traffic networks with the benefit of improved computational efficiency in simulation or optimization. We present, for the LWR-based DNL model that explicitly captures vehicle spillback, an in-depth study on the implementation of the continuum signal model, as its approximation accuracy depends on a number of factors and may deteriorate greatly under certain conditions. The proposed MPEC is solved on two test networks with three metaheuristic methods. Parallel computing is employed to significantly accelerate the solution procedure.     

A robust optimization approach for dynamic traffic signal control with emission considerations

We consider an analytical signal control problem on a signalized network whose traffic flow dynamic is described by the Lighthill–Whitham–Richards (LWR) model (Lighthill and Whitham, 1955; Richards, 1956). This problem explicitly addresses traffic-derived emissions as constraints or objectives. We seek to tackle this problem using a mixed integer mathematical programming approach. Such class of problems, which we call LWR-Emission (LWR-E), has been analyzed before to certain extent. Since mixed integer programs are practically efficient to solve in many cases (Bertsimas et al., 2011b), the mere fact of having integer variables is not the most significant challenge to solving LWR-E problems; rather, it is the presence of the potentially nonlinear and nonconvex emission-related constraints/objectives that render the program computationally expensive.To address this computational challenge, we proposed a novel reformulation of the LWR-E problem as a mixed integer linear program (MILP). This approach relies on the existence of a statistically valid macroscopic relationship between the aggregate emission rate and the vehicle occupancy on the same link. This relationship is approximated with certain functional forms and the associated uncertainties are handled explicitly using robust optimization (RO) techniques. The RO allows emissions-related constraints and/or objectives to be reformulated as linear forms under mild conditions. To further reduce the computational cost, we employ a link-based LWR model to describe traffic dynamics with the benefit of fewer (integer) variables and less potential traffic holding. The proposed MILP explicitly captures vehicle spillback, avoids traffic holding, and simultaneously minimizes travel delay and addresses emission-related concerns.     

A novel traffic signal control formulation

A novel traffic signal control formulation is developed through a mixed integer programming technique. The formulation considers dynamic traffic, uses dynamic traffic demand as input, and takes advantage of a convergent numerical approximation to the hydrodynamic model of traffic flow. As inherent from the underlying hydrodynamic model, this formulation covers the whole range of the fundamental relationships between speed, flow, and density. Kinematic waves of the stop-and-go traffic associated with traffic signals are also captured. Because of this property, one does not need to tune or switch the model for the different traffic conditions. It “automatically” adjusts to the different traffic conditions. We applied the model to three demand scenarios in a simple network. The results seemed promising. This model produced timing plans that are consistent with models that work for unsaturated conditions. In gridlock conditions, it produced a timing plan that was better than conventional queue management practices.     

An approximate dynamic programming approach for the empty container allocation problem

The objective of this study is to demonstrate the successful application of an approximate dynamic programming approach in deriving effective operational strategies for the relocation of empty containers in the containerized sea-cargo industry. A dynamic stochastic model for a simple two-ports two-voyages (TPTV) system is proposed first to demonstrate the effectiveness of the approximate optimal solution obtained through a simulation based approach known as the temporal difference (TD) learning for average cost minimization. An exact optimal solution can be obtained for this simple TPTV model. Approximate optimal results from the TPTV model utilizing a linear approximation architecture under the TD framework can then be compared to this exact solution. The results were found comparable and showed promising improvements over an existing commonly used heuristics. The modeling and solution approach can be extended to a realistic multiple-ports multiple-voyages (MPMV) system. Some results for the MPMV case are shown.     

Vehicle scheduling under stochastic trip times: An approximate dynamic programming approach

Due to unexpected demand surge and supply disruptions, road traffic conditions could exhibit substantial uncertainty, which often makes bus travelers encounter start delays of service trips and substantially degrades the performance of an urban transit system. Meanwhile, rapid advances of information and communication technologies have presented tremendous opportunities for intelligently scheduling a bus fleet. With the full consideration of delay propagation effects, this paper is devoted to formulating the stochastic dynamic vehicle scheduling problem, which dynamically schedules an urban bus fleet to tackle the trip time stochasticity, reduce the delay and minimize the total costs of a transit system. To address the challenge of “curse of dimensionality”, we adopt an approximate dynamic programming approach (ADP) where the value function is approximated through a three-layer feed-forward neural network so that we are capable of stepping forward to make decisions and solving the Bellman’s equation through sequentially solving multiple mixed integer linear programs. Numerical examples based on the realistic operations dataset of bus lines in Beijing have demonstrated that the proposed neural-network-based ADP approach not only exhibits a good learning behavior but also significantly outperforms both myopic and static polices, especially when trip time stochasticity is high.     

A linear programming formulation for autonomous intersection control within a dynamic traffic assignment and connected vehicle environment

This paper develops a novel linear programming formulation for autonomous intersection control (LPAIC) accounting for traffic dynamics within a connected vehicle environment. Firstly, a lane based bi-level optimization model is introduced to propagate traffic flows in the network, accounting for dynamic departure time, dynamic route choice, and autonomous intersection control in the context of system optimum network model. Then the bi-level optimization model is transformed to the linear programming formulation by relaxing the nonlinear constraints with a set of linear inequalities. One special feature of the LPAIC formulation is that the entries of the constraint matrix has only {−1, 0, 1} values. Moreover, it is proved that the constraint matrix is totally unimodular, the optimal solution exists and contains only integer values. It is also shown that the traffic flows from different lanes pass through the conflict points of the intersection safely and there are no holding flows in the solution. Three numerical case studies are conducted to demonstrate the properties and effectiveness of the LPAIC formulation to solve autonomous intersection control.     

Multi-modal traffic signal control with priority,signal actuation and coordination

Both coordinated-actuated signal control systems and signal priority control systems have been widely deployed for the last few decades. However, these two control systems are often conflicting with each due to different control objectives. This paper aims to address the conflicting issues between actuated-coordination and multi-modal priority control. Enabled by vehicle-to-infrastructure (v2i) communication in Connected Vehicle Systems, priority eligible vehicles, such as emergency vehicles, transit buses, commercial trucks, and pedestrians are able to send request for priority messages to a traffic signal controller when approaching a signalized intersection. It is likely that multiple vehicles and pedestrians will send requests such that there may be multiple active requests at the same time. A request-based mixed-integer linear program (MILP) is formulated that explicitly accommodate multiple priority requests from different modes of vehicles and pedestrians while simultaneously considering coordination and vehicle actuation. Signal coordination is achieved by integrating virtual coordination requests for priority in the formulation. A penalty is added to the objective function when the signal coordination is not fulfilled. This “soft” signal coordination allows the signal plan to adjust itself to serve multiple priority requests that may be from different modes. The priority-optimal signal timing is responsive to real-time actuations of non-priority demand by allowing phases to extend and gap out using traditional vehicle actuation logic. The proposed control method is compared with state-of-practice transit signal priority (TSP) both under the optimized signal timing plans using microscopic traffic simulation. The simulation experiments show that the proposed control model is able to reduce average bus delay, average pedestrian delay, and average passenger car delay, especially for highly congested condition with a high frequency of transit vehicle priority requests.     

Stochastic adaptive control model for traffic signal systems

An adaptive control model of a network of signalized intersections is proposed based on a discrete-time, stationary, Markov decision process. The model incorporates probabilistic forecasts of individual vehicle actuations at downstream inductance loop detectors that are derived from a macroscopic link transfer function. The model is tested both on a typical isolated traffic intersection and a simple network comprised of five four-legged signalized intersections, and compared to full-actuated control. Analyses of simulation results using this approach show significant improvement over traditional full-actuated control, especially for the case of high volume, but not saturated, traffic demand.     

Identification of oversaturated intersections using high-resolution traffic signal data

Conceptually, an oversaturated traffic intersection is defined as one where traffic demand exceeds the capacity. Such a definition, however, cannot be applied directly to identify oversaturated intersections because measuring traffic demand under congested conditions is not an easy task, particularly with fixed-location sensors. In this paper, we circumvent this issue by quantifying the detrimental effects of oversaturation on signal operations, both temporally and spatially. The detrimental effect is characterized temporally by a residual queue at the end of a cycle, which will require a portion of green time in the next cycle; or spatially by a spill-over from downstream traffic whereby usable green time is reduced because of the downstream blockage. The oversaturation severity index (OSI), in either the temporal dimension (T-OSI) or the spatial dimension (S-OSI) can then be measured using high-resolution traffic signal data by calculating the ratio between the unusable green time due to detrimental effects and the total available green time in a cycle. To quantify the T-OSI, in this paper, we adopt a shockwave-based queue estimation algorithm to estimate the residual queue length. S-OSI can be identified by a phenomenon denoted as “Queue-Over-Detector (QOD)”, which is the condition when high occupancy on a detector is caused by downstream congestion. We believe that the persistence duration and the spatial extent with OSI greater than zero provide an important indicator for measuring traffic network performance so that corresponding congestion mitigation strategies can be prepared. The proposed algorithms for identifying oversaturated intersections and quantifying the oversaturation severity index have been field-tested using traffic signal data from a major arterial in the Twin Cities of Minnesota.     

Time-dependent green vehicle routing problem with stochastic vehicle speeds: An approximate dynamic programming algorithm

This paper addresses a Time Dependent Capacitated Vehicle Routing Problem with stochastic vehicle speeds and environmental concerns. The problem has been formulated as a Markovian Decision Process. As distinct from the traditional attempts on the problem, while estimating the amount of fuel consumption and emissions, the model takes time-dependency and stochasticity of the vehicle speeds into account. The Time Dependent Capacitated Vehicle Routing Problem is known to be NP-Hard for even deterministic settings. Incorporating uncertainty to the problem increases complexity, which renders classical optimization methods infeasible. Therefore, we propose an Approximate Dynamic Programming based heuristic as a decision aid tool for the problem. The proposed Markovian Decision Model and Approximate Dynamic Programming based heuristic are flexible in terms that more environmentally friendly solutions can be obtained by changing the objective function from cost minimization to emissions minimization. The added values of the proposed decision support tools have been shown through computational analyses on several instances. The computational analyses show that incorporating vehicle speed stochasticity into decision support models has potential to improve the performance of resulting routes in terms of travel duration, emissions and travel cost. In addition, the proposed heuristic provides promising results within relatively short computation times.     

Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach

In a heavily congested metro line, unexpected disturbances often occur to cause the delay of the traveling passengers, infeasibility of the current timetable and reduction of the operational efficiency. Due to the uncertain and dynamic characteristics of passenger demands, the commonly used method to recover from disturbances in practice is to change the timetable and rolling stock manually based on the experiences and professional judgements. In this paper, we develop a stochastic programming model for metro train rescheduling problem in order to jointly reduce the time delay of affected passengers, their total traveling time and operational costs of trains. To capture the complexity of passenger traveling characteristics, the arriving ratio of passengers at each station is modeled as a non-homogeneous poisson distribution, in which the intensity function is treated as time-varying origin-to-destination passenger demand matrices. By considering the number of on-board passengers, the total energy usage is modeled as the difference between the tractive energy consumption and the regenerative energy. Then, we design an approximate dynamic programming based algorithm to solve the proposed model, which can obtain a high-quality solution in a short time. Finally, numerical examples with real-world data sets are implemented to verify the effectiveness and robustness of the proposed approaches.     

Modelling a signal controlled traffic stream using cellular automata

Cellular automata models have formed the theory for the development of several transportation models to simulate various types of elements such as vehicles, pedestrians or even railway traffic. Furthermore, they have been applied to simulate several scenarios from very simple (freeway traffic) to rather complicated ones (lane reduction and signal optimisation). However, the properties of the model when used to simulate a signal controlled traffic stream have not been dealt with in great detail. This paper discusses several issues that arise while using the model for the simulation of traffic at signalised intersections. It also investigates the relationships between the randomisation parameter of the model, the model dynamics and the estimated saturation flow. For the deterministic version of the model, the formulas describing traffic quantities at the intersection are derived and are dependent on the desired speed – a parameter of the model. For the stochastic version, one can adopt several different approaches for the application of the randomisation rule, depending on the simulation needs.     

An application of shock wave theory to traffic signal control

A real time control policy minimizing total intersection delays subject to queue length constraints at an isolated signalized intersection is developed in this paper. The policy is derived from a new traffic model which describes the simultaneous evolution of queue lengths of two conflicting traffic streams, controlled by a traffic light, in both time and space. The model is based on the examination of shock waves generated upstream of the stop lines by the intermittent service of traffic at the signal. The proposed policy was tested against the existing pre-timed control policy at a high volume intersection and it was found superior, especially when demands increase well above the saturation level.     

Dynamic traffic routing in a network with adaptive signal control

In real traffic networks, travellers’ route choice is affected by traffic control strategies. In this research, we capture the interaction between travellers’ route choice and traffic signal control in a coherent framework. For travellers’ route choice, a VANET (Vehicular Ad hoc NETwork) is considered, where travellers have access to the real-time traffic information through V2V/V2I (Vehicle to Vehicle/Vehicle to Infrastructure) infrastructures and make route choice decisions at each intersection using hyper-path trees. We test our algorithm and control strategy by simulation in OmNet++ (A network communication simulator) and SUMO (Simulation of Urban MObility) under several scenarios. The simulation results show that with the proposed dynamic routing, the overall travel cost significantly decreases. It is also shown that the proposed adaptive signal control reduces the average delay effectively, as well as reduces the fluctuation of the average speed within the whole network.     

Self-organizing traffic signals using secondary extension and dynamic coordination

Actuated traffic signal control logic has many advantages because of its responsiveness to traffic demands, short cycles, effective use of capacity leading to and recovering from oversaturation, and amenability to aggressive transit priority. Its main drawback has been its inability to provide good progression along arterials. However, the traditional way of providing progression along arterials, coordinated–actuated control with a common, fixed cycle length, has many drawbacks stemming from its long cycle lengths, inflexibility in recovering from priority interruptions, and ineffective use of capacity during periods of oversaturation. This research explores a new paradigm for traffic signal control, “self-organizing signals,” based on local actuated control but with some additional rules that create coordination mechanisms. The primary new rules proposed are for secondary extensions, in which the green may be held to serve an imminently arriving platoon, and dynamic coordination, in which small groups of closely spaced signals communicate with one another to cycle synchronously with the group’s critical intersection. Simulation tests in VISSIM performed on arterial corridors in Massachusetts and Arizona show overall delay reductions of up to 14% compared to an optimized coordinated–actuated scheme where there is no transit priority, and more than 30% in scenarios with temporary oversaturation. Tests also show that with self-organizing control, transit signal priority can be more effective than with coordinated–actuated control, reducing transit delay by about 60%, or 12 to 14 s per intersection with little impact on traffic delay.     

Multi-criteria analysis of optimal signal plans using microscopic traffic models

Increasing concerns on environment and natural resources, coupled with increasing demand for transport, put lots of pressure for improved efficiency and performance on transport systems worldwide. New technology nowadays enables fast innovation in transport, but it is the policy for deployment and operation with a systems perspective that often determines success. Smart traffic management has played important roles for continuous development of traffic systems especially in urban areas. There is, however, still lack of effort in current traffic management and planning practice prioritizing policy goals in environment and energy. This paper presents an application of a model-based framework to quantify environmental impacts and fuel efficiency of road traffic, and to evaluate optimal signal plans with respect not only to traffic mobility performance but also other important measures for sustainability. Microscopic traffic simulator is integrated with micro-scale emission model for estimation of emissions and fuel consumption at high resolution. A stochastic optimization engine is implemented to facilitate optimal signal planning for different policy goals, including delay, stop-and-goes, fuel economy etc. In order to enhance the validity of the modeling framework, both traffic and emission models are fine-tuned using data collected in a Chinese city. In addition, two microscopic traffic models are applied, and lead to consistent results for signal optimization. Two control schemes, fixed time and vehicle actuated, are optimized while multiple performance indexes are analyzed and compared for corresponding objectives. Solutions, representing compromise between different policies, are also obtained in the case study by optimizing an integrated performance index.     

Network traffic control based on a mesoscopic dynamic flow model

The paper focuses on Network Traffic Control based on aggregate traffic flow variables, aiming at signal settings which are consistent with within-day traffic flow dynamics. The proposed optimisation strategy is based on two successive steps: the first step refers to each single junction optimisation (green timings), the second to network coordination (offsets). Both of the optimisation problems are solved through meta-heuristic algorithms: the optimisation of green timings is carried out through a multi-criteria Genetic Algorithm whereas offset optimisation is achieved with the mono-criterion Hill Climbing algorithm. To guarantee proper queuing and spillback simulation, an advanced mesoscopic traffic flow model is embedded within the network optimisation method. The adopted mesoscopic traffic flow model also includes link horizontal queue modelling. The results attained through the proposed optimisation framework are compared with those obtained through benchmark tools.