Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers).The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-h traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.     

Stability and bifurcation in network traffic flow: A Poincaré map approach

Previous studies have shown that, in a diverge-merge network with two intermediate links (the DM network), the kinematic wave model always admits stationary solutions under constant boundary conditions, but periodic oscillations can develop from empty initial conditions. Such contradictory observations suggest that the stationary states be unstable. In this study we develop a systematic approach to investigate the stability property of stationary states in this and other networks within the framework of network kinematic wave theories. Based on the observation that kinematic waves propagate in a circular path when only one of the two intermediate links is congested, we derive a one-dimensional, discrete Poincaré map in the out-flux at a Poincaré section. We then prove that the fixed points of the Poincaré map correspond to stationary flow-rates on the two links. With Lyapunov’s first method, we demonstrate that the Poincaré map can be finite-time stable, asymptotically stable, or unstable. When unstable, the map is found to have periodical points of period two, but no chaotic solutions. We further analyze the bifurcation in the stability of the Poincaré map caused by varying route choice proportions. We apply the Poincaré map approach to analyzing traffic patterns in more general (DM)ⁿ and beltway networks, which are sufficient and necessary structures for network-induced unstable traffic and gridlock, respectively. This study demonstrates that the Poincaré map approach can be efficiently applied to analyze traffic dynamics in any road networks with circular information propagation and provides new insights into unstable traffic dynamics caused by interactions among network bottlenecks.     

Cooperative traffic control of a mixed network with two urban regions and a freeway

Currently most optimization methods for urban transport networks (i) are suited for networks with simplified dynamics that are far from real-sized networks or (ii) apply decentralized control, which is not appropriate for heterogeneously loaded networks or (iii) investigate good-quality solutions through micro-simulation models and scenario analysis, which make the problem intractable in real time. In principle, traffic management decisions for different sub-systems of a transport network (urban, freeway) are controlled by operational rules that are network specific and independent from one traffic authority to another. In this paper, the macroscopic traffic modeling and control of a large-scale mixed transportation network consisting of a freeway and an urban network is tackled. The urban network is partitioned into two regions, each one with a well-defined Macroscopic Fundamental Diagram (MFD), i.e. a unimodal and low-scatter relationship between region density and outflow. The freeway is regarded as one alternative commuting route which has one on-ramp and one off-ramp within each urban region. The urban and freeway flow dynamics are formulated with the tool of MFD and asymmetric cell transmission model, respectively. Perimeter controllers on the border of the urban regions operating to manipulate the perimeter interflow between the two regions, and controllers at the on-ramps for ramp metering are considered to control the flow distribution in the mixed network. The optimal traffic control problem is solved by a Model Predictive Control (MPC) approach in order to minimize total delay in the entire network. Several control policies with different levels of urban-freeway control coordination are introduced and tested to scrutinize the characteristics of the proposed controllers. Numerical results demonstrate how different levels of coordination improve the performance once compared with independent control for freeway and urban network. The approach presented in this paper can be extended to implement efficient real-world control strategies for large-scale mixed traffic networks.     

A sensitivity-based approach for adaptive decomposition of anticipatory network traffic control

Anticipatory optimal network control is defined as the problem of determining the set of control actions that minimizes a network-wide objective function. This not only takes into account local consequences on the propagation of flows, but also the global network-wide routing behavior of the users. Such an objective function is, in general, defined in a centralized setting, as knowledge regarding the whole network is needed to correctly compute it. Reaching a level of centralization sufficient to attain network-wide control objectives is however rarely realistic in practice. Multiple authorities are influencing different portions the network, separated either hierarchically or geographically. The distributed nature of networks and traffic directly influences the complexity of the anticipatory control problem.This is our motivation for this work, in which we introduce a decomposition mechanism for the global anticipatory network traffic control problem, based on dynamic clustering of traffic controllers. Rather than solving the full centralized problem, or blindly performing a full controller-wise decomposition, this technique allows recognizing when and which controllers should be grouped in clusters, and when, instead, these can be optimized separately.The practical relevance with respect to our motivation is that our approach allows identification of those network traffic conditions in which multiple actors need to actively coordinate their actions, or when unilateral action suffices for still approximating global optimality.This clustering procedure is based on well-known algebraic and statistical tools that exploit the network’s sensitivity to control and its structure to deduce coupling behavior. We devise several case studies in order to assess our newly introduced procedure’s performances, in comparison with fully decomposed and fully centralized anticipatory optimal network control, and show that our approach is able to outperform both centralized and decomposed procedures.     

India’s air traffic: towards an empirical framework for systems analysis

The paper explores patterns of flows within India’s Air Traffic System through the lens of carriers’ networks and route structures between 2006 and 2014. Through observations of frequency distributions and their distinct patterns an analytic framework is derived heuristically by means of classification and aggregation. The well-known skewed traffic distribution which spatially concentrates traffic around relatively few airports serves as the starting point for decomposing the air traffic system (ATS) into its constituent route types. Airline operations along distinct route classes allows for classifying individual carrier’s network features as an embedded part of the system. Discussion of their role includes a spatial component. Inferences about development paths – past, present, future – of the Indian commercial ATS can be made.     

Modelling the resilience,friability and costs of an air transport network affected by a large-scale disruptive event

This paper deals with developing a methodology for estimating the resilience, friability, and costs of an air transport network affected by a large-scale disruptive event. The network consists of airports and airspace/air routes between them where airlines operate their flights. Resilience is considered as the ability of the network to neutralize the impacts of disruptive event(s). Friability implies reducing the network’s existing resilience due to removing particular nodes/airports and/or links/air routes, and consequently cancelling the affected airline flights. The costs imply additional expenses imposed on airports, airlines, and air passengers as the potentially most affected actors/stakeholders due to mitigating actions such as delaying, cancelling and rerouting particular affected flights. These actions aim at maintaining both the network’s resilience and safety at the acceptable level under given conditions.Large scale disruptive events, which can compromise the resilience and friability of a given air transport network, include bad weather, failures of particular (crucial) network components, the industrial actions of the air transport staff, natural disasters, terrorist threats/attacks and traffic incidents/accidents.The methodology is applied to the selected real-life case under given conditions. In addition, this methodology could be used for pre-selecting the location of airline hub airport(s), assessing the resilience of planned airline schedules and the prospective consequences, and designing mitigating measures before, during, and in the aftermath of a disruptive event. As such, it could, with slight modifications, be applied to transport networks operated by other transport modes.     

Reprint of “Modelling the resilience,friability and costs of an air transport network affected by a large-scale disruptive event”

This paper deals with developing a methodology for estimating the resilience, friability, and costs of an air transport network affected by a large-scale disruptive event. The network consists of airports and airspace/air routes between them where airlines operate their flights. Resilience is considered as the ability of the network to neutralize the impacts of disruptive event(s). Friability implies reducing the network’s existing resilience due to removing particular nodes/airports and/or links/air routes, and consequently cancelling the affected airline flights. The costs imply additional expenses imposed on airports, airlines, and air passengers as the potentially most affected actors/stakeholders due to mitigating actions such as delaying, cancelling and rerouting particular affected flights. These actions aim at maintaining both the network’s resilience and safety at the acceptable level under given conditions.Large scale disruptive events, which can compromise the resilience and friability of a given air transport network, include bad weather, failures of particular (crucial) network components, the industrial actions of the air transport staff, natural disasters, terrorist threats/attacks and traffic incidents/accidents.The methodology is applied to the selected real-life case under given conditions. In addition, this methodology could be used for pre-selecting the location of airline hub airport(s), assessing the resilience of planned airline schedules and the prospective consequences, and designing mitigating measures before, during, and in the aftermath of a disruptive event. As such, it could, with slight modifications, be applied to transport networks operated by other transport modes.     

Nonlinear gating control for urban road traffic network using the network fundamental diagram

This work proposes a nonlinear model predictive controller for the urban gating problem. The system model is formalized based on a research on existing models of the network fundamental diagram and the perimeter control systems. For the existing models, modifications are suggested: additional state variables are allocated to describe the queue dynamics at the network gates. Using the extended model, a nonlinear model predictive controller is designed offering a ‘non‐greedy’ policy compared with previous, ‘greedy’ gating control designs. The greedy and non‐greedy nonlinear model predictive control (NMPC) controllers are compared with a greedy linear feedback proportional‐integral‐derivative (PID) controller in different traffic situations. The proposed non‐greedy NMPC controller outperforms the other two approaches in terms of travel distance performance and queue lengths. The performance results justify the consideration of queue lengths in dynamic modeling, and the use of NMPC approach for controller design. Copyright © 2014 John Wiley & Sons, Ltd.     

Strategic de-confliction in the presence of a large number of 4D trajectories using a causal modeling approach

This paper presents a strategic de-confliction algorithm based on causal modeling developed under the STREAM project and launched under the umbrella of the Single European Sky ATM Research (SESAR) Program. The basic underlying concept makes use of the enriched information included in the Shared Business Trajectories (SBTs) of the flights prior to takeoff (or in the Reference Business Trajectories (RBTs) if the flight is airborne) to allocate conflict-free trajectories in a traffic planning phase that should lead to an actual conflict-free scenario in the flight execution phase in the absence of flight and/or network uncertainties. The proposed approach could decrease the workload of the air traffic controllers, thus improving the Air Traffic Management (ATM) capacity while meeting the maximum possible expectations of the Airspace Users’ requirements in terms of horizontal flight efficiency. The main modules of the implemented system are also presented in this paper; these modules are designed to enable the processing of thousands of trajectories within a few seconds or minutes and encompass a global network scope with a planning horizon of approximately 2–3 h. The causal model applied for network conflict resolution and flight routing allocation is analyzed to demonstrate how the emergent dynamics (i.e., domino effects) of local trajectory amendments can be efficiently explored to identify conflict-free Pareto-efficient network scenarios. Various performance indicators can be taken into account in the multi-criteria optimization process, thus offering to the network manager a flexible tool for fostering a collaborative planning process.     

Recent developments in traffic flow modeling using macroscopic fundamental diagram

ABSTRACT

This paper presents an overview of the recent developments in traffic flow modelling and analysis using macroscopic fundamental diagram (MFD) as well as their applications. In recent literature, various aggregated traffic models have been proposed and studied to analyse traffic flow while enhancing network efficiency. Many of these studies have focused on models based on MFD that describes the relationship between aggregated flow and aggregated density of transport networks. The analysis of MFD has been carried out based on experimental data collected from sensors and GPS, as well as simulation models. Several factors are found to influence the existence and shape of MFD, including traffic demand, network and signal settings, and route choices. As MFD can well express the traffic dynamics of large urban transport networks, it has been extensively applied to traffic studies, including the development of network-wide control strategies, network partitioning, performance evaluation, and road pricing. This work also presents future extensions and research directions for MFD-based traffic modelling and applications.     

Personalized real-time traffic information provision: Agent-based optimization model and solution framework

The advancement of information and communication technology allows the use of more sophisticated information provision strategies for real-time congested traffic management in a congested network. This paper proposes an agent-based optimization modeling framework to provide personalized traffic information for heterogeneous travelers. Based on a space–time network, a time-dependent link flow based integer programming model is first formulated to optimize various information strategies, including elements of where and when to provide the information, to whom the information is given, and what alternative route information should be suggested. The analytical model can be solved efficiently using off-the-shelf commercial solvers for small-scale network. A Lagrangian Relaxation-based heuristic solution approach is developed for medium to large networks via the use of a mesoscopic dynamic traffic simulator.     

Dynamic traffic routing in a network with adaptive signal control

In real traffic networks, travellers’ route choice is affected by traffic control strategies. In this research, we capture the interaction between travellers’ route choice and traffic signal control in a coherent framework. For travellers’ route choice, a VANET (Vehicular Ad hoc NETwork) is considered, where travellers have access to the real-time traffic information through V2V/V2I (Vehicle to Vehicle/Vehicle to Infrastructure) infrastructures and make route choice decisions at each intersection using hyper-path trees. We test our algorithm and control strategy by simulation in OmNet++ (A network communication simulator) and SUMO (Simulation of Urban MObility) under several scenarios. The simulation results show that with the proposed dynamic routing, the overall travel cost significantly decreases. It is also shown that the proposed adaptive signal control reduces the average delay effectively, as well as reduces the fluctuation of the average speed within the whole network.     

Spatial and temporal characterization of travel patterns in a traffic network using vehicle trajectories

This paper presents a trajectory clustering method to discover spatial and temporal travel patterns in a traffic network. The study focuses on identifying spatially distinct traffic flow groups using trajectory clustering and investigating temporal traffic patterns of each spatial group. The main contribution of this paper is the development of a systematic framework for clustering and classifying vehicle trajectory data, which does not require a pre-processing step known as map-matching and directly applies to trajectory data without requiring the information on the underlying road network. The framework consists of four steps: similarity measurement, trajectory clustering, generation of cluster representative subsequences, and trajectory classification. First, we propose the use of the Longest Common Subsequence (LCS) between two vehicle trajectories as their similarity measure, assuming that the extent to which vehicles’ routes overlap indicates the level of closeness and relatedness as well as potential interactions between these vehicles. We then extend a density-based clustering algorithm, DBSCAN, to incorporate the LCS-based distance in our trajectory clustering problem. The output of the proposed clustering approach is a few spatially distinct traffic stream clusters, which together provide an informative and succinct representation of major network traffic streams. Next, we introduce the notion of Cluster Representative Subsequence (CRS), which reflects dense road segments shared by trajectories belonging to a given traffic stream cluster, and present the procedure of generating a set of CRSs by merging the pairwise LCSs via hierarchical agglomerative clustering. The CRSs are then used in the trajectory classification step to measure the similarity between a new trajectory and a cluster. The proposed framework is demonstrated using actual vehicle trajectory data collected from New York City, USA. A simple experiment was performed to illustrate the use of the proposed spatial traffic stream clustering in application areas such as network-level traffic flow pattern analysis and travel time reliability analysis.     

Network-oriented household activity pattern problem for system optimization

The recently emerging trend of self-driving vehicles and information sharing technologies, made available by private technology vendors, starts creating a revolutionary paradigm shift in the coming years for traveler mobility applications. By considering a deterministic traveler decision making framework at the household level in congested transportation networks, this paper aims to address the challenges of how to optimally schedule individuals’ daily travel patterns under the complex activity constraints and interactions. We reformulate two special cases of household activity pattern problem (HAPP) through a high-dimensional network construct, and offer a systematic comparison with the classical mathematical programming models proposed by Recker (1995). Furthermore, we consider the tight road capacity constraint as another special case of HAPP to model complex interactions between multiple household activity scheduling decisions, and this attempt offers another household-based framework for linking activity-based model (ABM) and dynamic traffic assignment (DTA) tools. Through embedding temporal and spatial relations among household members, vehicles and mandatory/optional activities in an integrated space-time-state network, we develop two 0–1 integer linear programming models that can seamlessly incorporate constraints for a number of key decisions related to vehicle selection, activity performing and ridesharing patterns under congested networks. The well-structured network models can be directly solved by standard optimization solvers, and further converted to a set of time-dependent state-dependent least cost path-finding problems through Lagrangian relaxation, which permit the use of computationally efficient algorithms on large-scale high-fidelity transportation networks.     

Agent-based en-route diversion: Dynamic behavioral responses and network performance represented by Macroscopic Fundamental Diagrams

This paper focuses on modeling agents’ en-route diversion behavior under information provision. The behavior model is estimated based on naïve Bayes rules and re-calibrated using a Bayesian approach. Stated-preference driving simulator data is employed for model estimation. Bluetooth-based field data is employed for re-calibration. Then the behavior model is integrated with a simulation-based dynamic traffic assignment model. A traffic incident scenario along with variable message signs (VMS) is designed and analyzed under the context of a real-world large-scale transportation network to demonstrate the integrated model and the impact of drivers’ dynamic en-route diversion behavior on network performance. Macroscopic Fundamental Diagram (MFD) is employed as a measurement to represent traffic dynamics. This research has quantitatively evaluated the impact of information provision and en-route diversion in a VMS case study. It proposes and demonstrates an original, complete, behaviorally sound, and cost-effective modeling framework for potential analyses and evaluations related to Advanced Traffic Information System (ATIS) and real-time operational applications.     

Animal dynamics based approach for modeling pedestrian crowd egress under panic conditions

Collective movement is important during emergencies such as natural disasters or terrorist attacks, when rapid egress is essential for escape. The development of quantitative theories and models to explain and predict the collective dynamics of pedestrians has been hindered by the lack of complementary data under emergency conditions. Collective patterns are not restricted to humans, but have been observed in other non-human biological systems. In this study, a mathematical model for crowd panic is derived from collective animal dynamics. The development and validation of the model is supported by data from experiments with panicking Argentine ants (Linepithema humile). A first attempt is also made to scale the model parameters for collective pedestrian traffic from those for ant traffic, by employing a scaling concept approach commonly used in biology.     

Modelling contact mode and frequency of interactions with social network members using the multiple discrete–continuous extreme value model

Communication patterns are an integral component of activity patterns and the travel induced by these activities. The present study aims to understand the determinants of the communication patterns (by the modes face-to-face, phone, e-mail and SMS) between people and their social network members. The aim is for this to eventually provide further insights into travel behaviour for social and leisure purposes. A social network perspective brings value to the study and modelling of activity patterns since leisure activities are influenced not only by traditional trip measures such as time and cost but also motivated extensively by the people involved in the activity. By using a multiple discrete-continuous extreme value model (Bhat, 2005), we can investigate the means of communication chosen to interact with a given social network member (multiple discrete choices) and the frequency of interaction by each mode (treated as continuous) at the same time. The model also allows us to investigate satiation effects for different modes of communication. Our findings show that in spite of people having increasingly geographically widespread networks and more diverse communication technologies, a strong underlying preference for face-to-face contact remains. In contrast with some of the existing work, we show that travel-related variables at the ego level are less important than specific social determinants which can be considered while making use of social network data.     

Boundedly rational user equilibria (BRUE): Mathematical formulation and solution sets

Boundedly rational user equilibria (BRUE) represent traffic flow distribution patterns where travellers can take any route whose travel cost is within an ‘indifference band’ of the shortest path cost. Those traffic flow patterns satisfying the above condition constitute a set, named the BRUE solution set. It is important to obtain all the BRUE flow patterns, because it can help predict the variation of the link flow pattern in a traffic network under the boundedly rational behavior assumption. However, the methodology of constructing the BRUE set has been lacking in the established literature. This paper fills the gap by constructing the BRUE solution set on traffic networks with fixed demands. After defining ε-BRUE, where ε is the indifference band for the perceived travel cost, we formulate the ε-BRUE problem as a nonlinear complementarity problem (NCP), so that a BRUE solution can be obtained by solving a BRUE–NCP formulation. To obtain the BRUE solution set encompassing all BRUE flow patterns, we propose a methodology of generating acceptable path set which may be utilized under the boundedly rational behavior assumption. We show that with the increase of the indifference band, the acceptable path set that contains boundedly rational equilibrium flows will be augmented, and the critical values of indifference band to augment these path sets can be identified by solving a family of mathematical programs with equilibrium constraints (MPEC) sequentially. The BRUE solution set can then be obtained by assigning all traffic demands to the acceptable path set. Various numerical examples are given to illustrate our findings.     

Network-wide traffic and environmental impacts of acceleration and deceleration among Eco-Driving Vehicles in different road configurations

Eco-Driving, a driver behaviour-based method, has featured in a number of national policy documents as part of CO₂ emission reduction or climate change strategies. This investigation comprises a detailed assessment of acceleration and deceleration in Eco-Driving Vehicles at different penetration levels in the vehicle fleet, under varying traffic composition and volume. The impacts of Eco-Driving on network-wide traffic and environmental performance at a number of speed-restricted road networks (30?km/h) is quantified using microsimulation. The results show that increasing levels of Eco-Driving in certain road networks result in significant environmental and traffic congestion detriments at the road network level in the presence of heavy traffic. Increases in CO₂ emissions of up to 18% were found. However, with the addition of vehicle-to-vehicle or vehicle-to-infrastructure communication technology which facilitates dynamic driving control on speed and acceleration/deceleration in vehicles, improvements in CO₂ emissions and traffic congestion are possible using Eco-Driving.     

Behavior-based consistency-seeking models as deployment alternatives to dynamic traffic assignment models

This paper proposes a behavior-based consistency-seeking (BBCS) model as an alternative to the dynamic traffic assignment paradigm for the real-time control of traffic systems under information provision. The BBCS framework uses a hybrid probabilistic–possibilistic model to capture the day-to-day evolution and the within-day dynamics of individual driver behavior. It considers heterogeneous driver classes based on the broad behavioral characteristics of drivers elicited from surveys and past studies on driver behavior. Fuzzy logic and if–then rules are used to model the various driver behavior classes. The approach enables the modeling of information characteristics and driver response to be more consistent with the real-world. The day-to-day evolution of driver behavior characteristics is reflected by updating the appropriate model parameters based on the current day’s experience. The within-day behavioral dynamics are reactive and capture drivers’ actions vis-à-vis the ambient driving conditions by updating the weights associated with the relevant if–then rules. The BBCS model is deployed by updating the ambient driver behavior class fractions so as to ensure consistency with the real-time traffic sensor measurements. Simulation experiments are conducted to investigate the real-time applicability of the proposed framework to a real-world network. The results suggest that the approach can reasonably capture the within-day variations in driver behavior model parameters and class fractions in the traffic stream. Also, they indicate that deployment-capable information strategies can be used to influence system performance. From a computational standpoint, the approach is real-time deployable.