Analysing queueing at toll plazas using a coupled,multiple-queue,queueing system model: application to toll plaza design

A vehicle approaching a toll plaza observes the queues at each of the available toll-lanes before choosing which to join. This choice process, the arrival process of vehicles and the service characteristics of the toll-booths, affect the queues and delay the drivers. In this paper, queueing at a toll plaza is modelled as a multiple-queue queueing system where the arrival process to a queue (toll-lane) is dependent on the state of all the queues. In the past, such systems have been modelled mathematically only for two queues and are not applicable for toll plazas with three or more toll-lanes. The proposed model determines the steady-state probability density function (pdf) for the queues at large toll plazas. This study is used to determine the number of toll-lanes or the length of the upstream queueing area required to achieve certain user-specified levels-of-service. Expected delay and maximum queue length are used as level-of-service measures. Indicative design charts are also provided.     

Design and evaluation of dynamic traffic management strategies for congested conditions

This paper presents a procedure for dynamic design and evaluation of traffic management strategies in oversaturated conditions. The method combines a dynamic control algorithm and a disutility function. The dynamic algorithm designs signal control parameters to manage formation and dissipation of queues on system links with explicit consideration of current and projected queue lengths and demands. The disutility function measures the relative performance of the dynamic control algorithm based on preset system performance goals. The user may statically select the management strategy, or alternatively the system may be instructed to set off different management schemes based on predefined performance thresholds. The problem was formulated as one of output maximization subject to state, control, and traffic management strategy choices. Solutions were obtained using genetic algorithms. Four traffic management plans were tested to show the capabilities of the new procedure. The results show that the procedure is able to generate suitable signal control schemes that are favorable to attaining the desired traffic management goals. The results showed that multiple, or hybrids of single measures of effectiveness may need to be examined in order to correctly assess system performance. The procedure has potential for real-time implementation in an intelligent transportation system setting.     

Local ramp metering with distant downstream bottlenecks: A comparative study

The well-known feedback ramp metering algorithm ALINEA can be applied for local ramp metering or included as a key component in a coordinated ramp metering system. ALINEA uses real-time occupancy measurements from the ramp flow merging area that may be at most a few hundred meters downstream of the metered on-ramp nose. In many practical cases, however, bottlenecks with smaller capacities than the merging area may exist further downstream, which suggests using measurements from those downstream bottlenecks. Recent theoretical and simulation studies indicate that ALINEA may lead to poorly damped closed-loop behavior in this case, but PI-ALINEA, a suitable Proportional-Integral (PI) extension of ALINEA, can lead to satisfactory control performance. This paper addresses the same local ramp-metering problem in the presence of far-downstream bottlenecks, with a particular focus on the employment of PI-ALINEA to tackle three distinct cases of bottleneck that may often be encountered in practice: (1) an uphill case; (2) a lane-drop case; and (3) an un-controlled downstream on-ramp case. Extensive simulation studies are conducted on the basis of a macroscopic traffic flow model to show that ALINEA is not capable of carrying out ramp metering in these bottleneck cases, while PI-ALINEA operates satisfactorily in all cases. A field application example of PI-ALINEA is also reported with regard to a real case of far downstream bottlenecks. With its control parameters appropriately tuned beforehand, PI-ALINEA is found to be universally applicable, with little fine-tuning required for field applications.     

Access control policies without inside queues: Their properties and public policy implications

An access control policy that eliminates all queues beyond the entry points to a network has obvious benefits, which include smooth travel and predictable travel times inside the network. Yet it has never been proven, to the best of our knowledge, whether excluding inside queues yields sub-optimal network performance or, in other words, allowing inside queues can actually further reduce the system travel cost. Moreover, it is not clear whether an optimal control policy derived from efficiency considerations can also be a fair policy to all road users. This paper provide answers to these questions in the context of a monocentric network. By analyzing the structure of the access control problem considering all feasible policies (with/without inside queues), we show that the minimal system cost realizable by access control can be obtained without directly solving a non-convex optimization program, and can indeed always be achieved by a control policy excluding all of the inside queues. These optimal policies are defined by a polyhedral set and a Finite Generation Algorithm can be applied to derive the analytical form of this set. The optimal policies are not unique in general, thus making it possible to achieve both minimal system cost and fairness simultaneously.     

A modified reinforcement learning algorithm for solving coordinated signalized networks

This study proposes Reinforcement Learning (RL) based algorithm for finding optimum signal timings in Coordinated Signalized Networks (CSN) for fixed set of link flows. For this purpose, MOdified REinforcement Learning algorithm with TRANSYT-7F (MORELTRANS) model is proposed by way of combining RL algorithm and TRANSYT-7F. The modified RL differs from other RL algorithms since it takes advantage of the best solution obtained from the previous learning episode by generating a sub-environment at each learning episode as the same size of original environment. On the other hand, TRANSYT-7F traffic model is used in order to determine network performance index, namely disutility index. Numerical application is conducted on medium sized coordinated signalized road network. Results indicated that the MORELTRANS produced slightly better results than the GA in signal timing optimization in terms of objective function value while it outperformed than the HC. In order to show the capability of the proposed model for heavy demand condition, two cases in which link flows are increased by 20% and 50% with respect to the base case are considered. It is found that the MORELTRANS is able to reach good solutions for signal timing optimization even if demand became increased.     

On the macroscopic stability of freeway traffic

A simple model of traffic flow is used to analyze the spatio-temporal distribution of flow and density on closed-loop homogeneous freeways with many ramps, which produce inflows and allow outflows. As we would expect, if the on-ramp demand is space-independent then this distribution tends toward uniformity in space if the freeway is either: (i) uncongested; or (ii) congested with queues on its on-ramps and enough inflow to cause the average freeway density to increase with time. In all other cases, however, including any recovery phase of a rush hour where the freeway’s average density declines, the distribution of flow and density quickly becomes uneven. This happens even under conditions of perfect symmetry, where the percentage of vehicles exiting at every off ramp is the same. The flow-density deviations from the average are shown to grow exponentially in time and propagate backwards in space with a fixed wave speed. A consequence of this type of instability is that, during recovery, gaps of uncongested traffic will quickly appear in the unevenly congested stream, reducing average flow. This extends the duration of recovery and invariably creates clockwise hysteresis loops on scatter-plots of average system flow vs. density during any rush hour that oversaturates the freeway. All these effects are quantified with formulas and verified with simulations. Some have been observed in real networks. In a more practical vein, it is also shown that the negative effects of instability diminish (i.e., freeway flows increase) if (a) some drivers choose to exit the freeway prematurely when it is too congested and/or (b) freeway access is regulated in a certain traffic-responsive way. These two findings could be used to improve the algorithms behind VMS displays for driver guidance (finding a), and on-ramp metering rates (finding b).     

Dynamic charging-while-driving systems for freight delivery services with electric vehicles: Traffic and energy modelling

This paper presents a research on traffic modelling developed for assessing traffic and energy performance of electric systems installed along roads for dynamic charging-while-driving (CWD) of fully electric vehicles (FEVs).The logic adopted by the developed traffic model is derived from a particular simulation scenario of electric charging: a freight distribution service operated using medium-sized vans. In this case, the CWD service is used to recover the state of charge of the FEV batteries to shortly start with further activities after arrival at the depot.The CWD system is assumed to be implemented in a multilane ring road with several intermediate on-ramp entrances, where the slowest lane is reserved for the dynamic charging of authorized electric vehicles. A specific traffic model is developed and implemented based on a mesoscopic approach, where energy requirements and charging opportunities affect driving and traffic behaviours. Overtaking manoeuvres as well as new entries in the CWD lane of vehicles that need to charge are modelled according to a cooperative driving system, which manages adequate time gaps between consecutive vehicles. Finally, a speed control strategy is simulated at a defined node to create an empty time-space slot in the CWD lane, by delaying the arriving vehicles. This simulated control, implemented to allow maintenance operations for CWD that may require clearing a charging zone for a short time slot, could also be applied to facilitate on-ramp merging manoeuvres.     

Definition of a merging assistant strategy using intelligent vehicles

In the area of active traffic management, new technologies provide opportunities to improve the use of current infrastructure. Vehicles equipped with in-car communication systems are capable of exchanging messages with the infrastructure and other vehicles. This new capability offers many opportunities for traffic management. This paper presents a novel merging assistant strategy that exploits the communication capabilities of intelligent vehicles. The proposed control requires the cooperation of equipped vehicles on the main carriageway in order to create merging gaps for on-ramp vehicles released by a traffic light. The aim is to reduce disruptions to the traffic flow created by the merging vehicles. This paper focuses on the analytical formulation of the control algorithm, and the traffic flow theories used to define the strategy. The dynamics of the gap formation derived from theoretical considerations are validated using a microscopic simulation. The validation indicates that the control strategy mostly developed from macroscopic theory well approximates microscopic traffic behaviour. The results present encouraging capabilities of the system. The size and frequency of the gaps created on the main carriageway, and the space and time required for their creation are compatible with a real deployment of the system. Finally, we summarise the results of a previous study showing that the proposed merging strategy reduces the occurrence of congestion and the number of late-merging vehicles. This innovative control strategy shows the potential of using intelligent vehicles for facilitating the merging manoeuvre through use of emerging communications technologies.     

On dynamics of traffic queues in a road network with route choice based on real time traffic information

Introducing real time traffic information into transportation network makes it necessary to consider development of queues and traffic flows as a dynamic process. This paper initiates a theoretical study of conditions under which this process is stable. A model is presented that describes within-one-day development of queues when drivers affected by real-time traffic information choose their paths en route. The model is reduced to a system of differential equations with delay. Equilibrium points of the system correspond to constant queue lengths. Stability of the system is investigated using characteristic values of the linearised minimal face flow. A traffic network example illustrating the method is provided.     

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers).The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-h traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.     

Automated transit headway control via adaptive signal priority

This paper reports on a study that developed a next‐generation Transit Signal Priority (TSP) strategy, Adaptive TSP, that controls adaptively transit operations of high frequency routes using traffic signals, thus automating the operations control task and relieving transit agencies of this burden. The underlying algorithm is based on Reinforcement Learning (RL), an emerging Artificial Intelligence method. The developed RL agent is responsible for determining the best duration of each signal phase such that transit vehicles can recover to the scheduled headway taking into consideration practical phase length constraints. A case study was carried out by employing the microscopic traffic simulation software Paramics to simulate transit and traffic operations at one signalized intersection along the King Streetcar route in downtown Toronto. The results show that the control policy learned by the agent could effectively reduce the transit headway deviation and causes smaller disruption to cross street traffic compared with the existing unconditional transit signal priority algorithm.     

Control Concepts for Facilitating Motorway On-ramp Merging Using Intelligent Vehicles

Abstract

Congestion at motorway junctions is a traffic phenomenon that degrades operation of infrastructure and can lead to breakdown of traffic flow and associated reduction in capacity. Advanced communication technologies open new possibilities to prevent or at least delay this phenomenon, and innovative active traffic management systems have been developed in the recent years for better control of motorway traffic. This paper presents a review of control strategies for facilitating motorway on-ramp merging using intelligent vehicles. First, the concepts of the control algorithms are reviewed chronologically divided into three types of intelligent vehicle: completely automated, equipped with cooperative adaptive cruise control and equipped with on-board display. Then, a common structure is identified, and the algorithms are presented based on their characteristics in order to identify similarities, dissimilarities, trends and possible future research directions. Finally, using a similar approach, a review of the methods used to evaluate these control strategies identifies important aspects that should be considered by further research on this topic.     

Human-like autonomous car-following model with deep reinforcement learning

This study proposes a framework for human-like autonomous car-following planning based on deep reinforcement learning (deep RL). Historical driving data are fed into a simulation environment where an RL agent learns from trial and error interactions based on a reward function that signals how much the agent deviates from the empirical data. Through these interactions, an optimal policy, or car-following model that maps in a human-like way from speed, relative speed between a lead and following vehicle, and inter-vehicle spacing to acceleration of a following vehicle is finally obtained. The model can be continuously updated when more data are fed in. Two thousand car-following periods extracted from the 2015 Shanghai Naturalistic Driving Study were used to train the model and compare its performance with that of traditional and recent data-driven car-following models. As shown by this study’s results, a deep deterministic policy gradient car-following model that uses disparity between simulated and observed speed as the reward function and considers a reaction delay of 1 s, denoted as DDPGvRT, can reproduce human-like car-following behavior with higher accuracy than traditional and recent data-driven car-following models. Specifically, the DDPGvRT model has a spacing validation error of 18% and speed validation error of 5%, which are less than those of other models, including the intelligent driver model, models based on locally weighted regression, and conventional neural network-based models. Moreover, the DDPGvRT demonstrates good capability of generalization to various driving situations and can adapt to different drivers by continuously learning. This study demonstrates that reinforcement learning methodology can offer insight into driver behavior and can contribute to the development of human-like autonomous driving algorithms and traffic-flow models.     

A dual control approach for repeated anticipatory traffic control with estimation of network flow sensitivity

This paper investigates a strategic signal control, which anticipates travelers' route choice response and determines signal timings to optimize network‐wide objectives. In general traffic assignment models are used for anticipating this route choice response. However, model‐reality mismatch usually brings suboptimal solutions to the real system. A repeated anticipatory control resolves the suboptimality and addresses the modeling error by learning from information on model bias. This paper extends the repeated control approach and focuses on the estimation of flow sensitivity as well as its influence on control, which is a crucial issue in implementation of model bias correction. The main objective of this paper is first to analyze the estimation error in the real flow derivative that is estimated from noisy measurements. A dual control method is then presented, improving both optimization objective function and derivative estimation during the control process. The proposed dual algorithm is tested on a simple network as well as on a midsize network. Numerical examples confirm the reliable performance of the new reality‐tracking control strategy and its ability to identify (local) optimal solutions on real traffic networks. Copyright © 2016 John Wiley & Sons, Ltd.     

Increasing the capacity of an isolated merge by metering its on-ramp

Measurements taken downstream of freeway/on-ramp merges have previously shown that discharge flow diminishes when a merge becomes an isolated bottleneck. By means of observation and experiment, we show here that metering an on-ramp can recover the higher discharge flow at a merge and thereby increase the merge capacity. Detailed observations were collected at a single merge using video. These data revealed that the reductions in discharge flow are triggered by a queue that forms near the merge in the freeway shoulder lane and then spreads laterally, as drivers change lanes to maneuver around slow traffic. Our experiments show that once restrictive metering mitigated this shoulder lane queue, high outflows often returned to the median lane. High merge outflows could be restored in all freeway lanes by then relaxing the metering rate so that inflows from the on-ramp increased. Although outflows recovered in this fashion were not sustained for periods greater than 13 min, the findings are the first real evidence that ramp metering can favorably affect the capacity of an isolated merge. Furthermore, these findings point to control strategies that might generate higher outflows for more prolonged periods and increase merge capacity even more. Finally, the findings uncover details of merge operation that are essential for developing realistic theories of merging traffic.     

Modeling and simulation of vehicle projection arrival–discharge process in adaptive traffic signal controls

Real‐time signal control operates as a function of the vehicular arrival and discharge process to satisfy a pre‐specified operational performance. This process is often predicted based on loop detectors placed upstream of the signal. In our newly developed signal control for diamond interchanges, a microscopic model is proposed to estimate traffic flows at the stop‐line. The model considers the traffic dynamics of vehicular detection, arrivals, and departures, by taking into account varying speeds, length of queues, and signal control. As the signal control is optimized over a rolling horizon that is divided into intervals, the vehicular detection for and projection into the corresponding horizon intervals are also modeled. The signal control algorithm is based on dynamic programming and the optimization of signal policy is performed using a certain performance measure involving delays, queue lengths, and queue storage ratios. The arrival–discharge model is embedded in the optimization algorithm and both are programmed into AIMSUN, a microscopic stochastic simulation program. AIMSUN is then used to simulate the traffic flow and implement the optimal signal control by accessing internal data including detected traffic demand and vehicle speeds. Sensitivity analysis is conducted to study the effect of selecting different optimization criteria on the signal control performance. It is concluded that the queue length and queue storage ratio are the most appropriate performance measures in real‐time signal control of interchanges. Copyright © 2010 John Wiley & Sons, Ltd.     

Adaptive traffic signal control using approximate dynamic programming

This paper presents a study on an adaptive traffic signal controller for real-time operation. The controller aims for three operational objectives: dynamic allocation of green time, automatic adjustment to control parameters, and fast revision of signal plans. The control algorithm is built on approximate dynamic programming (ADP). This approach substantially reduces computational burden by using an approximation to the value function of the dynamic programming and reinforcement learning to update the approximation. We investigate temporal-difference learning and perturbation learning as specific learning techniques for the ADP approach. We find in computer simulation that the ADP controllers achieve substantial reduction in vehicle delays in comparison with optimised fixed-time plans. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised, which can be achieved conveniently using the ADP approach.     

Orchestration of driving simulator scenarios based on dynamic actor preparation and automated action planning

In driving simulation, a scenario includes definitions of the road environment, the traffic situation, simulated vehicles’ interactions with the participant’s vehicle and measurements that need to be collected. The scenarios need to be designed in such a way that the research questions to be studied can be answered, which commonly imply exposing the participant for a couple of predefined specific situations that has to be both realistic and repeatable. This article presents an integrated algorithm based on Dynamic Actor Preparation and Automated Action Planning to control autonomous simulated vehicles in the simulation in order to generate predefined situations. This algorithm is thus able to plan driving actions for autonomous vehicles based on specific tasks with relevant contextual information as well as handling longitudinal transportation of simulated vehicles based on the contextual information in an automated manner. The conducted experiment shows that the algorithm is able to guarantee repeatability under autonomous traffic flow. The presented algorithm can benefit not only the driving simulation community, but also relevant areas, such as autonomous vehicle and in-vehicle device design by providing them with an algorithm for target pursue and driving task accomplishment, which can be used to design a human-vehicle cooperation system in the coming era of autonomous driving.     

Pareto-improving ramp metering strategies for reducing congestion in the morning commute

This paper presents an alternative approach to internalize congestion externality during the morning commute. We consider a linear freeway with multiple on-ramps and a downstream bottleneck and commuters accessing the freeway via different on-ramps try to arrive at work on time. Rather than charging congestion tolls as widely suggested by economists, we show that the old-fashioned engineering approach – ramp metering – can be a powerful tool to affect travelers’ departure time choice and thereby alter the congestion externality distribution among travelers. With carefully designed time-dependent metering plans, travelers from different origins can be channelized and will access the freeway bottleneck in different time periods, resulting in less total cost for the system compared to the no-metering case. The metering strategies are Pareto-improving, with travelers from the on-ramp with the highest priority having the smallest individual costs and travelers from the on-ramp with the lowest priority having their costs equal to those in the no-metering scenario. By changing the priority order of the ramps periodically, the benefit of the Pareto-improving metering strategies can be distributed evenly among all travelers. Numerical experiments show that the total user cost can be reduced by up to 40% with the proposed metering strategies. This study offers researchers and policy makers a different angle of looking at congestion externality, and the results provide an overview of the potential long term benefits that dynamic ramp metering strategies can achieve.