Asynchronous n-step Q-learning adaptive traffic signal control

Ensuring transportation systems are efficient is a priority for modern society. Intersection traffic signal control can be modeled as a sequential decision-making problem. To learn how to make the best decisions, we apply reinforcement learning techniques with function approximation to train an adaptive traffic signal controller. We use the asynchronous n-step Q-learning algorithm with a two hidden layer artificial neural network as our reinforcement learning agent. A dynamic, stochastic rush hour simulation is developed to test the agent’s performance. Compared against traditional loop detector actuated and linear Q-learning traffic signal control methods, our reinforcement learning model develops a superior control policy, reducing mean total delay by up 40% without compromising throughput. However, we find our proposed model slightly increases delay for left turning vehicles compared to the actuated controller, as a consequence of the reward function, highlighting the need for an appropriate reward function which truly develops the desired policy.     

Assessing the Potential Impacts of Connected Vehicles: Mobility,Environmental, and Safety Perspectives

The connected vehicle is a rapidly emerging paradigm aimed at deploying and developing a fully connected transportation system that enables data exchange among vehicles, infrastructure, and mobile devices to improve mobility, enhance safety, and reduce the adverse environmental impacts of the transportation systems. This study focuses on micromodeling and quantitatively assessing the potential impacts of the connected vehicle (CV) on mobility, safety, and the environment. To assess the benefits of CVs, a modeling framework is developed based on traffic microsimulation for a real network located in the city of Toronto, Canada, to mimic communication between enabled vehicles. In this study, we examine the effects of providing real-time routing guidance and advisory warning messages to CVs. In addition, to take into account the rerouting in nonconnected vehicles (non-CVs) in response to varying sources of information such as apps, global positioning systems (GPS), variable message signs (VMS), or simply seeing the traffic back up, the impact of fraction of non-CV vehicles was also considered and evaluated. Therefore, vehicles in this model are divided into; uninformed/unfamiliar not connected (non-CV), informed/familiar but not connected (non-CV) that get updates infrequently every 5 minutes or so (non-CV), and connected vehicles that receive information more frequently (CV). The results demonstrate the potential of connected vehicles to improve mobility, enhance safety, and reduce greenhouse gas emissions (GHGs) at the network-wide level. The results also show quantitatively how the market penetration of connected vehicles proportionally affects the performance of the traffic network. While the presented results are pertinent to the specifics of the road network modeled and cannot be generalized, the quantitative figures provide researchers and practitioners with ideas of what to expect from vehicle connectivity concerning mobility, safety, and environmental improvements.