Asynchronous n-step Q-learning adaptive traffic signal control

Ensuring transportation systems are efficient is a priority for modern society. Intersection traffic signal control can be modeled as a sequential decision-making problem. To learn how to make the best decisions, we apply reinforcement learning techniques with function approximation to train an adaptive traffic signal controller. We use the asynchronous n-step Q-learning algorithm with a two hidden layer artificial neural network as our reinforcement learning agent. A dynamic, stochastic rush hour simulation is developed to test the agent’s performance. Compared against traditional loop detector actuated and linear Q-learning traffic signal control methods, our reinforcement learning model develops a superior control policy, reducing mean total delay by up 40% without compromising throughput. However, we find our proposed model slightly increases delay for left turning vehicles compared to the actuated controller, as a consequence of the reward function, highlighting the need for an appropriate reward function which truly develops the desired policy.     

The combined effect of rain and wind on air–water gas exchange: A feasibility study

A series of experiments were conducted at University of Delaware's Air–Sea Interaction Laboratory to examine the combined effects of rain and wind on air–water gas exchange. During this study, ASIL WRX I, a combination of 3 rain rates and 4 wind speeds were used, for a total of 12 different environmental conditions. The SF₆ evasion method was used to determine the bulk gas transfer velocities, and airside profiles of wind and CO₂ were used to estimate flux–profiles of momentum and carbon dioxide. In addition to measurements of fluxes with and without rain in a wind–wave boundary layer, measurements of wave properties were also obtained. Rain is shown to alter the wind profile in the flume, and dampen surface waves. Also, SF₆ evasion indicates that with the present experimental setup, for most of the experimental conditions, rain and wind combine linearly to influence air–water gas exchange. Flux–profile relationships for marine atmospheric boundary layers, which were performed to scale up to field measurements, were explored by a comparison between SF₆-derived bulk fluxes and airside CO₂ profile measurements.