On the imputation of missing data for road traffic forecasting: New insights and novel techniques

Vehicle flow forecasting is of crucial importance for the management of road traffic in complex urban networks, as well as a useful input for route planning algorithms. In general traffic predictive models rely on data gathered by different types of sensors placed on roads, which occasionally produce faulty readings due to several causes, such as malfunctioning hardware or transmission errors. Filling in those gaps is relevant for constructing accurate forecasting models, a task which is engaged by diverse strategies, from a simple null value imputation to complex spatio-temporal context imputation models. This work elaborates on two machine learning approaches to update missing data with no gap length restrictions: a spatial context sensing model based on the information provided by surrounding sensors, and an automated clustering analysis tool that seeks optimal pattern clusters in order to impute values. Their performance is assessed and compared to other common techniques and different missing data generation models over real data captured from the city of Madrid (Spain). The newly presented methods are found to be fairly superior when portions of missing data are large or very abundant, as occurs in most practical cases.     

A hybrid approach to integrate fuzzy C-means based imputation method with genetic algorithm for missing traffic volume data estimation

Although various innovative traffic sensing technologies have been widely employed, incomplete sensor data is one of the most major problems to significantly degrade traffic data quality and integrity. In this study, a hybrid approach integrating the Fuzzy C-Means (FCM)-based imputation method with the Genetic Algorithm (GA) is develop for missing traffic volume data estimation based on inductance loop detector outputs. By utilizing the weekly similarity among data, the conventional vector-based data structure is firstly transformed into the matrix-based data pattern. Then, the GA is applied to optimize the membership functions and centroids in the FCM model. The experimental tests are conducted to verify the effectiveness of the proposed approach. The traffic volume data collected at different temporal scales were used as the testing dataset, and three different indicators, including root mean square error, correlation coefficient, and relative accuracy, are utilized to quantify the imputation performance compared with some conventional methods (Historical method, Double Exponential Smoothing, and Autoregressive Integrated Moving Average model). The results show the proposed approach outperforms the conventional methods under prevailing traffic conditions.     

Using high-resolution event-based data for traffic modeling and control: An overview

Research on using high-resolution event-based data for traffic modeling and control is still at early stage. In this paper, we provide a comprehensive overview on what has been achieved and also think ahead on what can be achieved in the future. It is our opinion that using high-resolution event data, instead of conventional aggregate data, could bring significant improvements to current research and practices in traffic engineering. Event data records the times when a vehicle arrives at and departs from a vehicle detector. From that, individual vehicle’s on-detector-time and time gap between two consecutive vehicles can be derived. Such detailed information is of great importance for traffic modeling and control. As reviewed in this paper, current research has demonstrated that event data are extremely helpful in the fields of detector error diagnosis, vehicle classification, freeway travel time estimation, arterial performance measure, signal control optimization, traffic safety, traffic flow theory, and environmental studies. In addition, the cost of event data collection is low compared to other data collection techniques since event data can be directly collected from existing controller cabinet without any changes on the infrastructure, and can be continuously collected in 24/7 mode. This brings many research opportunities as suggested in the paper.     

A hybrid deep learning based traffic flow prediction method and its understanding

Deep neural networks (DNNs) have recently demonstrated the capability to predict traffic flow with big data. While existing DNN models can provide better performance than shallow models, it is still an open issue of making full use of spatial-temporal characteristics of the traffic flow to improve their performance. In addition, our understanding of them on traffic data remains limited. This paper proposes a DNN based traffic flow prediction model (DNN-BTF) to improve the prediction accuracy. The DNN-BTF model makes full use of weekly/daily periodicity and spatial-temporal characteristics of traffic flow. Inspired by recent work in machine learning, an attention based model was introduced that automatically learns to determine the importance of past traffic flow. The convolutional neural network was also used to mine the spatial features and the recurrent neural network to mine the temporal features of traffic flow. We also showed through visualization how DNN-BTF model understands traffic flow data and presents a challenge to conventional thinking about neural networks in the transportation field that neural networks is purely a “black-box” model. Data from open-access database PeMS was used to validate the proposed DNN-BTF model on a long-term horizon prediction task. Experimental results demonstrated that our method outperforms the state-of-the-art approaches.     

Overview of missing physical commodity trade data and its imputation using data augmentation

The physical aspects of commodity trade are becoming increasingly important on a global scale for transportation planning, demand management for transportation facilities and services, energy use, and environmental concerns. Such aspects (for example, weight and volume) of commodities are vital for logistics industry to allow for medium-to-long term planning at the strategic level and identify commodity flow trends. However, incomplete physical commodity trade databases impede proper analysis of trade flow between various countries. The missing physical values could be due to many reasons such as, (1) non-compliance of reporter countries with the prescribed regulations by World Customs Organization (WCO) (2) confidentiality issues, (3) delays in processing of data, or (4) erroneous reporting. The traditional missing data imputation methods, such as the substitution by mean, substitution by linear interpolation/extrapolation using adjacent points, the substitution by regression, and the substitution by stochastic regression, have been proposed in the context of estimating physical aspects of commodity trade data. However, a major demerit of these single imputation methods is their failure to incorporate uncertainty associated with missing data. The use of computationally complex stochastic methods to improve the accuracy of imputed data has recently become possible with the advancement of computer technology. Therefore, this study proposes a sophisticated data augmentation algorithm in order to impute missing physical commodity trade data. The key advantage of the proposed approach lies in the fact that instead of using a point estimate as the imputed value, it simulates a distribution of missing data through multiple imputations to reflect uncertainty and to maintain variability in the data. This approach also provides the flexibility to include fundamental distributional property of the variables, such as physical quantity, monetary value, price elasticity of demand, price variation, and product differentiation, and their correlations to generate reasonable average estimates of statistical inferences. An overview and limitations of most commonly used data imputation approaches is presented, followed by the theoretical basis and imputation procedure of the proposed approach. Lastly, a case study is presented to demonstrate the merits of the proposed approach in comparison to traditional imputation methods.     

Efficient missing data imputing for traffic flow by considering temporal and spatial dependence

The missing data problem remains as a difficulty in a diverse variety of transportation applications, e.g. traffic flow prediction and traffic pattern recognition. To solve this problem, numerous algorithms had been proposed in the last decade to impute the missed data. However, few existing studies had fully used the traffic flow information of neighboring detecting points to improve imputing performance. In this paper, probabilistic principle component analysis (PPCA) based imputing method, which had been proven to be one of the most effective imputing methods without using temporal or spatial dependence, is extended to utilize the information of multiple points. We systematically examine the potential benefits of multi-point data fusion and study the possible influence of measurement time lags. Tests indicate that the hidden temporal–spatial dependence is nonlinear and could be better retrieved by kernel probabilistic principle component analysis (KPPCA) based method rather than PPCA method. Comparison proves that imputing errors can be notably reduced, if temporal–spatial dependence has been appropriately considered.     

Estimating historical hourly traffic volumes via machine learning and vehicle probe data: A Maryland case study

This paper focuses on the problem of estimating historical traffic volumes between sparsely-located traffic sensors, which transportation agencies need to accurately compute statewide performance measures. To this end, the paper examines applications of vehicle probe data, automatic traffic recorder counts, and neural network models to estimate hourly volumes in the Maryland highway network, and proposes a novel approach that combines neural networks with an existing profiling method. On average, the proposed approach yields 24% more accurate estimates than volume profiles, which are currently used by transportation agencies across the US to compute statewide performance measures. The paper also quantifies the value of using vehicle probe data in estimating hourly traffic volumes, which provides important managerial insights to transportation agencies interested in acquiring this type of data. For example, results show that volumes can be estimated with a mean absolute percent error of about 21% at locations where average number of observed probes is between 30 and 47 vehicles/h, which provides a useful guideline for assessing the value of probe vehicle data from different vendors.     

Comparison of advanced imputation algorithms for detection of transportation mode and activity episode using GPS data

Global Positioning System (GPS) technologies have been increasingly considered as an alternative to traditional travel survey methods to collect activity-travel data. Algorithms applied to extract activity-travel patterns vary from informal ad-hoc decision rules to advanced machine learning methods and have different accuracy. This paper systematically compares the relative performance of different algorithms for the detection of transportation modes and activity episodes. In particular, naive Bayesian, Bayesian network, logistic regression, multilayer perceptron, support vector machine, decision table, and C4.5 algorithms are selected and compared for the same data according to their overall error rates and hit ratios. Results show that the Bayesian network has a better performance than the other algorithms in terms of the percentage correctly identified instances and Kappa values for both the training data and test data, in the sense that the Bayesian network is relatively efficient and generalizable in the context of GPS data imputation.     

The use of a hazard-based duration model for imputation of missing personal income data

This paper suggests using a proportional hazard model to predict personal income, for the purpose of imputing missing income data in household travel surveys. The model has a hazard function that comprises two multiplicative components: (1) a non-parametric baseline hazard function that is dependent only on the income level and (2) a function that is dependent only on the other personal attributes of the survey respondents (excluding income). To estimate and validate the model, data is drawn from a travel characteristics survey conducted in Hong Kong in year 2001. The model is found to have a much higher accuracy when compared with a conventional ordered probit model based on the assumption that the logarithm of income is normally distributed.

Estimating traffic volumes for signalized intersections using connected vehicle data

Recently connected vehicle (CV) technology has received significant attention thanks to active pilot deployments supported by the US Department of Transportation (USDOT). At signalized intersections, CVs may serve as mobile sensors, providing opportunities of reducing dependencies on conventional vehicle detectors for signal operation. However, most of the existing studies mainly focus on scenarios that penetration rates of CVs reach certain level, e.g., 25%, which may not be feasible in the near future. How to utilize data from a small number of CVs to improve traffic signal operation remains an open question. In this work, we develop an approach to estimate traffic volume, a key input to many signal optimization algorithms, using GPS trajectory data from CV or navigation devices under low market penetration rates. To estimate traffic volumes, we model vehicle arrivals at signalized intersections as a time-dependent Poisson process, which can account for signal coordination. The estimation problem is formulated as a maximum likelihood problem given multiple observed trajectories from CVs approaching to the intersection. An expectation maximization (EM) procedure is derived to solve the estimation problem. Two case studies were conducted to validate our estimation algorithm. One uses the CV data from the Safety Pilot Model Deployment (SPMD) project, in which around 2800 CVs were deployed in the City of Ann Arbor, MI. The other uses vehicle trajectory data from users of a commercial navigation service in China. Mean absolute percentage error (MAPE) of the estimation is found to be 9–12%, based on benchmark data manually collected and data from loop detectors. Considering the existing scale of CV deployments, the proposed approach could be of significant help to traffic management agencies for evaluating and operating traffic signals, paving the way of using CVs for detector-free signal operation in the future.     

Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic data

Accurately modeling traffic speeds is a fundamental part of efficient intelligent transportation systems. Nowadays, with the widespread deployment of GPS-enabled devices, it has become possible to crowdsource the collection of speed information to road users (e.g. through mobile applications or dedicated in-vehicle devices). Despite its rather wide spatial coverage, crowdsourced speed data also brings very important challenges, such as the highly variable measurement noise in the data due to a variety of driving behaviors and sample sizes. When not properly accounted for, this noise can severely compromise any application that relies on accurate traffic data. In this article, we propose the use of heteroscedastic Gaussian processes (HGP) to model the time-varying uncertainty in large-scale crowdsourced traffic data. Furthermore, we develop a HGP conditioned on sample size and traffic regime (SSRC-HGP), which makes use of sample size information (probe vehicles per minute) as well as previous observed speeds, in order to more accurately model the uncertainty in observed speeds. Using 6 months of crowdsourced traffic data from Copenhagen, we empirically show that the proposed heteroscedastic models produce significantly better predictive distributions when compared to current state-of-the-art methods for both speed imputation and short-term forecasting tasks.     

Human-like autonomous car-following model with deep reinforcement learning

This study proposes a framework for human-like autonomous car-following planning based on deep reinforcement learning (deep RL). Historical driving data are fed into a simulation environment where an RL agent learns from trial and error interactions based on a reward function that signals how much the agent deviates from the empirical data. Through these interactions, an optimal policy, or car-following model that maps in a human-like way from speed, relative speed between a lead and following vehicle, and inter-vehicle spacing to acceleration of a following vehicle is finally obtained. The model can be continuously updated when more data are fed in. Two thousand car-following periods extracted from the 2015 Shanghai Naturalistic Driving Study were used to train the model and compare its performance with that of traditional and recent data-driven car-following models. As shown by this study’s results, a deep deterministic policy gradient car-following model that uses disparity between simulated and observed speed as the reward function and considers a reaction delay of 1 s, denoted as DDPGvRT, can reproduce human-like car-following behavior with higher accuracy than traditional and recent data-driven car-following models. Specifically, the DDPGvRT model has a spacing validation error of 18% and speed validation error of 5%, which are less than those of other models, including the intelligent driver model, models based on locally weighted regression, and conventional neural network-based models. Moreover, the DDPGvRT demonstrates good capability of generalization to various driving situations and can adapt to different drivers by continuously learning. This study demonstrates that reinforcement learning methodology can offer insight into driver behavior and can contribute to the development of human-like autonomous driving algorithms and traffic-flow models.     

A novel passenger flow prediction model using deep learning methods

Currently, deep learning has been successfully applied in many fields and achieved amazing results. Meanwhile, big data has revolutionized the transportation industry over the past several years. These two hot topics have inspired us to reconsider the traditional issue of passenger flow prediction. As a special structure of deep neural network (DNN), an autoencoder can deeply and abstractly extract the nonlinear features embedded in the input without any labels. By exploiting its remarkable capabilities, a novel hourly passenger flow prediction model using deep learning methods is proposed in this paper. Temporal features including the day of a week, the hour of a day, and holidays, the scenario features including inbound and outbound, and tickets and cards, and the passenger flow features including the previous average passenger flow and real-time passenger flow, are defined as the input features. These features are combined and trained as different stacked autoencoders (SAE) in the first stage. Then, the pre-trained SAE are further used to initialize the supervised DNN with the real-time passenger flow as the label data in the second stage. The hybrid model (SAE-DNN) is applied and evaluated with a case study of passenger flow prediction for four bus rapid transit (BRT) stations of Xiamen in the third stage. The experimental results show that the proposed method has the capability to provide a more accurate and universal passenger flow prediction model for different BRT stations with different passenger flow profiles.     

Traffic volume forecasting based on radial basis function neural network with the consideration of traffic flows at the adjacent intersections

The forecasting of short-term traffic flow is one of the key issues in the field of dynamic traffic control and management. Because of the uncertainty and nonlinearity, short-term traffic flow forecasting could be a challenging task. Artificial Neural Network (ANN) could be a good solution to this issue as it is possible to obtain a higher forecasting accuracy within relatively short time through this tool. Traditional methods for traffic flow forecasting generally based on a separated single point. However, it is found that traffic flows from adjacent intersections show a similar trend. It indicates that the vehicle accumulation and dissipation influence the traffic volumes of the adjacent intersections. This paper presents a novel method, which considers the travel flows of the adjacent intersections when forecasting the one of the middle. Computational experiments show that the proposed model is both effective and practical.     

Long short-term memory neural network for traffic speed prediction using remote microwave sensor data

Neural networks have been extensively applied to short-term traffic prediction in the past years. This study proposes a novel architecture of neural networks, Long Short-Term Neural Network (LSTM NN), to capture nonlinear traffic dynamic in an effective manner. The LSTM NN can overcome the issue of back-propagated error decay through memory blocks, and thus exhibits the superior capability for time series prediction with long temporal dependency. In addition, the LSTM NN can automatically determine the optimal time lags. To validate the effectiveness of LSTM NN, travel speed data from traffic microwave detectors in Beijing are used for model training and testing. A comparison with different topologies of dynamic neural networks as well as other prevailing parametric and nonparametric algorithms suggests that LSTM NN can achieve the best prediction performance in terms of both accuracy and stability.     

交通事故预警系统构建

针对交通安全现状及国内外交通预警发展现状的分析,阐明建立交通事故预警系统的必要性。分析了基于人、车、路、环境四要素的道路交通事故的成因,根据交通事故预警系统设计原则和建立预警系统的目的,采用相关理论,选用合适的交通信息采集技术,建立了交通事故预警系统。该系统包括驾驶员预警子系统、车辆防撞预警子系统、车辆状况预警子系统、道路安全预警子系统和交通气象预警子系统。     

A dynamic Bayesian network model for real-time crash prediction using traffic speed conditions data

Traffic crashes occurring on freeways/expressways are considered to relate closely to previous traffic conditions, which are time-varying. Meanwhile, most studies use volume/occupancy/speed parameters to predict the likelihood of crashes, which are invalid for roads where the traffic conditions are estimated using speed data extracted from sampled floating cars or smart phones. Therefore, a dynamic Bayesian network (DBN) model of time sequence traffic data has been proposed to investigate the relationship between crash occurrence and dynamic speed condition data. Moreover, the traffic conditions near the crash site were identified as several state combinations according to the level of congestion and included in the DBN model. Based on 551 crashes and corresponding speed information collected on expressways in Shanghai, China, DBN models were built with time series speed condition data and different state combinations. A comparative analysis of the DBN model using flow detector data and a static Bayesian network model was also conducted. The results show that, with only speed condition data and nine traffic state combinations, the DBN model can achieve a crash prediction accuracy of 76.4% with a false alarm rate of 23.7%. In addition, the results of transferability testing imply that the DBN models are applicable to other similar expressways with 67.0% crash prediction accuracy.     

A phase-based smoothing method for accurate traffic speed estimation with floating car data

In this paper, a novel freeway traffic speed estimation method based on probe data is presented. In contrast to other traffic speed estimators, it only requires velocity data from probes and does not depend on any additional data inputs such as density or flow information. In the first step the method determines the three traffic phases free flow, synchronized flow, and Wide Moving Jam (WMJ) described by Kerner et al. in space and time. Subsequently, reported data is processed with respect to the prevailing traffic phase in order to estimate traffic velocities. This two-step approach allows incorporating empirical features of phase fronts into the estimation procedure. For instance, downstream fronts of WMJs always propagate upstream with approximately constant velocity, and downstream fronts of synchronized flow phases usually stick to bottlenecks. The second step assures the validity of measured velocities is limited to the extent of its assigned phase. Effectively, velocity information in space-time can be estimated more distinctively and the result is therefore more accurate even if the input data density is low.The accuracy of the proposed Phase-Based Smoothing Method (PSM) is evaluated using real floating car data collected during two traffic congestions on the German freeway A99 and compared to the performance of the Generalized Adaptive Smoothing Method (GASM) as well as a naive algorithm. The quantitative and qualitative results show that the PSM reconstructs the congestion pattern more accurately than the other two. A subsequent analysis of the computational efficiency and sensitivity demonstrates its practical suitability.     

An improved cellular automata model for heterogeneous work zone traffic

This paper aims to develop an improved cellular automata (ICA) model for simulating heterogeneous traffic in work zone. The proposed ICA model includes the forwarding rules to update longitudinal speeds and positions of work zone vehicles. The randomization probability parameter used by the ICA is formulated as a function of the activity length, the transition length and the volumes of different types of vehicles traveling across work zone. Compared to the existing cellular automata models, the ICA model possesses a novel and realistic lateral speed and position updating rule so that the simulation of vehicle’s lateral movement in work zone is close to the reality. The ICA model is calibrated and validated microscopically and macroscopically by using the real work zone data. Comparisons of field data and ICA for trajectories, speed and speed–flow relationship in work zone show very close agreement. Finally, the proposed ICA model is applied to estimate traffic delay occurred in work zone.