Rationalizing Reliable Imputation Durations of Genetically Designed Time Delay Neural Network and Locally Weighted Regression Models

Abstract

Estimating missing values is known as data imputation. Previous research has shown that genetic algorithms (GAs) designed locally weighted regression (LWR) and time delay neural network (TDNN) models can generate more accurate hourly volume imputations for a period of 12 successive hours than traditional methods used by highway agencies. It would be interesting and important to further refine the models for imputing larger missing intervals. Therefore, a large number of genetically designed LWR and TDNN models are developed in this study and used to impute up to a week-long missing interval (168 hours) for sample traffic counts obtained from various groups of roads in Alberta, Canada. It is found that road type and functional class have considerable influences on reliable imputations. The reliable imputation durations range from 4–5 days for traffic counts with most unstable patterns to over 10 days for those with most stable patterns. The study results clearly show that calibrated GA-designed models can provide reliable imputations for missing data with ‘block patterns’, and demonstrate their further potentials in traffic data programs.     

On the imputation of missing data for road traffic forecasting: New insights and novel techniques

Vehicle flow forecasting is of crucial importance for the management of road traffic in complex urban networks, as well as a useful input for route planning algorithms. In general traffic predictive models rely on data gathered by different types of sensors placed on roads, which occasionally produce faulty readings due to several causes, such as malfunctioning hardware or transmission errors. Filling in those gaps is relevant for constructing accurate forecasting models, a task which is engaged by diverse strategies, from a simple null value imputation to complex spatio-temporal context imputation models. This work elaborates on two machine learning approaches to update missing data with no gap length restrictions: a spatial context sensing model based on the information provided by surrounding sensors, and an automated clustering analysis tool that seeks optimal pattern clusters in order to impute values. Their performance is assessed and compared to other common techniques and different missing data generation models over real data captured from the city of Madrid (Spain). The newly presented methods are found to be fairly superior when portions of missing data are large or very abundant, as occurs in most practical cases.     

Hybrid artificial neural network and locally weighted regression models for lane-based short-term urban traffic flow forecasting

ABSTRACT

In recent years, there has been considerable research interest in short-term traffic flow forecasting. However, forecasting models offering a high accuracy at a fine temporal resolution (e.g. 1 or 5?min) and lane level are still rare. In this study, a combination of genetic algorithm, neural network and locally weighted regression is used to achieve optimal prediction under various input and traffic settings. The genetically optimized artificial neural network (GA-ANN) and locally weighted regression (GA-LWR) models are developed and tested, with the former forecasting traffic flow every 5-min within a 30-min period and the latter for forecasting traffic flow of a particular 5-min period of each for four lanes of an urban arterial road in Beijing, China. In particular, for morning peak and off-peak traffic flow prediction, the GA-ANN 5-min traffic flow model results in average errors of 3–5% and most 95th percentile errors of 7–14% for each of the four lanes; for the peak and off-peak time traffic flow predictions, the GA-LWR 5-min traffic flow model results in average errors of 2–4% and most 95th percentile errors are lower than 10% for each of the four lanes. When compared to previous models that usually offer average errors greater than 6–15%, such empirical findings should be of interest to and instrumental for transportation authorities to incorporate in their city- or state-wide Advanced Traveller Information Systems (ATIS).     

Short-term traffic flow rate forecasting based on identifying similar traffic patterns

The ability to timely and accurately forecast the evolution of traffic is very important in traffic management and control applications. This paper proposes a non-parametric and data-driven methodology for short-term traffic forecasting based on identifying similar traffic patterns using an enhanced K-nearest neighbor (K-NN) algorithm. Weighted Euclidean distance, which gives more weight to recent measurements, is used as a similarity measure for K-NN. Moreover, winsorization of the neighbors is implemented to dampen the effects of dominant candidates, and rank exponent is used to aggregate the candidate values. Robustness of the proposed method is demonstrated by implementing it on large datasets collected from different regions and by comparing it with advanced time series models, such as SARIMA and adaptive Kalman Filter models proposed by others. It is demonstrated that the proposed method reduces the mean absolute percent error by more than 25%. In addition, the effectiveness of the proposed enhanced K-NN algorithm is evaluated for multiple forecast steps and also its performance is tested under data with missing values. This research provides strong evidence suggesting that the proposed non-parametric and data-driven approach for short-term traffic forecasting provides promising results. Given the simplicity, accuracy, and robustness of the proposed approach, it can be easily incorporated with real-time traffic control for proactive freeway traffic management.     

Estimating historical hourly traffic volumes via machine learning and vehicle probe data: A Maryland case study

This paper focuses on the problem of estimating historical traffic volumes between sparsely-located traffic sensors, which transportation agencies need to accurately compute statewide performance measures. To this end, the paper examines applications of vehicle probe data, automatic traffic recorder counts, and neural network models to estimate hourly volumes in the Maryland highway network, and proposes a novel approach that combines neural networks with an existing profiling method. On average, the proposed approach yields 24% more accurate estimates than volume profiles, which are currently used by transportation agencies across the US to compute statewide performance measures. The paper also quantifies the value of using vehicle probe data in estimating hourly traffic volumes, which provides important managerial insights to transportation agencies interested in acquiring this type of data. For example, results show that volumes can be estimated with a mean absolute percent error of about 21% at locations where average number of observed probes is between 30 and 47 vehicles/h, which provides a useful guideline for assessing the value of probe vehicle data from different vendors.     

Alternatives in assigning short‐term counts to seasonal adjustment factor groupings

The precise estimation of the annual average daily traffic (AADT) is a task of significant interest for many transportation authorities and Departments of Transportation. In this study, three methods are developed to improve the assignment of short‐term counts to seasonal adjustment factor (SAF) groupings: the traditional functional classification, a discriminant analysis (DA), and a new statistical approach based on a weighted coefficient of variation (WCV). The data analyzed within this study are generated from all available continuous counters within the State of Ohio between 2002 and 2006. The analysis is conducted using SAFs that are separately calculated for the total volume and the directional specific volumes of a site. The results show that the directionally based assignment errors are statistically lower at a 95% confidence interval when compared with those generated by the total volume analysis. It is also found that the hourly time‐of‐day factors are more important in the assignment process than the average daily traffic. The directionally based WCV produces a decline in the average mean absolute percentage error (MAPE) over the roadway functional classification by 58% and in the standard deviation of the absolute error (SDAE) by 70%. On the contrary, the directionally based DA lowers the MAPE and the SDAE by 35% and 60%, respectively. Copyright © 2012 John Wiley & Sons, Ltd.     

Estimating vehicle miles traveled combined with ITS data

‘Vehicle miles traveled’ (VMT) is an important performance measure for highway systems. Currently, VMT [or ‘annual average daily traffic’ (AADT)] is estimated from a combination of permanent counting stations and short-term counts done at specified locations as part of the Highway Performance Monitoring System (HPMS) mandated by the US Federal Highway Administration. However, on some roadway sections, Intelligent Transportation Systems (ITS) such as detectors and cameras also produce traffic data. The question addressed in this paper is whether and under what conditions ITS systems data could be used instead of HPMS short-term counts (called ‘coverage counts’)? This paper develops a methodology for determining a threshold number of missing daily traffic counts, or alternatively, the number of valid ITS data observations needed, in order to confidently replace the HPMS coverage counts with ITS data.

Because ITS counts, coverage counts, and actual ground counts (e.g. continuous counts) cannot be found coexisting on a roadway section, it is hard to compare them directly. In this paper, the Monte Carlo simulation method is employed to generate synthetic ITS counts and coverage counts from a set of relatively complete traffic counts collected at a continuous count station. Comparisons are made between simulated ITS counts, coverage counts, and actual ground counts. The simulation results indicate that when there are<330 daily traffic counts missing in a set of ITS counts in a year, that is, when there are at least 35 days of valid data, ITS counts can be used to derive a better AADT than using coverage counts. This result is applied to calculate the VMT for the Hampton Roads region in Virginia. The comparison between the VMTs derived with using and not using the threshold number indicates that these two VMTs are significantly different.     

Comparison of parametric and nonparametric models for traffic flow forecasting

Single point short-term traffic flow forecasting will play a key role in supporting demand forecasts needed by operational network models. Seasonal autoregressive integrated moving average (ARIMA), a classic parametric modeling approach to time series, and nonparametric regression models have been proposed as well suited for application to single point short-term traffic flow forecasting. Past research has shown seasonal ARIMA models to deliver results that are statistically superior to basic implementations of nonparametric regression. However, the advantages associated with a data-driven nonparametric forecasting approach motivate further investigation of refined nonparametric forecasting methods. Following this motivation, this research effort seeks to examine the theoretical foundation of nonparametric regression and to answer the question of whether nonparametric regression based on heuristically improved forecast generation methods approach the single interval traffic flow prediction performance of seasonal ARIMA models.     

Are vehicle trajectories simulated by dynamic traffic models relevant for estimating fuel consumption?

This paper questions the relevance of microscopic traffic models for estimating the impact of traffic strategies on fuel consumption. Urban driving cycles from the ARTEMIS database are simplified into piecewise linear speed profiles to mimic the classical outputs of microscopic traffic flow models. Fuel consumption is estimated for real and simplified trajectories and links between kinematics and the fuel consumption errors are investigated. Simplifying trajectories causes fuel consumption underestimation, from −1.2 to −5.2% on average according to the level of simplification; errors can approach −20% for some cycles. A focus on kinematic phases indicates that the maximum speed reached and the time decelerating are the main influences on fuel consumption. Finally, in the case where maximum speeds are estimated correctly, it is shown that errors committed at each kinematic phase when acceleration distributions are approximated by their mean values, converge towards small errors over complete cycles. A method is developed to quantify and reduce these errors.     

The changing accuracy of traffic forecasts

Researchers have improved travel demand forecasting methods in recent decades but invested relatively little to understand their accuracy. A major barrier has been the lack of necessary data. We compiled the largest known database of traffic forecast accuracy, composed of forecast traffic, post-opening counts and project attributes for 1291 road projects in the United States and Europe. We compared measured versus forecast traffic and identified the factors associated with accuracy. We found measured traffic is on average 6% lower than forecast volumes, with a mean absolute deviation of 17% from the forecast. Higher volume roads, higher functional classes, shorter time spans, and the use of travel models all improved accuracy. Unemployment rates also affected accuracy—traffic would be 1% greater than forecast on average, rather than 6% lower, if we adjust for higher unemployment during the post-recession years (2008 to 2014). Forecast accuracy was not consistent over time: more recent forecasts were more accurate, and the mean deviation changed direction. Traffic on projects that opened from the 1980s through early 2000s was higher on average than forecast, while traffic on more recent projects was lower on average than forecast. This research provides insight into the degree of confidence that planners and policy makers can expect from traffic forecasts and suggests that we should view forecasts as a range of possible outcomes rather than a single expected outcome.

     

An efficient realization of deep learning for traffic data imputation

Traffic data provide the basis for both research and applications in transportation control, management, and evaluation, but real-world traffic data collected from loop detectors or other sensors often contain corrupted or missing data points which need to be imputed for traffic analysis. For this end, here we propose a deep learning model named denoising stacked autoencoders for traffic data imputation. We tested and evaluated the model performance with consideration of both temporal and spatial factors. Through these experiments and evaluation results, we developed an algorithm for efficient realization of deep learning for traffic data imputation by training the model hierarchically using the full set of data from all vehicle detector stations. Using data provided by Caltrans PeMS, we have shown that the mean absolute error of the proposed realization is under 10 veh/5-min, a better performance compared with other popular models: the history model, ARIMA model and BP neural network model. We further investigated why the deep leaning model works well for traffic data imputation by visualizing the features extracted by the first hidden layer. Clearly, this work has demonstrated the effectiveness as well as efficiency of deep learning in the field of traffic data imputation and analysis.     

Validation of an extended discrete first-order model with variable speed limits

This paper validates the prediction model embedded in a model predictive controller (MPC) of variable speed limits (VSLs). The MPC controller was designed based on an extended discrete first-order model with a triangular fundamental diagram. In our previous work, the extended discrete first-order model was designed to reproduce the capacity drop and the propagation of jam waves, and it was validated with reasonable accuracy without the presence of VSLs. As VSLs influence traffic dynamics, the dynamics including VSLs needs to be validated, before it can be applied as a prediction model in MPC. For conceptual illustrations, we use two synthetic examples to show how the model reproduces the key mechanisms of VSLs that are applied by existing VSL control approaches. Furthermore, the model is calibrated by use of real traffic data from Dutch freeway A12, where the field test of a speed limit control algorithm (SPECIALIST) was conducted. In the calibration, the original model is extended by using a quadrangular fundamental diagram which keeps the linear feature of the model and represents traffic states at the under-critical branch more accurately. The resulting model is validated using various traffic data sets. The accuracy of the model is compared with a second-order traffic flow model. The performance of two models is comparable: both models reproduce accurate results matching with real data. Flow errors of the calibration and validation are around 10%. The extended discrete first-order model-based MPC controller has been demonstrated to resolve freeway jam waves efficiently by synthetic cases. It has a higher computation speed comparing to the second-order model-based MPC.     

Empirical analysis of short period traffic counts and their efficiency: the case of Indian traffic

Short period traffic counts (SPTCs) are conducted routinely to estimate the annual average daily traffic (AADT) at a particular site. This paper uses Indian traffic volume data to methodically and extensively study the effect of four aspects related to the design of SPTCs. These four aspects are: (i) for how long, (ii) on which days should SPTCs be carried out, (iii) how many times, and (iv) on which months should SPTCs be carried out? The analyses indicate that the best durations for conducting SPTCs are 3 days (starting with a Thursday) and 7 days, for total traffic and truck traffic, respectively. Further, these counts should be repeated twice a year keeping a separation of two months between the counts to obtain good estimates of AADT at minimal cost. An additional outcome of this study has been the determination of seasonal factor values for roads in developing economies, like India.     

Updating predictive accident models of modern rural single carriageway A-roads

Abstract

Reliable predictive accident models (PAMs) are essential to design and maintain safe road networks, and yet the models most commonly used in the UK were derived using data collected 20 to 30 years ago. Given that the national personal injury accident total fell by some 30% in the last 25 years, while road traffic increased by over 60%, significant errors in scheme appraisal and evaluation based on the models currently in use seem inevitable. In this paper, the temporal transferability of PAMs for modern rural single carriageway A-roads is investigated, and their predictive performance is evaluated against a recent data set. Despite the age of these models, the PAMs for predicting the total accidents provide a remarkably good fit to recent data and these are more accurate than models where accidents are disaggregated by type. The performance of the models can be improved by calibrating them against recent data.     

Short‐term travel speed prediction models in car navigation systems

The objective of this study is the development of the short‐term prediction models to predict average spot speeds of the subject location in the short‐term periods of 5, 10 and 15 minutes respectively. In this study, field data were used to see the comparison of the predictability of Regression Analysis, ARIMA, Kalman Filtering and Neural Network models. These field data were collected from image processing detectors at the urban expressway for 17 hours including both peak and non‐peak hours. Most of the results were reliable, but the results of models using Kalman Filtering and Neural Networks are more accurate and realistic than those of the others.     

Biased standard error estimations in transport model calibration due to heteroscedasticity arising from the variability of linear data projection

Reliable transport models calibrated from accurate traffic data are crucial for predicating transportation system performance and ensuring better traffic planning. However, due to the impracticability of collecting data from an entire population, methods of data inference such as the linear data projection are commonly adopted. A recent study has shown that systematic bias may be embedded in the parameters calibrated due to linearly projected data that do not account for scaling factor variability. Adjustment factors for reducing such biases in the calibrated parameters have been proposed for a generalized multivariate polynomial model. However, the effects of linear data projection on the dispersion of and confidence in the adjusted parameters have not been explored. Without appropriate statistics examining the statistical significance of the adjusted model, their validity in applications remains unknown and dubious. This study reveals that heteroscedasticity is inherently introduced by data projection with a varying scaling factor. Parameter standard errors that are estimated by linearly projected data without any appropriate treatments for non-homoscedasticity are definitely biased, and possibly above or below their true values. To ensure valid statistical tests of significance and prevent exposure to uninformed and unnecessary risk in applications, a generic analytical distribution-free (ADF) method and an equivalent scaling factor (ESF) method are proposed to adjust the parameter standard errors for a generalized multivariate polynomial model, based on the reported residual sum of squares. The ESF method transforms a transport model into a linear function of the scaling factor before calibration, which provides an alternative solution path for achieving unbiased parameter estimations. Simulation results demonstrate the robustness of the ESF method compared with the ADF method at high model nonlinearity. Case studies are conducted to illustrate the applicability of the ESF method for the parameter standard error estimations of six Macroscopic Bureau of Public Road functions, which are calibrated using real-world global positioning system data obtained from Hong Kong.     

Memory properties and fractional integration in transportation time-series

In transportation analyses, autoregressive integrated moving average (ARIMA) and generalized autoregressive conditional heteroskedasticity (GARCH) models have been widely used mainly because of their well established theoretical foundation and ease of application. However, they lack the ability to capture long memory properties and do not jointly treat the mean and variance (variability) of a time-series. We employ fractionally integrated dual memory models and compare results to classical time-series models in a traffic engineering context. Results indicate that dual memory models offer better representation of the original time-series than classical models; further, forcing the differentiation parameter of ARIMA model to equal 1 leads to over-inflated moving average terms and, consequently, to questionable models with artificial correlation structures.     

Random parameter models for estimating statewide daily bicycle counts using crowdsourced data

Persistent lack of non-motorized traffic counts can affect the evidence-based decisions of transportation planning and safety-concerned agencies in making reliable investments in bikeway and other non-motorized facilities. Researchers have used various approaches to estimate bicycles counts, such as scaling, direct-demand modeling, time series, and others. In recent years, an increasing number of studies have tried to use crowdsourced data for estimating the bicycle counts. Crowdsourced data only represents a small percentage of cyclists. This percentage, on the other hand, can change based on the location, facility type, meteorological, and other factors. Moreover, the autocorrelation observed in bicycle counts may be different from the autocorrelation structure observed among crowdsourced platform users, such as Strava. Strava users are more consistent; hence, the time series count data may be stationary, while bicycle demand may vary based on seasonal factors. In addition to seasonal variation, several time-invariant contributing factors (e.g., facility type, roadway characteristics, household income) affect bicycle demand, which needs to be accounted for when developing direct demand models. In this paper, we use a mixed-effects model with autocorrelated errors to predict daily bicycle counts from crowdsourced data across the state of Texas. Additionally, we supplement crowdsourced data with other spatial and temporal factors such as roadway facility, household income, population demographics, population density and weather conditions to predict bicycle counts. The results show that using a robust methodology, we can predict bicycle demand with a 29% margin of error, which is significantly lower than merely scaling the crowdsourced data (41%).     

A simple nonparametric car-following model driven by field data

Car-following models are always of great interest of traffic engineers and researchers. In the age of mass data, this paper proposes a nonparametric car-following model driven by field data. Different from most of the existing car-following models, neither driver’s behaviour parameters nor fundamental diagrams are assumed in the data-driven model. The model is proposed based on the simple k-nearest neighbour, which outputs the average of the most similar cases, i.e., the most likely driving behaviour under the current circumstance. The inputs and outputs are selected, and the determination of the only parameter k is introduced. Three simulation scenarios are conducted to test the model. The first scenario is to simulate platoons following real leaders, where traffic waves with constant speed and the detailed trajectories are observed to be consistent with the empirical data. Driver’s rubbernecking behaviour and driving errors are simulated in the second and third scenarios, respectively. The time–space diagrams of the simulated trajectories are presented and explicitly analysed. It is demonstrated that the model is able to well replicate periodic traffic oscillations from the precursor stage to the decay stage. Without making any assumption, the fundamental diagrams for the simulated scenario coincide with the empirical fundamental diagrams. These all validate that the model can well reproduce the traffic characteristics contained by the field data. The nonparametric car-following model exhibits traffic dynamics in a simple and parsimonious manner.     

Increasing the accuracy of loop detector counts using adaptive neural fuzzy inference system and genetic programming

Loop detectors are devices that are most commonly used for obtaining data at intersections. Multiple detectors are usually required to monitor a location, and this reduces the accuracy of detectors for collecting traffic volumes. The purpose of this paper is to increase the accuracy of loop detector counts using Adaptive Neural Fuzzy Inference System (ANFIS) and Genetic Programming (GP) based on detector volume and occupancy. These methods do not need microscopic analysis and are easy to employ. Four approaches for one intersection are used in a case study. Results show that the models can improve intersection detector counts significantly. Results also show that ANFIS produces more accurate counts compared to regression and GP.