An efficient realization of deep learning for traffic data imputation

Traffic data provide the basis for both research and applications in transportation control, management, and evaluation, but real-world traffic data collected from loop detectors or other sensors often contain corrupted or missing data points which need to be imputed for traffic analysis. For this end, here we propose a deep learning model named denoising stacked autoencoders for traffic data imputation. We tested and evaluated the model performance with consideration of both temporal and spatial factors. Through these experiments and evaluation results, we developed an algorithm for efficient realization of deep learning for traffic data imputation by training the model hierarchically using the full set of data from all vehicle detector stations. Using data provided by Caltrans PeMS, we have shown that the mean absolute error of the proposed realization is under 10 veh/5-min, a better performance compared with other popular models: the history model, ARIMA model and BP neural network model. We further investigated why the deep leaning model works well for traffic data imputation by visualizing the features extracted by the first hidden layer. Clearly, this work has demonstrated the effectiveness as well as efficiency of deep learning in the field of traffic data imputation and analysis.     

Estimation of missing traffic counts using factor,genetic, neural,and regression techniques

Analyses from some of the highway agencies show that up to 50% permanent traffic counts (PTCs) have missing values. It will be difficult to eliminate such a significant portion of data from traffic analysis. Literature review indicates that the limited research uses factor or autoregressive integrated moving average (ARIMA) models for predicting missing values. Factor-based models tend to be less accurate. ARIMA models only use the historical data. In this study, genetically designed neural network and regression models, factor models, and ARIMA models were developed. It was found that genetically designed regression models based on data from before and after the failure had the most accurate results. Average errors for refined models were lower than 1% and the 95th percentile errors were below 2% for counts with stable patterns. Even for counts with relatively unstable patterns, average errors were lower than 3% in most cases.     

On the imputation of missing data for road traffic forecasting: New insights and novel techniques

Vehicle flow forecasting is of crucial importance for the management of road traffic in complex urban networks, as well as a useful input for route planning algorithms. In general traffic predictive models rely on data gathered by different types of sensors placed on roads, which occasionally produce faulty readings due to several causes, such as malfunctioning hardware or transmission errors. Filling in those gaps is relevant for constructing accurate forecasting models, a task which is engaged by diverse strategies, from a simple null value imputation to complex spatio-temporal context imputation models. This work elaborates on two machine learning approaches to update missing data with no gap length restrictions: a spatial context sensing model based on the information provided by surrounding sensors, and an automated clustering analysis tool that seeks optimal pattern clusters in order to impute values. Their performance is assessed and compared to other common techniques and different missing data generation models over real data captured from the city of Madrid (Spain). The newly presented methods are found to be fairly superior when portions of missing data are large or very abundant, as occurs in most practical cases.     

Overview of missing physical commodity trade data and its imputation using data augmentation

The physical aspects of commodity trade are becoming increasingly important on a global scale for transportation planning, demand management for transportation facilities and services, energy use, and environmental concerns. Such aspects (for example, weight and volume) of commodities are vital for logistics industry to allow for medium-to-long term planning at the strategic level and identify commodity flow trends. However, incomplete physical commodity trade databases impede proper analysis of trade flow between various countries. The missing physical values could be due to many reasons such as, (1) non-compliance of reporter countries with the prescribed regulations by World Customs Organization (WCO) (2) confidentiality issues, (3) delays in processing of data, or (4) erroneous reporting. The traditional missing data imputation methods, such as the substitution by mean, substitution by linear interpolation/extrapolation using adjacent points, the substitution by regression, and the substitution by stochastic regression, have been proposed in the context of estimating physical aspects of commodity trade data. However, a major demerit of these single imputation methods is their failure to incorporate uncertainty associated with missing data. The use of computationally complex stochastic methods to improve the accuracy of imputed data has recently become possible with the advancement of computer technology. Therefore, this study proposes a sophisticated data augmentation algorithm in order to impute missing physical commodity trade data. The key advantage of the proposed approach lies in the fact that instead of using a point estimate as the imputed value, it simulates a distribution of missing data through multiple imputations to reflect uncertainty and to maintain variability in the data. This approach also provides the flexibility to include fundamental distributional property of the variables, such as physical quantity, monetary value, price elasticity of demand, price variation, and product differentiation, and their correlations to generate reasonable average estimates of statistical inferences. An overview and limitations of most commonly used data imputation approaches is presented, followed by the theoretical basis and imputation procedure of the proposed approach. Lastly, a case study is presented to demonstrate the merits of the proposed approach in comparison to traditional imputation methods.     

Modelling user satisfaction in public transport systems considering missing information

Collecting data to obtain insights into customer satisfaction with public transport services is very time-consuming and costly. Many factors such as service frequency, reliability and comfort during the trip have been found important drivers of customer satisfaction. Consequently, customer satisfaction surveys are quite lengthy, resulting in many interviews not being completed within the aboard time of the passengers/respondents. This paper questions as to whether it is possible to reduce the amount of information collected without a compromise on insights. To address this research question, we conduct a comparative analysis of different Ordered Probit models: one with a full list of attributes versus one with partial set of attributes. For the latter, missing information was imputed using three different methods that are based on modes, single imputations using predictive models and multiple imputation. Estimation results show that the partial model using the multiple imputation method behaves in a similar way to the model that is based on the full survey. This finding opens an opportunity to reduce interview time which is critical for most customer satisfaction surveys.

     

Decision-making on Large Infrastructure Projects: The Role of the Dutch Parliament

Abstract

Validating microscopic traffic simulation models incorporates several challenges because of the inadequacy and rareness of validation data, and the complexity of the car following and lane-changing processes. In addition, validation data were usually measured in aggregate form at the link level and not at the level of the individual vehicle. The majority of model validation attempts in the literature use average link measurements of traffic characteristics. However, validation techniques based on averages of traffic variables have several limitations including possible inconsistency between the field observed and simulation-estimated variables, and as such the resulting spatial–temporal traffic stream patterns.

Due to these inconsistencies, this paper introduces a novel approach to the validation of microscopic traffic simulation models. A three-stage procedure for validating microscopic simulation models is presented. The paper describes the field measurements, experimental setup, and the simulation-based analysis of the three stages. The purpose of the first stage is to validate a benchmark simulator (NETSIM) using limited field data. The second stage examines the spatial–temporal traffic patterns extracted from the benchmark simulator versus those extracted from the simulation model to be validated (I-SIM-S). Different traffic patterns were examined accounting for various factors, such as traffic flow, link speeds, and signal timing. The third stage compares the aggregate traffic measures extracted from the subject simulator against those extracted from the benchmark simulator.     

Estimating vehicle miles traveled combined with ITS data

‘Vehicle miles traveled’ (VMT) is an important performance measure for highway systems. Currently, VMT [or ‘annual average daily traffic’ (AADT)] is estimated from a combination of permanent counting stations and short-term counts done at specified locations as part of the Highway Performance Monitoring System (HPMS) mandated by the US Federal Highway Administration. However, on some roadway sections, Intelligent Transportation Systems (ITS) such as detectors and cameras also produce traffic data. The question addressed in this paper is whether and under what conditions ITS systems data could be used instead of HPMS short-term counts (called ‘coverage counts’)? This paper develops a methodology for determining a threshold number of missing daily traffic counts, or alternatively, the number of valid ITS data observations needed, in order to confidently replace the HPMS coverage counts with ITS data.

Because ITS counts, coverage counts, and actual ground counts (e.g. continuous counts) cannot be found coexisting on a roadway section, it is hard to compare them directly. In this paper, the Monte Carlo simulation method is employed to generate synthetic ITS counts and coverage counts from a set of relatively complete traffic counts collected at a continuous count station. Comparisons are made between simulated ITS counts, coverage counts, and actual ground counts. The simulation results indicate that when there are<330 daily traffic counts missing in a set of ITS counts in a year, that is, when there are at least 35 days of valid data, ITS counts can be used to derive a better AADT than using coverage counts. This result is applied to calculate the VMT for the Hampton Roads region in Virginia. The comparison between the VMTs derived with using and not using the threshold number indicates that these two VMTs are significantly different.     

A hybrid approach to integrate fuzzy C-means based imputation method with genetic algorithm for missing traffic volume data estimation

Although various innovative traffic sensing technologies have been widely employed, incomplete sensor data is one of the most major problems to significantly degrade traffic data quality and integrity. In this study, a hybrid approach integrating the Fuzzy C-Means (FCM)-based imputation method with the Genetic Algorithm (GA) is develop for missing traffic volume data estimation based on inductance loop detector outputs. By utilizing the weekly similarity among data, the conventional vector-based data structure is firstly transformed into the matrix-based data pattern. Then, the GA is applied to optimize the membership functions and centroids in the FCM model. The experimental tests are conducted to verify the effectiveness of the proposed approach. The traffic volume data collected at different temporal scales were used as the testing dataset, and three different indicators, including root mean square error, correlation coefficient, and relative accuracy, are utilized to quantify the imputation performance compared with some conventional methods (Historical method, Double Exponential Smoothing, and Autoregressive Integrated Moving Average model). The results show the proposed approach outperforms the conventional methods under prevailing traffic conditions.     

Comparing whole-link travel time models

In a model commonly used in dynamic traffic assignment the link travel time for a vehicle entering a link at time t is taken as a function of the number of vehicles on the link at time t. In an alternative recently introduced model, the travel time for a vehicle entering a link at time t is taken as a function of an estimate of the flow in the immediate neighbourhood of the vehicle, averaged over the time the vehicle is traversing the link. Here we compare the solutions obtained from these two models when applied to various inflow profiles. We also divide the link into segments, apply each model sequentially to the segments and again compare the results. As the number of segments is increased, the discretisation refined to the continuous limit, the solutions from the two models converge to the same solution, which is the solution of the Lighthill, Whitham, Richards (LWR) model for traffic flow. We illustrate the results for different travel time functions and patterns of inflows to the link. In the numerical examples the solutions from the second of the two models are closer to the limit solutions. We also show that the models converge even when the link segments are not homogeneous, and introduce a correction scheme in the second model to compensate for an approximation error, hence improving the approximation to the LWR model.     

Norm approximation method for handling traffic count inconsistencies in path flow estimator

Path flow estimator (PFE) is a one-stage network observer proposed to estimate path flows and hence origin–destination (O–D) flows from traffic counts in a transportation network. Although PFE does not require traffic counts to be collected on all network links when inferring unmeasured traffic conditions, it does require all available counts to be reasonably consistent. This requirement is difficult to fulfill in practice due to errors inherited in data collection and processing. The original PFE model handles this issue by relaxing the requirement of perfect replication of traffic counts through the specification of error bounds. This method enhances the flexibility of PFE by allowing the incorporation of local knowledge, regarding the traffic conditions and the nature of traffic data, into the estimation process. However, specifying appropriate error bounds for all observed links in real networks turns out to be a difficult and time-consuming task. In addition, improper specification of the error bounds could lead to a biased estimation of total travel demand in the network. This paper therefore proposes the norm approximation method capable of internally handling inconsistent traffic counts in PFE. Specifically, three norm approximation criteria are adopted to formulate three L_p-PFE models for estimating consistent path flows and O–D flows that simultaneously minimize the deviation between the estimated and observed link volumes. A partial linearization algorithm embedded with an iterative balancing scheme and a column generation procedure is developed to solve the three L_p-PFE models. In addition, the proposed L_p-PFE models are illustrated with numerical examples and the characteristics of solutions obtained by these models are discussed.     

A robust optimization approach for dynamic traffic signal control with emission considerations

We consider an analytical signal control problem on a signalized network whose traffic flow dynamic is described by the Lighthill–Whitham–Richards (LWR) model (Lighthill and Whitham, 1955; Richards, 1956). This problem explicitly addresses traffic-derived emissions as constraints or objectives. We seek to tackle this problem using a mixed integer mathematical programming approach. Such class of problems, which we call LWR-Emission (LWR-E), has been analyzed before to certain extent. Since mixed integer programs are practically efficient to solve in many cases (Bertsimas et al., 2011b), the mere fact of having integer variables is not the most significant challenge to solving LWR-E problems; rather, it is the presence of the potentially nonlinear and nonconvex emission-related constraints/objectives that render the program computationally expensive.To address this computational challenge, we proposed a novel reformulation of the LWR-E problem as a mixed integer linear program (MILP). This approach relies on the existence of a statistically valid macroscopic relationship between the aggregate emission rate and the vehicle occupancy on the same link. This relationship is approximated with certain functional forms and the associated uncertainties are handled explicitly using robust optimization (RO) techniques. The RO allows emissions-related constraints and/or objectives to be reformulated as linear forms under mild conditions. To further reduce the computational cost, we employ a link-based LWR model to describe traffic dynamics with the benefit of fewer (integer) variables and less potential traffic holding. The proposed MILP explicitly captures vehicle spillback, avoids traffic holding, and simultaneously minimizes travel delay and addresses emission-related concerns.     

Strava Metro data for bicycle monitoring: a literature review

ABSTRACT

Monitoring bicycle trips is no longer limited to traditional sources, such as travel surveys and counts. Strava, a popular fitness tracker, continuously collects human movement trajectories, and its commercial data service, Strava Metro, has enriched bicycle research opportunities over the last five years. Accrued knowledge from colleagues who have already utilised Strava Metro data can be valuable for those seeking expanded monitoring options. To convey such knowledge, this paper synthesises a data overview, extensive literature review on how the data have been applied to deal with drivers’ bicycle-related issues, and implications for future work. The review results indicate that Strava Metro data have the potential—although finite—to be used to identify various travel patterns, estimate travel demand, analyse route choice, control for exposure in crash models, and assess air pollution exposure. However, several challenges, such as the under-representativeness of the general population, bias towards and away from certain groups, and lack of demographic and trip details at the individual level, prevent researchers from depending entirely on the new data source. Cross-use with other sources and validation of reliability with official data could enhance the potentiality.     

Passenger travel characteristics and bus operational states: a study based on IC card and GPS data in Yinchuan,China

ABSTRACT

In recent years, public transport has been developing rapidly and producing large amounts of traffic data. Emerging big data-mining techniques enable the application of these data in a variety of ways. This study uses bus intelligent card (IC card) data and global positioning system (GPS) data to estimate passenger boarding and alighting stations. First, an estimation model for boarding stations is introduced to determine passenger boarding stations. Then, the authors propose an innovative uplink and downlink information identification model (UDI) to generate information for estimating alighting stations. Subsequently, the estimation model for the alighting stations is introduced. In addition, a transfer station identification model is also developed to determine transfer stations. These models are applied to Yinchuan, China to analyze passenger flow characteristics and bus operations. The authors obtain passenger flows based on stations (stops), bus lines, and traffic analysis zones (TAZ) during weekdays and weekends. Moreover, average bus operational speeds are obtained. These findings can be used in bus network planning and optimization as well as bus operation scheduling.     

GIS-based transit trip allocation methods converting stop-level boarding and alighting trips into TAZ trips

ABSTRACT

This study introduces a framework to improve the utilization of new data sources such as automated vehicle location (AVL) and automated passenger counting (APC) systems in transit ridership forecasting models. The direct application of AVL/APC data to travel forecasting requires an important intermediary step that links stops and activities – boarding and alighting – to the actual locations (at the traffic analysis zone (TAZ) level) that generated/attracted these trips. GIS-based transit trip allocation methods are developed with a focus on considering the case when the access shed spans multiple TAZs. The proposed methods improve practical applicability with easily obtained data. The performance of the proposed allocation methods is further evaluated using transit on-board survey data. The results show that the methods can effectively handle various conditions, particularly for major activity generators. The average errors between observed data and the proposed method are about 8% for alighting trips and 18% for boarding trips.     

Empirical analysis of short period traffic counts and their efficiency: the case of Indian traffic

Short period traffic counts (SPTCs) are conducted routinely to estimate the annual average daily traffic (AADT) at a particular site. This paper uses Indian traffic volume data to methodically and extensively study the effect of four aspects related to the design of SPTCs. These four aspects are: (i) for how long, (ii) on which days should SPTCs be carried out, (iii) how many times, and (iv) on which months should SPTCs be carried out? The analyses indicate that the best durations for conducting SPTCs are 3 days (starting with a Thursday) and 7 days, for total traffic and truck traffic, respectively. Further, these counts should be repeated twice a year keeping a separation of two months between the counts to obtain good estimates of AADT at minimal cost. An additional outcome of this study has been the determination of seasonal factor values for roads in developing economies, like India.     

A model for the visualization and analysis of elevator traffic

ABSTRACT

Analysis of elevator traffic in high rise buildings is critical to the performance evaluation of elevator group control systems (EGCS). Elevator dispatching methods or parking algorithms in an EGCS can be designed or modified according to analyses of traffic flow. However, interpretation of traffic flow based solely on numerical data may not be explicit and transparent for EGCS experts as well as for other non-expert building administration. In this study, we present a model for visualization and analysis of elevator traffic. First, we present an alternative approach for traffic analysis which we call route visualization. In the proposed approach, we initially decompose elevator traffic into its component parts and investigate each component independently. Then, using superposition of components we obtain a reconstructed model of overall traffic. This modeling approach provides component-based traffic analysis and representation of routes with intensities through data visualization. In the second part we introduce a multi-dimensional analysis of time parameters in ECGS. This approach provides a comparative analysis of several control algorithms such as dispatch or park algorithms for different combinations of traffic components.     

The estimation of a time-dependent OD trip table with vehicle trajectory samples

Abstract

In this paper we discuss a dynamic origin–destination (OD) estimation problem that has been used for identifying time-dependent travel demand on a road network. Even though a dynamic OD table is an indispensable data input for executing a dynamic traffic assignment, it is difficult to construct using the conventional OD construction method such as the four-step model. For this reason, a direct estimation method based on field traffic data such as link traffic counts has been used. However, the method does not account for a logical relationship between a travel demand pattern and socioeconomic attributes. In addition, the OD estimation method cannot guarantee the reliability of estimated results since the OD estimation problem has a property named the ‘underdetermined problem.’ In order to overcome such a problem, the method developed in this paper makes use of vehicle trajectory samples with link traffic counts. The new method is applied to numerical examples and shows promising capability for identifying a temporal and spatial travel demand pattern.     

Freight Data Assembling and Modeling: Methodologies and Practice

Abstract

In comparison to personal travel, freight movements within large metropolitan areas are much less studied. Most conventional transportation models and planning analysis that disregarded freight flows have been criticized on the plausibility of their results and conclusions. To alleviate these problems, this study proposes a non-survey based approach to assemble and process freight data in a systematic way. A freight origin–destination (OD) matrix of freight flows can be developed using secondary data sources. The estimated freight flows can be loaded together with conventional passenger flows onto the regional highway network of a large metropolitan area. As a case study, this non-survey based approach was applied to build a freight OD and study the traffic flows in Los Angeles. It concluded that this approach can be used to analyze urban freight movement in a low-cost way in which planning agencies can overcome the common omission of freight flow information in their transportation plans.     

Random parameter models for estimating statewide daily bicycle counts using crowdsourced data

Persistent lack of non-motorized traffic counts can affect the evidence-based decisions of transportation planning and safety-concerned agencies in making reliable investments in bikeway and other non-motorized facilities. Researchers have used various approaches to estimate bicycles counts, such as scaling, direct-demand modeling, time series, and others. In recent years, an increasing number of studies have tried to use crowdsourced data for estimating the bicycle counts. Crowdsourced data only represents a small percentage of cyclists. This percentage, on the other hand, can change based on the location, facility type, meteorological, and other factors. Moreover, the autocorrelation observed in bicycle counts may be different from the autocorrelation structure observed among crowdsourced platform users, such as Strava. Strava users are more consistent; hence, the time series count data may be stationary, while bicycle demand may vary based on seasonal factors. In addition to seasonal variation, several time-invariant contributing factors (e.g., facility type, roadway characteristics, household income) affect bicycle demand, which needs to be accounted for when developing direct demand models. In this paper, we use a mixed-effects model with autocorrelated errors to predict daily bicycle counts from crowdsourced data across the state of Texas. Additionally, we supplement crowdsourced data with other spatial and temporal factors such as roadway facility, household income, population demographics, population density and weather conditions to predict bicycle counts. The results show that using a robust methodology, we can predict bicycle demand with a 29% margin of error, which is significantly lower than merely scaling the crowdsourced data (41%).     

Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic data

Accurately modeling traffic speeds is a fundamental part of efficient intelligent transportation systems. Nowadays, with the widespread deployment of GPS-enabled devices, it has become possible to crowdsource the collection of speed information to road users (e.g. through mobile applications or dedicated in-vehicle devices). Despite its rather wide spatial coverage, crowdsourced speed data also brings very important challenges, such as the highly variable measurement noise in the data due to a variety of driving behaviors and sample sizes. When not properly accounted for, this noise can severely compromise any application that relies on accurate traffic data. In this article, we propose the use of heteroscedastic Gaussian processes (HGP) to model the time-varying uncertainty in large-scale crowdsourced traffic data. Furthermore, we develop a HGP conditioned on sample size and traffic regime (SSRC-HGP), which makes use of sample size information (probe vehicles per minute) as well as previous observed speeds, in order to more accurately model the uncertainty in observed speeds. Using 6 months of crowdsourced traffic data from Copenhagen, we empirically show that the proposed heteroscedastic models produce significantly better predictive distributions when compared to current state-of-the-art methods for both speed imputation and short-term forecasting tasks.