Automating a framework to extract and analyse transport related social media content: The potential and the challenges

Harnessing the potential of new generation transport data and increasing public participation are high on the agenda for transport stakeholders and the broader community. The initial phase in the program of research reported here proposed a framework for mining transport-related information from social media, demonstrated and evaluated it using transport-related tweets associated with three football matches as case studies. The goal of this paper is to extend and complement the previous published studies. It reports an extended analysis of the research results, highlighting and elaborating the challenges that need to be addressed before a large-scale application of the framework can take place. The focus is specifically on the automatic harvesting of relevant, valuable information from Twitter. The results from automatically mining transport related messages in two scenarios are presented i.e. with a small-scale labelled dataset and with a large-scale dataset of 3.7 m tweets. Tweets authored by individuals that mention a need for transport, express an opinion about transport services or report an event, with respect to different transport modes, were mined. The challenges faced in automatically analysing Twitter messages, written in Twitter’s specific language, are illustrated. The results presented show a strong degree of success in the identification of transport related tweets, with similar success in identifying tweets that expressed an opinion about transport services. The identification of tweets that expressed a need for transport services or reported an event was more challenging, a finding mirrored during the human based message annotation process. Overall, the results demonstrate the potential of automatic extraction of valuable information from tweets while pointing to areas where challenges were encountered and additional research is needed. The impact of a successful solution to these challenges (thereby creating efficient harvesting systems) would be to enable travellers to participate more effectively in the improvement of transport services.     

Fuzzy ontology-based sentiment analysis of transportation and city feature reviews for safe traveling

Traffic congestion is rapidly increasing in urban areas, particularly in mega cities. To date, there exist a few sensor network based systems to address this problem. However, these techniques are not suitable enough in terms of monitoring an entire transportation system and delivering emergency services when needed. These techniques require real-time data and intelligent ways to quickly determine traffic activity from useful information. In addition, these existing systems and websites on city transportation and travel rely on rating scores for different factors (e.g., safety, low crime rate, cleanliness, etc.). These rating scores are not efficient enough to deliver precise information, whereas reviews or tweets are significant, because they help travelers and transportation administrators to know about each aspect of the city. However, it is difficult for travelers to read, and for transportation systems to process, all reviews and tweets to obtain expressive sentiments regarding the needs of the city. The optimum solution for this kind of problem is analyzing the information available on social network platforms and performing sentiment analysis. On the other hand, crisp ontology-based frameworks cannot extract blurred information from tweets and reviews; therefore, they produce inadequate results. In this regard, this paper proposes fuzzy ontology-based sentiment analysis and semantic web rule language (SWRL) rule-based decision-making to monitor transportation activities (accidents, vehicles, street conditions, traffic volume, etc.) and to make a city-feature polarity map for travelers. This system retrieves reviews and tweets related to city features and transportation activities. The feature opinions are extracted from these retrieved data, and then fuzzy ontology is used to determine the transportation and city-feature polarity. A fuzzy ontology and an intelligent system prototype are developed by using Protégé web ontology language (OWL) and Java, respectively. The experimental results show satisfactory improvement in tweet and review analysis and opinion mining.     

Efficient multiple model particle filtering for joint traffic state estimation and incident detection

This article proposes an efficient multiple model particle filter (EMMPF) to solve the problems of traffic state estimation and incident detection, which requires significantly less computation time compared to existing multiple model nonlinear filters. To incorporate the on ramps and off ramps on the highway, junction solvers for a traffic flow model with incident dynamics are developed. The effectiveness of the proposed EMMPF is assessed using a benchmark hybrid state estimation problem, and using synthetic traffic data generated by a micro-simulation software. Then, the traffic estimation framework is implemented using field data collected on Interstate 880 in California. The results show the EMMPF is capable of estimating the traffic state and detecting incidents and requires an order of magnitude less computation time compared to existing algorithms, especially when the hybrid system has a large number of rare models.     

Mining and correlating traffic events from human sensor observations with official transport data using self-organizing-maps

Cities are complex systems, where related Human activities are increasingly difficult to explore within. In order to understand urban processes and to gain deeper knowledge about cities, the potential of location-based social networks like Twitter could be used a promising example to explore latent relationships of underlying mobility patterns. In this paper, we therefore present an approach using a geographic self-organizing map (Geo-SOM) to uncover and compare previously unseen patterns from social media and authoritative data. The results, which we validated with Live Traffic Disruption (TIMS) feeds from Transport for London, show that the observed geospatial and temporal patterns between special events (r = 0.73), traffic incidents (r = 0.59) and hazard disruptions (r = 0.41) from TIMS, are strongly correlated with traffic-related, georeferenced tweets. Hence, we conclude that tweets can be used as a proxy indicator to detect collective mobility events and may help to provide stakeholders and decision makers with complementary information on complex mobility processes.     

Incident detection methods using probe vehicles with on-board GPS equipment

Mobile communication instruments have made detecting traffic incidents possible by using floating traffic data. This paper studies the properties of traffic flow dynamics during incidents and proposes incident detection methods using floating data collected by probe vehicles equipped with on-board global positioning system (GPS) equipment. The proposed algorithms predict the time and location of traffic congestion caused by an incident. The detection rate and false rate of the models are examined using a traffic flow simulator, and the performance measures of the proposed methods are compared with those of previous methods.     

Effective arterial road incident detection: A Bayesian network based algorithm

Timely and accurate incident detection is an essential part of any successful advanced traffic management system. The complex nature of arterial road traffic makes automated incident detection a real challenge. Stable performance and strong transferability remain major issues concerning the existing incident detection algorithms. A new arterial road incident detection algorithm TSC_ar is presented in this paper. In this algorithm, Bayesian networks are used to quantitatively model the causal dependencies between traffic events (e.g. incident) and traffic parameters. Using real time traffic data as evidence, the Bayesian networks update the incident probability at each detection interval through two-way inference. An incident alarm is issued when the estimated incident probability exceeds the predefined decision threshold. The Bayesian networks allow us to subjectively build existing traffic knowledge into their conditional probability tables, which makes the knowledge base for incident detection robust and dynamic. Meanwhile, we incorporate intersection traffic signals into traffic data processing. A total of 40 different types of arterial road incidents are simulated to test the performance of the algorithm. The high detection rate of 88% is obtained while the false alarm rate of the algorithm is maintained as low as 0.62%. Most importantly, it is found that both the detection rate and false alarm rate are not sensitive to the incident decision thresholds. This is the unique feature of the TSC_ar algorithm, which suggests that the Bayesian network approach is advanced in enabling effective arterial road incident detection.     

Examination of factors affecting freeway incident clearance times: a comparison of the generalized F model and several alternative nested models

Traffic incidents are a principal cause of congestion on urban freeways, reducing capacity and creating risks for both involved motorists and incident response personnel. As incident durations increase, the risk of secondary incidents or crashes also becomes problematic. In response to these issues, many road agencies in metropolitan areas have initiated incident management programs aimed at detecting, responding to, and clearing incidents to restore freeways to full capacity as quickly and safely as possible. This study examined those factors that impact the time required by the Michigan Department of Transportation Freeway Courtesy Patrol to clear incidents that occurred on the southeastern Michigan freeway network. These models were developed using traffic flow data, roadway geometry information, and an extensive incident inventory database. A series of parametric hazard duration models were developed, each assuming a different underlying probability distribution for the hazard function. Although each modeling framework provided results that were similar in terms of the direction of factor effects, there was significant variability in terms of the estimated magnitude of these impacts. The generalized F distribution was shown to provide the best fit to the incident clearance time data, and the use of poorer fitting distributions was shown to result in severe over‐estimation or under‐estimation of factor effects. Those factors that were found to impact incident clearance times included the time of day and month when the incident occurred, the geometric and traffic characteristics of the freeway segment, and the characteristics of each incident. Copyright © 2012 John Wiley & Sons, Ltd.     

Reprint of: Modelling the impact of traffic incidents on travel time reliability

Traffic incidents are recognised as one of the key sources of non-recurrent congestion that often leads to reduction in travel time reliability (TTR), a key metric of roadway performance. A method is proposed here to quantify the impacts of traffic incidents on TTR on freeways. The method uses historical data to establish recurrent speed profiles and identifies non-recurrent congestion based on their negative impacts on speeds. The locations and times of incidents are used to identify incidents among non-recurrent congestion events. Buffer time is employed to measure TTR. Extra buffer time is defined as the extra delay caused by traffic incidents. This reliability measure indicates how much extra travel time is required by travellers to arrive at their destination on time with 95% certainty in the case of an incident, over and above the travel time that would have been required under recurrent conditions. An extra buffer time index (EBTI) is defined as the ratio of extra buffer time to recurrent travel time, with zero being the best case (no delay). A Tobit model is used to identify and quantify factors that affect EBTI using a selected freeway segment in the Southeast Queensland, Australia network. Both fixed and random parameter Tobit specifications are tested. The estimation results reveal that models with random parameters offer a superior statistical fit for all types of incidents, suggesting the presence of unobserved heterogeneity across segments. What factors influence EBTI depends on the type of incident. In addition, changes in TTR as a result of traffic incidents are related to the characteristics of the incidents (multiple vehicles involved, incident duration, major incidents, etc.) and traffic characteristics.     

An investigation of incident frequency,duration and lanes blockage for determining traffic delay

Traffic delay caused by incidents is closely related to three variables: incident frequency, incident duration, and the number of lanes blocked by an incident that is directly related to the bottleneck capacity. Relatively, incident duration has been more extensively studied than incident frequency and the number of lanes blocked in an incident. In this study, we provide an investigation of the influencing factors for all of these three variables based on an incident data set that was collected in New York City (NYC). The information about the incidents derived from the identification can be used by incident management agencies in NYC for strategic policy decision making and daily incident management and traffic operation. In identifying the influencing factors for incident frequency, a set of models, including Poisson and Negative Binomial regression models and their zero‐inflated models, were considered. An appropriate model was determined based on a model decision‐making tree. The influencing factors for incident duration were identified based on hazard‐based models where Exponential, Weibull, Log‐logistic, and Log‐normal distributions were considered for incident duration. For the number of lanes blocked in an incident, the identification of the influencing factors was based on an Ordered Probit model which can better capture the order inherent in the number of lanes blocked in an incident. As identified in this study, rain is the only factor that significantly influenced incident frequency. For incident duration and the number of lanes blocked in an incident, various factors had significant impact. As concluded in this study, there is a strong need to identify the influencing factors in terms of different types of incidents and the roadways where the incidents occured.     

A methodology to estimate travel time using dynamic traffic assignment (DTA) under incident conditions

This paper presents results from a research case study that examined the distribution of travel time of origin–destination (OD) pairs on a transportation network under incident conditions. Using a transportation simulation dynamic traffic assignment (DTA) model, incident on a transportation network is executed under normal conditions, incident conditions without traveler information availability, and incident conditions assuming that users had perfect knowledge of the incident conditions and could select paths to avoid the incident location. The results suggest that incidents have a different impact on different OD pairs. The results confirm that an effective traveler information system has the potential to ease the impacts of incident conditions network wide. Yet it is also important to note that the use of information may detriment some OD pairs while benefiting other OD pairs. The methodology demonstrated in this paper provides insights into the usefulness of embedding a fully calibrated DTA model into the analysis tools of a traffic management and information center.     

Characterization of incidents on an urban arterial road

The early warning of incidents on urban arterial roads in a congested city can reduce delay, accidents and pollutant emission. Freeway incident detection systems implemented in recent years may not be suitable for arterial incidents. Arterial incident detection is more difficult. The traffic flow on an arterial road is not conserved from the upstream end of a road link to the downstream end because urban traffic does turn in and out of side‐streets, car‐parks and local residences. Roadside friction such as kerbside parking and shopping traffic also tends to create apparent incidents which are in fact frequent and normal events. This paper develops a definition for an arterial incident and describes a case study on an arterial road in Melbourne, Australia. The study shows that detectors upstream of an incident are more useful for incident detection than downstream detectors. It also identifies occupancy and speed as the appropriate parameters to characterise and detect arterial incidents.     

Simulation studies of information propagation in a self-organizing distributed traffic information system

This paper investigates the feasibility of a self-organizing, completely distributed traffic information system based upon vehicle-to-vehicle communication technologies. Unlike centralized traffic information systems, the proposed system does not need public infrastructure investment as a prerequisite for implementation. Due to the complexity of the proposed system, simulation is selected as the primary approach in the feasibility studies. A simulation framework is built based on an existing microscopic traffic simulation model for the simulation studies. The critical questions for building the proposed market-driven system are examined both from communication requirements and traffic engineering points of view. Traffic information propagation both in freeway and arterial networks via information exchange among IVC-equipped vehicles is tested within the simulation framework. Results on the probability of successful IVC and traffic information propagation distance obtained from the simulation studies are generated and analyzed under incident-free and incident conditions for various roadway formats and parameter combinations. Comparisons between the speed of the incident information wave and the speed of the corresponding traffic shock wave due to the incident are analyzed for different scenarios as the most crucial aspect of the information propagation as a potential foundation for application in such a decentralized traffic information system.     

A new methodology for incident detection and characterization on surface streets

In this paper, a new methodology is presented for real-time detection and characterization of incidents on surface streets. The proposed automatic incident detection approach is capable of detecting incidents promptly as well as characterizing incidents in terms of time-varying lane-changing fractions and queue lengths in blocked lanes, lanes blocked due to incidents, and incident duration. The architecture of the proposed incident detection approach consists of three sequential procedures: (1) Symptom Identification for identification of incident symptoms, (2) Signal Processing for real-time prediction of incident-related lane traffic characteristics and (3) Pattern Recognition for incident recognition. Lane traffic counts and occupancy are the only two major types of input data, which can be readily collected from point detectors. The primary techniques utilized in this paper include: (1) a discrete-time, nonlinear, stochastic system with boundary constraints to predict real-time lane-changing fractions and queue lengths and (2) a pattern-recognition-based algorithm employing modified sequential probability ratio tests (MSPRT) to detect incidents. Off-line tests based on simulated as well as video-based real data were conducted to assess the performance of the proposed algorithm. The test results have indicated the feasibility of achieving real-time incident detection using the proposed methodology.     

Competing risk mixture model and text analysis for sequential incident duration prediction

Predicting the duration of traffic incidents sequentially during the incident clearance period is helpful in deploying efficient measures and minimizing traffic congestion related to such incidents. This study proposes a competing risk mixture hazard-based model to analyze the effect of various factors on traffic incident duration and predict the duration sequentially. First, topic modeling, a text analysis technique, is used to process the textual features of the traffic incident to extract time-dependent topics. Given four specific clearance methods and the uncertainty of these methods when used during traffic incidents, the proposed mixture model uses the multinomial logistic model and parametric hazard-based model to assess the influence of covariates on the probability of clearance methods and on the duration of the incident. Subsequently, the performance of estimated mixture model in sequentially predicting the incident duration is compared with that of the non-mixture model. The prediction results show that the presented mixture model outperforms the non-mixture model.     

An exploratory hazard-based analysis of highway incident duration

The statistical analysis of highway incident duration has become an increasingly import research topic due to the impact that highway incidents (vehicle accidents and disablements) have on traffic congestion. In addition, there is a growing need to evaluate incident management programs that seek to reduce incident duration and incident-induced traffic congestion. We apply hazard-based duration models to statistically evaluate the time it takes detect/report, respond to, and clear incidents. Two-year data from Washington State's incident response team program were used to estimate the hazard models. The model estimation results show that a wide variety of factors significantly affect incident times (i.e. detection/reporting, response, and clearance times), and that different distributional assumptions for the hazard function are appropriate for the different incident times being considered. It was also found that the estimated coefficients were not stable between the two years of data used in model estimation. The findings of this paper provide an important demonstration of method and an empirical basis to assess incident management programs.     

Automated detection of lane-blocking freeway incidents using artificial neural networks

A major source of urban freeway delay in the U.S. is non-recurring congestion caused by incidents. The automated detection of incidents is an important function of a freeway traffic management center. A number of incident detection algorithms, using inductive loop data as input, have been developed over the past several decades, and a few of them are being deployed at urban freeway systems in major cities. These algorithms have shown varying degrees of success in their detection performance. In this paper, we present a new incident detection technique based on artificial neural networks (ANNs). Three types of neural network models, namely the multi-layer feedforward (MLF), the self-organizing feature map (SOFM) and adaptive resonance theory 2 (ART2), were developed to classify traffic surveillance data obtained from loop detectors, with the objective of using the classified output to detect lane-blocking freeway incidents. The models were developed with simulation data from a study site and tested with both simulation and field data at the same site. The MLF was found to have the highest potential, among the three ANNs, to achieve a better incident detection performance. The MLF was also tested with limited field data collected from three other freeway locations to explore its transferability. Our results and analyzes with data from the study site as well as the three test sites have shown that the MLF consistently detected most of the lane-blocking incidents and typically gave a false alarm rate lower than the California, McMaster and Minnesota algorithms currently in use.     

The impacts of connected vehicle technology on network-wide traffic operation and fuel consumption under various incident scenarios

ABSTRACT

Incidents are a major source of traffic congestion and can lead to long and unpredictable delays, deteriorating traffic operations and adverse environmental impacts. The emergence of connected vehicles and communication technologies has enabled travelers to use real-time traffic information. The ability to exchange traffic information among vehicles has tremendous potential impacts on network performance especially in the case of non-recurrent congestion. To this end, this paper utilizes a microscopic simulation model of traffic in El Paso, Texas to investigate the impacts of incidents on traffic operation and fuel consumption at different market penetration rates (MPR) of connected vehicles. Several scenarios are implemented and tested to determine the impacts of incidents on network performance in an urban area. The scenarios are defined by changing the duration of incidents and the number of lanes closed. This study also shows how communication technology affects network performance in response to congestion. The results of the study demonstrate the potential effectiveness of connected vehicle technology in improving network performance. For an incident with a duration of 900?s and MPR of 80%, total fuel consumption and total travel time decreased by approximately 20%; 26% was observed in network-wide travel time and fuel consumption at 100% MPR.     

A fuzzy-based system for incident detection in urban street networks

Detecting incidents on urban streets or arterials using loop detector data is quite challenging. The pattern of the incident could be quite similar to non-incident cases as intersections get congested. This paper describes the development of a fuzzy logic for incident detection. An Integrated System for Incident Management (

-sim) was developed. An integral component of such system is a microscopic simulator,

-sim-s, an object-oriented model that allows for virtual detector installations at different locations, modeling different intersection layouts, traffic control types and timing, and link characteristics.

-sim-s was utilized to generate various incident scenarios and extracting associated detectors’ accumulative counts. A data clustering technique was utilized to consolidate the various incident scenarios into a single data set for the development of the Fuzzy Logic for incident detection at intersections (

-sim-fl). The

-sim-fl uses the detector data as well as other link properties in flagging detecting incidents.The

-sim-fl can be used to indicate the possibility of an incident, a stalled vehicle, or a sort of traffic disturbance. The devised logic was validated using separate simulation-based incident scenarios.     

Operational impacts of incident quick clearance legislation: a simulation analysis

Many states in the US have enacted quick clearance laws requiring drivers of vehicles involved in minor incidents to move their vehicles from travel lanes prior to the arrival of first responders. Since little is known about the effectiveness of these laws, this research sought to find the benefit–cost ratio of advertising quick clearance legislation to improve driver compliance, and compare it with benefit–cost ratios of other incident management strategies, particularly traffic cameras, freeway service patrols, and traffic sensors. The analysis used traffic simulation that applied application programming interfaces to produce random spatial and temporal occurrence of incidents, including incident start times, durations, and locations, based on normal distributions developed from field data, to test before and after the law scenarios. The results provide decision makers with support for prioritizing funding between these incident management strategies and indicated that investments in the advertisement of this law was beneficial. Copyright © 2011 John Wiley & Sons, Ltd.     

Modeling the time to the next primary and secondary incident: A semi-Markov stochastic process approach

Incidents are notorious for their delays to road users. Secondary incidents – i.e., incidents that occur within a certain temporal and spatial distance from the first/primary incident – can further complicate clearance and add to delays. While there are numerous studies on the empirical analysis of incident data, to the best of our knowledge, an analytical model that can be used for primary and secondary incident management planning that explicitly considers both the stochastic as well as the dynamic nature of traffic does not exist. In this paper, we present such a complementary model using a semi-Markov stochastic process approach. The model allows for unprecedented generality in the modeling of stochastics during incidents on freeways. Particularly, we relax the oftentimes restrictive Poisson assumption (in the modeling of vehicle arrivals, vehicle travel times, and incidence occurrence and recovery times) and explicitly model secondary incidents. Numerical case studies are provided to illustrate the proposed model.