Reinforcement learning approach for train rescheduling on a single-track railway

Optimal rail network infrastructure and rolling stock utilization can be achieved with use of different scheduling tools by extensive planning a long time before actual operations. The initial train timetable takes into account possible smaller disturbances, which can be compensated within the schedule. Bigger disruptions, such as accidents, rolling stock breakdown, prolonged passenger boarding, and changed speed limit cause delays that require train rescheduling. In this paper, we introduce a train rescheduling method based on reinforcement learning, and more specifically, Q-learning. We present here the Q-learning principles for train rescheduling, which consist of a learning agent and its actions, environment and its states, as well as rewards. The use of the proposed approach is first illustrated on a simple rescheduling problem comprising a single-lane track with three trains. The evaluation of the approach is performed on extensive set of experiments carried out on a real-world railway network in Slovenia. The empirical results show that Q-learning lead to rescheduling solutions that are at least equivalent and often superior to those of several basic rescheduling methods that do not rely on learning agents. The solutions are learned within reasonable computational time, a crucial factor for real-time applications.     

Closing the loop in real-time railway control: Framework design and impacts on operations

Railway traffic is heavily affected by disturbances and/or disruptions, which are often cause of delays and low performance of train services. The impact and the propagation of such delays can be mitigated by relying on automatic tools for rescheduling traffic in real-time. These tools predict future track conflict based on current train information and provide suitable control measures (e.g. reordering, retiming and/or rerouting) by using advanced mathematical models. A growing literature is available on these tools, but their effects on real operations are blurry and not yet well known, due to the very scarce implementation of such systems in practice.In this paper we widen the knowledge on how automatic real-time rescheduling tools can influence train performance when interfaced with railway operations. To this purpose we build up a novel traffic control framework that couples the state-of-the art automatic rescheduling tool ROMA, with the realistic railway traffic simulation environment EGTRAIN, used as a surrogate of the real field. At regular times ROMA is fed with current traffic information measured from the field (i.e. EGTRAIN) in order to predict possible conflicts and compute (sub) optimal control measures that minimize the max consecutive delay on the network. We test the impact of the traffic control framework based on different types of interaction (i.e. open loop, multiple open loop, closed loop) between the rescheduling tool and the simulation environment as well as different combinations of parameter values (such as the rescheduling interval and prediction horizon). The influence of different traffic prediction models (assuming e.g. aggressive versus conservative driving behaviour) is also investigated together with the effects on traffic due to control delays of the dispatcher in implementing the control measures computed by the rescheduling tool.Results obtained for the Dutch railway corridor Utrecht–Den Bosch show that a closed loop interaction outperforms both the multiple open loop and the open loop approaches, especially with large control delays and limited information on train entrance delays and dwell times. A slow rescheduling frequency and a large prediction horizon improve the quality of the control measure. A limited control delay and a conservative prediction of train speed help filtering out uncertain traffic dynamics thereby increasing the effectiveness of the implemented measures.     

Train rescheduling model with train delay and passenger impatience time in urban subway network

This paper considers the train rescheduling problem with train delay in urban subway network. With the objective of minimizing the negative effect of train delay to passengers, which is quantified with a weighted combination of travel time cost and the cost of giving up the planned trips, train rescheduling model is proposed to jointly synchronize both train delay operation constraints and passenger behavior choices. Space–time network is proposed to describe passenger schedule‐based path choices and obtain the shortest travel times. Impatience time is defined to describe the intolerance of passengers to train delay. By comparing the increased travel time due to train delay with the passenger impatience time, a binary variable is defined to represent whether the passenger will give up their planned trips or not. The proposed train rescheduling model is implemented using genetic algorithm, and the model effectiveness is further examined through numerical experiments of real‐world urban subway train timetabling test. Duration effects of the train delay to the optimization results are analyzed. Copyright © 2017 John Wiley & Sons, Ltd.     

Robust single-track train dispatching model under a dynamic and stochastic environment: A scenario-based rolling horizon solution approach

After a major service disruption on a single-track rail line, dispatchers need to generate a series of train meet-pass plans at different decision times of the rescheduling stage. The task is to recover the impacted train schedule from the current and future disturbances and minimize the expected additional delay under different forecasted operational conditions. Based on a stochastic programming with recourse framework, this paper incorporates different probabilistic scenarios in the rolling horizon decision process to recognize (1) the input data uncertainty associated with predicted segment running times and segment recovery times and (2) the possibilities of rescheduling decisions after receiving status updates. The proposed model periodically optimizes schedules for a relatively long rolling horizon, while selecting and disseminating a robust meet-pass plan for every roll period. A multi-layer branching solution procedure is developed to systematically generate and select meet-pass plans under different stochastic scenarios. Illustrative examples and numerical experiments are used to demonstrate the importance of robust disruption handling under a dynamic and stochastic environment. In terms of expected total train delay time, our experimental results show that the robust solutions are better than the expected value-based solutions by a range of 10-30%.     

Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach

In a heavily congested metro line, unexpected disturbances often occur to cause the delay of the traveling passengers, infeasibility of the current timetable and reduction of the operational efficiency. Due to the uncertain and dynamic characteristics of passenger demands, the commonly used method to recover from disturbances in practice is to change the timetable and rolling stock manually based on the experiences and professional judgements. In this paper, we develop a stochastic programming model for metro train rescheduling problem in order to jointly reduce the time delay of affected passengers, their total traveling time and operational costs of trains. To capture the complexity of passenger traveling characteristics, the arriving ratio of passengers at each station is modeled as a non-homogeneous poisson distribution, in which the intensity function is treated as time-varying origin-to-destination passenger demand matrices. By considering the number of on-board passengers, the total energy usage is modeled as the difference between the tractive energy consumption and the regenerative energy. Then, we design an approximate dynamic programming based algorithm to solve the proposed model, which can obtain a high-quality solution in a short time. Finally, numerical examples with real-world data sets are implemented to verify the effectiveness and robustness of the proposed approaches.     

Passenger oriented railway disruption management by adapting timetables and rolling stock schedules

In passenger railway operations, unforeseen events require railway operators to adjust their timetable and their resource schedules. The passengers will also adapt their routes to their destinations. When determining the new timetable and rolling stock schedule, the railway operator has to take passenger behavior into account. The operator should increase the capacity of trains for which the operator expects more demand than on a regular day. Furthermore, the operator could increase the frequency of the trains that serve stations with an additional demand.This paper describes a real-time disruption management approach which integrates the rescheduling of the rolling stock and the timetable by taking the changed passenger demand into account. The timetable decisions are limited to additional stops of trains at stations at which they normally would not call. Several variants of the approach are suggested, with the difference in how to determine which additional stops should be executed.Real-time rescheduling requires fast solutions. Therefore a heuristic approach is used. We demonstrate the performance of the several variants of our algorithm on realistic instances of Netherlands Railways, the major railway operator in the Netherlands.     

Hybrid predictive control for real-time optimization of public transport systems’ operations based on evolutionary multi-objective optimization

A hybrid predictive control formulation based on evolutionary multi-objective optimization to optimize real-time operations of public transport systems is presented. The state space model includes bus position, expected load and arrival time at stops. The system is based on discrete events, and the possible operator control actions are: holding vehicles at stations and skipping some stations. The controller (operator) pursues the minimization of a dynamic objective function to generate better operational decisions under uncertain demand at bus stops. In this work, a multi-objective approach is conducted to include different goals in the optimization process that could be opposite. In this case, the optimization was defined in terms of two objectives: waiting time minimization on one side, and impact of the strategies on the other. A genetic algorithm method is proposed to solve the multi-objective dynamic problem. From the conducted experiments considering a single bus line corridor, we found that the two objectives are opposite but with a certain degree of overlapping, in the sense that in all cases both objectives significantly improve the level of service with respect to the open-loop scenario by regularizing the headways. On average, the observed trade-off validates the proposed multi-objective methodology for the studied system, allowing dynamically finding the pseudo-optimal Pareto front and making real-time decisions based on different optimization criteria reflected in the proposed objective function compounds.     

A bi-level model for single-line rail timetable design with consideration of demand and capacity

This paper proposes a bi-level model to solve the timetable design problem for an urban rail line. The upper level model aims at determining the headways between trains to minimize total passenger cost, which includes not only the usual perceived travel time cost, but also penalties during travel. With the headways given by the upper level model, passengers’ arrival times at their origin stops are determined by the lower level model, in which the cost-minimizing behavior of each passenger is taken into account. To make the model more realistic, explicit capacity constraints of individual trains are considered. With these constraints, passengers cannot board a full train, but wait in queues for the next coming train. A two-stage genetic algorithm incorporating the method of successive averages is introduced to solve the bi-level model. Two hypothetical examples and a real world case are employed to evaluate the effectiveness of the proposed bi-level model and algorithm. Results show that the bi-level model performs well in reducing total passenger cost, especially in reducing waiting time cost and penalties. And the section loading-rates of trains in the optimized timetable are more balanced than the even-headway timetable. The sensitivity analyses show that passenger’s desired arrival time interval at destination and crowding penalty factor have a high influence on the optimal solution. And with the dispersing of passengers' desired arrival time intervals or the increase of crowding penalty factor, the section loading-rates of trains become more balanced.     

Estimation of train dwell time at short stops based on track occupation event data: A study at a Dutch railway station

Train dwell time is one of the most unpredictable components of railway operations, mainly because of the varying volumes of alighting and boarding passengers. However, for reliable estimations of train running times and route conflicts on main lines, it is necessary to obtain accurate estimations of dwell times at the intermediate stops on the main line, the so‐called short stops. This is a great challenge for a more reliable, efficient and robust train operation. Previous research has shown that the dwell time is highly dependent on the number of boarding and alighting passengers. However, these numbers are usually not available in real time. This paper discusses the possibility of a dwell time estimation model at short stops without passenger demand information by means of a statistical analysis of track occupation data from the Netherlands. The analysis showed that the dwell times are best estimated for peak and off‐peak hours separately. The peak‐hour dwell times are estimated using a linear regression model of train length, dwell times at previous stops and dwell times of the preceding trains. The off‐peak‐hour dwell times are estimated using a non‐parametric regression model, in particular, the k‐nearest neighbor model. There are two major advantages of the proposed estimation models. First, the models do not need passenger flow data, which is usually impossible to obtain in real time in practice. Second, detailed parameters of rolling stock configuration and platform layout are not required, which makes the model more generic and eases implementation. A case study at Dutch railway stations shows that the estimation accuracy is 85.8%–88.5% during peak hours and 80.1% during off‐peak hours, which is relatively high. We conclude that the estimation of dwell times at short stop stations without passenger data is possible. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimizing sustainable feeder bus operation considering realistic networks and heterogeneous demand

A mathematical model is developed to optimize social and fiscal sustainable operation of a feeder bus system considering realistic network and heterogeneous demand. The objective total profit is a nonlinear, mixed integer function, which is maximized by optimizing the number of stops, headway, and fare. The stops are located which maximize the ridership. The demand elasticity for the bus service is dependent on passengers' access distance, wait time, in‐vehicle time, and fare. An optimization algorithm is developed to search for the optimal solution that maximizes the profit. The modeling approach is applied to planning a bus transit system within Woodbridge, New Jersey. Copyright © 2011 John Wiley & Sons, Ltd.     

A multi-instrumented approach to observing the activity rescheduling decision process

Activity scheduling simulation models represent an emerging and proposing approach to forecasting travel demand. The most significant developmental challenge is the lack of empirical data on how people actually proceed through the scheduling and conflict resolution process. This paper develops a new methodology to collect data about the rescheduling decision process. The data collection involves six stages: preplanned schedule interview, coding of the preplanned schedule, second-by-second Global Positioning System tracking, internet-based prompted recall diary, detection of rescheduling decisions (via comparison of planned versus executed activities), and a final in-depth interview probing the how and why of rescheduling decisions. Each stage of the methodology is described in detail with example results drawn from a pilot study. Key discoveries include: elicitation of multiple preplanned schedule reporting methods (verbal, point-form, calendar); discovery that activity attributes (time, location, involved persons) are planned on significantly different time horizons and include partial elaboration; and provision of new insights into how and why rescheduling decisions are made. A method for automatically tracking rescheduling decisions was also discovered. Overall, the new methodology has potential to contribute to the development of more realistic models of the entire scheduling process, especially rescheduling and conflict resolution sub-models.     

Six reasons why supply‐oriented indicators systematically overestimate service quality in public transport

Supply‐oriented measures of quality lead to a systematic overestimate of quality as experienced by travellers in public transport. An example is a train with an average occupation rate for seats being 50%, where, nevertheless, the occupation rate observed by travellers is much higher when some parts of the trajectory are busy. Similar examples are discussed for waiting times at stops, probabilities of arriving in time, probabilities of getting a connection and walking distances to bus stops. A plea is then made for putting more effort in measuring demand‐oriented quality measures.     

Automated real-time railway traffic control: an experimental analysis of reliability,resilience and robustness

Railway transportation provides sustainable, fast and safe transport. Its attractiveness is linked to a broad concept of service reliability: the capability to adhere to a timetable in the presence of delays perturbing traffic. To counter these phenomena, real-time rescheduling can be used, changing train orders and times, according to rules of thumb, or mathematical optimization models, minimizing delays or maximizing punctuality. In the literature, different indices of robustness, reliability and resilience are defined for railway traffic. We review and evaluate these indices applied to railway traffic control, comparing optimal rescheduling approaches such as Open Loop and Closed Loop control, to a typical First-Come-First-Served dispatching rule, and following the timetable (no-action). This experimental analysis clarifies the benefits of automated traffic control for infrastructure managers, railway operators and passengers. The timetable order, normally used in assessing a-priori reliability, systematically overestimates unreliability of operations that can be reduced by real-time control.     

Train design features affecting boarding and alighting of passengers

Accurately predicting train dwell time is critical to running an effective and efficient service. With high‐density passenger services, large numbers of passengers must be able to board and alight the train quickly – and within scheduled dwell times. Using a specially constructed train mock‐up in a pedestrian movement laboratory, the experiments outlined in this paper examine the impact of train carriage design factors such as door width, seat type, platform edge doors and horizontal gap on the time taken by passengers to board and alight. The findings illustrate that the effectiveness of design features depends on whether there are a majority of passengers boarding or alighting. An optimum door width should be between 1.7 and 1.8 m. The use of a central pole and platform edge doors produced no major effects, but a 200 mm horizontal gap could increase the movement of passengers. There is no clear effect of the type of seats and neither the standbacks between 50, 300 and 500 mm. Further research will look for the relationship between the dwell time and the characteristics of passengers such as personal space. Copyright © 2017 John Wiley & Sons, Ltd.     

Integrating capacity analysis with high-speed railway timetabling: A minimum cycle time calculation model with flexible overtaking constraints and intelligent enumeration

Compared with most optimization methods for capacity evaluation, integrating capacity analysis with timetabling can reveal the types of train line plans and operating rules that have a positive influence on improving capacity utilization as well as yielding more accurate analyses. For most capacity analyses and cyclic timetabling methods, the cycle time is a constant (e.g., one or two hours). In this paper, we propose a minimum cycle time calculation (MCTC) model based on the periodic event scheduling problem (PESP) for a given train line plan, which is promising for macroscopic train timetabling and capacity analysis. In accordance with train operating rules, a non-collision constraint and a series of flexible overtaking constraints (FOCs) are constructed based on variations of the original binary variables in the PESP. Because of the complexity of the PESP, an iterative approximation (IA) method for integration with the CPLEX solver is proposed. Finally, two hypothetical cases are considered to analyze railway capacity, and several influencing factors are studied, including train regularity, train speed, line plan specifications (train stops), overtaking and train heterogeneity. The MCTC model and IA method are used to test a real-world case involving the timetable of the Beijing–Shanghai high-speed railway in China.     

Express guideway transit: A case for further development in transit automation

Safe and reliable coupling and decoupling of cars from a moving train is feasible with further developments in linear motor propulsion and control of transit vehicles. This allows the last car of a train to decouple and stop at a station for a relative long dwell time, before it accelerates and is coupled to a following train. Controlled doors in front and rear of the transit vehicle permit passengers to walk through the train to the car which stops at their destination. A proposed transit system using these features is described and compared to Bombardier's Advanced Rapid Transit. Potential advantages are high schedule speed, uncrowded trains, smaller and more stations, low energy requirements and a smaller vehicle fleet.     

Waiting time perceptions at transit stops and stations: Effects of basic amenities,gender, and security

Waiting time in transit travel is often perceived negatively and high-amenity stops and stations are becoming increasingly popular as strategies for mitigating transit riders’ aversion to waiting. However, beyond recent evidence that realtime transit arrival information reduces perceived waiting time, there is limited empirical evidence as to which other specific station and stop amenities can effectively influence user perceptions of waiting time. To address this knowledge gap, the authors conducted a passenger survey and video-recorded waiting passengers at different types of transit stops and stations to investigate differences between survey-reported waiting time and video-recorded actual waiting time. Results from the survey and video observations show that the reported wait time on average is about 1.21 times longer than the observed wait time. Regression analysis was employed to explain the variation in riders’ reported waiting time as a function of their objectively observed waiting time, as well as station and stop amenities, weather, time of the day, personal demographics, and trip characteristics. Based on the regression results, most waits at stops with no amenities are perceived at least 1.3 times as long as they actually are. Basic amenities including benches and shelters significantly reduce perceived waiting times. Women waiting for more than 10 min in perceived insecure surroundings report waits as dramatically longer than they really are, and longer than do men in the same situation. The authors recommend a focus on providing basic amenities at stations and stops as broadly as possible in transit systems, and a particular focus on stops on low-frequency routes and in less safe areas for security measures.     

Not All Transfers Are Created Equal: Towards a Framework Relating Transfer Connectivity to Travel Behaviour

Abstract

Walking from origins to transit stops, transferring between transit lines and walking from transit stops to destinations—all add to the burden of transit travel, sometimes to a very large degree. Transfers in particular can be stressful and/or time‐consuming for travellers, discouraging transit use. As such, transit facilities that reduce the burdens of walking, waiting and transferring can substantially increase transit system efficacy and use. In this paper, we argue that transit planning research on transit stops and stations, and transit planning practice frequently lack a clear conceptual framework relating transit waits and transfers with what we know about travel behaviour. Therefore, we draw on the concepts of transfer penalties and value of time in the travel behaviour/economics literature to develop a framework that situates transfer penalties within the total travel generalized costs of a transit trip. For example, value of time is important in relating actual time of waiting and walking to the perceived time of travel. We also draw on research to classify factors most important to users’ perspectives and travel behaviour—transfer costs, time scheduling and five transfer facility attributes: (1) access, (2) connection and reliability, (3) information, (4) amenities, and (5) security and safety. Using this framework, we seek to explicitly relate improvements of transfer stops/stations with components of transfer penalties and changes in travel behaviour (through a reduction in transfer penalties). We conclude that the employment of such a framework can help practitioners better apply the most effective improvements to transit stops and transfer facilities.     

Integrated optimization of bus priority operations in connected vehicle environment

Most previous works associated with transit signal priority merely focus on the optimization of signal timings, ignoring both bus speed and dwell time at bus stops. This paper presents a novel approach to optimize the holding time at bus stops, signal timings, and bus speed to provide priority to buses at isolated intersections. The objective of the proposed model is to minimize the weighted average vehicle delays of the intersection, which includes both bus delay and impact on nearby intersection traffic, ensuring that buses clear these intersections without being stopped by a red light. A set of formulations are developed to explicitly capture the interaction between bus speed, bus holding time, and transit priority signal timings. Experimental analysis is used to show that the proposed model has minimal negative impacts on general traffic and outperforms the no priority, signal priority only, and signal priority with holding control strategies (no bus speed adjustment) in terms of reducing average bus delays and stops. A sensitivity analysis further demonstrates the potential of the proposed approach to be applied to bus priority control systems in real‐time under different traffic demands, bus stop locations, and maximum speed limits. Copyright © 2016 John Wiley & Sons, Ltd.     

A delay root cause discovery and timetable adjustment model for enhancing the punctuality of railway services

Knock-on delay, which is the key factor in punctuality of railway service, is mainly related to two factors including the quality of timetable in the planning phase and disturbances which may result in unscheduled trains’ waiting or meeting in operation phase. If the delay root cause and the interactions among the factors responsible for these can be clearly clarified, then the punctuality of railway operations can be enhanced by taking reactions such as timetable adjustment, rescheduling or rerouting of railway traffic in case of disturbances. These delay reasons can be used to predict the lengths of railway disruptions and effective reactions can be applied in disruption management. In this work, a delay root cause discovery model is proposed, which integrates heterogeneous railway operation data sources to reconstruct the details of the railway operations. A supervised decision tree method following the machine learning and data mining techniques is designed to estimate the key factors in knock-on delays. It discovers the root cause delay factor by logically analyzing the scheduled or un-scheduled trains meetings and overtaking behaviors, and the subsequent delay propagations. Experiment results show that the proposed decision tree can predict the delay reason with the accuracy of 83%, and it can be further enhance to 90% if the delay cause is only considered “prolonged passengers boarding” and “meeting or overtaking” factors. The delay root cause can be discovered by the proposed model, verified by frequency filtering in operation records, and resolved by the adjustment of timetable which is an important reference for the next timetable rescheduling. The results of this study can be applied to railway operation decision support and disruption management, especially with regard to timetable rescheduling, trains resequencing or rerouting, system reliability analysis, and service quality improvements.