Mining missing train logs from Smart Card data

This paper shows how to recover the arrival times of trains from the gate times of metro passengers from Smart Card data. Such technique is essential when a log, the set of records indicating the actual arrival and departure time of each bus or train at each station and also a critical component in reliability analysis of a transportation system, is missing partially or entirely. The procedure reconstructs each train as a sequence of the earliest exit times, called S-epochs, among its alighting passengers at each stations. The procedure first constructs a set of passengers, also known as reference passengers, whose routing choices are easily identifiable. The procedure then computes, from the exit times of the reference passengers, a set of tentative S-epochs based on a detection measure whose validity relies on an extreme-value characteristic of the platform-to-gate movement of alighting passengers. The tentative S-epochs are then finalized to be a true one, or rejected, based on their consistencies with bounds and/or interpolation from prescribed S-epochs of adjacent trains and stations. Tested on 12 daily sets of trains, with varying degrees of missing logs, from three entire metro lines, the method restored the arrival times of 95% of trains within the error of 24 s even when 100% of logs was missing. The mining procedure can also be applied to trains operating under special strategies such as short-turning and skip-stop. The recovered log seems precise enough for the current reliability analysis performed by the city of Seoul.