Direct generation of level of service maps from images using convolutional and long short-term memory networks

Congestion in transport stations could result in stampede development and deadly crush situations. Closed circuit television (CCTV) cameras enable station managers to monitor the crowd and reduce overcrowding risks. However, identifying congestion conditions is a very laborious task for a human operator who has to monitor multiple locations at the same time. This paper presents a new approach to automated image-based identification of congestion as measured by level of service (LOS), which is the most widely accepted standard for measuring congestion. Existing methods for measuring LOS based on crowd density estimation from images have the disadvantages that, crowd density cannot be estimated accurately. In addition, the calculation of flow parameters involves a complex process, and consequently these parameters are not indicative of congestion in real-time. This paper proposes a novel method based on machine learning to directly classify LOS without calculating flow parameters. In the proposed method, visual features extracted by a deep convolutional neural network are classified using a support vector machine classifier and the classification results are further refined by using a long short-term memory network. A second contribution of this research is to develop a web-based LOS map visualization platform to monitor pedestrian distribution and variation of distribution in real-time. Experimental evaluation at Flinders Street Station in Melbourne shows that this method can achieve an accuracy of 81.9% and efficiency of 0.40?seconds per frame in LOS classification using CCTV images.