一种基于策略梯度强化学习的列车智能控制方法

近年来,我国已初步建成巨大的城市轨道交通和高速铁路网络,逐步开始走向提升整体运营效率的新阶段。城市轨道交通系统的大规模和高密度运营,使得系统能耗急剧增长。现有的自动驾驶控制方法基于已有的模型,能够完成在正常场景下的自动驾驶。基于现有列车自动驾驶技术的控制原理和优秀司机的驾驶经验,提出一种列车智能控制方法,以减小列车的牵引能耗。首先,建立列车控制专家系统,能满足乘客舒适度要求;在此基础上,利用神经网络作为列车驾驶控制器,设计了一种基于策略的强化学习算法,优化神经网络的参数,以适应变化的运营场景。基于地铁现场运行数据仿真结果表明,该智能算法比现有算法具有更好地节能效果和准时性。

A Policy-Based Reinforcement Learning Algorithm for Intelligent Train Control

In recent years,China has built a huge urban rail and high-speed rail network and is moving toward a new stage of overall operational efficiency improvement.The large-scale and high-frequency operation of urban rail systems has led to a dramatic increase in energy consumption.The current ATO control strategy can deal with the automatic driving control in normal situations.To cut down the traction energy consumption of trains,this paper proposed an intelligent train control approach considering the automatic train operation(ATO)principle and the driving experience of drivers.The train control expert system which can meet the passenger riding comfort requirement was firstly concluded from the literature,standards,and driving experience.Based on the expert system,a neural network was used as the driving control actor to choose the coasting strategy for trains.Then,a policy-based reinforcement learning algorithm was presented to optimize the parameters of the neural network and the control strategy such that the controller can suit to different scenarios.Finally,a case study conducted based on the real-world data of Beijing Yizhuang line illustrates the effectiveness of the proposed approach.The simulation results indicate that compared with the current ATO control algorithm,the proposed algorithm has a better energy-efficient and punctual performance.