More Like this

1. A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles

   https://doi.org/10.1109/ASPDAC.2018.8297305  

   Zhao, Pu; Wang, Yanzhi; Chang, Naehyuck; Zhu, Qi; Lin, Xue (January 2018, 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC))

   Hybrid electric vehicles employ a hybrid propulsion system to combine the energy efficiency of electric motor and a long driving range of internal combustion engine, thereby achieving a higher fuel economy as well as convenience compared with conventional ICE vehicles. However, the relatively complicated powertrain structures of HEVs necessitate an effective power management policy to determine the power split between ICE and EM. In this work, we propose a deep reinforcement learning framework of the HEV power management with the aim of improving fuel economy. The DRL technique is comprised of an offline deep neural network construction phase and an online deep Q-learning phase. Unlike traditional reinforcement learning, DRL presents the capability of handling the high dimensional state and action space in the actual decision-making process, making it suitable for the HEV power management problem. Enabled by the DRL technique, the derived HEV power management policy is close to optimal, fully model-free, and independent of a prior knowledge of driving cycles. Simulation results based on actual vehicle setup over real-world and testing driving cycles demonstrate the effectiveness of the proposed framework on optimizing HEV fuel economy. 

   more » « less
2. PnP-DRL: A Plug-and-Play Deep Reinforcement Learning Approach for Experience-Driven Networking

   Xu, Zhiyuan; Wu, Kun; Zhang, Weiyi; Tang, Jian; Wang, Yanzhi; Xue, Guoliang (August 2021, IEEE journal on selected areas in communications)

   null (Ed.)

   While Deep Reinforcement Learning has emerged as a de facto approach to many complex experience-driven networking problems, it remains challenging to deploy DRL into real systems. Due to the random exploration or half-trained deep neural networks during the online training process, the DRL agent may make unexpected decisions, which may lead to system performance degradation or even system crash. In this paper, we propose PnP-DRL, an offline-trained, plug and play DRL solution, to leverage the batch reinforcement learning approach to learn the best control policy from pre-collected transition samples without interacting with the system. After being trained without interaction with systems, our Plug and Play DRL agent will start working seamlessly, without additional exploration or possible disruption of the running systems. We implement and evaluate our PnP-DRL solution on a prevalent experience-driven networking problem, Dynamic Adaptive Streaming over HTTP (DASH). Extensive experimental results manifest that 1) The existing batch reinforcement learning method has its limits; 2) Our approach PnP-DRL significantly outperforms classical adaptive bitrate algorithms in average user Quality of Experience (QoE); 3) PnP-DRL, unlike the state-of-the-art online DRL methods, can be off and running without learning gaps, while achieving comparable performances. 

   more » « less
3. Control and Powertrain Management for Multi-Autonomous Hybrid Vehicles

   https://doi.org/10.1115/1.4043110  

   Ghasemi, Masood; Song, Xingyong (July 2019, Journal of Dynamic Systems, Measurement, and Control)

   The need for less fuel consumption and the trend of higher level of autonomy together urge the power optimization in multihybrid autonomous vehicles. Both the multivehicle coordination control and the hybrid powertrain energy management should be optimized to maximize fuel savings. In this paper, we intend to have a computationally efficient framework to optimize them individually and then evaluate the overall control performance. The optimization is conducted in series. First is at the multivehicle system's level where the distributed locally optimal solution is given for vehicles with nonlinear dynamics. Second, the powertrain management optimization is conducted at the hybrid powertrain level. We provide an analytical formulation of the powertrain optimization for each hybrid vehicle by using Pontryagin's minimum principle (PMP). By approximating the optimal instantaneous fuel consumption rate as a polynomial of the engine speed, we can formulate the optimization problem into a set of algebraic equations, which enables the computationally efficient real-time implementation. To justify the applicability of the methodology in real-time, we give directions on numerical iterative solutions for these algebraic equations. The analysis on the stability of the method is shown through statistical analysis. Finally, further simulations are given to confirm the efficacy and the robustness of the proposed optimal approach. An off-road example is given in the simulation, although the framework developed can be applied to on-road scenario as well. 

   more » « less
4. Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles – A Deep Reinforcement Learning Approach

   https://doi.org/10.1109/MT-ITS49943.2021.9529289  

   Jiang, Liming; Xie, Yuanchang; Wen, Xiao; Chen, Danjue; Li, Tienan; Evans, Nicholas G. (January 2021, 2021 7th International Conference on Models and Technologies for Intelligent Transportation Systems)

   Stop-and-go traffic poses significant challenges to the efficiency and safety of traffic operations, and its impacts and working mechanism have attracted much attention. Recent studies have shown that Connected and Automated Vehicles (CAVs) with carefully designed longitudinal control have the potential to dampen the stop-and-go wave based on simulated vehicle trajectories. In this study, Deep Reinforcement Learning (DRL) is adopted to control the longitudinal behavior of CAVs and real-world vehicle trajectory data is utilized to train the DRL controller. It considers a Human-Driven (HD) vehicle tailed by a CAV, which are then followed by a platoon of HD vehicles. Such an experimental design is to test how the CAV can help to dampen the stop-and-go wave generated by the lead HD vehicle and contribute to smoothing the following HD vehicles’ speed profiles. The DRL control is trained using real-world vehicle trajectories, and eventually evaluated using SUMO simulation. The results show that the DRL control decreases the speed oscillation of the CAV by 54% and 8%-28% for those following HD vehicles. Significant fuel consumption savings are also observed. Additionally, the results suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically. 

   more » « less
5. Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles – A Deep Reinforcement Learning Approach

   Jiang, L (June 2021, Proceedings of the 7th International IEEE Conference on Models and Technologies for Intelligent Transportation Systems)

   null (Ed.)

   Stop-and-go traffic poses significant challenges to the efficiency and safety of traffic operations, and its impacts and working mechanism have attracted much attention. Recent studies have shown that Connected and Automated Vehicles (CAVs) with carefully designed longitudinal control have the potential to dampen the stop-and-go wave based on simulated vehicle trajectories. In this study, Deep Reinforcement Learning (DRL) is adopted to control the longitudinal behavior of CAVs and real-world vehicle trajectory data is utilized to train the DRL controller. It considers a Human-Driven (HD) vehicle tailed by a CAV, which are then followed by a platoon of HD vehicles. Such an experimental design is to test how the CAV can help to dampen the stop-and-go wave generated by the lead HD vehicle and contribute to smoothing the following HD vehicles’ speed profiles. The DRL control is trained using realworld vehicle trajectories, and eventually evaluated using SUMO simulation. The results show that the DRL control decreases the speed oscillation of the CAV by 54% and 8%-28% for those following HD vehicles. Significant fuel consumption savings are also observed. Additionally, the results suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically. 

   more » « less