An official website of the United States government
Hybrid electric vehicles employ a hybrid propulsion system to combine the energy efficiency of electric motor and a long driving range of internal combustion engine, thereby achieving a higher fuel economy as well as convenience compared with conventional ICE vehicles. However, the relatively complicated powertrain structures of HEVs necessitate an effective power management policy to determine the power split between ICE and EM. In this work, we propose a deep reinforcement learning framework of the HEV power management with the aim of improving fuel economy. The DRL technique is comprised of an offline deep neural network construction phase and an online deep Q-learning phase. Unlike traditional reinforcement learning, DRL presents the capability of handling the high dimensional state and action space in the actual decision-making process, making it suitable for the HEV power management problem. Enabled by the DRL technique, the derived HEV power management policy is close to optimal, fully model-free, and independent of a prior knowledge of driving cycles. Simulation results based on actual vehicle setup over real-world and testing driving cycles demonstrate the effectiveness of the proposed framework on optimizing HEV fuel economy.
more »
« less
Control and Powertrain Management for Multi-Autonomous Hybrid Vehicles
The need for less fuel consumption and the trend of higher level of autonomy together urge the power optimization in multihybrid autonomous vehicles. Both the multivehicle coordination control and the hybrid powertrain energy management should be optimized to maximize fuel savings. In this paper, we intend to have a computationally efficient framework to optimize them individually and then evaluate the overall control performance. The optimization is conducted in series. First is at the multivehicle system's level where the distributed locally optimal solution is given for vehicles with nonlinear dynamics. Second, the powertrain management optimization is conducted at the hybrid powertrain level. We provide an analytical formulation of the powertrain optimization for each hybrid vehicle by using Pontryagin's minimum principle (PMP). By approximating the optimal instantaneous fuel consumption rate as a polynomial of the engine speed, we can formulate the optimization problem into a set of algebraic equations, which enables the computationally efficient real-time implementation. To justify the applicability of the methodology in real-time, we give directions on numerical iterative solutions for these algebraic equations. The analysis on the stability of the method is shown through statistical analysis. Finally, further simulations are given to confirm the efficacy and the robustness of the proposed optimal approach. An off-road example is given in the simulation, although the framework developed can be applied to on-road scenario as well.
more »
« less
- Award ID(s):
- 1826410
- PAR ID:
- 10126748
- Date Published:
- Journal Name:
- Journal of Dynamic Systems, Measurement, and Control
- Volume:
- 141
- Issue:
- 7
- ISSN:
- 0022-0434
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Hybrid electric vehicles can achieve better fuel economy than conventional vehicles by utilizing multiple power sources. While these power sources have been controlled by rule-based or optimization-based control algorithms, recent studies have shown that machine learning-based control algorithms such as online Deep Reinforcement Learning (DRL) can effectively control the power sources as well. However, the optimization and training processes for the online DRL-based powertrain control strategy can be very time and resource intensive. In this paper, a new offline–online hybrid DRL strategy is presented where offline vehicle data are exploited to build an initial model and an online learning algorithm explores a new control policy to further improve the fuel economy. In this manner, it is expected that the agent can learn an environment consisting of the vehicle dynamics in a given driving condition more quickly compared to the online algorithms, which learn the optimal control policy by interacting with the vehicle model from zero initial knowledge. By incorporating a priori offline knowledge, the simulation results show that the proposed approach not only accelerates the learning process and makes the learning process more stable, but also leads to a better fuel economy compared to online only learning algorithms.more » « less
-
Eco-driving has garnered considerable research attention owing to its potential socio-economic impact, including enhanced public health and mitigated climate change effects through the reduction of greenhouse gas emissions. With an expectation of more autonomous vehicles (AVs) on the road, an eco-driving strategy in hybrid traffic networks encompassing AV and human-driven vehicles (HDVs) with the coordination of traffic lights is a challenging task. The challenge is partially due to the insufficient infrastructure for collecting, transmitting, and sharing real-time traffic data among vehicles, facilities, and traffic control centers, and the following decision-making of agents involved in traffic control. Additionally, the intricate nature of the existing traffic network, with its diverse array of vehicles and facilities, contributes to the challenge by hindering the development of a mathematical model for accurately characterizing the traffic network. In this study, we utilized the Simulation of Urban Mobility (SUMO) simulator to tackle the first challenge through computational analysis. To address the second challenge, we employed a model-free reinforcement learning (RL) algorithm, proximal policy optimization, to decide the actions of AV and traffic light signals in a traffic network. A novel eco-driving strategy was proposed by introducing different percentages of AV into the traffic flow and collaborating with traffic light signals using RL to control the overall speed of the vehicles, resulting in improved fuel consumption efficiency. Average rewards with different penetration rates of AV (5%, 10%, and 20% of total vehicles) were compared to the situation without any AV in the traffic flow (0% penetration rate). The 10% penetration rate of AV showed a minimum time of convergence to achieve average reward, leading to a significant reduction in fuel consumption and total delay of all vehicles.more » « less
-
null (Ed.)For energy-efficient Connected and Automated Vehicle (CAV) Eco-driving control on signalized arterials under uncertain traffic conditions, this paper explicitly considers traffic control devices (e.g., road markings, traffic signs, and traffic signals) and road geometry (e.g., road shapes, road boundaries, and road grades) constraints in a data-driven optimization-based Model Predictive Control (MPC) modeling framework. This modeling framework uses real-time vehicle driving and traffic signal data via Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communications. In the MPC-based control model, this paper mathematically formulates location-based traffic control devices and road geometry constraints using the geographic information from High-Definition (HD) maps. The location-based traffic control devices and road geometry constraints have the potential to improve the safety, energy, efficiency, driving comfort, and robustness of connected and automated driving on real roads by considering interrupted flow facility locations and road geometry in the formulation. We predict a set of uncertain driving states for the preceding vehicles through an online learning-based driving dynamics prediction model. We then solve a constrained finite-horizon optimal control problem with the predicted driving states to obtain a set of Eco-driving references for the controlled vehicle. To obtain the optimal acceleration or deceleration commands for the controlled vehicle with the set of Eco-driving references, we formulate a Distributionally Robust Stochastic Optimization (DRSO) model (i.e., a special case of data-driven optimization models under moment bounds) with Distributionally Robust Chance Constraints (DRCC) with location-based traffic control devices and road geometry constraints. We design experiments to demonstrate the proposed model under different traffic conditions using real-world connected vehicle trajectory data and Signal Phasing and Timing (SPaT) data on a coordinated arterial with six actuated intersections on Fuller Road in Ann Arbor, Michigan from the Safety Pilot Model Deployment (SPMD) project.more » « less
-
In the urban corridor with a mixed traffic composition of connected and automated vehicles (CAVs) alongside human-driven vehicles (HDVs), vehicle operations are intricately influenced by both individual driving behaviors and the presence of signalized intersections. Therefore, the development of a coordinated control strategy that effectively accommodates these dual factors becomes imperative to enhance the overall quality of traffic flow. This study proposes a bi-level structure crafted to decouple the joint effects of the vehicular driving behaviors and corridor signal offsets setting. The objective of this structure is to optimize both the average travel time (ATT) and fuel consumption (AFC). At the lower-level, three types of car-following models while considering driving modes are presented to illustrate the desired driving behaviors of HDVs and CAVs. Moreover, a trigonometry function method combined with a rolling horizon scheme is proposed to generate the eco-trajectory of CAVs in the mixed traffic flow. At the upper-level, a multi-objective optimization model for corridor signal offsets is formulated to minimize ATT and AFC based on the lower-level simulation outputs. Additionally, a revised Non-Dominated Sorting Genetic Algorithm II (NSGA-II) is adopted to identify the set of Pareto-optimal solutions for corridor signal offsets under different CAV penetration rates (CAV PRs). Numerical experiments are conducted within a corridor that encompasses three signalized intersections. The performance of our proposed eco-driving strategy is validated in comparison to the intelligent driver model (IDM) and green light optimal speed advisory (GLOSA) algorithm in single-vehicle simulation. Results show that our proposed strategy yields reduced travel time and fuel consumption to both IDM and GLOSA. Subsequently, the effectiveness of our proposed coordinated control strategy is validated across various CAV PRs. Results indicated that the optimal AFC can be reduced by 4.1%–32.2% with CAV PRs varying from 0.2 to 1, and the optimal ATT can be saved by 2.3% maximum. Furthermore, sensitivity analysis is conducted to evaluate the impact of CAV PRs and V/C ratios on the optimal ATT and AFC.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1115/1.4043110
