skip to main content

Explore Research Products in the NSF-PAR

This content will become publicly available on October 31, 2024

Title: Reinforcement Learning Based Fast Charging of Electric Vehicle Battery Packs

Range anxiety and lack of adequate access to fast charging are proving to be important impediments to electric vehicle (EV) adoption. While many techniques to fast charging EV batteries (model-based & model-free) have been developed, they have focused on a single Lithium-ion cell. Extensions to battery packs are scarce, often considering simplified architectures (e.g., series-connected) for ease of modeling. Computational considerations have also restricted fast-charging simulations to small battery packs, e.g., four cells (for both series and parallel connected cells). Hence, in this paper, we pursue a model-free approach based on reinforcement learning (RL) to fast charge a large battery pack (comprising 444 cells). Each cell is characterized by an equivalent circuit model coupled with a second-order lumped thermal model to simulate the battery behavior. After training the underlying RL, the developed model will be straightforward to implement with low computational complexity. In detail, we utilize a Proximal Policy Optimization (PPO) deep RL as the training algorithm. The RL is trained in such a way that the capacity loss due to fast charging is minimized. The pack’s highest cell surface temperature is considered an RL state, along with the pack’s state of charge. Finally, in a detailed case study, the results are compared with the constant current-constant voltage (CC-CV) approach, and the outperformance of the RL-based approach is demonstrated. Our proposed PPO model charges the battery as fast as a CC-CV with a 5C constant stage while maintaining the temperature as low as a CC-CV with a 4C constant stage.

  more » « less
Award ID(s):
2312196
NSF-PAR ID:
10538262
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Society of Automotive Engineers International
Date Published:
Format(s):
Medium: X
Location:
Greenville, South Carolina, United States
Sponsoring Org:
National Science Foundation
More Like this
  1. ( , 2021 Modeling, Estimation and Control Conference)
    null (Ed.)
    Fast charging of lithium-ion batteries is crucial to increase desirability for consumers and hence accelerate the adoption of electric vehicles. A major barrier to shorter charge times is the accelerated aging of the battery at higher charging rates, which can be driven by lithium plating, increased solid electrolyte interphase growth due to elevated temperatures, and particle cracking due to mechanical stress. Lithium plating depends on the overpotential of the negative electrode, and mechanical stress depends on the concentration gradient, both of which cannot be measured directly. Techniques based on physics-based models of the battery and optimal control algorithms have been developed to this end. While these methods show promise in reducing degradation, their optimization algorithms' complexity can limit their implementation. In this paper, we present a method based on the constant current constant voltage (CC-CV) charging scheme, called CC-CVησT (VEST). The new approach is simpler to implement and can be used with any model to impose varying levels of constraints on variables pertinent to degradation, such as plating potential and mechanical stress. We demonstrate the new CC-CVησT charging using an electrochemical model with mechanical and thermal effects included. Furthermore, we discuss how uncertainties can be accounted for by considering safety margins for the plating and stress constraints. 
    more » « less
  2. ; ( , Journal of energy storage)
    Uwe Sauer, Dirk (Ed.)
    ABSTRACT State of Charge (SoC) and discharge capacity of the batteries are parameters that cannot be determined directly from the battery monitoring and control system and requires estimation. Current and voltage sensors have inherent error and delay leading to inaccurate measurements leading to inaccurate SoC and discharge capacity estimations. These sensors also have an additional cost to the battery system. This paper proposes a sensorless approach to estimate parameters of Vanadium Redox Flow Batteries (VRFBs) for both CC and CV charging methods by estimating battery current in CV mode and terminal voltage in CC mode. The results of estimations by the sensorless approach show a maximum relative error of 0.0035 in estimating terminal voltage in CC charging and a maximum relative error of 0.045 in estimating charging current in CV mode. Furthermore, long- term operation of vanadium redox flow batteries causes ion diffusions across the membrane and the depletion of active materials, which leads to capacity fading in VRFBs and inaccurate SoC estimation. To address the inaccuracy of SoC estimation in the long-term use of the battery, the capacity fading model is also considered for VRFBs in this paper. Experimental results show a 19% electrolyte volume change in the positive and negative tanks after 200 cycles of charge/discharge due to the bulk electrolyte transfer between the positive and negative sides of the battery system. This change of electrolyte volume results in 13.73% capacity fading after 200 cycles of charging/discharging. The SoC also changes by 7.1% after 200 cycles, due to the capacity and electrolyte volume loss, which shows the necessity of considering capacity fading in long-term use of the battery. 
    more » « less
  3. ; ; ; ( , International Conference on Renewable Energy and APplications)
    In this paper, a high efficiency, decoupled on-board battery charger is proposed. The proposed topology is composed of two LLC resonant converters sharing the same full-bridge inverter with constant switching frequency. The outputs of two LLC resonant converters are connected in series. One of the LLC resonant converters is operated at the resonant frequency, which is the highest efficiency operation point. The magnetic control is adopted for the second LLC resonant converter to fulfill the closed-loop control of charge voltage and current for constant voltage (CV) and constant current (CC) charge modes. The proposed topology can achieve zero voltage switching (ZVS) for all primary switches and zero current switching (ZCS) for all secondary diodes during both CC and CV modes. Furthermore, thanks to constant frequency operation, the electromagnetic interference (EMI) filter design is simplified. Simulation and experimental studies are presented to verify the feasibility and validity of the proposed topology. 
    more » « less
  4. ; ; ( , IEEE Transactions on Sustainable Energy)
    This paper proposes a methodology to increase the lifetime of the central battery energy storage system (CBESS) in an islanded building-level DC microgrid (MG) and enhance the voltage quality of the system by employing the supercapacitor (SC) of electric vehicles (EVs) that utilize battery-SC hybrid energy storage systems. To this end, an adaptive filtration-based (FB) current-sharing strategy is proposed in the voltage feedback control loop of the MG that smooths the CBESS current to increase its lifetime by allocating a portion of the high-frequency current variations to the EV charger. The bandwidth of this filter is adjusted using a data-driven algorithm to guarantee that only the EV's SC absorbs the high-frequency current variations, thereby enabling the EV's battery energy storage system (BESS) to follow its standard constant current-constant voltage (CC-CV) charging profile. Therefore, the EV's SC can coordinate with the CBESS without impacting the charging profile of the EV's BESS. Also, a small-signal stability analysis is provided indicating that the proposed approach improves the marginal voltage stability of the DC MG leading to better transient response and higher voltage quality. Finally, the performance of the proposed EV charging is validated using MATLAB/Simulink and hardware-in-the-loop (HIL) testing. 
    more » « less
  5. ; ; ; ( , Biotechnology and Bioengineering)
    Abstract

    Reinforcement learning (RL), a subset of machine learning (ML), could optimize and control biomanufacturing processes, such as improved production of therapeutic cells. Here, the process of CAR T‐cell activation by antigen‐presenting beads and their subsequent expansion is formulated in silico. The simulation is used as an environment to train RL‐agents to dynamically control the number of beads in culture to maximize the population of robust effector cells at the end of the culture. We make periodic decisions of incremental bead addition or complete removal. The simulation is designed to operate in OpenAI Gym, enabling testing of different environments, cell types, RL‐agent algorithms, and state inputs to the RL‐agent. RL‐agent training is demonstrated with three different algorithms (PPO, A2C, and DQN), each sampling three different state input types (tabular, image, mixed); PPO‐tabular performs best for this simulation environment. Using this approach, training of the RL‐agent on different cell types is demonstrated, resulting in unique control strategies for each type. Sensitivity to input‐noise (sensor performance), number of control step interventions, and advantages of pre‐trained RL‐agents are also evaluated. Therefore, we present an RL framework to maximize the population of robust effector cells in CAR T‐cell therapy production.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email