skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Enhancing HVAC energy management through multi-zone occupant-centric approach: A multi-agent deep reinforcement learning solution
Title: Enhancing HVAC energy management through multi-zone occupant-centric approach: A multi-agent deep reinforcement learning solution
Occupant-centric HVAC control places a premium on factors including thermal comfort and electricity cost to guarantee occupant satisfaction. Traditional approaches, reliant on static models for occupant behaviors, fall short in capturing intra-day behavioral variations, resulting in imprecise thermal comfort evaluations and suboptimal HVAC energy management, especially in multi-zone systems with diverse occupant profiles. To address this issue, this paper proposes a novel occupant-centric multi-zone HVAC control approach that intelligently schedules cooling and heating setpoints using Multi-agent Deep Reinforcement Learning (MADRL). This approach systematically takes into account stochastic occupant behavior models, such as dynamic clothing insulation adjustments, metabolic rates, and occupancy patterns. Simulation results demonstrate the efficacy of the proposed approach. Comparative case studies show that the proposed MADRL-based, occupant-centric HVAC control reduces electricity costs by 51.09% compared to rule-based approaches and 4.34% compared to single-agent DRL while maintaining multi-zonal thermal comfort for occupants.  more » « less
Award ID(s):
1856084
PAR ID:
10611193
Author(s) / Creator(s):
; ;
Publisher / Repository:
Elsevier
Date Published:
Journal Name:
Energy and Buildings
Volume:
303
Issue:
C
ISSN:
0378-7788
Page Range / eLocation ID:
113770
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Symposium on Simulation in Architecture and Urban Design (SIMAUD 2020))
    Thermal comfort and energy efficiency are always the two most significant objectives in HVAC operations. However, for conventional HVAC systems, the pursuit of high energy efficiency may be at the expense of satisfactory thermal comfort. Therefore, even if centralized HVAC systems nowadays have higher energy efficiency than before in office buildings, most of them cannot adapt the dynamic occupant behaviors or individual thermal comfort. In order to realize high energy efficiency while still maintain satisfactory thermal environment for occupants indoors, the integrated hybrid HVAC system has been developed for years such as task-ambient conditioning system. Moreover, the occupant-based HVAC control system such as human- in-the-loop has also been investigated so that the system can be adaptive based on occupant behaviors. However, most of research related to personalized air-conditioning system only focuses on field-study with limited scale (i.e. only one office room), this paper has proposed a co- simulation model in energyplus to simulate the hybrid cooling system with synthetic thermal comfort distributions based on global comfort database I&II. An optimization framework on cooling set-point is proposed with the objective of energy performance and the constraints of thermal comfort distribution developed by unsupervised Gaussian mixture model (GMM) clustering and kernel density estimation (KDE). The co-simulation results have illustrated that with the proposed optimization algorithm and the hybrid cooling system, HVAC demand power has decreased 5.3% on average with at least 90% of occupants feeling satisfied. 
    more » « less
  2. ; ; ; ; ; (, 2022 IEEE Power & Energy Society General Meeting (PESGM))
    This paper proposes a home energy management system (HEMS) while considering the residential occupant’s clothing integrated thermal comfort and electrical vehicles (EV) state-of-charge (SOC) concern. An adaptive dynamic program- ming (ADP) based HEMS model is proposed to optimally determine the setpoints of heating, ventilation, air conditioning (HVAC), the donning/doffing decisions for the clothing conditions and charging/discharging of EV while taking into account the uncertainties in outside temperature and EV arrival SOC. We use model predictive control (MPC) to simulate a multi-day energy management of a residential house equipped with the proposed HEMS. The proposed HEMS is compared with a baseline case without the HEMS. The simulation results show that a 47.5% of energy cost saving can be achieved by the proposed HEMS while maintaining satisfactory occupant thermal comfort and negligible EV SOC concerns. 
    more » « less
  3. ; ; (, Sustainability)
    Rooftop photovoltaics (PV) and electrical vehicles (EV) have become more economically viable to residential customers. Most existing home energy management systems (HEMS) only focus on the residential occupants’ thermal comfort in terms of indoor temperature and humidity while neglecting their other behaviors or concerns. This paper aims to integrate residential PV and EVs into the HEMS in an occupant-centric manner while taking into account the occupants’ thermal comfort, clothing behaviors, and concerns on the state-of-charge (SOC) of EVs. A stochastic adaptive dynamic programming (ADP) model was proposed to optimally determine the setpoints of heating, ventilation, air conditioning (HVAC), occupant’s clothing decisions, and the EV’s charge/discharge schedule while considering uncertainties in the outside temperature, PV generation, and EV’s arrival SOC. The nonlinear and nonconvex thermal comfort model, EV SOC concern model, and clothing behavior model were holistically embedded in the ADP-HEMS model. A model predictive control framework was further proposed to simulate a residential house under the time of use tariff, such that it continually updates with optimal appliance schedules decisions passed to the house model. Cosimulations were carried out to compare the proposed HEMS with a baseline model that represents the current operational practice. The result shows that the proposed HEMS can reduce the energy cost by 68.5% while retaining the most comfortable thermal level and negligible EV SOC concerns considering the occupant’s behaviors. 
    more » « less
  4. ; ; ; (, Building and Environment)
    The model of personalized thermal comfort can be learned via various machine learning algorithms and used to improve the individuals’ thermal comfort levels with potentially less energy consumption of HVAC systems. However, the learning of such a model typically requires a substantial number of thermal votes from the considered occupant, and the environmental conditions needed for collecting some votes may be undesired by the occupant in order to obtain a model with good generalization ability. In this paper, we propose to use a meta-learning algorithm to reduce the required number of personalized thermal votes so that a personalized thermal comfort model can be obtained with only a small number of feedback. With the learned meta-model, we derive a method based on the backpropagation of neural networks to quickly identify the best environmental and personal conditions for a specific occupant. The proposed identification algorithm has an additional advantage that the thermal comfort, indicated by the mean thermal sensation value, improves incrementally during the data collection process. We use the ASHRAE global thermal comfort database II to verify that the meta-learning algorithm can achieve an improved prediction accuracy after using 5 thermal sensation votes from an occupant to make adaptations. In addition, we show the effectiveness of the fast identification algorithm for the best personalized thermal environmental conditions with a thermal sensation generation model built from the PMV model. 
    more » « less
  5. ; ; (, IEEE Transactions on Sustainable Computing)
    Reinforcement learning (RL) methods can be used to develop a controller for the heating, ventilation, and air conditioning (HVAC) systems that both saves energy and ensures high occupants’ thermal comfort levels. However, the existing works typically require on-policy data to train an RL agent, and the occupants’ personalized thermal preferences are not considered, which is limited in the real-world scenarios. This paper designs a high-performance model-based offline RL algorithm for personalized HVAC systems. The proposed algorithm can quickly adapt to different occupants’ thermal preferences with a few thermal feedbacks, guaranteeing the high occupants’ personalized thermal comfort levels efficiently. First, we use a meta-supervised learning algorithm to train an occupant's thermal preference model. Then, we train an ensemble neural network to predict the thermal states of the considered zone. In addition, the obtained ensemble networks can indicate the regions in the state and action spaces covered by the offline dataset. With the personalized thermal preference model updated via meta-testing, model-based RL is used to derive the optimal HVAC controller. Since the proposed algorithm only requires offline datasets and a few online thermal feedbacks for training, it contributes to a more practical deployment of the RL algorithm to HVAC systems. We use the ASHRAE database II to verify the effectiveness and advantage of the meta-learning algorithm for modeling different occupants’ thermal preferences. Numerical simulations on the EnergyPlus environment demonstrate that the proposed algorithm can guarantee personalized thermal preferences with a slight increase of power consumption of 1.91% compared with the model-based RL algorithm with on-policy data aggregation. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email