skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Fast Human-in-the-Loop Control for HVAC Systems via Meta-Learning and Model-Based Offline Reinforcement Learning
Reinforcement learning (RL) methods can be used to develop a controller for the heating, ventilation, and air conditioning (HVAC) systems that both saves energy and ensures high occupants’ thermal comfort levels. However, the existing works typically require on-policy data to train an RL agent, and the occupants’ personalized thermal preferences are not considered, which is limited in the real-world scenarios. This paper designs a high-performance model-based offline RL algorithm for personalized HVAC systems. The proposed algorithm can quickly adapt to different occupants’ thermal preferences with a few thermal feedbacks, guaranteeing the high occupants’ personalized thermal comfort levels efficiently. First, we use a meta-supervised learning algorithm to train an occupant's thermal preference model. Then, we train an ensemble neural network to predict the thermal states of the considered zone. In addition, the obtained ensemble networks can indicate the regions in the state and action spaces covered by the offline dataset. With the personalized thermal preference model updated via meta-testing, model-based RL is used to derive the optimal HVAC controller. Since the proposed algorithm only requires offline datasets and a few online thermal feedbacks for training, it contributes to a more practical deployment of the RL algorithm to HVAC systems. We use the ASHRAE database II to verify the effectiveness and advantage of the meta-learning algorithm for modeling different occupants’ thermal preferences. Numerical simulations on the EnergyPlus environment demonstrate that the proposed algorithm can guarantee personalized thermal preferences with a slight increase of power consumption of 1.91% compared with the model-based RL algorithm with on-policy data aggregation.  more » « less
Award ID(s):
1837021
PAR ID:
10467763
Author(s) / Creator(s):
; ;
Publisher / Repository:
Institute of Electrical and Electronics Engineers (IEEE)
Date Published:
Journal Name:
IEEE Transactions on Sustainable Computing
Volume:
8
Issue:
3
ISSN:
2377-3790
Page Range / eLocation ID:
504 to 521
Subject(s) / Keyword(s):
HVAC Data models Computational modeling Prediction algorithms Atmospheric modeling Optimization Buildings
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Building and Environment)
    The model of personalized thermal comfort can be learned via various machine learning algorithms and used to improve the individuals’ thermal comfort levels with potentially less energy consumption of HVAC systems. However, the learning of such a model typically requires a substantial number of thermal votes from the considered occupant, and the environmental conditions needed for collecting some votes may be undesired by the occupant in order to obtain a model with good generalization ability. In this paper, we propose to use a meta-learning algorithm to reduce the required number of personalized thermal votes so that a personalized thermal comfort model can be obtained with only a small number of feedback. With the learned meta-model, we derive a method based on the backpropagation of neural networks to quickly identify the best environmental and personal conditions for a specific occupant. The proposed identification algorithm has an additional advantage that the thermal comfort, indicated by the mean thermal sensation value, improves incrementally during the data collection process. We use the ASHRAE global thermal comfort database II to verify that the meta-learning algorithm can achieve an improved prediction accuracy after using 5 thermal sensation votes from an occupant to make adaptations. In addition, we show the effectiveness of the fast identification algorithm for the best personalized thermal environmental conditions with a thermal sensation generation model built from the PMV model. 
    more » « less
  2. ; ; ; ; (, Symposium on Simulation in Architecture and Urban Design (SIMAUD 2020))
    Thermal comfort and energy efficiency are always the two most significant objectives in HVAC operations. However, for conventional HVAC systems, the pursuit of high energy efficiency may be at the expense of satisfactory thermal comfort. Therefore, even if centralized HVAC systems nowadays have higher energy efficiency than before in office buildings, most of them cannot adapt the dynamic occupant behaviors or individual thermal comfort. In order to realize high energy efficiency while still maintain satisfactory thermal environment for occupants indoors, the integrated hybrid HVAC system has been developed for years such as task-ambient conditioning system. Moreover, the occupant-based HVAC control system such as human- in-the-loop has also been investigated so that the system can be adaptive based on occupant behaviors. However, most of research related to personalized air-conditioning system only focuses on field-study with limited scale (i.e. only one office room), this paper has proposed a co- simulation model in energyplus to simulate the hybrid cooling system with synthetic thermal comfort distributions based on global comfort database I&II. An optimization framework on cooling set-point is proposed with the objective of energy performance and the constraints of thermal comfort distribution developed by unsupervised Gaussian mixture model (GMM) clustering and kernel density estimation (KDE). The co-simulation results have illustrated that with the proposed optimization algorithm and the hybrid cooling system, HVAC demand power has decreased 5.3% on average with at least 90% of occupants feeling satisfied. 
    more » « less
  3. ; ; ; ; ; ; (, IEEE)
    Advanced Driver Assistance Systems (ADAS) are increasingly important in improving driving safety and comfort, with Adaptive Cruise Control (ACC) being one of the most widely used. However, pre-defined ACC settings may not always align with driver's preferences and habits, leading to discomfort and potential safety issues. Personalized ACC (P-ACC) has been proposed to address this problem, but most existing research uses historical driving data to imitate behaviors that conform to driver preferences, neglecting real-time driver feedback. To bridge this gap, we propose a cloud-vehicle collaborative P-ACC framework that incorporates driver feedback adaptation in real time. The framework is divided into offline and online parts. The offline component records the driver's naturalistic car-following trajectory and uses inverse reinforcement learning (IRL) to train the model on the cloud. In the online component, driver feedback is used to update the driving gap preference in real time. The model is then retrained on the cloud with driver's takeover trajectories, achieving incremental learning to better match driver's preference. Human-in-the-loop (HuiL) simulation experiments demonstrate that our proposed method significantly reduces driver intervention in automatic control systems by up to 62.8%. By incorporating real-time driver feedback, our approach enhances the comfort and safety of P-ACC, providing a personalized and adaptable driving experience. 
    more » « less
  4. ; ; (, International Conference on. Computing in Civil Engineering (i3CE): Smart Cities, Sustainability, and Resilience)
    Understanding occupants’ thermal comfort is essential for the effective operation of Heating, Ventilation, and Air Conditioning (HVAC) systems. Existing studies of the “human-in-the-loop” HVAC control generally suffer from: (1) excessive reliance on cumbersome human feedback; and (2) intrusiveness caused by conventional data collection methods. To address these limitations, this paper investigates the low-cost thermal camera as a non-intrusive approach to assess thermal comfort in real time using facial skin temperature. The framework developed can automatically detect occupants, extract facial regions, measure skin temperature, and interpret thermal comfort with minimal interruption or participation of occupants. The framework is validated using the facial skin temperature collected from twelve occupants. Personal comfort models trained from different machine learning algorithms are compared and results show that Random Forest model can achieve an accuracy of 85% and also suggest that the skin temperature of ears, nose, and cheeks are most indicative of thermal comfort. 
    more » « less
  5. ; ; (, Proceedings of the 2021 Winter Simulation Conference)
    S. Kim, B. Feng (Ed.)
    The thermal comfort of individuals is considered an important factor that affects the health, well-being, and productivity of the occupants. However, only a small proportion of people are satisfied with the thermal environment of their current workplace. Therefore, this paper proposes a novel framework to simulate and optimize thermal comfort by controlling room conditions and matching them with occupants. The method is developed based on personalized thermal comfort prediction models and the Large Neighborhood Search (LNS) algorithm. To illustrate and validate the algorithm, a case study is provided. The results compare the thermal comfort of the occupants before and after the optimization and show a significant improvement in the thermal comfort. The proposed simulation method is proven to be feasible and efficient in providing an optimal match of occupants and rooms with specific settings, and therefore, can be of great value for the decision-making of the building management. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email