skip to main content


Title: A Deep Reinforcement Learning Based Approach for Home Energy Management System
Home energy management system (HEMS) enables residents to actively participate in demand response (DR) programs. It can autonomously optimize the electricity usage of home appliances to reduce the electricity cost based on time-varying electricity prices. However, due to the existence of randomness in the pricing process of the utility and resident's activities, developing an efficient HEMS is challenging. To address this issue, we propose a novel home energy management method for optimal scheduling of different kinds of home appliances based on deep reinforcement learning (DRL). Specifically, we formulate the home energy management problem as an MDP considering the randomness of real-time electricity prices and resident's activities. A DRL approach based on proximal policy optimization (PPO) is developed to determine the optimal DR scheduling strategy. The proposed approach does not need any information on the appliances' models and distribution knowledge of the randomness. Simulation results verify the effectiveness of our proposed approach.  more » « less
Award ID(s):
1917275
NSF-PAR ID:
10158893
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT 2020)
Page Range / eLocation ID:
1 to 5
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Smart appliances’ run schedule and electric vehicles charging can be managed over a smart grid enabled home area network (HAN) to reduce electricity demand at critical times and add more plug-in electric vehicles to the grid, which eventually lower customers’ energy bills and reduce greenhouse gas emissions. Short range radio-based wireless communication technologies commonly adopted in a HAN are vulnerable to cyber attacks due to their wide interception range. In this work, a low-cost solution is proposed for securing the low-volume data exchange of sensitive tasks (e.g., key management and mutual authentication). Our approach utilizes the emerging concept of retro-reflector based visible light communication (Retro-VLC), where smart appliances, IoT sensors and other electric devices perform the sensitive data exchange with the HAN gateway via the secure Retro-VLC channel. To conduct the feasibility study, a multi-pixel Retro-VLC link is prototyped to enable quadrature amplitude modulation. The bit error rate of Retro-VLC is studied analytically, numerically and experimentally. A heterogeneous Retro-VLC + WLAN connection is implemented by socket programming. In addition, the working range, sniffing range, and key exchange latency are measured. The results validate the applicability of the Retro-VLC based solution. 
    more » « less
  2. Abstract

    Recent increases in renewable power generation challenge the operation of the power grid: generation rates fluctuate in time and are not synchronized with power demand fluctuations. Demand response (DR) consists of adjusting user electricity demand to match available power supply. Chemical plants are appealing candidates for DR programs; they offer large, concentrated loads that can be modulated via production scheduling. Price‐based DR is a common means of engaging industrial entities; its benefits increase significantly when a longer (typically, a few days) scheduling time horizon is considered. DR production scheduling comes with its own challenges, related to uncertainty in future (i.e., forecast) electricity prices and product demand. In this work, we provide a framework for DR production scheduling under uncertainty based on a chance‐constrained formulation that also accounts for the dynamics of the production facility. The ideas are illustrated with an air separation unit case study.

     
    more » « less
  3. Rooftop photovoltaics (PV) and electrical vehicles (EV) have become more economically viable to residential customers. Most existing home energy management systems (HEMS) only focus on the residential occupants’ thermal comfort in terms of indoor temperature and humidity while neglecting their other behaviors or concerns. This paper aims to integrate residential PV and EVs into the HEMS in an occupant-centric manner while taking into account the occupants’ thermal comfort, clothing behaviors, and concerns on the state-of-charge (SOC) of EVs. A stochastic adaptive dynamic programming (ADP) model was proposed to optimally determine the setpoints of heating, ventilation, air conditioning (HVAC), occupant’s clothing decisions, and the EV’s charge/discharge schedule while considering uncertainties in the outside temperature, PV generation, and EV’s arrival SOC. The nonlinear and nonconvex thermal comfort model, EV SOC concern model, and clothing behavior model were holistically embedded in the ADP-HEMS model. A model predictive control framework was further proposed to simulate a residential house under the time of use tariff, such that it continually updates with optimal appliance schedules decisions passed to the house model. Cosimulations were carried out to compare the proposed HEMS with a baseline model that represents the current operational practice. The result shows that the proposed HEMS can reduce the energy cost by 68.5% while retaining the most comfortable thermal level and negligible EV SOC concerns considering the occupant’s behaviors. 
    more » « less
  4. Fine-grained monitoring of everyday appliances can provide better feedback to the consumers and motivate them to change behavior in order to reduce their energy usage. It also helps to detect abnormal power consumption events, long-term appliance malfunctions and potential safety concerns. Commercially available plug meters can be used for individual appliance monitoring but for an entire house, each such individual plug meters are expensive and tedious to setup. Alternative methods relying on Non-Intrusive Load Monitoring techniques help disaggregate electricity consumption data and learn about the individual appliance's power states and signatures. However fine-grained events (e.g., appliance malfunctions, abnormal power consumption, etc.) remain undetected and thus inferred contexts (such as safety hazards etc.) become invisible. In this work, we correlate an appliance's inherent acoustic noise with its energy consumption pattern individually and in presence of multiple appliances. We initially investigate classification techniques to establish the relationship between appliance power and acoustic states for efficient energy disaggregation and abnormal events detection. While promising, this approach fails when there are multiple appliances simultaneously in `ON' state. To further improve the accuracy of our energy disaggregation algorithm, we propose a probabilistic graphical model, based on a variation of Factorial Hidden Markov Model (FHMM) for multiple appliances energy disaggregation. We combine our probabilistic model with the appliances acoustic analytics and postulate a hybrid model for energy disaggregation. Our approach helps to improve the performance of energy disaggregation algorithms and provide critical insights on appliance longevity, abnormal power consumption, consumer behavior and their everyday lifestyle activities. We evaluate the performance of our proposed algorithms on real data traces and show that the fusion of acoustic and power signatures can successfully detect a number of appliances with 95% accuracy. 
    more » « less
  5. This paper proposes a home energy management system (HEMS) while considering the residential occupant’s clothing integrated thermal comfort and electrical vehicles (EV) state-of-charge (SOC) concern. An adaptive dynamic program- ming (ADP) based HEMS model is proposed to optimally determine the setpoints of heating, ventilation, air conditioning (HVAC), the donning/doffing decisions for the clothing conditions and charging/discharging of EV while taking into account the uncertainties in outside temperature and EV arrival SOC. We use model predictive control (MPC) to simulate a multi-day energy management of a residential house equipped with the proposed HEMS. The proposed HEMS is compared with a baseline case without the HEMS. The simulation results show that a 47.5% of energy cost saving can be achieved by the proposed HEMS while maintaining satisfactory occupant thermal comfort and negligible EV SOC concerns. 
    more » « less