More Like this

1. Enhancing HVAC energy management through multi-zone occupant-centric approach: A multi-agent deep reinforcement learning solution

   https://doi.org/10.1016/j.enbuild.2023.113770  

   Liu, Xuebo; Wu, Yingying; Wu, Hongyu (January 2024, Energy and Buildings)

   Occupant-centric HVAC control places a premium on factors including thermal comfort and electricity cost to guarantee occupant satisfaction. Traditional approaches, reliant on static models for occupant behaviors, fall short in capturing intra-day behavioral variations, resulting in imprecise thermal comfort evaluations and suboptimal HVAC energy management, especially in multi-zone systems with diverse occupant profiles. To address this issue, this paper proposes a novel occupant-centric multi-zone HVAC control approach that intelligently schedules cooling and heating setpoints using Multi-agent Deep Reinforcement Learning (MADRL). This approach systematically takes into account stochastic occupant behavior models, such as dynamic clothing insulation adjustments, metabolic rates, and occupancy patterns. Simulation results demonstrate the efficacy of the proposed approach. Comparative case studies show that the proposed MADRL-based, occupant-centric HVAC control reduces electricity costs by 51.09% compared to rule-based approaches and 4.34% compared to single-agent DRL while maintaining multi-zonal thermal comfort for occupants. 

   more » « less
2. Run-time accuracy reconfigurable stochastic computing for dynamic reliability and power management: work-in-progress

   Yu, S.; Zhou, H.; Amrouch, H.; Henkel, J.; Tan, S. X.-D. (October 2020, Proc. International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (CASES’20), ESWEEK 2020)

   In this paper, we propose a novel accuracy-reconfigurable stochastic computing (ARSC) framework for dynamic reliability and power management. Different than the existing stochastic computing works, where the accuracy versus power/energy trade-off is carried out in the design time, the new ARSC design can change accuracy or bit-width of the data in the run-time so that it can accommodate the long-term aging effects by slowing the system clock frequency at the cost of accuracy while maintaining the throughput of the computing. We validate the ARSC concept on a discrete cosine transformation (DCT) and inverse DCT designs for image compressing/decompressing applications, which are implemented on Xilinx Spartan-6 family XC6SLX45 platform. Experimental results show that the new design can easily mitigate the long-term aging-induced effects by accuracy trade-off while maintaining the throughput of the whole computing process using simple frequency scaling. We further show that one-bit precision loss for the input data, which translated to 3.44dB of the accuracy loss in term of Peak Signal to Noise Ratio (PSNR) for images, we can sufficiently compensate the NBTI induced aging effects in 10 years while maintaining the pre-aging computing throughput of 7.19 frames per second. At the same time, we can save 74\% power consumption by 10.67dB of accuracy loss. The proposed ARSC computing framework also allows much aggressive frequency scaling, which can lead to order of magnitude power savings compared to the traditional dynamic voltage and frequency scaling (DVFS) techniques. 

   more » « less
3. A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles

   https://doi.org/10.1109/ASPDAC.2018.8297305  

   Zhao, Pu; Wang, Yanzhi; Chang, Naehyuck; Zhu, Qi; Lin, Xue (January 2018, 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC))

   Hybrid electric vehicles employ a hybrid propulsion system to combine the energy efficiency of electric motor and a long driving range of internal combustion engine, thereby achieving a higher fuel economy as well as convenience compared with conventional ICE vehicles. However, the relatively complicated powertrain structures of HEVs necessitate an effective power management policy to determine the power split between ICE and EM. In this work, we propose a deep reinforcement learning framework of the HEV power management with the aim of improving fuel economy. The DRL technique is comprised of an offline deep neural network construction phase and an online deep Q-learning phase. Unlike traditional reinforcement learning, DRL presents the capability of handling the high dimensional state and action space in the actual decision-making process, making it suitable for the HEV power management problem. Enabled by the DRL technique, the derived HEV power management policy is close to optimal, fully model-free, and independent of a prior knowledge of driving cycles. Simulation results based on actual vehicle setup over real-world and testing driving cycles demonstrate the effectiveness of the proposed framework on optimizing HEV fuel economy. 

   more » « less
4. A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

   An, Qing; Segarra, Santiago; Dick, Chris; Sabharwal, Ashutosh; Doost-Mohammady, Rahman (September 2023, IEEE transactions on machine learning in communications and networking)

   The large number of antennas in massive MIMO systems allows the base station to communicate with multiple users at the same time and frequency resource with multi-user beamforming. However, highly correlated user channels could drastically impede the spectral efficiency that multi-user beamforming can achieve. As such, it is critical for the base station to schedule a suitable group of users in each time and frequency resource block to achieve maximum spectral efficiency while adhering to fairness constraints among the users. In this paper, we consider the resource scheduling problem for massive MIMO systems with its optimal solution known to be NP-hard. Inspired by recent achievements in deep reinforcement learning (DRL) to solve problems with large action sets, we propose \name{}, a dynamic scheduler for massive MIMO based on the state-of-the-art Soft Actor-Critic (SAC) DRL model and the K-Nearest Neighbors (KNN) algorithm. Through comprehensive simulations using realistic massive MIMO channel models as well as real-world datasets from channel measurement experiments, we demonstrate the effectiveness of our proposed model in various channel conditions. Our results show that our proposed model performs very close to the optimal proportionally fair (Opt-PF) scheduler in terms of spectral efficiency and fairness with more than one order of magnitude lower computational complexity in medium network sizes where Opt-PF is computationally feasible. Our results also show the feasibility and high performance of our proposed scheduler in networks with a large number of users and resource blocks. 

   more » « less
5. A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

   Qing An; Santiago Segarra; Chris Dick; Ashutosh Sabharwal; Rahman Doost-Mohammady (September 2023, IEEE transactions on machine learning in communications and networking)

   The large number of antennas in massive MIMO systems allows the base station to communicate with multiple users at the same time and frequency resource with multi-user beamforming. However, highly correlated user channels could drastically impede the spectral efficiency that multi-user beamforming can achieve. As such, it is critical for the base station to schedule a suitable group of users in each time and frequency resource block to achieve maximum spectral efficiency while adhering to fairness constraints among the users. In this paper, we consider the resource scheduling problem for massive MIMO systems with its optimal solution known to be NP-hard. Inspired by recent achievements in deep reinforcement learning (DRL) to solve problems with large action sets, we propose SMART, a dynamic scheduler for massive MIMO based on the state-of-the-art Soft Actor-Critic (SAC) DRL model and the K-Nearest Neighbors (KNN) algorithm. Through comprehensive simulations using realistic massive MIMO channel models as well as real-world datasets from channel measurement experiments, we demonstrate the effectiveness of our proposed model in various channel conditions. Our results show that our proposed model performs very close to the optimal proportionally fair (Opt-PF) scheduler in terms of spectral efficiency and fairness with more than one order of magnitude lower computational complexity in medium network sizes where Opt-PF is computationally feasible. Our results also show the feasibility and high performance of our proposed scheduler in networks with a large number of users and resource blocks. 

   more » « less