skip to main content


Title: Thermal and RGB-D Sensor Fusion for Non-Intrusive Human Thermal Comfort Assessment
It is well established that thermal comfort is an influential factor in human health and wellbeing. Uncomfortable thermal environments can reduce occupants’ comfort and productivity, and cause symptoms of sick building syndrome. To harness the built environment as a medium to support human health, well-being, and engagement, it is significantly important to understand occupants’ thermal comfort in real time. To this end, this study proposes a non-intrusive method to collect occupants’ facial skin temperature and interpret their thermal comfort conditions by fusing the thermal and RGB-D images collected from multiple low-cost thermographic and Kinect sensors. This study distinguishes from existing methods of thermal comfort assessment in three ways: 1) it is a truly non-intrusive data collection approach which has a minimal interruption or participation of building occupants; 2) the proposed approach can simultaneously identify and interpret multiple occupants’ thermal comfort; 3) it uses low-cost thermographic and RGB-D cameras which can be rapidly deployed and reconfigured to adapt to various settings. This approach was experimentally evaluated in a transient heating environment (room temperature increased from 23 to 27 °C) to verify its applicability in real operational built environments. In total, all 6 subjects observed moderate to strong positive correlations between the ambient room temperature and subjects’ facial skin temperature collected using the proposed approach. Additionally, all 6 subjects have voted different thermal sensations at the beginning (the first 5 minutes) and at the end (the last 5 minutes) of the heating experiment, which can be reflected by the significant differences in the mean skin temperature of these two periods (p < .001). Results of this pilot study demonstrate the feasibility of applying the proposed non-intrusive approach to real multi-occupancy environments to dynamically interpret occupants’ thermal comfort and optimize the operation of building heating, ventilation and air conditioning (HVAC) systems.  more » « less
Award ID(s):
1804321
NSF-PAR ID:
10110474
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
CIB World Building Congress 2019, Hong Kong
Issue:
2019
Page Range / eLocation ID:
1-10
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. About 40% of the energy produced globally is consumed within buildings, primarily for providing occupants with comfortable work and living spaces. However, despite the significant impacts of such energy consumption on the environment, the lack of thermal comfort among occupants is a common problem that can lead to health complications and reduced productivity. To address this problem, it is particularly important to understand occupants’ thermal comfort in real-time to dynamically control the environment. This study investigates an infrared thermal camera network to extract skin temperature features and predict occupants’ thermal preferences at flexible distances and angles. This study distinguishes from existing methods in two ways: (1) the proposed method is a non-intrusive data collection approach which does not require human participation or personal devices; (2) it uses low-cost thermal cameras and RGB-D sensors which can be rapidly reconfigured to adapt to various settings and has little or no hardware infrastructure dependency. The proposed camera network is verified using the facial skin temperature collected from 16 subjects in a multi-occupancy experiment. The results show that all 16 subjects observed a statistically higher skin temperature as the room temperature increases. The variations in skin temperature also correspond to the distinct comfort states reported by the subjects. The post-experiment evaluation suggests that the networked thermal cameras have a minimal interruption of building occupants. The proposed approach demonstrates the potential to transition the human physiological data collection from an intrusive and wearable device-based approach to a truly non-intrusive and scalable approach. 
    more » « less
  2. Understanding occupants’ thermal sensation and comfort is essential to defining the operational settings for Heating, Ventilation and Air Conditioning (HVAC) systems in buildings. Due to the continuous impact of human and environmental factors, occupants’ thermal sensation and comfort level can change over time. Thus, to dynamically control the environment, thermal comfort should be monitored in real time. This paper presents a novel non-intrusive infrared thermography framework to estimate an occupant’s thermal comfort level by measuring skin temperature collected from different facial regions using low- cost thermal cameras. Unlike existing methods that rely on placing sensors directly on humans for skin temperature measurement, the proposed framework is able to detect the presence of occupants, extract facial regions, measure skin temperature features, and interpret thermal comfort conditions with minimal interruption of the building occupants. The method is validated by collecting thermal comfort data from a total of twelve subjects under cooling, heating and steady-state experiments. The results demonstrate that ears, nose and cheeks are most indicative of thermal comfort and the proposed framework can be used to assess occupants’ thermal comfort with an average accuracy of 85%. 
    more » « less
  3. Understanding occupants’ thermal comfort is essential for the effective operation of Heating, Ventilation, and Air Conditioning (HVAC) systems. Existing studies of the “human-in-the-loop” HVAC control generally suffer from: (1) excessive reliance on cumbersome human feedback; and (2) intrusiveness caused by conventional data collection methods. To address these limitations, this paper investigates the low-cost thermal camera as a non-intrusive approach to assess thermal comfort in real time using facial skin temperature. The framework developed can automatically detect occupants, extract facial regions, measure skin temperature, and interpret thermal comfort with minimal interruption or participation of occupants. The framework is validated using the facial skin temperature collected from twelve occupants. Personal comfort models trained from different machine learning algorithms are compared and results show that Random Forest model can achieve an accuracy of 85% and also suggest that the skin temperature of ears, nose, and cheeks are most indicative of thermal comfort. 
    more » « less
  4. This study presents a vision-based approach that employs RGB video images as the sole source for inferring thermoregulation states in the human body in response to thermal condition variations in indoor environments. The primary objective is to contribute to our envisioned thermoregulation-based HVAC control that leverages actual thermal demands from end-users’ thermoregulation states for increased energy efficiency. Given that the envisioned control system calls for measurement techniques under four constraints of non-intrusiveness, applicability, sensitivity, and ubiquity (i.e., feasibility and scalability), this study investigated the potentials of ubiquitously obtainable RGB video images (through webcams or smartphones). Using photoplethysmography(PPG), a well-known optical technique for measuring blood volume changes in the microvascular bed of skin, we have leveraged the mechanism of blood flow control to the skin surface (blood vessels' dilation and constriction)for heat dissipation regulations, reflected in PPG signal's amplitude. Given the subtle variations of PPG signals and their susceptibility to noise, we proposed a framework that uses a combination of independent component analysis and adaptive filtering to reduce unwanted and in-band artifacts while preserving the amplitude information of PPG signals that indicates thermophysiological states. The framework was experimentally evaluated using transient thermal conditions to account for applicability and sensitivity attributes. Therefore, without considering an acclimation time for stabilized thermoregulation states, human subjects were exposed to varying temperatures (∼20–30 °C) while reporting their thermal sensations. In total, for 10 human subjects out of 15, a positive correlation between vision-based indicators, skin temperature, and thermal sensations were observed demonstrating promising potential in inferring thermal sensations of occupants with sufficient sensitivity 
    more » « less
  5. null (Ed.)
    Mental workload represents the mental resources an individual devotes to a task. In a building environment, understanding how ambient thermal conditions affect occupants' mental workload offers an opportunity to achieve optimal thermal settings for the heating, ventilation, and air conditioning (HVAC) systems. However, directly measuring mental workload on a large and continuous scale requires occupants to perform subjective tests or wear electroencephalogram (EEG) or similar devices, which is impractical. This paper assesses the feasibility of using infrared facial thermography captured by a low-cost thermal camera to disclose mental workload. An experiment was conducted to measure the facial skin temperature while subjects performed cognitive tasks in three different thermal environments, representing occupants' thermal sensation of slightly cool, neutral, and slightly warm. Mental workload was measured using an EEG headset to eliminate subjective bias. The correlations between facial temperature and mental workload vary with different individuals and thermal conditions. Relatively strong correlations are found in the neutral environment and in the regions of ears, mouth, and neck. The results also suggest that future work should collect data under extended experiment duration. This is because it was observed that the response of facial skin temperature to mental workload varies with task type; thus, increasing the repetitiveness for each type of task or using more challenging tasks in the experiment could potentially lead to more insights on this relationship. 
    more » « less