skip to main content

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, February 13 until 2:00 AM ET on Friday, February 14 due to maintenance. We apologize for the inconvenience.


This content will become publicly available on June 1, 2025

Title: Occupancy Estimation from Blurred Video: A Multifaceted Approach with Privacy Consideration
Building occupancy information is significant for a variety of reasons, from allocation of resources in smart buildings to responding during emergency situations. As most people spend more than 90% of their time indoors, a comfortable indoor environment is crucial. To ensure comfort, traditional HVAC systems condition rooms assuming maximum occupancy, accounting for more than 50% of buildings’ energy budgets in the US. Occupancy level is a key factor in ensuring energy efficiency, as occupancy-controlled HVAC systems can reduce energy waste by conditioning rooms based on actual usage. Numerous studies have focused on developing occupancy estimation models leveraging existing sensors, with camera-based methods gaining popularity due to their high precision and widespread availability. However, the main concern with using cameras for occupancy estimation is the potential violation of occupants’ privacy. Unlike previous video-/image-based occupancy estimation methods, we addressed the issue of occupants’ privacy in this work by proposing and investigating both motion-based and motion-independent occupancy counting methods on intentionally blurred video frames. Our proposed approach included the development of a motion-based technique that inherently preserves privacy, as well as motion-independent techniques such as detection-based and density-estimation-based methods. To improve the accuracy of the motion-independent approaches, we utilized deblurring methods: an iterative statistical technique and a deep-learning-based method. Furthermore, we conducted an analysis of the privacy implications of our motion-independent occupancy counting system by comparing the original, blurred, and deblurred frames using different image quality assessment metrics. This analysis provided insights into the trade-off between occupancy estimation accuracy and the preservation of occupants’ visual privacy. The combination of iterative statistical deblurring and density estimation achieved a 16.29% counting error, outperforming our other proposed approaches while preserving occupants’ visual privacy to a certain extent. Our multifaceted approach aims to contribute to the field of occupancy estimation by proposing a solution that seeks to balance the trade-off between accuracy and privacy. While further research is needed to fully address this complex issue, our work provides insights and a step towards a more privacy-aware occupancy estimation system.

 
more » « less
Award ID(s):
2039089
PAR ID:
10547805
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
MDPI
Date Published:
Journal Name:
Sensors
Volume:
24
Issue:
12
ISSN:
1424-8220
Page Range / eLocation ID:
3739
Subject(s) / Keyword(s):
occupancy counting deblurring deep learning machine learning image processing privacy
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Cognitive buildings use data on how occupants respond to the built environment to proactively make occupant-centric adjustments to lighting, temperature, ventilation, and other environmental parameters. However, sensors that unobtrusively and ubiquitously measure occupant responses are lacking. Here we show that Doppler-radar based sensors, which can sense small physiological motions, provide accurate occupancy detection and estimation of vital signs in challenging, realistic circumstances. Occupancy was differentiated from an empty room over 93% of the time in a 3.4 m × 8.5 m conference room with a single sensor in both wall and ceiling-mounted configurations. Occupancy was successfully detected while an occupant was under the table, visibly blocked from the sensor, a scenario where infrared, ultrasound, and video-based occupancy sensors would fail. Heart and respiratory rates were detected in all seats in the conference room with a single ceiling-mounted sensor. The occupancy sensor can be used to control HVAC and lighting with a short, 1–2 min delay and to provide information for space utilization optimization. Heart and respiratory rate sensing could provide additional feedback to future human-building interactive systems that use vital signs to determine how occupant comfort and wellness is changing with time. 
    more » « less
  2. People spend approximately 90% of their time indoors, making effective indoor air quality (IAQ) monitoring crucial for occupants’ well-being. Traditional IAQ monitoring primarily focuses on carbon dioxide ( ) levels to inform the operation of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, HVAC systems often overlook other critical IAQ metrics, such as volatile organic compounds (VOC), which may correspond better to occupant activities in some cases. This naturalistic study, conducted over four months at the University of Virginia, addresses this significant gap by observing changes in VOC and levels across various times, events, and spaces, including conference rooms, single occupancy offices, and common open-space areas. We aimed to determine whether can be the only representative of IAQ for dynamically adjusting the ventilation rates within this testbed. A key focus was on poor IAQ instances where levels were below the recommended levels, but VOC concentrations exceeded them, potentially impacting occupants’ health and well-being. Our results revealed that in the studied conference room, poor IAQ conditions prevailed 71% of the time during occupancy, in contrast to lower rates in single occupancy offices (11%, 7%, and 16%). Notably, while social events influenced levels less, VOC levels significantly increased in all open-space areas. These findings challenge the conventional reliance on monitoring for IAQ management, highlighting the necessity of incorporating comprehensive IAQ metrics in HVAC systems. The study underscores the critical need for dynamic HVAC systems that adapt to real-time IAQ conditions, a vital step towards enhancing indoor environmental quality in various settings. 
    more » « less
  3. Predicting the occupancy related information in an environment has been investigated to satisfy the myriad requirements of various evolving pervasive, ubiquitous, opportunistic and participatory sensing applications. Infrastructure and ambient sensors based techniques have been leveraged largely to determine the occupancy of an environment incurring a significant deployment and retrofitting costs. In this paper, we advocate an infrastructure-less zero-configuration multimodal smartphone sensor-based techniques to detect fine-grained occupancy information. We propose to exploit opportunistically smartphones' acoustic sensors in presence of human conversation and motion sensors in absence of any conversational data. We develop a novel speaker estimation algorithm based on unsupervised clustering of overlapped and non-overlapped conversational data to determine the number of occupants in a crowded environment. We also design a hybrid approach combining acoustic sensing opportunistically with locomotive model to further improve the occupancy detection accuracy. We evaluate our algorithms in different contexts, conversational, silence and mixed in presence of 10 domestic users. Our experimental results on real-life data traces collected from 10 occupants in natural setting show that using this hybrid approach we can achieve approximately 0.76 error count distance for occupancy detection accuracy on average. 
    more » « less
  4. Occupancy detection helps enable various emerging smart environment applications ranging from opportunistic HVAC (heating, ventilation, and air-conditioning) control, effective meeting management, healthy social gathering, and public event planning and organization. Ubiquitous availability of smartphones and wearable sensors with the users for almost 24 hours helps revitalize a multitude of novel applications. The inbuilt microphone sensor in smartphones plays as an inevitable enabler to help detect the number of people conversing with each other in an event or gathering. A large number of other sensors such as accelerometer and gyroscope help count the number of people based on other signals such as locomotive motion. In this work, we propose multimodal data fusion and deep learning approach relying on the smartphone’s microphone and accelerometer sensors to estimate occupancy. We first demonstrate a novel speaker estimation algorithm for people counting and extend the proposed model using deep nets for handling large-scale fluid scenarios with unlabeled acoustic signals. We augment our occupancy detection model with a magnetometer-dependent fingerprinting-based localization scheme to assimilate the volume of location-specific gathering. We also propose crowdsourcing techniques to annotate the semantic location of the occupant. We evaluate our approach in different contexts: conversational, silence, and mixed scenarios in the presence of 10 people. Our experimental results on real-life data traces in natural settings show that our cross-modal approach can achieve approximately 0.53 error count distance for occupancy detection accuracy on average. 
    more » « less
  5. Thermal comfort and energy efficiency are always the two most significant objectives in HVAC operations. However, for conventional HVAC systems, the pursuit of high energy efficiency may be at the expense of satisfactory thermal comfort. Therefore, even if centralized HVAC systems nowadays have higher energy efficiency than before in office buildings, most of them cannot adapt the dynamic occupant behaviors or individual thermal comfort. In order to realize high energy efficiency while still maintain satisfactory thermal environment for occupants indoors, the integrated hybrid HVAC system has been developed for years such as task-ambient conditioning system. Moreover, the occupant-based HVAC control system such as human- in-the-loop has also been investigated so that the system can be adaptive based on occupant behaviors. However, most of research related to personalized air-conditioning system only focuses on field-study with limited scale (i.e. only one office room), this paper has proposed a co- simulation model in energyplus to simulate the hybrid cooling system with synthetic thermal comfort distributions based on global comfort database I&II. An optimization framework on cooling set-point is proposed with the objective of energy performance and the constraints of thermal comfort distribution developed by unsupervised Gaussian mixture model (GMM) clustering and kernel density estimation (KDE). The co-simulation results have illustrated that with the proposed optimization algorithm and the hybrid cooling system, HVAC demand power has decreased 5.3% on average with at least 90% of occupants feeling satisfied. 
    more » « less