skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: SenseTribute: Smart Home Occupant Identification via Fusion Across On-Object Sensing Devices
Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute, a novel occupant identification solution that makes use of existing and prevalent on-object sensors that are originally designed to monitor the status of objects they are attached to. SenseTribute extracts richer information content from such on-object sensors and analyzes the data to accurately identify the person interacting with the objects. This approach is based on the physical phenomenon that different occupants interact with objects in different ways. Moreover, SenseTribute may not rely on users’ true identities, so the approach works even without labeled training data. However, resolution of information from a single on-object sensor may not be sufficient to differentiate occupants, which may lead to errors in identification. To overcome this problem, SenseTribute operates over a sequence of events within a user activity, leveraging recent work on activity segmentation. We evaluate SenseTribute using real-world experiments by deploying sensors on five distinct objects in a kitchen and inviting participants to interact with the objects. We demonstrate that SenseTribute can correctly identify occupants in 96% of trials without labeled training data, while per-sensor identification yields only 74% accuracy even with training data.  more » « less
Award ID(s):
1645759
PAR ID:
10048438
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
4th ACM Int'l Conference on Systems for Energy-Efficient Built Environments (BuildSys'17)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The key to optimal occupant comfort as well as resource utilization in a smart building is to provide personalized control over smart appliances. Additionally, with an exponentially growing Internet-of-Things (IoT), reducing the need of frequent user attention and effort involving building management to control and manage an enormous number of smart devices becomes inevitable. One crucial step to enable occupant-specific personalized spaces in smart buildings is accurate identification of different occupants. In this paper, we introduce SolarWalk to show that small and unobtrusive indoor photovoltaic harvesters can identify occupants in smart home scenarios. The key observations are that i) photovoltaics are commonly used as a power source for many indoor energy-harvesting devices, ii) a PV cell's output voltage is perturbed differently when different persons pass in close range, creating an unique signature voltage trace, and iii) the voltage pattern can also determine the person' walking direction. SolarWalk identifies occupants in a smart home by training a classifier with their shadow voltage traces. SolarWalk achieves an average accuracy of 88% to identify five occupants in a home and on average 77% accurate to determine whether someone entered or exited the room. SolarWalk enables an accurate occupant identification system that is non-invasive, ubiquitous, and does not require dedicated hardware and rigorous installation. 
    more » « less
  2. In this paper, we present a multiple concurrent occupant identification approach through footstep-induced floor vibration sensing. Identification of human occupants is useful in a variety of indoor smart structure scenarios, with applications in building security, space allocation, and healthcare. Existing approaches leverage sensing modalities such as vision, acoustic, RF, and wearables, but are limited due to deployment constraints such as line-of-sight requirements, sensitivity to noise, dense sensor deployment, and requiring each walker to wear/carry a device. To overcome these restrictions, we use footstep-induced structural vibration sensing. Footstep-induced signals contain information about the occupants' unique gait characteristics, and propagate through the structural medium, which enables sparse and passive identification of indoor occupants. The primary research challenge is that multiple-person footstep-induced vibration responses are a mixture of structurally-codependent overlapping individual responses with unknown timing, spectral content, and mixing ratios. As such, it is difficult to determine which part of the signal corresponds to each occupant. We overcome this challenge through a recursive sparse representation approach based on cosine distance that identifies each occupant in a footstep event in the order that their signals are generated, reconstructs their portion of the signal, and removes it from the mixed response. By leveraging sparse representation, our approach can simultaneously identify and separate mixed/overlapping responses, and the use of the cosine distance error function reduces the influence of structural codependency on the multiple walkers' signals. In this way, we isolate and identify each of the multiple occupants' footstep responses. We evaluate our approach by conducting real-world walking experiments with three concurrent walkers and achieve an average F1 score for identifying all persons of 0.89 (1.3x baseline improvement), and with a 10-person "hybrid" dataset (simulated combination of single-walker real-world data), we identify 2, 3, and 4 concurrent walkers with a trace-level accuracy of 100%, 93%, and 73%, respectively, and observe as much as a 2.9x error reduction over a naive baseline approach. 
    more » « less
  3. Smart home electronic devices invisibly collect, process, and exchange information with each other and with remote services, often without a home occupants' knowledge or consent. These devices may be mobile or fixed and may have wireless or wired network connections. Detecting and identifying all devices present in a home is a necessary first step to control the flow of data, but there exists no universal mechanism to detect and identify all electronic devices in a space. In this paper we present ICED (Identification and Classification of Electronic Devices), a system that can (i) identify devices from a known set of devices, and (ii) detect the presence of previously unseen devices. ICED, based on harmonic radar technology, collects measurements at the first harmonic of the radar's transmit frequency. We find that the harmonic response contains enough information to infer the type of device. It works when the device has no wireless network interface, is powered off, or attempts to evade detection. We evaluate performance on a collection of 17 devices and find that by transmitting a range of frequencies we correctly identify known devices with 97.6% accuracy and identify previously unseen devices as ‘unknown’ with 69.0% balanced accuracy. 
    more » « less
  4. Abstract Cognitive buildings use data on how occupants respond to the built environment to proactively make occupant-centric adjustments to lighting, temperature, ventilation, and other environmental parameters. However, sensors that unobtrusively and ubiquitously measure occupant responses are lacking. Here we show that Doppler-radar based sensors, which can sense small physiological motions, provide accurate occupancy detection and estimation of vital signs in challenging, realistic circumstances. Occupancy was differentiated from an empty room over 93% of the time in a 3.4 m × 8.5 m conference room with a single sensor in both wall and ceiling-mounted configurations. Occupancy was successfully detected while an occupant was under the table, visibly blocked from the sensor, a scenario where infrared, ultrasound, and video-based occupancy sensors would fail. Heart and respiratory rates were detected in all seats in the conference room with a single ceiling-mounted sensor. The occupancy sensor can be used to control HVAC and lighting with a short, 1–2 min delay and to provide information for space utilization optimization. Heart and respiratory rate sensing could provide additional feedback to future human-building interactive systems that use vital signs to determine how occupant comfort and wellness is changing with time. 
    more » « less
  5. Smart buildings promise to adapt environmental conditions to the needs of occupants based on statistical analytics applied to various monitored data. While sensors for accurate monitoring of building parameters such as temperature, lighting, and air-quality abound, currently available occupancy sensors are limited to sensing of presence only, with limited accuracy. Doppler radar sensors have shown great promise for unobtrusive recognition and monitoring of occupant presence, count, activity, and cardiopulmonary vital signs. With such measures, a smart building can optimize operations not only for the most efficient use of energy and space, but also to create healthy and sustainable environments that support occupant wellness, comfort, and productivity. This paper presents an overview of Doppler radar occupancy sensors for smart building applications. 
    more » « less