In this paper, we present a multiple concurrent occupant identification approach through footstep-induced floor vibration sensing. Identification of human occupants is useful in a variety of indoor smart structure scenarios, with applications in building security, space allocation, and healthcare. Existing approaches leverage sensing modalities such as vision, acoustic, RF, and wearables, but are limited due to deployment constraints such as line-of-sight requirements, sensitivity to noise, dense sensor deployment, and requiring each walker to wear/carry a device. To overcome these restrictions, we use footstep-induced structural vibration sensing. Footstep-induced signals contain information about the occupants' unique gait characteristics, and propagate through the structural medium, which enables sparse and passive identification of indoor occupants. The primary research challenge is that multiple-person footstep-induced vibration responses are a mixture of structurally-codependent overlapping individual responses with unknown timing, spectral content, and mixing ratios. As such, it is difficult to determine which part of the signal corresponds to each occupant. We overcome this challenge through a recursive sparse representation approach based on cosine distance that identifies each occupant in a footstep event in the order that their signals are generated, reconstructs their portion of the signal, and removes it from the mixed response. By leveraging sparse representation, our approach can simultaneously identify and separate mixed/overlapping responses, and the use of the cosine distance error function reduces the influence of structural codependency on the multiple walkers' signals. In this way, we isolate and identify each of the multiple occupants' footstep responses. We evaluate our approach by conducting real-world walking experiments with three concurrent walkers and achieve an average F1 score for identifying all persons of 0.89 (1.3x baseline improvement), and with a 10-person "hybrid" dataset (simulated combination of single-walker real-world data), we identify 2, 3, and 4 concurrent walkers with a trace-level accuracy of 100%, 93%, and 73%, respectively, and observe as much as a 2.9x error reduction over a naive baseline approach.
more »
« less
SenseTribute: Smart Home Occupant Identification via Fusion Across On-Object Sensing Devices
Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute, a novel occupant identification solution that makes use of existing and prevalent on-object sensors that are originally designed to monitor the status of objects they are attached to. SenseTribute extracts richer information content from such on-object sensors and analyzes the data to accurately identify the person interacting with the objects. This approach is based on the physical phenomenon that different occupants interact with objects in different ways. Moreover, SenseTribute may not rely on users’ true identities, so the approach works even without labeled training data. However, resolution of information from a single on-object sensor may not be sufficient to differentiate occupants, which may lead to errors in identification. To overcome this problem, SenseTribute operates over a sequence of events within a user activity, leveraging recent work on activity segmentation. We evaluate SenseTribute using real-world experiments by deploying sensors on five distinct objects in a kitchen and inviting participants to interact with the objects. We demonstrate that SenseTribute can correctly identify occupants in 96% of trials without labeled training data, while per-sensor identification yields only 74% accuracy even with training data.
more »
« less
- Award ID(s):
- 1645759
- NSF-PAR ID:
- 10048438
- Date Published:
- Journal Name:
- 4th ACM Int'l Conference on Systems for Energy-Efficient Built Environments (BuildSys'17)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The key to optimal occupant comfort as well as resource utilization in a smart building is to provide personalized control over smart appliances. Additionally, with an exponentially growing Internet-of-Things (IoT), reducing the need of frequent user attention and effort involving building management to control and manage an enormous number of smart devices becomes inevitable. One crucial step to enable occupant-specific personalized spaces in smart buildings is accurate identification of different occupants. In this paper, we introduce SolarWalk to show that small and unobtrusive indoor photovoltaic harvesters can identify occupants in smart home scenarios. The key observations are that i) photovoltaics are commonly used as a power source for many indoor energy-harvesting devices, ii) a PV cell's output voltage is perturbed differently when different persons pass in close range, creating an unique signature voltage trace, and iii) the voltage pattern can also determine the person' walking direction. SolarWalk identifies occupants in a smart home by training a classifier with their shadow voltage traces. SolarWalk achieves an average accuracy of 88% to identify five occupants in a home and on average 77% accurate to determine whether someone entered or exited the room. SolarWalk enables an accurate occupant identification system that is non-invasive, ubiquitous, and does not require dedicated hardware and rigorous installation.more » « less
-
more » « less
Typically to a roboticist, a plan is the outcome of other work, a synthesized object that realizes ends defined by some problem; plans qua plans are seldom treated as first-class objects of study. Plans designate functionality: a plan can be viewed as defining a robot’s behavior throughout its execution. This informs and reveals many other aspects of the robot’s design, including: necessary sensors and action choices, history, state, task structure, and how to define progress. Interrogating sets of plans helps in comprehending the ways in which differing executions influence the interrelationships between these various aspects. Revisiting Erdmann’s theory of action-based sensors, a classical approach for characterizing fundamental information requirements, we show how plans (in their role of designating behavior) influence sensing requirements. Using an algorithm for enumerating plans, we examine how some plans for which no action-based sensor exists can be transformed into sets of sensors through the identification and handling of features that preclude the existence of action-based sensors. We are not aware of those obstructing features having been previously identified. Action-based sensors may be treated as standalone reactive plans; we relate them to the set of all possible plans through a lattice structure. This lattice reveals a boundary between plans with action-based sensors and those without. Some plans, specifically those that are not reactive plans and require some notion of internal state, can never have associated action-based sensors. Even so, action-based sensors can serve as a framework to explore and interpret how such plans make use of state.
-
more » « less
The use of audio and video modalities for Human Activity Recognition (HAR) is common, given the richness of the data and the availability of pre-trained ML models using a large corpus of labeled training data. However, audio and video sensors also lead to significant consumer privacy concerns. Researchers have thus explored alternate modalities that are less privacy-invasive such as mmWave doppler radars, IMUs, motion sensors. However, the key limitation of these approaches is that most of them do not readily generalize across environments and require significant in-situ training data. Recent work has proposed cross-modality transfer learning approaches to alleviate the lack of trained labeled data with some success. In this paper, we generalize this concept to create a novel system called VAX (Video/Audio to 'X'), where training labels acquired from existing Video/Audio ML models are used to train ML models for a wide range of 'X' privacy-sensitive sensors. Notably, in VAX, once the ML models for the privacy-sensitive sensors are trained, with little to no user involvement, the Audio/Video sensors can be removed altogether to protect the user's privacy better. We built and deployed VAX in ten participants' homes while they performed 17 common activities of daily living. Our evaluation results show that after training, VAX can use its onboard camera and microphone to detect approximately 15 out of 17 activities with an average accuracy of 90%. For these activities that can be detected using a camera and a microphone, VAX trains a per-home model for the privacy-preserving sensors. These models (average accuracy = 84%) require no in-situ user input. In addition, when VAX is augmented with just one labeled instance for the activities not detected by the VAX A/V pipeline (~2 out of 17), it can detect all 17 activities with an average accuracy of 84%. Our results show that VAX is significantly better than a baseline supervised-learning approach of using one labeled instance per activity in each home (average accuracy of 79%) since VAX reduces the user burden of providing activity labels by 8x (~2 labels vs. 17 labels).
-
Occupant behavior has a significant impact on building systems’ operations and efficiency. As a result, several innovative approaches have been introduced to quantify the dynamics of occupants within indoor environments, such as interactions with different building systems and the impact of various feedback and interventions to reduce the building energy consumption. To achieve this, researchers have highlighted the importance of reducing energy consumption without impacting occupant comfort. As a result, there is an increasing body of research evaluating how different theories of behavior across a variety of disciplines can explain occupant interactions with building systems. Future progress in this area calls for an in-depth understanding of behavioral theories in explaining occupant interactions with different building systems. In this paper, we have used a structured literature review approach to investigate how different psychological, sociological, and economic theories have been applied to explain occupant interactions with heating and cooling (HVAC systems), opening windows and ventilation, lighting and shading, electronic appliances, domestic hot water, as well as energy conservation behaviors. Throughout the paper, we identify the most common theories and methodologies applied within the existing research, general findings related to how occupants interact with different building systems, as well as a number of identified gaps within the literature. Finally, we provide a discussion on directions for future research studies in this area under each building system.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
