An official website of the United States government
The use of audio and video modalities for Human Activity Recognition (HAR) is common, given the richness of the data and the availability of pre-trained ML models using a large corpus of labeled training data. However, audio and video sensors also lead to significant consumer privacy concerns. Researchers have thus explored alternate modalities that are less privacy-invasive such as mmWave doppler radars, IMUs, motion sensors. However, the key limitation of these approaches is that most of them do not readily generalize across environments and require significant in-situ training data. Recent work has proposed cross-modality transfer learning approaches to alleviate the lack of trained labeled data with some success. In this paper, we generalize this concept to create a novel system called VAX (Video/Audio to 'X'), where training labels acquired from existing Video/Audio ML models are used to train ML models for a wide range of 'X' privacy-sensitive sensors. Notably, in VAX, once the ML models for the privacy-sensitive sensors are trained, with little to no user involvement, the Audio/Video sensors can be removed altogether to protect the user's privacy better. We built and deployed VAX in ten participants' homes while they performed 17 common activities of daily living. Our evaluation results show that after training, VAX can use its onboard camera and microphone to detect approximately 15 out of 17 activities with an average accuracy of 90%. For these activities that can be detected using a camera and a microphone, VAX trains a per-home model for the privacy-preserving sensors. These models (average accuracy = 84%) require no in-situ user input. In addition, when VAX is augmented with just one labeled instance for the activities not detected by the VAX A/V pipeline (~2 out of 17), it can detect all 17 activities with an average accuracy of 84%. Our results show that VAX is significantly better than a baseline supervised-learning approach of using one labeled instance per activity in each home (average accuracy of 79%) since VAX reduces the user burden of providing activity labels by 8x (~2 labels vs. 17 labels).
more »
« less
AeroSense: Sensing Aerosol Emissions from Indoor Human Activities
The types of human activities occupants are engaged in within indoor spaces significantly contribute to the spread of airborne diseases through emitting aerosol particles. Today, ubiquitous computing technologies can inform users of common atmosphere pollutants for indoor air quality. However, they remain uninformed of the rate of aerosol generated directly from human respiratory activities, a fundamental parameter impacting the risk of airborne transmission. In this paper, we present AeroSense, a novel privacy-preserving approach using audio sensing to accurately predict the rate of aerosol generated from detecting the kinds of human respiratory activities and determining the loudness of these activities. Our system adopts a privacy-first as a key design choice; thus, it only extracts audio features that cannot be reconstructed into human audible signals using two omnidirectional microphone arrays. We employ a combination of binary classifiers using the Random Forest algorithm to detect simultaneous occurrences of activities with an average recall of 85%. It determines the level of all detected activities by estimating the distance between the microphone and the activity source. This level estimation technique yields an average of 7.74% error. Additionally, we developed a lightweight mask detection classifier to detect mask-wearing, which yields a recall score of 75%. These intermediary outputs are critical predictors needed for AeroSense to estimate the amounts of aerosol generated from an active human source. Our model to predict aerosol is a Random Forest regression model, which yields 2.34 MSE and 0.73 r2 value. We demonstrate the accuracy of AeroSense by validating our results in a cleanroom setup and using advanced microbiological technology. We present results on the efficacy of AeroSense in natural settings through controlled and in-the-wild experiments. The ability to estimate aerosol emissions from detected human activities is part of a more extensive indoor air system integration, which can capture the rate of aerosol dissipation and inform users of airborne transmission risks in real time.
more »
« less
- PAR ID:
- 10627061
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Volume:
- 8
- Issue:
- 2
- ISSN:
- 2474-9567
- Page Range / eLocation ID:
- 1 to 30
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Grassian, Vicki (Ed.)Nanocluster aerosol (NCA: particles in the size range of 1–3 nm) are a critically important, yet understudied, class of atmospheric aerosol particles. NCA efficiently deposit in the human respiratory system and can translocate to vital organs. Due to their high surface area-to-mass ratios, NCA are associated with a heightened propensity for bioactivity and toxicity. Despite the human health relevance of NCA, little is known regarding the prevalence of NCA in indoor environments where people spend the majority of their time. In this study, we quantify the formation and transformation of indoor atmospheric NCA down to 1 nm via high-resolution online nanoparticle measurements during propane gas cooking in a residential building. We observed a substantial pool of sub-1.5 nm NCA in the indoor atmosphere during cooking periods, with aerosol number concentrations often dominated by the newly formed NCA. Indoor atmospheric NCA emission factors can reach up to ~10^16 NCA/kg-fuel during propane gas cooking and can exceed those for vehicles with gasoline and diesel engines. Such high emissions of combustion-derived indoor NCA can result in substantial NCA respiratory exposures and dose rates for children and adults, significantly exceeding that for outdoor traffic-associated NCA. Combustion-derived indoor NCA undergo unique size-dependent physical transformations, strongly influenced by particle coagulation and condensation of low-volatility cooking vapors. We show that indoor atmospheric NCA need to be measured directly and cannot be predicted using conventional indoor air pollution markers such as PM2.5 mass concentrations and NOx (NO + NO2) mixing ratios.more » « less
-
Abstract Fluid-mechanics research has focused primarily on droplets/aerosols being expelled from infected individuals and transmission of well-mixed aerosols indoors. However, aerosol collisions with susceptible hosts earlier in the spread, as well as aerosol deposition in the nasal cavity, have been relatively overlooked. In this paper, two simple fluid models are presented to gain a better understanding of the collision and deposition between a human and aerosols. The first model is based on the impact of turbulent diffusion coefficients and air flow in a room on the collisions between aerosols and humans. Infection rates can be determined based on factors such as air circulation and geometry as an infection zone expands from an infected host. The second model clarifies how aerosols of different sizes adhere to different parts of the respiratory tract. Based on the inhalation rate and the nasal cavity shape, the critical particle size and the deposition location can be determined. Our study offers simple fluid models to understand the effects of geometric factors and air flows on the aerosol transmission and deposition.more » « less
-
Audio-based human activity recognition (HAR) is very popular because many human activities have unique sound signatures that can be detected using machine learning (ML) approaches. These audio-based ML HAR pipelines often use common featurization techniques, such as extracting various statistical and spectral features by converting time domain signals to the frequency domain (using an FFT) and using them to train ML models. Some of these approaches also claim privacy benefits by preventing the identification of human speech. However, recent deep learning-based automatic speech recognition (ASR) models pose new privacy challenges to these featurization techniques. In this paper, we systematically evaluate various featurization approaches for audio data, assessing their privacy risks through metrics like speech intelligibility (PER and WER) while considering the utility tradeoff in terms of ML-based activity recognition accuracy. Our findings reveal the susceptibility of these approaches to speech content recovery when exposed to recent ASR models, especially under re-tuning or retraining conditions. Notably, fine-tuned ASR models achieved an average Phoneme Error Rate (PER) of 39.99% and Word Error Rate (WER) of 44.43% in speech recognition for these approaches. To overcome these privacy concerns, we propose Kirigami, a lightweight machine learning-based audio speech filter that removes human speech segments reducing the efficacy of ASR models (70.48% PER and 101.40% WER) while also maintaining HAR accuracy (76.0% accuracy). We show that Kirigami can be implemented on common edge microcontrollers with limited computational capabilities and memory, providing a path to deployment on a variety of IoT devices. Finally, we conducted a real-world user study and showed the robustness of Kirigami on a laptop and an ARM Cortex-M4F microcontroller under three different background noises.more » « less
-
null (Ed.)Abstract Background Aerosol transmission of COVID-19 is the subject of ongoing policy debate. Characterizing aerosol produced by people with COVID-19 is critical to understanding the role of aerosols in transmission. Objective We investigated the presence of virus in size-fractioned aerosols from six COVID-19 patients admitted into mixed acuity wards in April of 2020. Methods Size-fractionated aerosol samples and aerosol size distributions were collected from COVID-19 positive patients. Aerosol samples were analyzed for viral RNA, positive samples were cultured in Vero E6 cells. Serial RT-PCR of cells indicated samples where viral replication was likely occurring. Viral presence was also investigated by western blot and transmission electron microscopy (TEM). Results SARS-CoV-2 RNA was detected by rRT-PCR in all samples. Three samples confidently indicated the presence of viral replication, all of which were from collected sub-micron aerosol. Western blot indicated the presence of viral proteins in all but one of these samples, and intact virions were observed by TEM in one sample. Significance Observations of viral replication in the culture of submicron aerosol samples provides additional evidence that airborne transmission of COVID-19 is possible. These results support the use of efficient respiratory protection in both healthcare and by the public to limit transmission.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1145/3659593
