More Like this

1. IoT Augmented Physical Scale Model of a Suburban Home

   https://doi.org/10.1109/ICCWorkshops49005.2020.9145040  

   Burns, Thomas ; Fichthorn, Gregory ; Zehtabian, Sharare ; Bacanli, Salih Safa ; Razghandi, Mina ; Boloni, Ladislau ; Turgut, Damla ( June 2020 , IEEE ICC 2020 Workshop on Convergent IoT (C-IoT))

   null (Ed.)

   Green homes require informed energy management decisions. For instance, it is preferable that a comfortable internal temperature is achieved through natural, energy-efficient means such as opening doors or lowering shades as opposed to turning on the air conditioning. This requires the control agent to understand the complex system dynamics of the home: will opening the window raise or lower the temperature in this particular situation? Unfortunately, developing mathematical models of a suburban home situated in its natural environment is a significant challenge, while performing real-world experiments is costly, takes a long time and depends on external circumstances beyond the control of the experimenter. In this paper, we describe the architecture of a physical, small scale model of a suburban home and its immediate exterior environment. Specific scenarios can be enacted using Internet of Things (IoT) actuators that control the doors and windows. We use a suite of IoT sensors to collect data during the scenario. We use deep learning-based temporal regression models to make predictions about the impact of specific actions on the temperature and humidity in the home. 

   more » « less
2. Near real-time ASL recognition using a millimeter wave radar

   https://doi.org/10.1117/12.2588616  

   Adeoluwa, Oladipupo ; Kearney, Sean J. ; Kurtoglu, Emre ; Connors, Charles ; Gurbuz, Sevgi Zubeyde ( April 2021 , Proceedings of SPIE (International Society of Optics and Photonics))

   Raynal, Ann M. ; Ranney, Kenneth I. (Ed.)

   Most research in technologies for the Deaf community have focused on translation using either video or wearable devices. Sensor-augmented gloves have been reported to yield higher gesture recognition rates than camera-based systems; however, they cannot capture information expressed through head and body movement. Gloves are also intrusive and inhibit users in their pursuit of normal daily life, while cameras can raise concerns over privacy and are ineffective in the dark. In contrast, RF sensors are non-contact, non-invasive and do not reveal private information even if hacked. Although RF sensors are unable to measure facial expressions or hand shapes, which would be required for complete translation, this paper aims to exploit near real-time ASL recognition using RF sensors for the design of smart Deaf spaces. In this way, we hope to enable the Deaf community to benefit from advances in technologies that could generate tangible improvements in their quality of life. More specifically, this paper investigates near real-time implementation of machine learning and deep learning architectures for the purpose of sequential ASL signing recognition. We utilize a 60 GHz RF sensor which transmits a frequency modulation continuous wave (FMWC waveform). RF sensors can acquire a unique source of information that is inaccessible to optical or wearable devices: namely, a visual representation of the kinematic patterns of motion via the micro-Doppler signature. Micro-Doppler refers to frequency modulations that appear about the central Doppler shift, which are caused by rotational or vibrational motions that deviate from principle translational motion. In prior work, we showed that fractal complexity computed from RF data could be used to discriminate signing from daily activities and that RF data could reveal linguistic properties, such as coarticulation. We have also shown that machine learning can be used to discriminate with 99% accuracy the signing of native Deaf ASL users from that of copysigning (or imitation signing) by hearing individuals. Therefore, imitation signing data is not effective for directly training deep models. But, adversarial learning can be used to transform imitation signing to resemble native signing, or, alternatively, physics-aware generative models can be used to synthesize ASL micro-Doppler signatures for training deep neural networks. With such approaches, we have achieved over 90% recognition accuracy of 20 ASL signs. In natural environments, however, near real-time implementations of classification algorithms are required, as well as an ability to process data streams in a continuous and sequential fashion. In this work, we focus on extensions of our prior work towards this aim, and compare the efficacy of various approaches for embedding deep neural networks (DNNs) on platforms such as a Raspberry Pi or Jetson board. We examine methods for optimizing the size and computational complexity of DNNs for embedded micro-Doppler analysis, methods for network compression, and their resulting sequential ASL recognition performance. 

   more » « less
3. Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles

   https://doi.org/10.1145/3447814  

   Jia, Xiaowei ; Willard, Jared ; Karpatne, Anuj ; Read, Jordan S. ; Zwart, Jacob A. ; Steinbach, Michael ; Kumar, Vipin ( May 2021 , ACM/IMS Transactions on Data Science)

   null (Ed.)

   Physics-based models are often used to study engineering and environmental systems. The ability to model these systems is the key to achieving our future environmental sustainability and improving the quality of human life. This article focuses on simulating lake water temperature, which is critical for understanding the impact of changing climate on aquatic ecosystems and assisting in aquatic resource management decisions. General Lake Model (GLM) is a state-of-the-art physics-based model used for addressing such problems. However, like other physics-based models used for studying scientific and engineering systems, it has several well-known limitations due to simplified representations of the physical processes being modeled or challenges in selecting appropriate parameters. While state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This article proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improves the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physics-based models (by over 20% even with very little training data), while generating outputs consistent with physical laws. An important aspect of our PGRNN approach lies in its ability to incorporate the knowledge encoded in physics-based models. This allows training the PGRNN model using very few true observed data while also ensuring high prediction accuracy. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where physics-based (also known as mechanistic) models are used. 

   more » « less
4. Machine learning based online full-chip heatmap estimation

   Sadiqbatcha, S ; Zhao, Y. ; Zhang, J. ; Amrouch, H. ; Henkel, J. ; Tan, Sheldon ( January 2020 , Asia South Pacific Design Automation Conference (ASP-DAC’20))

   In tis work, we propose a novel approach to real-time estimation of full-chip transient heatmaps for commercial processors based on machine learning. The model derived in this work supplements the temperature data sensed from the existing on-chip sensors, allowing for the development of more robust runtime power and thermal control schemes that can take advantage of the additional thermal information that is otherwise not available. The new approach involves offline acquisition of accurate spatial and temporal heatmaps using an infrared thermal imaging setup while nominal working conditions are maintained on the chip. To build the dynamic thermal model, we apply Long-Short-Term-Memory (LSTM) neutral networks with system-level variables such as chip frequency, instruction counts, and other performance metrics as inputs. To reduce the dimensionality of the model, 2D spatial discrete cosine transformation (DCT) is first performed on the heatmaps so that they can be expressed with just their dominant DCT frequencies. Our study shows that only $6\times 6$ DCT coefficients are required to maintain sufficient accuracy across a variety of workloads. Experimental results show that the proposed approach can estimate the full-chip heatmaps with less than 1.4C root-mean-square-error and take only 19ms for each inference which suits well for real-time use. 

   more » « less
5. Learning From Others Without Sacrificing Privacy: Simulation Comparing Centralized and Federated Machine Learning on Mobile Health Data

   https://doi.org/10.2196/23728  

   Liu, Jessica Chia ; Goetz, Jack ; Sen, Srijan ; Tewari, Ambuj ( January 2021 , JMIR mHealth and uHealth)

   null (Ed.)

   Background The use of wearables facilitates data collection at a previously unobtainable scale, enabling the construction of complex predictive models with the potential to improve health. However, the highly personal nature of these data requires strong privacy protection against data breaches and the use of data in a way that users do not intend. One method to protect user privacy while taking advantage of sharing data across users is federated learning, a technique that allows a machine learning model to be trained using data from all users while only storing a user’s data on that user’s device. By keeping data on users’ devices, federated learning protects users’ private data from data leaks and breaches on the researcher’s central server and provides users with more control over how and when their data are used. However, there are few rigorous studies on the effectiveness of federated learning in the mobile health (mHealth) domain. Objective We review federated learning and assess whether it can be useful in the mHealth field, especially for addressing common mHealth challenges such as privacy concerns and user heterogeneity. The aims of this study are to describe federated learning in an mHealth context, apply a simulation of federated learning to an mHealth data set, and compare the performance of federated learning with the performance of other predictive models. Methods We applied a simulation of federated learning to predict the affective state of 15 subjects using physiological and motion data collected from a chest-worn device for approximately 36 minutes. We compared the results from this federated model with those from a centralized or server model and with the results from training individual models for each subject. Results In a 3-class classification problem using physiological and motion data to predict whether the subject was undertaking a neutral, amusing, or stressful task, the federated model achieved 92.8% accuracy on average, the server model achieved 93.2% accuracy on average, and the individual model achieved 90.2% accuracy on average. Conclusions Our findings support the potential for using federated learning in mHealth. The results showed that the federated model performed better than a model trained separately on each individual and nearly as well as the server model. As federated learning offers more privacy than a server model, it may be a valuable option for designing sensitive data collection methods. 

   more » « less