More Like this

1. Smart Bracelets for Remote Monitoring of Wearers' Physical and Affective State

   https://doi.org/10.1109/MECO49872.2020.9134093  

   Rishe, Naphtali David; Adjouadi, Malek; Ortega, Francisco (June 2020, 2020 9th Mediterranean Conference on Embedded Computing (MECO))

   null (Ed.)

   Smart bracelets able to interpret the wearer's emotional state and communicate it to a remote decision-support facility will have broad applications in healthcare, elder care, the military, and other fields. While there are existing commercial embedded devices, such as the Apple Watch, that have health-monitoring sensors, such devices cannot sufficiently support a real-time health-monitoring system with battery-efficient remote data delivery. Ongoing R&D is developing solutions capable of monitoring multiple psycho-physiological signals. Possible hardware configurations include wrist-worn devices and sensors across an augmented reality headset (e.g., HoloLens 2). The device should carry an array of sensors of psycho-physiological signals, including a galvanic skin response sensor, motion sensor, skin temperature sensor, and a heart rate sensor. Output from these sensors can be intelligently fused to monitor the affective state and to determine specific trigger events for the wearer. To enable real-time remote monitoring applications, the device needs to be low-power to allow persistent monitoring while prolonging usage before recharging. For many applications, specialized sensor arrays are required, e.g. a galvanic skin response sensor. An application-flexible device would allow adding/removing sensors and would provide a choice of communication modules (e.g., Bluetooth 5.0 low-energy vs ZigBee). Appropriate configurations of the device would support applications in military health monitoring, drug-addiction mitigation, autistic trigger monitoring, and augmented reality exploration. A configuration example is: motion sensors (3-axis accelerometers, gyroscopes, and magnetometers to track steps, falls, and energy usage), a heart-rate sensor (e.g., an optical-based heart rate sensor with a single monitoring zone using the process of photoplethysmography (PPS)), at least a Bluetooth 5.0 (but a different communication device may be needed depending on the use case), and flash memory to temporarily store data when the device is not remotely communicating. The wearables field has greatly advanced in the quality of sensors; the fusion of multi-sensor data is the current frontier. 

   more » « less
2. A Novel Headset System Synchronizing Vision and EEG testing for a Rapid Assessment and Diagnosis of Concussions and Other Brain Injuries

   https://doi.org/10.54941/ahfe1002125  

   Edquilang, David; Feng, Jeff (January 2022, AHFE International)

   Millions of concussions happen each year in the US alone. A proportionally large number of these concussions are due to high impact sports injury. Currently, there exists no solution to quickly monitor brain functions and test the oculomotor functions of individuals who have suffered a traumatic brain injury in order to diagnose them as having suffered a concussion. What is presently done to diagnose concussions is a CT scan or MRI, which are lengthy procedures to schedule, set up, and conduct; and furthermore, takes additional time to analyze the results in order to arrive at a diagnosis. This prolongation of the diagnosing process is inherently problematic since the longer time it takes between time of injury and time of diagnosis, there is greater risk of decisions and actions which can worsen damage to the brain. The sooner a concussion can be diagnosed, the sooner and better the treatment can be performed for recovery. In order to ameliorate this issue, we seek to develop a device to perform the function of diagnosis and monitoring of brain activity in a more rapid and timely manner. Literature review into the anatomy of vestibular and ocular brain functions was performed; as well as research into various testing and monitoring methodologies of these vestibular and ocular functions. One such method that has proven to be a reliable method for diagnosis is Vestibular Ocular Motor Screening (VOMS), which is a visual and balance test performed by a doctor with a patient. Further research was also done into existing technologies whose functionalities would allow the device in order to perform brain monitoring, visual testing, and ultimately diagnosis; namely EEG, VR, and infrared eye tracking. Currently, very few devices on the market take advantage of these technologies together for medical uses. A device incorporating these technologies together allows would allow for more consistent administering of visual tests and real-time monitoring of brain activity. With a functional prototype, user testing is to be performed in order to assess the function and viability of the device. 

   more » « less
3. HAR-CTGAN: A Mobile Sensor Data Generation Tool for Human Activity Recognition

   https://doi.org/10.1109/BigData55660.2022.10020848  

   DeOliveira, Joshua; Gerych, Walter; Koshkarova, Aruzhan; Rundensteiner, Elke; Agu, Emmanuel (December 2022, 2022 IEEE International Conference on Big Data (Big Data))

   Human activity recognition (HAR) is the process of using mobile sensor data to determine the physical activities performed by individuals. HAR is the backbone of many mobile healthcare applications, such as passive health monitoring systems, early diagnosing systems, and fall detection systems. Effective HAR models rely on deep learning architectures and big data in order to accurately classify activities. Unfortunately, HAR datasets are expensive to collect, are often mislabeled, and have large class imbalances. State-of-the-art approaches to address these challenges utilize Generative Adversarial Networks (GANs) for generating additional synthetic data along with their labels. Problematically, these HAR GANs only synthesize continuous features — features that are represented by real numbers — recorded from gyroscopes, accelerometers, and other sensors that produce continuous data. This is limiting since mobile sensor data commonly has discrete features that provide additional context such as device location and the time-of-day, which have been shown to substantially improve HAR classification. Hence, we studied Conditional Tabular Generative Adversarial Networks (CTGANs) for data generation to synthesize mobile sensor data containing both continuous and discrete features, a task never been done by state-of-the-art approaches. We show HAR-CTGANs generate data with greater realism resulting in allowing better downstream performance in HAR models, and when state-of-the-art models were modified with HAR-CTGAN characteristics, downstream performance also improves. 

   more » « less
4. Designing Deep Neural Networks Robust to Sensor Failure in Mobile Health Environments

   Mamun, Abdullah; Mirzadeh, Seyed Iman; Ghasemzadeh, Hassan (January 2022, The 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2022))

   Missing data is a very common challenge in health monitoring systems and one reason for that is that they are largely dependent on different types of sensors. A critical characteristic of the sensor-based prediction systems is their dependency on hardware, which is prone to physical limitations that add another layer of complexity to the algorithmic component of the system. For instance, it might not be realistic to assume that the prediction model has access to all sensors at all times. This can happen in the real-world setup if one or more sensors on a device malfunction or temporarily have to be disabled due to power limitations. The consequence of such a scenario is that the model faces ``missing input data'' from those unavailable sensors at the deployment time, and as a result, the quality of prediction can degrade significantly. While the missing input data is a very well-known problem, to the best of our knowledge, no study has been done to efficiently minimize the performance drop when one or more sensors may be unavailable for a significant amount of time. The sensor failure problem investigated in this paper can be viewed as a spatial missing data problem, which has not been explored to date. In this work, we show that the naive known methods of dealing with missing input data such as zero-filling or mean-filling are not suitable for sensor-based prediction and we propose an algorithm that can reconstruct the missing input data for unavailable sensors. Moreover, we show that on the MobiAct, MotionSense, and MHEALTH activity classification benchmarks, our proposed method can outperform the baselines by large accuracy margins of 8.2%, 15.1%, and 11.6%, respectively. 

   more » « less
5. Using Machine Learning to Train a Wearable Device for Measuring Students’ Cognitive Load during Problem-Solving Activities Based on Electrodermal Activity, Body Temperature, and Heart Rate: Development of a Cognitive Load Tracker for Both Personal and Classroom Use

   https://doi.org/10.3390/s20174833  

   Romine, William L.; Schroeder, Noah L.; Graft, Josephine; Yang, Fan; Sadeghi, Reza; Zabihimayvan, Mahdieh; Kadariya, Dipesh; Banerjee, Tanvi (September 2020, Sensors)

   null (Ed.)

   Automated tracking of physical fitness has sparked a health revolution by allowing individuals to track their own physical activity and health in real time. This concept is beginning to be applied to tracking of cognitive load. It is well known that activity in the brain can be measured through changes in the body’s physiology, but current real-time measures tend to be unimodal and invasive. We therefore propose the concept of a wearable educational fitness (EduFit) tracker. We use machine learning with physiological data to understand how to develop a wearable device that tracks cognitive load accurately in real time. In an initial study, we found that body temperature, skin conductance, and heart rate were able to distinguish between (i) a problem solving activity (high cognitive load), (ii) a leisure activity (moderate cognitive load), and (iii) daydreaming (low cognitive load) with high accuracy in the test dataset. In a second study, we found that these physiological features can be used to predict accurately user-reported mental focus in the test dataset, even when relatively small numbers of training data were used. We explain how these findings inform the development and implementation of a wearable device for temporal tracking and logging a user’s learning activities and cognitive load. 

   more » « less