skip to main content


Title: Easy-Assist: An Intelligent Haptic-based Affective framework for Assisted Living
Unlike the younger population that uses wearables such as smartwatches for monitoring health on a daily basis, elderly people need assistance in the use of technology and interpreting the data obtained through these smart connected frameworks. The current monitoring systems are primarily designed to monitor the physiological signals on a daily basis. The aim of this proposed research, Easy-Assist, is to help older people to maintain their emotional well-being. This research is focused on developing a wearable affective framework, which can help in detecting the emotions of the user in addition to monitoring their physiological signals. The proposed framework can be used in an automated assisted living environment, where the user's emotional state can be balanced using a haptic-based emotional elicitation system after the user's emotion is recognized, detected and interpreted in real-time. The proposed framework is validated using a fall detection algorithm deployed in a custom-built watch wearable, built using off-the-shelf components and an emotion detection framework built using a single board computer. A dataset of 21700 samples acquired using the proposed framework yielded a maximum efficiency of 97.25%, 96 %, and 94 %, in classifying the state and emotion classes into Alert, Active and Normal classes respectively, using multi-class SVM model. The overall latency of the proposed research was in few orders of milli-seconds.  more » « less
Award ID(s):
1924117 1924112
NSF-PAR ID:
10157977
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2020 IEEE International Conference on Consumer Electronics (ICCE)
Page Range / eLocation ID:
1 to 5
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Overindulgence of harmful substances such as drugs or alcohol, called substance abuse, can directly affect a person's health and their day-to-day activities. The younger population become more vulnerable to such use of psychoactive substances due to lack of awareness of the long-term hazardous effects these substances can have on their health. Additionally, these individuals tend to develop severe mental disorders as they grow older. With the boom of Internet of Things (IoT), the use of wearable sensors such as smartwatches and smartphones has tremendously increased. These wearables help in monitoring a person's physiological signal and keep them informed of one's health. In this research, we propose an edge-intelligent IoT-based wearable that can assist in substance-abuse detection by monitoring their physiological signals on daily basis. The proposed system helps in monitoring the substance abuse and craving of the individual and help the healthcare provider to start an early intervention as required. The proposed system is validated using a custom-built wearable, i-SAD, which was developed as a dedicated substance abuse wearable using commercially available off-the-shelf components. The proposed wearable design was validated using medical quality wearable and yielded a correlation of 0.89 for accelerometer values and 0.92 for average heart rate values. 
    more » « less
  2. We present a System for Processing In-situ Bio-signal Data for Emotion Recognition and Sensing (SPIDERS)- a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user's facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions (eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements), using the bio-signals acquired from three non-contact sensors (IR camera, proximity sensor, IMU). SPIDERS distinguishes between different classes of apparent and real emotion states based on the aforementioned four bio-signals. We prototype advanced functionalities including facial expression detection and real emotion classification with a landmarks and optical flow based facial expression detector that leverages changes in a user's eyebrows and eye shapes to achieve up to 83.87% accuracy, as well as a pupillometry-based real emotion classifier with higher accuracy than other low-cost wearable platforms that use sensors requiring skin contact. SPIDERS costs less than $20 to assemble and can continuously run for up to 9 hours before recharging. We demonstrate that SPIDERS is a truly wireless and portable platform that has the capability to impact a wide range of applications, where knowledge of the user's emotional state is critical. 
    more » « less
  3. Stress plays a critical role in our lives, impacting our productivity and our long-term physiological and psychological well-being. This has motivated the development of stress monitoring solutions to better understand stress, its impact on productivity and teamwork, and help users adapt their habits toward more sustainable stress levels. However, today's stress monitoring solutions remain obtrusive, requiring active user participation (e.g., self-reporting), interfering with people's daily activities, and often adding more burden to users looking to reduce their stress. In this paper, we introduce WiStress, the first system that can passively monitor a user's stress levels by relying on wireless signals. WiStress does not require users to actively provide input or to wear any devices on their bodies. It operates by transmitting ultra-low-power wireless signals and measuring their reflections off the user's body. WiStress introduces two key innovations. First, it presents the first machine learning network that can accurately and robustly extract heartbeat intervals (IBI's) from wireless reflections without constraints on a user's daily activities. Second, it introduces a stress classification framework that combines the extracted heartbeats with other wirelessly captured stress-related features in order to infer a subject's stress level. We built a prototype of WiStress and tested it on 22 different subjects across different environments in both stress-induced and free-living conditions. Our results demonstrate that WiStress has high accuracy (84%-95%) in inferring a person's stress level in a fully-automated way, paving the way for ubiquitous sensing systems that can monitor stress and provide feedback to improve productivity, health, and well-being. 
    more » « less
  4. Affective studies provide essential insights to address emotion recognition and tracking. In traditional open-loop structures, a lack of knowledge about the internal emotional state makes the system incapable of adjusting stimuli parameters and automatically responding to changes in the brain. To address this issue, we propose to use facial electromyogram measurements as biomarkers to infer the internal hidden brain state as feedback to close the loop. In this research, we develop a systematic way to track and control emotional valence, which codes emotions as being pleasant or obstructive. Hence, we conduct a simulation study by modeling and tracking the subject's emotional valence dynamics using state-space approaches. We employ Bayesian filtering to estimate the person-specific model parameters along with the hidden valence state, using continuous and binary features extracted from experimental electromyogram measurements. Moreover, we utilize a mixed-filter estimator to infer the secluded brain state in a real-time simulation environment. We close the loop with a fuzzy logic controller in two categories of regulation: inhibition and excitation. By designing a control action, we aim to automatically reflect any required adjustments within the simulation and reach the desired emotional state levels. Final results demonstrate that, by making use of physiological data, the proposed controller could effectively regulate the estimated valence state. Ultimately, we envision future outcomes of this research to support alternative forms of self-therapy by using wearable machine interface architectures capable of mitigating periods of pervasive emotions and maintaining daily well-being and welfare. 
    more » « less
  5. Suicide is a negative outcome of combination of complex personal, social and mental health factors, which forces the individual to consider it as the only way out to their problems. Suicide has become one of the most significant causes of death in the United States. A major cause of this ominous event is due to mental illnesses such as major depression and bipolar depression. With an aim to increase the awareness of suicide and improve the quality of life of people who are vulnerable or prone to suicidal ideation, this research focuses on developing an IoT-based framework that can help in continuously monitoring physiological and behavioral signals of the individual. The proposed framework, M-SID, is designed based on a hypothesis to capture rapid variation of suicidal ideation using physiological signals. The proposed research is validated with the help of a custom-built hardware and the results are verified using a commercially available wrist band. 
    more » « less