skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Experimental Study of Long Short-Term Memory and Transformer Models for Fall Detection on Smartwatches
Falls are the second leading cause of unintentional injury deaths worldwide. While numerous wearable fall detection devices incorporating AI models have been developed, none of them are used successfully in a fall detection application running on commodity-based smartwatches in real time. The system misses some falls, and generates an annoying amount of False Positives for practical use. We have investigated and experimented with an LSTM model for fall detection on a smartwatch. Even though the LSTM model has high accuracy during offline testing, the good performance of offline LSTM models cannot be translated to the equivalence of real-time performance. Transformers, on the other hand, can learn long-sequence data and patterns intrinsic to the data due to their self-attention mechanism. This paper compares three variants of LSTM and two variants of Transformer models for learning fall patterns. We trained all models using fall and activity data from three datasets, and the real-time testing of the model was performed using the SmartFall App. Our findings showed that in the offline training, the CNN-LSTM model was better than the Transformer model for all the datasets. However, the Transformer is a preferable choice for deployment in real-time fall detection applications.  more » « less
Award ID(s):
2123749
PAR ID:
10626687
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Sensors
Date Published:
Journal Name:
Sensors
Volume:
24
Issue:
19
ISSN:
1424-8220
Page Range / eLocation ID:
6235
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, Applied Sciences)
    Real-time fall detection using a wearable sensor remains a challenging problem due to high gait variability. Furthermore, finding the type of sensor to use and the optimal location of the sensors are also essential factors for real-time fall-detection systems. This work presents real-time fall-detection methods using deep learning models. Early detection of falls, followed by pneumatic protection, is one of the most effective means of ensuring the safety of the elderly. First, we developed and compared different data-segmentation techniques for sliding windows. Next, we implemented various techniques to balance the datasets because collecting fall datasets in the real-time setting has an imbalanced nature. Moreover, we designed a deep learning model that combines a convolution-based feature extractor and deep neural network blocks, the LSTM block, and the transformer encoder block, followed by a position-wise feedforward layer. We found that combining the input sequence with the convolution-learned features of different kernels tends to increase the performance of the fall-detection model. Last, we analyzed that the sensor signals collected by both accelerometer and gyroscope sensors can be leveraged to develop an effective classifier that can accurately detect falls, especially differentiating falls from near-falls. Furthermore, we also used data from sixteen different body parts and compared them to determine the better sensor position for fall-detection methods. We found that the shank is the optimal position for placing our sensors, with an F1 score of 0.97, and this could help other researchers collect high-quality fall datasets. 
    more » « less
  2. ; ; ; (, Sensors)
    Deep learning models rely heavily on extensive training data, but obtaining sufficient real-world data remains a major challenge in clinical fields. To address this, we explore methods for generating realistic synthetic multivariate fall data to supplement limited real-world samples collected from three fall-related datasets: SmartFallMM, UniMib, and K-Fall. We apply three conventional time-series augmentation techniques, a Diffusion-based generative AI method, and a novel approach that extracts fall segments from public video footage of older adults. A key innovation of our work is the exploration of two distinct approaches: video-based pose estimation to extract fall segments from public footage, and Diffusion models to generate synthetic fall signals. Both methods independently enable the creation of highly realistic and diverse synthetic data tailored to specific sensor placements. To our knowledge, these approaches and especially their application in fall detection represent rarely explored directions in this research area. To assess the quality of the synthetic data, we use quantitative metrics, including the Fréchet Inception Distance (FID), Discriminative Score, Predictive Score, Jensen–Shannon Divergence (JSD), and Kolmogorov–Smirnov (KS) test, and visually inspect temporal patterns for structural realism. We observe that Diffusion-based synthesis produces the most realistic and distributionally aligned fall data. To further evaluate the impact of synthetic data, we train a long short-term memory (LSTM) model offline and test it in real time using the SmartFall App. Incorporating Diffusion-based synthetic data improves the offline F1-score by 7–10% and boosts real-time fall detection performance by 24%, confirming its value in enhancing model robustness and applicability in real-world settings. 
    more » « less
  3. ; ; ; ; (, Sensors)
    Falls in the elderly are associated with significant morbidity and mortality. While numerous fall detection devices incorporating AI and machine learning algorithms have been developed, no known smartwatch-based system has been used successfully in real-time to detect falls for elderly persons. We have developed and deployed a SmartFall system on a commodity-based smartwatch which has been trialled by nine elderly participants. The system, while being usable and welcomed by the participants in our trials, has two serious limitations. The first limitation is the inability to collect a large amount of personalized data for training. When the fall detection model, which is trained with insufficient data, is used in the real world, it generates a large amount of false positives. The second limitation is the model drift problem. This means an accurate model trained using data collected with a specific device performs sub-par when used in another device. Therefore, building one model for each type of device/watch is not a scalable approach for developing smartwatch-based fall detection system. To tackle those issues, we first collected three datasets including accelerometer data for fall detection problem from different devices: the Microsoft watch (MSBAND), the Huawei watch, and the meta-sensor device. After that, a transfer learning strategy was applied to first explore the use of transfer learning to overcome the small dataset training problem for fall detection. We also demonstrated the use of transfer learning to generalize the model across the heterogeneous devices. Our preliminary experiments demonstrate the effectiveness of transfer learning for improving fall detection, achieving an F1 score higher by over 10% on average, an AUC higher by over 0.15 on average, and a smaller false positive prediction rate than the non-transfer learning approach across various datasets collected using different devices with different hardware specifications. 
    more » « less
  4. ; ; ; ; (, IEEE Big Data 2023)
    Social media cyberbullying has a detrimental effect on human life. As online social networking grows daily, the amount of hate speech also increases. Such terrible content can cause depression and actions related to suicide. This paper proposes a trustable LSTM Autoencoder Network for cyberbullying detection on social media using synthetic data. We have demonstrated a cutting-edge method to address data availability difficulties by producing machine-translated data. However, several languages such as Hindi and Bangla still lack adequate investigations due to a lack of datasets. We carried out experimental identification of aggressive comments on Hindi, Bangla, and English datasets using the proposed model and traditional models, including Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), LSTM-Autoencoder, Word2vec, Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-trained Transformer 2 (GPT-2) models. We employed evaluation metrics such as f1-score, accuracy, precision, and recall to assess the models’ performance. Our proposed model outperformed all the models on all datasets, achieving the highest accuracy of 95%. Our model achieves state-of-the-art results among all the previous works on the dataset we used in this paper. 
    more » « less
  5. ; ; ; (, Microchimica Acta)
    A portable electrochemical aptasensor integrated with machine learning was developed for rapid and on-site detection of Staphylococcus aureus (S. aureus) in food and beverage samples. The aptasensor was fabricated using screen-printed carbon electrodes (SPCEs) modified with gold nanoparticles (AuNPs) and functionalized with an Iron-regulated Surface Determinant Protein A (IsdA)-specific aptamer for the detection of S. aureus. Approximately 2,000 cyclic voltammetry (CV) data points were collected for six different food and beverage matrices spiked with varying concentrations of S. aureus (1, 10, 500, and 1000 colony-forming unit (CFU)/mL). Each CV scan was repeated 10 times, linearly averaged, and baseline corrected before model input. Noise filtering and normalization were performed to ensure consistent feature representation across training and testing datasets. Machine learning models, including Convolutional Neural Networks (CNNs) and Transformer architectures, were applied to classify the samples. The CNN model demonstrated superior performance, with a test loss of 0.0402 and a test accuracy of 99.21%. In contrast, the Transformer model achieved a test loss of 0.2014 and an accuracy of 94.21%. To enhance usability, an Android application was developed using the Network Enabled Technologies (NET) framework, enabling real-time inference of bacterial concentration directly from CV data on mobile devices (e.g. smartphones). This system demonstrates potential for a rapid, accurate, and scalable solution for real-world food safety monitoring. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email