skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Predicting Lower Extremity Joint Kinematics Using Multi-Modal Data in the Lab and Outdoor Environment
Predicting future walking joint kinematics is crucial for assistive device control, especially in variable walking environments. Traditional optical motion capture systems provide kinematics data but require laborious post-processing, whereas IMU based systems provide direct calculations but add delays due to data collection and algorithmic processes. Predicting future kinematics helps to compensate for these delays, enabling the system real-time. Furthermore, these predicted kinematics could serve as target trajectories for assistive devices such as exoskeletal robots and lower limb prostheses. However, given the complexity of human mobility and environmental factors, this prediction remains to be challenging. To address this challenge, we propose the Dual-ED-Attention-FAM-Net, a deep learning model utilizing two encoders, two decoders, a temporal attention module, and a feature attention module. Our model outperforms the state-of-the-art LSTM model. Specifically, for Dataset A, using IMUs and a combination of IMUs and videos, RMSE values decrease from 4.45° to 4.22° and from 4.52° to 4.15°, respectively. For Dataset B, IMUs and IMUs combined with pressure insoles result in RMSE reductions from 7.09° to 6.66° and from 7.20° to 6.77°, respectively. Additionally, incorporating other modalities alongside IMUs helps improve the performance of the model.  more » « less
Award ID(s):
2246671
PAR ID:
10518485
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
University of Hawaii at Manoa, Association for Information Systems IEEE Computer Society Press
Date Published:
Journal Name:
Proceedings of the Hawaii International Conference on System Sciences
ISSN:
0073-1129
Format(s):
Medium: X
Location:
Hawaii
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC))
    Step length is a critical gait parameter that allows a quantitative assessment of gait asymmetry. Gait asymmetry can lead to many potential health threats such as joint degeneration, difficult balance control, and gait inefficiency. Therefore, accurate step length estimation is essential to understand gait asymmetry and provide appropriate clinical interventions or gait training programs. The conventional method for step length measurement relies on using foot-mounted inertial measurement units (IMUs). However, this may not be suitable for real-world applications due to sensor signal drift and the potential obtrusiveness of using distal sensors. To overcome this challenge, we propose a deep convolutional neural network-based step length estimation using only proximal wearable sensors (hip goniometer, trunk IMU, and thigh IMU) capable of generalizing to various walking speeds. To evaluate this approach, we utilized treadmill data collected from sixteen able-bodied subjects at different walking speeds. We tested our optimized model on the overground walking data. Our CNN model estimated the step length with an average mean absolute error of 2.89 ± 0.89 cm across all subjects and walking speeds. Since wearable sensors and CNN models are easily deployable in real-time, our study findings can provide personalized real-time step length monitoring in wearable assistive devices and gait training programs. 
    more » « less
  2. ; ; ; ; (, Scientific Data)
    Abstract Tasks of daily living are often sporadic, highly variable, and asymmetric. Analyzing these real-world non-cyclic activities is integral for expanding the applicability of exoskeletons, protheses, wearable sensing, and activity classification to real life, and could provide new insights into human biomechanics. Yet, currently available biomechanics datasets focus on either highly consistent, continuous, and symmetric activities, such as walking and running, or only a single specific non-cyclic task. To capture a more holistic picture of lower limb movements in everyday life, we collected data from 12 participants performing 20 non-cyclic activities (e.g. sit-to-stand, jumping, squatting, lunging, cutting) as well as 11 cyclic activities (e.g. walking, running) while kinematics (motion capture and IMUs), kinetics (force plates), and electromyography (EMG) were collected. This dataset provides normative biomechanics for a highly diverse range of activities and common tasks from a consistent set of participants and sensors. 
    more » « less
  3. ; ; ; ; (, IEEE Transactions on Biomedical Engineering)
    Objective: Real-time measurement of biological joint moment could enhance clinical assessments and generalize exoskeleton control. Accessing joint moments outside clinical and laboratory settings requires harnessing non-invasive wearable sensor data for indirect estimation. Previous approaches have been primarily validated during cyclic tasks, such as walking, but these methods are likely limited when translating to non-cyclic tasks where the mapping from kinematics to moments is not unique. Methods: We trained deep learning models to estimate hip and knee joint moments from kinematic sensors, electromyography (EMG), and simulated pressure insoles from a dataset including 10 cyclic and 18 non-cyclic activities. We assessed estimation error on combinations of sensor modalities during both activity types. Results: Compared to the kinematics-only baseline, adding EMG reduced RMSE by 16.9% at the hip and 30.4% at the knee (p<0.05) and adding insoles reduced RMSE by 21.7% at the hip and 33.9% at the knee (p<0.05). Adding both modalities reduced RMSE by 32.5% at the hip and 41.2% at the knee (p<0.05) which was significantly higher than either modality individually (p<0.05). All sensor additions improved model performance on non-cyclic tasks more than cyclic tasks (p<0.05). Conclusion: These results demonstrate that adding kinetic sensor information through EMG or insoles improves joint moment estimation both individually and jointly. These additional modalities are most important during non-cyclic tasks, tasks that reflect the variable and sporadic nature of the real-world. Significance: Improved joint moment estimation and task generalization is pivotal to developing wearable robotic systems capable of enhancing mobility in everyday life. 
    more » « less
  4. (, Proceedings of the 17th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering)
    Inertial measurement units (IMUs) could provide an attractive alternative to video motion capture systems for measuring walking in a non-laboratory setting. This study applied an optimization method to an 18-DOF lower body model to convert synthetic IMU data into corresponding joint angles. 
    more » « less
  5. ; ; ; ; ; ; (, 2019 Design of Medical Devices Conference)
    This paper presents a portable inertial measurement unit (IMU)-based motion sensing system and proposed an adaptive gait phase detection approach for non-steady state walking and multiple activities (walking, running, stair ascent, stair descent, squat) monitoring. The algorithm aims to overcome the limitation of existing gait detection methods that are time-domain thresholding based for steady-state motion and are not versatile to detect gait during different activities or different gait patterns of the same activity. The portable sensing suit is composed of three IMU sensors (wearable sensors for gait phase detection) and two footswitches (ground truth measurement and not needed for gait detection of the proposed algorithm). The acceleration, angular velocity, Euler angle, resultant acceleration, and resultant angular velocity from three IMUs are used as the input training data and the data of two footswitches used as the training label data (single support, double support, swing phase). Three methods 1) Logistic Regression (LR), 2) Random Forest Classifier (RF), and 3) Artificial Neural Network (NN) are used to build the gait phase detection models. The result shows our proposed gait phase detection with Random Forest Classifier can achieve 98.94% accuracy in walking, 98.45% in running, 99.15% in stair-ascent, 99.00% in stair-descent, and 99.63% in squatting. It demonstrates that our sensing suit can not only detect the gait status in any transient state but also generalize to multiple activities. Therefore, it can be implemented in real-time monitoring of human gait and control of assistive devices. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email