More Like this

1. Wearable Sensor-Based Step Length Estimation During Overground Locomotion Using a Deep Convolutional Neural Network

   https://doi.org/10.1109/EMBC46164.2021.9630060  

   Jin, Heejoo; Kang, Inseung; Choi, Gayeon; Molinaro, Dean D.; Young, Aaron J. (November 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC))

   Step length is a critical gait parameter that allows a quantitative assessment of gait asymmetry. Gait asymmetry can lead to many potential health threats such as joint degeneration, difficult balance control, and gait inefficiency. Therefore, accurate step length estimation is essential to understand gait asymmetry and provide appropriate clinical interventions or gait training programs. The conventional method for step length measurement relies on using foot-mounted inertial measurement units (IMUs). However, this may not be suitable for real-world applications due to sensor signal drift and the potential obtrusiveness of using distal sensors. To overcome this challenge, we propose a deep convolutional neural network-based step length estimation using only proximal wearable sensors (hip goniometer, trunk IMU, and thigh IMU) capable of generalizing to various walking speeds. To evaluate this approach, we utilized treadmill data collected from sixteen able-bodied subjects at different walking speeds. We tested our optimized model on the overground walking data. Our CNN model estimated the step length with an average mean absolute error of 2.89 ± 0.89 cm across all subjects and walking speeds. Since wearable sensors and CNN models are easily deployable in real-time, our study findings can provide personalized real-time step length monitoring in wearable assistive devices and gait training programs. 

   more » « less
2. A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control

   https://doi.org/10.1017/wtc.2022.18  

   Zhang, Qiang; Fragnito, Natalie; Bao, Xuefeng; Sharma, Nitin (January 2022, Wearable Technologies)

   Abstract Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging’s region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ( $ p $  < .001) and increased the prediction coefficient of determination by 20.13% ( $ p $  < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy. 

   more » « less
3. A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

   https://doi.org/10.3390/jimaging10050103  

   Hao, Yu; Yang, Fan; Huang, Hao; Yuan, Shuaihang; Rangan, Sundeep; Rizzo, John-Ross; Wang, Yao; Fang, Yi (May 2024, Journal of Imaging)

   People with blindness and low vision (pBLV) encounter substantial challenges when it comes to comprehensive scene recognition and precise object identification in unfamiliar environments. Additionally, due to the vision loss, pBLV have difficulty in accessing and identifying potential tripping hazards independently. Previous assistive technologies for the visually impaired often struggle in real-world scenarios due to the need for constant training and lack of robustness, which limits their effectiveness, especially in dynamic and unfamiliar environments, where accurate and efficient perception is crucial. Therefore, we frame our research question in this paper as: How can we assist pBLV in recognizing scenes, identifying objects, and detecting potential tripping hazards in unfamiliar environments, where existing assistive technologies often falter due to their lack of robustness? We hypothesize that by leveraging large pretrained foundation models and prompt engineering, we can create a system that effectively addresses the challenges faced by pBLV in unfamiliar environments. Motivated by the prevalence of large pretrained foundation models, particularly in assistive robotics applications, due to their accurate perception and robust contextual understanding in real-world scenarios induced by extensive pretraining, we present a pioneering approach that leverages foundation models to enhance visual perception for pBLV, offering detailed and comprehensive descriptions of the surrounding environment and providing warnings about potential risks. Specifically, our method begins by leveraging a large-image tagging model (i.e., Recognize Anything Model (RAM)) to identify all common objects present in the captured images. The recognition results and user query are then integrated into a prompt, tailored specifically for pBLV, using prompt engineering. By combining the prompt and input image, a vision-language foundation model (i.e., InstructBLIP) generates detailed and comprehensive descriptions of the environment and identifies potential risks in the environment by analyzing environmental objects and scenic landmarks, relevant to the prompt. We evaluate our approach through experiments conducted on both indoor and outdoor datasets. Our results demonstrate that our method can recognize objects accurately and provide insightful descriptions and analysis of the environment for pBLV. 

   more » « less
4. A Lightweight Exoskeleton-Based Portable Gait Data Collection System

   https://doi.org/10.3390/s21030781  

   Haque, Md Rejwanul; Imtiaz, Masudul H.; Kwak, Samuel T.; Sazonov, Edward; Chang, Young-Hui; Shen, Xiangrong (February 2021, Sensors)

   null (Ed.)

   For the controller of wearable lower-limb assistive devices, quantitative understanding of human locomotion serves as the basis for human motion intent recognition and joint-level motion control. Traditionally, the required gait data are obtained in gait research laboratories, utilizing marker-based optical motion capture systems. Despite the high accuracy of measurement, marker-based systems are largely limited to laboratory environments, making it nearly impossible to collect the desired gait data in real-world daily-living scenarios. To address this problem, the authors propose a novel exoskeleton-based gait data collection system, which provides the capability of conducting independent measurement of lower limb movement without the need for stationary instrumentation. The basis of the system is a lightweight exoskeleton with articulated knee and ankle joints. To minimize the interference to a wearer’s natural lower-limb movement, a unique two-degrees-of-freedom joint design is incorporated, integrating a primary degree of freedom for joint motion measurement with a passive degree of freedom to allow natural joint movement and improve the comfort of use. In addition to the joint-embedded goniometers, the exoskeleton also features multiple positions for the mounting of inertia measurement units (IMUs) as well as foot-plate-embedded force sensing resistors to measure the foot plantar pressure. All sensor signals are routed to a microcontroller for data logging and storage. To validate the exoskeleton-provided joint angle measurement, a comparison study on three healthy participants was conducted, which involves locomotion experiments in various modes, including overground walking, treadmill walking, and sit-to-stand and stand-to-sit transitions. Joint angle trajectories measured with an eight-camera motion capture system served as the benchmark for comparison. Experimental results indicate that the exoskeleton-measured joint angle trajectories closely match those obtained through the optical motion capture system in all modes of locomotion (correlation coefficients of 0.97 and 0.96 for knee and ankle measurements, respectively), clearly demonstrating the accuracy and reliability of the proposed gait measurement system. 

   more » « less
5. Continuous Classification of Locomotion in Response to Task Complexity and Anticipatory State

   https://doi.org/10.3389/fbioe.2021.628050  

   Kazemimoghadam, Mahdieh; Fey, Nicholas P. (April 2021, Frontiers in Bioengineering and Biotechnology)

   Objective Intent recognition in lower-extremity assistive devices (e.g., prostheses and exoskeletons) is typically limited to either recognition of steady-state locomotion or changes of terrain (e.g., level ground to stair) occurring in a straight-line path and under anticipated condition. Stability is highly affected during non-steady changes of direction such as cuts especially when they are unanticipated, posing high risk of fall-related injuries. Here, we studied the influence of changes of direction and user anticipation on task recognition, and accordingly introduced classification schemes accommodating such effects. Methods A linear discriminant analysis (LDA) classifier continuously classified straight-line walking, sidestep/crossover cuts (single transitions), and cuts-to-stair locomotion (mixed transitions) performed under varied task anticipatory conditions. Training paradigms with varying levels of anticipated/unanticipated exposures and analysis windows of size 100–600 ms were examined. Results More accurate classification of anticipated relative to unanticipated tasks was observed. Including bouts of target task in the training data was necessary to improve generalization to unanticipated locomotion. Only up to two bouts of target task were sufficient to reduce errors to <20% in unanticipated mixed transitions, whereas, in single transitions and straight walking, substantial unanticipated information (i.e., five bouts) was necessary to achieve similar outcomes. Window size modifications did not have a significant influence on classification performance. Conclusion Adjusting the training paradigm helps to achieve classification schemes capable of adapting to changes of direction and task anticipatory state. Significance The findings could provide insight into developing classification schemes that can adapt to changes of direction and user anticipation. They could inform intent recognition strategies for controlling lower-limb assistive to robustly handle “unknown” circumstances, and thus deliver increased level of reliability and safety. 

   more » « less