Attention:The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, November 12th until 12:00 AM ET on Friday, November 13th due to maintenance. We apologize for the inconvenience.


Title: Navigation Training for Persons With Visual Disability Through Multisensory Assistive Technology: Mixed Methods Experimental Study
BackgroundVisual disability is a growing problem for many middle-aged and older adults. Conventional mobility aids, such as white canes and guide dogs, have notable limitations that have led to increasing interest in electronic travel aids (ETAs). Despite remarkable progress, current ETAs lack empirical evidence and realistic testing environments and often focus on the substitution or augmentation of a single sense. ObjectiveThis study aims to (1) establish a novel virtual reality (VR) environment to test the efficacy of ETAs in complex urban environments for a simulated visual impairment (VI) and (2) evaluate the impact of haptic and audio feedback, individually and combined, on navigation performance, movement behavior, and perception. Through this study, we aim to address gaps to advance the pragmatic development of assistive technologies (ATs) for persons with VI. MethodsThe VR platform was designed to resemble a subway station environment with the most common challenges faced by persons with VI during navigation. This environment was used to test our multisensory, AT-integrated VR platform among 72 healthy participants performing an obstacle avoidance task while experiencing symptoms of VI. Each participant performed the task 4 times: once with haptic feedback, once with audio feedback, once with both feedback types, and once without any feedback. Data analysis encompassed metrics such as completion time, head and body orientation, and trajectory length and smoothness. To evaluate the effectiveness and interaction of the 2 feedback modalities, we conducted a 2-way repeated measures ANOVA on continuous metrics and a Scheirer-Ray-Hare test on discrete ones. We also conducted a descriptive statistical analysis of participants’ answers to a questionnaire, assessing their experience and preference for feedback modalities. ResultsResults from our study showed that haptic feedback significantly reduced collisions (P=.05) and the variability of the pitch angle of the head (P=.02). Audio feedback improved trajectory smoothness (P=.006) and mitigated the increase in the trajectory length from haptic feedback alone (P=.04). Participants reported a high level of engagement during the experiment (52/72, 72%) and found it interesting (42/72, 58%). However, when it came to feedback preferences, less than half of the participants (29/72, 40%) favored combined feedback modalities. This indicates that a majority preferred dedicated single modalities over combined ones. ConclusionsAT is crucial for individuals with VI; however, it often lacks user-centered design principles. Research should prioritize consumer-oriented methodologies, testing devices in a staged manner with progression toward more realistic, ecologically valid settings to ensure safety. Our multisensory, AT-integrated VR system takes a holistic approach, offering a first step toward enhancing users’ spatial awareness, promoting safer mobility, and holds potential for applications in medical treatment, training, and rehabilitation. Technological advancements can further refine such devices, significantly improving independence and quality of life for those with VI.  more » « less
Award ID(s):
2345139 2236097 2129076 1928614
PAR ID:
10559615
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
JMIR Publications
Date Published:
Journal Name:
JMIR Rehabilitation and Assistive Technologies
Volume:
11
ISSN:
2369-2529
Page Range / eLocation ID:
e55776
Subject(s) / Keyword(s):
Sponsoring Org:
National Science Foundation
More Like this
  1. BackgroundAugmented sensory feedback (ASF) in virtual reality (VR) can enhance motor performance, yet it is not standardly employed with rehabilitation protocols and its effects on physiological and perceptual states remain underexplored. AimThis study examined how different ASF modalities—visual, haptic, and combined visual + haptic—modulate motor performance, psychophysiological responses, and user perceptions during an upper-limb VR training task. MethodsTwenty neurotypical adults controlled a virtual robotic arm via semi-isometric muscle contractions, while receiving one of four ASF training conditions: no feedback (NF), visual feedback (VF), haptic feedback (HF), or combined (multimodal) visual + haptic feedback (VHF). Improved performance (minimizing motion pathlengths and task completion time), changes in physiological signals (EEG band power, EMG amplitude, electrodermal activity, heart rate), and perceptual ratings (agency, motivation, utility) were assessed before and after each training condition. ResultsVF produced greater efficiency in motor output with improved performance (primary metrics of study) in conjunction with reduced EMG, along with increased electrodermal activity suggestive of higher arousal. VHF elicited significant post-training increases in EEG alpha and beta power. Motivation and utility ratings were significantly higher for VF and VHF compared to HF and NF, while agency ratings remained stable. Across all conditions, improved performance correlated with increased alpha power, reduced EMG and heart rate, and higher motivation and utility. ConclusionThese findings indicate that ASF modality differentially shapes motor, physiological, and perceptual responses. Future work should establish whether these responses generalize to clinical groups, such as those with neuromotor impairment. Ultimately, adaptive VR systems leveraging psychophysiological responses to optimize feedback in real time—balancing exertion, cognitive load, and engagement during rehabilitative training—may be key to accelerating gains in motor function. 
    more » « less
  2. null (Ed.)
    Technological advancements and increased access have prompted the adoption of head- mounted display based virtual reality (VR) for neuroscientific research, manual skill training, and neurological rehabilitation. Applications that focus on manual interaction within the virtual environment (VE), especially haptic-free VR, critically depend on virtual hand-object collision detection. Knowledge about how multisensory integration related to hand-object collisions affects perception-action dynamics and reach-to-grasp coordination is needed to enhance the immersiveness of interactive VR. Here, we explored whether and to what extent sensory substitution for haptic feedback of hand-object collision (visual, audio, or audiovisual) and collider size (size of spherical pointers representing the fingertips) influences reach-to-grasp kinematics. In Study 1, visual, auditory, or combined feedback were compared as sensory substitutes to indicate the successful grasp of a virtual object during reach-to-grasp actions. In Study 2, participants reached to grasp virtual objects using spherical colliders of different diameters to test if virtual collider size impacts reach-to-grasp. Our data indicate that collider size but not sensory feedback modality significantly affected the kinematics of grasping. Larger colliders led to a smaller size-normalized peak aperture. We discuss this finding in the context of a possible influence of spherical collider size on the perception of the virtual object’s size and hence effects on motor planning of reach-to-grasp. Critically, reach-to-grasp spatiotemporal coordination patterns were robust to manipulations of sensory feedback modality and spherical collider size, suggesting that the nervous system adjusted the reach (transport) component commensurately to the changes in the grasp (aperture) component. These results have important implications for research, commercial, industrial, and clinical applications of VR. 
    more » « less
  3. Spinal cord injury (SCI) impairs motor function and requires rigorous rehabilitative therapy, motivating the development of approaches that are engaging and customizable. Virtual reality (VR) motor training with augmented sensory feedback (ASF) offers a promising pathway to enhance functional outcomes, yet it remains unclear how ASF modalities affect performance and underlying psychophysiological states in persons with SCI. Five participants with chronic incomplete cervical-level SCI controlled a virtual robotic arm with semi-isometric upper-body contractions while undergoing ASF training with either visual feedback (VF) or combined visual plus haptic feedback (VHF). Motor performance (pathlength, completion time), psychophysiological measures (EEG, EMG, EDA, HR), and perceptual ratings (agency, motivation, utility) were assessed before and after ASF training. VF significantly reduced pathlength (−12.5%, p = 0.0011) and lowered EMG amplitude (−32.5%, p = 0.0063), suggesting the potential for improved motor performance and neuromuscular efficiency. VHF did not significantly improve performance, but trended toward higher cortical engagement. EEG analyses showed VF significantly decreased alpha and beta activity after training, whereas VHF trended toward mild increases. Regression revealed improved performance was significantly (p < 0.05) associated with changes in alpha power, EMG, EDA, and self-reported motivation. ASF type may differentially shape performance and psychophysiological responses in SCI participants. These preliminary findings suggest VR-based ASF as a potent multidimensional tool for personalizing rehabilitation. 
    more » « less
  4. IntroductionThis study examined how motor task difficulty and augmented sensory feedback (ASF) modality influence task performance, physiological responses, and perceptual experience during virtual reality (VR)-based upper-limb motor training. Building on related brace- and VR-based ASF platforms previously examined in neurotypical and spinal cord injury cohorts, the long-term goal is to support neurorehabilitation-oriented adaptive VR training; however, the present study used neurotypical participants as an initial validation step. MethodsFourteen right-handed neurotypical participants used a supportive upper-limb brace and surface electromyography (EMG) for myoelectric control of a virtual robotic arm in a two-dimensional tracking task. A 2 × 2 within-subject design crossed ASF modality (visual feedback versus visual + haptic feedback) with task difficulty (straight-line versus sinusoidal trajectory). Outcomes included normalized path length, task completion time, movement jerk, alpha- and beta-band electroencephalogram (EEG) power over motor cortex, heart rate, EMG amplitude, EMG-displacement-based virtual work, and perceptual ratings of agency, motivation, and utility. Short-term pre-to-post percentage changes were computed for each measure. ResultsVHF straight-line training produced the largest reductions in normalized path length (-4.69%) and completion time (-6.12%), whereas VHF sinusoid training produced the largest reductions in movement jerk (-5.67%) and virtual work (-13.33%). ASF modality affected path length, completion time, jerk, EMG amplitude, and virtual work, whereas task difficulty modulated beta-band EEG power and the physiological index. Changes in path length were positively associated with changes in completion time and EMG amplitude. DiscussionThese findings suggest that ASF modality and task difficulty can differentially shape short-term behavioral, physiological, and perceptual responses in healthy participants. The exploratory distribution of participant-specific best-case conditions motivates future studies using independent trial subsets to evaluate whether behavioral and physiological measures can support individualized VR training. 
    more » « less
  5. Kim, Jaehwan (Ed.)
    Visual impairment represents a critical challenge for our society with 285 million affect worldwide; alarmingly, the prevalence is expected to triple by 2050. Supporting mobility is a chief priority for assistive technologies. In recent years, the integration of computer vision and haptic technologies has led to a number of wearable electronic travel aids (ETAs). Previously, we have proposed an ETA comprised of a computer vision system and a wearable haptic device in the form of a belt. The belt encompasses a two-by-five array of piezoelectric-based macrofiber composite (MFC) actuators, which can generate vibrations on the abdomen when an oscillating voltage is applied across their electrodes. The computer vision system identifies position and distance of surrounding obstacles egocentrically and drives the actuators relative to the salience of the potential hazard(s). Despite promising pilots, the design, control, and optimization of the ETA requires substantial, potentially high-risk, and tedious training and testing to accommodate patient-specific behavioral idiosyncrasies and a variety of visual impairments. To address these issues, we employ a virtual reality (VR) platform that offers simulations of visual impairment by disease type and severity with front-end control. We review our early work on the first three visual impairments piloted in the platform, each with three levels of severity: mild, moderate and severe. The VR environment is interfaced with the ETA, which provides feedback to the user based on the position of virtual obstacles. These simulations allow safe, controlled, repeatable experiments with ETAs that can be performed with varying degrees of visual perception. Our framework can become a paradigm for the development and testing of ETAs, with other potential applications in disability awareness, education, and training. 
    more » « less