skip to main content


Title: Empirical Evaluation of Calibration and Long-term Carryover Effects of Reverberation on Egocentric Auditory Depth Perception in VR
Distance compression, which refers to the underestimation of ego-centric distance to objects, is a common problem in immersive virtual environments. Besides visually compensating the compressed distance, several studies have shown that auditory information can be an alternative solution for this problem. In particular, reverberation time (RT) has been proven to be an effective method to compensate distance compression. To further explore the feasibility of applying audio information to improve distance perception, we investigate whether users’ egocentric distance perception can be calibrated, and whether the calibrated effect can be carried over and even sustain for a longer duration. We conducted a study to understand the perceptual learning and carryover effects by using RT as stimuli for users to perceive distance in IVEs. The results show that the carryover effect exists after calibration, which indicates people can learn to perceive distances by attuning reverberation time, and the accuracy even remains a constant level after 6 months. Our findings could potentially be utilized to improve the distance perception in VR systems as the calibration of auditory distance perception in VR could sustain for several months. This could eventually avoid the burden of frequent training regimens.  more » « less
Award ID(s):
2007435
NSF-PAR ID:
10437067
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
IEEE International Conference on Virtual Reality and 3D User Interfaces (IEEE VR)
Page Range / eLocation ID:
232 to 240
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. As applications for virtual reality (VR) and augmented reality (AR) technology increase, it will be important to understand how users perceive their action capabilities in virtual environments. Feedback about actions may help to calibrate perception for action opportunities (affordances) so that action judgments in VR and AR mirror actors’ real abilities. Previous work indicates that walking through a virtual doorway while wielding an object can calibrate the perception of one’s passability through feedback from collisions. In the current study, we aimed to replicate this calibration through feedback using a different paradigm in VR while also testing whether this calibration transfers to AR. Participants held a pole at 45°and made passability judgments in AR (pretest phase). Then, they made passability judgments in VR and received feedback on those judgments by walking through a virtual doorway while holding the pole (calibration phase). Participants then returned to AR to make posttest passability judgments. Results indicate that feedback calibrated participants’ judgments in VR. Moreover, this calibration transferred to the AR environment. In other words, after experiencing feedback in VR, passability judgments in VR and in AR became closer to an actor’s actual ability, which could make training applications in these technologies more effective.

     
    more » « less
  2. This research investigates the effect of scaling in virtual reality to improve the reach of users with Parkinson’s disease (PD). People with PD have limited reach, often due to impaired postural stability. We investigated how virtual reality (VR) can improve reach during and after VR exposure. Participants played a VR game where they smashed water balloons thrown at them by crossing their midsection. The distance the balloons were thrown at increased and decreased based on success or failure. Their perception of the distance and their hand were scaled in three counterbalanced conditions: under-scaled (scale = 0:83), not-scaled (scale = 1), and over-scaled (scale = 1:2), where the scale value is the ratio between the virtual reach that they perceive in the virtual environment (VE) and their actual reach. In each study condition, six data were measured - 1. Real World Reach (pre-exposure), 2. Virtual Reality Baseline Reach, 3. Virtual Reality Not-Scaled Reach, 4. Under-Scaled Reach, 5. Over-Scaled Reach, and 6. Real World Reach (post-exposure). Our results show that scaling a person’s movement in virtual reality can help improve reach. Therefore, we recommend including a scaling factor in VR games for people with Parkinson’s disease. 
    more » « less
  3. While tremendous advances in visual and auditory realism have been made for virtual and augmented reality (VR/AR), introducing a plausible sense of physicality into the virtual world remains challenging. Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users. However, existing VR/AR solutions either completely ignore the force inputs from the users or rely on obtrusive sensing devices that compromise user experience. By identifying users' muscle activation patterns while engaging in VR/AR, we design a learning-based neural interface for natural and intuitive force inputs. Specifically, we show that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities. Fuelled by a neural-network-based model, our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. Through an interactive psychophysical study, we show that human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by our interface. We further demonstrate that our interface enables ubiquitous control via finger tapping. Ultimately, we envision our findings to push forward research towards more realistic physicality in future VR/AR. 
    more » « less
  4. Battery life is an increasingly urgent challenge for today's untethered VR and AR devices. However, the power efficiency of head-mounted displays is naturally at odds with growing computational requirements driven by better resolution, refresh rate, and dynamic ranges, all of which reduce the sustained usage time of untethered AR/VR devices. For instance, the Oculus Quest 2, under a fully-charged battery, can sustain only 2 to 3 hours of operation time. Prior display power reduction techniques mostly target smartphone displays. Directly applying smartphone display power reduction techniques, however, degrades the visual perception in AR/VR with noticeable artifacts. For instance, the "power-saving mode" on smartphones uniformly lowers the pixel luminance across the display and, as a result, presents an overall darkened visual perception to users if directly applied to VR content. Our key insight is that VR display power reduction must be cognizant of the gaze-contingent nature of high field-of-view VR displays. To that end, we present a gaze-contingent system that, without degrading luminance, minimizes the display power consumption while preserving high visual fidelity when users actively view immersive video sequences. This is enabled by constructing 1) a gaze-contingent color discrimination model through psychophysical studies, and 2) a display power model (with respect to pixel color) through real-device measurements. Critically, due to the careful design decisions made in constructing the two models, our algorithm is cast as a constrained optimization problem with a closed-form solution, which can be implemented as a real-time, image-space shader. We evaluate our system using a series of psychophysical studies and large-scale analyses on natural images. Experiment results show that our system reduces the display power by as much as 24% (14% on average) with little to no perceptual fidelity degradation. 
    more » « less
  5. null (Ed.)
    Abstract Background How the brain develops accurate models of the external world and generates appropriate behavioral responses is a vital question of widespread multidisciplinary interest. It is increasingly understood that brain signal variability—posited to enhance perception, facilitate flexible cognitive representations, and improve behavioral outcomes—plays an important role in neural and cognitive development. The ability to perceive, interpret, and respond to complex and dynamic social information is particularly critical for the development of adaptive learning and behavior. Social perception relies on oxytocin-regulated neural networks that emerge early in development. Methods We tested the hypothesis that individual differences in the endogenous oxytocinergic system early in life may influence social behavioral outcomes by regulating variability in brain signaling during social perception. In study 1, 55 infants provided a saliva sample at 5 months of age for analysis of individual differences in the oxytocinergic system and underwent electroencephalography (EEG) while listening to human vocalizations at 8 months of age for the assessment of brain signal variability. Infant behavior was assessed via parental report. In study 2, 60 infants provided a saliva sample and underwent EEG while viewing faces and objects and listening to human speech and water sounds at 4 months of age. Infant behavior was assessed via parental report and eye tracking. Results We show in two independent infant samples that increased brain signal entropy during social perception is in part explained by an epigenetic modification to the oxytocin receptor gene ( OXTR ) and accounts for significant individual differences in social behavior in the first year of life. These results are measure-, context-, and modality-specific: entropy, not standard deviation, links OXTR methylation and infant behavior; entropy evoked during social perception specifically explains social behavior only; and only entropy evoked during social auditory perception predicts infant vocalization behavior. Conclusions Demonstrating these associations in infancy is critical for elucidating the neurobiological mechanisms accounting for individual differences in cognition and behavior relevant to neurodevelopmental disorders. Our results suggest that an epigenetic modification to the oxytocin receptor gene and brain signal entropy are useful indicators of social development and may hold potential diagnostic, therapeutic, and prognostic value. 
    more » « less