Our brain perceives the world by exploiting multisensory cues to extract information about various aspects of external stimuli. The sensory cues from the same stimulus should be integrated to improve perception, and otherwise segregated to distinguish different stimuli. In reality, however, the brain faces the challenge of recognizing stimuli without knowing in advance the sources of sensory cues. To address this challenge, we propose that the brain conducts integration and segregation concurrently with complementary neurons. Studying the inference of heading-direction via visual and vestibular cues, we develop a network model with two reciprocally connected modules modeling interacting visual-vestibular areas. In each module, there are two groups of neurons whose tunings under each sensory cue are either congruent or opposite. We show that congruent neurons implement integration, while opposite neurons compute cue disparity information for segregation, and the interplay between two groups of neurons achieves efficient multisensory information processing.
more »
« less
Influence of sensory modality and control dynamics on human path integration
Path integration is a sensorimotor computation that can be used to infer latent dynamical states by integrating self-motion cues. We studied the influence of sensory observation (visual/vestibular) and latent control dynamics (velocity/acceleration) on human path integration using a novel motion-cueing algorithm. Sensory modality and control dynamics were both varied randomly across trials, as participants controlled a joystick to steer to a memorized target location in virtual reality. Visual and vestibular steering cues allowed comparable accuracies only when participants controlled their acceleration, suggesting that vestibular signals, on their own, fail to support accurate path integration in the absence of sustained acceleration. Nevertheless, performance in all conditions reflected a failure to fully adapt to changes in the underlying control dynamics, a result that was well explained by a bias in the dynamics estimation. This work demonstrates how an incorrect internal model of control dynamics affects navigation in volatile environments in spite of continuous sensory feedback.
more »
« less
- Award ID(s):
- 1707400
- NSF-PAR ID:
- 10380463
- Date Published:
- Journal Name:
- eLife
- Volume:
- 11
- ISSN:
- 2050-084X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Redirected and amplified head movements have the potential to provide more natural interaction with virtual environments (VEs) than using controller-based input, which causes large discrepancies between visual and vestibular self-motion cues and leads to increased VR sickness. However, such amplified head movements may also exacerbate VR sickness symptoms over no amplification. Several general methods have been introduced to reduce VR sickness for controller-based input inside a VE, including a popular vignetting method that gradually reduces the field of view. In this paper, we investigate the use of vignetting to reduce VR sickness when using amplified head rotations instead of controllerbased input. We also investigate whether the induced VR sickness is a result of the user’s head acceleration or velocity by introducing two different modes of vignetting, one triggered by acceleration and the other by velocity. Our dependent measures were pre and post VR sickness questionnaires as well as estimated discomfort levels that were assessed each minute of the experiment. Our results show interesting effects between a baseline condition without vignetting, as well as the two vignetting methods, generally indicating that the vignetting methods did not succeed in reducing VR sickness for most of the participants and, instead, lead to a significant increase. We discuss the results and potential explanations of our findings.more » « less
-
Objective: To determine if a vestibular prosthesis could improve function in subjects with severe vestibular damage and could be used it as a scientific tool to investigate central vestibular processing. Background: Damage to the vestibular labyrinth is common and usually permanent. We therefore developed and tested a vestibular implant (VI) that is designed to mimic the information normally provided by the vestibular labyrinth to determine if we can reduce vestibular-mediated deficits and study temporal integration of sensory cues in the brain. Design/Methods: Monkeys had electrodes implanted in the semicircular canals of one ear and then severe bilateral vestibular damage was induced with aminoglycosides. Eye movements, perception, and balance were tested before and after vestibular damage and with the VI activated, which supplied head motion information to the brain via electrical stimulation delivered by the implanted electrodes. Humans also had electrode implantation (done in conjunction with a cochlear implant, CI) and they were tested on a temporal binding psychophysical task Results: Stimulation provided by VI in vestibulopathic monkeys improved their balance, perception of spatial orientation, and eye movement responses. Timing experiments in humans using CI and VI stimuli showed that unlike past experiments that used motion to generate the vestibular signal, CI and VI signals were received by the cerebral cortex with the same latency and were perceived as simultaneous, but this timing perception was highly sensitive to adaption. Conclusions: VI improves oculomotor, postural, and perceptual behavior in vestibulopathic monkeys and could prove to be an effective way to improve these functions in patients with permanent labyrinthine damage. Timing experiments show that when novel stimuli are used, the brain synthesizes them in accordance with their arrival at the cortex, but that experience can rapidly recalibrate this timing relationship, which may be why normal stimuli that are experienced habitually lack this characteristic.more » « less
-
more » « less
Abstract Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further suggest that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. Gravitational and inertial acceleration stimulate the otoliths in an equivalent manner, so we also analyze the dynamics of human head orientation to better understand how knowledge of these dynamics can constrain solutions to the problem of gravitoinertial ambiguity. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing, including both frequency segregation and probabilistic internal model accounts. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.
-
more » « less
Synopsis Arboreal embryos of phyllomedusine treefrogs hatch prematurely to escape snake predation, cued by vibrations in their egg clutches during attacks. However, escape success varies between species, from ∼77% in Agalychnis callidryas to just ∼9% in A. spurrelli at 1 day premature. Both species begin responding to snake attacks at similar developmental stages, when vestibular mechanosensory function begins, suggesting that sensory ability does not limit the hatching response in A. spurrelli. Agalychnis callidryas clutches are thick and gelatinous, while A. spurrelli clutches are thinner and stiffer. We hypothesized that this structural difference alters the egg motion excited by attacks. Since vibrations excited by snakes must propagate through clutches to reach embryos, we hypothesized that the species difference in attack-induced hatching may reflect effects of clutch biomechanics on the cues available to embryos. Mechanics predicts that thinner, stiffer structures have higher free vibration frequencies, greater spatial attenuation, and faster vibration damping than thicker, more flexible structures. We assessed clutch biomechanics by embedding small accelerometers in clutches of both species and recording vibrations during standardized excitation tests at two distances from the accelerometer. Analyses of recorded vibrations showed that A. spurrelli clutches have higher free vibration frequencies and greater vibration damping than A. callidryas clutches. Higher frequencies elicit less hatching in A. callidryas, and greater damping could reduce the amount of vibration embryos can perceive. To directly test if clutch structure affects escape success in snake attacks, we transplanted A. spurrelli eggs into A. callidryas clutches and compared their escape rates with untransplanted, age-matched conspecific controls. We also performed reciprocal transplantation of eggs between pairs of A. callidryas clutches as a method control. Transplanting A. spurrelli embryos into A. callidryas clutches nearly tripled their escape success (44%) compared to conspecific controls (15%), whereas transplanting A. callidryas embryos into different A. callidryas clutches only increased escape success by 10%. At hatching competence, A. callidryas eggs are no longer jelly-encapsulated, while A. spurrelli eggs retain their jelly coat. Therefore, we compared the hatching response and latency of A. spurrelli in de-jellied eggs and their control, jelly-encapsulated siblings using manual egg-jiggling to simulate predation cues. Embryos in de-jellied eggs were more likely to hatch and hatched faster than control siblings. Together, our results suggest that the properties of parentally produced egg-clutch structures, including their vibration biomechanics, constrain the information available to A. spurrelli embryos and contribute to interspecific differences in hatching responses to predator attacks.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.7554/eLife.63405
