More Like this

1. Haptic and Audio Interaction Design

   https://doi.org/10.1007/978-3-031-15019-7_5  

   Jones, Lynette A. ; Ho, Hsin-Ni ( August 2022 , Haptic and Audio Interaction Design)

   Saitis, C. ; Farkhatdinov, I ; Papetti, S. (Ed.)

   There are fundamental differences between the tactile and thermal sensory systems that must be accommodated when designing multisensory cutaneous displays for use in virtual or teleoperated robotic environments. In this review we highlight the marked temporal and spatial differences between the senses of cold and warmth as revealed in psychophysical experiments. Cold and warmth are distinct senses with marked differences in the time taken to respond to stimulation and in their temporal filtering processes. Such variations must be taken into account when time-varying profiles of thermal stimulation are delivered to the skin concurrent with tactile stimulation since the resulting sensations will not be perceived on the same time scale. Although it is often reported that the thermal senses are markedly inferior to the sense of touch with respect to their spatial acuity, it is also clear that there is considerable variability across the body in the accuracy with which thermal stimuli can be localized. The distal to proximal gradient in thermal acuity suggests that locations other than the palmar surface of the hand are better suited for displaying thermal cues, in contrast to the situation for tactile inputs. As was noted for temporal processes, there are differences between localizing warmth and cold stimuli, with localization being superior for cold. These properties provide benchmarks that can be used in designing thermal and multisensory displays. 

   more » « less
2. Neural Correlates of Audiovisual Temporal Binding Window in Individuals With Schizotypal and Autistic Traits: Evidence From Resting‐State Functional Connectivity

   https://doi.org/10.1002/aur.2456  

   Zhou, Han‐yu ; Wang, Yong‐ming ; Zhang, Rui‐ting ; Cheung, Eric F. C. ; Pantelis, Christos ; Chan, Raymond C. K. ( December 2020 , Autism Research)

   Temporal proximity is an important clue for multisensory integration. Previous evidence indicates that individuals with autism and schizophrenia are more likely to integrate multisensory inputs over a longer temporal binding window (TBW). However, whether such deficits in audiovisual temporal integration extend to subclinical populations with high schizotypal and autistic traits are unclear. Using audiovisual simultaneity judgment (SJ) tasks for nonspeech and speech stimuli, our results suggested that the width of the audiovisual TBW was not significantly correlated with self‐reported schizotypal and autistic traits in a group of young adults. Functional magnetic resonance imaging (fMRI) resting‐state activity was also acquired to explore the neural correlates underlying inter‐individual variability of TBW width. Across the entire sample, stronger resting‐state functional connectivity (rsFC) between the left superior temporal cortex and the left precuneus, and weaker rsFC between the left cerebellum and the right dorsal lateral prefrontal cortex were correlated with a narrower TBW for speech stimuli. Meanwhile, stronger rsFC between the left anterior superior temporal gyrus and the right inferior temporal gyrus was correlated with a wider audiovisual TBW for non‐speech stimuli. The TBW‐related rsFC was not affected by levels of subclinical traits. In conclusion, this study indicates that audiovisual temporal processing may not be affected by autistic and schizotypal traits and rsFC between brain regions responding to multisensory information and timing may account for the inter‐individual difference in TBW width.

   Individuals with ASD and schizophrenia are more likely to perceive asynchronous auditory and visual events as occurring simultaneously even if they are well separated in time. We investigated whether similar difficulties in audiovisual temporal processing were present in subclinical populations with high autistic and schizotypal traits. We found that the ability to detect audiovisual asynchrony was not affected by different levels of autistic and schizotypal traits. We also found that connectivity of some brain regions engaging in multisensory and timing tasks might explain an individual's tendency to bind multisensory information within a wide or narrow time window.Autism Res2021, 14: 668–680. © 2020 International Society for Autism Research and Wiley Periodicals LLC

    

   more » « less
3. Multimodality as universality: Designing inclusive accessibility to graphical information

   https://doi.org/10.3389/feduc.2023.1071759  

   Doore, Stacy A. ; Dimmel, Justin ; Kaplan, Toni M. ; Guenther, Benjamin A. ; Giudice, Nicholas A. ( April 2023 , Frontiers in Education)

   Graphical representations are ubiquitous in the learning and teaching of science, technology, engineering, and mathematics (STEM). However, these materials are often not accessible to the over 547,000 students in the United States with blindness and significant visual impairment, creating barriers to pursuing STEM educational and career pathways. Furthermore, even when such materials are made available to visually impaired students, access is likely through literalized modes (e.g., braille, verbal description), which is problematic as these approaches (1) do not directly convey spatial information and (2) are different from the graphic-based materials used by students without visual impairment. The purpose of this study was to design and evaluate a universally accessible system for communicating graphical representations in STEM classes. By combining a multisensory vibro-audio interface and an app running on consumer mobile hardware, the system is meant to work equally well for all students, irrespective of their visual status. We report the design of the experimental system and the results of an experiment where we compared learning performance with the system to traditional (visual or tactile) diagrams for sighted participants (n = 20) and visually impaired participants (n =9) respectively. While the experimental multimodal diagrammatic system (MDS) did result in significant learning gains for both groups of participants, the results also revealed no statistically significant differences in the capacity for learning from graphical information across both comparison groups. Likewise, there were no statistically significant differences in the capacity for learning from graphical information between the stimuli presented through the experimental system and the traditional (visual or tactile) diagram control conditions, across either participant group. These findings suggest that both groups were able to learn graphical information from the experimental system as well as traditional diagram presentation materials. This learning modality was supported without the need for conversion of the diagrams to make them accessible for participants who required tactile materials. The system also provided additional multisensory information for sighted participants to interpret and answer questions about the diagrams. Findings are interpreted in terms of new universal design principles for producing multisensory graphical representations that would be accessible to all learners.

    

   more » « less
4. Piezo2 integrates mechanical and thermal cues in vertebrate mechanoreceptors

   https://doi.org/10.1073/pnas.1910213116  

   Zheng, Wang ; Nikolaev, Yury A. ; Gracheva, Elena O. ; Bagriantsev, Sviatoslav N. ( August 2019 , Proceedings of the National Academy of Sciences)

   Tactile information is detected by thermoreceptors and mechanoreceptors in the skin and integrated by the central nervous system to produce the perception of somatosensation. Here we investigate the mechanism by which thermal and mechanical stimuli begin to interact and report that it is achieved by the mechanotransduction apparatus in cutaneous mechanoreceptors. We show that moderate cold potentiates the conversion of mechanical force into excitatory current in all types of mechanoreceptors from mice and tactile-specialist birds. This effect is observed at the level of mechanosensitive Piezo2 channels and can be replicated in heterologous systems using Piezo2 orthologs from different species. The cold sensitivity of Piezo2 is dependent on its blade domains, which render the channel resistant to cold-induced perturbations of the physical properties of the plasma membrane and give rise to a different mechanism of mechanical activation than that of Piezo1. Our data reveal that Piezo2 is an evolutionarily conserved mediator of thermal–tactile integration in cutaneous mechanoreceptors. 

   more » « less
5. A Normative Theory for Causal Inference and Bayes Factor Computation in Neural Circuits

   Zhang, Wenhao ; Wu, Si ; Doiron, Brent ; Lee, Tai Sing ( January 2019 , Advances in neural information processing systems)

   This study provides a normative theory for how Bayesian causal inference can be implemented in neural circuits. In both cognitive processes such as causal reasoning and perceptual inference such as cue integration, the nervous systems need to choose different models representing the underlying causal structures when making inferences on external stimuli. In multisensory processing, for example, the nervous system has to choose whether to integrate or segregate inputs from different sensory modalities to infer the sensory stimuli, based on whether the inputs are from the same or different sources. Making this choice is a model selection problem requiring the computation of Bayes factor, the ratio of likelihoods between the integration and the segregation models. In this paper, we consider the causal inference in multisensory processing and propose a novel generative model based on neural population code that takes into account both stimulus feature and stimulus reliability in the inference. In the case of circular variables such as heading direction, our normative theory yields an analytical solution for computing the Bayes factor, with a clear geometric interpretation, which can be implemented by simple additive mechanisms with neural population code. Numerical simulation shows that the tunings of the neurons computing Bayes factor are consistent with the "opposite neurons" discovered in dorsal medial superior temporal (MSTd) and the ventral intraparietal (VIP) areas for visual-vestibular processing. This study illuminates a potential neural mechanism for causal inference in the brain. 

   more » « less