More Like this

1. Learning from Silence and Noise for Visual Sound Source Localization

   Juanola, Xavier; Morais, Giovanal; Fuentes, Magdalena; Haro, Gloria (November 2025, BMVA - the British Machine Vision Association.)

   Visual sound source localization is a fundamental perception task that aims to detect the location of sounding sources in a video given its audio. Despite recent progress, we identify two shortcomings in current methods: 1) most approaches perform poorly in cases with low audio-visual semantic correspondence such as silence, noise, and offscreen sounds, i.e. in the presence of negative audio; and 2) most prior evaluations are limited to positive cases, where both datasets and metrics convey scenarios with a single visible sound source in the scene. To address this, we introduce three key contributions. First, we propose a new training strategy that incorporates silence and noise, which improves performance in positive cases, while being more robust against negative sounds. Our resulting self-supervised model, SSL-SaN, achieves state-of-the-art performance compared to other self-supervised models, both in sound localization and cross-modal retrieval. Second, we propose a new metric that quantifies the trade-off between alignment and separability of auditory and visual features across positive and negative audio-visual pairs. Third, we present IS3+, an extended and improved version of the IS3 synthetic dataset with negative audio. Our data, metrics and code are available on the Project page. 

   more » « less
2. Parvalbumin and somatostatin interneurons support distinct computations during auditory scene analysis

   https://doi.org/10.64898/2026.07.08.736456  

   Qu, Zhili; Nocon, Jian Carlo; Sen, Kamal; Gritton, Howard J (July 2026, bioRxiv)

   Abstract Auditory scene analysis requires cortical circuits to preserve target-sound representations while suppressing interference from competing sound sources. Although spatial separation between tar­get and masker sounds can substantially improve perception, the inhibitory circuit mechanisms that transform spatial separation into improved target sound representations remain poorly un­derstood. Here, we used a multi-speaker auditory environment to present target sounds alone (clean) or together with spatially distributed competing white noise (masked) trials while record­ing auditory cortical activity in awake mice. By combining electrophysiology, cell-type-specific optogenetic suppression, and spike-distance-based neural classifiers, we examined how parval­bumin (PV) and somatostatin (SST) interneurons contribute to spatial processing in complex auditory scenes. PV and SST interneurons made dissociable, context-dependent contributions. PV suppression broadly altered spatial tuning properties, including tuning width, modulation depth, centroid, and sparseness, and reduced target-sound discriminability under clean condi­tions. In contrast, SST suppression produced comparatively modest effects on spatial tuning but selectively impaired neural discrimination only during masked trials thereby weakening the im­provement in target discrimination normally associated with spatial release from masking. Thus, the interneuron population that most strongly altered spatial tuning was not the population that most strongly disrupted masked discrimination. This reveals a dissociation between cortical in­hibitory mechanisms supporting spatial receptive-field structure and those supporting target–masker segregation. Together, these findings suggest that PV-mediated inhibition refines the fi­delity of auditory cortical representations, whereas SST-mediated inhibition enhances separation of simultaneously spatially competing sound sources, providing complementary inhibitory mecha­nisms for auditory scene analysis. 

   more » « less
3. Two stages of bandwidth scaling drives efficient neural coding of natural sounds

   https://doi.org/10.1371/journal.pcbi.1010862  

   He, Fengrong; Stevenson, Ian H.; Escabí, Monty A. (February 2023, PLOS Computational Biology)

   Wei, Xue-Xin (Ed.)

   Theories of efficient coding propose that the auditory system is optimized for the statistical structure of natural sounds, yet the transformations underlying optimal acoustic representations are not well understood. Using a database of natural sounds including human speech and a physiologically-inspired auditory model, we explore the consequences of peripheral (cochlear) and mid-level (auditory midbrain) filter tuning transformations on the representation of natural sound spectra and modulation statistics. Whereas Fourier-based sound decompositions have constant time-frequency resolution at all frequencies, cochlear and auditory midbrain filters bandwidths increase proportional to the filter center frequency. This form of bandwidth scaling produces a systematic decrease in spectral resolution and increase in temporal resolution with increasing frequency. Here we demonstrate that cochlear bandwidth scaling produces a frequency-dependent gain that counteracts the tendency of natural sound power to decrease with frequency, resulting in a whitened output representation. Similarly, bandwidth scaling in mid-level auditory filters further enhances the representation of natural sounds by producing a whitened modulation power spectrum (MPS) with higher modulation entropy than both the cochlear outputs and the conventional Fourier MPS. These findings suggest that the tuning characteristics of the peripheral and mid-level auditory system together produce a whitened output representation in three dimensions (frequency, temporal and spectral modulation) that reduces redundancies and allows for a more efficient use of neural resources. This hierarchical multi-stage tuning strategy is thus likely optimized to extract available information and may underlies perceptual sensitivity to natural sounds. 

   more » « less
4. Representations of spatial saliency in auditory cortex are selectively organized through temporal coordination

   https://doi.org/10.64898/2026.06.10.731488  

   Irani-Cereceda, Martin; Qu, Zhili; Sen, Kamal; Sadaghiani, Sepideh; Gritton, Howard J (June 2026, bioRxiv)

   SUMMARY Selective processing of behaviorally relevant sounds in complex environments requires dynamic modulation of auditory representations. Here, we show that behaviorally relevant sound locations selectively organize auditory cortical activity through temporal coordination rather than changes in firing rate. Using laminar electrophysiology in mice, we recorded activity in primary auditory cortex (A1) activity during a task in which reward was associated with a specific sound location while neutral, non-rewarded sounds were presented across multiple spatial positions. Neurons exhibited increased response reliability, temporal precision, and spike-field coupling to gamma oscillations selectively for sounds presented from behaviorally relevant locations, even when those sounds were unrewarded. These effects were absent during passive listening. Neurons with stronger gamma coupling also had sharper spatial tuning and greater trial-to-trial reliability, linking temporal coordination to representational fidelity. These findings identify gamma-mediated synchronization as a mechanism through which spatial relevance dynamically reshapes auditory encoding. 

   more » « less
5. A sonic One Health framework: characterizing health of the environment, animals, and humans using a soundscape perspective

   https://doi.org/10.1093/biosci/biaf109  

   Pijanowski, Bryan C; Rivas_Fuenzalida, Francisco; Kang, Jian; Werner, Courtney; Francis, Alex; McPherson, Malinda; Heinz, Michael; Grimes, Sarah; Post, Jennifer; Rhodes, O Eugene; et al (August 2025, BioScience)

   Abstract We present a sonic One Health framework with environmental, animal and human health integration centering on understanding positive and negative effects from the sounds of nature or the sounds from society. Our framework includes a sonic One Health system that describes sound and health; a sonic One Health network that describes specialists needed in our sonic One Health system; a human sonic effects model that reviews how sounds effect the auditory, neurological, cardiovascular, hormonal, and immunological systems; and an integrated set of sonic One Health remediation pathways. We review the literature in noise and health, environmental psychology as it relates to restorative effects of being in nature, and approaches to monitoring sound and perception. We also present three brief case studies, describe future directions for research across eight themes, discuss novel and challenging aspects of our framework, and describe the role that AI may play in our framework. 

   more » « less