An official website of the United States government
Neurons in higher-order visual areas integrate information through a canonical computation called normalization. The strength of normalization is highly heterogeneous across neurons, and this heterogeneity correlates with attention-mediated modulations in neural responses. However, the circuit mechanism underlying the heterogeneous normalization strength is unclear. In this work, we study normalization in a spiking neuron network model of visual cortex. Our model reveals that the heterogeneity of normalization strength is highly correlated with the inhibitory current each neuron receives. The correlation between inhibition and other synaptic inputs explains the experimentally observed dependence of spike count correlations on normalization strength. Further, we find that neurons with stronger normalization encode information more efficiently, and that networks with more heterogeneity in normalization encode visual stimuli with higher information and capacity. Together, our model provides a mechanistic explanation of heterogeneous normalization strengths in the visual cortex and sheds light on the computational benefits of neuronal heterogeneity.
more »
« less
PolarTag: Invisible Data with Light Polarization
Visual tags (e.g., barcodes, QR codes) are ubiquitous in modern day life, though they rely on obtrusive geometric patterns to encode data, degrading the overall user experience. We propose a new paradigm of passive visual tags which utilizes light polarization to imperceptibly encode data using cheap, widely-available components. The tag and its data can be extracted from background scenery using off-the-shelf cameras with inexpensive LCD shutters attached atop camera lenses. We examine the feasibility of this design with real-world experiments. Initial results show zero bit errors at distances up to 3.0~m, an angular-detection range of \ang110, and robustness to manifold ambient light and occlusion scenarios.
more »
« less
- PAR ID:
- 10191708
- Date Published:
- Journal Name:
- HotMobile '20: The 21st International Workshop on Mobile Computing Systems and Applications
- Page Range / eLocation ID:
- 74 to 79
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We release ASFx Eval, a curated dataset of audiocards which are structured metadata with attributes pertinent to sound design such as noun-verb pairs and example visual context . Using Pixtral-12B-2409 we generated 500 audiocards for corresponding audio files in Adobe Audition Sound Effects and manually screened them for incoherence and hallucinations. These audiocards can be used to evaluate audio language models on tasks such as structured metadata generation, text-audio retrieval, and audio captioning, by pairing them with the corresponding audio file using the filename column in the csv below. For example, the audiocard with filename "Impacts/Impact Metal Spatula Hits 08.wav" corresponds to the "Impact Metal Spatula Hits 08.wav" sound effect under the "Impacts" category. Each of the remaining columns corresponds to an audiocard field. We also provide Universal Category System (UCS) category and subcategory tags. UCS is an ontology developed for sound design and commonly used to organize professional libraries. We generated UCS tags from a text classifier that achieved 96% accuracy on a separate dataset with human annotated tags. These tags can be used to evaluate audio language models and audio representations on UCS category and subcategory prediction for sound effects library management.more » « less
-
Abstract: Neurons in higher-order visual areas integrate information through a canonical computation called normalization. The strength of normalization is highly heterogeneous across neurons, and this heterogeneity correlates with attention-mediated modulations in neural responses. However, the circuit mechanism underlying the heterogeneous normalization strength is unclear. In this work, we study normalization in a spiking neuron network model of visual cortex. Our model reveals that the heterogeneity of normalization strength is highly correlated with the inhibitory current each neuron receives. The correlation between inhibition and other synaptic inputs explains the experimentally observed dependence of spike count correlations on normalization strength. Further, we find that neurons with stronger normalization encode information more efficiently, and that networks with more heterogeneity in normalization have more information and higher capacity in encoding visual stimuli. Together, our model provides a mechanistic explanation of heterogeneous normalization strengths in the visual cortex, and sheds light on the computational benefits of neuronal heterogeneity.more » « less
-
Humans can recognize their whole-body movements even when displayed as dynamic dot patterns. The sparse depiction of whole-body movements, coupled with a lack of visual experience watching ourselves in the world, has long implicated nonvisual mechanisms to self-action recognition. Using general linear modeling and multivariate analyses on human brain imaging data from male and female participants, we aimed to identify the neural systems for this ability. First, we found that cortical areas linked to motor processes, including frontoparietal and primary somatomotor cortices, exhibit greater engagement and functional connectivity when recognizing self-generated versus other-generated actions. Next, we show that these regions encode self-identity based on motor familiarity, even after regressing out idiosyncratic visual cues using multiple regression representational similarity analysis. Last, we found the reverse pattern for unfamiliar individuals: encoding localized to occipitotemporal visual regions. These findings suggest that self-awareness from actions emerges from the interplay of motor and visual processes.more » « less
-
In-situ visual observations of marine organisms is crucial to developing behavioural understandings and their relations to their surrounding ecosystem. Typically, these observations are collected via divers, tags, and remotely-operated or human-piloted vehicles. Recently, however, autonomous underwater vehicles equipped with cameras and embedded computers with GPU capabilities are being developed for a variety of applications, and in particular, can be used to supplement these existing data collection mechanisms where human operation or tags are more difficult. Existing approaches have focused on using fully-supervised tracking methods, but labelled data for many underwater species are severely lacking. Semi-supervised trackers may offer alternative tracking solutions because they require less data than fully-supervised counterparts. However, because there are not existing realistic underwater tracking datasets, the performance of semi-supervised tracking algorithms in the marine domain is not well understood. To better evaluate their performance and utility, in this paper we provide (1) a novel dataset specific to marine animals located at http://warp.whoi.edu/vmat/, (2) an evaluation of state-of-the-art semi-supervised algorithms in the context of underwater animal tracking, and (3) an evaluation of real-world performance through demonstrations using a semi-supervised algorithm on-board an autonomous underwater vehicle to track marine animals in the wild.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.1145/3376897.3377854
