Understanding the neural basis of the remarkable human cognitive capacity to learn novel concepts from just one or a few sensory experiences constitutes a fundamental problem. We propose a simple, biologically plausible, mathematically tractable, and computationally powerful neural mechanism for few-shot learning of naturalistic concepts. We posit that the concepts that can be learned from few examples are defined by tightly circumscribed manifolds in the neural firing-rate space of higher-order sensory areas. We further posit that a single plastic downstream readout neuron learns to discriminate new concepts based on few examples using a simple plasticity rule. We demonstrate the computational power of our proposal by showing that it can achieve high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network (DNN) models of these representations and can even learn novel visual concepts specified only through linguistic descriptors. Moreover, we develop a mathematical theory of few-shot learning that links neurophysiology to predictions about behavioral outcomes by delineating several fundamental and measurable geometric properties of neural representations that can accurately predict the few-shot learning performance of naturalistic concepts across all our numerical simulations. This theory reveals, for instance, that high-dimensional manifolds enhance the ability to learn new concepts from few examples. Intriguingly, we observe striking mismatches between the geometry of manifolds in the primate visual pathway and in trained DNNs. We discuss testable predictions of our theory for psychophysics and neurophysiological experiments.
Deep neural networks (DNNs) optimized for visual tasks learn representations that align layer depth with the hierarchy of visual areas in the primate brain. One interpretation of this finding is that hierarchical representations are necessary to accurately predict brain activity in the primate visual system. To test this interpretation, we optimized DNNs to directly predict brain activity measured with fMRI in human visual areas V1-V4. We trained a single-branch DNN to predict activity in all four visual areas jointly, and a multi-branch DNN to predict each visual area independently. Although it was possible for the multi-branch DNN to learn hierarchical representations, only the single-branch DNN did so. This result shows that hierarchical representations are not necessary to accurately predict human brain activity in V1-V4, and that DNNs that encode brain-like visual representations may differ widely in their architecture, ranging from strict serial hierarchies to multiple independent branches.
more » « less- PAR ID:
- 10420703
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 14
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The recent publications of the inter-areal connectomes for mouse, marmoset, and macaque cortex have allowed deeper comparisons across rodent vs. primate cortical organization. In general, these show that the mouse has very widespread, “all-to-all” inter-areal connectivity (i.e. a “highly dense” connectome in a graph theoretical framework), while primates have a more modular organization. In this review, we highlight the relevance of these differences to function, including the example of primary visual cortex (V1) which, in the mouse, is interconnected with all other areas, therefore including other primary sensory and frontal areas. We argue that this dense inter-areal connectivity benefits multimodal associations, at the cost of reduced functional segregation. Conversely, primates have expanded cortices with a modular connectivity structure, where V1 is almost exclusively interconnected with other visual cortices, themselves organized in relatively segregated streams, and hierarchically higher cortical areas such as prefrontal cortex provide top–down regulation for specifying precise information for working memory storage and manipulation. Increased complexity in cytoarchitecture, connectivity, dendritic spine density, and receptor expression additionally reveal a sharper hierarchical organization in primate cortex. Together, we argue that these primate specializations permit separable deconstruction and selective reconstruction of representations, which is essential to higher cognition.
-
null (Ed.)Abstract The mammalian sensory neocortex consists of hierarchically organized areas reciprocally connected via feedforward (FF) and feedback (FB) circuits. Several theories of hierarchical computation ascribe the bulk of the computational work of the cortex to looped FF-FB circuits between pairs of cortical areas. However, whether such corticocortical loops exist remains unclear. In higher mammals, individual FF-projection neurons send afferents almost exclusively to a single higher-level area. However, it is unclear whether FB-projection neurons show similar area-specificity, and whether they influence FF-projection neurons directly or indirectly. Using viral-mediated monosynaptic circuit tracing in macaque primary visual cortex (V1), we show that V1 neurons sending FF projections to area V2 receive monosynaptic FB inputs from V2, but not other V1-projecting areas. We also find monosynaptic FB-to-FB neuron contacts as a second motif of FB connectivity. Our results support the existence of FF-FB loops in primate cortex, and suggest that FB can rapidly and selectively influence the activity of incoming FF signals.more » « less
-
Andreas Krause, Barbara Engelhardt (Ed.)Reconstructing natural images from fMRI recordings is a challenging task of great importance in neuroscience. The current architectures are bottlenecked because they fail to effectively capture the hierarchical processing of visual stimuli that takes place in the human brain. Motivated by that fact, we introduce a novel neural network architecture for the problem of neural decoding. Our architecture uses Hierarchical Variational Autoencoders (HVAEs) to learn meaningful representations of natural images and leverages their latent space hierarchy to learn voxel-to-image mappings. By mapping the early stages of the visual pathway to the first set of latent variables and the higher visual cortex areas to the deeper layers in the latent hierarchy, we are able to construct a latent variable neural decoding model that replicates the hierarchical visual information processing. Our model achieves better reconstructions compared to the state of the art and our ablation study indicates that the hierarchical structure of the latent space is responsible for that performance.more » « less
-
Abstract Attention promotes the selection of behaviorally relevant sensory signals from the barrage of sensory information available. Visual attention modulates the gain of neuronal activity in all visual brain areas examined, although magnitudes of gain modulations vary across areas. For example, attention gain magnitudes in the dorsal lateral geniculate nucleus (LGN) and primary visual cortex (V1) vary tremendously across fMRI measurements in humans and electrophysiological recordings in behaving monkeys. We sought to determine whether these discrepancies are due simply to differences in species or measurement, or more nuanced properties unique to each visual brain area. We also explored whether robust and consistent attention effects, comparable to those measured in humans with fMRI, are observable in the LGN or V1 of monkeys. We measured attentional modulation of multiunit activity in the LGN and V1 of macaque monkeys engaged in a contrast change detection task requiring shifts in covert visual spatial attention. Rigorous analyses of LGN and V1 multiunit activity revealed robust and consistent attentional facilitation throughout V1, with magnitudes comparable to those observed with fMRI. Interestingly, attentional modulation in the LGN was consistently negligible. These findings demonstrate that discrepancies in attention effects are not simply due to species or measurement differences. We also examined whether attention effects correlated with the feature selectivity of recorded multiunits. Distinct relationships suggest that attentional modulation of multiunit activity depends upon the unique structure and function of visual brain areas.