An official website of the United States government
Perception is fallible. Humans know this, and so do some nonhuman animals like macaque monkeys. When monkeys report more confidence in a perceptual decision, that decision is more likely to be correct. It is not known how neural circuits in the primate brain assess the quality of perceptual decisions. Here, we test two hypotheses. First, that decision confidence is related to the structure of population activity in the sensory cortex. And second, that this relation differs from the one between sensory activity and decision content. We trained macaque monkeys to judge the orientation of ambiguous stimuli and additionally report their confidence in these judgments. We recorded population activity in the primary visual cortex and used decoders to expose the relationship between this activity and the choice-confidence reports. Our analysis validated both hypotheses and suggests that perceptual decisions arise from a neural computation downstream of visual cortex that estimates the most likely interpretation of a sensory response, while decision confidence instead reflects a computation that evaluates whether this sensory response will produce a reliable decision. Our work establishes a direct link between neural population activity in the sensory cortex and the metacognitive ability to introspect about the quality of perceptual decisions.
more »
« less
Calibrating Bayesian decoders of neural spiking activity
Accurately decoding external variables from observations of neural activity is a major challenge in systems neuroscience. Bayesian decoders, that provide probabilistic estimates, are some of the most widely used. Here we show how, in many common settings, the probabilistic predictions made by traditional Bayesian decoders are overconfident. That is, the estimates for the decoded stimulus or movement variables are more certain than they should be. We then show how Bayesian decoding with latent variables, taking account of low-dimensional shared variability in the observations, can improve calibration, although additional correction for overconfidence is still needed. Using data from males, we examine: 1) decoding the direction of grating stimuli from spike recordings in primary visual cortex in monkeys, 2) decoding movement direction from recordings in primary motor cortex in monkeys, 3) decoding natural images from multi-region recordings in mice, and 4) decoding position from hippocampal recordings in rats. For each setting we characterize the overconfidence, and we describe a possible method to correct miscalibration post-hoc. Properly calibrated Bayesian decoders may alter theoretical results on probabilistic population coding and lead to brain machine interfaces that more accurately reflect confidence levels when identifying external variables. Significance Statement Bayesian decoding is a statistical technique for making probabilistic predictions about external stimuli or movements based on recordings of neural activity. These predictions may be useful for robust brain machine interfaces or for understanding perceptual or behavioral confidence. However, the probabilities produced by these models do not always match the observed outcomes. Just as a weather forecast predicting a 50% chance of rain may not accurately correspond to an outcome of rain 50% of the time, Bayesian decoders of neural activity can be miscalibrated as well. Here we identify and measure miscalibration of Bayesian decoders for neural spiking activity in a range of experimental settings. We compare multiple statistical models and demonstrate how overconfidence can be corrected.
more »
« less
- PAR ID:
- 10516259
- Publisher / Repository:
- Society for Neuroscience
- Date Published:
- Journal Name:
- The Journal of Neuroscience
- ISSN:
- 0270-6474
- Page Range / eLocation ID:
- e2158232024
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Sensory data about most natural task-relevant variables are entangled with task-irrelevant nuisance variables. The neurons that encode these relevant signals typically constitute a nonlinear population code. Here we present a theoretical framework for quantifying how the brain uses or decodes its nonlinear information. Our theory obeys fundamental mathematical limitations on information content inherited from the sensory periphery, describing redundant codes when there are many more cortical neurons than primary sensory neurons. The theory predicts that if the brain uses its nonlinear population codes optimally, then more informative patterns should be more correlated with choices. More specifically, the theory predicts a simple, easily computed quantitative relationship between fluctuating neural activity and behavioral choices that reveals the decoding efficiency. This relationship holds for optimal feedforward networks of modest complexity, when experiments are performed under natural nuisance variation. We analyze recordings from primary visual cortex of monkeys discriminating the distribution from which oriented stimuli were drawn, and find these data are consistent with the hypothesis of near-optimal nonlinear decoding.more » « less
-
Abstract Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.more » « less
-
Correlations between the spiking of pairs of neurons are often used to study the brain’s representation of sensory or motor variables and neural circuit function and dysfunction. Previous statistical techniques have shown how time-averaged spike-spike correlations can be predicted by the time-averaged relationships between the individual neurons and the local field potential (LFP). However, spiking and LFP are both nonstationary, and spike-spike correlations have nonstationary structure that cannot be accounted for by time-averaged approaches. Here we develop parametric models that predict spike-spike correlations using a small number of LFP-based predictors, and we then apply these models to the problem of tracking changes in spike-spike correlations over time. Parametric models allow for flexibility in the choice of which LFP recording channels and frequency bands to use for prediction, and coefficients directly indicate which LFP features drive correlated spiking. Here we demonstrate our methods in simulation and test the models on experimental data from large-scale multi-electrode recordings in the mouse hippocampus and visual cortex. In single time windows, we find that our parametric models can be as accurate as previous nonparametric approaches, while also being flexible and interpretable. We then demonstrate how parametric models can be applied to describe nonstationary spike-spike correlations measured in sequential time windows. We find that although the patterns of both cortical and hippocampal spike-spike correlations vary over time, these changes are, at least partially, predicted by models that assume a fixed spike-field relationship. This approach may thus help to better understand how the dynamics of spike-spike correlations are related to functional brain states. Since spike-spike correlations are increasingly used as features for decoding external variables from neural activity, these models may also have the potential to improve the accuracy of adaptive decoders and brain machine interfaces.more » « less
-
Andreas Krause, Barbara Engelhardt (Ed.)Reconstructing natural images from fMRI recordings is a challenging task of great importance in neuroscience. The current architectures are bottlenecked because they fail to effectively capture the hierarchical processing of visual stimuli that takes place in the human brain. Motivated by that fact, we introduce a novel neural network architecture for the problem of neural decoding. Our architecture uses Hierarchical Variational Autoencoders (HVAEs) to learn meaningful representations of natural images and leverages their latent space hierarchy to learn voxel-to-image mappings. By mapping the early stages of the visual pathway to the first set of latent variables and the higher visual cortex areas to the deeper layers in the latent hierarchy, we are able to construct a latent variable neural decoding model that replicates the hierarchical visual information processing. Our model achieves better reconstructions compared to the state of the art and our ablation study indicates that the hierarchical structure of the latent space is responsible for that performance.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1523/JNEUROSCI.2158-23.2024
