An official website of the United States government
Humans and other primates can robustly report whether they've seen specific images before, even when those images are extremely similar to ones they've previously seen. Multiple lines of evidence suggest that pattern separation computations in the hippocampus (HC) contribute to this behavior by shaping the fidelity of visual memory. However, unclear is whether HC uniquely determines memory fidelity or whether computations in other brain areas also contribute. To investigate, we recorded neural signals from inferotemporal cortex (ITC) and HC of two rhesus monkeys (1 male, 1 female) as they performed a memory task in which they judged whether images were novel or exactly repeated in the presence of visually similar lure images with a range of visual similarities. We found behavioral evidence for sharpening, reflected as memory performance that was nonlinearly transformed relative to a benchmark defined by visual representations in ITC. As expected, we found that behavioral sharpening aligned with visual memory representations in HC. Surprisingly, and unaccounted for by HC pattern separation proposals, we also found neural correlates of behavioral sharpening reflected in ITC. These results, coupled with further analysis of the data, suggest that ITC contributes to shaping the fidelity of visual memory in the transformation from visual processing to memory storage and signaling.
more »
« less
Sharpened visual memory representations are reflected in inferotemporal cortex
Humans and other primates can robustly report whether they've seen specific images before, even when those images are extremely similar to ones they've previously seen. Multiple lines of evidence suggest that pattern separation computations in the hippocampus (HC) contribute to this behavior by shaping the fidelity of visual memory. However, unclear is whether HC uniquely determines memory fidelity or whether computations in other brain areas also contribute. To investigate, we recorded neural signals from inferotemporal cortex (ITC) and HC of two rhesus monkeys (1 male, 1 female) as they performed a memory task in which they judged whether images were novel or exactly repeated in the presence of visually similar lure images with a range of visual similarities. We found behavioral evidence for sharpening, reflected as memory performance that was nonlinearly transformed relative to a benchmark defined by visual representations in ITC. As expected, we found that behavioral sharpening aligned with visual memory representations in HC. Surprisingly, and unaccounted for by HC pattern separation proposals, we also found neural correlates of behavioral sharpening reflected in ITC. These results, coupled with further analysis of the data, suggest that ITC contributes to shaping the fidelity of visual memory in the transformation from visual processing to memory storage and signaling. Significance StatementVisual recognition memories are stored with remarkable visual fidelity, allowing humans and other primates to distinguish images they have encountered from visually similar images they have not. This fidelity has long been attributed to computations in the hippocampus that sharpen visual representations before memory storage (pattern separation). Unclear is how this proposal aligns with other evidence that visual memories are stored within high-level visual cortex itself, before signals reach the hippocampus. Here we demonstrate that, like the hippocampus, inferotemporal cortex also reflects sharpened visual memory representations, suggesting that visual cortex contributes to shaping the visual fidelity of visual memory.
more »
« less
- Award ID(s):
- 2043255
- PAR ID:
- 10661271
- Publisher / Repository:
- Journal of Neuroscience
- Date Published:
- Journal Name:
- The Journal of Neuroscience
- ISSN:
- 0270-6474
- Page Range / eLocation ID:
- e0833252025
- Subject(s) / Keyword(s):
- familiarity hippocampus inferior temporal cortex inferotemporal cortex medial temporal cortex recognition memory
- Format(s):
- Medium: X
- Associated Dataset(s):
- View Associated Dataset(s) >>
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Summary While humans and other primates are generally good at remembering the images they have seen, they systematically remember some images better than others (“image memorability”). Here, we leverage image memorability to resolve a puzzle around how the brain transforms seeing into familiarity. Namely, the neural signal driving familiarity reports is thought to be repetition suppression, an overall reduction in the vigor of the population response in brain regions, including inferotemporal cortex (ITC). However, within ITC, more memorable images evoke higher firing rates than less memorable ones, even when they are repeated. These two observations conflict: ifreducedfiring leads to stronger memory signaling, then why are the images that inducegreaterfiring more memorable? To investigate this “familiarity-memorability paradox”, we compared neural activity in ITC and the hippocampus (HC) as two macaque monkeys performed a single-exposure image familiarity task. We found a resolution to the paradox, reflected in two pieces of evidence. First, memory and memorability were not completely overlapping in ITC, suggesting that familiarity signals could be extracted independently of memorability modulation. Second, responses in HC reflected a transformation of ITC responses that extracted memory, leaving behind stimulus-evoked firing modulation. This transformation could be accounted for by a linear decoder that selectively extracts memory and preserves larger ITC memory signals for more memorable images. These results resolve the familiarity-memorability paradox by demonstrating that ITC familiarity signals are a component (but not the entirety) of repetition suppression, and they establish the existence of a previously undescribed medial temporal lobe computation. SignificanceNeural representations of image memorability create a paradox: familiarity reports are thought to derive fromreducedfiring in inferotemporal cortex (ITC), but more memorable images evoke more vigorous responses there. To resolve this contradiction, we compared ITC with an area downstream, the hippocampus (HC), as macaque monkeys performed a single-exposure familiarity task. We found evidence for a computation between ITC and HC that selectively extracts ITC memory signals. This transformation could be explained by a linear decoder that preserves larger memory signals for more memorable images. This work resolves the familiarity-memorability paradox by showing that ITC memory signals have been misunderstood, and it reveals a previously undescribed role for the medial temporal lobe in familiarity behavior.more » « less
-
Episodic memories are records of personally experienced events, coded neurally via the hippocampus and sur- rounding medial temporal lobe cortex. Information about the neural signal corresponding to a memory representation can be measured in fMRI data when the pattern across voxels is examined. Prior studies have found that similarity in the voxel patterns across repetition of a to-be-remembered stimulus predicts later memory retrieval, but the results are inconsistent across studies. The current study investigates the possibility that cognitive goals (defined here via the task instructions given to participants) during encoding affect the voxel pattern that will later support memory retrieval, and therefore that neural representations cannot be interpreted based on the stimulus alone. The behavioral results showed that exposure to variable cognitive tasks across repetition of events benefited subsequent memory retrieval. Voxel patterns in the hippocampus indicated a significant interaction between cognitive tasks (variable vs. consistent) and memory (remembered vs. forgotten) such that reduced voxel pattern similarity for repeated events with variable cognitive tasks, but not consistent cognitive tasks, sup- ported later memory success. There was no significant interaction in neural pattern similarity between cognitive tasks and memory success in medial temporal cortices or lateral occipital cortex. Instead, higher similarity in voxel patterns in right medial temporal cortices was associated with later memory retrieval, regardless of cognitive task. In conclusion, we found that the relationship between pattern similarity across repeated encoding and memory success in the hippocampus (but not medial temporal lobe cortex) changes when the cognitive task during encoding does or does not vary across repetitions of the event.more » « less
-
Thanks to patients Phineas Gage and Henry Molaison, we have long known that behavioral control depends on the frontal lobes, whereas declarative memory depends on the medial temporal lobes (MTL). For decades, cognitive functions—behavioral control, declarative memory—have served as labels for characterizing the division of labor in cortex. This approach has made enormous contributions to understanding how the brain enables the mind, providing a systems-level explanation of brain function that constrains lower-level investigations of neural mechanism. Today, the approach has evolved such that functional labels are often applied to brain networks rather than focal brain regions. Furthermore, the labels have diversified to include both broadly-defined cognitive functions (declarative memory, visual perception) and more circumscribed mental processes (recollection, familiarity, priming). We ask whether a process—a high-level mental phenomenon corresponding to an introspectively-identifiable cognitive event—is the most productive label for dissecting memory. For example, recollection conflates a neurocomputationaloperation(pattern completion-based retrieval) with a class ofrepresentational content(associative, high-dimensional memories). Because a full theory of memory must identify operations and representations separately, and specify how they interact, we argue that processes like recollection constitute inadequate labels for characterizing neural mechanisms. Instead, we advocate considering the component operations and representations of processes like recollection in isolation. For the organization of memory, the evidence suggests that pattern completion is recapitulated widely across the ventral visual stream and MTL, but the division of labor between sites within this pathway can be explained by representational content.more » « less
-
Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (lFPC). Our results indicate that lFPC holds previous causal choices in a ‘pending’ state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1523/JNEUROSCI.0833-25.2025
