An official website of the United States government
Humans and other primates can robustly report whether they've seen specific images before, even when those images are extremely similar to ones they've previously seen. Multiple lines of evidence suggest that pattern separation computations in the hippocampus (HC) contribute to this behavior by shaping the fidelity of visual memory. However, unclear is whether HC uniquely determines memory fidelity or whether computations in other brain areas also contribute. To investigate, we recorded neural signals from inferotemporal cortex (ITC) and HC of two rhesus monkeys (1 male, 1 female) as they performed a memory task in which they judged whether images were novel or exactly repeated in the presence of visually similar lure images with a range of visual similarities. We found behavioral evidence for sharpening, reflected as memory performance that was nonlinearly transformed relative to a benchmark defined by visual representations in ITC. As expected, we found that behavioral sharpening aligned with visual memory representations in HC. Surprisingly, and unaccounted for by HC pattern separation proposals, we also found neural correlates of behavioral sharpening reflected in ITC. These results, coupled with further analysis of the data, suggest that ITC contributes to shaping the fidelity of visual memory in the transformation from visual processing to memory storage and signaling. Significance StatementVisual recognition memories are stored with remarkable visual fidelity, allowing humans and other primates to distinguish images they have encountered from visually similar images they have not. This fidelity has long been attributed to computations in the hippocampus that sharpen visual representations before memory storage (pattern separation). Unclear is how this proposal aligns with other evidence that visual memories are stored within high-level visual cortex itself, before signals reach the hippocampus. Here we demonstrate that, like the hippocampus, inferotemporal cortex also reflects sharpened visual memory representations, suggesting that visual cortex contributes to shaping the visual fidelity of visual memory.
more »
« less
Resolving a paradox about how vision is transformed into familiarity
Summary While humans and other primates are generally good at remembering the images they have seen, they systematically remember some images better than others (“image memorability”). Here, we leverage image memorability to resolve a puzzle around how the brain transforms seeing into familiarity. Namely, the neural signal driving familiarity reports is thought to be repetition suppression, an overall reduction in the vigor of the population response in brain regions, including inferotemporal cortex (ITC). However, within ITC, more memorable images evoke higher firing rates than less memorable ones, even when they are repeated. These two observations conflict: ifreducedfiring leads to stronger memory signaling, then why are the images that inducegreaterfiring more memorable? To investigate this “familiarity-memorability paradox”, we compared neural activity in ITC and the hippocampus (HC) as two macaque monkeys performed a single-exposure image familiarity task. We found a resolution to the paradox, reflected in two pieces of evidence. First, memory and memorability were not completely overlapping in ITC, suggesting that familiarity signals could be extracted independently of memorability modulation. Second, responses in HC reflected a transformation of ITC responses that extracted memory, leaving behind stimulus-evoked firing modulation. This transformation could be accounted for by a linear decoder that selectively extracts memory and preserves larger ITC memory signals for more memorable images. These results resolve the familiarity-memorability paradox by demonstrating that ITC familiarity signals are a component (but not the entirety) of repetition suppression, and they establish the existence of a previously undescribed medial temporal lobe computation. SignificanceNeural representations of image memorability create a paradox: familiarity reports are thought to derive fromreducedfiring in inferotemporal cortex (ITC), but more memorable images evoke more vigorous responses there. To resolve this contradiction, we compared ITC with an area downstream, the hippocampus (HC), as macaque monkeys performed a single-exposure familiarity task. We found evidence for a computation between ITC and HC that selectively extracts ITC memory signals. This transformation could be explained by a linear decoder that preserves larger memory signals for more memorable images. This work resolves the familiarity-memorability paradox by showing that ITC memory signals have been misunderstood, and it reveals a previously undescribed role for the medial temporal lobe in familiarity behavior.
more »
« less
- Award ID(s):
- 2043255
- PAR ID:
- 10661274
- Publisher / Repository:
- bioRxiv
- Date Published:
- Format(s):
- Medium: X
- Institution:
- University of Pennsylvania
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Humans and other primates can robustly report whether they've seen specific images before, even when those images are extremely similar to ones they've previously seen. Multiple lines of evidence suggest that pattern separation computations in the hippocampus (HC) contribute to this behavior by shaping the fidelity of visual memory. However, unclear is whether HC uniquely determines memory fidelity or whether computations in other brain areas also contribute. To investigate, we recorded neural signals from inferotemporal cortex (ITC) and HC of two rhesus monkeys (1 male, 1 female) as they performed a memory task in which they judged whether images were novel or exactly repeated in the presence of visually similar lure images with a range of visual similarities. We found behavioral evidence for sharpening, reflected as memory performance that was nonlinearly transformed relative to a benchmark defined by visual representations in ITC. As expected, we found that behavioral sharpening aligned with visual memory representations in HC. Surprisingly, and unaccounted for by HC pattern separation proposals, we also found neural correlates of behavioral sharpening reflected in ITC. These results, coupled with further analysis of the data, suggest that ITC contributes to shaping the fidelity of visual memory in the transformation from visual processing to memory storage and signaling.more » « less
-
The prefrontal cortex is larger than would be predicted by body size or visual cortex volume in great apes compared with monkeys. Because prefrontal cortex is critical for working memory, we hypothesized that recognition memory tests would engage working memory in orangutans more robustly than in rhesus monkeys. In contrast to working memory, the familiarity response that results from repetition of an image is less cognitively taxing and has been associated with nonfrontal brain regions. Across three experiments, we observed a striking species difference in the control of behavior by these two types of memory. First, we found that recognition memory performance in orangutans was controlled by working memory under conditions in which this memory system plays little role in rhesus monkeys. Second, we found that unlike the case in monkeys, familiarity was not involved in recognition memory performance in orangutans, shown by differences with monkeys across three different measures. Memory in orangutans was not improved by use of novel images, was always impaired by a concurrent cognitive load, and orangutans did not accurately identify images seen minutes ago. These results are surprising and puzzling, but do support the view that prefrontal expansion in great apes favored working memory. At least in orangutans, increased dependence on working memory may come at a cost in terms of the availability of familiarity.more » « less
-
Episodic memories are records of personally experienced events, coded neurally via the hippocampus and sur- rounding medial temporal lobe cortex. Information about the neural signal corresponding to a memory representation can be measured in fMRI data when the pattern across voxels is examined. Prior studies have found that similarity in the voxel patterns across repetition of a to-be-remembered stimulus predicts later memory retrieval, but the results are inconsistent across studies. The current study investigates the possibility that cognitive goals (defined here via the task instructions given to participants) during encoding affect the voxel pattern that will later support memory retrieval, and therefore that neural representations cannot be interpreted based on the stimulus alone. The behavioral results showed that exposure to variable cognitive tasks across repetition of events benefited subsequent memory retrieval. Voxel patterns in the hippocampus indicated a significant interaction between cognitive tasks (variable vs. consistent) and memory (remembered vs. forgotten) such that reduced voxel pattern similarity for repeated events with variable cognitive tasks, but not consistent cognitive tasks, sup- ported later memory success. There was no significant interaction in neural pattern similarity between cognitive tasks and memory success in medial temporal cortices or lateral occipital cortex. Instead, higher similarity in voxel patterns in right medial temporal cortices was associated with later memory retrieval, regardless of cognitive task. In conclusion, we found that the relationship between pattern similarity across repeated encoding and memory success in the hippocampus (but not medial temporal lobe cortex) changes when the cognitive task during encoding does or does not vary across repetitions of the event.more » « less
-
While viewing a visual stimulus, we often cannot tell whether it is inherently memorable or forgettable. However, the memorability of a stimulus can be quantified and partially predicted by a collection of conceptual and perceptual factors. Higher-level properties that represent the “meaningfulness” of a visual stimulus to viewers best predict whether it will be remembered or forgotten across a population. Here, we hypothesize that the feelings evoked by an image, operationalized as the valence and arousal dimensions of affect, significantly contribute to the memorability of scene images. We ran two complementary experiments to investigate the influence of affect on scene memorability, in the process creating a new image set (VAMOS) of hundreds of natural scene images for which we obtained valence, arousal, and memorability scores. From our first experiment, we found memorability to be highly reliable for scene images that span a wide range of evoked arousal and valence. From our second experiment, we found that both valence and arousal are significant but weak predictors of image memorability. Scene images were most memorable if they were slightly negatively valenced and highly arousing. Images that were extremely positive or unarousing were most forgettable. Valence and arousal together accounted for less than 8% of the variance in image memorability. These findings suggest that evoked affect contributes to the overall memorability of a scene image but, like other singular predictors, does not fully explain it. Instead, memorability is best explained by an assemblage of visual features that combine, in perhaps unintuitive ways, to predict what is likely to stick in our memory.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Posted Content:
- https://doi.org/10.1101/2025.06.13.659490
