More Like this

1. Integration of event experiences to build relational knowledge in the human brain

   https://doi.org/10.1093/cercor/bhad260  

   Leshinskaya, Anna; Nguyen, Mitchell_A; Ranganath, Charan (July 2023, Cerebral Cortex)

   Abstract We investigated how the human brain integrates experiences of specific events to build general knowledge about typical event structure. We examined an episodic memory area important for temporal relations, anterior-lateral entorhinal cortex, and a semantic memory area important for action concepts, middle temporal gyrus, to understand how and when these areas contribute to these processes. Participants underwent functional magnetic resonance imaging while learning and recalling temporal relations among novel events over two sessions 1 week apart. Across distinct contexts, individual temporal relations among events could either be consistent or inconsistent with each other. Within each context, during the recall phase, we measured associative coding as the difference of multivoxel correlations among related vs unrelated pairs of events. Neural regions that form integrative representations should exhibit stronger associative coding in the consistent than the inconsistent contexts. We found evidence of integrative representations that emerged quickly in anterior-lateral entorhinal cortex (at session 1), and only subsequently in middle temporal gyrus, which showed a significant change across sessions. A complementary pattern of findings was seen with signatures during learning. This suggests that integrative representations are established early in anterior-lateral entorhinal cortex and may be a pathway to the later emergence of semantic knowledge in middle temporal gyrus. 

   more » « less
2. Evidence for a Spoken Word Lexicon in the Auditory Ventral Stream

   https://doi.org/10.1162/nol_a_00108  

   Damera, Srikanth R.; Chang, Lillian; Nikolov, Plamen P.; Mattei, James A.; Banerjee, Suneel; Glezer, Laurie S.; Cox, Patrick H.; Jiang, Xiong; Rauschecker, Josef P.; Riesenhuber, Maximilian (July 2023, Neurobiology of Language)

   Abstract The existence of a neural representation for whole words (i.e., a lexicon) is a common feature of many models of speech processing. Prior studies have provided evidence for a visual lexicon containing representations of whole written words in an area of the ventral visual stream known as the visual word form area. Similar experimental support for an auditory lexicon containing representations of spoken words has yet to be shown. Using functional magnetic resonance imaging rapid adaptation techniques, we provide evidence for an auditory lexicon in the auditory word form area in the human left anterior superior temporal gyrus that contains representations highly selective for individual spoken words. Furthermore, we show that familiarization with novel auditory words sharpens the selectivity of their representations in the auditory word form area. These findings reveal strong parallels in how the brain represents written and spoken words, showing convergent processing strategies across modalities in the visual and auditory ventral streams. 

   more » « less
3. Temporal context and latent state inference in the hippocampal splitter signal

   https://doi.org/10.7554/eLife.82357  

   Duvelle, Éléonore; Grieves, Roddy M; van der Meer, Matthijs AA (January 2023, eLife)

   The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by ‘splitter cells’, hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function. 

   more » « less
4. On the Role of Prediction in Streaming Hierarchical Learning

   Mounir, Ramy (October 2024, Digital Commons at University of South Florida)

   In today's world, AI systems need to make sense of large amounts of data as it unfolds in real-time, whether it's a video from surveillance and monitoring cameras, streams of egocentric footage, or sequences in other domains such as text or audio. The ability to break these continuous data streams into meaningful events, discover nested structures, and predict what might happen next at different levels of abstraction is crucial for applications ranging from passive surveillance systems to sensory-motor autonomous learning. However, most existing models rely heavily on large, annotated datasets with fixed data distributions and offline epoch-based training, which makes them impractical for handling the unpredictability and scale of dynamic real-world environments. This dissertation tackles these challenges by introducing a set of predictive models designed to process streaming data efficiently, segment events, and build sequential memory models without supervision or data storage. First, we present a single-layer predictive model that segments long, unstructured video streams by detecting temporal events and spatially localizing objects in each frame. The model is applied to wildlife monitoring footage, where it processes continuous, high-frame-rate video and successfully detects and tracks events without supervision. It operates in an online streaming manner to perform simultaneous training and inference without storing or revisiting the processed data. This approach alleviates the need for manual labeling, making it ideal for handling long-duration, real-world video footage. Building on this, we introduce STREAMER, a multi-layered architecture that extends the single-layer model into a hierarchical predictive framework. STREAMER segments events at different levels of abstraction, capturing the compositional structure of activities in egocentric videos. By dynamically adapting to various timescales, it creates a hierarchy of nested events and forms more complex and abstract representations of the input data. Finally, we propose the Predictive Attractor Model (PAM), which builds biologically plausible memory models of sequential data. Inspired by neuroscience, PAM uses sparse distributed representations and local learning rules to avoid catastrophic forgetting, allowing it to continually learn and make predictions without overwriting previous knowledge. Unlike many traditional models, PAM can generate multiple potential future outcomes conditioned on the same context, which allows for handling uncertainty in generative tasks. Together, these models form a unified framework of predictive learning that addresses multiple challenges in event understanding and temporal data analyses. By using prediction as the core mechanism, they segment continuous data streams into events, discover hierarchical structures across multiple levels of abstraction, learn semantic event representations, and model sequences without catastrophic forgetting. 

   more » « less
5. Numeric social-media posts engage people with climate science

   https://doi.org/10.1093/pnasnexus/pgae250  

   Peters, Ellen; Markowitz, David M; Nadratowski, Ariel; Shoots-Reinhard, Brittany (June 2024, PNAS Nexus)

   Rand, David (Ed.)

   Abstract Innumeracy (lack of math skills) among nonscientists often leads climate scientists and others to avoid communicating numbers due to concerns that the public will not understand them and may disengage. However, people often report preferring to receive numbers; providing them also can improve decisions. Here, we demonstrated that the presence vs. absence of at least one Arabic integer in climate-related social-media posts increased sharing up to 31.7% but, counter to hypothesis, decreased liking of messages 5.2% in two preregistered observational studies (climate scientists on Twitter, N  > 8 million Tweets; climate subreddit, N  > 17,000 posts and comments). We speculated that the decreased liking was due, not to reduced engagement, but to more negative feelings towards climate-related content described with numeric precision. A preregistered within-participant experiment (N = 212) then varied whether climate consequences were described using Arabic integers (e.g. “90%”) or another format (e.g. verbal terms, “almost all”). The presence of Arabic integers about consequences led to more sharing, wanting to find out more, and greater trust and perceptions of an expert messenger; perceived trust and expertise appeared to mediate effects on sharing and wanting to find out more. Arabic integers about consequences again led to more negative feelings about the Tweets as if numbers clarified the dismaying magnitude of climate threats. Our results indicate that harnessing the power of numbers could increase public trust and concern regarding this defining issue of our time. Communicators, however, should also consider counteracting associated negative feelings—that could halt action—by providing feasible solutions to increase people's self-efficacy. 

   more » « less