More Like this

1. Chunks are not "Content-Free": Hierarchical Representations Preserve Perceptual Detail within Chunks.

   Allen, Michael G; Destefano, Isabella; Brady, Timothy F. (July 2021, Proceedings of the Annual Conference of the Cognitive Science Society)

   null (Ed.)

   Chunks allow us to use long-term knowledge to efficiently represent the world in working memory. Most views of chunking assume that when we use chunks, this results in the loss of specific perceptual details, since it is presumed the contents of chunks are decoded from long-term memory rather than reflecting the exact details of the item that was presented. However, in two experiments, we find that in situations where participants make use of chunks to improve visual working memory, access to instance-specific perceptual detail (that cannot be retrieved from long-term memory) increased, rather than decreased. This supports an alternative view: that chunks facilitate the encoding and retention into memory of perceptual details as part of structured, hierarchical memories, rather than serving as mere “content-free” pointers. It also provides a strong contrast to accounts in which working memory capacity is assumed to be exhaustively described by the number of chunks remembered. 

   more » « less
2. Hierarchical Tree-Based Sequential Event Prediction with Application in the Aviation Accident Report

   https://doi.org/10.1109/ICDE51399.2021.00178  

   Zhao, Xinyu; Yan, Hao; Liu, Yongming (August 2021, Proceedings - 2021 IEEE 37th International Conference on Data Engineering, ICDE 2021)

   Sequential event prediction is a well-studied area and has been widely used in proactive management, recommender systems and healthcare. One major assumption of the existing sequential event prediction methods is that similar event sequence patterns in the historical record will repeat themselves, enabling us to predict future events. However, in reality, the assumption becomes less convincing when we are trying to predict rare or unique sequences. Furthermore, the representation of the event may be complex with hierarchical structures. In this paper, we aim to solve this issue by taking advantage of the multi-level or hierarchical representation of these rare events. We proposed to build a sequential Encoder-Decoder framework to predict the event sequences. More specifically, in the encoding layer, we built a hierarchical embedding representation for the events. In the decoding layer, we first predict the high-level events and the low-level events are generated according to a hierarchical graphical structure. We propose to link the encoding decoding layers with the temporal models for future event prediction. In this article, we further discussed applying the proposed model into the failure event prediction according to the aviation accident reports and have shown improved accuracy and model interpretability. 

   more » « less
3. Control of working memory by phase–amplitude coupling of human hippocampal neurons

   https://doi.org/10.1038/s41586-024-07309-z  

   Daume, Jonathan; Kamiński, Jan; Schjetnan, Andrea_G P; Salimpour, Yousef; Khan, Umais; Kyzar, Michael; Reed, Chrystal M; Anderson, William S; Valiante, Taufik A; Mamelak, Adam N; et al (May 2024, Nature)

   Abstract Retaining information in working memory is a demanding process that relies on cognitive control to protect memoranda-specific persistent activity from interference^1,2. However, how cognitive control regulates working memory storage is unclear. Here we show that interactions of frontal control and hippocampal persistent activity are coordinated by theta–gamma phase–amplitude coupling (TG-PAC). We recorded single neurons in the human medial temporal and frontal lobe while patients maintained multiple items in their working memory. In the hippocampus, TG-PAC was indicative of working memory load and quality. We identified cells that selectively spiked during nonlinear interactions of theta phase and gamma amplitude. The spike timing of these PAC neurons was coordinated with frontal theta activity when cognitive control demand was high. By introducing noise correlations with persistently active neurons in the hippocampus, PAC neurons shaped the geometry of the population code. This led to higher-fidelity representations of working memory content that were associated with improved behaviour. Our results support a multicomponent architecture of working memory^1,2, with frontal control managing maintenance of working memory content in storage-related areas^3–5. Within this framework, hippocampal TG-PAC integrates cognitive control and working memory storage across brain areas, thereby suggesting a potential mechanism for top-down control over sensory-driven processes. 

   more » « less
4. Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex

   https://doi.org/10.1371/journal.pcbi.1011801  

   Jiang, Linxing Preston; Rao, Rajesh_P N (February 2024, PLOS Computational Biology)

   Rubin, Jonathan (Ed.)

   We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network’s hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world. 

   more » « less
5. Co-operation of long-term and working memory representations in simultaneous chaining by rhesus monkeys ( Macaca mulatta )

   https://doi.org/10.1177/1747021819838432  

   Templer, Victoria L.; Gazes, Regina Paxton; Hampton, Robert R. (April 2019, Quarterly Journal of Experimental Psychology)

   We studied the memory representations that control execution of action sequences by training rhesus monkeys ( Macaca mulatta) to touch sets of five images in a predetermined arbitrary order (simultaneous chaining). In Experiment 1, we found that this training resulted in mental representations of ordinal position rather than learning associative chains, replicating the work of others. We conducted novel analyses of performance on probe tests consisting of two images “derived” from the full five-image lists (i.e., test B, D from list A→B→C→D→E). We found a “first item effect” such that monkeys responded most quickly to images that occurred early in the list in which they had been learned, indicating that monkeys covertly execute known lists mentally until an image on the screen matches the one stored in memory. Monkeys also made an ordinal comparison of the two images presented at test based on long-term memory of positional information, resulting in a “symbolic distance effect.” Experiment 2 indicated that ordinal representations were based on absolute, rather than on relative, positional information because subjects did not link two lists into one large list after linking training, unlike what occurs in transitive inference. We further examined the contents of working memory during list execution in Experiments 3 and 4 and found evidence for a prospective, rather than a retrospective, coding of position in the lists. These results indicate that serial expertise in simultaneous chaining results in robust absolute ordinal coding in long-term memory, with rapidly updating prospective coding of position in working memory during list execution. 

   more » « less