skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective
Abstract Reinforcement learning and working memory are two core processes of human cognition and are often considered cognitively, neuroscientifically, and algorithmically distinct. Here, we show that the brain networks that support them actually overlap significantly and that they are less distinct cognitive processes than often assumed. We review literature demonstrating the benefits of considering each process to explain properties of the other and highlight recent work investigating their more complex interactions. We discuss how future research in both computational and cognitive sciences can benefit from one another, suggesting that a key missing piece for artificial agents to learn to behave with more human-like efficiency is taking working memory's role in learning seriously. This review highlights the risks of neglecting the interplay between different processes when studying human behavior (in particular when considering individual differences). We emphasize the importance of investigating these dynamics to build a comprehensive understanding of human cognition.  more » « less
Award ID(s):
2020844
PAR ID:
10343191
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Journal of Cognitive Neuroscience
Volume:
34
Issue:
4
ISSN:
0898-929X
Page Range / eLocation ID:
551 to 568
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Nature Reviews Psychology)
    Schubert, Teresa (Ed.)
    Drawing is a cognitive tool that makes the invisible contents of mental life visible. Humans use this tool to produce a remarkable variety of pictures, from realistic portraits to schematic diagrams. Despite this variety and the prevalence of drawn images, the psychological mechanisms that enable drawings to be so versatile have yet to be fully explored. In this Review, we synthesize contemporary work in multiple areas of psychology, computer science and neuroscience that examines the cognitive processes involved in drawing production and comprehension. This body of findings suggests that the balance of contributions from perception, memory and social inference during drawing production varies depending on the situation, resulting in some drawings that are more realistic and other drawings that are more abstract. We also consider the use of drawings as a research tool for investigating various aspects of cognition, as well as the role that drawing has in facilitating learning and communication. Taken together, information about how drawings are used in different contexts illuminates the central role of visually grounded abstractions in human thought and behaviour. 
    more » « less
  2. ; ; ; ; (, Integrative and Comparative Biology)
    Synopsis The term “cognitive template” originated from work in human-based cognitive science to describe a literal, stored, neural representation used in recognition tasks. As the study of cognition has expanded to nonhuman animals, the term has diffused to describe a wider range of animal cognitive tools and strategies that guide action through the recognition of and discrimination between external states. One potential reason for this nonstandardized meaning and variable employment is that researchers interested in the broad range of animal recognition tasks enjoy the simplicity of the cognitive template concept and have allowed it to become shorthand for many dissimilar or unknown neural processes without deep scrutiny of how this metaphor might comport with underlying neurophysiology. We review the functional evidence for cognitive templates in fields such as perception, navigation, communication, and learning, highlighting any neural correlates identified by these studies. We find that the concept of cognitive templates has facilitated valuable exploration at the interface between animal behavior and cognition, but the quest for a literal template has failed to attain mechanistic support at the level of neurophysiology. This may be the result of a misled search for a single physical locus for the “template” itself. We argue that recognition and discrimination processes are best treated as emergent and, as such, may not be physically localized within single structures of the brain. Rather, current evidence suggests that such tasks are accomplished through synergies between multiple distributed processes in animal nervous systems. We thus advocate for researchers to move toward a more ecological, process-oriented conception, especially when discussing the neural underpinnings of recognition-based cognitive tasks. 
    more » « less
  3. ; ; ; (, Annual Review of Vision Science)
    Animals live in visually complex environments. As a result, visual systems have evolved mechanisms that simplify visual processing and allow animals to focus on the information that is most relevant to adaptive decision making. This review explores two key mechanisms that animals use to efficiently process visual information: categorization and specialization. Categorization occurs when an animal's perceptual system sorts continuously varying stimuli into a set of discrete categories. Specialization occurs when particular classes of stimuli are processed using distinct cognitive operations that are not used for other classes of stimuli. We also describe a nonadaptive consequence of simplifying heuristics: visual illusions, where visual perception consistently misleads the viewer about the state of the external world or objects within it. We take an explicitly comparative approach by exploring similarities and differences in visual cognition across human and nonhuman taxa. Considering areas of convergence and divergence across taxa provides insight into the evolution and function of visual systems and associated perceptual strategies. 
    more » « less
  4. ; ; (, Annals of the New York Academy of Sciences)
    Abstract The remarkable successes of convolutional neural networks (CNNs) in modern computer vision are by now well known, and they are increasingly being explored as computational models of the human visual system. In this paper, we ask whether CNNs might also provide a basis for modeling higher‐level cognition, focusing on the core phenomena of similarity and categorization. The most important advance comes from the ability of CNNs to learn high‐dimensional representations of complex naturalistic images, substantially extending the scope of traditional cognitive models that were previously only evaluated with simple artificial stimuli. In all cases, the most successful combinations arise when CNN representations are used with cognitive models that have the capacity to transform them to better fit human behavior. One consequence of these insights is a toolkit for the integration of cognitively motivated constraints back into CNN training paradigms in computer vision and machine learning, and we review cases where this leads to improved performance. A second consequence is a roadmap for how CNNs and cognitive models can be more fully integrated in the future, allowing for flexible end‐to‐end algorithms that can learn representations from data while still retaining the structured behavior characteristic of human cognition. 
    more » « less
  5. ; (, ACM)
    Working memory is an important component of cognition that infuences key cognitive processes, such as language. As such, working memory should play a key role in cognitive models for languagecapable robots. The ways in which working memory bufers are organized within a robot’s architecture can inform processes such as Referring Expression Generation. Thus, it is important to understand how information and resources within working memory may be organized to lead to human-like robotic language. Previous work on the DIARC cognitive architecture described an entitylevel, feature-based working memory framework in which each known entity had its own dedicated working memory bufer. This paper expands on that framework and proposes a new resource management strategy in which sets of entities that belong to the same type share a single working memory bufer.We end the paper with a brief discussion of how this novel strategy compares to the previously implemented entity-level strategy. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email