skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Linear reinforcement learning in planning, grid fields, and cognitive control
Abstract It is thought that the brain’s judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. Building on control engineering, here we introduce a model for decision making in the brain that reuses a temporally abstracted map of future events to enable biologically-realistic, flexible choice at the expense of specific, quantifiable biases. It replaces the classic nonlinear, model-based optimization with a linear approximation that softly maximizes around (and is weakly biased toward) a default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably flexible replanning with biases and cognitive control. It also provides insight into how the brain can represent maps of long-distance contingencies stably and componentially, as in entorhinal response fields, and exploit them to guide choice even under changing goals.  more » « less
Award ID(s):
1822571
PAR ID:
10300308
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Nature Communications
Volume:
12
Issue:
1
ISSN:
2041-1723
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Nature Communications)
    Abstract Decision-makers often process new evidence selectively, depending on their current beliefs about the world. We asked whether such confirmation biases result from biases in the encoding of sensory evidence in the brain, or alternatively in the utilization of encoded evidence for behavior. Human participants estimated the source of a sequence of visual-spatial evidence samples while we measured cortical population activity with magnetoencephalography. Halfway through the sequence, participants were prompted to judge the more likely source category. We find that processing of subsequent evidence depends on its consistency with the previously chosen category. Evidence encoded in parietal cortex contributes more to the estimation report when that evidence is consistent with the previous choice compared to when it contradicts that choice. Our results indicate that information contradicting pre-existing beliefs has little impact on subsequent behavior, despite being precisely encoded in the brain. This provides room for deliberative control to counteract confirmation biases. 
    more » « less
  2. (, Current Opinion in Behavioral Sciences)
    Our ability to overcome habitual responses in favor of goal-driven novel responses depends on frontoparietal cognitive control networks (CCNs). Recent and ongoing work is revealing the brain network and information processes that allow CCNs to generate cognitive flexibility. First, working memory processes necessary for flexible maintenance and manipulation of goal-relevant representations were recently found to depend on short-term network plasticity (in contrast to persistent activity) within CCN regions. Second, compositional (i.e. abstract and reusable) rule representations maintained within CCNs have been found to reroute network activity flows from stimulus to response, enabling flexible behavior. Together, these findings suggest cognitive flexibility is enhanced by CCN-coordinated network mechanisms, utilizing compositional reuse of neural representations and network flows to flexibly accomplish task goals. 
    more » « less
  3. ; ; ; (, IEEE Conference on Decision and Control)
    In this paper, we propose a framework to control brain-wide functional connectivity by selectively acting on the brain's structure and parameters. Functional connectivity, which measures the degree of correlation between neural activities in different brain regions, can be used to distinguish between healthy and certain diseased brain dynamics and, possibly, as a control parameter to restore healthy functions. In this work, we use a collection of interconnected Kuramoto oscillators to model oscillatory neural activity, and show that functional connectivity is essentially regulated by the degree of synchronization between different clusters of oscillators. Then, we propose a minimally invasive method to correct the oscillators' interconnections and frequencies to enforce arbitrary and stable synchronization patterns among the oscillators and, consequently, a desired pattern of functional connectivity. Additionally, we show that our synchronization-based framework is robust to parameter mismatches and numerical inaccuracies, and validate it using a realistic neurovascular model to simulate neural activity and functional connectivity in the human brain. 
    more » « less
  4. ; ; ; (, Physica A: Statistical Mechanics and its Applications)
    For pedestrians moving without spatial constraints, extensive research has been devoted to develop methods of density estimation. In this paper we present a new approach based on Voronoi cells, offering a means to estimate density for individuals in small, unbounded pedestrian groups. A thorough evaluation of existing methods, encompassing both Lagrangian and Eulerian approaches employed in similar contexts, reveals notable limitations. Specifically, these methods turn out to be ill-defined for realistic density estimation along a pedestrian’s trajectory, exhibiting systematic biases and fluctuations that depend on the choice of parameters. There is thus a need for a parameter-independent method to eliminate this bias. We propose a modification of the widely used Voronoi-cell based density estimate to accommodate pedestrian groups, irrespective of their size. The advantages of this modified Voronoi method are that it is an instantaneous method that requires only knowledge of the pedestrians’ positions at a give time, does not depend on the choice of parameter values, gives us a realistic estimate of density in an individual’s neighborhood, and has appropriate physical meaning for both small and large human crowds in a wide variety of situations. We conclude with general remarks about the meaning of density measurements for small groups of pedestrians. 
    more » « less
  5. ; ; (, IEEE International Conference on Industrial Internet (ICII))
    Cellular networks with D2D links are increasingly being explored for mission-critical applications (e.g., real-time control and AR/VR) which require predictable communication reliability. Thus it is critical to control interference among concurrent transmissions in a predictable manner to ensure the required communication reliability. To this end, we propose a Unified Cellular Scheduling (UCS) framework that, based on the Physical-Ratio-K (PRK) interference model, schedules uplink, downlink, and D2D transmissions in a unified manner to ensure predictable communication reliability while maximizing channel spatial reuse. UCS also provides a simple, effective approach to mode selection that maximizes the communication capacity for each involved communication pair. UCS effectively uses multiple channels for high throughput as well as resilience to channel fading and external interference. Leveraging the availability of base stations (BSes) as well as high-speed, out-of-band connectivity between BSes, UCS effectively orchestrates the functionalities of BSes and user equipment (UE) for light-weight control signaling and ease of incremental deployment and integration with existing cellular standards. We have implemented UCS using the open-source, standards-compliant cellular networking platform OpenAirInterface, and we have validated the UCS design and implementation using the USRP B210 software-defined radios in the ORBIT wireless testbed. We have also evaluated UCS through high-fidelity, at-scale simulation studies; we observe that UCS ensures predictable communication reliability while achieving a higher channel spatial reuse rate than existing mechanisms, and that the distributed UCS framework enables a channel spatial reuse rate statistically equal to that in the state-of-the-art centralized scheduling algorithm iOrder. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email