skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Linear reinforcement learning in planning, grid fields, and cognitive control
Abstract It is thought that the brain’s judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. Building on control engineering, here we introduce a model for decision making in the brain that reuses a temporally abstracted map of future events to enable biologically-realistic, flexible choice at the expense of specific, quantifiable biases. It replaces the classic nonlinear, model-based optimization with a linear approximation that softly maximizes around (and is weakly biased toward) a default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably flexible replanning with biases and cognitive control. It also provides insight into how the brain can represent maps of long-distance contingencies stably and componentially, as in entorhinal response fields, and exploit them to guide choice even under changing goals.  more » « less
Award ID(s):
1822571
PAR ID:
10300308
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Nature Communications
Volume:
12
Issue:
1
ISSN:
2041-1723
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. (, Current Opinion in Behavioral Sciences)
    Our ability to overcome habitual responses in favor of goal-driven novel responses depends on frontoparietal cognitive control networks (CCNs). Recent and ongoing work is revealing the brain network and information processes that allow CCNs to generate cognitive flexibility. First, working memory processes necessary for flexible maintenance and manipulation of goal-relevant representations were recently found to depend on short-term network plasticity (in contrast to persistent activity) within CCN regions. Second, compositional (i.e. abstract and reusable) rule representations maintained within CCNs have been found to reroute network activity flows from stimulus to response, enabling flexible behavior. Together, these findings suggest cognitive flexibility is enhanced by CCN-coordinated network mechanisms, utilizing compositional reuse of neural representations and network flows to flexibly accomplish task goals. 
    more » « less
  2. ; ; ; (, IEEE Conference on Decision and Control)
    In this paper, we propose a framework to control brain-wide functional connectivity by selectively acting on the brain's structure and parameters. Functional connectivity, which measures the degree of correlation between neural activities in different brain regions, can be used to distinguish between healthy and certain diseased brain dynamics and, possibly, as a control parameter to restore healthy functions. In this work, we use a collection of interconnected Kuramoto oscillators to model oscillatory neural activity, and show that functional connectivity is essentially regulated by the degree of synchronization between different clusters of oscillators. Then, we propose a minimally invasive method to correct the oscillators' interconnections and frequencies to enforce arbitrary and stable synchronization patterns among the oscillators and, consequently, a desired pattern of functional connectivity. Additionally, we show that our synchronization-based framework is robust to parameter mismatches and numerical inaccuracies, and validate it using a realistic neurovascular model to simulate neural activity and functional connectivity in the human brain. 
    more » « less
  3. ; ; (, IEEE International Conference on Industrial Internet (ICII))
    Cellular networks with D2D links are increasingly being explored for mission-critical applications (e.g., real-time control and AR/VR) which require predictable communication reliability. Thus it is critical to control interference among concurrent transmissions in a predictable manner to ensure the required communication reliability. To this end, we propose a Unified Cellular Scheduling (UCS) framework that, based on the Physical-Ratio-K (PRK) interference model, schedules uplink, downlink, and D2D transmissions in a unified manner to ensure predictable communication reliability while maximizing channel spatial reuse. UCS also provides a simple, effective approach to mode selection that maximizes the communication capacity for each involved communication pair. UCS effectively uses multiple channels for high throughput as well as resilience to channel fading and external interference. Leveraging the availability of base stations (BSes) as well as high-speed, out-of-band connectivity between BSes, UCS effectively orchestrates the functionalities of BSes and user equipment (UE) for light-weight control signaling and ease of incremental deployment and integration with existing cellular standards. We have implemented UCS using the open-source, standards-compliant cellular networking platform OpenAirInterface, and we have validated the UCS design and implementation using the USRP B210 software-defined radios in the ORBIT wireless testbed. We have also evaluated UCS through high-fidelity, at-scale simulation studies; we observe that UCS ensures predictable communication reliability while achieving a higher channel spatial reuse rate than existing mechanisms, and that the distributed UCS framework enables a channel spatial reuse rate statistically equal to that in the state-of-the-art centralized scheduling algorithm iOrder. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; ; et al (, Geoscientific Model Development)
    Abstract. The ocean mixed layer is the interface between the ocean interior and the atmosphere or sea ice and plays a key role in climate variability. It isthus critical that numerical models used in climate studies are capable of a good representation of the mixed layer, especially its depth. Here weevaluate the mixed-layer depth (MLD) in six pairs of non-eddying (1∘ grid spacing) and eddy-rich (up to 1/16∘) models from theOcean Model Intercomparison Project (OMIP), forced by a common atmospheric state. For model evaluation, we use an updated MLD dataset computed fromobservations using the OMIP protocol (a constant density threshold). In winter, low-resolution models exhibit large biases in the deep-waterformation regions. These biases are reduced in eddy-rich models but not uniformly across models and regions. The improvement is most noticeable inthe mode-water formation regions of the Northern Hemisphere. Results in the Southern Ocean are more contrasted, with biases of either sign remainingat high resolution. In eddy-rich models, mesoscale eddies control the spatial variability in MLD in winter. Contrary to a hypothesis that thedeepening of the mixed layer in anticyclones would make the MLD larger globally, eddy-rich models tend to have a shallower mixed layer at mostlatitudes than coarser models do. In addition, our study highlights the sensitivity of the MLD computation to the choice of a reference level andthe spatio-temporal sampling, which motivates new recommendations for MLD computation in future model intercomparison projects. 
    more » « less
  5. ; ; ; (, Monthly Notices of the Royal Astronomical Society)
    null (Ed.)
    ABSTRACT Photometric galaxy surveys constitute a powerful cosmological probe but rely on the accurate characterization of their redshift distributions using only broad-band imaging, and can be very sensitive to incomplete or biased priors used for redshift calibration. A hierarchical Bayesian model has recently been developed to estimate those from the robust combination of prior information, photometry of single galaxies, and the information contained in the galaxy clustering against a well-characterized tracer population. In this work, we extend the method so that it can be applied to real data, developing some necessary new extensions to it, especially in the treatment of galaxy clustering information, and we test it on realistic simulations. After marginalizing over the mapping between the clustering estimator and the actual density distribution of the sample galaxies, and using prior information from a small patch of the survey, we find the incorporation of clustering information with photo-z’s tightens the redshift posteriors and overcomes biases in the prior that mimic those happening in spectroscopic samples. The method presented here uses all the information at hand to reduce prior biases and incompleteness. Even in cases where we artificially bias the spectroscopic sample to induce a shift in mean redshift of $$\Delta \bar{z} \approx 0.05,$$ the final biases in the posterior are $$\Delta \bar{z} \lesssim 0.003.$$ This robustness to flaws in the redshift prior or training samples would constitute a milestone for the control of redshift systematic uncertainties in future weak lensing analyses. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email