An official website of the United States government
Extensive studies in rodents show that place cells in the hippocampus have firing patterns that are highly correlated with the animal's location in the environment and are organized in layers of increasing field sizes or scales along its dorsoventral axis. In this study, we use a spatial cognition model to show that different field sizes could be exploited to adapt the place cell representation to different environments according to their size and complexity. Specifically, we provide an in-depth analysis of how to distribute place cell fields according to the obstacles in cluttered environments to optimize learning time and path optimality during goal-oriented spatial navigation tasks. The analysis uses a reinforcement learning (RL) model that assumes that place cells allow encoding the state. While previous studies have suggested exploiting different field sizes to represent areas requiring different spatial resolutions, our work analyzes specific distributions that adapt the representation to the environment, activating larger fields in open areas and smaller fields near goals and subgoals (e.g., obstacle corners). In addition to assessing how the multi-scale representation may be exploited in spatial navigation tasks, our analysis and results suggest place cell representations that can impact the robotics field by reducing the total number of cells for path planning without compromising the quality of the paths learned.
more »
« less
A Computational Model for a Multi-Goal Spatial Navigation Task inspired by Rodent Studies
We present a biologically-inspired computational model of the rodent hippocampus based on recent studies of the hippocampus showing that its longitudinal axis is involved in complex spatial navigation. While both poles of the hippocampus, i.e. septal (dorsal) and temporal (ventral), encode spatial information; the septal area has traditionally been attributed more to navigation and action selection; whereas the temporal pole has been more involved with learning and motivation. In this work we hypothesize that the septal-temporal organization of the hippocampus axis also provides a multi-scale spatial representation that may be exploited during complex rodent navigation. To test this hypothesis, we developed a multi-scale model of the hippocampus evaluated it with a simulated rat on a multi-goal task, initially in a simplified environment, and then on a more complex environment where multiple obstacles are introduced. In addition to the hippocampus providing a spatial representation of the environment, the model includes an actor-critic framework for the motivated learning of the different tasks.
more »
« less
- Award ID(s):
- 1703225
- PAR ID:
- 10180133
- Date Published:
- Journal Name:
- IJCNN
- Page Range / eLocation ID:
- 1 to 8
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract A large body of evidence shows that the hippocampus is necessary for successful spatial navigation. Various studies have shown anatomical and functional differences between the dorsal (DHC) and ventral (VHC) portions of this structure. The DHC is primarily involved in spatial navigation and contains cells with small place fields. The VHC is primarily involved in context and emotional encoding contains cells with large place fields and receives major projections from the medial prefrontal cortex. In the past, spatial navigation experiments have used relatively simple tasks that may not have required a strong coordination along the dorsoventral hippocampal axis. In this study, we tested the hypothesis that the DHC and VHC may be critical for goal‐directed navigation in obstacle‐rich environments. We used a learning task in which animals memorize the location of a set of rewarded feeders, and recall these locations in the presence of small or large obstacles. We report that bilateral DHC or VHC inactivation impaired spatial navigation in both large and small obstacle conditions. Importantly, this impairment did not result from a deficit in the spatial memory for the set of feeders (i.e., recognition of the goal locations) because DHC or VHC inactivation did not affect recall performance when there was no obstacle on the maze. We also show that the behavioral performance of the animals was correlated with several measures of maze complexity and that these correlations were significantly affected by inactivation only in the large object condition. These results suggest that as the complexity of the environment increases, both DHC and VHC are required for spatial navigation.more » « less
-
The hippocampus consists of a stereotyped neuronal circuit repeated along the septal-temporal axis. This transverse circuit contains distinct subfields with stereotyped connectivity that support crucial cognitive processes, including episodic and spatial memory. However, comprehensive measurements across the transverse hippocampal circuit in vivo are intractable with existing techniques. Here, we developed an approach for two-photon imaging of the transverse hippocampal plane in awake mice via implanted glass microperiscopes, allowing optical access to the major hippocampal subfields and to the dendritic arbor of pyramidal neurons. Using this approach, we tracked dendritic morphological dynamics on CA1 apical dendrites and characterized spine turnover. We then used calcium imaging to quantify the prevalence of place and speed cells across subfields. Finally, we measured the anatomical distribution of spatial information, finding a non-uniform distribution of spatial selectivity along the DG-to-CA1 axis. This approach extends the existing toolbox for structural and functional measurements of hippocampal circuitry.more » « less
-
A new housing development in a familiar neighborhood, a wrong turn that ends up lengthening a Sunday stroll: our internal representation of the world requires constant updating, and we need to be able to associate events separated by long intervals of time to finetune future outcome. This often requires neural connections to be altered. A brain region known as the hippocampus is involved in building and maintaining a map of our environment. However, signals from other brain areas can activate silent neurons in the hippocampus when the body is in a specific location by triggering cellular events called dendritic calcium spikes. Milstein et al. explored whether dendritic calcium spikes in the hippocampus could also help the brain to update its map of the world by enabling neurons to stop being active at one location and to start responding at a new position. Experiments in mice showed that calcium spikes could change which features of the environment individual neurons respond to by strengthening or weaking connections between specific cells. Crucially, this mechanism allowed neurons to associate event sequences that unfold over a longer timescale that was more relevant to the ones encountered in day-to-day life. A computational model was then put together, and it demonstrated that dendritic calcium spikes in the hippocampus could enable the brain to make better spatial decisions in future. Indeed, these spikes are driven by inputs from brain regions involved in complex cognitive processes, potentially enabling the delayed outcomes of navigational choices to guide changes in the activity and wiring of neurons. Overall, the work by Milstein et al. advances the understanding of learning and memory in the brain and may inform the design of better systems for artificial learning.more » « less
-
In public spaces shared with humans, ensuring multi-robot systems navigate without collisions while respecting social norms is challenging, particularly with limited communication. Although current robot social navigation techniques leverage advances in reinforcement learning and deep learning, they frequently overlook robot dynamics in simulations, leading to a simulation-to-reality gap. In this paper, we bridge this gap by presenting a new multi-robot social navigation environment crafted using Dec-POSMDP and multi-agent reinforcement learning. Furthermore, we introduce SAMARL: a novel benchmark for cooperative multi-robot social navigation. SAMARL employs a unique spatial-temporal transformer combined with multi-agent reinforcement learning. This approach effectively captures the complex interactions between robots and humans, thus promoting cooperative tendencies in multi-robot systems. Our extensive experiments reveal that SAMARL outperforms existing baseline and ablation models in our designed environment. Demo videos for this work can be found at: https://sites.google.com/view/samarlmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/IJCNN.2019.8851852
