An official website of the United States government
Autonomous mobile robots deployed in outdoor environments must reason about different types of terrain for both safety (e.g., prefer dirt over mud) and deployer preferences (e.g., prefer dirt path over flower beds). Most existing solutions to this preference-aware path planning problem use semantic segmentation to classify terrain types from camera images, and then ascribe costs to each type. Unfortunately, there are three key limitations of such approaches -- they 1) require pre-enumeration of the discrete terrain types, 2) are unable to handle hybrid terrain types (e.g., grassy dirt), and 3) require expensive labelled data to train visual semantic segmentation. We introduce Visual Representation Learning for Preference-Aware Path Planning (VRL-PAP), an alternative approach that overcomes all three limitations: VRL-PAP leverages unlabeled human demonstrations of navigation to autonomously generate triplets for learning visual representations of terrain that are viewpoint invariant and encode terrain types in a continuous representation space. The learned representations are then used along with the same unlabeled human navigation demonstrations to learn a mapping from the representation space to terrain costs. At run time, VRL-PAP maps from images to representations and then representations to costs to perform preference-aware path planning. We present empirical results from challenging outdoor settings that demonstrate VRL-PAP 1) is successfully able to pick paths that reflect demonstrated preferences, 2) is comparable in execution to geometric navigation with a highly detailed manually annotated map (without requiring such annotations), 3) is able to generalize to novel terrain types with minimal additional unlabeled demonstrations.
more »
« less
This content will become publicly available on May 19, 2026
Safer Online Replanning for Construction Robots via NLP-Informed Potential Fields and BIM Semantics
Autonomous navigation in construction environments is particularly challenging due to dynamic obstacles and uncertain surroundings. While recent advances in Building Information Modeling (BIM)-based planning have leveraged spatial and semantic information to improve navigation, most prior work assumes precise localization of the BIM model to enable global path planning. In contrast, this paper introduces an online replanning framework that registers obstacles on discovery within BIM and replans according to the updated semantic map. Our method integrates object-aware path planning by utilizing large language models (LLMs) to extract semantic danger sentiments from BIM-annotated objects and their spatial information about the mission environment. Additionally, we demonstrate practical feasibility by integrating a path tracking control, ensuring generated paths are not only safer but also realistically executable by mobile robots. Experimental results demonstrate an improved obstacle avoidance by 2.8× compared to traditional A* algorithms in dynamically updated environments.
more »
« less
- Award ID(s):
- 2047138
- PAR ID:
- 10617090
- Publisher / Repository:
- IEEE International Conference on Robotics and Automation
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Extensive studies in rodents show that place cells in the hippocampus have firing patterns that are highly correlated with the animal's location in the environment and are organized in layers of increasing field sizes or scales along its dorsoventral axis. In this study, we use a spatial cognition model to show that different field sizes could be exploited to adapt the place cell representation to different environments according to their size and complexity. Specifically, we provide an in-depth analysis of how to distribute place cell fields according to the obstacles in cluttered environments to optimize learning time and path optimality during goal-oriented spatial navigation tasks. The analysis uses a reinforcement learning (RL) model that assumes that place cells allow encoding the state. While previous studies have suggested exploiting different field sizes to represent areas requiring different spatial resolutions, our work analyzes specific distributions that adapt the representation to the environment, activating larger fields in open areas and smaller fields near goals and subgoals (e.g., obstacle corners). In addition to assessing how the multi-scale representation may be exploited in spatial navigation tasks, our analysis and results suggest place cell representations that can impact the robotics field by reducing the total number of cells for path planning without compromising the quality of the paths learned.more » « less
-
Indoor navigation in complex building environments poses significant challenges, particularly for individuals who are unfamiliar with their surroundings. Mixed reality (MR) technologies have emerged as a promising solution to enhance situational awareness and facilitate navigation within indoor spaces. However, there is a lack of spatial data for indoor environments, including outdated floor plans and limited real-time operational data. This paper presents the development of a mixed-reality application for indoor building navigation and evacuation. The application uses feature extraction for location sensing and situational awareness to provide accurate and reliable navigation in any indoor environment using Microsoft HoloLens. The application can track the user's position and orientation and give the user-specific information on how to evacuate the building. This information is then used to generate navigation instructions for the user. We demonstrate how this mixed reality HoloLens application can provide spatially contextualized 3D visualizations that promote spatial knowledge acquisition and situational awareness. These 3D visualizations are developed as an emergency evacuation and navigation tool to aid the building occupants in safe and quick evacuation. Experimental results demonstrate the effectiveness of the application, providing 3D visualizations of multilevel spaces and aiding individuals in understanding their position and evacuation path during emergencies. We believe that adopting mixed reality technologies, such as the HoloLens, can greatly enhance individuals' ability to navigate large-scale environments during emergencies by promoting spatial knowledge acquisition and supporting cognitive mapping.more » « less
-
Navigation among dynamic obstacles is a fundamental task in robotics that has been modeled in various ways. In Safe Interval Path Planning, location is discretized to a grid, time is continuous, future trajectories of obstacles are assumed known, and planning takes place offline. In this work, we define the Real-time Safe Interval Path Planning problem setting, in which the agent plans online and must issue its next action within a strict time bound. Unlike in classical real-time heuristic search, the cost-to-go in Real-time Safe Interval Path Planning is a function of time rather than a scalar. We present several algorithms for this setting and prove that they learn admissible heuristics. Empirical evaluation shows that the new methods perform better than classical approaches under a variety of conditions.more » « less
-
null (Ed.)From navigation in unfamiliar environments to career plan- ning, people typically first sample information before com- mitting to a plan. However, most studies find that people adopt myopic strategies when sampling information. Here we challenge those findings by investigating whether contingency planning is a driver of information sampling. To this aim, we developed a novel navigation task that is a shortest path find- ing problem under uncertainty of bridge closures. Participants (n = 109) were allowed to sample information on bridge sta- tuses prior to committing to a path. We developed a computa- tional model in which the agent samples information based on the cost of switching to a contingency plan. We find that this model fits human behavior well and is qualitatively similar to the approximated optimal solution. Together, this suggests that humans use contingency planning as a driver of information sampling.more » « less
