An official website of the United States government
Navigating complex real-world environments requires understanding the semantic context and effectively making decisions. Existing solutions leave room for improvements: traditional reactive approaches that do not maintain a map often struggle in complex environments, map-dependent methods demand significant effort in mapping processes, and learning-based methods rely on large training datasets and face the difficulty of generalization. To address these challenges, we propose a novel visual semantic navigation framework that combines data-driven semantic understanding, Pareto-optimal decision-making, and image-space planning. Our approach uses a local environmental representation callednavigability image, which allows the robot to assess immediate traversability without relying onapriorimapping or navigation data. Building on this, we introducePareto-Optimal Visual Navigation(POVNav), a decision-making framework in the image space that identifies appropriate subgoals, constructs collision-free paths, and generates control commands using visual servoing. This framework also supports selective navigation behaviors, such as avoiding traversable yet slippery grasslands to prevent getting stuck, by dynamically adjusting the navigability criteria within the local representation. POVNav is lightweight, operating solely with a monocular camera and without requiring map storage or training data collection, making it highly versatile for different robotic platforms and environments. Extensive year-round real-world experiments validated its efficacy in both structured indoor environments and unstructured outdoor settings, including dense forest trails and snow-covered roads. Field experiments using various image segmentation techniques demonstrated its robustness and adaptability across a wide range of conditions. Additionally, we demonstrate that POVNav successfully guides a robot through narrow pipes in a culvert inspection task. Overall, we showcase the utility of POVNav in real-world scenarios, highlighting its flexibility and computational efficiency for autonomous robots in complex environments.
more »
« less
Understanding Human Dynamic Sampling Objectives to Enable Robot-assisted Scientific Decision Making
Truly collaborative scientific field data collection between human scientists and autonomous robot systems requires a shared understanding of the search objectives and tradeoffs faced when making decisions. Therefore, critical to developing intelligent robots to aid human experts is an understanding of how scientists make such decisions and how they adapt their data collection strategies when presented with new informationin situ. In this study, we examined the dynamic data collection decisions of 108 expert geoscience researchers using a simulated field scenario. Human data collection behaviors suggested two distinct objectives: an information-based objective to maximize information coverage and a discrepancy-based objective to maximize hypothesis verification. We developed a highly simplified quantitative decision model that allows the robot to predict potential human data collection locations based on the two observed human data collection objectives. Predictions from the simple model revealed a transition from information-based to discrepancy-based objective as the level of information increased. The findings will allow robotic teammates to connect experts’ dynamic science objectives with the adaptation of their sampling behaviors and, in the long term, enable the development of more cognitively compatible robotic field assistants.
more »
« less
- Award ID(s):
- 2240075
- PAR ID:
- 10497542
- Publisher / Repository:
- ACM Transactions on Human-Robot Interaction
- Date Published:
- Journal Name:
- ACM Transactions on Human-Robot Interaction
- Volume:
- 13
- Issue:
- 1
- ISSN:
- 2573-9522
- Page Range / eLocation ID:
- 1 to 17
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Policy summarization is a computational paradigm for explaining the behavior and decision-making processes of autonomous robots to humans. It summarizes robot policies via exemplary demonstrations, aiming to improve human understanding of robotic behaviors. This understanding is crucial, especially since users often make critical decisions about robot deployment in the real world. Previous research in policy summarization has predominantly focused on simulated robots and environments, overlooking its application to physically embodied robots. Our work fills this gap by combining current policy summarization methods with a novel, interactive user interface that involves physical interaction with robots. We conduct human-subject experiments to assess our explanation system, focusing on the impact of different explanation modalities in policy summarization. Our findings underscore the unique advantages of combining virtual and physical training environments to effectively communicate robot behavior to human users.more » « less
-
Hideki Aoyama; Keiich Shirase (Ed.)An integral part of information-centric smart manufacturing is the adaptation of industrial robots to complement human workers in a collaborative manner. While advancement in sensing has enabled real-time monitoring of workspace, understanding the semantic information in the workspace, such as parts and tools, remains a challenge for seamless robot integration. The resulting lack of adaptivity to perform in a dynamic workspace have limited robots to tasks with pre-defined actions. In this paper, a machine learning-based robotic object detection and grasping method is developed to improve the adaptivity of robots. Specifically, object detection based on the concept of single-shot detection (SSD) and convolutional neural network (CNN) is investigated to recognize and localize objects in the workspace. Subsequently, the extracted information from object detection, such as the type, position, and orientation of the object, is fed into a multi-layer perceptron (MLP) to generate the desired joint angles of robotic arm for proper object grasping and handover to the human worker. Network training is guided by forward kinematics of the robotic arm in a self-supervised manner to mitigate issues such as singularity in computation. The effectiveness of the developed method is validated on an eDo robotic arm in a human-robot collaborative assembly case study.more » « less
-
Using the context of human-supervised object collection tasks, we explore policies for a robot to seek assistance from a human supervisor and avoid loss of human trust in the robot. We consider a human-robot interaction scenario in which a mobile manipulator chooses to collect objects either autonomously or through human assistance; while the human supervisor monitors the robot’s operation, assists when asked, or intervenes if the human perceives that the robot may not accomplish its goal. We design an optimal assistance-seeking policy for the robot using a Partially Observable Markov Decision Process (POMDP) setting in which human trust is a hidden state and the objective is to maximize collaborative performance. We conduct two sets of human-robot interaction experiments. The data from the first set of experiments is used to estimate POMDP parameters, which are used to compute an optimal assistance-seeking policy that is used in the second experiment. For most participants, the estimated POMDP reveals that humans are more likely to intervene when their trust is low and the robot is performing a high-complexity task; and that the robot asking for assistance in high-complexity tasks can increase human trust in the robot. Our experimental results show that the proposed trust-aware policy yields superior performance compared with an optimal trust-agnostic policy.more » « less
-
null (Ed.)Abstract How do scientists generate and weight candidate queries for hypothesis testing, and how does learning from observations or experimental data impact query selection? Field sciences offer a compelling context to ask these questions because query selection and adaptation involves consideration of the spatiotemporal arrangement of data, and therefore closely parallels classic search and foraging behavior. Here we conduct a novel simulated data foraging study—and a complementary real-world case study—to determine how spatiotemporal data collection decisions are made in field sciences, and how search is adapted in response to in-situ data. Expert geoscientists evaluated a hypothesis by collecting environmental data using a mobile robot. At any point, participants were able to stop the robot and change their search strategy or make a conclusion about the hypothesis. We identified spatiotemporal reasoning heuristics, to which scientists strongly anchored, displaying limited adaptation to new data. We analyzed two key decision factors: variable-space coverage, and fitting error to the hypothesis. We found that, despite varied search strategies, the majority of scientists made a conclusion as the fitting error converged. Scientists who made premature conclusions, due to insufficient variable-space coverage or before the fitting error stabilized, were more prone to incorrect conclusions. We found that novice undergraduates used the same heuristics as expert geoscientists in a simplified version of the scenario. We believe the findings from this study could be used to improve field science training in data foraging, and aid in the development of technologies to support data collection decisions.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1145/3623383
