An official website of the United States government
This research introduces a virtual reality (VR) training system for improving human–robot collaboration (HRC) in industrial disassembly tasks, particularly for e-waste recycling. Conventional training approaches frequently fail to provide sufficient adaptability, immediate feedback, or scalable solutions for complex industrial workflows. The implementation leverages Quest Pro’s body-tracking capabilities to enable ergonomic, immersive interactions with planned eye-tracking integration for improved interactivity and accuracy. The Niryo One robot aids users in hands-on disassembly while generating synthetic data to refine robot motion planning models. A Robot Operating System (ROS) bridge enables the seamless simulation and control of various robotic platforms using Unified Robotics Description Format (URDF) files, bridging virtual and physical training environments. A Long Short-Term Memory (LSTM) model predicts user interactions and robotic motions, optimizing trajectory planning and minimizing errors. Monte Carlo dropout-based uncertainty estimation enhances prediction reliability, ensuring adaptability to dynamic user behavior. Initial technical validation demonstrates the platform’s potential, with preliminary testing showing promising results in task execution efficiency and human–robot motion alignment, though comprehensive user studies remain for future work. Limitations include the lack of multi-user scenarios, potential tracking inaccuracies, and the need for further real-world validation. This system establishes a sandbox training framework for HRC in disassembly, leveraging VR and AI-driven feedback to improve skill acquisition, task efficiency, and training scalability across industrial applications.
more »
« less
JEDAI: A System for Skill-Aligned Explainable Robot Planning
This paper presents JEDAI Explains Decision-Making AI (JEDAI), an AI system designed for outreach and educational efforts aimed at non-AI experts. JEDAI features a novel synthesis of research ideas from integrated task and motion planning and explainable AI. JEDAI helps users create high-level, intuitive plans while ensuring that they will be executable by the robot. It also provides users customized explanations about errors and helps improve their understanding of AI planning as well as the limits and capabilities of the underlying robot system.
more »
« less
- Award ID(s):
- 1942856
- PAR ID:
- 10342142
- Date Published:
- Journal Name:
- Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present V.Ra, a visual and spatial programming system for robot-IoT task authoring. In V.Ra, programmable mobile robots serve as binding agents to link the stationary IoTs and perform collaborative tasks. We establish an ecosystem that coherently connects the three key elements of robot task planning (human-robot-IoT) with one single AR-SLAM device. Users can perform task authoring in an analogous manner with the Augmented Reality (AR) interface. Then placing the device onto the mobile robot directly transfers the task plan in a what-you-do-is-what-robot-does (WYDWRD) manner. The mobile device mediates the interactions between the user, robot and IoT oriented tasks, and guides the path planning execution with the SLAM capability.more » « less
-
We present V.Ra, a visual and spatial programming system for robot-IoT task authoring. In V.Ra, programmable mobile robots serve as binding agents to link the stationary IoTs and perform collaborative tasks. We establish an ecosystem that coherently connects the three key elements of robot task planning , the human, robot and IoT, with one single mobile AR device. Users can perform task authoring with the Augmented Reality (AR) handheld interface, then placing the AR device onto the mobile robot directly transfers the task plan in a what-you-do-is-what-robot-does (WYDWRD) manner. The mobile device mediates the interactions between the user, robot, and the IoT oriented tasks, and guides the path planning execution with the embedded simultaneous localization and mapping (SLAM) capability. We demonstrate that V.Ra enables instant, robust and intuitive room-scale navigatory and interactive task authoring through various use cases and preliminary studies.more » « less
-
With the introduction of Industry 5.0, there is a growing focus on human-robot collaboration and the empowerment of human workers through the se of robotic technologies. Collaborative robots, or cobots, are well suited for filling the needs of industry. Cobots have a prioritization on safety and collaboration, giving them the unique ability to work in close proximity with people. This has the potential impact of increasing task productivity and efficiency while reducing ergonomic strain on human workers, as cobots can collaborate on tasks as teammates and support their human collaborators. However, effectively deploying and using cobots requires multidisciplinary knowledge spanning fields such as human factors and ergonomics, economics, and human-robot interaction. This knowledge barrier represents a growing challenge in industry, as workers lack the skills necessary to effectively leverage and realize the potential of cobots within their applications, resulting in cobots often being used non-collaboratively as a form of cheap automation. This presents several research opportunities for the creation of new cobot systems that support users in the creation of cobot interactions. The goal of this dissertation is to explore the use of abstraction and scaffolding supports within cobot systems to assist users in building human-robot collaborations. Specifically, this research (1) presents updates to the design of systems for planning and programming collaborative tasks, and (2) evaluates each system to understand how it can support user creation of cobot interactions. First, I present the CoFrame cobot programming system, a tool built on prior work, and illustrate how it supports user creation and understanding of cobot programs. Then, I present the evaluation of the system with domain experts, novices, and a real-world deployment to understand in which ways CoFrame does and does not successfully support users. I then describe the Allocobot system for allocating work and planning collaborative interactions, describing how it encodes multiple models of domain knowledge within its representation. Finally, I evaluate the Allocobot system in two real-world scenarios to understand how it produces and optimizes viable interaction plans.more » « less
-
While the ultimate goal of natural-language based Human-Robot Interaction (HRI) may be free-form, mixed-initiative dialogue,social robots deployed in the near future will likely primarily engage in wakeword-driven interaction, in which users’ commands are prefaced by a wakeword such as “Hey, Robot.” This style of interaction helps to allay user privacy concerns, as the robot’s full speech recognition module need not be employed until the target wakeword is used. Unfortunately, there are a number of concerns in the popular media surrounding this style of interaction, with consumers fearing that it is training users (in particular,children) to be rude towards technology, and by extension, rude towards other humans. In this paper, we present a study that demonstrates how an alternate style of wakeword, i.e., “Excuse me, Robot” may allay this concern, by priming users to phrase commands as Indirect Speech Actsmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- The DOI is not currently available.
