More Like this

1. Evaluating Methods for End-User Creation of Robot Task Plans

   Paxton, Chris ; Jonathan, Felix ; Hundt, Andrew ; Bilge, Mutlu ; Hager, Gregory D. ( October 2018 , 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems)

   How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effective. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as "pick up component A from the right side of the table." Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks. 

   more » « less
2. CoSTAR: Instructing collaborative robots with behavior trees and vision

   https://doi.org/10.1109/ICRA.2017.7989070  

   Paxton, Chris ; Hundt, Andrew ; Jonathan, Felix ; Guerin, Kelleher ; Hager, Gregory D. ( May 2017 , ICRA 2017)

   For collaborative robots to become useful, end users who are not robotics experts must be able to instruct them to perform a variety of tasks. With this goal in mind, we developed a system for end‐user creation of robust task plans with a broad range of capabilities. CoSTAR: the Collaborative System for Task Automation and Recognition} is our winning entry in the 2016 KUKA Innovation Award competition at the Hannover Messe trade show, which this year focused on Flexible Manufacturing. CoSTAR is unique in how it creates natural abstractions that use perception to represent the world in a way users can both understand and utilize to author capable and robust task plans. Our Behavior Tree‐based task editor integrates high‐level information from known object segmentation and pose estimation with spatial reasoning and robot actions to create robust task plans. We describe the crossplatform design and implementation of this system on multiple industrial robots and evaluate its suitability for a wide variety of use cases. 

   more » « less
3. V.Ra: An In-Situ Visual Authoring System for Robot-IoT Task Planning with Augmented Reality

   https://doi.org/10.1145/3322276.3322278  

   Cao, Yuanzhi ; Xu, Zhuangying ; Li, Fan ; Zhong, Wentao ; Huo, Ke ; Ramani, Karthik ( June 2019 , Proceedings of the 2019 on Designing Interactive Systems Conference)

   We present V.Ra, a visual and spatial programming system for robot-IoT task authoring. In V.Ra, programmable mobile robots serve as binding agents to link the stationary IoTs and perform collaborative tasks. We establish an ecosystem that coherently connects the three key elements of robot task planning , the human, robot and IoT, with one single mobile AR device. Users can perform task authoring with the Augmented Reality (AR) handheld interface, then placing the AR device onto the mobile robot directly transfers the task plan in a what-you-do-is-what-robot-does (WYDWRD) manner. The mobile device mediates the interactions between the user, robot, and the IoT oriented tasks, and guides the path planning execution with the embedded simultaneous localization and mapping (SLAM) capability. We demonstrate that V.Ra enables instant, robust and intuitive room-scale navigatory and interactive task authoring through various use cases and preliminary studies. 

   more » « less
4. V.Ra: An In-Situ Visual Authoring System for Robot-IoT Task Planning with Augmented Reality

   https://doi.org/10.1145/3290607.3312797  

   Cao, Yuanzhi ; Xu, Zhuangying ; Li, Fan ; Zhong, Wentao ; Huo, Ke ; Ramani, Karthik ( May 2019 , Conference on Human Factors in Computing Systems - Proceedings, 2 May 2019)

   We present V.Ra, a visual and spatial programming system for robot-IoT task authoring. In V.Ra, programmable mobile robots serve as binding agents to link the stationary IoTs and perform collaborative tasks. We establish an ecosystem that coherently connects the three key elements of robot task planning (human-robot-IoT) with one single AR-SLAM device. Users can perform task authoring in an analogous manner with the Augmented Reality (AR) interface. Then placing the device onto the mobile robot directly transfers the task plan in a what-you-do-is-what-robot-does (WYDWRD) manner. The mobile device mediates the interactions between the user, robot and IoT oriented tasks, and guides the path planning execution with the SLAM capability. 

   more » « less
5. Authr: A Task Authoring Environment for Human-Robot Teams

   https://doi.org/10.1145/3379337.3415872  

   Schoen, Andrew ; Henrichs, Curt ; Strohkirch, Mathias ; Mutlu, Bilge ( October 2020 , UIST '20: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology)

   Collaborative robots promise to transform work across many industries and promote “human-robot teaming” as a novel paradigm. However, realizing this promise requires the understanding of how existing tasks, developed for and performed by humans, can be effectively translated into tasks that robots can singularly or human-robot teams can collaboratively perform. In the interest of developing tools that facilitate this process we present Authr, an end-to-end task authoring environment that assists engineers at manufacturing facilities in translating existing manual tasks into plans applicable for human-robot teams and simulates these plans as they would be performed by the human and robot. We evaluated Authr with two user studies, which demonstrate the usability and effectiveness of Authr as an interface and the benefits of assistive task allocation methods for designing complex tasks for human-robot teams. We discuss the implications of these findings for the design of software tools for authoring human-robot collaborative plans. 

   more » « less