skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: The effect of challenging work environment on human-robot interaction and cognitive load during teleoperation: a case study of teleoperated excavator in a virtual experiment
Construction sites typically involve a risky, dynamic, and challenging work environment. Despite numerous safety training programs and regulations, accidents still occur in construction sites, especially when working with construction robotics. To alleviate this problem in the most fundamental way, teleoperation that allows operators to work remotely has been studied. Teleoperated construction robots have the great potential to be used in various contexts for extreme and hazardous construction sites. Here, work conditions for human-robot interaction in construction differ from those in other structured and controlled environments like manufacturing factories, and thus there is a need for the associated studies. In this paper, we aim to measure and analyze the performance of human-robot interaction and the cognitive load of human operators in dynamic and challenging construction work environments (hazardous risks such as underground utility strikes and working under time constraints).  more » « less
Award ID(s):
2026574
PAR ID:
10385448
Author(s) / Creator(s):
;
Date Published:
Journal Name:
The 2022 International Conference on Robotics and Automation (ICRA)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. With the construction sector primed to incorporate such advanced technologies as artificial intelligence (AI), robots, and machines, these advanced tools will require a deep understanding of human–robot trust dynamics to support safety and productivity. Although other disciplines have broadly investigated human trust-building with robots, the discussion within the construction domain is still nascent, raising concerns because construction workers are increasingly expected to work alongside robots or cobots, and to communicate and interact with drones. Without a better understanding of how construction workers can appropriately develop and calibrate their trust in their robotic counterparts, the implementation of advanced technologies may raise safety and productivity issues within these already-hazardous jobsites. Consequently, this study conducted a systematic review of the human–robot trust literature to (1) understand human–robot trust-building in construction and other domains; and (2) establish a roadmap for investigating and fostering worker–robot trust in the construction industry. The proposed worker–robot trust-building roadmap includes three phases: static trust based on the factors related to workers, robots, and construction sites; dynamic trust understood via measuring, modeling, and interpreting real-time trust behaviors; and adaptive trust, wherein adaptive calibration strategies and adaptive training facilitate appropriate trust-building. This roadmap sheds light on a progressive procedure to uncover the appropriate trust-building between workers and robots in the construction industry. 
    more » « less
  2. Shane, Jennifer S; Madson, Katherine M; Mo, Yunjeong Leah; Poleacovschi, Cristina; Sturgill, Roy E (Ed.)
    Teleoperation is widely used in hazardous and uncertain site settings, allowing scheduled procedures to be carried out across long distances while workers are away from the sites. Teleoperators in off-sites collect both the site information and feedback from the interfaces which provide synthesized information that a robot collects. This interface mainly conveys visionary information for the operator’s intuitiveness such as the spatial awareness of objects and surroundings. To achieve a rich visual understanding of the site, the interface should fully contain and intuitively convey the associated contextual information. Excessive or unintuitive information not only makes it difficult for operators to exert their full potential but also increases their cognitive load. This study explores how different visual interface configurations affect operators’ work performance and their cognitive load during the teleoperation task. The findings from the experimental studies are expected to help develop human-centered interfaces for teleoperation in the context of construction tasks and provide the cornerstone for not only an intuitive but fruitfully informative interface in a provided task setting. 
    more » « less
  3. With the advancement of automation and robotic technologies, the teleoperation has been leveraged as a promising solution for human workers in a hazardous construction work environment. Since human operators and construction sites are separated in a distance, teleoperation requires a seamless human-machine interface, an intermediate medium, to communicate between humans and machines in construction sites. Several types of teleoperation interfaces including conventional joysticks, haptic devices, graphic user interfaces, auditory interfaces, and tactile interfaces have been developed to control and command construction robotics remotely. The ultimate goal of human-machine interfaces for remote operations is to make intuitive sensory channels that can provide and receive enough information while reducing the associated cognitive and physical load on human operators. Previously developed interfaces have tried to achieve such goals, but each interface still has challenges that should be assessed for enhancing the future teleoperation application in construction workplaces. This paper examines different human-machine interfaces for excavator teleoperation in terms of its on-site usability and intuitiveness. The capabilities of the interfaces for excavator teleoperation are evaluated based on their limitations and requirements. The outcome is expected to provide better understanding of teleoperation interfaces for excavators and guiding future directions for addressing underlying challenges. 
    more » « less
  4. The objective of this research is to evaluate vision-based pose estimation methods for on-site construction robots. The prospect of human-robot collaborative work on construction sites introduces new workplace hazards that must be mitigated to ensure safety. Human workers working on tasks alongside construction robots must perceive the interaction to be safe to ensure team identification and trust. Detecting the robot pose in real-time is thus a key requirement in order to inform the workers and to enable autonomous operation. Vision-based (marker-less, marker-based) and sensor-based (IMU, UWB) are two of the main methods for estimating robot pose. The marker-based and sensor-based methods require some additional preinstalled sensors or markers, whereas the marker-less method only requires an on-site camera system, which is common on modern construction sites. In this research, we develop a marker-less pose estimation system, which is based on a convolutional neural network (CNN) human pose estimation algorithm: stacked hourglass networks. The system is trained with image data collected from a factory setup environment and labels of excavator pose. We use a KUKA robot arm with a bucket mounted on the end-effector to represent a robotic excavator in our experiment. We evaluate the marker-less method and compare the result with the robot’s ground truth pose. The preliminary results show that the marker-less method is capable of estimating the pose of the excavator based on a state-of-the-art human pose estimation algorithm. 
    more » « less
  5. Robots have the potential to enhance safety on construction job sites by assuming hazardous tasks. While existing safety research on physical human-robot interaction (pHRI) primarily addresses collision risks, ensuring inherently safe collaborative workflows is equally important. For example, ergonomic optimization in co-manipulation is an important safety consideration in pHRI. While frameworks such as Rapid Entire Body Assessment (REBA) have been an industry standard for these interventions, their lack of a rigorous mathematical structure poses challenges for using them with optimization algorithms. Previous works have tackled this gap by developing approximations or statistical approaches that are error-prone or data-dependent. This paper presents a framework using Reinforcement Learning for precise ergonomic optimization that generalizes to different types of tasks. To ensure practicality and safe experimentations, the training leverages Inverse Kinematics in virtual reality to simulate human movement mechanics. Results of a comparison between the developed framework and ergonomically naive approaches are presented. 
    more » « less