More Like this

1. Exploring Human-Machine Interfaces for Teleoperation of Excavator

   https://doi.org/10.1061/9780784483961.079  

   Lee, Jin Sol; Ham, Youngjib (March 2022, ASCE Construction Research Congress 2022)

   With the advancement of automation and robotic technologies, the teleoperation has been leveraged as a promising solution for human workers in a hazardous construction work environment. Since human operators and construction sites are separated in a distance, teleoperation requires a seamless human-machine interface, an intermediate medium, to communicate between humans and machines in construction sites. Several types of teleoperation interfaces including conventional joysticks, haptic devices, graphic user interfaces, auditory interfaces, and tactile interfaces have been developed to control and command construction robotics remotely. The ultimate goal of human-machine interfaces for remote operations is to make intuitive sensory channels that can provide and receive enough information while reducing the associated cognitive and physical load on human operators. Previously developed interfaces have tried to achieve such goals, but each interface still has challenges that should be assessed for enhancing the future teleoperation application in construction workplaces. This paper examines different human-machine interfaces for excavator teleoperation in terms of its on-site usability and intuitiveness. The capabilities of the interfaces for excavator teleoperation are evaluated based on their limitations and requirements. The outcome is expected to provide better understanding of teleoperation interfaces for excavators and guiding future directions for addressing underlying challenges. 

   more » « less
2. Perception and Action Augmentation for Teleoperation Assistance in Freeform Telemanipulation

   https://doi.org/10.1145/3643804  

   Lin, Tsung-Chi; Krishnan, Achyuthan Unni; Li, Zhi (March 2024, ACM Transactions on Human-Robot Interaction)

   Teleoperation enables controlling complex robot systems remotely, providing the ability to impart human expertise from a distance. However, these interfaces can be complicated to use as it is difficult to contextualize information about robot motion in the workspace from the limited camera feedback. Thus, it is required to study the best manner in which assistance can be provided to the operator that reduces interface complexity and effort required for teleoperation. Some techniques that provide assistance to the operator while freeform teleoperating include: (1) perception augmentation, like augmented reality visual cues and additional camera angles, increasing the information available to the operator; (2) action augmentation, like assistive autonomy and control augmentation, optimized to reduce the effort required by the operator while teleoperating. In this article, we investigate: (1) which aspects of dexterous telemanipulation require assistance; (2) the impact of perception and action augmentation in improving teleoperation performance; and (3) what factors impact the usage of assistance and how to tailor these interfaces based on the operators’ needs and characteristics. The findings from this user study and resulting post-study surveys will help identify task-based and user-preferred perception and augmentation features for teleoperation assistance. 

   more » « less
3. Identifying and Understanding Eye Movement Patterns during Teleoperated Lunar Construction Tasks

   Seo, M; Kim, M; Ham, Y (September 2025, American Society of Civil Engineers (ASCE))

   Effective visual scanning and perception are essential to perform teleoperation construction tasks. In long-distance teleoperation systems, such as between Earth and the Moon, communication delays of at least 3 seconds impact operator performance and situational awareness, influencing eye movement patterns that vary with time-delay conditions and task types. This study examines features of eye movement patterns during time-delayed teleoperation tasks in lunar construction environment. We analyzed and quantified eye movement trajectories using recurrence quantification analysis to identify spatiotemporal patterns and gain insights into how operators interact with visual scenes under time-delayed and task-specific conditions. By visualizing these patterns, we interpreted their temporal and spatial characteristics and compared them in different situations. Our findings, which address a gap in understanding operator eye movement patterns during teleoperation tasks, have potential practical implications for improving visual perception by identifying quantitative patterns along with qualitative interpretations. This study will contribute to visual interface design in teleoperation systems for lunar construction, providing actionable insights for the design and operation of such systems. 

   more » « less
4. Telemanipulation via Virtual Reality Interfaces with Enhanced Environment Models

   https://doi.org/10.1109/IROS51168.2021.9636005  

   Murphy Wonsick, Tarık Keleștemur (January 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   Extreme environments, such as search and rescue missions, defusing bombs, or exploring extraterrestrial planets, are unsafe environments for humans to be in. Robots enable humans to explore and interact in these environments through remote presence and teleoperation and virtual reality provides a medium to create immersive and easy-to-use teleoperation interfaces. However, current virtual reality interfaces are still very limited in their capabilities. In this work, we aim to advance robot teleoperation virtual reality interfaces by developing an environment reconstruction methodology capable of recognizing objects in a robot’s environment and rendering high fidelity models inside a virtual reality headset. We compare our proposed environment reconstruction method against traditional point cloud streaming by having operators plan waypoint trajectories to accomplish a pick-and-place task. Overall, our results show that users find our environment reconstruction method more usable and less cognitive work compared to raw point cloud streaming. 

   more » « less
5. Intuitive, Efficient and Ergonomic Tele-Nursing Robot Interfaces: Design Evaluation and Evolution

   https://doi.org/10.1145/3526108  

   Lin, Tsung-Chi; Krishnan, Achyuthan Unni; Li, Zhi (April 2022, ACM Transactions on Human-Robot Interaction)

   Tele-nursing robots provide a safe approach for patient-caring in quarantine areas. For effective nurse-robot collaboration, ergonomic teleoperation and intuitive interfaces with low physical and cognitive workload must be developed. We propose a framework to evaluate the control interfaces to iteratively develop an intuitive, efficient, and ergonomic teleoperation interface. The framework is a hierarchical procedure that incorporates general to specific assessment and its role in design evolution. We first present pre-defined objective and subjective metrics used to evaluate three representative contemporary teleoperation interfaces. The results indicate that teleoperation via human motion mapping outperforms the gamepad and stylus interfaces. The trade-off with using motion mapping as a teleoperation interface is the non-trivial physical fatigue. To understand the impact of heavy physical demand during motion mapping teleoperation, we propose an objective assessment of physical workload in teleoperation using electromyography (EMG). We find that physical fatigue happens in the actions that involve precise manipulation and steady posture maintenance. We further implemented teleoperation assistance in the form of shared autonomy to eliminate the fatigue-causing component in robot teleoperation via motion mapping. The experimental results show that the autonomous feature effectively reduces the physical effort while improving the efficiency and accuracy of the teleoperation interface. 

   more » « less