Search for: All records

Abstract ROV operations are mainly performed via a traditional control kiosk and limited data feedback methods, such as the use of joysticks and camera view displays equipped on a surface vessel. This traditional setup requires significant personnel on board (POB) time and imposes high requirements for personnel training. This paper proposes a virtual reality (VR) based haptic-visual ROV teleoperation system that can substantially simplify ROV teleoperation and enhance the remote operator's situational awareness. This study leverages the recent development in Mixed Reality (MR) technologies, sensory augmentation, sensing technologies, and closed-loop control, to visualize and render complex underwater environmental data in an intuitive and immersive way. The raw sensor data will be processed with physics engine systems and rendered as a high-fidelity digital twin model in game engines. Certain features will be visualized and displayed via the VR headset, whereas others will be manifested as haptic and tactile cues via our haptic feedback systems. We applied a simulation approach to test the developed system. With our developed system, a high-fidelity subsea environment is reconstructed based on the sensor data collected from an ROV including the bathymetric, hydrodynamic, visual, and vehicle navigational measurements. Specifically, the vehicle is equipped with a navigation sensor system for real-time state estimation, an acoustic Doppler current profiler for far-field flow measurement, and a bio-inspired artificial literal-line hydrodynamic sensor system for near-field small-scale hydrodynamics. Optimized game engine rendering algorithms then visualize key environmental features as augmented user interface elements in a VR headset, such as color-coded vectors, to indicate the environmental impact on the performance and function of the ROV. In addition, augmenting environmental feedback such as hydrodynamic forces are translated into patterned haptic stimuli via a haptic suit for indicating drift-inducing flows in the near field. A pilot case study was performed to verify the feasibility and effectiveness of the system design in a series of simulated ROV operation tasks. ROVs are widely used in subsea exploration and intervention tasks, playing a critical role in offshore inspection, installation, and maintenance activities. The innovative ROV teleoperation feedback and control system will lower the barrier for ROV pilot jobs. 

The utilization of remote operated vehicles (ROVs) has become essential across various subsea industries, such as oil and gas exploration and offshore wind energy, yet significant challenges remain in achieving effective human-ROV interaction. Despite advancements, ROV operations are hindered by complex control systems, high physical and cognitive demands on pilots, and a lack of sensory feedback mechanisms that fully convey the underwater environment’s dynamics. This study addresses these gaps by surveying ROV pilots and industry stakeholders to identify prevalent operational challenges, essential skills, and perspectives on integrating novel teleoperation technologies, including mixed reality and haptic feedback. Findings reveal a strong industry interest in technologies that enhance situational awareness and ease control demands, although concerns remain regarding practical integration and operator fatigue. By highlighting the critical skills required and potential benefits of human-centered augmentation systems, this study provides insights to inform future ergonomic designs, training frameworks, and technology development aimed at advancing safe and effective ROV teleoperation.