Search for: All records

Abstract Teleoperation can enable human intervention to help handle instances of failure in autonomy thus allowing for much safer deployment of autonomous vehicle technology. Successful teleoperation requires recreating the environment around the remote vehicle using camera data received over wireless communication channels. This paper develops a new predictive display system to tackle the significant time delays encountered in receiving camera data over wireless networks. First, a new high gain observer is developed for estimating the position and orientation of the ego vehicle. The novel observer is shown to perform accurate state estimation using only GNSS and gyroscope sensor readings. A vector field method which fuses the delayed camera and Lidar data is then presented. This method uses sparse 3D points obtained from Lidar and transforms them using the state estimates from the high gain observer to generate a sparse vector field for the camera image. Polynomial based interpolation is then performed to obtain the vector field for the complete image which is then remapped to synthesize images for accurate predictive display. The method is evaluated on real-world experimental data from the nuScenes and KITTI datasets. The performance of the high gain observer is also evaluated and compared with that of the EKF. The synthesized images using the vector field based predictive display are compared with ground truth images using various image metrics and offer vastly improved performance compared to delayed images. 

In this paper we develop a novel disruptionresilient approach for real-time, high-resolution sensor data delivery over multiple wireless channels for military autonomous systems such as drones, autonomous vehicles and robots. We design two innovative neural multiple description codecs (neural MDCs) which compress and encode images into multiple independently decodable and mutually refineable streams. Our approach not only achieves high compression efficiency, but also enables the effective use of multiple diverse radio channels for real-time delivery of high-resolution sensor data while ensuring disruption resiliency. Using benchmark image/video sensor datasets as well as real-world 5G traces, we evaluate and demonstrate the efficacy of both neural MDC codecs for highresolution sensor data streaming over multiple radio channels under various jamming scenarios.