More Like this

1. Multi-Modal Vehicle Data Delivery via Commercial 5G Mobile Networks: An Initial Study

   https://doi.org/10.1109/ICDCSW60045.2023.00026  

   Carpenter, Jason; Ye, Wei; Qian, Feng; Zhang, Zhi-Li (July 2023, 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW))

   Support for connected and autonomous vehicles (CAVs) is a major use case of 5G networks. Due to their large from factors, CAVs can be equipped with multiple radio antennas, cameras, LiDAR and other sensors. In other words, they are "giant" mobile integrated communications and sensing devices. The data collected can not only facilitate edge-assisted autonomous driving, but also enable intelligent radio resource allocation by cellular networks. In this paper we conduct an initial study to assess the feasibility of delivering multi-modal sensory data collected by vehicles over emerging commercial 5G networks. We carried out an "in-the-wild" drive test and data collection campaign between Minneapolis and Chicago using a vehicle equipped with a 360° camera, a LiDAR device, multiple smart phones and a professional 5G network measurement tool. Using the collected multi-modal data, we conduct trace-driven experiments in a local streaming testbed to analyze the requirements and performance of streaming multi-modal sensor data over existing 4G/5G networks. We reveal several notable findings and point out future research directions. 

   more » « less
2. Malicious Attacks against Multi-Sensor Fusion in Autonomous Driving

   https://doi.org/10.1145/3636534.3649372  

   Zhu, Yi; Miao, Chenglin; Xue, Hongfei; Yu, Yunnan; Su, Lu; Qiao, Chunming (May 2024, ACM)

   Multi-sensor fusion has been widely used by autonomous vehicles (AVs) to integrate the perception results from different sensing modalities including LiDAR, camera and radar. Despite the rapid development of multi-sensor fusion systems in autonomous driving, their vulnerability to malicious attacks have not been well studied. Although some prior works have studied the attacks against the perception systems of AVs, they only consider a single sensing modality or a camera-LiDAR fusion system, which can not attack the sensor fusion system based on LiDAR, camera, and radar. To fill this research gap, in this paper, we present the first study on the vulnerability of multi-sensor fusion systems that employ LiDAR, camera, and radar. Specifically, we propose a novel attack method that can simultaneously attack all three types of sensing modalities using a single type of adversarial object. The adversarial object can be easily fabricated at low cost, and the proposed attack can be easily performed with high stealthiness and flexibility in practice. Extensive experiments based on a real-world AV testbed show that the proposed attack can continuously hide a target vehicle from the perception system of a victim AV using only two small adversarial objects. 

   more » « less
3. A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

   https://doi.org/10.1109/SMC52423.2021.9658639  

   Islam, Muhammad Mobaidul; Newaz, Abdullah Al; Karimoddini, Ali (October 2021, 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC))

   This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. To deal with the challenges associated with the autonomous driving scenarios, an integrated tracking and detection framework is proposed. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates both in RGB and depth images. To provide accurate information, the detection phase is further enhanced by fusing multi-modal sensor information using the Kalman filter. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene. We evaluate our framework on a real public driving dataset. Experimental results demonstrate that the proposed method achieves significant performance improvement over a baseline method that solely uses image-based pedestrian detection. 

   more » « less
4. ICanC: Improving Camera-based Object Detection and Energy Consumption in Low-Illumination Environments

   Ma, Daniel; Zhong, Ren; Shi, Weisong (March 2025, arXiv)

   This paper introduces ICanC (pronounced "I Can See"), a novel system designed to enhance object detection and optimize energy efficiency in autonomous vehicles (AVs) operating in low-illumination environments. By leveraging the complementary capabilities of LiDAR and camera sensors, ICanC improves detection accuracy under conditions where camera performance typically declines, while significantly reducing unnecessary headlight usage. This approach aligns with the broader objective of promoting sustainable transportation. ICanC comprises three primary nodes: the Obstacle Detector, which processes LiDAR point cloud data to fit bounding boxes onto detected objects and estimate their position, velocity, and orientation; the Danger Detector, which evaluates potential threats using the information provided by the Obstacle Detector; and the Light Controller, which dynamically activates headlights to enhance camera visibility solely when a threat is detected. Experiments conducted in physical and simulated environments demonstrate ICanC's robust performance, even in the presence of significant noise interference. The system consistently achieves high accuracy in camera-based object detection when headlights are engaged, while significantly reducing overall headlight energy consumption. These results position ICanC as a promising advancement in autonomous vehicle research, achieving a balance between energy efficiency and reliable object detection. 

   more » « less
5. 3D-FAD: Informativeness-guided Multimodal 3D Object Detection for Federated Autonomous Driving

   https://doi.org/10.1145/3820499  

   Gan, Maolin; Liu, Li; Alam, Samiul; Liu, Jiancheng; Zhang, Mi; Zeng, Huacheng; Cao, Zhichao (July 2026, ACM Transactions on Sensor Networks)

   Autonomous driving has been adopted in various applications, including taxis, logistics, and agriculture, to address labor shortages and enhance safety. A critical component is 3D object detection, relying on deep neural networks (DNNs) that integrate LiDAR and camera data for robust road planning. Federated Autonomous Driving (FAD) utilizes Federated Learning (FL) to train these DNNs, addressing the demand for diverse training datasets while preserving client privacy. However, extending the existing FAD framework to 3D object detection remains challenging due to insufficient consideration of modality heterogeneity and informativeness. The article introduces3D-FAD, a pioneering informativeness-guided multimodal FAD framework.3D-FADis the first to explicitly integrate modality informativeness into multimodal 3D object detection for FAD by a convolution-based encoder. It utilizes this informativeness for efficient client selection and aggregation at the server. Furthermore,3D-FADintroduces Masked Modality AutoEncoders to impute deficient LiDAR features for clients with limited LiDAR FOV and low-cost cameras in real-world autonomous driving. Our experiments on real-world autonomous driving datasets nuScene and Waymo show that3D-FADoutperforms baselines by up to 34.1% and 39.2% in key metrics, along with better performance across different object classes and environments. 

   more » « less