An official website of the United States government
Interest in cooperative perception is growing quickly due to its remarkable performance in improving perception capabilities for connected and automated vehicles. This improvement is crucial, especially for automated driving scenarios in which perception performance is one of the main bottlenecks to the development of safety and efficiency. However, current cooperative perception methods typically assume that all collaborating vehicles have enough communication bandwidth to share all features with an identical spatial size, which is impractical for real-world scenarios. In this paper, we propose Adaptive Cooperative Perception, a new cooperative perception framework that is not limited by the aforementioned assumptions, aiming to enable cooperative perception under more realistic and challenging conditions. To support this, a novel feature encoder is proposed and named Pillar Attention Encoder. A pillar attention mechanism is designed to extract the feature data while considering its significance for the perception task. An adaptive feature filter is proposed to adjust the size of the feature data for sharing by considering the importance value of the feature. Experiments are conducted for cooperative object detection from multiple vehicle-based and infrastructure-based LiDAR sensors under various communication conditions. Results demonstrate that our method can successfully handle dynamic communication conditions and improve the mean Average Precision by 10.18% when compared with the state-of-the-art feature encoder.
more »
« less
Dynamic Feature Sharing for Cooperative Perception from Point Clouds
Perceiving the surrounding environment is critical to enable cooperative driving automation, which is regarded as a transformative solution to improving our transportation system. Cooperative perception, by cooperating information from spatially separated nodes, can innately unlock the bottleneck caused by physical occlusions and has become an important research topic. Although cooperative perception aims to resolve practical problems, most of the current research work is designed based on the default assumption that the communication capacities of collaborated perception entities are identical. In this work, we introduce a fundamental approach - Dynamic Feature Sharing (DFS) - for cooperative perception from a more pragmatic context. Specifically, a DFS-based cooperative perception framework is designed to dynamically reduce the feature data required for sharing among the cooperating entities. Convolution-based Priority Filtering (CPF) is proposed to enable DFS under different communication constraints (e.g., bandwidth) by filtering the feature data according to the designed priority values. Zero-shot experiments demonstrate that the proposed CPF method can improve cooperative perception performance by approximately +22% under a dynamic communication-capacity condition and up to +130% when the communication bandwidth is reduced by 90 %.
more »
« less
- Award ID(s):
- 2152258
- PAR ID:
- 10510965
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 979-8-3503-9946-2
- Page Range / eLocation ID:
- 3970 to 3976
- Format(s):
- Medium: X
- Location:
- Bilbao, Spain
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Utilizing the latest advances in Artificial Intelligence (AI), the computer vision community is now witnessing an unprecedented evolution in all kinds of perception tasks, particularly in object detection. Based on multiple spatially separated perception nodes, Cooperative Perception (CP) has emerged to significantly advance the perception of automated driving. However, current cooperative object detection methods mainly focus on ego-vehicle efficiency without considering the practical issues of system-wide costs. In this paper, we introduce VINet, a unified deep learning-based CP network for scalable, lightweight, and heterogeneous cooperative 3D object detection. VINet is the first CP method designed from the standpoint of large-scale systemlevel implementation and can be divided into three main phases: (1) Global Pre-Processing and Lightweight Feature Extraction which prepare the data into global style and extract features for cooperation in a lightweight manner; (2) Two-Stream Fusion which fuses the features from scalable and heterogeneous perception nodes; and (3) Central Feature Backbone and 3D Detection Head which further process the fused features and generate cooperative detection results. An open-source data experimental platform is designed and developed for CP dataset acquisition and model evaluation. The experimental analysis shows that VINet can reduce 84% system-level computational cost and 94% system-level communication cost while improving the 3D detection accuracy.more » « less
-
Abstract Sensing is an essential part in autonomous driving and intelligent transportation systems. It enables the vehicle to better understand itself and its surrounding environment. Vehicular networks support information sharing among different vehicles and hence enable the multi‐vehicle multi‐sensor cooperative sensing, which can greatly improve the sensing performance. However, there are a couple of issues to be addressed. First, the multi‐sensor data fusion needs to deal with heterogeneous data formats. Second, the cooperative sensing process needs to deal with low data quality and perception blind spots for some vehicles. In order to solve the above problems, in this paper the occupancy grid map is adopted to facilitate the fusion of multi‐vehicle and multi‐sensor data. The dynamic target detection frame and pixel information of the camera data are mapped to the static environment of the LiDAR point cloud, and the space‐based occupancy probability distribution kernel density estimation characterization fusion data is designed , and the occupancy grid map based on the probability level and the spatial level is generated. Real‐world experiments show that the proposed fusion framework is better compatible with the data information of different sensors and expands the sensing range by involving the collaborations among multiple vehicles in vehicular networks.more » « less
-
Multi-Party Computation (MPC) is a technique enabling data from several sources to be used in a secure computation revealing only the result while protecting the original data, facilitating shared utilization of data sets gathered by different entities. The presence of Field Programmable Gate Array (FPGA) hardware in datacenters can provide accelerated computing as well as low latency, high bandwidth communication that bolsters the performance of MPC and lowers the barrier to using MPC for many applications. In this work, we propose a Secret Sharing FPGA design based on the protocol described by Araki et al. We compare our hardware design to the original authors' software implementations of Secret Sharing and to work accelerating MPC protocols based on Garbled Circuits with FPGAs. Our conclusion is that Secret Sharing in the datacenter is competitive and when implemented on FPGA hardware was able to use at least 10x fewer computer resources than the original work using CPUs.more » « less
-
Vehicle to Vehicle (V2V) communication allows vehicles to wirelessly exchange information on the surrounding environment and enables cooperative perception. It helps prevent accidents, increase the safety of the passengers, and improve the traffic flow efficiency. However, these benefits can only come when the vehicles can communicate with each other in a fast and reliable manner. Therefore, we investigated two areas to improve the communication quality of V2V: First, using beamforming to increase the bandwidth of V2V communication by establishing accurate and stable collaborative beam connection between vehicles on the road; second, ensuring scalable transmission to decrease the amount of data to be transmitted, thus reduce the bandwidth requirements needed for collaborative perception of autonomous driving vehicles. Beamforming in V2V communication can be achieved by utilizing image-based and LIDAR’s 3D data-based vehicle detection and tracking. For vehicle detection and tracking simulation, we tested the Single Shot Multibox Detector deep learning-based object detection method that can achieve a mean Average Precision of 0.837 and the Kalman filter for tracking. For scalable transmission, we simulate the effect of varying pixel resolutions as well as different image compression techniques on the file size of data. Results show that without compression, the file size for only transmitting the bounding boxes containing detected object is up to 10 times less than the original file size. Similar results are also observed when the file is compressed by lossless and lossy compression to varying degrees. Based on these findings using existing databases, the impact of these compression methods and methods of effectively combining feature maps on the performance of object detection and tracking models will be further tested in the real-world autonomous driving system.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ITSC57777.2023.10422242
