An official website of the United States government
Object detection plays a pivotal in autonomous driving by enabling the vehicles to perceive and comprehend their environment, thereby making informed decisions for safe navigation. Camera data provides rich visual context and object recognition, while LiDAR data offers precise distance measurements and 3D mapping. Multi-modal object detection models are gaining prominence in incorporating these data types, which is essential for the comprehensive perception and situational awareness needed in autonomous vehicles. Although graphics processing units (GPUs) and field-programmable gate arrays (FPGAs) are promising hardware options for this application, the complex knowledge required to efficiently adapt and optimize multi-modal detection models for FPGAs presents a significant barrier to their utilization on this versatile and efficient platform. In this work, we evaluate the performance of camera and LiDARbased detection models on GPU and FPGA hardware, aiming to provide a specialized understanding for translating multi-modal detection models to suit the unique architecture of heterogeneous hardware platforms in autonomous driving systems. We focus on critical metrics from both system and model performance aspects. Based on our quantitative implications, we propose foundational insights and guidance for the design of camera and LiDAR-based multi-modal detection models on diverse hardware platforms.
more »
« less
Multi-Modal Vehicle Data Delivery via Commercial 5G Mobile Networks: An Initial Study
Support for connected and autonomous vehicles (CAVs) is a major use case of 5G networks. Due to their large from factors, CAVs can be equipped with multiple radio antennas, cameras, LiDAR and other sensors. In other words, they are "giant" mobile integrated communications and sensing devices. The data collected can not only facilitate edge-assisted autonomous driving, but also enable intelligent radio resource allocation by cellular networks. In this paper we conduct an initial study to assess the feasibility of delivering multi-modal sensory data collected by vehicles over emerging commercial 5G networks. We carried out an "in-the-wild" drive test and data collection campaign between Minneapolis and Chicago using a vehicle equipped with a 360° camera, a LiDAR device, multiple smart phones and a professional 5G network measurement tool. Using the collected multi-modal data, we conduct trace-driven experiments in a local streaming testbed to analyze the requirements and performance of streaming multi-modal sensor data over existing 4G/5G networks. We reveal several notable findings and point out future research directions.
more »
« less
- PAR ID:
- 10484811
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW)
- ISBN:
- 979-8-3503-2812-7
- Page Range / eLocation ID:
- 157 to 162
- Format(s):
- Medium: X
- Location:
- Hong Kong, Hong Kong
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Connected Autonomous Vehicles (CAVs) have achieved significant improvements in recent years. The CAVs can share sensor data to improve autonomous driving performance and enhance road safety. CAV architecture depends on roadside edge servers for latency-sensitive applications. The roadside edge servers are equipped with high-performance embedded edge computing devices that perform calculations with low power requirements. As the number of vehicles varies over different times of the day and vehicles can request for different CAV applications, the computation requirements for roadside edge computing platform can also vary. Hence, a framework for dynamic deployment of edge computing platforms can ensure CAV applications’ performance and proper usage of the devices. In this paper, we propose R-CAV – a framework for drone-based roadside edge server deployment that provides roadside units (RSUs) based on the computation requirement. Our proof of concept implementation for object detection algorithm using Nvidia Jetson nano demonstrates the proposed framework's feasibility. We posit that the framework will enhance the intelligent transport system vision by ensuring CAV applications’ quality of service.more » « less
-
Vehicle-to-everything (V2X) collaborative perception has emerged as a promising solution to address the limitations of single-vehicle perception systems. However, existing V2X datasets are limited in scope, diversity, and quality. To address these gaps, we present Mixed Signals, a comprehensive V2X dataset featuring 45.1k point clouds and 240.6k bounding boxes collected from three connected autonomous vehicles (CAVs) equipped with two different configurations of LiDAR sensors, plus a roadside unit with dual LiDARs. Our dataset provides point clouds and bounding box annotations across 10 classes, ensuring reliable data for perception training. We provide detailed statistical analysis on the quality of our dataset and extensively benchmark existing V2X methods on it. Mixed Signals is ready-to-use, with precise alignment and consistent annotations across time and viewpoints. We hope our work advances research in the emerging, impactful field of V2X perception. Dataset details at https://mixedsignalsdataset.cs.cornell.edu/.more » « less
-
null (Ed.)Recent research has demonstrated the potential benefits of connected, autonomous vehicles (CAVs) to the performance of urban networks. Specifically, several proposals have been made for policies and related technologies that either perform more efficiently when the proportion of CAVs is relatively high or that exclude human driven vehicles (HDVs) altogether. This same body of research has also identified several challenges faced by such networks, especially in the context of shared autonomous vehicles (SAVs). We propose a lane-use policy for networks of exclusively CAVs with the goal of preserving priority within any two-class, arbitrary priority assignment regime. We investigate the merits of such a policy by adopting a simple occupancy based, two-class priority scheme in a network of SAVs. We will demonstrate that by granting and preserving priority for occupied vehicles, average travel times and speeds for passengers are improved with limited degradation in these measures for other, i.e. unoccupied, vehicles. The proposed lane-use policy is developed on realistic physical limitations of the street network and without the need for trajectory reservations.more » « less
-
Unmanned vehicles, equipped with radiation detection sensors, can serve as a valuable aid to personnel responding to radiological incidents. The use of tele-operated ground vehicles avoids human exposure to hazardous environments, which in addition to radioactive contamination, might present other risks to personnel. Autonomous unmanned vehicles using algorithms for radioisotope classification, source localization, and efficient exploration allow these vehicles to conduct surveys with reduced human supervision allowing teams to address larger areas in less time. This work presents systems for autonomous radiation search with results presented in several proof-of-concept demonstrations.more » « less
