Access to high-quality data is an important barrier in the digital analysis of urban settings, including applications within computer vision and urban design. Diverse forms of data collected from sensors in areas of high activity in the urban environment, particularly at street intersections, are valuable resources for researchers interpreting the dynamics between vehicles, pedestrians, and the built environment. In this paper, we present a high-resolution audio, video, and LiDAR dataset of three urban intersections in Brooklyn, New York, totaling almost 8 unique hours. The data were collected with custom Reconfigurable Environmental Intelligence Platform (REIP) sensors that were designed with the ability to accurately synchronize multiple video and audio inputs. The resulting data are novel in that they are inclusively multimodal, multi-angular, high-resolution, and synchronized. We demonstrate four ways the data could be utilized — (1) to discover and locate occluded objects using multiple sensors and modalities, (2) to associate audio events with their respective visual representations using both video and audio modes, (3) to track the amount of each type of object in a scene over time, and (4) to measure pedestrian speed using multiple synchronized camera views. In addition to these use cases, our data are available for other researchers to carry out analyses related to applying machine learning to understanding the urban environment (in which existing datasets may be inadequate), such as pedestrian-vehicle interaction modeling and pedestrian attribute recognition. Such analyses can help inform decisions made in the context of urban sensing and smart cities, including accessibility-aware urban design and Vision Zero initiatives.
Multi-Modal Vehicle Data Delivery via Commercial 5G Mobile Networks: An Initial Study
Support for connected and autonomous vehicles (CAVs) is a major use case of 5G networks. Due to their large from factors, CAVs can be equipped with multiple radio antennas, cameras, LiDAR and other sensors. In other words, they are "giant" mobile integrated communications and sensing devices. The data collected can not only facilitate edge-assisted autonomous driving, but also enable intelligent radio resource allocation by cellular networks. In this paper we conduct an initial study to assess the feasibility of delivering multi-modal sensory data collected by vehicles over emerging commercial 5G networks. We carried out an "in-the-wild" drive test and data collection campaign between Minneapolis and Chicago using a vehicle equipped with a 360° camera, a LiDAR device, multiple smart phones and a professional 5G network measurement tool. Using the collected multi-modal data, we conduct trace-driven experiments in a local streaming testbed to analyze the requirements and performance of streaming multi-modal sensor data over existing 4G/5G networks. We reveal several notable findings and point out future research directions.
more »
« less
- NSF-PAR ID:
- 10484811
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW)
- Page Range / eLocation ID:
- 157 to 162
- Format(s):
- Medium: X
- Location:
- Hong Kong, Hong Kong
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
1. Description of the objectives and motivation for the contribution to ECE education The demand for wireless data transmission capacity is increasing rapidly and this growth is expected to continue due to ongoing prevalence of cellular phones and new and emerging bandwidth-intensive applications that encompass high-definition video, unmanned aerial systems (UAS), intelligent transportation systems (ITS) including autonomous vehicles, and others. Meanwhile, vital military and public safety applications also depend on access to the radio frequency spectrum. To meet these demands, the US federal government is beginning to move from the proven but inefficient model of exclusive frequency assignments to a more-efficient, shared-spectrum approach in some bands of the radio frequency spectrum. A STEM workforce that understands the radio frequency spectrum and applications that use the spectrum is needed to further increase spectrum efficiency and cost-effectiveness of wireless systems over the next several decades to meet anticipated and unanticipated increases in wireless data capacity. 2. Relevant background including literature search examples if appropriate CISCO Systems’ annual survey indicates continued strong growth in demand for wireless data capacity. Meanwhile, undergraduate electrical and computer engineering courses in communication systems, electromagnetics, and networks tend to emphasize mathematical and theoretical fundamentals and higher-layer protocols, with less focus on fundamental concepts that are more specific to radio frequency wireless systems, including the physical and media access control layers of wireless communication systems and networks. An efficient way is needed to introduce basic RF system and spectrum concepts to undergraduate engineering students in courses such as those mentioned above who are unable to, or had not planned to take a full course in radio frequency / microwave engineering or wireless systems and networks. We have developed a series of interactive online modules that introduce concepts fundamental to wireless communications, the radio frequency spectrum, and spectrum sharing, and seek to present these concepts in context. The modules include interactive, JavaScript-based simulation exercises intended to reinforce the concepts that are presented in the modules through narrated slide presentations, text, and external links. Additional modules in development will introduce advanced undergraduate and graduate students and STEM professionals to configuration and programming of adaptive frequency-agile radios and spectrum management systems that can operate efficiently in congested radio frequency environments. Simulation exercises developed for the advanced modules allow both manual and automatic control of simulated radio links in timed, game-like simulations, and some exercises will enable students to select from among multiple pre-coded controller strategies and optionally edit the code before running the timed simulation. Additionally, we have developed infrastructure for running remote laboratory experiments that can also be embedded within the online modules, including a web-based user interface, an experiment management framework, and software defined radio (SDR) application software that runs in a wireless testbed initially developed for research. Although these experiments rely on limited hardware resources and introduce additional logistical considerations, they provide additional realism that may further challenge and motivate students. 3. Description of any assessment methods used to evaluate the effectiveness of the contribution, Each set of modules is preceded and followed by a survey. Each individual module is preceded by a quiz and followed by another quiz, with pre- and post-quiz questions drawn from the same pool. The pre-surveys allow students to opt in or out of having their survey and quiz results used anonymously in research. 4. Statement of results. The initial modules have been and are being used by three groups of students: (1) students in an undergraduate Introduction to Communication Systems course; (2) an interdisciplinary group of engineering students, including computer science students, who are participating in related undergraduate research project; and (3) students in a graduate-level communications course that includes both electrical and computer engineers. Analysis of results from the first group of students showed statistically significant increases from pre-quiz to post-quiz for each of four modules on fundamental wireless communication concepts. Results for the other students have not yet been analyzed, but also appear to show substantial pre-quiz to post-quiz increases in mean scores.more » « less
-
Connected Autonomous Vehicles (CAVs) have achieved significant improvements in recent years. The CAVs can share sensor data to improve autonomous driving performance and enhance road safety. CAV architecture depends on roadside edge servers for latency-sensitive applications. The roadside edge servers are equipped with high-performance embedded edge computing devices that perform calculations with low power requirements. As the number of vehicles varies over different times of the day and vehicles can request for different CAV applications, the computation requirements for roadside edge computing platform can also vary. Hence, a framework for dynamic deployment of edge computing platforms can ensure CAV applications’ performance and proper usage of the devices. In this paper, we propose R-CAV – a framework for drone-based roadside edge server deployment that provides roadside units (RSUs) based on the computation requirement. Our proof of concept implementation for object detection algorithm using Nvidia Jetson nano demonstrates the proposed framework's feasibility. We posit that the framework will enhance the intelligent transport system vision by ensuring CAV applications’ quality of service.more » « less
-
more » « less
5G Millimeter Wave (mmWave) technology holds great promise for Connected Autonomous Vehicles (CAVs) due to its ability to achieve data rates in the Gbps range. However, mmWave suffers from a high beamforming overhead and requirement of line of sight (LOS) to maintain a strong connection. For Vehicle-to-Infrastructure (V2I) scenarios, where CAVs connect to roadside units (RSUs), these drawbacks become apparent. Because vehicles are dynamic, there is a large potential for link blockages. These blockages are detrimental to the connected applications running on the vehicle, such as cooperative perception and remote driver takeover. Existing RSU selection schemes base their decisions on signal strength and vehicle trajectory alone, which is not enough to prevent the blockage of links. Many modern CAVs motion planning algorithms routinely use other vehicle’s near-future path plans, either by explicit communication among vehicles, or by prediction. In this paper, we make use of the knowledge of other vehicle’s near future path plans to further improve the RSU association mechanism for CAVs. We solve the RSU association algorithm by converting it to a shortest path problem with the objective to maximize the total communication bandwidth. We evaluate our approach, titled B-AWARE, in simulation using Simulation of Urban Mobility (SUMO) and Digital twin for self-dRiving Intelligent VEhicles (DRIVE) on 12 highway and city street scenarios with varying traffic density and RSU placements. Simulations show B-AWARE results in a 1.05× improvement of the potential datarate in the average case and 1.28× in the best case vs. the state-of-the-art. But more impressively, B-AWARE reduces the time spent with no connection by 42% in the average case and 60% in the best case as compared to the state-of-the-art methods. This is a result of B-AWARE reducing nearly 100% of blockage occurrences.
-
The continuous increase in demanding for availability and ultra-reliability of low-latency and broadband wireless connections is instigating further research in the standardization of next-generation mobile systems. 6G networks, among other benefits, should offer global ubiquitous mobility thanks to the utilization of the Space segment as an intelligent yet autonomous ecosystem. In this framework, multi-layered networks will take charge of providing connectivity by implementing Cloud-Radio Access Network (C-RAN) functionalities on heterogeneous nodes distributed over aerial and orbital segments. Unmanned Aerial Vehicles (UAVs), High-Altitude Plat-forms (HAPs), and small satellites compose the Space ecosystem encompassing the 3D networks. Recently, a lot of interest has been raised about splitting operations to distribute baseband processing functionalities among such nodes to balance the computational load and reduce the power consumption. This work focuses on the hardware development of C-RAN physical (PHY-) layer operations to derive their computational and energy demand. More in detail, the 5G Downlink Shared Channel (DLSCH) and the Physical Downlink Shared Channel (PDSCH) are first simulated in MATLAB environment to evaluate the variation of computational load depending on the selected splitting options and number of antennas available at transmitter (TX) and receiver (RX) side. Then, the PHY-layer processing chain is software-implemented and the various splitting options are tested on low-cost processors, such as Raspberry Pi (RP) 3B+ and 4B. By overclocking the RPs, we compute the execution time and we derive the instruction count (IC) per program for each considered splitting option so to achieve the mega instructions per second (MIPS) for the expected processing time. Finally, by comparing the performance achieved by the employed RPs with that of Nvidia Jetson Nano (JN) processor used as benchmark, we shall discuss about size, weight, power and cost (SWaP-C)...more » « less
