skip to main content

Explore Research Products in the NSF-PAR

Title: 3D Point Cloud Generation with Millimeter-Wave Radar
Emerging autonomous driving systems require reliable perception of 3D surroundings. Unfortunately, current mainstream perception modalities, i.e., camera and Lidar, are vulnerable under challenging lighting and weather conditions. On the other hand, despite their all-weather operations, today's vehicle Radars are limited to location and speed detection. In this paper, we introduce MILLIPOINT, a practical system that advances the Radar sensing capability to generate 3D point clouds. The key design principle of MILLIPOINT lies in enabling synthetic aperture radar (SAR) imaging on low-cost commodity vehicle Radars. To this end, MILLIPOINT models the relation between signal variations and Radar movement, and enables self-tracking of Radar at wavelength-scale precision, thus realize coherent spatial sampling. Furthermore, MILLIPOINT solves the unique problem of specular reflection, by properly focusing on the targets with post-imaging processing. It also exploits the Radar's built-in antenna array to estimate the height of reflecting points, and eventually generate 3D point clouds. We have implemented MILLIPOINT on a commodity vehicle Radar. Our evaluation results show that MILLIPOINT effectively combats motion errors and specular reflections, and can construct 3D point clouds with much higher density and resolution compared with the existing vehicle Radar solutions.  more » « less
Award ID(s):
1925767 1952942
NSF-PAR ID:
10300638
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume:
4
Issue:
4
ISSN:
2474-9567
Page Range / eLocation ID:
1 to 23
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , IEEE Conference on Computer Vision and Pattern Recognition)
    null (Ed.)
    Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving. While they generate fine-grained point clouds or high-resolution images with rich information in good weather conditions, they fail in adverse weather (e.g., fog) where opaque particles distort lights and significantly reduce visibility. Thus, existing methods relying on lidar or camera experience significant performance degradation in rare but critical adverse weather conditions. To remedy this, we resort to exploiting complementary radar, which is less impacted by adverse weather and becomes prevalent on vehicles. In this paper, we present Multimodal Vehicle Detection Network (MVDNet), a two-stage deep fusion detector, which first generates proposals from two sensors and then fuses region-wise features between multimodal sensor streams to improve final detection results. To evaluate MVDNet, we create a procedurally generated training dataset based on the collected raw lidar and radar signals from the open-source Oxford Radar Robotcar. We show that the proposed MVDNet surpasses other state-of-the-art methods, notably in terms of Average Precision (AP), especially in adverse weather conditions. The code and data are available at https://github.com/qiank10/MVDNet. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al ( , Atmospheric Measurement Techniques)
    Abstract. Meteor radars have become widely used instruments to study atmospheric dynamics, particularly in the 70 to 110 km altitude region. Thesesystems have been proven to provide reliable and continuous measurements of horizontal winds in the mesosphere and lower thermosphere. Recently,there have been many attempts to utilize specular and/or transverse scatter meteor measurements to estimate vertical winds and vertical windvariability. In this study we investigate potential biases in vertical wind estimation that are intrinsic to the meteor radar observation geometryand scattering mechanism, and we introduce a mathematical debiasing process to mitigate them. This process makes use of a spatiotemporal Laplacefilter, which is based on a generalized Tikhonov regularization. Vertical winds obtained from this retrieval algorithm are compared to UA-ICON modeldata. This comparison reveals good agreement in the statistical moments of the vertical velocity distributions. Furthermore, we present the firstobservational indications of a forward scatter wind bias. It appears to be caused by the scattering center's apparent motion along the meteortrajectory when the meteoric plasma column is drifted by the wind. The hypothesis is tested by a radiant mapping of two meteor showers. Finally, weintroduce a new retrieval algorithm providing a physically and mathematically sound solution to derive vertical winds and wind variability frommultistatic meteor radar networks such as the Nordic Meteor Radar Cluster (NORDIC) and the Chilean Observation Network De meteOr Radars(CONDOR). The new retrieval is called 3DVAR+DIV and includes additional diagnostics such as the horizontal divergence and relative vorticity toensure a physically consistent solution for all 3D winds in spatially resolved domains. Based on this new algorithm we obtained vertical velocitiesin the range of w = ± 1–2 m s−1 for most of the analyzed data during 2 years of collection, which is consistent with the values reportedfrom general circulation models (GCMs) for this timescale and spatial resolution. 
    more » « less
  3. ; ; ; ; ; ; ; ( , Earth, Planets and Space)
    Abstract

    In recent years, multistatic specular meteor radars (SMRs) have been introduced to study the Mesosphere and Lower Thermosphere (MLT) dynamics with increasing spatial and temporal resolution. SMRs, compared to other ground-based observations, have the advantage of continuously measuring the region between 80 and 100 km independent of weather, season, or time of day. In this paper, frequency spectra of MLT horizontal winds are explored through observations from a campaign using the SIMONe (Spread-spectrum Interferometric Multistatic meteor radar Observing Network) approach conducted in northern Germany in 2018 (hereafter SIMONe 2018). The 7-day SIMONe 2018 comprised of fourteen multistatic SMR links and allows us to build a substantial database of specular meteor trail events, collecting more than one hundred thousand detections per day within a geographic area of$$\sim$$∼500 km$$\times$$×500 km. We have implemented two methods to obtain the frequency spectra of the horizontal wind components: (1) Mean Wind Estimation (MWE) and (2) Wind field Correlation Function Inversion (WCFI), which utilizes the mean and the covariances of the line of sight velocities, respectively. Monte Carlo simulations of a gravity wave spectral model were implemented to validate and compare both methods. The simulation analyses suggest that the WCFI helps us to capture the energy of smaller scale wind fluctuations than those capture with MWE. Characterization of the spectral slope of the horizontal wind at different MLT altitudes has been conducted on the SIMONe 2018, and it provides evidence that gravity waves with periods smaller than 7 h and greater than 2 h dominate with horizontal structures significantly larger than 500 km. In the future, these analyses can be extended to understand the significance of small-scale fluctuations in the MLT, which were not possible with conventional MWE methods.

    Graphical Abstract

     
    more » « less
  4. ; ; ; ; ; ; ; ( , Earth and Space Science)

    There are few observational techniques for measuring the distribution of kinetic energy within the mesosphere with a wide range of spatial and temporal scales. This study describes a method for estimating the three‐dimensional mesospheric wind field correlation function from specular meteor trail echoes. Each radar echo provides a measurement of a one‐dimensional projection of the wind velocity vector at a randomly sampled point in space and time. The method relies on using pairs of such measurements to estimate the correlation function of the wind with different spatial and temporal lags. The method is demonstrated using a multistatic meteor radar data set that includes ≈105meteor echoes observed during a 24‐hr time period. The new method is found to be in good agreement with the well‐established technique for estimating horizontal mean winds. High‐resolution correlation functions with temporal, horizontal, and vertical lags are also estimated from the data. The temporal correlation function is used to retrieve the kinetic energy spectrum, which includes the semidiurnal mode and a 3‐hr period wave. The horizontal and vertical correlation functions of the wind are then used to derive second‐order structure functions, which are found to be compatible with the Kolmogorov prediction for spectral distribution of kinetic energy in the turbulent inertial range. The presented method can be used to extend the capabilities of specular meteor radars. It is relatively flexible and has a multitude of applications beyond what has been shown in this study.

     
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al ( , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition)
    Advances in perception for self-driving cars have accel- erated in recent years due to the availability of large-scale datasets, typically collected at specific locations and under nice weather conditions. Yet, to achieve the high safety re- quirement, these perceptual systems must operate robustly under a wide variety of weather conditions including snow and rain. In this paper, we present a new dataset to enable robust autonomous driving via a novel data collection pro- cess — data is repeatedly recorded along a 15 km route un- der diverse scene (urban, highway, rural, campus), weather (snow, rain, sun), time (day/night), and traffic conditions (pedestrians, cyclists and cars). The dataset includes im- ages and point clouds from cameras and LiDAR sensors, along with high-precision GPS/INS to establish correspon- dence across routes. The dataset includes road and object annotations using amodal masks to capture partial occlu- sions and 3D bounding boxes. We demonstrate the unique- ness of this dataset by analyzing the performance of base- lines in amodal segmentation of road and objects, depth estimation, and 3D object detection. The repeated routes opens new research directions in object discovery, contin- ual learning, and anomaly detection. Link to Ithaca365: https://ithaca365.mae.cornell.edu/ 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email