skip to main content

Explore Research Products in the NSF-PAR

Title: 3D Point Cloud Generation with Millimeter-Wave Radar
Emerging autonomous driving systems require reliable perception of 3D surroundings. Unfortunately, current mainstream perception modalities, i.e., camera and Lidar, are vulnerable under challenging lighting and weather conditions. On the other hand, despite their all-weather operations, today's vehicle Radars are limited to location and speed detection. In this paper, we introduce MILLIPOINT, a practical system that advances the Radar sensing capability to generate 3D point clouds. The key design principle of MILLIPOINT lies in enabling synthetic aperture radar (SAR) imaging on low-cost commodity vehicle Radars. To this end, MILLIPOINT models the relation between signal variations and Radar movement, and enables self-tracking of Radar at wavelength-scale precision, thus realize coherent spatial sampling. Furthermore, MILLIPOINT solves the unique problem of specular reflection, by properly focusing on the targets with post-imaging processing. It also exploits the Radar's built-in antenna array to estimate the height of reflecting points, and eventually generate 3D point clouds. We have implemented MILLIPOINT on a commodity vehicle Radar. Our evaluation results show that MILLIPOINT effectively combats motion errors and specular reflections, and can construct 3D point clouds with much higher density and resolution compared with the existing vehicle Radar solutions.  more » « less
Award ID(s):
1925767 1952942
NSF-PAR ID:
10300638
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume:
4
Issue:
4
ISSN:
2474-9567
Page Range / eLocation ID:
1 to 23
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , IEEE Conference on Computer Vision and Pattern Recognition)
    null (Ed.)
    Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving. While they generate fine-grained point clouds or high-resolution images with rich information in good weather conditions, they fail in adverse weather (e.g., fog) where opaque particles distort lights and significantly reduce visibility. Thus, existing methods relying on lidar or camera experience significant performance degradation in rare but critical adverse weather conditions. To remedy this, we resort to exploiting complementary radar, which is less impacted by adverse weather and becomes prevalent on vehicles. In this paper, we present Multimodal Vehicle Detection Network (MVDNet), a two-stage deep fusion detector, which first generates proposals from two sensors and then fuses region-wise features between multimodal sensor streams to improve final detection results. To evaluate MVDNet, we create a procedurally generated training dataset based on the collected raw lidar and radar signals from the open-source Oxford Radar Robotcar. We show that the proposed MVDNet surpasses other state-of-the-art methods, notably in terms of Average Precision (AP), especially in adverse weather conditions. The code and data are available at https://github.com/qiank10/MVDNet. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al ( , Bulletin of the American Meteorological Society)
    Abstract Weather radar networks have great potential for continuous and long-term monitoring of aerial biodiversity of birds, bats, and insects. Biological data from weather radars can support ecological research, inform conservation policy development and implementation, and increase the public’s interest in natural phenomena such as migration. Weather radars are already used to study animal migration, quantify changes in populations, and reduce aerial conflicts between birds and aircraft. Yet efforts to establish a framework for the broad utilization of operational weather radar for biodiversity monitoring are at risk without suitable data policies and infrastructure in place. In Europe, communities of meteorologists and ecologists have made joint efforts toward sharing and standardizing continent-wide weather radar data. These efforts are now at risk as new meteorological data exchange policies render data useless for biodiversity monitoring. In several other parts of the world, weather radar data are not even available for ecological research. We urge policy makers, funding agencies, and meteorological organizations across the world to recognize the full potential of weather radar data. We propose several actions that would ensure the continued capability of weather radar networks worldwide to act as powerful tools for biodiversity monitoring and research. 
    more » « less
  3. ; ( , Monthly Weather Review)
    Abstract

    The scientific community has long acknowledged the importance of high-temporal-resolution radar observations to advance science research and improve high-impact weather prediction. Development of innovative rapid-scan radar technologies over the past two decades has enabled radar volume scans of 10–60 s compared to 3–5 min with traditional parabolic dish research radars and the WSR-88D radar network. This review examines the impact of rapid-scan radar technology, defined as radars collecting volume scans in 1 min or less, on atmospheric science research spanning different subdisciplines and evaluates the strengths and weaknesses of the use of rapid-scan radars. In particular, a significant body of literature has accumulated for tornado and severe thunderstorm research and forecasting applications, in addition to a growing number of studies of convection. Convection research has benefited substantially from more synchronous vertical views, but could benefit more substantially by leveraging multi-Doppler wind retrievals and complementary in situ and remote sensors. In addition, several years of forecast evaluation studies are synthesized from radar testbed experiments, and the benefits of assimilating rapid-scan radar observations are analyzed. Although the current body of literature reflects the considerable utility of rapid-scan radars to science research, a weakness is that limited advancements in understanding of the physical mechanisms behind observed features have been enabled. There is considerable opportunity to bridge the gap in physical understanding with the current technology using coordinated efforts to include rapid-scan radars in field campaigns and expanding the breadth of meteorological phenomena studied.

    Significance Statement

    Recently developed rapid-scan radar technologies, capable of collecting volumetric (i.e., three-dimensional) measurements in 10–60 s, have improved temporal sampling of weather phenomena. This review examines the impact of these radar observations from the past two decades on science research and emerging operational capabilities. Substantial breadth and impact of research is evident for tornado research and forecasting applications, in addition to documentation of other rapidly evolving phenomena associated with deep convection, such as tornadoes, hail, lightning, and tropical cyclones. This review identifies the strengths and weaknesses of how these radars have been used in scientific research to inform future studies, emerging from the increasing availability and capability of rapid-scan radars. In addition, this review synthesizes research that can benefit future operational radar decisions.

     
    more » « less
  4. ; ; ( , Proceedings of the annual International Conference on Mobile Computing and Networking)
    Unmanned aerial vehicles (UAVs) rely on optical sensors such as cameras and lidar for autonomous operation. However, such optical sensors are error-prone in bad lighting, inclement weather conditions including fog and smoke, and around textureless or transparent surfaces. In this paper, we ask: is it possible to fly UAVs without relying on optical sensors, i.e., can UAVs fly without seeing? We present BatMobility, a lightweight mmWave radar-only perception system for UAVs that eliminates the need for optical sensors. BatMobility enables two core functionalities for UAVs – radio flow estimation (a novel FMCW radar-based alternative for optical flow based on surface-parallel doppler shift) and radar-based collision avoidance. We build BatMobility using commodity sensors and deploy it as a real-time system on a small off-the-shelf quadcopter running an unmodified flight controller. Our evaluation shows that BatMobility achieves comparable or better performance than commercial-grade optical sensors across a wide range of scenarios. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al ( , Atmospheric Measurement Techniques)
    Abstract. Meteor radars have become widely used instruments to study atmospheric dynamics, particularly in the 70 to 110 km altitude region. Thesesystems have been proven to provide reliable and continuous measurements of horizontal winds in the mesosphere and lower thermosphere. Recently,there have been many attempts to utilize specular and/or transverse scatter meteor measurements to estimate vertical winds and vertical windvariability. In this study we investigate potential biases in vertical wind estimation that are intrinsic to the meteor radar observation geometryand scattering mechanism, and we introduce a mathematical debiasing process to mitigate them. This process makes use of a spatiotemporal Laplacefilter, which is based on a generalized Tikhonov regularization. Vertical winds obtained from this retrieval algorithm are compared to UA-ICON modeldata. This comparison reveals good agreement in the statistical moments of the vertical velocity distributions. Furthermore, we present the firstobservational indications of a forward scatter wind bias. It appears to be caused by the scattering center's apparent motion along the meteortrajectory when the meteoric plasma column is drifted by the wind. The hypothesis is tested by a radiant mapping of two meteor showers. Finally, weintroduce a new retrieval algorithm providing a physically and mathematically sound solution to derive vertical winds and wind variability frommultistatic meteor radar networks such as the Nordic Meteor Radar Cluster (NORDIC) and the Chilean Observation Network De meteOr Radars(CONDOR). The new retrieval is called 3DVAR+DIV and includes additional diagnostics such as the horizontal divergence and relative vorticity toensure a physically consistent solution for all 3D winds in spatially resolved domains. Based on this new algorithm we obtained vertical velocitiesin the range of w = ± 1–2 m s−1 for most of the analyzed data during 2 years of collection, which is consistent with the values reportedfrom general circulation models (GCMs) for this timescale and spatial resolution. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email