An official website of the United States government
We presentCoSense, a system that enables coexistence of networking and sensing on next-generation millimeter-wave (mmWave) picocells for traffic monitoring and pedestrian safety at intersections in all weather conditions. Although existing wireless signal-based object detection systems are available, they suffer from limited resolution and their outputs may not provide sufficient discriminatory information in complex scenes, such as traffic intersections.CoSenseproposes using 5G picocells, which operate at mmWave frequency bands and provide higher data rates and higher sensing resolution than traditional wireless technology. However, it is difficult to run sensing applications and data transfer simultaneously on mmWave devices due to potential interference, and using special-purpose sensing hardware can prohibit deployment of sensing applications to a large number of existing and future inexpensive mmWave devices. Additionally, mmWave devices are vulnerable to weak reflectivity and specularity challenges, which may result in loss of information about objects and pedestrians. To overcome these challenges,CoSensedesign customized deep learning models that not only can recover missing information about the target scene but also enable coexistence of networking and sensing. We evaluateCoSenseon diverse data samples captured at traffic intersections and demonstrate that it can detect and locate pedestrians and vehicles, both qualitatively and quantitatively, without significantly affecting the networking throughput.
more »
« less
mmBox: Harnessing Millimeter-Wave Signals for Reliable Vehicle and Pedestrians Detection
Object detection plays a pivotal role in various fields, for example, a smart traffic system relies on the detected results for decision-making. However, existing studies predominately utilize optical camera and LiDAR, which exhibit limitations in adverse outdoor environments, such as foggy weather. To address these challenges, millimeter-waves (mmWaves) attract researchers’ attention to detect objects in severe conditions since they can work effectively in low-visibility conditions and overcome small obstacles. Yet, previous mmWave-based works have shown limited performance, such as no shape information for objects. Therefore, we design and implement a two-stage system,mmBox, to accurately predict bounding boxes with depth for vehicles and pedestrians, which first generates heatmaps in different dimensions and then leverages a deep learning model to extract features for predictions. To evaluate the performance ofmmBox, we collected real-world mmWave reflections from urban traffic intersections and dense-fog environments. The extensive evaluation metrics show remarkable accuracy and the low latency of our model.
more »
« less
- PAR ID:
- 10558409
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- ACM Transactions on Internet of Things
- Volume:
- 5
- Issue:
- 4
- ISSN:
- 2691-1914
- Page Range / eLocation ID:
- 1 to 30
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Millimeter wave (mmWave) sensing has recently gained attention for its robustness in challenging environments. When visual sensors such as cameras fail to perform, mmWave radars can be used to provide reliable performance. However, the poor scattering performance and lack of texture in millimeter waves can make it difficult for radars to identify objects in some situations precisely. In this paper, we take insight from camera fiducials which are very easily identifiable by a camera, and present R-fiducial tags, which smartly augment the current infrastructure to enable myriad applications with mmwave radars. R-fiducial acts as fiducials for mmwave sensing, similar to camera fiducials, and can be reliably identified by a mmwave radar. We identify a set of requirements for millimeter wave fiducials and show how R-fiducial meets them all. R-fiducial uses a novel spread-spectrum modulation technique to provide low latency with high reliability. Our evaluations show that R-fiducial can be reliably detected with a 100% detection rate up to 25 meters with a 120-degree field of view and a few milliseconds of latency. We also conduct experiments and case studies in adverse and low visibility conditions to demonstrate the potential of R-fiducial in a variety of applications.more » « less
-
To support the demand of multi-Gbps sensory data exchanges for enhancing (semi)-autonomous driving, millimeter-wave bands (mmWave) vehicular-to-infrastructure (V2I) communications have attracted intensive attention. Unfortunately, the vulnerability to blockages over mmWave bands poses significant design challenges, which can be hardly addressed by manipulating end transceivers, such as beamforming techniques. In this paper, we propose to enhance mmWave V2I communications by augmenting the transmission environments through reflection, where highly-reflective cheap metallic plates are deployed as tunable reflectors without damaging the aesthetic nature of the environments. In this way, alternative indirect line-of-sight (LOS) links are established by adjusting the angle of reflectors. Our fundamental challenge is to adapt the time-consuming reflector angle tuning to the highly dynamic vehicular environment. By using deep reinforcement learning, we propose the learning-based Fast Reflection (LFR) algorithm, which autonomously learns from the observable traffic pattern to select desirable reflector angles in advance for probably blocked vehicles in near future. Simulation results demonstrate our proposal could effectively augment mmWave V2I transmission environments with significant performance gain.more » « less
-
In this paper, we ask, "Can millimeter-wave (mmWave) radars sense objects not directly illuminated by the radar - for instance, objects located outside the transmit beamwidth, behind occlusions, or placed fully behind the radar?" Traditionally, mmWave radars are limited to sense objects that are directly illuminated by the radar and scatter its signals directly back. In practice, however, radar signals scatter to other intermediate objects in the environment and undergo multiple bounces before being received back at the radar. In this paper, we present Hydra, a framework to explicitly model and exploit multi-bounce paths for sensing. Hydra enables standalone mmWave radars to sense beyond-field-of-view objects without prior knowledge of the environment. We extensively evaluate the localization performance of Hydra with an off-the-shelf mmWave radar in five different environments with everyday objects. Exploiting multi-bounce via Hydra provides 2×-10× improvement in the median beyond-field-of-view localization error over baselines.more » « less
-
This paper proposes SquiggleMilli, a system that approximates traditional Synthetic Aperture Radar (SAR) imaging on mobile millimeter-wave (mmWave) devices. The system is capable of imaging through obstructions, such as clothing, and under low visibility conditions. Unlike traditional SAR that relies on mechanical controllers or rigid bodies, SquiggleMilli is based on the hand-held, fluidic motion of the mmWave device. It enables mmWave imaging in hand-held settings by re-thinking existing motion compensation, compressed sensing, and voxel segmentation. Since mmWave imaging suffers from poor resolution due to specularity and weak reflectivity, the reconstructed shapes could be imperceptible by machines and humans. To this end, SquiggleMilli designs a machine learning model to recover the high spatial frequencies in the object to reconstruct an accurate 2D shape and predict its 3D features and category. We have customized SquiggleMilli for security applications, but the model is adaptable to other applications with limited training samples. We implement SquiggleMilli on off-the-shelf components and demonstrate its performance improvement over the traditional SAR qualitatively and quantitatively.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1145/3695883
