An official website of the United States government
We presentCoSense, a system that enables coexistence of networking and sensing on next-generation millimeter-wave (mmWave) picocells for traffic monitoring and pedestrian safety at intersections in all weather conditions. Although existing wireless signal-based object detection systems are available, they suffer from limited resolution and their outputs may not provide sufficient discriminatory information in complex scenes, such as traffic intersections.CoSenseproposes using 5G picocells, which operate at mmWave frequency bands and provide higher data rates and higher sensing resolution than traditional wireless technology. However, it is difficult to run sensing applications and data transfer simultaneously on mmWave devices due to potential interference, and using special-purpose sensing hardware can prohibit deployment of sensing applications to a large number of existing and future inexpensive mmWave devices. Additionally, mmWave devices are vulnerable to weak reflectivity and specularity challenges, which may result in loss of information about objects and pedestrians. To overcome these challenges,CoSensedesign customized deep learning models that not only can recover missing information about the target scene but also enable coexistence of networking and sensing. We evaluateCoSenseon diverse data samples captured at traffic intersections and demonstrate that it can detect and locate pedestrians and vehicles, both qualitatively and quantitatively, without significantly affecting the networking throughput.
more »
« less
This content will become publicly available on November 30, 2025
mmBox: Harnessing Millimeter-Wave Signals for Reliable Vehicle and Pedestrians Detection
Object detection plays a pivotal role in various fields, for example, a smart traffic system relies on the detected results for decision-making. However, existing studies predominately utilize optical camera and LiDAR, which exhibit limitations in adverse outdoor environments, such as foggy weather. To address these challenges, millimeter-waves (mmWaves) attract researchers’ attention to detect objects in severe conditions since they can work effectively in low-visibility conditions and overcome small obstacles. Yet, previous mmWave-based works have shown limited performance, such as no shape information for objects. Therefore, we design and implement a two-stage system,mmBox, to accurately predict bounding boxes with depth for vehicles and pedestrians, which first generates heatmaps in different dimensions and then leverages a deep learning model to extract features for predictions. To evaluate the performance ofmmBox, we collected real-world mmWave reflections from urban traffic intersections and dense-fog environments. The extensive evaluation metrics show remarkable accuracy and the low latency of our model.
more »
« less
- PAR ID:
- 10558409
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- ACM Transactions on Internet of Things
- Volume:
- 5
- Issue:
- 4
- ISSN:
- 2691-1914
- Page Range / eLocation ID:
- 1 to 30
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Millimeter wave (mmWave) sensing has recently gained attention for its robustness in challenging environments. When visual sensors such as cameras fail to perform, mmWave radars can be used to provide reliable performance. However, the poor scattering performance and lack of texture in millimeter waves can make it difficult for radars to identify objects in some situations precisely. In this paper, we take insight from camera fiducials which are very easily identifiable by a camera, and present R-fiducial tags, which smartly augment the current infrastructure to enable myriad applications with mmwave radars. R-fiducial acts as fiducials for mmwave sensing, similar to camera fiducials, and can be reliably identified by a mmwave radar. We identify a set of requirements for millimeter wave fiducials and show how R-fiducial meets them all. R-fiducial uses a novel spread-spectrum modulation technique to provide low latency with high reliability. Our evaluations show that R-fiducial can be reliably detected with a 100% detection rate up to 25 meters with a 120-degree field of view and a few milliseconds of latency. We also conduct experiments and case studies in adverse and low visibility conditions to demonstrate the potential of R-fiducial in a variety of applications.more » « less
-
To support the demand of multi-Gbps sensory data exchanges for enhancing (semi)-autonomous driving, millimeter-wave bands (mmWave) vehicular-to-infrastructure (V2I) communications have attracted intensive attention. Unfortunately, the vulnerability to blockages over mmWave bands poses significant design challenges, which can be hardly addressed by manipulating end transceivers, such as beamforming techniques. In this paper, we propose to enhance mmWave V2I communications by augmenting the transmission environments through reflection, where highly-reflective cheap metallic plates are deployed as tunable reflectors without damaging the aesthetic nature of the environments. In this way, alternative indirect line-of-sight (LOS) links are established by adjusting the angle of reflectors. Our fundamental challenge is to adapt the time-consuming reflector angle tuning to the highly dynamic vehicular environment. By using deep reinforcement learning, we propose the learning-based Fast Reflection (LFR) algorithm, which autonomously learns from the observable traffic pattern to select desirable reflector angles in advance for probably blocked vehicles in near future. Simulation results demonstrate our proposal could effectively augment mmWave V2I transmission environments with significant performance gain.more » « less
-
This paper proposes SquiggleMilli, a system that approximates traditional Synthetic Aperture Radar (SAR) imaging on mobile millimeter-wave (mmWave) devices. The system is capable of imaging through obstructions, such as clothing, and under low visibility conditions. Unlike traditional SAR that relies on mechanical controllers or rigid bodies, SquiggleMilli is based on the hand-held, fluidic motion of the mmWave device. It enables mmWave imaging in hand-held settings by re-thinking existing motion compensation, compressed sensing, and voxel segmentation. Since mmWave imaging suffers from poor resolution due to specularity and weak reflectivity, the reconstructed shapes could be imperceptible by machines and humans. To this end, SquiggleMilli designs a machine learning model to recover the high spatial frequencies in the object to reconstruct an accurate 2D shape and predict its 3D features and category. We have customized SquiggleMilli for security applications, but the model is adaptable to other applications with limited training samples. We implement SquiggleMilli on off-the-shelf components and demonstrate its performance improvement over the traditional SAR qualitatively and quantitatively.more » « less
-
We propose MiShape, a millimeter-wave (mmWave) wireless signal based imaging system that generates high-resolution human silhouettes and predicts 3D locations of body joints. The system can capture human motions in real-time under low light and low-visibility conditions. Unlike existing vision-based motion capture systems, MiShape is privacy non-invasive and can generalize to a wide range of motion tracking applications at-home. To overcome the challenges with low-resolution, specularity, and aliasing in images from Commercial-Off-The-Shelf (COTS) mmWave systems, MiShape designs deep learning models based on conditional Generative Adversarial Networks and incorporates the rules of human biomechanics. We have customized MiShape for gait monitoring, but the model is well adaptive to any tracking applications with limited fine-tuning samples. We experimentally evaluate MiShape with real data collected from a COTS mmWave system for 10 volunteers, with diverse ages, gender, height, and somatotype, performing different poses. Our experimental results demonstrate that MiShape delivers high-resolution silhouettes and accurate body poses on par with an existing vision-based system, and unlocks the potential of mmWave systems, such as 5G home wireless routers, for privacy-noninvasive healthcare applications.more » « less
