skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: CoSense: Deep Learning Augmented Sensing for Coexistence with Networking in Millimeter-Wave Picocells
We presentCoSense, a system that enables coexistence of networking and sensing on next-generation millimeter-wave (mmWave) picocells for traffic monitoring and pedestrian safety at intersections in all weather conditions. Although existing wireless signal-based object detection systems are available, they suffer from limited resolution and their outputs may not provide sufficient discriminatory information in complex scenes, such as traffic intersections.CoSenseproposes using 5G picocells, which operate at mmWave frequency bands and provide higher data rates and higher sensing resolution than traditional wireless technology. However, it is difficult to run sensing applications and data transfer simultaneously on mmWave devices due to potential interference, and using special-purpose sensing hardware can prohibit deployment of sensing applications to a large number of existing and future inexpensive mmWave devices. Additionally, mmWave devices are vulnerable to weak reflectivity and specularity challenges, which may result in loss of information about objects and pedestrians. To overcome these challenges,CoSensedesign customized deep learning models that not only can recover missing information about the target scene but also enable coexistence of networking and sensing. We evaluateCoSenseon diverse data samples captured at traffic intersections and demonstrate that it can detect and locate pedestrians and vehicles, both qualitatively and quantitatively, without significantly affecting the networking throughput.  more » « less
Award ID(s):
2018966 2144505 2342833
PAR ID:
10557749
Author(s) / Creator(s):
;
Publisher / Repository:
ACM
Date Published:
Journal Name:
ACM Transactions on Internet of Things
Volume:
5
Issue:
3
ISSN:
2691-1914
Page Range / eLocation ID:
1 to 35
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, ACM Transactions on Internet of Things)
    Object detection plays a pivotal role in various fields, for example, a smart traffic system relies on the detected results for decision-making. However, existing studies predominately utilize optical camera and LiDAR, which exhibit limitations in adverse outdoor environments, such as foggy weather. To address these challenges, millimeter-waves (mmWaves) attract researchers’ attention to detect objects in severe conditions since they can work effectively in low-visibility conditions and overcome small obstacles. Yet, previous mmWave-based works have shown limited performance, such as no shape information for objects. Therefore, we design and implement a two-stage system,mmBox, to accurately predict bounding boxes with depth for vehicles and pedestrians, which first generates heatmaps in different dimensions and then leverages a deep learning model to extract features for predictions. To evaluate the performance ofmmBox, we collected real-world mmWave reflections from urban traffic intersections and dense-fog environments. The extensive evaluation metrics show remarkable accuracy and the low latency of our model. 
    more » « less
  2. ; (, ACM Transactions on Internet of Things)
    In this work, we proposeMiSleep, a deep learning augmented millimeter-wave (mmWave) wireless system to monitor human sleep posture by predicting the 3D location of the body joints of a person during sleep. Unlike existing vision- or wearable-based sleep monitoring systems,MiSleepis not privacy-invasive and does not require users to wear anything on their body.MiSleepleverages knowledge of human anatomical features and deep learning models to solve challenges in existing mmWave devices with low-resolution and aliased imaging, and specularity in signals.MiSleepbuilds the model by learning the relationship between mmWave reflected signals and body postures from thousands of existing samples. Since a practical sleep also involves sudden toss-turns, which could introduce errors in posture prediction,MiSleepdesigns a state machine based on the reflected signals to classify the sleeping states into rest or toss-turn, and predict the posture only during the rest states. We evaluateMiSleepwith real data collected from Commercial-Off-The-Shelf mmWave devices for 8 volunteers of diverse ages, genders, and heights performing different sleep postures. We observe thatMiSleepidentifies the toss-turn events start time and duration within 1.25 s and 1.7 s of the ground truth, respectively, and predicts the 3D location of body joints with a median error of 1.3 cm only and can perform even under the blankets, with accuracy on par with the existing vision-based system, unlocking the potential of mmWave systems for privacy-noninvasive at-home healthcare applications. 
    more » « less
  3. ; ; ; ; ; ; (, in Proc. ACM MobiSys’22, 2022)
    Crowded metropolises present unique challenges to the potential deployment of autonomous vehicles. Safety of pedestrians cannot be compromised and personal privacy must be preserved. Smart city intersections will be at the core of Artificial Intelligence (AI)-powered citizen-friendly traffic management systems for such metropolises. Hence, the main objective of this work is to develop an experimentation framework for designing applications in support of secure and efficient traffic intersections in urban areas. We integrated a camera and a programmable edge computing node, deployed within the COSMOS testbed in New York City, with an Eclipse sensiNact data platform provided by Kentyou. We use this pipeline to collect and analyze video streams in real-time to support smart city applications. In this demo, we present a video analytics pipeline that analyzes the video stream from a COSMOS’ street-level camera to extract traffic/crowd-related information and sends it to a dedicated dashboard for real-time visualization and further assessment. This is done without sending the raw video, in order to avoid violating pedestrians’ privacy. 
    more » « less
  4. ; ; ; (, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)
    We propose MiShape, a millimeter-wave (mmWave) wireless signal based imaging system that generates high-resolution human silhouettes and predicts 3D locations of body joints. The system can capture human motions in real-time under low light and low-visibility conditions. Unlike existing vision-based motion capture systems, MiShape is privacy non-invasive and can generalize to a wide range of motion tracking applications at-home. To overcome the challenges with low-resolution, specularity, and aliasing in images from Commercial-Off-The-Shelf (COTS) mmWave systems, MiShape designs deep learning models based on conditional Generative Adversarial Networks and incorporates the rules of human biomechanics. We have customized MiShape for gait monitoring, but the model is well adaptive to any tracking applications with limited fine-tuning samples. We experimentally evaluate MiShape with real data collected from a COTS mmWave system for 10 volunteers, with diverse ages, gender, height, and somatotype, performing different poses. Our experimental results demonstrate that MiShape delivers high-resolution silhouettes and accurate body poses on par with an existing vision-based system, and unlocks the potential of mmWave systems, such as 5G home wireless routers, for privacy-noninvasive healthcare applications. 
    more » « less
  5. (, in Proc. IEEE PerCom’23, Mar. 2023)
    The density and complexity of urban environments present significant challenges for autonomous vehicles. Moreover, ensuring pedestrians’ safety and protecting personal privacy are crucial considerations in these environments. Smart city intersections and AI-powered traffic management systems will be essential for addressing these challenges. Therefore, our research focuses on creating an experimental framework for the design of applications that support the secure and efficient management of traffic intersections in urban areas. We integrated two cameras (street-level and bird’s eye view), both viewing an intersection, and a programmable edge computing node, deployed within the COSMOS testbed in New York City, with a central management platform provided by Kentyou. We designed a pipeline to collect and analyze the video streams from both cameras and obtain real-time traffic/pedestrian-related information to support smart city applications. The obtained information from both cameras is merged, and the results are sent to a dedicated dashboard for real-time visualization and further assessment (e.g., accident prevention). The process does not require sending the raw videos in order to avoid violating pedestrians’ privacy. In this demo, we present the designed video analytic pipelines and their integration with Kentyou central management platform. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email