An official website of the United States government
Video scene analysis is a well-investigated area where researchers have devoted efforts to detect and classify people and objects in the scene. However, real-life scenes are more complex: the intrinsic states of the objects (e.g., machine operating states or human vital signals) are often overlooked by vision-based scene analysis. Recent work has proposed a radio frequency (RF) sensing technique, wireless vibrometry, that employs wireless signals to sense subtle vibrations from the objects and infer their internal states. We envision that the combination of video scene analysis with wireless vibrometry form a more comprehensive understanding of the scene, namely "rich scene analysis". However, the RF sensors used in wireless vibrometry only provide time series, and it is challenging to associate these time series data with multiple real-world objects. We propose a real-time RF-vision sensor fusion system, Capricorn, that efficiently builds a cross-modal correspondence between visual pixels and RF time series to better understand the complex natures of a scene. The vision sensors in Capricorn model the surrounding environment in 3D and obtain the distances of different objects. In the RF domain, the distance is proportional to the signal time-of-flight (ToF), and we can leverage the ToF to separate the RF time series corresponding to each object. The RF-vision sensor fusion in Capricorn brings multiple benefits. The vision sensors provide environmental contexts to guide the processing of RF data, which helps us select the most appropriate algorithms and models. Meanwhile, the RF sensor yields additional information that is originally invisible to vision sensors, providing insight into objects' intrinsic states. Our extensive evaluations show that Capricorn real-timely monitors multiple appliances' operating status with an accuracy of 97%+ and recovers vital signals like respirations from multiple people. A video (https://youtu.be/b-5nav3Fi78) demonstrates the capability of Capricorn.
more »
« less
Delay-Tolerant Data Fusion for Underwater Acoustic Tracking Networks
We consider a network of distributed underwater sensors whose task is to monitor the movement of objects across an area. The sensors measure the strength of signals emanated by the objects and convey the measurements to the local fusion centers. Multiple fusion centers are deployed to cover an arbitrarily large area. The fusion centers communicate with each other to achieve consensus on the estimated locations of the moving objects. We introduce two efficient methods for data fusion of distributed partial estimates when delay in communication is not negligible. We concentrate on the minimum mean squared error (MMSE) global estimator, and evaluate the performance of these fusion methods in the context of multiple-object tracking via extended Kalman filtering. Numerical results show the superior performance compared to the case when delay is ignored.
more »
« less
- Award ID(s):
- 1726512
- PAR ID:
- 10316507
- Date Published:
- Journal Name:
- Underwater Communications and Networking
- ISSN:
- 2471-1268
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Paolo Spagnolo (Ed.)This paper discusses asynchronous distributed inference in object tracking. Unlike many studies, which assume that the delay in communication between partial estimators and the central station is negligible, our study focuses on the problem of asynchronous distributed inference in the presence of delays. We introduce an efficient data fusion method for combining the distributed estimates, where delay in communications is not negligible. To overcome the delay, predictions are made for the state of the system based on the most current available information from partial estimators. Simulation results show the efficacy of the methods proposed.more » « less
-
Collaborative object localization aims to collaboratively estimate locations of objects observed from multiple views or perspectives, which is a critical ability for multi-agent systems such as connected vehicles. To enable collaborative localization, several model-based state estimation and learning-based localization methods have been developed. Given their encouraging performance, model-based state estimation often lacks the ability to model the complex relationships among multiple objects, while learning-based methods are typically not able to fuse the observations from an arbitrary number of views and cannot well model uncertainty. In this paper, we introduce a novel spatiotemporal graph filter approach that integrates graph learning and model-based estimation to perform multi-view sensor fusion for collaborative object localization. Our approach models complex object relationships using a new spatiotemporal graph representation and fuses multi-view observations in a Bayesian fashion to improve location estimation under uncertainty. We evaluate our approach in the applications of connected autonomous driving and multiple pedestrian localization. Experimental results show that our approach outperforms previous techniques and achieves the state-of-the-art performance on collaborative localization.more » « less
-
We study a heterogeneous two-tier wireless sensor network in which N heterogeneous access points (APs) collect sensing data from densely distributed sensors and then forward the data to M heterogeneous fusion centers (FCs). This heterogeneous node deployment problem is modeled as a quantization problem with distortion defined as the total power consumption of the network. The necessary conditions of the optimal AP and FC node deployment are explored in this paper. We provide a variation of Voronoi diagrams as the optimal cell partition for this network, and show that each AP should be placed between its connected FC and the geometric center of its cell partition. In addition, we propose a heterogeneous two-tier Lloyd-like algorithm to optimize the node deployment. Simulation results show that our proposed algorithm outperforms the existing methods like Minimum Energy Routing, Agglomerative Clustering, and Divisive Clustering, on average.more » « less
-
Given sensor units distributed throughout an environment, we consider the problem of consolidating readings into a single coherent view when sensors wish to limit knowledge of their specific readings. Standard fusion methods make no guarantees about what curious participants may learn. For applications where privacy guarantees are required, we introduce a fusion approach that limits what can be inferred. First, it forms an aggregate stream, oblivious to the underlying sensor data, and then evaluates that stream on a combinatorial filter. This is achieved via secure multi-party computation techniques built on cryptographic primitives, which we extend and apply to the problem of fusing discrete sensor signals. We prove that the extensions preserve security under the model of semi-honest adversaries. Also, for a simple target tracking case study, we examine a proof-of-concept implementation: analyzing the (empirical) running times for components in the architecture and suggesting directions for future improvement.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/UComms50339.2021.9598132
