More Like this

1. Vi-Fi: Associating Moving Subjects across Vision and Wireless Sensors

   https://doi.org/10.1109/IPSN54338.2022.00024  

   Liu H, Alali A (May 2022, International Conference on Information Processing in Sensor Networks (IPSN 2022))

   In this paper, we present Vi-Fi, a multi-modal system that leverages a user’s smartphone WiFi Fine Timing Measurements (FTM) and inertial measurement unit (IMU) sensor data to associate the user detected on a camera footage with their corresponding smartphone identifier (e.g. WiFi MAC address). Our approach uses a recurrent multi-modal deep neural network that exploits FTM and IMU measurements along with distance between user and camera (depth information) to learn affinity matrices. As a baseline method for comparison, we also present a traditional non deep learning approach that uses bipartite graph matching. To facilitate evaluation, we collected a multi-modal dataset that comprises camera videos with depth information (RGB-D), WiFi FTM and IMU measurements for multiple participants at diverse real-world settings. Using association accuracy as the key metric for evaluating the fidelity of Vi-Fi in associating human users on camera feed with their phone IDs, we show that Vi-Fi achieves between 81% (real-time) to 91% (offline) association accuracy 

   more » « less
2. TRACER: Efficient Object Re-Identification in Networked Cameras through Adaptive Query Processing

   Chunduri, Pramod; Lu, Yao; Arulraj, Joy (July 2025, https://arxiv.org/abs/2507.09448)

   Efficiently re-identifying and tracking objects across a network of cameras is crucial for applications like traffic surveillance. Spatula is the state-of-the-art video database man- agement system (VDBMS) for processing RE-ID queries. However, it suffers from two limitations. Its spatio-temporal filtering scheme has limited accuracy on large camera networks due to localized camera history. It is not suitable for critical video analytics applications that require high recall due to lack of support for adaptive query processing. In this paper, we present TRACER, a novel VDBMS for efficiently processing RE-ID queries using an adaptive query pro- cessing framework. TRACER selects the optimal camera to process at each time step by training a recurrent network to model long- term historical correlations. To accelerate queries under a high recall constraint, TRACER incorporates a probabilistic adaptive search model that processes camera feeds in incremental search windows and dynamically updates the sampling probabilities using an exploration-exploitation strategy. To address the paucity of benchmarks for the RE-ID task due to privacy concerns, we present a novel synthetic benchmark for generating multi-camera RE-ID datasets based on real-world traffic distribution. Our evaluation shows that TRACER outperforms the state-of-the-art cross-camera analytics system by 3.9⇥ on average across diverse datasets. 

   more » « less
3. Distributed Multi-robot Tracking of Unknown Clustered Targets with Noisy Measurements

   https://doi.org/10.1007/978-3-031-51497-5_10  

   Chen, Jun; Dames, Philip; Park, Shinkyu (January 2024, Springer Nature Switzerland)

   Distributed multi-target tracking is a canonical task for multi-robot systems, encompassing applications from environmental monitoring to disaster response to surveillance. In many situations, the distribution of unknown objects in a search area is irregular, with objects are likely to distribute in clusters instead of evenly distributed. In this paper, we develop a novel distributed multi-robot multi-target tracking algorithm for effectively tracking clustered targets from noisy measurements. Our algorithm contains two major components. Firstly, both the instantaneous and cumulative target density are estimated, providing the best guess of current target states and long-term coarse distribution of clusters, respectively. Secondly, the power diagram is implemented in Lloyd’s algorithm to optimize task space assignment for each robot to trade-off between tracking detected targets in clusters and searching for potential targets outside clusters. We demonstrate the efficacy of our proposed method and show that our method outperforms of other candidates in tracking accuracy through a set of simulations. 

   more » « less
4. ViFiT: Reconstructing Vision Trajectories from IMU and Wi-Fi Fine Time Measurements

   https://doi.org/10.1145/3615984.3616503  

   Cao, Bryan Bo; Alali, Abrar; Liu, Hansi; Meegan, Nicholas; Gruteser, Marco; Dana, Kristin; Ashok, Ashwin; Jain, Shubham (October 2023, ACM)

   Tracking subjects in videos is one of the most widely used functions in camera-based IoT applications such as security surveillance, smart city traffic safety enhancement, vehicle to pedestrian communication and so on. In computer vision domain, tracking is usually achieved by first detecting subjects, then associating detected bounding boxes across video frames. Typically, frames are transmitted to a remote site for processing, incurring high latency and network costs. To address this, we propose ViFiT, a transformerbased model that reconstructs vision bounding box trajectories from phone data (IMU and Fine Time Measurements). It leverages a transformer’s ability of better modeling long-term time series data. ViFiT is evaluated on Vi-Fi Dataset, a large-scale multimodal dataset in 5 diverse real world scenes, including indoor and outdoor environments. Results demonstrate that ViFiT outperforms the state-of-the-art approach for cross-modal reconstruction in LSTM Encoder-Decoder architecture X-Translator and achieves a high frame reduction rate as 97.76% with IMU and Wi-Fi data. 

   more » « less
5. The Architectural Implications of Multi-modal Detection Models for Autonomous Driving Systems

   Li, Yunge; Saha, Shaibal; Xu, Lanyu (May 2024, 2024 IEEE International Conference on Mobility, Operations, Services and Technologies (MOST))

   Object detection plays a pivotal in autonomous driving by enabling the vehicles to perceive and comprehend their environment, thereby making informed decisions for safe navigation. Camera data provides rich visual context and object recognition, while LiDAR data offers precise distance measurements and 3D mapping. Multi-modal object detection models are gaining prominence in incorporating these data types, which is essential for the comprehensive perception and situational awareness needed in autonomous vehicles. Although graphics processing units (GPUs) and field-programmable gate arrays (FPGAs) are promising hardware options for this application, the complex knowledge required to efficiently adapt and optimize multi-modal detection models for FPGAs presents a significant barrier to their utilization on this versatile and efficient platform. In this work, we evaluate the performance of camera and LiDARbased detection models on GPU and FPGA hardware, aiming to provide a specialized understanding for translating multi-modal detection models to suit the unique architecture of heterogeneous hardware platforms in autonomous driving systems. We focus on critical metrics from both system and model performance aspects. Based on our quantitative implications, we propose foundational insights and guidance for the design of camera and LiDAR-based multi-modal detection models on diverse hardware platforms. 

   more » « less