An official website of the United States government
With the wider adoption of edge computing services, intelligent edge devices, and high-speed V2X communication, compute-intensive tasks for autonomous vehicles, such as object detection using camera, LiDAR, and/or radar data, can be partially offloaded to road-side edge servers. However, data privacy becomes a major concern for vehicular edge computing, as sensitive sensor data from vehicles can be observed and used by edge servers. We aim to address the privacy problem by protecting both vehicles’ sensor data and the detection results. In this paper, we present vehicle–edge cooperative deep-learning networks with privacy protection for object-detection tasks, named vePOD for short. In vePOD, we leverage the additive secret sharing theory to develop secure functions for every layer in an object-detection convolutional neural network (CNN). A vehicle’s sensor data is split and encrypted into multiple secret shares, each of which is processed on an edge server by going through the secure layers of a detection network. The detection results can only be obtained by combining the partial results from the participating edge servers. We have developed proof-of-concept detection networks with secure layers: vePOD Faster R-CNN (two-stage detection) and vePOD YOLO (single-stage detection). Experimental results on public datasets show that vePOD does not degrade the accuracy of object detection and, most importantly, it protects data privacy for vehicles. The execution of a vePOD object-detection network with secure layers is orders of magnitude faster than the existing approaches for data privacy. To the best of our knowledge, this is the first work that targets privacy protection in object-detection tasks with vehicle–edge cooperative computing.
more »
« less
This content will become publicly available on December 9, 2026
Privacy-Preserving Driver Monitoring on the Edges: Transformer-Based Processing of Secret Shares from Video Streams
Modern vehicles increasingly rely on advanced driver monitoring systems (DMS) to ensure safety and enhance the driving experience. These systems assess driver status to prevent accidents caused by fatigue, inattentiveness, or intoxication. While some DMS applications process video data on vehicle, many rely on edge or cloud-based solutions, raising significant privacy concerns due to the storage of sensor data from vehicles. Existing approaches, such as de-identification and homomorphic encryption, either impose heavy computational overhead on vehicles or insufficiently address privacy. To overcome these limitations, we present the Privacy-preserving Driver Monitoring System (PDMS), a novel framework based on the additive secret sharing theory and privacy-preserving Transformer-based deep learning models. PDMS creates randomized secret shares from driver’s facial video data on vehicle, processes them independently through privacy-preserving Transformer models on edges, and securely aggregates partial results on vehicle, ensuring vehicles’ sensor data and final results remain protected. This approach reduces the computational load on the vehicle, enabling cost-effective and scalable DMS solutions that protect the privacy of the driver both in transit and in processing. Our contributions include the design and optimization of the PDMS system, incorporating privacy-preserving DNN layers that are capable of processing randomized secret shares. Furthermore, we present a practical system that utilizes a vision transformer (ViT)-based gaze estimation model, demonstrating the effectiveness of PDMS through comprehensive experiments.
more »
« less
- Award ID(s):
- 2231519
- PAR ID:
- 10667358
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- ACM Transactions on Internet of Things
- ISSN:
- 2691-1914
- Page Range / eLocation ID:
- 1-31
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper introduces PAVE (Pedestrian Awareness Via Edge analytics), a scalable real-time video analytics system that uses street cameras to enhance pedestrian safety while preserving their privacy. PAVE processes live camera streams on an edge server to track pedestrians and vehicles in realtime, predict vehicles’ trajectories, and identify danger zones where pedestrians are present. The coordinates of these zones are sent to pedestrians’ mobile devices via a custom iOS app, which locally determines if they are at risk without sharing any data with the edge server, hence preserving privacy. Moreover, anonymized metadata, including real-time location and speed/direction of pedestrians and vehicles, are visualized on a public map. PAVE’s effectiveness was validated through deployment on the NSF COSMOS testbed, processing live video from cameras in diverse urban environments. Live field tests show that PAVE can alert at-risk pedestrians ∼0.9 s before a vehicle reaches them. Through extensive profiling, we show that optimizing memory/compute configuration per pipeline stage can reduce latency by up to 10× compared to the default operating system configurations.more » « less
-
The integration of connected autonomous vehicles (CAVs) has significantly enhanced driving convenience, but it has also raised serious privacy concerns, particularly regarding the personal identifiable information (PII) stored on infotainment systems. Recent advances in connected and autonomous vehicle control, such as multi-agent system (MAS)-based hierarchical architectures and privacy-preserving strategies for mixed-autonomy platoon control, underscore the increasing complexity of privacy management within these environments. Rental cars with infotainment systems pose substantial challenges, as renters often fail to delete their data, leaving it accessible to subsequent renters. This study investigates the risks associated with PII in connected vehicles and emphasizes the necessity of automated solutions to ensure data privacy. We introduce the Vehicle Inactive Profile Remover (VIPR), an innovative automated solution designed to identify and delete PII left on infotainment systems. The efficacy of VIPR is evaluated through surveys, hands-on experiments with rental vehicles, and a controlled laboratory environment. VIPR achieved a 99.5% success rate in removing user profiles, with an average deletion time of 4.8 s or less, demonstrating its effectiveness in mitigating privacy risks. This solution highlights VIPR as a critical tool for enhancing privacy in connected vehicle environments, promoting a safer, more responsible use of connected vehicle technology in society.more » « less
-
Multi-user environments present significant challenges in coordinating diverse preferences and resolving conflicts around shared resources. Current systems use a single-agent approach that struggles to balance individual needs with collective objectives. We introduce MALLM, a novel framework that deploys personalized LLM-based agents for each user on edge devices. MALLM integrates multi-sensor data fusion with a structured multi-agent decision-making mechanism, processing all data locally for enhanced privacy. Our edge-computing architecture enables real-time deliberation through evidence-based argumentation and consensus formation algorithms. The system continuously refines user profiles through sensor data while managing computational resources e!ciently. We evaluate MALLM through two case studies-health monitoring and personalized comfort management- demonstrating improved conflict resolution and resource e!ciency compared to conventional approaches. Our results show that MALLM e''ectively balances competing user priorities while preserving privacy in complex shared environments.more » « less
-
Many AI platforms, including traffic monitoring systems, use Federated Learning (FL) for decentralized sensor data processing for learning-based applications while preserving privacy and ensuring secured information transfer. On the other hand, applying supervised learning to large data samples, like high-resolution images requires intensive human labor to label different parts of a data sample. Multiple Instance Learning (MIL) alleviates this challenge by operating over labels assigned to the ’bag’ of instances. In this paper, we introduce Federated Multiple-Instance Learning (FedMIL). This framework applies federated learning to boost the training performance in video-based MIL tasks such as vehicle accident detection using distributed CCTV networks. However, data sources in decentralized settings are not typically Independently and Identically Distributed (IID), making client selection imperative to collectively represent the entire dataset with minimal clients. To address this challenge, we propose DPPQ, a framework based on the Determinantal Point Process (DPP) with a quality-based kernel to select clients with the most diverse datasets that achieve better performance compared to both random selection and current DPP-based client selection methods even with less data utilization in the majority of non-IID cases. This offers a significant advantage for deployment on edge devices with limited computational resources, providing a reliable solution for training AI models in massive smart sensor networks.more » « less
