With the fast development of Fifth-/Sixth-Generation (5G/6G) communications and the Internet of Video Things (IoVT), a broad range of mega-scale data applications emerge (e.g., all-weather all-time video). These network-based applications highly depend on reliable, secure, and real-time audio and/or video streams (AVSs), which consequently become a target for attackers. While modern Artificial Intelligence (AI) technology is integrated with many multimedia applications to help enhance its applications, the development of General Adversarial Networks (GANs) also leads to deepfake attacks that enable manipulation of audio or video streams to mimic any targeted person. Deepfake attacks are highly disturbing and can mislead the public, raising further challenges in policy, technology, social, and legal aspects. Instead of engaging in an endless AI arms race “fighting fire with fire”, where new Deep Learning (DL) algorithms keep making fake AVS more realistic, this paper proposes a novel approach that tackles the challenging problem of detecting deepfaked AVS data leveraging Electrical Network Frequency (ENF) signals embedded in the AVS data as a fingerprint. Under low Signal-to-Noise Ratio (SNR) conditions, Short-Time Fourier Transform (STFT) and Multiple Signal Classification (MUSIC) spectrum estimation techniques are investigated to detect the Instantaneous Frequency (IF) of interest. For reliable authentication, we enhanced the ENF signal embedded through an artificial power source in a noisy environment using the spectral combination technique and a Robust Filtering Algorithm (RFA). The proposed signal estimation workflow was deployed on a continuous audio/video input for resilience against frame manipulation attacks. A Singular Spectrum Analysis (SSA) approach was selected to minimize the false positive rate of signal correlations. Extensive experimental analysis for a reliable ENF edge-based estimation in deepfaked multimedia recordings is provided to facilitate the need for distinguishing artificially altered media content.
more »
« less
Detecting Compromised Edge Smart Cameras using Lightweight Environmental Fingerprint Consensus
Rapid advances in the Internet of Video Things (IoVT) deployment in modern smart cities has enabled secure infrastructures with minimal human intervention. However, attacks on audio-video inputs affect the reliability of large-scale multimedia surveillance systems as attackers are able to manipulate the perception of live events. For example, Deepfake audio/video attacks and frame duplication attacks can cause significant security breaches. This paper proposes a Lightweight Environmental Fingerprint Consensus based detection of compromised smart cameras in edge surveillance systems (LEFC). LEFC is a partial decentralized authentication mechanism that leverages Electrical Network Frequency (ENF) as an environmental fingerprint and distributed ledger technology (DLT). An ENF signal carries randomly fluctuating spatio-temporal signatures, which enable digital media authentication. With the proposed DLT consensus mechanism named Proof-of-ENF (PoENF) as a backbone, LEFC can estimate and authenticate the media recording and detect byzantine nodes controlled by the perpetrator. The experimental evaluation shows feasibility and effectiveness of proposed LEFC scheme under a distributed byzantine network environment.
more »
« less
- Award ID(s):
- 2039342
- NSF-PAR ID:
- 10359340
- Date Published:
- Journal Name:
- The 19th ACM Conference on Embedded Networked Sensor Systems
- Page Range / eLocation ID:
- 505 to 510
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Wysocki, Bryant T. ; Holt, James ; Blowers, Misty (Ed.)The information era has gained a lot of traction due to the abundant digital media contents through technological broadcasting resources. Among the information providers, the social media platform has remained a popular platform for the widespread reach of digital content. Along with accessibility and reach, social media platforms are also a huge venue for spreading misinformation since the data is not curated by trusted authorities. With many malicious participants involved, artificially generated media or strategically altered content could potentially result in affecting the integrity of targeted organizations. Popular content generation tools like DeepFake have allowed perpetrators to create realistic media content by manipulating the targeted subject with a fake identity or actions. Media metadata like time and location-based information are altered to create a false perception of real events. In this work, we propose a Decentralized Electrical Network Frequency (ENF)-based Media Authentication (DEMA) system to verify the metadata information and the digital multimedia integrity. Leveraging the environmental ENF fingerprint captured by digital media recorders, altered media content is detected by exploiting the ENF consistency based on its time and location of recording along with its spatial consistency throughout the captured frames. A decentralized and hierarchical ENF map is created as a reference database for time and location verification. For digital media uploaded to a broadcasting service, the proposed DEMA system correlates the underlying ENF fingerprint with the stored ENF map to authenticate the media metadata. With the media metadata intact, the embedded ENF in the recording is compared with a reference ENF based on the time of recording, and a correlation-based metric is used to evaluate the media authenticity. In case of missing metadata, the frames are divided spatially to compare the ENF consistency throughout the recording.more » « less
-
null (Ed.)The current centralized model of the electricity market is not efficient in performing distributed energy transactions required for the transactive smart grid. One of the prominent solutions to this issue is to integrate blockchain technologies, which promise transparent, tamper-proof, and secure transaction systems specifically suitable for the decentralized and distributed energy markets. Blockchain has already been shown to successfully operate in a microgrid peer-to-peer (P2P) energy market. The prime determinant of different blockchain implementations is the consensus algorithm they use to reach consensus on which blocks/transactions to accept as valid in a distributed environment. Although different blockchain implementations have been proposed independently for P2P energy market in the microgrid, quantitative experimental analyses and comparison of the consensus algorithms that the different blockchains may use for energy markets, has not been studied. Identifying the right consensus algorithm to use is essential for scalability and operation of the energy market. To this end, we evaluate three popular consensus algorithms: (i) proof of work (PoW), (ii) proof of authority (PoA), and (iii) Istanbul Byzantine fault tolerance (IBFT), running them on a network of nodes set up using a network of docker nodes to form a microgrid energy market. Using a series of double auctions, we assess each algorithm's viability using different metrics, such as time to reach consensus and scalability. The results indicate that PoA is the most efficient and scalable consensus algorithm to hold double auctions in the smart grid. We also identified the minimum hardware specification necessary for devices such as smart meters, which may run these consensus algorithmsmore » « less
-
The current centralized model of the electricity market is not efficient in performing distributed energy transactions required for the transactive smart grid. One of the prominent solutions to this issue is to integrate blockchain technologies, which promise transparent, tamper-proof, and secure transaction systems specifically suitable for the decentralized and distributed energy markets. Blockchain has already been shown to successfully operate in a microgrid peer-to-peer (P2P) energy market. The prime determinant of different blockchain implementations is the consensus algorithm they use to reach consensus on which blocks/transactions to accept as valid in a distributed environment. Although different blockchain implementations have been proposed independently for P2P energy market in the microgrid, quantitative experimental analyses and comparison of the consensus algorithms that the different blockchains may use for energy markets, has not been studied. Identifying the right consensus algorithm to use is essential for scalability and operation of the energy market. To this end, we evaluate three popular consensus algorithms: (i) proof of work (PoW), (ii) proof of authority (PoA), and (iii) Istanbul Byzantine fault tolerance (IBFT), running them on a network of nodes set up using a network of docker nodes to form a microgrid energy market. Using a series of double auctions, we assess each algorithm’s viability using different metrics, such as time to reach consensus and scalability. The results indicate that PoA is the most efficient and scalable consensus algorithm to hold double auctions in the smart grid. We also identified the minimum hardware specification necessary for devices such as smart meters, which may run these consensus algorithms.more » « less
-
A Continuous Articulatory Gesture Based Liveness Detection for Voice Authentication on Smart DevicesVoice biometrics is drawing increasing attention to user authentication on smart devices. However, voice biometrics is vulnerable to replay attacks, where adversaries try to spoof voice authentication systems using pre-recorded voice samples collected from genuine users. To this end, we propose VoiceGesture, a liveness detection solution for voice authentication on smart devices such as smartphones and smart speakers. With audio hardware advances on smart devices, VoiceGesture leverages built-in speaker and microphone pairs on smart devices as Doppler Radar to sense articulatory gestures for liveness detection during voice authentication. The experiments with 21 participants and different smart devices show that VoiceGesture achieves over 99% and around 98% detection accuracy for text-dependent and text-independent liveness detection, respectively. Moreover, VoiceGesture is robust to different device placements, low audio sampling frequency, and supports medium range liveness detection on smart speakers in various use scenarios, including smart homes and smart vehicles.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3485730.3493684
