More Like this

1. Defending against Backdoors in Federated Learning with Robust Learning Rate

   Ozdayi, Mustafa Safa ; Kantarcioglu, Murat ; Gel, Yulia R. ( May 2021 , Proceedings of the AAAI Conference on Artificial Intelligence)

   null (Ed.)

   Federated learning (FL) allows a set of agents to collaboratively train a model without sharing their potentially sensitive data. This makes FL suitable for privacy-preserving applications. At the same time, FL is susceptible to adversarial attacks due to decentralized and unvetted data. One important line of attacks against FL is the backdoor attacks. In a backdoor attack, an adversary tries to embed a backdoor functionality to the model during training that can later be activated to cause a desired misclassification. To prevent backdoor attacks, we propose a lightweight defense that requires minimal change to the FL protocol. At a high level, our defense is based on carefully adjusting the aggregation server's learning rate, per dimension and per round, based on the sign information of agents' updates. We first conjecture the necessary steps to carry a successful backdoor attack in FL setting, and then, explicitly formulate the defense based on our conjecture. Through experiments, we provide empirical evidence that supports our conjecture, and we test our defense against backdoor attacks under different settings. We observe that either backdoor is completely eliminated, or its accuracy is significantly reduced. Overall, our experiments suggest that our defense significantly outperforms some of the recently proposed defenses in the literature. We achieve this by having minimal influence over the accuracy of the trained models. In addition, we also provide convergence rate analysis for our proposed scheme. 

   more » « less
2. Evaluation of Various Defense Techniques Against Targeted Poisoning Attacks in Federated Learning

   https://doi.org/10.1109/MASS56207.2022.00102  

   Richards, Charles ; Khemani, Sofia ; Li, Feng ( October 2022 , IEEE)

   Federated Learning (FL) allows individual clients to train a global model by aggregating local model updates each round. This results in collaborative model training while main-taining the privacy of clients' sensitive data. However, malicious clients can join the training process and train with poisoned data or send artificial model updates in targeted poisoning attacks. Many defenses to targeted poisoning attacks rely on anomaly-detection based metrics which remove participants that deviate from the majority. Similarly, aggregation-based defenses aim to reduce the impact of outliers, while L2-norm clipping tries to scale down the impact of malicious models. However, oftentimes these defenses misidentify benign clients as malicious or only work under specific attack conditions. In our paper, we examine the effectiveness of two anomaly -detection metrics on three different aggregation methods, in addition to the presence of L2-norm clipping and weight selection, across two different types of attacks. We also combine different defenses in order to examine their interaction and examine each defense when no attack is present. We found minimum aggregation to be the most effective defense against label-flipping attacks, whereas both minimum aggregation and geometric median worked well against distributed backdoor attacks. Using random weight selection significantly deteriorated defenses against both attacks, whereas the use of clipping made little difference. Finally, the main task accuracy was directly correlated with the BA in the label-flipping attack and generally was close to the MA in benign scenarios. However, in the DBA the MA and BA are inversely correlated and the MA fluctuates greatly. 

   more » « less
3. Privacy-Engineered Value Decomposition Networks for Cooperative Multi-Agent Reinforcement Learning

   https://doi.org/10.1109/CDC49753.2023.10384184  

   Gohari, Parham ; Hale, Matthew ; Topcu, Ufuk ( December 2023 , Proceedings of the 62nd IEEE Conference on Decision and Control (CDC))

   In cooperative multi-agent reinforcement learning (Co-MARL), a team of agents must jointly optimize the team's longterm rewards to learn a designated task. Optimizing rewards as a team often requires inter-agent communication and data sharing, leading to potential privacy implications. We assume privacy considerations prohibit the agents from sharing their environment interaction data. Accordingly, we propose Privacy-Engineered Value Decomposition Networks (PE-VDN), a Co-MARL algorithm that models multi-agent coordination while provably safeguarding the confidentiality of the agents' environment interaction data. We integrate three privacy-engineering techniques to redesign the data flows of the VDN algorithm-an existing Co-MARL algorithm that consolidates the agents' environment interaction data to train a central controller that models multi-agent coordination-and develop PE-VDN. In the first technique, we design a distributed computation scheme that eliminates Vanilla VDN's dependency on sharing environment interaction data. Then, we utilize a privacy-preserving multi-party computation protocol to guar-antee that the data flows of the distributed computation scheme do not pose new privacy risks. Finally, we enforce differential privacy to preempt inference threats against the agents' training data-past environment interactions-when they take actions based on their neural network predictions. We implement PE-VDN in StarCraft Multi-Agent Competition (SMAC) and show that it achieves 80% of Vanilla VDN's win rate while maintaining differential privacy levels that provide meaningful privacy guarantees. The results demonstrate that PE-VDN can safeguard the confidentiality of agents' environment interaction data without sacrificing multi-agent coordination. 

   more » « less
4. Deletion inference, reconstruction, and compliance in machine (un)learning

   https://doi.org/10.56553/popets-2022-0079  

   Gao, Ji ; Garg, Sanjam ; Mahmoody, Mohammad ; Vasudevan, Prashant Nalini ( July 2022 , Proceedings on Privacy Enhancing Technologies)

   Privacy attacks on machine learning models aim to identify the data that is used to train such models. Such attacks, traditionally, are studied on static models that are trained once and are accessible by the adversary. Motivated to meet new legal requirements, many machine learning methods are recently extended to support machine unlearning, i.e., updating models as if certain examples are removed from their training sets, and meet new legal requirements. However, privacy attacks could potentially become more devastating in this new setting, since an attacker could now access both the original model before deletion and the new model after the deletion. In fact, the very act of deletion might make the deleted record more vulnerable to privacy attacks. Inspired by cryptographic definitions and the differential privacy framework, we formally study privacy implications of machine unlearning. We formalize (various forms of) deletion inference and deletion reconstruction attacks, in which the adversary aims to either identify which record is deleted or to reconstruct (perhaps part of) the deleted records. We then present successful deletion inference and reconstruction attacks for a variety of machine learning models and tasks such as classification, regression, and language models. Finally, we show that our attacks would provably be precluded if the schemes satisfy (variants of) deletion compliance (Garg, Goldwasser, and Vasudevan, Eurocrypt’20). 

   more » « less
5. Mitigating Data Poisoning Attacks On a Federated Learning-Edge Computing Network

   https://doi.org/10.1109/CCNC49032.2021.9369581  

   Doku, Ronald ; Rawat, Danda B. ( January 2021 , 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC))

   null (Ed.)

   Edge Computing (EC) has seen a continuous rise in its popularity as it provides a solution to the latency and communication issues associated with edge devices transferring data to remote servers. EC achieves this by bringing the cloud closer to edge devices. Even though EC does an excellent job of solving the latency and communication issues, it does not solve the privacy issues associated with users transferring personal data to the nearby edge server. Federated Learning (FL) is an approach that was introduced to solve the privacy issues associated with data transfers to distant servers. FL attempts to resolve this issue by bringing the code to the data, which goes against the traditional way of sending the data to remote servers. In FL, the data stays on the source device, and a Machine Learning (ML) model used to train the local data is brought to the end device instead. End devices train the ML model using local data and then send the model updates back to the server for aggregation. However, this process of asking random devices to train a model using its local data has potential risks such as a participant poisoning the model using malicious data for training to produce bogus parameters. In this paper, an approach to mitigate data poisoning attacks in a federated learning setting is investigated. The application of the approach is highlighted, and the practical and secure nature of this approach is illustrated as well using numerical results. 

   more » « less