More Like this

1. Model-Targeted Poisoning Attacks with Provable Convergence

   Suya, Fnu; Mahloujifar, Saeed; Suri, Anshuman; Evans, David; Tian, Yuan (January 2021, 38th International Conference on Machine Learning)

   null (Ed.)

   In a poisoning attack, an adversary with control over a small fraction of the training data attempts to select that data in a way that induces a corrupted model that misbehaves in favor of the adversary. We consider poisoning attacks against convex machine learning models and propose an efficient poisoning attack designed to induce a specified model. Unlike previous model-targeted poisoning attacks, our attack comes with provable convergence to any attainable target classifier. The distance from the induced classifier to the target classifier is inversely proportional to the square root of the number of poisoning points. We also provide a lower bound on the minimum number of poisoning points needed to achieve a given target classifier. Our method uses online convex optimization, so finds poisoning points incrementally. This provides more flexibility than previous attacks which require a priori assumption about the number of poisoning points. Our attack is the first model-targeted poisoning attack that provides provable convergence for convex models, and in our experiments, it either exceeds or matches state-of-the-art attacks in terms of attack success rate and distance to the target model. 

   more » « less
2. PORTFILER: Port-Level Network Profiling for Self-Propagating Malware Detection

   Ongun, Talha; Spohngellert, Oliver; Miller, Benjamin; Boboila, Simona; Oprea, Alina; Eliassi-Rad, Tina; Hiser, Jason; Nottingham, Alastair; Davidson, Jack; Veeraraghavan, Malathi (January 2021, IEEE Conference on Communications and Network Security)

   null (Ed.)

   Recent self-propagating malware (SPM) campaigns compromised hundred of thousands of victim machines on the Internet. It is challenging to detect these attacks in their early stages, as adversaries utilize common network services, use novel techniques, and can evade existing detection mechanisms. We propose PORTFILER (PORT-Level Network Traffic ProFILER), a new machine learning system applied to network traffic for detecting SPM attacks. PORTFILER extracts port-level features from the Zeek connection logs collected at a border of a monitored network, applies anomaly detection techniques to identify suspicious events, and ranks the alerts across ports for investigation by the Security Operations Center (SOC). We propose a novel ensemble methodology for aggregating individual models in PORTFILER that increases resilience against several evasion strategies compared to standard ML baselines. We extensively evaluate PORTFILER on traffic collected from two university networks, and show that it can detect SPM attacks with different patterns, such as WannaCry and Mirai, and performs well under evasion. Ranking across ports achieves precision over 0.94 and false positive rates below 8 × 10−4 in the top 100 highly ranked alerts. When deployed on the university networks, PORTFILER detected anomalous SPM-like activity on one of the campus networks, confirmed by the university SOC as malicious. PORTFILER also detected a Mirai attack recreated on the two university networks with higher precision and recall than deep learning based autoencoder methods. 

   more » « less
3. Insights on Using Deep Learning to Spoof Inertial Measurement Units for Stealthy Attacks on UAVs

   https://doi.org/10.1109/MILCOM55135.2022.10017482  

   Kim, Kyo Hyun; Kara, Denizkhan; Paruchuri, Vineetha; Mohan, Sibin; Kimberly, Greg; Osipychev, Denis; Kim, Jae H.; Eckhardt, Josh D.; Pajic, Miroslav (November 2022, MILCOM, Military Communications Conference)

   Unmanned Aerial Vehicles (UAVs) find increasing use in mission critical tasks both in civilian and military operations. Most UAVs rely on Inertial Measurement Units (IMUs) to calculate vehicle attitude and track vehicle position. Therefore, an incorrect IMU reading can cause a vehicle to destabilize, and possibly even crash. In this paper, we describe how a strategic adversary might be able to introduce spurious IMU values that can deviate a vehicle from its mission-specified path while at the same time evade customary anomaly detection mechanisms, thereby effectively perpetuating a “stealthy attack” on the system. We explore the feasibility of a Deep Neural Network (DNN) that uses a vehicle's state information to calculate the applicable IMU values to perpetrate such an attack. The eventual goal is to cause a vehicle to perturb enough from its mission parameters to compromise mission reliability, while, from the operator's perspective, the vehicle still appears to be operating normally. 

   more » « less
4. Hibernated Backdoor: A Mutual Information Empowered Backdoor Attack to Deep Neural Networks

   https://doi.org/10.1609/aaai.v36i9.21272  

   Ning, Rui; Li, Jiang; Xin, Chunsheng; Wu, Hongyi; Wang, Chonggang (June 2022, Proceedings of the AAAI Conference on Artificial Intelligence)

   We report a new neural backdoor attack, named Hibernated Backdoor, which is stealthy, aggressive and devastating. The backdoor is planted in a hibernated mode to avoid being detected. Once deployed and fine-tuned on end-devices, the hibernated backdoor turns into the active state that can be exploited by the attacker. To the best of our knowledge, this is the first hibernated neural backdoor attack. It is achieved by maximizing the mutual information (MI) between the gradients of regular and malicious data on the model. We introduce a practical algorithm to achieve MI maximization to effectively plant the hibernated backdoor. To evade adaptive defenses, we further develop a targeted hibernated backdoor, which can only be activated by specific data samples and thus achieves a higher degree of stealthiness. We show the hibernated backdoor is robust and cannot be removed by existing backdoor removal schemes. It has been fully tested on four datasets with two neural network architectures, compared to five existing backdoor attacks, and evaluated using seven backdoor detection schemes. The experiments demonstrate the effectiveness of the hibernated backdoor attack under various settings. 

   more » « less
5. On the Exploitability of Instruction Tuning

   Shu, Manli; Wang, Jiongxiao; Zhu, Chen; Geiping, Jonas; Xiao, Chaowei; Goldstein, Tom (September 2023, Thirty-seventh Conference on Neural Information Processing Systems)

   Instruction tuning is an effective technique to align large language models (LLMs) with human intent. In this work, we investigate how an adversary can exploit instruction tuning by injecting specific instruction-following examples into the training data that intentionally changes the model's behavior. For example, an adversary can achieve content injection by injecting training examples that mention target content and eliciting such behavior from downstream models. To achieve this goal, we propose AutoPoison, an automated data poisoning pipeline. It naturally and coherently incorporates versatile attack goals into poisoned data with the help of an oracle LLM. We showcase two example attacks: content injection and over-refusal attacks, each aiming to induce a specific exploitable behavior. We quantify and benchmark the strength and the stealthiness of our data poisoning scheme. Our results show that AutoPoison allows an adversary to change a model's behavior by poisoning only a small fraction of data while maintaining a high level of stealthiness in the poisoned examples. We hope our work sheds light on how data quality affects the behavior of instruction-tuned models and raises awareness of the importance of data quality for responsible deployments of LLMs. 

   more » « less