More Like this

1. Using Knowledge Graphs and Reinforcement Learning for Malware Analysis

   https://doi.org/10.1109/BigData50022.2020.9378491  

   Piplai, Aritran; Ranade, Priyanka; Kotal, Anantaa; Mittal, Sudip; Narayanan, Sandeep Nair; Joshi, Anupam (December 2020, 2020 IEEE International Conference on Big Data (Big Data))

   null (Ed.)

   Machine learning algorithms used to detect attacks are limited by the fact that they cannot incorporate the back-ground knowledge that an analyst has. This limits their suitability in detecting new attacks. Reinforcement learning is different from traditional machine learning algorithms used in the cybersecurity domain. Compared to traditional ML algorithms, reinforcement learning does not need a mapping of the input-output space or a specific user-defined metric to compare data points. This is important for the cybersecurity domain, especially for malware detection and mitigation, as not all problems have a single, known, correct answer. Often, security researchers have to resort to guided trial and error to understand the presence of a malware and mitigate it.In this paper, we incorporate prior knowledge, represented as Cybersecurity Knowledge Graphs (CKGs), to guide the exploration of an RL algorithm to detect malware. CKGs capture semantic relationships between cyber-entities, including that mined from open source. Instead of trying out random guesses and observing the change in the environment, we aim to take the help of verified knowledge about cyber-attack to guide our reinforcement learning algorithm to effectively identify ways to detect the presence of malicious filenames so that they can be deleted to mitigate a cyber-attack. We show that such a guided system outperforms a base RL system in detecting malware. 

   more » « less
2. Generating Fake Cyber Threat Intelligence Using Transformer-Based Models

   Ranade, P.; Piplai, A.; Mittal, S.; Joshi, A.; Finin, T (July 2021, International Joint Conference on Neural Networks 2021 (IJCNN 2021))

   Cyber-defense systems are being developed to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. A potential risk is that fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems. Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing the model to learn incorrect inputs to serve their malicious needs. In this paper, we automatically generate fake CTI text descriptions using transformers. We show that given an initial prompt sentence, a public language model like GPT-2 with fine-tuning, can generate plausible CTI text with the ability of corrupting cyber-defense systems. We utilize the generated fake CTI text to perform a data poisoning attack on a Cybersecurity Knowledge Graph (CKG) and a cybersecurity corpus. The poisoning attack introduced adverse impacts such as returning incorrect reasoning outputs, representation poisoning, and corruption of other dependent AI-based cyber defense systems. We evaluate with traditional approaches and conduct a human evaluation study with cybersecurity professionals and threat hunters. Based on the study, professional threat hunters were equally likely to consider our fake generated CTI as true. 

   more » « less
3. Optimal Attack and Defense for Reinforcement Learning

   https://doi.org/10.1609/aaai.v38i13.29346  

   McMahan, Jeremy; Wu, Young; Zhu, Xiaojin; Xie, Qiaomin (March 2024, Proceedings of the AAAI Conference on Artificial Intelligence)

   To ensure the usefulness of Reinforcement Learning (RL) in real systems, it is crucial to ensure they are robust to noise and adversarial attacks. In adversarial RL, an external attacker has the power to manipulate the victim agent's interaction with the environment. We study the full class of online manipulation attacks, which include (i) state attacks, (ii) observation attacks (which are a generalization of perceived-state attacks), (iii) action attacks, and (iv) reward attacks. We show the attacker's problem of designing a stealthy attack that maximizes its own expected reward, which often corresponds to minimizing the victim's value, is captured by a Markov Decision Process (MDP) that we call a meta-MDP since it is not the true environment but a higher level environment induced by the attacked interaction. We show that the attacker can derive optimal attacks by planning in polynomial time or learning with polynomial sample complexity using standard RL techniques. We argue that the optimal defense policy for the victim can be computed as the solution to a stochastic Stackelberg game, which can be further simplified into a partially-observable turn-based stochastic game (POTBSG). Neither the attacker nor the victim would benefit from deviating from their respective optimal policies, thus such solutions are truly robust. Although the defense problem is NP-hard, we show that optimal Markovian defenses can be computed (learned) in polynomial time (sample complexity) in many scenarios. 

   more » « less
4. Quantitative Measurement of Cyber Resilience: Modeling and Experimentation

   https://doi.org/10.1145/3703159  

   Weisman, Michael J; Kott, Alexander; Ellis, Jason E; Murphy, Brian J; Parker, Travis W; Smith, Sidney; Vandekerckhove, Joachim (January 2025, ACM Transactions on Cyber-Physical Systems)

   Cyber resilience is the ability of a system to resist and recover from a cyber attack, thereby restoring the system’s functionality. Effective design and development of a cyber resilient system requires experimental methods and tools for quantitative measuring of cyber resilience. This article describes an experimental method and test bed for obtaining resilience-relevant data as a system (in our case—a truck) traverses its route, in repeatable, systematic experiments. We model a truck equipped with an autonomous cyber-defense system and which also includes inherent physical resilience features. When attacked by malware, this ensemble of cyber-physical features (i.e., “bonware”) strives to resist and recover from the performance degradation caused by the malware’s attack. We propose parsimonious mathematical models to aid in quantifying systems’ resilience to cyber attacks. Using the models, we identify quantitative characteristics obtainable from experimental data and show that these characteristics can serve as useful quantitative measures of cyber resilience. 

   more » « less
5. Curriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading

   https://doi.org/10.1109/TPWRS.2022.3213487  

   Matavalam, Amarsagar Reddy; Guddanti, Kishan Prudhvi; Weng, Yang; Ajjarapu, Venkataramana (October 2022, IEEE Transactions on Power Systems)

   This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning (RL) frameworks to effectively learn agents that control the grid's topology to prevent thermal cascading. Typical RL-based topology controllers fail to perform well due to the large search/optimization space. Here, we propose an actor-critic-based agent to address the problem's combinatorial nature and train the agent using the RL environment developed by RTE, the French TSO. To address the challenge of the large optimization space, a curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment using network physics for enhanced agent learning. Further, a parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations. Without these modifications to the training procedure, the RL agent failed for most test scenarios, illustrating the importance of properly integrating domain knowledge of physical systems for real-world RL learning. The agent was tested by RTE for the 2019 learning to run the power network challenge and was awarded the 2nd place in accuracy and 1st place in speed. The developed code is open-sourced for public use. Analysis of a simple system proves the enhancement in training RL-agents using the curriculum. 

   more » « less