An official website of the United States government
MITRE ATT&CK is an open-source taxonomy of adversary tactics, techniques, and procedures based on real-world observations. Increasingly, organizations leverage ATT&CK technique "coverage" as the basis for evaluating their security posture, while Endpoint Detection and Response (EDR) and Security Indicator and Event Management (SIEM) products integrate ATT&CK into their design as well as marketing. However, the extent to which ATT&CK coverage is suitable to serve as a security metric remains unclear— Does ATT&CK coverage vary meaningfully across different products? Is it possible to achieve total coverage of ATT&CK? Do endpoint products that detect the same attack behaviors even claim to cover the same ATT&CK techniques? In this work, we attempt to answer these questions by conducting a comprehensive (and, to our knowledge, the first) analysis of endpoint detection products' use of MITRE ATT&CK. We begin by evaluating 3 ATT&CK-annotated detection rulesets from major commercial providers (Carbon Black, Splunk, Elastic) and a crowdsourced ruleset (Sigma) to identify commonalities and underutilized regions of the ATT&CK matrix. We continue by performing a qualitative analysis of unimplemented ATT&CK techniques to determine their feasibility as detection rules. Finally, we perform a consistency analysis of ATT&CK labeling by examining 37 specific threat entities for which at least 2 products include specific detection rules. Combined, our findings highlight the limitations of overdepending on ATT&CK coverage when evaluating security posture; most notably, many techniques are unrealizable as detection rules, and coverage of an ATT&CK technique does not consistently imply coverage of the same real-world threats.
more »
« less
A Quantitative Analysis of Offensive Cyber Operation (OCO) Automation Tools
The ecosystem for automated offensive security tools has grown in recent years. As more tools automate offensive security techniques via Artificial Intelligence (AI) and Machine Learning (ML), it may result in vulnerabilities due to adversarial attacks. Therefore, it is imperative that research is conducted to help understand the techniques used by these security tools. Our work explores the current state of the art in offensive security tools. First, we employ an abstract model that can be used to understand what phases of an Offensive Cyber Operation (OCO) can be automated. We then adopt a generalizable taxonomy, and apply it to automation tools (such as normal automation and the use of artificial intelligence in automation). We then curated a dataset of tools and research papers and quantitatively analyzed it. Our work resulted in a public dataset that includes analysis of (n=57) papers and OCO tools that are mapped to the the MITRE ATT&CK Framework enterprise techniques, applicable phases of our OCO model, and the details of the automation technique. The results show a need for a granular expansion on the ATT&CK Exploit Public-Facing application technique. A critical finding is that most OCO tools employed Simple Rule Based automation, hinting at a lucrative research opportunity for the use of Artificial Intelligence (AI) and Machine Learning (ML) in future OCO tooling.
more »
« less
- Award ID(s):
- 1921813
- PAR ID:
- 10430173
- Date Published:
- Journal Name:
- ARES '22: Proceedings of the 17th International Conference on Availability, Reliability and Security
- Page Range / eLocation ID:
- 1 to 11
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This Innovative Practice Work-in-Progress paper presents a virtual, proactive, and collaborative learning paradigm that can engage learners with different backgrounds and enable effective retention and transfer of the multidisciplinary AI-cybersecurity knowledge. While progress has been made to better understand the trustworthiness and security of artificial intelligence (AI) techniques, little has been done to translate this knowledge to education and training. There is a critical need to foster a qualified cybersecurity workforce that understands the usefulness, limitations, and best practices of AI technologies in the cybersecurity domain. To address this import issue, in our proposed learning paradigm, we leverage multidisciplinary expertise in cybersecurity, AI, and statistics to systematically investigate two cohesive research and education goals. First, we develop an immersive learning environment that motivates the students to explore AI/machine learning (ML) development in the context of real-world cybersecurity scenarios by constructing learning models with tangible objects. Second, we design a proactive education paradigm with the use of hackathon activities based on game-based learning, lifelong learning, and social constructivism. The proposed paradigm will benefit a wide range of learners, especially underrepresented students. It will also help the general public understand the security implications of AI. In this paper, we describe our proposed learning paradigm and present our current progress of this ongoing research work. In the current stage, we focus on the first research and education goal and have been leveraging cost-effective Minecraft platform to develop an immersive learning environment where the learners are able to investigate the insights of the emerging AI/ML concepts by constructing related learning modules via interacting with tangible AI/ML building blocks.more » « less
-
Abstract In agriculture, important unanswered questions about machine learning and artificial intelligence (ML/AI) include will ML/AI change how food is produced and will ML algorithms replace or partially replace farmers in the decision process. As ML/AI technologies become more accurate, they have the potential to improve profitability while reducing the impact of agriculture on the environment. However, despite these benefits, there are many adoption barriers including cost, and that farmers may be reluctant to adopt a decision tool they do not understand. The goal of this special issue is to discuss cutting‐edge research on the use of ML/AI technologies in agriculture, barriers to the adoption of these technologies, and how technologies can affect our current workforce. The papers are separated into three sections: Machine Learning within Crops, Pasture, and Irrigation; Machine Learning in Predicting Crop Disease; and Society and Policy of Machine Learning.more » « less
-
De_Vita, R; Espinal, X; Laycock, P; Shadura, O (Ed.)Differentiable Programming could open even more doors in HEP analysis and computing to Artificial Intelligence/Machine Learning. Current common uses of AI/ML in HEP are deep learning networks – providing us with sophisticated ways of separating signal from background, classifying physics, etc. This is only one part of a full analysis – normally skims are made to reduce dataset sizes by applying selection cuts, further selection cuts are applied, perhaps new quantities calculated, and all of that is fed to a deep learning network. Only the deep learning network stage is optimized using the AI/ML gradient decent technique. Differentiable programming offers us a way to optimize the full chain, including selection cuts that occur during skimming. This contribution investigates applying selection cuts in front of a simple neural network using differentiable programming techniques to optimize the complete chain on toy data. There are several well-known problems that must be solved – e.g., selection cuts are not differentiable, and the interaction of a selection cut and a network during training is not well understood. This investigation was motived by trying to automate reduced dataset skims and sizes during analysis – HL-LHC analyses have potentially multi-TB dataset sizes and an automated way of reducing those dataset sizes and understanding the trade-offs would help the analyser make a judgement between time, resource usages, and physics accuracy. This contribution explores the various techniques to apply a selection cut that are compatible with differentiable programming and how to work around issues when it is bolted onto a neural network. Code is available.more » « less
-
As societies rely increasingly on computers for critical functions, the importance of cybersecurity becomes ever more paramount. Even in recent months there have been attacks that halted oil production, disrupted online learning at the height of COVID, and put medical records at risk at prominent hospitals. This constant threat of privacy leaks and infrastructure disruption has led to an increase in the adoption of artificial intelligence (AI) techniques, mainly machine learning (ML), in state-of-the-art cybersecurity approaches. Oftentimes, these techniques are borrowed from other disciplines without context and devoid of the depth of understanding as to why such techniques are best suited to solve the problem at hand. This is largely due to the fact that in many ways cybersecurity curricula have failed to keep up with advances in cybersecurity research and integrating AI and ML into cybersecurity curricula is extremely difficult. To address this gap, we propose a new methodology to integrate AI and ML techniques into cybersecurity education curricula. Our methodology consists of four components: i) Analysis of Literature which aims to understand the prevalence of AI and ML in cybersecurity research, ii) Analysis of Cybersecurity Curriculum that intends to determine the materials already present in the curriculum and the possible intersection points in the curricula for the new AI material, iii) Design of Adaptable Modules that aims to design highly adaptable modules that can be directly used by cybersecurity educators where new AI material can naturally supplement/substitute for concepts or material already present in the cybersecurity curriculum, and iv) Curriculum Level Evaluation that aims to evaluate the effectiveness of the proposed methodology from both student and instructor perspectives. In this paper, we focus on the first component of our methodology - Analysis of Literature and systematically analyze over 5000 papers that were published in the top cybersecurity conferences during the last five years. Our results clearly indicate that more than 78% of the cybersecurity papers mention AI terminology. To determine the prevalence of the use of AI, we randomly selected 300 papers and performed a thorough analysis. Our results show that more than 19% of the papers implement ML techniques. These findings suggest that AI and ML techniques should be considered for future integration into cybersecurity curriculum to better align with advancements in the field.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3538969.3544414
