An official website of the United States government
MITRE ATT&CK is an open-source taxonomy of adversary tactics, techniques, and procedures based on real-world observations. Increasingly, organizations leverage ATT&CK technique "coverage" as the basis for evaluating their security posture, while Endpoint Detection and Response (EDR) and Security Indicator and Event Management (SIEM) products integrate ATT&CK into their design as well as marketing. However, the extent to which ATT&CK coverage is suitable to serve as a security metric remains unclear— Does ATT&CK coverage vary meaningfully across different products? Is it possible to achieve total coverage of ATT&CK? Do endpoint products that detect the same attack behaviors even claim to cover the same ATT&CK techniques? In this work, we attempt to answer these questions by conducting a comprehensive (and, to our knowledge, the first) analysis of endpoint detection products' use of MITRE ATT&CK. We begin by evaluating 3 ATT&CK-annotated detection rulesets from major commercial providers (Carbon Black, Splunk, Elastic) and a crowdsourced ruleset (Sigma) to identify commonalities and underutilized regions of the ATT&CK matrix. We continue by performing a qualitative analysis of unimplemented ATT&CK techniques to determine their feasibility as detection rules. Finally, we perform a consistency analysis of ATT&CK labeling by examining 37 specific threat entities for which at least 2 products include specific detection rules. Combined, our findings highlight the limitations of overdepending on ATT&CK coverage when evaluating security posture; most notably, many techniques are unrealizable as detection rules, and coverage of an ATT&CK technique does not consistently imply coverage of the same real-world threats.
more »
« less
Examining Penetration Tester Behavior in the Collegiate Penetration Testing Competition
Penetration testing is a key practice toward engineering secure software. Malicious actors have many tactics at their disposal, and software engineers need to know what tactics attackers will prioritize in the first few hours of an attack. Projects like MITRE ATT&CK™ provide knowledge, but how do people actually deploy this knowledge in real situations? A penetration testing competition provides a realistic, controlled environment with which to measure and compare the efficacy of attackers. In this work, we examine the details of vulnerability discovery and attacker behavior with the goal of improving existing vulnerability assessment processes using data from the 2019 Collegiate Penetration Testing Competition (CPTC). We constructed 98 timelines of vulnerability discovery and exploits for 37 unique vulnerabilities discovered by 10 teams of penetration testers. We grouped related vulnerabilities together by mapping to Common Weakness Enumerations and MITRE ATT&CK™. We found that (1) vulnerabilities related to improper resource control (e.g., session fixation) are discovered faster and more often, as well as exploited faster, than vulnerabilities related to improper access control (e.g., weak password requirements), (2) there is a clear process followed by penetration testers of discovery/collection to lateral movement/pre-attack. Our methodology facilitates quicker analysis of vulnerabilities in future CPTC events.
more »
« less
- Award ID(s):
- 1922169
- PAR ID:
- 10401365
- Date Published:
- Journal Name:
- ACM Transactions on Software Engineering and Methodology
- Volume:
- 31
- Issue:
- 3
- ISSN:
- 1049-331X
- Page Range / eLocation ID:
- 1 to 25
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Industrial control systems (ICS) are systems used in critical infrastructures for supervisory control, data acquisition, and industrial automation. ICS systems have complex, component-based architectures with many different hardware, software, and human factors interacting in real time. Despite the importance of security concerns in industrial control systems, there has not been a comprehensive study that examined common security architectural weaknesses in this domain. Therefore, this paper presents the first in-depth analysis of 988 vulnerability advisory reports for Industrial Control Systems developed by 277 vendors. We performed a detailed analysis of the vulnerability reports to measure which components of ICS have been affected the most by known vulnerabilities, which security tactics were affected most often in ICS and what are the common architectural security weaknesses in these systems. Our key findings were: (1) Human-Machine Interfaces, SCADA configurations, and PLCs were the most affected components, (2) 62.86% of vulnerability disclosures in ICS had an architectural root cause, (3) the most common architectural weaknesses were “Improper Input Validation”, followed by “Im-proper Neutralization of Input During Web Page Generation” and “Improper Authentication”, and (4) most tactic-related vulnerabilities were related to the tactics “Validate Inputs”, “Authenticate Actors” and “Authorize Actors”.more » « less
-
As a new format of mobile application, mini-programs, which function within a larger app and are built with HTML, CSS, and JavaScript web technology, have become the way to do almost everything in China. Many researchers have done the ecosystem or developing study, while the permission problem has not been investigated yet. In this paper, we present our studies on the permission management of mini-programs and conduct a systematic study on 9 popular mobile host app ecosystems that host over 7 million mini-programs. After testing over 2,580 APIs, we extracted a common abstract model for mini-programs’ permission control and revealed six categories of potential security vulnerabilities due to improper permission management. It is alarming that the current popular mobile app ecosystems (i.e., host apps) under study have at least one security vulnerability due to the mini-programs’ improper permission management. We present the corresponding attack methods to dissect these potential weaknesses further to exploit the discovered vulnerabilities. To prove that the revealed vulnerabilities may cause severe consequences in real-world use, we show three kinds of attacks without privileges or cracking the host apps. We have responsibly disclosed the newly discovered vulnerabilities, and two CVEs were issued. Finally, we put forward systematic suggestions to strengthen the standardization of mini-programs.more » « less
-
In the wake of a cybersecurity incident, it is crucial to promptly discover how the threat actors breached security in order to assess the impact of the incident and to develop and deploy countermeasures that can protect against further attacks. To this end, defenders can launch a cyber-forensic investigation, which discovers the techniques that the threat actors used in the incident. A fundamental challenge in such an investigation is prioritizing the investigation of particular techniques since the investigation of each technique requires time and effort, but forensic analysts cannot know which ones were actually used before investigating them. To ensure prompt discovery, it is imperative to provide decision support that can help forensic analysts with this prioritization. A recent study demonstrated that data-driven decision support, based on a dataset of prior incidents, can provide state-of-the-art prioritization. However, this data-driven approach, called DISCLOSE, is based on a heuristic that utilizes only a subset of the available information and does not approximate optimal decisions. To improve upon this heuristic, we introduce a principled approach for data-driven decision support for cyber-forensic investigations. We formulate the decision-support problem using a Markov decision process, whose states represent the states of a forensic investigation. To solve the decision problem, we propose a Monte Carlo tree search based method, which relies on a k-NN regression over prior incidents to estimate state-transition probabilities. We evaluate our proposed approach on multiple versions of the MITRE ATT&CK dataset, which is a knowledge base of adversarial techniques and tactics based on real-world cyber incidents, and demonstrate that our approach outperforms DISCLOSE in terms of techniques discovered per effort spent.more » « less
-
The ecosystem for automated offensive security tools has grown in recent years. As more tools automate offensive security techniques via Artificial Intelligence (AI) and Machine Learning (ML), it may result in vulnerabilities due to adversarial attacks. Therefore, it is imperative that research is conducted to help understand the techniques used by these security tools. Our work explores the current state of the art in offensive security tools. First, we employ an abstract model that can be used to understand what phases of an Offensive Cyber Operation (OCO) can be automated. We then adopt a generalizable taxonomy, and apply it to automation tools (such as normal automation and the use of artificial intelligence in automation). We then curated a dataset of tools and research papers and quantitatively analyzed it. Our work resulted in a public dataset that includes analysis of (n=57) papers and OCO tools that are mapped to the the MITRE ATT&CK Framework enterprise techniques, applicable phases of our OCO model, and the details of the automation technique. The results show a need for a granular expansion on the ATT&CK Exploit Public-Facing application technique. A critical finding is that most OCO tools employed Simple Rule Based automation, hinting at a lucrative research opportunity for the use of Artificial Intelligence (AI) and Machine Learning (ML) in future OCO tooling.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1145/3514040
