Software depends on upstream projects that regularly fix vulnerabilities, but the documentation of those vulnerabilities is often unreliable or unavailable. Automating the collection of existing vulnerability fixes is essential for downstream projects to reliably update their dependencies due to the sheer number of dependencies in modern software. Prior efforts rely solely on incomplete databases or imprecise or inaccurate statistical analysis of upstream repositories. In this paper, we introduce Differential Alert Analysis (DAA) to discover vulnerability fixes in software projects. In contrast to statistical analysis, DAA leverages static analysis security testing (SAST) tools, which reason over code context and semantics. We provide a language-independent implementation of DAA and show that for Python and Java based projects, DAA has high precision for a ground-truth dataset of vulnerability fixes — even with noisy and low-precision SAST tools. We then use DAA in two large-scale empirical studies covering several prominent ecosystems, finding hundreds of resolved alerts, including many never publicly disclosed. DAA thus provides a powerful, accurate primitive for software projects, code analysis tools, vulnerability databases, and researchers to characterize and enhance the security of software supply chains.
more »
« less
Vulnerability coverage for adequacy security testing
Mainstream software applications and tools are the configurable platforms with an enormous number of parameters along with their values. Certain settings and possible interactions between these parameters may harden (or soften) the security and robustness of these applications against some known vulnerabilities. However, the large number of vulnerabilities reported and associated with these tools make the exhaustive testing of these tools infeasible against these vulnerabilities infeasible. As an instance of general software testing problem, the research question to address is whether the system under test is robust and secure against these vulnerabilities. This paper introduces the idea of "vulnerability coverage," a concept to adequately test a given application for a certain classes of vulnerabilities, as reported by the National Vulnerability Database (NVD). The deriving idea is to utilize the Common Vulnerability Scoring System (CVSS) as a means to measure the fitness of test inputs generated by evolutionary algorithms and then through pattern matching identify vulnerabilities that match the generated vulnerability vectors and then test the system under test for those identified vulnerabilities. We report the performance of two evolutionary algorithms (i.e., Genetic Algorithms and Particle Swarm Optimization) in generating the vulnerability pattern vectors.
more »
« less
- PAR ID:
- 10186624
- Date Published:
- Journal Name:
- SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing
- Page Range / eLocation ID:
- 540 to 543
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Symbolic execution is a popular software testing technique that can systematically examine program code to find bugs. Owing to the prevalence of software vulnerabilities, symbolic execution has been extensively used to detect software vulnerabilities. A major challenge of using symbolic execution to find complicated vulnerabilities such as use-after-free is to not only direct symbolic execution to explore relevant program paths but also explore the paths in a specific order. In this paper, we describe a targeted symbolic execution, called UAFDetect, for finding use-after-free vulnerabilities efficiently. UAFDetect guides symbolic execution to focus on paths that are likely to cause a use-after-free by pruning paths that are unlikely or infeasible to cause that. It uses dynamic typestate analysis to identify unlikely paths and static control flow analysis to identify infeasible paths. UAFDetect performs typestate analysis to detect the occurrence of use-after-free vulnerabilities. Upon the detection of a vulnerability, UAFDetect generates an exploit to trigger the vulnerability. We develop the prototype of UAFDetect and evaluate it on real-world use-after-free vulnerabilities. The evaluation demonstrates that UAFDetect can find use-after-free vulnerabilities effectively and efficiently.more » « less
-
Consistent growth in the software sector of the world economies has attracted both targeted and mass-scale attacks by cybercriminals. Producing reliable and secure software is difficult because of its growing complexity and the increasing number of sophisticated attacks. Developers can’t afford to believe that their security measures during development are perfect and impenetrable. In fact, many new software security vulnerabilities are discovered on a daily basis. Therefore, it is vital to identify and resolve those security vulnerabilities as early as possible. Security Vulnerability Testing (SVT), as an active defense, is the key to the agile detection and prevention of known and unknown security vulnerabilities. However, many software engineers lack the awareness of the importance of security vulnerability and the necessary knowledge and skills at the testing and operational stages. As a first step towards filling this gap, this paper advocates for building skills in selecting proper benchmarks for the assessment of SVT tools to enable distinguishing valuable security tools from trivial ones. Thus, we provide a set of requirements in fulfillment of this need, primarily addressing newcomers and researcher to the discipline.more » « less
-
Context: Applying vulnerability detection techniques is one of many tasks using the limited resources of a software project. Objective: The goal of this research is to assist managers and other decision- makers in making informed choices about the use of software vulnerability detection techniques through an empirical study of the efficiency and effectiveness of four techniques on a Java-based web application. Method: We apply four different categories of vulnerability detection techniques – systematic manual penetration testing (SMPT), exploratory manual penetration testing (EMPT), dynamic application security testing (DAST), and static application security testing (SAST) – to an open-source medical records system. Results: We found the most vulnerabilities using SAST. However, EMPT found more severe vulnerabilities. With each technique, we found unique vulnerabilities not found using the other techniques. The efficiency of manual techniques (EMPT, SMPT) was comparable to or better than the efficiency of automated techniques (DAST, SAST) in terms of Vulnerabilities per Hour (VpH). Conclusions: The vulnerability detection technique practitioners should select may vary based on the goals and available resources of the project. If the goal of an organization is to find “all” vulnerabilities in a project, they need to use as many techniques as their resources allow.more » « less
-
As the number and severity of security incidents continue to increase, remediating vulnerabilities and weaknesses has become a daunting task due to the sheer number of known vulnerabilities. Different scoring systems have been developed to provide qualitative and quantitative assessments of the severity of common vulnerabilities and weaknesses, and guide the prioritization of vulnerability remediation. However, these scoring systems provide only generic rankings of common weaknesses, which do not consider the specific vulnerabilities that exist in each system. To address this limitation, and building on recent principled approaches to vulnerability scoring, we propose new common weakness scoring metrics that consider the findings of vulnerability scanners, including the number of instances of each vulnerability across a system, and enable system-specific rankings that can provide actionable intelligence to security administrators. We built a small testbed to evaluate the proposed metrics against an existing metric, and show that the results are consistent with our intuition.more » « less