An official website of the United States government
Metrics available to practitioners for determining the security of open-source software packages often overlook potential security issues, such as latent vulnerabilities, that may arise in the package’s dependencies. However, security risks in the dependencies of a package can still increase the risk to the package itself. The goal of this work is to aid developers and software architects in making good component choices by analyzing the relevance and utility of security measures of the entire dependency tree associated with a package. We propose a set of approaches for aggregating security measures from OpenSSF Scorecard that include information from the dependency tree of a package (aggregate scores). We evaluated the aggregate scores in two ways. For one, we performed statistical analysis on packages in the Go ecosystem. We determined the effect size of aggregation and assessed whether aggregation influences the measure’s relationship with vulnerability count, a common security risk measure. The other evaluation was a survey to see if practitioners would prefer metrics that are aggregated compared to metrics that focus only on the root package. Our results indicate that for some security risk measures, such as the number of Binary Artifacts in a package, aggregation has a negligible statistical effect. However, for the same Binary Artifacts measure, most survey participants preferred to have aggregate scores. In contrast, most respondents were less concerned about whether maintainers of the package’s dependencies performed Code Review, as long as maintainers performed Code Review on the root package.
more »
« less
Do Software Security Practices Yield Fewer Vulnerabilities?
Due to the ever-increasing number of security breaches, practitioners are motivated to produce more secure software. In the United States, the White House Office released a memorandum on Executive Order (EO) 14028 that mandates organizations provide self-attestation of the use of secure software development practices. The OpenSSF Scorecard project allows practitioners to measure the use of software security practices automatically. However, little research has been done to determine whether the use of security practices improves package security, particularly which security practices have the biggest impact on security outcomes. The goal of this study is to assist practitioners and researchers in making informed decisions on which security practices to adopt through the development of models between software security practice scores and security vulnerability counts. To that end, we developed five supervised machine learning models for npm and PyPI packages using the OpenSSF Scorecard security practices scores and aggregate security scores as predictors and the number of externally-reported vulnerabilities as a target variable. Our models found that four security practices (Maintained, Code Review, Branch Protection, and Security Policy) were the most important practices influencing vulnerability count. However, we had low R2 (ranging from 9% to 12%) when we tested the models to predict vulnerability counts. Additionally, we observed that the number of reported vulnerabilities increased rather than reduced as the aggregate security score of the packages increased. Both findings indicate that additional factors may influence the package vulnerability count. Other factors, such as the scarcity of vulnerability data, time to implicate security practices vs. time to detect vulnerabilities, and the need for more adequate scripts to detect security practices, may impede the data-driven studies to indicate that practice can aid in reducing externally-reported vulnerabilities. We suggest that vulnerability count and security score data be refined so that these measures can be used to provide actionable guidance on security practices.
more »
« less
- Award ID(s):
- 2207008
- PAR ID:
- 10516081
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 979-8-3503-0037-6
- Page Range / eLocation ID:
- 292 to 303
- Format(s):
- Medium: X
- Location:
- Melbourne, Australia
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Security vulnerabilities in an application open the ways to security dangers and attacks, which can easily jeopardize the system executing that application. Therefore, it is important to develop vulnerability-free applications. The best approach would be to counteract against potential vulnerabilities during the coding with secure programming practices. Software security proactive control education for secure portable and web application advancement is of enormous interests in the Information Technology (IT) fields. In this paper, we proposed and developed innovative learning modules for software security proactive control based on several real-world scenarios to broaden and promote proactive control for secure software development in computing education.more » « less
-
Mainstream software applications and tools are the configurable platforms with an enormous number of parameters along with their values. Certain settings and possible interactions between these parameters may harden (or soften) the security and robustness of these applications against some known vulnerabilities. However, the large number of vulnerabilities reported and associated with these tools make the exhaustive testing of these tools infeasible against these vulnerabilities infeasible. As an instance of general software testing problem, the research question to address is whether the system under test is robust and secure against these vulnerabilities. This paper introduces the idea of "vulnerability coverage," a concept to adequately test a given application for a certain classes of vulnerabilities, as reported by the National Vulnerability Database (NVD). The deriving idea is to utilize the Common Vulnerability Scoring System (CVSS) as a means to measure the fitness of test inputs generated by evolutionary algorithms and then through pattern matching identify vulnerabilities that match the generated vulnerability vectors and then test the system under test for those identified vulnerabilities. We report the performance of two evolutionary algorithms (i.e., Genetic Algorithms and Particle Swarm Optimization) in generating the vulnerability pattern vectors.more » « less
-
null (Ed.)The ubiquitous usage of robots in modern society necessitates secure development of robotics systems. Practitioners who engage in robot development can benefit from a systematic study that investigates the categories of vulnerabilities that appear in robotics systems. The goal of this paper is to help practitioners mitigate vulnerabilities in robotics systems by conducting an empirical study of vulnerabilities in robotics systems. We conduct an empirical study where we analyze 176 robotics-related vulnerabilities collected from the Robot Vulnerability Database (RVD). Our findings show that: (i) robotics-related vulnerabilities can be classified into nine categories; (ii) memory-related vulnerabilities are the most frequent category, (iii) 92.6% of the reported vulnerabilities are software-related, and (iv) software components in robotics systems include more critical vulnerabilities compared to that of hardware components. Based on our findings, we provide a list of development activities that can be used to mitigate vulnerabilities for robotics systems.more » « less
