skip to main content


Title: Do Software Security Practices Yield Fewer Vulnerabilities?
Due to the ever-increasing number of security breaches, practitioners are motivated to produce more secure software. In the United States, the White House Office released a memorandum on Executive Order (EO) 14028 that mandates organizations provide self-attestation of the use of secure software development practices. The OpenSSF Scorecard project allows practitioners to measure the use of software security practices automatically. However, little research has been done to determine whether the use of security practices improves package security, particularly which security practices have the biggest impact on security outcomes. The goal of this study is to assist practitioners and researchers in making informed decisions on which security practices to adopt through the development of models between software security practice scores and security vulnerability counts. To that end, we developed five supervised machine learning models for npm and PyPI packages using the OpenSSF Scorecard security practices scores and aggregate security scores as predictors and the number of externally-reported vulnerabilities as a target variable. Our models found that four security practices (Maintained, Code Review, Branch Protection, and Security Policy) were the most important practices influencing vulnerability count. However, we had low R2 (ranging from 9% to 12%) when we tested the models to predict vulnerability counts. Additionally, we observed that the number of reported vulnerabilities increased rather than reduced as the aggregate security score of the packages increased. Both findings indicate that additional factors may influence the package vulnerability count. Other factors, such as the scarcity of vulnerability data, time to implicate security practices vs. time to detect vulnerabilities, and the need for more adequate scripts to detect security practices, may impede the data-driven studies to indicate that practice can aid in reducing externally-reported vulnerabilities. We suggest that vulnerability count and security score data be refined so that these measures can be used to provide actionable guidance on security practices.  more » « less
Award ID(s):
2207008
NSF-PAR ID:
10516081
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
IEEE
Date Published:
ISBN:
979-8-3503-0037-6
Page Range / eLocation ID:
292 to 303
Format(s):
Medium: X
Location:
Melbourne, Australia
Sponsoring Org:
National Science Foundation
More Like this
  1. Security vulnerabilities in an application open the ways to security dangers and attacks, which can easily jeopardize the system executing that application. Therefore, it is important to develop vulnerability-free applications. The best approach would be to counteract against potential vulnerabilities during the coding with secure programming practices. Software security proactive control education for secure portable and web application advancement is of enormous interests in the Information Technology (IT) fields. In this paper, we proposed and developed innovative learning modules for software security proactive control based on several real-world scenarios to broaden and promote proactive control for secure software development in computing education. 
    more » « less
  2. Mainstream software applications and tools are the configurable platforms with an enormous number of parameters along with their values. Certain settings and possible interactions between these parameters may harden (or soften) the security and robustness of these applications against some known vulnerabilities. However, the large number of vulnerabilities reported and associated with these tools make the exhaustive testing of these tools infeasible against these vulnerabilities infeasible. As an instance of general software testing problem, the research question to address is whether the system under test is robust and secure against these vulnerabilities. This paper introduces the idea of "vulnerability coverage," a concept to adequately test a given application for a certain classes of vulnerabilities, as reported by the National Vulnerability Database (NVD). The deriving idea is to utilize the Common Vulnerability Scoring System (CVSS) as a means to measure the fitness of test inputs generated by evolutionary algorithms and then through pattern matching identify vulnerabilities that match the generated vulnerability vectors and then test the system under test for those identified vulnerabilities. We report the performance of two evolutionary algorithms (i.e., Genetic Algorithms and Particle Swarm Optimization) in generating the vulnerability pattern vectors. 
    more » « less
  3. null (Ed.)
    The ubiquitous usage of robots in modern society necessitates secure development of robotics systems. Practitioners who engage in robot development can benefit from a systematic study that investigates the categories of vulnerabilities that appear in robotics systems. The goal of this paper is to help practitioners mitigate vulnerabilities in robotics systems by conducting an empirical study of vulnerabilities in robotics systems. We conduct an empirical study where we analyze 176 robotics-related vulnerabilities collected from the Robot Vulnerability Database (RVD). Our findings show that: (i) robotics-related vulnerabilities can be classified into nine categories; (ii) memory-related vulnerabilities are the most frequent category, (iii) 92.6% of the reported vulnerabilities are software-related, and (iv) software components in robotics systems include more critical vulnerabilities compared to that of hardware components. Based on our findings, we provide a list of development activities that can be used to mitigate vulnerabilities for robotics systems. 
    more » « less
  4. Consistent growth in the software sector of the world economies has attracted both targeted and mass-scale attacks by cybercriminals. Producing reliable and secure software is difficult because of its growing complexity and the increasing number of sophisticated attacks. Developers can’t afford to believe that their security measures during development are perfect and impenetrable. In fact, many new software security vulnerabilities are discovered on a daily basis. Therefore, it is vital to identify and resolve those security vulnerabilities as early as possible. Security Vulnerability Testing (SVT), as an active defense, is the key to the agile detection and prevention of known and unknown security vulnerabilities. However, many software engineers lack the awareness of the importance of security vulnerability and the necessary knowledge and skills at the testing and operational stages. As a first step towards filling this gap, this paper advocates for building skills in selecting proper benchmarks for the assessment of SVT tools to enable distinguishing valuable security tools from trivial ones. Thus, we provide a set of requirements in fulfillment of this need, primarily addressing newcomers and researcher to the discipline. 
    more » « less
  5. null (Ed.)
    Secure software development is a challenging task requiring consideration of many possible threats and mitigations.This paper investigates how and why programmers, despite a baseline of security experience, make security-relevant errors.To do this, we conducted an in-depth analysis of 94 submissions to a secure-programming contest designed to mimic real-world constraints: correctness, performance, and security.In addition to writing secure code, participants were asked to search for vulnerabilities in other teams’ programs; in total, teams submitted 866 exploits against the submissions we considered. Over an intensive six-month period, we used iterative open coding to manually, but systematically, characterize each submitted project and vulnerability (including vulnerabilities we identified ourselves). We labeled vulnerabilities by type, attacker control allowed, and ease of exploitation,and projects according to security implementation strategy.Several patterns emerged. For example, simple mistakes were least common: only 21% of projects introduced such an error.Conversely, vulnerabilities arising from a misunderstanding of security concepts were significantly more common, appearing in 78% of projects. Our results have implications for improving secure-programming APIs, API documentation,vulnerability-finding tools, and security education. 
    more » « less