Search for: All records

Abstract ContextPractitioners prefer to achieve performance without sacrificing productivity when developing scientific software. The Julia programming language is designed to develop performant computer programs without sacrificing productivity by providing a syntax that is scripting in nature. According to the Julia programming language website, the common projects are data science, machine learning, scientific domains, and parallel computing. While Julia has yielded benefits with respect to productivity, programs written in Julia can include security weaknesses, which can hamper the security of Julia-based scientific software. A systematic derivation of security weaknesses can facilitate secure development of Julia programs—an area that remains under-explored. ObjectiveThe goal of this paper is to help practitioners securely develop Julia programs by conducting an empirical study of security weaknesses found in Julia programs. MethodWe apply qualitative analysis on 4,592 Julia programs used in 126 open-source Julia projects to identify security weakness categories. Next, we construct a static analysis tool calledJuliaStaticAnalysisTool (JSAT) that automatically identifies security weaknesses in Julia programs. We apply JSAT to automatically identify security weaknesses in 558 open-source Julia projects consisting of 25,008 Julia programs. ResultsWe identify 7 security weakness categories, which include the usage of hard-coded password and unsafe invocation. From our empirical study we identify 23,839 security weaknesses. On average, we observe 24.9% Julia source code files to include at least one of the 7 security weakness categories. ConclusionBased on our research findings, we recommend rigorous inspection efforts during code reviews. We also recommend further development and application of security static analysis tools so that security weaknesses in Julia programs can be detected before execution. 

While Kubernetes enables practitioners to rapidly deploy their software and perform container orchestration efficiently, security of the Kubernetes-based deployment infrastructure is a concern for industry practitioners. A systematic understanding of how dynamic analysis can be used for securing Kubernetes deployments can aid practitioners in securing their Kubernetes deployments. We present an experience report, where we describe empirical findings from three dynamic application security testing (DAST) tools on a Kubernetes deployment used by 'Company-Z'. From our empirical study, we find (i) 3,442 recommended security configurations are violated in 'Company-Z's' Kubernetes deployment; and (ii) of the three studied DAST tools, Kubescape and Kubebench provide the highest support with respect to detecting 14 types of recommended security configurations. Based on our findings, we recommend practitioners to apply DAST tools for their Kubernetes deployments, and security researchers to investigate how to detect configuration violations dynamically in the Kubernetes deployment. 

Kubernetes is a popular open source software (OSS) tool to manage containers at scale. Despite being beneficial for rapid deployment, Kubernetes-based software deployments are susceptible to defects that can lead to serious consequences. A systematic analysis of development-related factors that cause defects can aid practitioners on how to mitigate these defects. We conduct an exploratory empirical study where we use causal analysis to quantify the impact of one development factor called minor contributors, which refers to practitioners who author < 5% of the total code. By analyzing 29,028 commits from 157 OSS repositories, we observe (i) 5.6% of the 29,028 commits to be authored by minor contributors; and (ii) authorship of minor contributors to impact defects in configuration scripts. Based on our findings, we recommend researchers to (1) further investigate the characteristics of minor contributors; and (2) identify other development-related factors that may have a causal impact on defects in Kubernetes configuration scripts. 

Machine learning (ML) deployment projects are used by practitioners to automatically deploy ML models. While ML deployment projects aid practitioners, security vulnerabilities in these projects can make ML deployment infrastructure susceptible to security attacks. A systematic characterization of vulnerabilities can aid in identifying activities to secure ML deployment projects used by practitioners. We conduct an empirical study with 149 vulnerabilities mined from 12 open source ML deployment projects to characterize vulnerabilities in ML deployment projects. From our empirical study, we (i) find 68 of the 149 vulnerabilities are critically or highly severe; (ii) derive 10 consequences of vulnerabilities, e.g., unauthorized access to trigger ML deployments; and (iii) observe established quality assurance activities, such as code review to be used in the ML deployment projects. We conclude our paper by providing a set of recommendations for practitioners and researchers. Dataset used for our paper is available online. 

Despite being beneficial for rapid delivery of software, Kubernetes deployments can be susceptible to security attacks, which can cause serious consequences. A systematic characterization of how community-prescribed security configurations, i.e., security configurations that are recommended by security experts, can aid practitioners to secure their Kubernetes deployments. To that end, we conduct an empirical study with 53 security configurations recommended by the Center for Internet Security (CIS), 20 survey respondents, and 544 configuration files obtained from the open source software (OSS) and proprietary domains. From our empirical study, we observe: (i) practitioners can be unaware of prescribed security configurations as 5% ~40% of the survey respondents are unfamiliar with 16 prescribed configurations; and (ii) for Company-A and OSS respectively, 18.0% and 17.9% of the configuration files include at least one violation of prescribed configurations. From our evaluation with 5 static application security testing (SAST) tools we find (i) only Kubescape to support all of the prescribed security configuration categories; (ii) the highest observed precision to be 0.41 and 0.43 respectively, for the Company-A and OSS datasets; and (iii) the highest observed recall to be respectively, 0.53 and 0.65 for the Company-A and OSS datasets. Our findings show a disconnect between what CIS experts recommend for Kubernetes-related configurations and what happens in practice. We conclude the paper by providing recommendations for practitioners and researchers. Dataset used for the paper is publicly available online. 

The replication package contains data and scripts used to generate results reported in the paper. The replication package does not contain any data from Company-A to abide by the non-disclose agreement signed between the authors and Company-A. 

Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5\% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures. 

In infrastructure as code (IaC), state reconciliation is the process of querying and comparing the infrastructure state prior to changing the infrastructure. As state reconciliation is pivotal to manage IaC-based computing infrastructure at scale, defects related to state reconciliation can create large-scale consequences. A categorization of state reconciliation defects, i.e., defects related to state reconciliation, can aid in understanding the nature of state reconciliation defects. We conduct an empirical study with 5,110 state reconciliation defects where we apply qualitative analysis to categorize state reconciliation defects. From the identified defect categories, we derive heuristics to design prompts for a large language model (LLM), which in turn are used for validation of state reconciliation. From our empirical study, we identify 8 categories of state reconciliation defects, amongst which 3 have not been reported for previously-studied software systems. The most frequently occurring defect category is inventory, i.e., the category of defects that occur when managing infrastructure inventory. Using an LLM with heuristics-based paragraph style prompts, we identify 9 previously unknown state reconciliation defects of which 7 have been accepted as valid defects, and 4 have already been fixed. Based on our findings, we conclude the paper by providing a set of recommendations for researchers and practitioners. 

Context:Compilers are the fundamental tools for software development. Thus, compiler defects can disrupt development productivity and propagate errors into developer-written software source code. Categorizing defects in compilers can inform practitioners and researchers about the existing defects in compilers and techniques that can be used to identify defects systematically. Objective:The goal of this paper is to help researchers understand the nature of defects in compilers by conducting a review of Internet artifacts and peer-reviewed publications that study defect characteristics of compilers. Methodology:We conduct a multi-vocal literature review (MLR) with 26 publications and 32 Internet artifacts to characterize compiler defects. Results:From our MLR, we identify 13 categories of defects, amongst which optimization defects have been the most reported defects in our artifacts publications. We observed 15 defect identification techniques tailored for compilers and no single technique identifying all observed defect categories. Conclusion:Our MLR lays the groundwork for practitioners and researchers to identify defects in compilers systematically.