An official website of the United States government
Infrastructure as code (IaC) scripts, such as Ansible scripts, are used to provision computing infrastructure at scale. Existence of bugs in IaC test scripts, such as, configuration and security bugs, can be consequential for the provisioned computing infrastructure. A characterization study of bugs in IaC test scripts is the first step to understand the quality concerns that arise during testing of IaC scripts, and also provide recommendations for practitioners on quality assurance. We conduct an empirical study with 4,831 Ansible test scripts mined from 104 open source software (OSS) repositories where we quantify bug frequency, and categorize bugs in test scripts. We further categorize testing patterns, i.e., recurring coding patterns in test scripts, which also correlate with appearance of bugs. From our empirical study, we observe 1.8% of 4,831 Ansible test scripts to include a bug, and 45.2% of the 104 repositories to contain at least one test script that includes bugs. We identify 7 categories of bugs, which includes security bugs and performance bugs that are related with metadata extraction. We also identify 3 testing patterns that correlate with appearance of bugs: 'assertion roulette’, 'local only testing’, and 'remote mystery guest‘. Based on our findings, we advocate for detection and mitigation of the 3 testing patterns as these patterns can have negative implications for troubleshooting failures, reproducible deployments of software, and provisioning of computing infrastructure.
more »
« less
Security Smells in Ansible and Chef Scripts: A Replication Study
Context: Security smells are recurring coding patterns that are indicative of security weakness and require further inspection. As infrastructure as code (IaC) scripts, such as Ansible and Chef scripts, are used to provision cloud-based servers and systems at scale, security smells in IaC scripts could be used to enable malicious users to exploit vulnerabilities in the provisioned systems. Goal: The goal of this article is to help practitioners avoid insecure coding practices while developing infrastructure as code scripts through an empirical study of security smells in Ansible and Chef scripts. Methodology: We conduct a replication study where we apply qualitative analysis with 1,956 IaC scripts to identify security smells for IaC scripts written in two languages: Ansible and Chef. We construct a static analysis tool called Security Linter for Ansible and Chef scripts (SLAC) to automatically identify security smells in 50,323 scripts collected from 813 open source software repositories. We also submit bug reports for 1,000 randomly selected smell occurrences. Results: We identify two security smells not reported in prior work: missing default in case statement and no integrity check. By applying SLAC we identify 46,600 occurrences of security smells that include 7,849 hard-coded passwords. We observe agreement for 65 of the responded 94 bug reports, which suggests the relevance of security smells for Ansible and Chef scripts amongst practitioners. Conclusion: We observe security smells to be prevalent in Ansible and Chef scripts, similarly to that of the Puppet scripts. We recommend practitioners to rigorously inspect the presence of the identified security smells in Ansible and Chef scripts using (i) code review, and (ii) static analysis tools.
more »
« less
- Award ID(s):
- 2026869
- PAR ID:
- 10273685
- Date Published:
- Journal Name:
- ACM Transactions on Software Engineering and Methodology
- Volume:
- 30
- Issue:
- 1
- ISSN:
- 1049-331X
- Page Range / eLocation ID:
- 1 to 31
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Feldt, Robert; Zimmermann, Thomas (Ed.)Context Despite being beneficial for managing computing infrastructure at scale, Ansible scripts include security weaknesses, such as hard-coded passwords. Security weaknesses can propagate into tasks, i.e., code constructs used for managing computing infrastructure with Ansible. Propagation of security weaknesses into tasks makes the provisioned infrastructure susceptible to security attacks. A systematic characterization of task infection, i.e., the propagation of security weaknesses into tasks, can aid practitioners and researchers in understanding how security weaknesses propagate into tasks and derive insights for practitioners to develop Ansible scripts securely. Objective The goal of the paper is to help practitioners and researchers understand how Ansible-managed computing infrastructure is impacted by security weaknesses by conducting an empirical study of task infections in Ansible scripts. Method We conduct an empirical study where we quantify the frequency of task infections in Ansible scripts. Upon detection of task infections, we apply qualitative analysis to determine task infection categories. We also conduct a survey with 23 practitioners to determine the prevalence and severity of identified task infection categories. With logistic regression analysis, we identify development factors that correlate with presence of task infections. Results In all, we identify 1,805 task infections in 27,213 scripts. We identify six task infection categories: anti-virus, continuous integration, data storage, message broker, networking, and virtualization. From our survey, we observe tasks used to manage data storage infrastructure perceived to have the most severe consequences. We also find three development factors, namely age, minor contributors, and scatteredness to correlate with the presence of task infections. Conclusion Our empirical study shows computing infrastructure managed by Ansible scripts to be impacted by security weaknesses. We conclude the paper by discussing the implications of our findings for practitioners and researchers.more » « less
-
The practice of infrastructure as code (IaC) recommends automated management of computing infrastructure with application of quality assurance, such as linting and testing. To that end, researchers recently have investigated quality concerns in IaC test manifests by deriving a catalog of test smells. The relevance of the identified smells need to be quantified by obtaining feedback from practitioners. Such feedback can help the IaC community understand if smells have relevance amongst practitioners, and derive future research directions. We survey 30 practitioners to assess the relevance of three Ansible test smell categories namely, assertion roulette, local only testing, and remote mystery guest. We observe local only testing to be the most agreed upon test smell category, whereas, assertion roulette is the least agreed upon test smell category. Our findings provide a nuanced perspective of test smells for IaC, and lays the groundwork for future research.more » « less
-
Infrastructure as Code (IaC) scripts, such as Puppet scripts, provide practitioners the opportunity to provision computing infrastructure automatically at scale. Poorly written IaC scripts impact various facets of quality (such as security and maintainability) and, in turn, may lead to serious consequences. Many of the ill-effects can be avoided or rectified easily by following recommendations derived from research and best practices gleaned from experience. While researchers have investigated methods to improve quality aspects of Puppet scripts, such research needs to be summarized and synthesized for industry practitioners. In this article, we summarize recent research in the IaC domain by discussing key quality issues, specifically security and maintainability smells, that may arise in an IaC script. We also mine open-source repositories from three organizations (Mozilla, Openstack, and Wikimedia) and report our observations on the identified smells. Furthermore, we also synthesize recommendations from the literature for software practitioners that could improve the quality of IaC scripts. Software development teams dealing with large computing infrastructure can get benefited from the actionable recommended practices. In addition, researchers in the domain may use this study to find opportunities to improve the state-of-the-art.more » « less
-
Despite being beneficial in automated provisioning of computing infrastructure at scale, infrastructure as code (IaC) scripts are susceptible to containing secrets, such as hard-coded passwords. A derivation of practices related to secret management for IaC can help practitioners to secure their secrets, potentially aiding them to securely develop IaC scripts. The goal of the paper is to help practitioners in secure development of infrastructure as code (IaC) scripts by identifying practices for secret management in IaC. We conduct a grey literature review with 38 Internet artifacts to identify 12 practices. We identify practices that are applicable for all IaC languages, e.g., prioritized encryption, as well as language-specific practices, such as state separation for Terraform. Our findings can be beneficial for (i) practitioners who can apply the identified practices to secure secrets in IaC development, and (ii) researchers who can investigate how the secret management process can be improved to facilitate secure development of IaC scripts.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1145/3408897
