More Like this

1. Come for syntax, stay for speed, write secure code: an empirical study of security weaknesses in Julia programs

   https://doi.org/10.1007/s10664-024-10606-w  

   Zhang, Yue; Murphy, Justin; Rahman, Akond (January 2025, Empirical Software Engineering)

   Abstract ContextPractitioners prefer to achieve performance without sacrificing productivity when developing scientific software. The Julia programming language is designed to develop performant computer programs without sacrificing productivity by providing a syntax that is scripting in nature. According to the Julia programming language website, the common projects are data science, machine learning, scientific domains, and parallel computing. While Julia has yielded benefits with respect to productivity, programs written in Julia can include security weaknesses, which can hamper the security of Julia-based scientific software. A systematic derivation of security weaknesses can facilitate secure development of Julia programs—an area that remains under-explored. ObjectiveThe goal of this paper is to help practitioners securely develop Julia programs by conducting an empirical study of security weaknesses found in Julia programs. MethodWe apply qualitative analysis on 4,592 Julia programs used in 126 open-source Julia projects to identify security weakness categories. Next, we construct a static analysis tool calledJuliaStaticAnalysisTool (JSAT) that automatically identifies security weaknesses in Julia programs. We apply JSAT to automatically identify security weaknesses in 558 open-source Julia projects consisting of 25,008 Julia programs. ResultsWe identify 7 security weakness categories, which include the usage of hard-coded password and unsafe invocation. From our empirical study we identify 23,839 security weaknesses. On average, we observe 24.9% Julia source code files to include at least one of the 7 security weakness categories. ConclusionBased on our research findings, we recommend rigorous inspection efforts during code reviews. We also recommend further development and application of security static analysis tools so that security weaknesses in Julia programs can be detected before execution. 

   more » « less
2. State Reconciliation Defects in Infrastructure as Code

   https://doi.org/10.1145/3660790  

   Hassan, Md Mahadi; Salvador, John; Santu, Shubhra_Kanti Karmaker; Rahman, Akond (July 2024, Proceedings of the ACM on Software Engineering)

   In infrastructure as code (IaC), state reconciliation is the process of querying and comparing the infrastructure state prior to changing the infrastructure. As state reconciliation is pivotal to manage IaC-based computing infrastructure at scale, defects related to state reconciliation can create large-scale consequences. A categorization of state reconciliation defects, i.e., defects related to state reconciliation, can aid in understanding the nature of state reconciliation defects. We conduct an empirical study with 5,110 state reconciliation defects where we apply qualitative analysis to categorize state reconciliation defects. From the identified defect categories, we derive heuristics to design prompts for a large language model (LLM), which in turn are used for validation of state reconciliation. From our empirical study, we identify 8 categories of state reconciliation defects, amongst which 3 have not been reported for previously-studied software systems. The most frequently occurring defect category is inventory, i.e., the category of defects that occur when managing infrastructure inventory. Using an LLM with heuristics-based paragraph style prompts, we identify 9 previously unknown state reconciliation defects of which 7 have been accepted as valid defects, and 4 have already been fixed. Based on our findings, we conclude the paper by providing a set of recommendations for researchers and practitioners. 

   more » « less
3. What questions do developers ask about Julia?

   https://doi.org/10.1145/3476883.3520205  

   Bose, Dibyendu Brinto; Gannod, Gerald C.; Rahman, Akond; Cottrell, Kaitlyn (April 2022, 2022 ACM Southeast Conference)

   The programming language Julia is designed to solve the 'two language problem', where developers who write scientific software can achieve desired performance, without sacrificing productivity. Since its inception in 2012, developers who have been using other programming languages have transitioned to Julia. A systematic investigation of the questions that developers ask about Julia can help in understanding the challenges that developers face while using Julia. Such understanding can be helpful (i) for toolsmiths who can construct tools so that developers can maximize their experience of using Julia, and (ii) for Julia language maintainers with empirical evidence on areas to improve the language as well as the Julia ecosystem. We conduct an empirical study with 3,093 Stack Overflow posts where we identify 13 categories of questions related to Julia-based software development. We observe developers to ask about a diverse set of topics, such as GC, Julia's garbage collector, JuMP, a domain-specific language constructed using Julia, and symbols, a metaprogramming utility in Julia. Based on our emerging results, we recommend enhancing support for developers with Julia-based tools and techniques for cross language transfer, type-related assistance, and package resolution. 

   more » « less
4. Quality Assurance for Infrastructure Orchestrators: Emerging Results from Ansible

   https://doi.org/10.1109/ICSA-C57050.2023.00073  

   Zhang, Yue; Rahman, Muktadir; Wu, Fan; Rahman, Akond (April 2023, 2023 IEEE 20th International Conference on Software Architecture Companion (ICSA-C))

   Infrastructure as code (IaC) is the practice of automatically managing computing infrastructure at scale. Despite yielding multiple benefits for organizations, the practice of IaC is susceptible to quality concerns, which can lead to large-scale consequences. While researchers have studied quality concerns in IaC manifests, quality aspects of infrastructure orchestrators, i.e., tools that implement the practice of IaC, remain an under-explored area. A systematic investigation of defects in infrastructure orchestrators can help foster further research in the domain of IaC. From our empirical study with 22,445 commits mined from the Ansible infrastructure orchestrator we observe (i) a defect density of 17.9 per KLOC, (ii) 12 categories of Ansible components for which defects appear, and (iii) the ‘Module’ component to include more defects than the other 11 components. Based on our empirical study, we provide recommendations for researchers to conduct future research to enhance the quality of infrastructure orchestrators. 

   more » « less
5. Gang of Eight: A Defect Taxonomy for Infrastructure as Code Scripts

   Rahman, Akond; Farhana, Effat; Parnin, Chris; Williams, Laurie (May 2020, 42nd ACM/IEEE International Conference on Software Engineering (ICSE))

   Defects in infrastructure as code (IaC) scripts can have serious consequences, for example, creating large-scale system outages. A taxonomy of IaC defects can be useful for understanding the nature of defects, and identifying activities needed to fix and prevent defects in IaC scripts. The goal of this paper is to help practitioners improve the quality of infrastructure as code (IaC) scripts by developing a defect taxonomy for IaC scripts through qualitative analysis. We develop a taxonomy of IaC defects by applying qualitative analysis on 1,448 defect-related commits collected from open source software (OSS) repositories of the Openstack organization. We conduct a survey with 66 practitioners to assess if they agree with the identified defect categories included in our taxonomy. We quantify the frequency of identified defect categories by analyzing 80,425 commits collected from 291 OSS repositories spanning across 2005 to 2019. Our defect taxonomy for IaC consists of eight categories, including a category specific to IaC called idempotency (i.e., defects that lead to incorrect system provisioning when the same IaC script is executed multiple times). We observe the surveyed 66 practitioners to agree most with idempotency. The most frequent defect category is configuration data i.e., providing erroneous configuration data in IaC scripts. Our taxonomy and the quantified frequency of the defect categories may help in advancing the science of IaC script quality. 

   more » « less