An official website of the United States government
We present an ethnographic study of secure software development processes in a software company using the anthropological research method of participant observation. Two PhD students in computer science trained in qualitative methods were embedded in a software company for 1.5 years of total research time. The researchers participated in everyday work activities such as coding and meetings, and observed software (in)security phenomena both through investigating historical data (code repositories and ticketing system records), and through pen-testing the developed software and observing developers’ and management’s reactions to the discovered vulnerabilities. Our study found that 1) security vulnerabilities are sometimes intentionally introduced and/or overlooked due to the difficulty in managing the various stakeholders’ responsibilities in an economic ecosystem, and cannot be simply blamed on developers’ lack of knowledge or skills; 2) accidental vulnerabilities discovered in the pen-testing process produce different reactions in the development team, often times contrary to what a security researcher would predict. These findings highlight the nuanced nature of the root causes of software vulnerabilities and indicate the need to take into account a significant amount of contextual information to understand how and why software vulnerabilities emerge during software development. Rather than simply addressing deficits in developer knowledge or practice, this research sheds light on at times forgotten human factors that significantly impact the security of software developed by actual companies. Our analysis also shows that improving software security in the development process can benefit from a co-creation model, where security experts work side by side with software developers to better identify security concerns and provide tools that are readily applicable within the specific context of the software development workflow.
more »
« less
This content will become publicly available on August 13, 2026
Context Matters: Qualitative Insights into Developers’ Approaches and Challenges with Software Composition Analysis
Software Composition Analysis (SCA) is an important part in the software security lifecycle. Establishing the individual software components and versions that make up an application allows for identifying and remediating vulnerabilities. However, SCA tools have not kept up with the ever growing number of new vulnerabilities each year. Developers are flooded with vulnerability alerts and often struggle to quickly remediate critical issues with external components. We conducted 20 interviews with developers to investigate their processes and challenges around using SCA in their software projects. Interviews covered how SCA tools are integrated into workflows, how reports are interpreted and acted upon, and what challenges were encountered. We find that SCA tools are most often integrated into build pipelines and that users report that information in SCA alerts is too generic and lack context, specifically context on infrastructure, network configurations, reachability, and exploitability. Based on our findings we conclude that context matters throughout the SCA process, including for evaluating impact, when to trigger SCA scan runners, and how to integrate and communicate tool findings.
more »
« less
- Award ID(s):
- 2207008
- PAR ID:
- 10609315
- Publisher / Repository:
- USENIX
- Date Published:
- Format(s):
- Medium: X
- Location:
- Seattle, WA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Software Supply Chain Security (SSC) involves numerous stakeholders, processes and tools that work together to deliver a software product. A vulnerability in one element can cascade through the entire system and potentially affect thousands of dependents and millions of end users. Despite the SSC’s importance and the increasing awareness around its security, existing research mainly focuses on either technical aspects, exploring various attack vectors and their mitigation, or it empirically studies developers’ challenges, but mainly within the open source context. To better develop supportive tooling and education, we need to understand how developers consider and mitigate supply chain security challenges. We conducted 18 semi-structured interviews with experienced developers actively working in industry to gather in-depth insights into their experiences, encountered challenges, and the effectiveness of various strategies to secure the SSC. We find that the developers are generally interested in securing the supply chain, but encounter many obstacles in implementing effective security measures, both specific to SSC security and for general security. Developers also mention a wide set of approaches and methods to secure their projects, but mostly report general secure software engineering methodologies and seem to be mostly unaware of SSC specific threats and mitigations.more » « less
-
Background and Context. Research software in the Computing Education Research (CER) domain frequently encounters issues with scalability and sustained adoption, which limits its educational impact. Despite the development of numerous CER programming (CER-P) tools designed to enhance learning and instruction, many fail to see widespread use or remain relevant over time. Previous research has primarily examined the challenges educators face in adopting and reusing CER tools, with few focusing on understanding the barriers to scaling and adoption practices from the tool developers’ perspective. Objectives. To address this, we conducted semi-structured interviews with 16 tool developers within the computing education community, focusing on the challenges they encounter and the practices they employ in scaling their CER-P tools. Method. Our study employs thematic analysis of the semi-structured interviews conducted with developers of CER-P tools. Findings. Our analysis revealed several barriers to scaling highlighted by participants, including funding issues, maintenance burdens, and the challenge of ensuring tool interoperability for a broader user base. Despite these challenges, developers shared various practices and strategies that facilitated some degree of success in scaling their tools. These strategies include the development of teaching materials and units of curriculum, active marketing within the academic community, and the adoption of flexible design principles to facilitate easier adaptation and use by educators and students. Implications. Our findings lay the foundation for further discussion on potential community action initiatives, such as the repository of CS tools and the community of tool developers, to allow educators to discover and integrate tools more easily in their classrooms and support tool developers by exchanging design practices to build high-quality education tools. Furthermore, our study suggests the potential benefits of exploring alternative funding models.more » « less
-
In December, 2020, Apple began requiring developers to disclose their data collection and use practices to generate a “privacy label” for their application. The use of mobile application Software Development Kits (SDKs) and third-party libraries, coupled with a typical lack of expertise in privacy, makes it challenging for developers to accurately report their data collection and use practices. In this work we discuss the design and evaluation of a tool to help iOS developers generate privacy labels. The tool combines static code analysis to identify likely data collection and use practices with interactive functionality designed to prompt developers to elucidate analysis results and carefully reflect on their applications’ data practices. We conducted semi-structured interviews with iOS developers as they used an initial version of the tool. We discuss how these results motivated us to develop an enhanced software tool, Privacy Label Wiz, that more closely resembles interactions developers reported to be most useful in our semi-structured interviews. We present findings from our interviews and the enhanced tool motivated by our study. We also outline future directions for software tools to better assist developers communicating their mobile app’s data practices to different audiences.more » « less
-
Software depends on upstream projects that regularly fix vulnerabilities, but the documentation of those vulnerabilities is often unreliable or unavailable. Automating the collection of existing vulnerability fixes is essential for downstream projects to reliably update their dependencies due to the sheer number of dependencies in modern software. Prior efforts rely solely on incomplete databases or imprecise or inaccurate statistical analysis of upstream repositories. In this paper, we introduce Differential Alert Analysis (DAA) to discover vulnerability fixes in software projects. In contrast to statistical analysis, DAA leverages static analysis security testing (SAST) tools, which reason over code context and semantics. We provide a language-independent implementation of DAA and show that for Python and Java based projects, DAA has high precision for a ground-truth dataset of vulnerability fixes — even with noisy and low-precision SAST tools. We then use DAA in two large-scale empirical studies covering several prominent ecosystems, finding hundreds of resolved alerts, including many never publicly disclosed. DAA thus provides a powerful, accurate primitive for software projects, code analysis tools, vulnerability databases, and researchers to characterize and enhance the security of software supply chains.more » « less
