More Like this

1. Literature Review of Rust Analysis Tools

   Marlatt, Braxton; Carter, Caden; Goodall, Chadwick; Song, Jia (October 2025, Springer Nature, In Proceedings of the International Conference on Smart Technologies and Reliable Systems)

   The Rust Programming Language, developed by Mozilla, has gained popularity for combining performance and memory safety. Its ownership and borrowing mechanisms ensure memory safety and reduce common bugs, making Rust safe. In contrast, Unsafe Rust introduces potential vulnerabilities that can undermine these guarantees and cause difficult security flaws. Unsafe Rust allows low-level operations, interaction with other languages, and optimizations, but bypasses most security checks, so its use should be limited, reviewed, and validated. Static and dynamic code analysis tools help validate that unsafe regions do not contain vulnerabilities. Due to Rust’s rapid growth, documentation of such tools is sparse and difficult to navigate. This research reviews, analyzes, and tests multiple static code analysis tools, aiming to provide developers with a comprehensive resource tailored to Unsafe Rust. The results include insights into each tool’s functionality, strengths, weaknesses, and use cases, equipping developers to write safer, more secure, and reliable Rust code. 

   more » « less
2. HILDE: Intentional Code Generation via Human-in-the-Loop Decoding

   Gonzalez, Emmanuel Anaya; Rothkopf, Raven; Lerner, Sorin; Polikarpova, Nadia (October 2025, 2025 IEEE Symposium on Visual Languages and Human-Centric Computing)

   While AI programming tools hold the promise of increasing programmers’ capabilities and productivity to a remarkable degree, they often exclude users from essential decision making processes, causing many to effectively “turn off their brains” and over-rely on solutions provided by these systems. These behaviors can have severe consequences in critical domains, like software security. We propose Human-in-the-Loop Decoding, a novel interaction technique that allows users to observe and directly influence LLM decisions during code generation, in order to align the model’s output with their personal requirements. We implement this technique in HILDE, a code completion assistant that highlights critical decisions made by the LLM and provides local alternatives for the user to explore. In a within-subjects study (N=18) on security-related tasks, we found that HILDE led participants to generate significantly fewer vulnerabilities and better align code generation with their goals compared to a traditional code completion assistant. 

   more » « less
3. HILDE: Intentional Code Generation via Human-in-the-Loop Decoding

   Gonzalez, Emmanuel Anaya; Rothkopf, Raven; Lerner, Sorin; Polikarpova, Nadia (October 2025, Proceedings)

   While AI programming tools hold the promise of increasing programmers’ capabilities and productivity to a remarkable degree, they often exclude users from essential decision making processes, causing many to effectively “turn off their brains” and over-rely on solutions provided by these systems. These behaviors can have severe consequences in critical domains, like software security. We propose Human-in-the-Loop Decoding, a novel interaction technique that allows users to observe and directly influence LLM decisions during code generation, in order to align the model’s output with their personal requirements. We implement this technique in HILDE, a code completion assistant that highlights critical decisions made by the LLM and provides local alternatives for the user to explore. In a within-subjects study (N=18) on security-related tasks, we found that HILDE led participants to generate significantly fewer vulnerabilities and better align code generation with their goals compared to a traditional code completion assistant. 

   more » « less
4. Rethinking the Evaluation of Secure Code Generation

   Dai, Shih-Chieh; Xu, Jun; Tao, Guanhong (April 2026, 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26),)

   Large language models (LLMs) are widely used in software development. However, the code generated by LLMs often contains vulnerabilities. Several secure code generation methods have been proposed to address this issue, but their current evaluation schemes leave several concerns unaddressed. Specifically, most existing studies evaluate security and functional correctness separately, using different datasets. That is, they assess vulnerabilities using securityrelated code datasets while validating functionality with general code datasets. In addition, prior research primarily relies on a single static analyzer, CodeQL, to detect vulnerabilities in generated code, which limits the scope of security evaluation. In this work, we conduct a comprehensive study to systematically assess the improvements introduced by four state-of-the-art secure code generation techniques. Specifically, we apply both security inspection and functionality validation to the same generated code and evaluate these two aspects together. We also employ three popular static analyzers and two LLMs to identify potential vulnerabilities in the generated code. Our study reveals that existing techniques often compromise the functionality of generated code to enhance security. Their overall performance remains limited when evaluating security and functionality together. In fact, many techniques even degrade the performance of the base LLM by more than 50%. Our further inspection reveals that these techniques often either remove vulnerable lines of code entirely or generate “garbage code” that is unrelated to the intended task. Moreover, the commonly used static analyzer CodeQL fails to detect several vulnerabilities, further obscuring the actual security improvements achieved by existing techniques. Our study serves as a guideline for a more rigorous and comprehensive evaluation of secure code generation performance in future work. 

   more » « less
5. Pairing Security Advisories with Vulnerable Functions Using Open-Source LLMs

   Dunlap, Trevor; Meyers, John S; Reaves, Brad; Enck, William (July 2024, Proceedings of the Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA))

   As the reliance on open-source software dependencies increases, managing the security vulnerabilities in these dependencies becomes complex. State-of-the-art industry tools use reachability analysis of code to alert developers when security vulnerabilities in dependencies are likely to impact their projects. These tools heavily rely on precisely identifying the location of the vulnerability within the dependency, specifically vulnerable functions. However, the process of identifying vulnerable functions is currently either manual or uses a naive automated approach that falsely assumes all changed functions in a security patch link are vulnerable. In this paper, we explore using open-source large language models (LLMs) to improve pairing security advisories with vulnerable functions. We explore various prompting strategies, learning paradigms (i.e., zero-shot vs. few-shot), and show our approach generalizes to other open-source LLMs. Compared to the naive automated approach, we show a 173% increase in precision while only having an 18% decrease in recall. The significant increase in precision to enhance vulnerable function identification lays the groundwork for downstream techniques that depend on this critical information for security analysis and threat mitigation. 

   more » « less