More Like this

1. Stronger universal and transferable attacks by suppressing refusals

   Huang, David; Shah, Avidan; Araujo, Alexandre; Wagner, David; Sitawarin, Chawin (April 2025, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers))

   Making large language models (LLMs) safe for mass deployment is a complex and ongoing challenge. Efforts have focused on aligning models to human preferences (RLHF), essentially embedding a “safety feature” into the model’s parameters. The Greedy Coordinate Gradient (GCG) algorithm (Zou et al., 2023b) emerges as one of the most popular automated jailbreaks, an attack that circumvents this safety training. So far, it is believed that such optimization-based attacks (unlike hand-crafted ones) are sample-specific. To make them universal and transferable, one has to incorporate multiple samples and models into the objective function. Contrary to this belief, we find that the adversarial prompts discovered by such optimizers are inherently prompt-universal and transferable, even when optimized on a single model and a single harmful request. To further exploit this phenomenon, we introduce IRIS, a new objective to these optimizers to explicitly deactivate the safety feature to create an even stronger universal and transferable attack. Without requiring a large number of queries or accessing output token probabilities, our universal and transferable attack achieves a 25% success rate against the state-of-the-art Circuit Breaker defense (Zou et al., 2024), compared to 2.5% by white-box GCG. Crucially, IRIS also attains state-of-the-art transfer rates on frontier models: GPT-3.5-Turbo (90%), GPT-4o-mini (86%), GPT-4o (76%), o1-mini (54%), o1-preview (48%), o3-mini (66%), and deepseek-reasoner (90%). 

   more » « less
2. Do LLMs Make Cybersecurity Mistakes? An Evaluation of LLMs Performance on Cybersecurity Concept Inventories

   https://doi.org/10.1109/FIE63693.2025.11328704  

   Huang, Shan; Herman, Geoffrey L; Sherman, Alan T (November 2025, IEEE)

   This full research paper evaluates the capabilities of large language models (LLMs) in cybersecurity education through two scenario-based multiple-choice questions concept inventories: the Cybersecurity Concept Inventory (CCI) and the Cybersecurity Curriculum Assessment (CCA). The test models included Llama3, Claude 3.5, DeepSeek, and OpenAI's GPT series (GPT-3.5, GPT-4, GPT-4o, and GPT-o1). We evaluated these LLMs in a zero-shot setting to minimize external influencing factors and compared the LLMs' performance to that of students previously studied. We also applied the LLMs' responses to the derived Item Response Theory (IRT) model to compare their knowledge levels. Quantitative results revealed that GPT-o1 outperformed all other models and average students in correctness and knowledge level, correctly answering 92% of the CCI and 72% of CCA test items. Additionally, the research investigates whether GPT-o1 makes mistakes through expert review of GPTo1's output. Expert review of GPT-o1's reasoning patterns uncovered instances of insightful reasoning but also highlighted ways in which GPT-o1's answers reflect persistent student mistakes, such as biases, overgeneralizations, and logical inconsistencies. This work highlights the great potential of GPT-o1 for introductory cybersecurity education for providing detailed explanations and structured reasoning for novice learners. 

   more » « less
3. Measuring and Improving the Efficiency of Python Code Generated by LLMs Using CoT Prompting and Fine-Tuning

   https://doi.org/10.1109/ACCESS.2025.3585742  

   Jonnala, Ramya; Yang, Jeong; Lee, Young; Liang, Gongbo; Cao, Zechun (July 2025, IEEE Access)

   The burgeoning sophistication of Artificial Intelligence (AI) has catalyzed the rapid proliferation of Large Language Models (LLMs) within software development. These models are increasingly employed to automate the generation of functionally correct code, address complex computational problems, and facilitate the debugging of existing software systems. However, LLM-generated code often faces challenges due to inherent inefficiencies, including redundant logical structures, factually inconsistent content (hallucinations), and programming errors. To address this issue, our research rigorously evaluated the computational efficiency of Python code generated by three prominent LLMs: GPT-4o-Mini, GPT-3.5-Turbo, and GPT-4-Turbo. The evaluation metrics encompass execution time, memory utilization, and peak memory consumption, while maintaining the functional correctness of the generated code. Leveraging the EffiBench benchmark datasets within the Google Vertex AI Workbench environment, across a spectrum of machine configurations, the study implemented a consistent seed parameter to ensure experimental reproducibility. Furthermore, we investigated the impact of two distinct optimization strategies: Chain-of-Thought (CoT) prompting and model fine-tuning. Our findings reveal a significant enhancement in efficiency metrics for GPT-4o-Mini and GPT-3.5-Turbo when employing CoT prompting; however, this trend was not observed for GPT-4-Turbo. Based on its promising performance with CoT prompting, we selected the GPT-4o-Mini model for subsequent fine-tuning, aiming to further enhance both its computational efficiency and accuracy. However, contrary to our expectations, fine-tuning the GPT-4o-Mini model led to a discernible degradation in both its accuracy and computational efficiency. In conclusion, this study provides empirical evidence suggesting that the deployment of high-CPU machine configurations, in synergy with the utilization of the GPT-4o-Mini model and CoT prompting techniques, yields demonstrably more efficient and accurate LLM-generated Python code, particularly within computationally intensive application scenarios. 

   more » « less
4. Generating Effective Distractors for Introductory Programming Challenges: LLMs vs Humans

   https://doi.org/10.1145/3706468.3706529  

   Hassany, Mohammad; Brusilovsky, Peter; Savelka, Jaromir; Lekshmi_Narayanan, Arun Balajiee; Akhuseyinoglu, Kamil; Agarwal, Arav; Hendrawan, Rully Agus (March 2025, ACM)

   As large language models (LLMs) show great promise in generating a wide spectrum of educational materials, robust yet cost-effective assessment of the quality and effectiveness of such materials becomes an important challenge. Traditional approaches, including expert-based quality assessment and student-centered evaluation, are resource-consuming, and do not scale efficiently. In this work, we explored the use of pre-existing student learning data as a promising approach to evaluate LLM-generated learning materials. Specifically, we used a dataset where students were completing the program construction challenges by picking the correct answers among human-authored distractors to evaluate the quality of LLM-generated distractors for the same challenges. The dataset included responses from 1,071 students across 22 classes taught from Fall 2017 to Spring 2023. We evaluated five prominent LLMs (OpenAI-o1, GPT-4, GPT-4o, GPT-4o-mini, and Llama-3.1-8b) across three different prompts to see which combinations result in more effective distractors, i.e., those that are plausible (often picked by students), and potentially based on common misconceptions. Our results suggest that GPT-4o was the most effective model, matching close to 50% of the functional distractors originally authored by humans. At the same time, all of the evaluated LLMs generated many novel distractors, i.e., those that did not match the pre-existing human-authored ones. Our preliminary analysis shows that those appear to be promising. Establishing their effectiveness in real-world classroom settings is left for future work. 

   more » « less
5. Zero-Shot Anomaly Detection in Laser Powder Bed Fusion Using Multimodal RAG and Large Language Models

   https://doi.org/10.1115/1.4070585  

   Naghavi_Khanghah, Kiarash; Chen, Zhiling; Romeo, Lela; Yang, Qian; Malhotra, Rajiv; Imani, Farhad; Xu, Hongyi (December 2025, Journal of Mechanical Design)

   Additive manufacturing (AM) enables the fabrication of complex designs while minimizing waste, but faces challenges related to defects and process anomalies. This study presents a novel multimodal retrieval-augmented generation (RAG)-based framework that automates anomaly detection across various additive manufacturing processes leveraging retrieved information from the literature, including images and descriptive text, rather than training datasets. This framework integrates text and image retrieval from the scientific literature and multimodal generation models to perform zero-shot anomaly identification, classification, and explanation generation in a laser powder bed fusion (L-PBF) setting. The proposed framework is evaluated on four L-PBF manufacturing datasets from the Oak Ridge National Laboratory, featuring various printer makes, models, and materials. This evaluation demonstrates the framework's adaptability and generalizability across diverse images without requiring additional training. Comparative analysis using Qwen2-VL-2B and GPT-4o-mini as multimodal large language model (MLLM) within the proposed framework highlights that GPT-4o-mini outperforms Qwen2-VL-2B and proportional random baseline in manufacturing anomalies classification. Additionally, the evaluation of the RAG system confirms that incorporating retrieval mechanisms improves average accuracy by 12% by reducing the risk of hallucination and providing additional information. The proposed framework can be continuously updated by integrating emerging research, allowing seamless adaptation to the evolving landscape of AM technologies. This scalable, automated, and zero-shot-capable framework streamlines AM anomaly analysis, enhancing efficiency and accuracy. 

   more » « less