An official website of the United States government
Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into “The Good” (beneficial LLM applications), “The Bad” (offensive applications), and “The Ugly” (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs’ potential to both bolster and jeopardize cybersecurity.
more »
« less
This content will become publicly available on July 3, 2026
The Dual Role of Large Language Models in Network Security: Survey and Research Trends
Large language models (LLMs) have profoundly shaped various domains, including several types of network systems. With their powerful capabilities, LLMs have recently been proposed to enhance network security. However, the development of LLMs can introduce new risks due to their potential vulnerabilities and misuse. In this paper, we are motivated to review the dual role of LLMs in network security. Our goal is to explore how LLMs impact network security and ultimately shed light on how to evaluate LLMs from a network security perspective. We further discuss several future research directions regarding how to scientifically enable LLMs to assist with network security.
more »
« less
- Award ID(s):
- 2321271
- PAR ID:
- 10615554
- Publisher / Repository:
- ACM
- Date Published:
- ISBN:
- 979-8-4007-1531-0
- Format(s):
- Medium: X
- Location:
- Arlington, VA, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
With the growing development and deployment of large language models (LLMs) in both industrial and academic fields, their security and safety concerns have become increasingly critical. However, recent studies indicate that LLMs face numerous vulnerabilities, including data poisoning, prompt injections, and unauthorized data exposure, which conventional methods have struggled to address fully. In parallel, blockchain technology, known for its data immutability and decentralized structure, offers a promising foundation for safeguarding LLMs. In this survey, we aim to comprehensively assess how to leverage blockchain technology to enhance LLMs' security and safety. Besides, we propose a new taxonomy of blockchain for large language models (BC4LLMs) to systematically categorize related works in this emerging field. Our analysis includes novel frameworks and definitions to delineate security and safety in the context of BC4LLMs, highlighting potential research directions and challenges at this intersection.Through this study, we aim to stimulate targeted advancements in blockchain-integrated LLM security.more » « less
-
Several recent research efforts have proposed Machine Learning (ML)-based solutions that can detect complex patterns in network traffic for a wide range of network security problems. However, without understanding how these black-box models are making their decisions, network operators are reluctant to trust and deploy them in their production settings. One key reason for this reluctance is that these models are prone to the problem of underspecification, defined here as the failure to specify a model in adequate detail. Not unique to the network security domain, this problem manifests itself in ML models that exhibit unexpectedly poor behavior when deployed in real-world settings and has prompted growing interest in developing interpretable ML solutions (e.g., decision trees) for “explaining” to humans how a given black-box model makes its decisions. However, synthesizing such explainable models that capture a given black-box model’s decisions with high fidelity while also being practical (i.e., small enough in size for humans to comprehend) is challenging. In this paper, we focus on synthesizing high-fidelity and low-complexity decision trees to help network operators determine if their ML models suffer from the problem of underspecification. To this end, we present TRUSTEE, a framework that takes an existing ML model and training dataset generate a high-fidelity, easy-to-interpret decision tree, and associated trust report. Using published ML models that are fully reproducible, we show how practitioners can use TRUSTEE to identify three common instances of model underspecification, i.e., evidence of shortcut learning, spurious correlations, and vulnerability to out-of-distribution samples.more » « less
-
The exponential growth of digital content has generated massive textual datasets, necessitating the use of advanced analytical approaches. Large Language Models (LLMs) have emerged as tools that are capable of processing and extracting insights from massive unstructured textual datasets. However, how to leverage LLMs for text analytics Information Systems (IS) research is currently unclear. To assist the IS community in understanding how to operationalize LLMs, we propose a Text Analytics for Information Systems Research (TAISR) framework. Our proposed framework provides detailed recommendations grounded in IS and LLM literature on how to conduct meaningful text analytics IS research for design science, behavioral, and econometric streams. We conducted three business intelligence case studies using our TAISR framework to demonstrate its application in several IS research contexts. We also outline the potential challenges and limitations of adopting LLMs for IS. By offering a systematic approach and evidence of its utility, our TAISR framework contributes to future IS research streams looking to incorporate powerful LLMs for text analytics.more » « less
-
Year after year, computing systems continue to grow in complexity at an exponential rate. While this can have far-ranging positive impacts on society, it has become extremely difficult to ensure the security of these systems in the field. Hardware security - in conjunction with more traditional cybersecurity topics like software and network security - is critical for designing secure systems. Moving forward, hardware security education must ensure the next generation of engineers have the knowledge and tools to address this growing challenge. A good foundation in hardware security draws on concepts from several different fields, including fundamental hardware design principles, signal processing and statistics, and even machine learning for modeling complex physical processes. It can be difficult to convey the material in a manageable way, even to advanced undergraduate students. In this paper, we describe how we have leveraged Python, and its rich ecosystem of open-source libraries, and scaffolding with Jupyter notebooks, to bridge the gap between theory and implementation of hardware security topics, helping students learn through experience.more » « less
