An official website of the United States government
Generative adversarial networks (GANs) are promising for a range of applications, including image translation and denoising, as well as synthetic data generation. These applications can be mapped to memristive crossbar arrays (MCAs) for ultra-high energy efficiency and portability. However, conductance variation within analog crossbars degrades the quality of the GAN outputs and necessitates robust post-manufacturing testing. We propose a two-stage adaptive test framework for compute-in-memory (CiM) based GANs, comprising an exhaustive test and a compact test. The exhaustive test measures the inception score of a device under test (DUT) by applying a large number of noise vectors, called the exhaustive noise set. To reduce test time, a compact test estimates the inception score of a DUT from a carefully chosen subset of these vectors, called the compact noise set. The compact noise set is determined by a binary mask optimized with a novel backpropagation-guided algorithm to minimize the difference between the estimated and true inception scores of the DUTs. Finally, to leverage both the accuracy of the exhaustive test and the speed of the compact test, the proposed adaptive test framework first applies the compact test to every DUT. Only the DUTs that yield low confidence in classifications are then subjected to the exhaustive test. Experiments show that this adaptive approach achieves less than 1% test escapes while offering up to 7.26× speedup compared to exhaustive test.
more »
« less
This content will become publicly available on May 25, 2027
Confidence Driven Compact Testing of Compute-in-Memory Based Language Models
Language models (LM) have revolutionized natural language processing (NLP) tasks such as question answering and code generation. However, escalating energy costs have accelerated the use of analog compute-in-memory (CiM) based architectures for energy-efficient language model inference. A key barrier to widespread adoption of such architectures is posed by device-to-device and cycle-to-cycle conductance variations within analog crossbars of CiM based systems that degrade text gener ation quality. To ensure reliable operation of analog CiM-based LMs (also referred to as device under tests or DUTs), fast and accurate functional testing of CiM structures is necessary post manufacture. We first assess the impact of conductance variations within crossbars on a LM’s log-perplexity, an evaluation metric for the quality of text generation. To derive a functional test, we propose confidence driven compact testing of CiM-based LMs. In this framework, we down-select text sequences from a benchmarking dataset to create a compact sequence set. During testing, we iteratively apply sequences from the compact sequence set to the CiM based LM and estimate a confidence interval of the DUT’s log-perplexity. Depending on the log-perplexity confidence interval (LPCI) and a predefined performance (log perplexity) threshold, a DUT is classified as ”good” or ”bad”. The combination of confidence driven testing and sequence down selection achieves 14.8-272x speedup compared to exhaustive testing while maintaining 0% critical test escape for GPT-2.
more »
« less
- Award ID(s):
- 2414361
- PAR ID:
- 10686259
- Publisher / Repository:
- IEEE
- Date Published:
- Subject(s) / Keyword(s):
- Language Model, Transformer, Functional Test
- Format(s):
- Medium: X
- Location:
- Estonia
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Compute-in-memory (CiM) based convolutional neural network (CNN) accelerators achieve low-power inference, utilizing memristive crossbar arrays for matrix multiplications. However, inherent conductance variations within the crossbar introduce computational errors. These errors propagate to the CNN output and cause image misclassification, leading to sub stantial accuracy degradation. This paper addresses the critical challenge of efficient and reliable post-manufacture testing for CiM-based CNN accelerators. We propose a novel test image sampling methodology, which iteratively applies sampled im ages from the CNN’s testing dataset using progressive random sampling (PRS) to a device under test (DUT) and estimates a confidence interval for the DUT accuracy. Based on the confidence interval and the acceptable accuracy threshold, the test labels a DUT as ”pass” or ”fail”. Furthermore, if we have access to an initial set of DUTs, we apply the images from the CNN’s testing dataset to these DUTs and leverage the DUT outputs to rank-order test images. We develop a sequential estimation test (SET) framework, where the images from the CNN’s testing dataset are sequentially applied according to a predetermined rank and the test terminates when a DUT can be confidently labeled as “pass” or “fail” based on the applied images. In each case, the number of applied test images adapts to the quality of the DUT. Experiments show that PRS and SET achieve 2.2× and 4.6× speedup compared to state-of-the-art test methodologies.more » « less
-
Moens, Marie-Francine; Huang, Xuanjing; Specia, Lucia; Yih, Scott Wen-tau (Ed.)Pre-trained Transformer language models (LM) have become go-to text representation encoders. Prior research fine-tunes deep LMs to encode text sequences such as sentences and passages into single dense vector representations for efficient text comparison and retrieval. However, dense encoders require a lot of data and sophisticated techniques to effectively train and suffer in low data situations. This paper finds a key reason is that standard LMs’ internal attention structure is not ready-to-use for dense encoders, which needs to aggregate text information into the dense representation. We propose to pre-train towards dense encoder with a novel Transformer architecture, Condenser, where LM prediction CONditions on DENSE Representation. Our experiments show Condenser improves over standard LM by large margins on various text retrieval and similarity tasks.more » « less
-
Aligning language models (LMs) with preferences is an important problem in natural language generation. A key challenge is that preferences are typically provided at the sequence level while LM training and generation both occur at the token level. There is, therefore, a granularity mismatch between the preference and the LM training losses, which may complicate the learning problem. In this paper, we address this issue by developing an alternate training process, where we iterate between grounding the sequence-level preference into token-level training guidance, and improving the LM with the learned guidance. For guidance learning, we design a framework that extends the pairwise-preference learning in imitation learning to both variable-length LM generation and the utilization of the preference among multiple generations. For LM training, based on the amount of supervised data, we present two minimalist learning objectives that utilize the learned guidance. In experiments, our method performs competitively on two distinct representative LM tasks — discrete-prompt generation and text summarization. Source codes are released at https://github.com/Shentao-YANG/Preference_Grounded_Guidance.more » « less
-
Neural language models (LMs) represent facts about the world described by text. Sometimes these facts derive from training data (in most LMs, a representation of the word banana encodes the fact that bananas are fruits). Sometimes facts derive from input text itself (a representation of the sentence I poured out the bottle encodes the fact that the bottle became empty). We describe REMEDI, a method for learning to map statements in natural language to fact encodings in an LM’s internal representation system. REMEDI encodings can be used as knowledge editors: when added to LM hidden representations, they modify downstream generation to be consistent with new facts. REMEDI encodings may also be used as probes: when compared to LM representations, they reveal which properties LMs already attribute to mentioned entities, in some cases making it possible to predict when LMs will generate outputs that conflict with background knowledge or input text. REMEDI thus links work on probing, prompting, and LM editing, and offers steps toward general tools for fine-grained inspection and control of knowledge in LMs.more » « less
