More Like this

1. Learning Assisted Post-Manufacture Testing and Tuning of RRAM-Based DNNs for Yield Recovery

   https://doi.org/10.23919/DATE58400.2024.10546505  

   Ma, Kwondo; Saha, Anurup; Amarnath, Chandramouli; Chatterjee, Abhijit (March 2024, IEEE)

   Variability-induced accuracy degradation of RRAM based DNNs is of great concern due to their significant potential for use in future energy-efficient machine learning architectures. To address this, we propose a two-step process. First, an enhanced testing procedure is used to predict DNN accuracy from a set of compact test stimuli (images). This test response (signature) is simply the concatenated vectors of output neurons of intermediate final DNN layers over the compact test images applied. DNNs with a predicted accuracy below a threshold are then tuned based on this signature vector. Using a clustering based approach, the signature is mapped to the optimal tuning parameter values of the DNN (determined using off-line training of the DNN via backpropagation) in a single step, eliminating any post-manufacture training of the DNN weights (expensive). The tuning parameters themselves consist of the gains and offsets of the ReLU activation of neurons of the DNN on a per-layer basis and can be tuned digitally. Tuning is achieved in less than a second of tuning time, with yield improvements of over 45% with a modest accuracy reduction of 4% compared to digital DNNs. 

   more » « less
2. SiPT: Signature-Based Predictive Testing of RRAM Crossbar Arrays for Deep Neural Networks

   https://doi.org/10.1109/TETC.2025.3533895  

   Ma, Kwondo; Saha, Anurup; Amarnath, Chandramouli; Chatterjee, Abhijit (October 2025, IEEE Transactions on Emerging Topics in Computing)

   Resistive Random-Access Memory (RRAM) crossbar array-based Deep Neural Networks (DNNs) are increasingly attractive for implementing ultra-low-power computing for AI. However, RRAM-based DNNs face inherent challenges from manufacturing process variability, which can compromise their performance (classification accuracy) and functional safety. One way to test these DNNs is to apply the exhaustive set of test images to each DNN to ascertain its performance; however, this is expensive and time-consuming. We propose a signature-based predictive testing (SiPT) in which a small subset of test images is applied to each DNN and the classification accuracy of the DNN is predicted directly from observations of the intermediate and final layer outputs of the network. This saves the test cost while allowing binning of RRAMbased DNNs for performance. To further improve the test efficiency of SiPT, we create the optimized compact set of test images, leveraging image filters and enhancements to synthesize images and develop a cascaded test structure, incorporating multiple sets of SiPT modules trained on compact test subsets of varying sizes. Through experimentation across diverse test cases, we demonstrate the viability of our SiPT framework under the RRAM process variations, showing test efficiency improvements. 

   more » « less
3. Signature Driven Post-Manufacture Testing and Tuning of RRAM Spiking Neural Networks for Yield Recovery

   Saha, A; Amarnath, C; Ma, K; Chatterjee, A (May 2025, Proceeding, IEEE/SIGDA Asian South Pacific Design Automation Conference)

   IEEE (Ed.)

   Resistive random access Memory (RRAM) based spiking neural networks (SNN) are becoming increasingly attractive for pervasive energy-efficient classification tasks. However, such networks suffer from degradation of performance (as determined by classification accuracy) due to the effects of process variations on fabricated RRAM devices resulting in loss of manufacturing yield. To address such yield loss, a two-step approach is developed. First, an alternative test framework is used to predict the performance of fabricated RRAM based SNNs using the SNN response to a small subset of images from the test image dataset, called the SNN response signature (to minimize test cost). This diagnoses those SNNs that need to be performance-tuned for yield recovery. Next, SNN tuning is performed by modulating the spiking thresholds of the SNN neurons on a layer-by-layer basis using a trained regressor that maps the SNN response signature to the optimal spiking thresholdvalues during tuning. The optimal spiking threshold values are determined by an off-line optimization algorithm. Experiments show that the proposed framework can reduce the number of out-of-spec SNN devices by up to 54% and improve yield by as much as 8.6%. 

   more » « less
4. EGIS: Entropy Guided Image Synthesis for Dataset-Agnostic Testing of RRAM-Based DNNs

   Saha, A; Amarnath, C; Ma, K; Chatterjee, A (March 2025, IEEE)

   While resistive random access memory (RRAM) based deep neural networks (DNN) are important for low-power inference in IoT and edge applications, they are vulnerable to the effects of manufacturing process variations that degrade their performance (classification accuracy). However, to test the same post-manufacture, the (image) dataset used to train the associated machine learning applications may not be available to the RRAM crossbar manufacturer for privacy reasons. As such, the performance of DNNs needs to be assessed with carefully crafted dataset-agnostic synthetic test images that expose anomalies in the crossbar manufacturing process to the maximum extent possible. In this work, we propose a dataset-agnostic post-manufacture testing framework for RRAM-based DNNs using Entropy Guided Image Synthesis (EGIS). We first create a synthetic image dataset such that the DNN outputs corresponding to the synthetic images minimize an entropy-based loss metric. Next, a small subset (consisting of 10-20 images) of the synthetic image dataset, called the compact image dataset, is created to expedite testing. The response of the device under test (DUT) to the compact image dataset is passed to a machine learning based outlier detector for pass/fail labeling of the DUT. It is seen that the test accuracy using such synthetic test images is very close to that of contemporary test methods. 

   more » « less
5. Selecting and Composing Learning Rate Policies for Deep Neural Networks

   https://doi.org/10.1145/3570508  

   Wu, Yanzhao; Liu, Ling (April 2023, ACM Transactions on Intelligent Systems and Technology)

   The choice of learning rate (LR) functions and policies has evolved from a simple fixed LR to the decaying LR and the cyclic LR, aiming to improve the accuracy and reduce the training time of Deep Neural Networks (DNNs). This article presents a systematic approach to selecting and composing an LR policy for effective DNN training to meet desired target accuracy and reduce training time within the pre-defined training iterations. It makes three original contributions. First, we develop an LR tuning mechanism for auto-verification of a given LR policy with respect to the desired accuracy goal under the pre-defined training time constraint. Second, we develop an LR policy recommendation system (LRBench) to select and compose good LR policies from the same and/or different LR functions through dynamic tuning, and avoid bad choices, for a given learning task, DNN model, and dataset. Third, we extend LRBench by supporting different DNN optimizers and show the significant mutual impact of different LR policies and different optimizers. Evaluated using popular benchmark datasets and different DNN models (LeNet, CNN3, ResNet), we show that our approach can effectively deliver high DNN test accuracy, outperform the existing recommended default LR policies, and reduce the DNN training time by 1.6-6.7× to meet a targeted model accuracy. 

   more » « less