More Like this

1. Testing DNN image classifiers for confusion & bias errors

   https://doi.org/10.1145/3377812.3390799  

   Tian, Yuchi; Zhong, Ziyuan; Ordonez, Vicente; Kaiser, Gail; Ray, Baishakhi (June 2020, ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings)

   null (Ed.)

   We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others. Most existing DNN testing techniques focus on per-image violations, so fail to detect class-level confusions or biases. We developed a testing technique to automatically detect class-based confusion and bias errors in DNN-driven image classification software. We evaluated our implementation, DeepInspect, on several popular image classifiers with precision up to 100% (avg. 72.6%) for confusion errors, and up to 84.3% (avg. 66.8%) for bias errors. 

   more » « less
2. Using Undervolting as an on-Device Defense Against Adversarial Machine Learning Attacks

   https://doi.org/10.1109/HOST49136.2021.9702287  

   Majumdar, Saikat; Samavatian, Mohammad Hossein; Barber, Kristin; Teodorescu, Radu (December 2021, 2021 IEEE International Symposium on Hardware Oriented Security and Trust (HOST))

   Deep neural network (DNN) classifiers are powerful tools that drive a broad spectrum of important applications, from image recognition to autonomous vehicles. Unfortunately, DNNs are known to be vulnerable to adversarial attacks that affect virtually all state-of-the-art models. These attacks make small imperceptible modifications to inputs that are sufficient to induce the DNNs to produce the wrong classification. In this paper we propose a novel, lightweight adversarial correction and/or detection mechanism for image classifiers that relies on undervolting (running a chip at a voltage that is slightly below its safe margin). We propose using controlled undervolting of the chip running the inference process in order to introduce a limited number of compute errors. We show that these errors disrupt the adversarial input in a way that can be used either to correct the classification or detect the input as adversarial. We evaluate the proposed solution in an FPGA design and through software simulation. We evaluate 10 attacks and show average detection rates of 77% and 90% on two popular DNNs. 

   more » « less
3. DeepCert: Verification of Contextually Relevant Robustness for Neural Network Image Classifiers

   https://doi.org/10.1007/978-3-030-83903-1_5  

   Paterson, Colin; Wu, Haoze; Grese, John; Calinescu, Radu; Pasareanu, Corina S.; Barrett, Clark (September 2021, Computer Safety, Reliability, and Security (SAFECOMP '21))

   Habli, Ibrahim; Sujan, Mark; Bitsch, Friedemann (Ed.)

   We introduce DeepCert, a tool-supported method for verifying the robustness of deep neural network (DNN) image classifiers to contextually relevant perturbations such as blur, haze, and changes in image contrast. While the robustness of DNN classifiers has been the subject of intense research in recent years, the solutions delivered by this research focus on verifying DNN robustness to small perturbations in the images being classified, with perturbation magnitude measured using established 𝐿𝑝 norms. This is useful for identifying potential adversarial attacks on DNN image classifiers, but cannot verify DNN robustness to contextually relevant image perturbations, which are typically not small when expressed with 𝐿𝑝 norms. DeepCert addresses this underexplored verification problem by supporting: (1) the encoding of real-world image perturbations; (2) the systematic evaluation of contextually relevant DNN robustness, using both testing and formal verification; (3) the generation of contextually relevant counterexamples; and, through these, (4) the selection of DNN image classifiers suitable for the operational context (i) envisaged when a potentially safety-critical system is designed, or (ii) observed by a deployed system. We demonstrate the effectiveness of DeepCert by showing how it can be used to verify the robustness of DNN image classifiers build for two benchmark datasets (‘German Traffic Sign’ and ‘CIFAR-10’) to multiple contextually relevant perturbations. 

   more » « less
4. Provable Instance Specific Robustness via Linear Constraints

   Humayun, A.I; Casco-Rodriguez, Josue; Balestriero, Randall; Baraniuk, Richard (July 2023, 2nd AdvML Frontiers Workshop at International Conference on Machine Learning 2023)

   Deep Neural Networks (DNNs) trained for classification tasks are vulnerable to adversarial attacks. But not all the classes are equally vulnerable. Adversarial training does not make all classes or groups equally robust as well. For example, in classification tasks with long-tailed distributions, classes are asymmetrically affected during adversarial training, with lower robust accuracy for less frequent classes. In this regard, we propose a provable robustness method by leveraging the continuous piecewise-affine (CPA) nature of DNNs. Our method can impose linearity constraints on the decision boundary, as well as the DNN CPA partition, without requiring any adversarial training. Using such constraints, we show that the margin between the decision boundary and minority classes can be increased in a provable manner. We also present qualitative and quantitative validation of our method for class-specific robustness. Our code is available at https: //github.com/Josuelmet/CROP 

   more » « less
5. Built-In Functional Testing of Analog In-Memory Accelerators for Deep Neural Networks

   https://doi.org/10.3390/electronics11162592  

   Mishra , Abhishek Kumar; Das, Anup Kumar; Kandasamy, Nagarajan (August 2022, Electronics)

   The paper develops a methodology for the online built-in self-testing of deep neural network (DNN) accelerators to validate the correct operation with respect to their functional specifications. The DNN of interest is realized in the hardware to perform in-memory computing using non-volatile memory cells as computational units. Assuming a functional fault model, we develop methods to generate pseudorandom and structured test patterns to detect hardware faults. We also develop a test-sequencing strategy that combines these different classes of tests to achieve high fault coverage. The testing methodology is applied to a broad class of DNNs trained to classify images from the MNIST, Fashion-MNIST, and CIFAR-10 datasets. The goal is to expose hardware faults which may lead to the incorrect classification of images. We achieve an average fault coverage of 94% for these different architectures, some of which are large and complex. 

   more » « less