The reliability of emerging neuromorphic compute fabrics is of great concern due to their widespread use in critical data-intensive applications. Ensuring such reliability is difficult due to the intensity of underlying computations (billions of parameters), errors induced by low power operation and the complex relationship between errors in computations and their effect on network performance accuracy. We study the problem of designing error-resilient neuromorphic systems where errors can stem from: (a) soft errors in computation of matrix-vector multiplications and neuron activations, (b) malicious trojan and adversarial security attacks and (c) effects of manufacturing process variations on analog crossbar arrays that can affect DNN accuracy. The core principle of error detection relies on embedded
predictive neuron checks using invariants derived from the statistics of nominal neuron activation patterns of hidden layers of a neural network. Algorithmic encodings of hidden neuron function are also used to derive invariants for checking. A key contribution is designing checks that are robust to the inherent nonlinearity of neuron computations with minimal impact on error detection coverage. Once errors are detected, they are corrected using probabilistic methods due to the difficulties involved in exact error diagnosis in such complex systems. The technique is scalable across soft errors as well as a range of security attacks. The effects of manufacturing process variations are handled through the use of compact tests from which DNN performance can be assessed using learning techniques. Experimental results on a variety of neuromorphic test systems: DNNs, spiking networks and hyperdimensional computing are presented.
more »
« less
Error Resilient Neuromorphic Networks Using Checker Neurons
The last decade has seen tremendous advances in
the application of artificial neural networks to solving problems
that mimic human intelligence. Many of these systems are
implemented using traditional digital compute engines where
errors can occur during memory accesses or during numerical
computation. While such networks are inherently error
resilient, specific errors can result in incorrect decisions. This
work develops a low overhead error detection and correction
approach for multilayer artificial neural networks, here the
hidden layer functions are approximated using checker neurons.
Experimental results show that a high coverage of injected errors
can be achieved with extremely low computational overhead
using consistency properties of the encoded checks. A key side
benefit is that the checks can flag errors when the network is
presented outlier data that do not correspond to data with which
the network is trained to operate.
more »
« less
- Award ID(s):
- 1723997
- NSF-PAR ID:
- 10098271
- Date Published:
- Journal Name:
- International On-Line Testing Symposium
- Page Range / eLocation ID:
- 135 to 138
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this research, a low cost error detection and correction approach is developed for multilayer perceptron networks, where checker neurons are used to encode hidden layer functions using independent training experiments. Error detection and correction is predicated on validating consistency properties of the encoded checks and shows that high coverage of injected errors can be achieved with extremely low computational overhead.more » « less
-
The materials’ consolidation, especially ceramics, is very important in advanced research development and industrial technologies. Science of sintering with all incoming novelties is the base of all these processes. A very important question in all of this is how to get the more precise structure parameters within the morphology of different ceramic materials. In that sense, the advanced procedure in collecting precise data in submicro-processes is also in direction of advanced miniaturization. Our research, based on different electrophysical parameters, like relative capacitance, breakdown voltage, and [Formula: see text], has been used in neural networks and graph theory successful applications. We extended furthermore our neural network back propagation (BP) on sintering parameters’ data. Prognosed mapping we can succeed if we use the coefficients, implemented by the training procedure. In this paper, we continue to apply the novelty from the previous research, where the error is calculated as a difference between the designed and actual network output. So, the weight coefficients contribute in error generation. We used the experimental data of sintered materials’ density, measured and calculated in the bulk, and developed possibility to calculate the materials’ density inside of consolidated structures. The BP procedure here is like a tool to come down between the layers, with much more precise materials’ density, in the points on morphology, which are interesting for different microstructure developments and applications. We practically replaced the errors’ network by density values, from ceramic consolidation. Our neural networks’ application novelty is successfully applied within the experimental ceramic material density [Formula: see text] [kg/m 3 ], confirming the direction way to implement this procedure in other density cases. There are many different mathematical tools or tools from the field of artificial intelligence that can be used in such or similar applications. We choose to use artificial neural networks because of their simplicity and their self-improvement process, through BP error control. All of this contributes to the great improvement in the whole research and science of sintering technology, which is important for collecting more efficient and faster results.more » « less
-
more » « less
Abstract Accurate and timely storm surge forecasts are essential during tropical cyclone events in order to assess the magnitude and location of the impacts. Coupled ocean‐atmosphere dynamical models provide accurate measures of storm surge but remain too computationally expensive to run for real‐time forecasting purposes. Therefore, it is common to utilize a parametric vortex model, implemented within a hydrodynamic model, which decreases computational time at the expense of forecast accuracy. Recently, data‐driven neural networks are being implemented as an alternative due to their combined efficiency and high accuracy. This work seeks to examine how an artificial neural network (ANN) can be used to make accurate storm surge predictions, and explores the added value of using a recurrent neural network (RNN). In particular, it is concerned with determining the parameters needed to successfully implement a neural network model for the Mid‐Atlantic Bight region. The neural network models were trained with modeled data resulting from coupling of the Hybrid Weather Research and Forecasting cyclone model (HWCM) and the Advanced Circulation Model. An ensemble of synthetic, but physically plausible, cyclones were simulated using the HWCM and used as input for the hydrodynamic model. Tests of the ANN were conducted to investigate the optimal lead‐time configuration of the input data and the neural network architecture needed to minimize storm surge forecast errors. Results highlight the accuracy of the ANN in forecasting moderate storm surge levels, while indicating a deficiency in capturing the magnitude of the peak values, which is improved in the implementation of the RNN.
-
Homomorphic Encryption (HE) is one of the most promising security solutions to emerging Machine Learning as a Service (MLaaS). Several Leveled-HE (LHE)-enabled Convolutional Neural Networks (LHECNNs) are proposed to implement MLaaS to avoid the large bootstrapping overhead. However, prior LHECNNs have to pay significant computational overhead but achieve only low inference accuracy, due to their polynomial approximation activations and poolings. Stacking many polynomial approximation activation layers in a network greatly reduces the inference accuracy, since the polynomial approximation activation errors lead to a low distortion of the output distribution of the next batch normalization layer. So the polynomial approximation activations and poolings have become the obstacle to a fast and accurate LHECNN model. In this paper, we propose a Shift-accumulation-based LHE-enabled deep neural network (SHE) for fast and accurate inferences on encrypted data. We use the binary-operation-friendly leveled-TFHE (LTFHE) encryption scheme to implement ReLU activations and max poolings. We also adopt the logarithmic quantization to accelerate inferences by replacing expensive LTFHE multiplications with cheap LTFHE shifts. We propose a mixed bitwidth accumulator to expedite accumulations. Since the LTFHE ReLU activations, max poolings, shifts and accumulations have small multiplicative depth, SHE can implement much deeper network architectures with more convolutional and activation layers. Our experimental results show SHE achieves the state-of-the-art inference accuracy and reduces the inference latency by 76.21% ~ 94.23% over prior LHECNNs on MNIST and CIFAR-10.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/IOLTS.2018.8474075
