Search for: All records

The data precision can significantly affect the accuracy and overhead metrics of hardware accelerators for different applications such as artificial neural networks (ANNs). This paper evaluates the inference and training of multi-layer perceptrons (MLPs), in which initially IEEE standard floating-point (FP) precisions (half, single and double) are utilized separately and then compared with mixed-precision FP formats. The mixed-precision calculations are investigated for three critical propagation modules (activation functions, weight updates, and accumulation units). Compared with applying a simple low-precision format, the mixed-precision format prevents an accuracy loss and the occurrence of overflow/underflow in the MLPs while potentially incurring in less hardware overhead in terms of area/power. As the multiply-accumulation is the most dominant operation in trending ANNs, a fully pipelined hardware implementation for the fused multiply-add units is proposed for different IEEE FP formats to achieve a very high operating frequency. 

This paper deals with the fault tolerance of Triplet Networks (TNs). Results based on extensive analysis and simulation by fault injection are presented for new schemes. As in accordance with technical literature, stuck-at faults are considered in the fault model for the training process. Simulation by fault injection shows that the TNs are not sensitive to this type of fault in the general case; however, an unexcepted failure (leading to network convergence to false solutions) can occur when the faults are in the negative subnetwork. Analysis for this specific case is provided and remedial solutions are proposed (namely the use of the loss function with regularized anchor outputs for stuck-at 0 faults and a modified margin for stuck-at 1/-1 faults). Simulation proves that false solutions can be very efficiently avoided by utilizing the proposed techniques. Random bit-flip faults are then considered in the fault model for the inference process. This paper analyzes the error caused by bit-flips on different bit positions in a TN with Floating-Point (FP) format and compares it with a fault- tolerant Stochastic Computing (SC) implementation. Analysis and simulation of the TNs confirm that the main degradation is caused by bit-flips on the exponent bits. Therefore, protection schemes are proposed to handle those errors; they replace least significant bits of the FP numbers with parity bits for both single- and multi-bit errors. The proposed methods achieve superior performance compared to other low-cost fault tolerant schemes found in the technical literature by reducing the classification accuracy loss of TNs by 96.76% (97.74%) for single-bit (multi-bit errors).  

In the mean field integrate-and-fire model, the dynamics of a typical neuronwithin a large network is modeled as a diffusion-jump stochastic process whosejump takes place once the voltage reaches a threshold. In this work, the maingoal is to establish the convergence relationship between the regularizedprocess and the original one where in the regularized process, the jumpmechanism is replaced by a Poisson dynamic, and jump intensity within theclassically forbidden domain goes to infinity as the regularization parametervanishes. On the macroscopic level, the Fokker-Planck equation for the processwith random discharges (i.e. Poisson jumps) are defined on the whole space,while the equation for the limit process is on the half space. However, withthe iteration scheme, the difficulty due to the domain differences has beengreatly mitigated and the convergence for the stochastic process and the firingrates can be established. Moreover, we find a polynomial-order convergence forthe distribution by a re-normalization argument in probability theory. Finally,by numerical experiments, we quantitatively explore the rate and the asymptoticbehavior of the convergence for both linear and nonlinear models.