Spiking Neural Networks (SNNs) [60] are machine learning approaches designed with spike-based computations [46] and bio-inspired learning algorithms [15] (See Appendix A for background on SNNs). SNN-based workloads are typically executed on event-driven neuromorphic hardware such as TrueNorth [35], Loihi [34], and DYNAP-SE [64]. These hardware platforms are extremely energyefficient, thanks to their event-driven activation and their tile-based distributed architecture with in-place neural computations and synaptic storage [72]. We investigate the internal architecture of neurons and synapses in DYNAP-SE (see Figures 3b and4b), and found that these circuits consist of transistors built using bulk CMOS or FinFet technologies [1,18,44]. 1 When operated at a high voltage and temperature, the transistor's parameters strongly drift from their nominal values. This is called aging. In fact, in scaled technology nodes, this aging happens even under nominal conditions and from the very start of using the devices leading to the so-called soft breakdown. The most important breakdown mechanism is the Bias Temperature Instability (BTI) [50,51,99]. Strongly depending on the workload, BTI is highly variable and it is largely reversible under nominal conditions on removal of the stress voltage. So it leads only to parametric time-dependent variability, affecting mainly delay and leakage power. If the neurons and synapses in a neuromorphic hardware are used continuously for long duration at elevated operating conditions, the parameter drifts cannot be reversed [100], leading to permanent functional degradation of the circuit and eventually, hardware faults [53,68,91]. The permanent fault rates in integrated circuits can be described by the bathtub curve as shown in Figure 1. Post manufacturing, integrated circuits (IC) are characterized by high failure rates as these circuits are subjected to manufacturing tests, such as stuck-at, at-speed, burn-in, etc., which filters out defective circuits and circuits with short lifetime. The probability of the successful circuits surviving for a longer period of time, increases. The failure rate, therefore, decreases over time. This phase is known as the infant mortality period. This is followed by a period of constant failure rate, often referred as useful life. The last phase is known as the wear-out or the aging phase and is characterized by increasing fault rate. Recent studies on reliability reveal that, if wear-out is not addressed from early device usage stage (e.g., the beginning of useful life period), circuits can age faster than anticipated with the wear-out phase settling earlier in life (shown by the red dashed line in the figure).
To address time-dependent variability or aging, circuit designers often set worst-case and hence highly pessimistic reliability-related extra design margins, which unnecessarily constrain performance. Our objective is to analyze the circuit aging in neuromorphic hardware at real-time and take corrective measures at the architecture-level to reverse the parameter drifts based on the utilization of neuron and synapse circuits within a machine learning workload.
Recently, Non-Volatile Memory (NVM) is used in neuromorphic hardware to implement highdensity and low-energy synaptic storage [14]. Several NVMs are explored for this purpose -Oxidebased Resistive RAM (OxRRAM) [62], Phase Change Memory (PCM) [66], Ferro-Electric RAM [65], and Spin-Transfer Torque Magnetic or Spin-Orbit-Torque RAM (STT-and SoT-MRAM) [97]. 2NVMs require either high voltage (OxRRAM, PCM and FeFET) or high current (MRAM) to operate, which accelerates the aging of transistors in neuron and synapse circuits in a neuromorphic hardware [2,55,83,84,88]. Aggressive device scaling increases power density and temperature, which makes reliability even worse. Therefore, circuit aging is emerging as one of the primary reliability concerns for neuromorphic hardware designed with NVMs [16].
The reliability problem we are addressing in this work is due to high voltage operations of NVMs. That can also occur in other system contexts, 3 but it is in particular an issue for SNNs due to the following reasons. To address this high voltage NVM problem, periodic de-stress of the peripheral circuit is necessary, which impacts inter-spike interval (ISI) when machine learning models are executed on these circuits. The performance (e.g., accuracy) of SNNs depends on ISI. Therefore, the reliability issues of NVMs lead to performance issues in SNNs.
Prior works on mapping machine learning workloads to neuromorphic hardware have mostly focused on compilation techniques, with the objective of improving machine learning performance on hardware. Examples of such approaches include hardware utilization-based mapping [3,4,8,10,11,24,40,47,48,86], energy-based mapping [9,32,94], and endurance-based mapping [92,93,95]. The recently-proposed approach RENEU [88] is the only compile-time based technique that maps the neurons and synapses to the hardware to improve the long-term, i.e., the lifetime reliability. Although compile-time based aging mitigation approaches have unique advantages such as low computation overhead, predictability, and performance guarantee, they are often conservative and therefore, may miss significant performance and reliability improvement opportunities. Dynamic approaches are flexible, adaptive, and potentially more effective in a highly dynamic environment, such as ones where the inference data deviates strongly from training examples. We show that both performance and reliability can be improved significantly if neuron and synapse circuits are de-stressed periodically at run-time based on current data.
On the run-time front, very few approaches address the run-time management of neuromorphic computing. 4 In [10], the authors propose a fast approach to remap online learning SNNs on a neuromorphic hardware after every learning epoch to improve model performance. In DTRO [2], the authors propose a hybrid approach to estimate the reliability degradation for machine learning workloads at design-time using training data, and use this information to de-stress all hardware circuits during run-time at fixed intervals, without actually tracking the circuit aging. The effectiveness of this approach is limited to supervised techniques only and the availability of representative training data. To this end, we make the following three key observations. • Observation 1: Workload, which includes synaptic weights and their activation on neuromorphic hardware, is specific to the machine learning task being executed and its input. • Observation 2: De-stressing all circuits in the hardware periodically, without tracking the actual aging, introduces long latency in spike generation and propagation, which impacts inter-spike interval, leading to information loss in SNNs. • Observation 3: Compared to long-term aging under elevated stress conditions, which is permanent and irreversible, short-term aging under nominal conditions is heavily workload-dependent (and hence to some extent controllable), and partially reversible.
Based on these three observations, we introduce NCRTM, a run-time reliability manager for neuromorphic hardware to de-stress neuron and synapse circuits in the hardware only when needed, by dynamically tracking their short-term aging during the execution of machine learning tasks. NCRTM extends our earlier work DTRO [2] with the following new contributions.
• We introduce an intelligent run-time manager NCRTM, which improves the long-term reliability of neuromorphic hardware by controlling its short-term aging when executing machine learning tasks. • We develop a run-time performance monitoring and reliability estimation framework using statistics collected from the neuromorphic hardware. • We show that NCRTM can be applied to both supervised and unsupervised machine learning approaches and scenarios where the number of training examples are limited. • We evaluate NCRTM with machine learning workloads designed using Convolution Neural Network (CNN), Multi-layer Perceptron (MLP), and Recurrent Neural Network (RNN) models on a state-of-the-art neuromorphic hardware simulator. Overall, NCRTM mitigates the aging-related reliability problems in neuromorphic computing by dynamically de-stressing neuron and synapse circuits in response to their short-term aging, with the objective of meeting a reliability target. NCRTM de-stresses these circuits only when it is absolutely necessary to do so, otherwise reducing the performance impact by scheduling all de-stress operations off the critical path by tracking the latency impact of de-stress operations on inter-spike interval (ISI), a key performance measure in SNNs.
Figure 2 illustrates how the proposed approach differs from two reliability-oriented state-of-the-art approaches. Figure 2a illustrates a design-time approach such as RENEU [88], where neurons and synapses are mapped to the hardware to increase the long term, i.e., the lifetime reliability. This approach estimates the aging in neuron and synapse circuits using representative training examples. There are no corrective online measures in place to control the aging, should the aging exceed a critical threshold or the workload behavior changes, for instance, when encountering unseen data at run-time. Figure 2b illustrates a hybrid approach such as DTRO [2], where neurons and synapses are mapped to the hardware using a reliability-oriented mapping technique (e.g., RENEU [88]). Additionally, all neuron and synapse circuits in the hardware are periodically de-stressed to control the aging. The de-stress interval is determined using training examples. The drawbacks of such an approach are the following. First, by not tracking the actual aging in real-time, such approach can introduce significant latency in interrupting normal operation, even when the aging is much below the critical threshold. This is especially critical for SNNs because the performance of machine learning workloads, e.g., their accuracy, depends on the precise times of spikes (see Appendix A). Second, the effectiveness of such a hybrid approach depends heavily on the training data, which may not always be representative. In fact, hybrid approaches present significant limitations for unsupervised applications or applications with limited training data. Figure 2c illustrates NCRTM, the proposed run-time approach for reliability management in neuromorphic hardware. NCRTM tracks the aging in neuron and synapse circuits at real-time during the execution of machine learning workloads and de-stresses these circuits only when their aging exceeds a critical threshold. By implementing age tracking and control at run-time, the de-stress decisions of NCRTM are made based on current data. Therefore, NCRTM is relevant for both supervised and unsupervised machine learning approaches.
To the best of our knowledge, NCRTM is the first work for run-time reliability management of neuromorphic hardware. In Section 7, we evaluate NCRTM against these state-of-the-art reliability management approaches using both supervised and unsupervised machine learning workloads.
In this section, we introduce the background necessary to understand our proposed run-time manager NCRTM. Background on SNNs are provided as Appendix.
We consider tile-based neuromorphic hardware [6,17], where the tiles are interconnected using networks-on-chip (NoC) or Segmented Bus [7]. This is similar to several contemporary neuromorphic architectures such as DYNAP-SE [64], Loihi [34], and TrueNorth [35]. Each tile in the hardware consists of a crossbar for synaptic storage, a set of input and output neurons, and a performance monitoring unit, which in its simplest form is a spike counter (SC). A crossbar, shown in Figure 3a, is a 3D organization of top electrodes (TEs), which form the rows and bottom electrodes (BEs), which form the columns. A synaptic cell is connected at a crosspoint, i.e., at the intersection of each row and column via an access transistor as shown in Figure 3b. Pre-synaptic neurons are mapped along the TEs and post-synaptic neurons along the BEs. The synaptic weight between a pre-and a post-synaptic neuron is programmed as conductance of the corresponding synaptic cell at the crosspoint. A pre-synaptic neuron's voltage (𝑉 𝑖 ) applied on the TE is multiplied by the conductance (𝐺 𝑖 ) to generate current 𝐼 𝑖 = 𝑉 𝑖 • 𝐺 𝑖 (according to Ohm's Law). This current propagates to the post-synaptic neuron to raise its action potential. Current summation occurs on each BE according to Kirchoff's Current Law, when integrating excitation from other pre-synaptic neurons. This implements 𝑖 𝐼 𝑖 = 𝑖 𝑉 𝑖 • 𝐺 𝑖 . This is the in-memory multiply accumulate logic implemented inside a crossbar. Figure 3a illustrates the integration of input excitation from two pre-synaptic neurons to one post-synaptic neuron via the synaptic weights 𝑤 1 and 𝑤 2 , respectively. This forms the data plane of the neuromorphic hardware. The control plane of the hardware consists of control signals, which enable specific access transistors (see Figure 3b) to facilitate current flow in the crossbar. The NVM device of a synaptic cell, shown as a resistive element in Figure 3b, can be implemented for instance with HfO2-based OxRAM or chalcogenide-based PCM as shown in Figure 3c. But our approach is not limited to these specific NVM technologies. Figure 4a shows the currents and voltages on the path from the pre-synaptic neuron 𝑁 𝑖 to the post-synaptic neuron 𝑁 𝑗 . The input current 𝐼 𝑖 𝑖𝑛 𝑗 is converted into voltage 𝑉 𝑖 𝑠𝑝𝑘 using the neuron 𝑁 𝑖 . This voltage is multiplied with the conductance 𝐺 𝑖 (representing synaptic weight) to generate the current 𝐼 𝑗 𝑖𝑛 𝑗 . This current is converted to voltage 𝑉 𝑗 𝑠𝑝𝑘 using neuron 𝑁 𝑗 . Figure 4b illustrate the internal architecture of a Leaky-Integrate-and-Fire neuron [45]. The current 𝐼 inj injected into the neuron is proportional to the weighted sum of excitation from all of its pre-synaptic connections. The PMOS and NMOS transistors in the neuron and the reference voltages raise the neuron's membrane voltage. When the voltage crosses a threshold, a spike is generated. The spike voltage (𝑉 spk ) must be sufficiently high to propagate current through the synaptic cell connected at the output of the neuron. This design, however, is also undermined by large power consumption, due to the very same problems present in the Axon-Hillock circuit. In [14] van Schaik proposed a circuit with an amplifier at the input, to compare the voltage on the capacitor with a desired spiking threshold voltage.
As the input exceeds the spiking threshold, the amplifier Certain NVMs such as PCM, OxRRAM, and FeFET requires high voltages to operate. We consider the case of PCM-based neuromorphic hardware, which requires ≈ 3V [87] to propagate current through it. This high voltage causes aging of the access transistor in each synaptic cell in a crossbar and also of the transistors in each neuron connected along the TEs and the BEs of the crossbar.
High voltage operations of transistors introduce many reliability issues such as Time-Dependent Dielectric Breakdown (TDDB), Bias Temperature Instability (BTI), and Hot-Carrier Injection (HCI). These are the dominant causes of aging in scaled technology nodes (45nm and below) [61]. In older nodes, Electromigration (EM) also plays a key role [23, 25-30, 69, 79].
Transistor aging is accelerated when it is stressed, i.e., exposed to high overdrive voltage, where overdrive voltage is defined as the voltage between transistor gate and source (𝑉 𝐺𝑆 ) in excess of the threshold voltage (𝑉 th ), where 𝑉 th is the minimum voltage required between gate and source to turn the transistor on. With this understanding, we provide a brief background of these three failure mechanisms.
• TDDB: This is a failure mechanism in a CMOS device, when the gate oxide breaks down as a result of long-time application of relatively low electric field (as opposed to immediate breakdown, which is caused by strong electric field) [76]. The lifetime of a CMOS device is measured in terms of its mean time to failure (MTTF) as
where 𝐴 and 𝛾 are material-related constants, and 𝑉 is the overdrive gate voltage of the CMOS device.
• BTI: This is a failure mechanism in a CMOS device where positive charges are trapped at the oxide-semiconductor boundary underneath the gate [41]. BTI manifests as 1) decrease in drain current and transconductance, and 2) increase in off current and threshold voltage. The BTI lifetime of the device is
where 𝐴 and 𝛾 are material-related constants, 𝐸 𝑎 is the activation energy, 𝐾 is the Boltzmann constant, 𝑇 is the temperature, and 𝑉 is the overdrive gate voltage of the CMOS device. • HCI: This is a failure mechanism in a CMOS device, when a carrier (electron or hole) gains sufficient kinetic energy to overcome the potential barrier of the conducting channel and gets trapped in the gate dielectric, permanently changing its switching characteristic [98]. Unlike the TDDB and BTI failure mechanisms, for which silicon-characterized reliability models are available from foundries, characterized models for HCI failure mechanisms are still in development for scaled nodes. Among these failure mechanisms, BTI is generally accepted as the most important mechanism for sub-10 nm nodes [59,73,78]. HCI mostly occurs there under strong voltage/current conditions and TDDB has become less important because technologists have stopped pursuing ultra-high k values in the dielectric.
We expand on the three observations made in Section 1.
To illustrate the application and input-dependent neuron activation, Figure 5 plots the spike firing rate of 100 randomly-selected neurons in AlexNet [52], a state-of-the-art CNN used for Imagenet classification. We report results for two randomly-selected training and test images. We observe that spike firing rates of neurons depend on the image presented to the AlexNet CNN. Therefore, reliability improvement strategies based on design-time analysis with training examples may not be optimal when they are applied at run-time to process in-field data, a limitation of our prior work [2]. We address this limitation by designing our proposed run-time framework NCRTM, which can adapt its decisions based on current data.
To demonstrate the workload-dependent nature of spike firing rate, Figure 6 plots the minimum, maximum, and average spike rate of all neurons in 10 machine learning workloads (see Section 6) for test examples from their respective dataset. We observe that spike rates of neurons are strongly workload-dependent, and therefore, a workload-specific strategy is needed to optimally control the reliability aspect. This is precisely the objective of NCRTM.
In SNNs, information is encoded in spike times. Inter-spike interval (ISI) defines the performance metric in SNNs. To demonstrate how ISI is impacted by reliability-oriented decisions, Figure 7(a)
shows the spike train generated by a neuron in AlexNet when processing a reference image. Each spike injects current into the crossbar to flow through the NVM cell. Figure 7(b) illustrates the voltage of the on-chip charge pump that supply the reference voltages in the neuron to generate this spike train. The charge pump is operated at 1.8𝑉 for the entire 60ms interval, boosting its voltage to 3𝑉 only to generate spikes. Aging of the transistors in the neuron is 8.3 units (see Section 5.3 for aging computation) and the average ISI is 5.9ms (See Section 5.4 for ISI computation). Figure 7(c) illustrates the charge pump's operating voltage when it is discharged to 1.2𝑉 after generating every spike and boosted again to 1.8𝑉 before generating the next. This is to de-stress the transistors in the neuron. Once de-stressed, the neuron becomes unavailable to generate spikes, introducing latency Although it is possible to estimate the worst-case ISI degradation at compile-time using SpiNe-Map [9] and other similar approaches, such estimation can deviate significantly from the actual case in a highly dynamic environment, where testing data is different from the training examples (see Figure 5). Therefore, a run-time manager is desirable to dynamically adapt the reliability-oriented decisions to limit ISI degradation.
In this work, we demonstrate our approach for BTI-related failures. The general principle applies to any other failure mechanism. BTI aging manifests as: 1) A decrease in drain current and transconductance, and 2) An increase in off current and threshold voltage. When operated at a high voltage and temperature, these parameters strongly drift from their nominal values. In fact, in scaled technology nodes, this BTI aging happens even under nominal conditions and from the very start of using the devices leading to the so-called soft breakdown [33,50,51,99]. Recent works such as [41,42,50,51,70,77,99] suggest that BTI is the collective response of two independent defects -the as-grown hole traps (AHTs) and generated defects (GDs). AHTs and a small proportion of GDs can be recovered by annealing at high temperatures if the BTI stress voltage is removed (de-stress). Figure 8 illustrates the stress and recovery of the threshold voltage of a CMOS transistor on application of a high (𝑉 spk ) and a low voltage (𝑉 idle ). We observe that both stress and recovery depends on the time of exposure to the corresponding voltage level. This implies that when a neuron is idle, the BTI aging of the neuron recovers from stress. Figure 9a shows the shift in threshold voltage of a NMOS transistor in a neuron with a constant firing rate of 50Hz and the neuron circuit de-stressed once every second (see Section 6 for the simulation setup). Figure 9b shows the results using the same setup, but with the neuron circuit de-stressed once every 100ms. As this figure clearly shows, with longer de-stress interval (e.g., once every second), the transistor aging becomes irreversible. Therefore, the shift in threshold voltage of the transistor is higher than the case with shorter de-stress interval (e.g., once every 100ms).
Figure 10 shows an example where four spikes (R1, R2, R3, & R4) are generated from a neuron. These spikes are generated with some idle time between them (based on its input excitation). The figure illustrates the hybrid approach DTRO [2] (see Fig. 2b), where the neuron is de-stressed at run-time after generating 3 spikes. This fixed number is decided statically, considering the neuron's activation in some training example. This is illustrated in the top right corner of the figure, where we observe that the BTI aging (A BTI ) exceeds the aging threshold of 10 units after generating three spikes based on the idle periods between spikes in the training example.
Using this static approach, the de-stress operation is initiated upon generating R3, which delays the generation of R4 due to the non-zero latency of the de-stress operation. This causes a change in the ISI, which may lead to performance loss in SNNs (see Appendix B). At the time when the neuron circuit is being de-stressed, the BTI aging is below the threshold because a CMOS transistor recovers partially from BTI stress when idle. The length of the idle periods at run-time can be different from those used at design-time when the analysis is performed as shown in this example. Therefore, the static approach will unnecessarily introduce performance penalty in such a situation. Using fixed interval for de-stress (instead of counting the spikes) will also lead to a similar situation because the number of spikes within the de-stress interval still remains unknown at run-time, being dependent on the input excitation.
Figure 10 also shows NCRTM, our dynamic reliability management policy, where the de-stress operation of the neuron circuit is initiated by tracking its aging. NCRTM can generate the spike R4 because the aging of the neuron is lower than the aging threshold at the time of generating the spike. This is because NCRTM models both the stress and recovery of circuit aging at run-time. There is no change in ISI. Therefore, NCRTM is better than the static approach both in terms of reliability and performance. This example demonstrates one scenario with sparse neuron activation. One can also imagine a counter scenario where the neuron is activated too frequently. In this case, the static policy can lead violating the critical threshold because it cannot adapt the de-stress interval at run-time. NCRTM can adjust its de-stress interval at run-time by tracking the aging (both stress and recover). In Section 7.4 we show only a marginal performance impact for workloads with frequent activation. Therefore, NCRTM is better than the static policy, when it comes to managing workload-specific circuit aging.
Based on the three observations in Section 4 and the motivating example in Section 5.1, we introduce NCRTM, a run-time reliability manager for neuromorphic hardware. Figure 11 NCRTM is implemented as a controller to mitigating the aging of neuron circuits in each tile. To do so, NCRTM estimates maximum aging of the neurons in each tile by recording the number of spikes within a time window (see our aging formulation in Section 5.3). If the aging of a tile exceeds a threshold (𝑡ℎ_𝑎), 5 NCRTM schedules the de-stress of the tile by making an entry in the de-stress queue (DSQ). However, the tile may not be de-stressed immediately. NCRTM de-stresses a tile opportunistically by estimating the change in ISI (called ISI distortion) that may result due to offlining the tile (see our ISI formulation in Section 5.4). During de-stresses, a very low voltage is applied to all the neurons in a tile for a time duration 𝑡𝐷𝑆𝐶 (discharge cycle time). This allows the transistors in the neurons to reverse the threshold voltage drift Δ𝑉 th . The recovery time 𝑡𝐷𝑆𝐶 is modeled using the framework presented in [100].
Fundamental to the aging and ISI computation in NCRTM is a technique to estimate the number of spikes for each neuron. The spike counter (SC) in each tile can facilitate counting the spikes in a time interval. However, not all neuromorphic hardware is equipped with SC. Therefore, we present an alternative software-based technique for implementing spike counting.
Dynamic Reliability Management in Neuromorphic Computing 1:13 5.2.1 Spike Counting in Software. To understand spike counting in software, we describe the spike communication mechanism in neuromorphic hardware. Spikes from the post-synaptic neurons in a tile are converted into an address-event representation (AER) and broadcasted on the interconnect via the AER encoder. Figure 12 shows an example explaining the principles behind AER. Here, four neurons in a tile spike at time 3, 0, 1 and 2 time units, respectively. The encoder encodes these four spikes in order to be communicated on the interconnect.
network and 400 billion SOPS per watt for networks with high spike rates and high number of active synapses, whereas todays most energy-efficient supercomputer achieves 4.5 billion floatingpoint operations per second (FLOPS) per watt [25]. Although the metric units are different, the computational capability can by some means be indicated by the number of operations per second.
This project is conducting the research on communication mechanisms and architecture of neuromorphic computing. In this section, some existent and conventional protocols and architectures are introduced as the basis of this research.
Address-event representation (AER) is a communication protocol originally proposed as a method to communicate sparse neural events between neuromorphic chips. Massive interconnections among individual neurons or neuron clusters are allocated to the reduced number of channels by time division multiplexing. According to the protocol, each spike is represented by its location and spiking time. We propose to count the spikes from each neuron by snooping on the interconnect. We implement counters, one for each neuron of the hardware. When a de-stress operation is initiated for a tile, all the counters for the neurons in the tile are reset to start counting the spikes for the next interval. The total storage overhead needed for implementing software-based spike counting for the 12-tile architecture of Figure 11 is 12 * 128 * 16 bits = 24Kb, with 128 post-synaptic neurons per tile. However, continuous snooping on the bus can introduce performance overhead. In the future, we plan to extend the interconnect routers to facilitate recording the spike packets. This will allow NCRTM to poll these readings periodically. With this necessary background, we now introduce our model for estimating aging (Section 5.3) and ISI (Section 5.4).
Equation 2 equates the MTTF of a CMOS transistor for a given overdrive voltage. BTI failures can also be modeled using the Weibull distribution with a scale parameter 𝛼 and a slope parameter 𝛽. Reliability at time 𝑡 can be written as
with the corresponding MTTF computed as
where Γ is the Gamma function. Using the expressions for MTTF from Equations 2 and 3, and rearranging, we obtain the expression for the scale parameter 𝛼 as
The aging (A), i.e., the degradation of the CMOS transistor can be expressed as
where the scaling factor 𝛼 (𝑉 𝑖 ) can be calculated using Eq. 5.
ACM J. Emerg We note that a neuron suffers aging when generating a spike. Each spike is of fixed voltage 𝑉 𝑠𝑝𝑘 (see Figure 4b) and a fixed time duration to the order of few ms. Therefore, both 𝑉 𝑖 and 𝑡 𝑖 in Equation 6 are constant. This allows us to express the aging formulation as
where 𝑛 is the number of spikes generated by the neuron, Δ𝑡 is the fixed spike duration, and 𝑉 is the fixed spike voltage. Equation 7 allows us to represent the aging in terms of the number of spikes generated by a neuron and the unit aging parameter Δ𝑡 𝛼 (𝑉 ) , which represents the aging per spike. This simplified formulation allows to estimate the aging in each neuron by simply counting the number of spikes it generates. Hence the performance overhead can be kept negligible.
To define ISI, we consider a tile consisting of 𝑁 post-synaptic neurons and a finite interval of time [0,𝑇 ] for which the tile is active without undergoing a de-stress operation. The post-synaptic neurons generate 𝐾 spikes in this interval, which are organized based on their generation time and the source neuron as
where 𝑡 𝑛 𝑖 is the time of the 𝑖 th spike generated by the 𝑛 th neuron and 𝐾 = 𝑁 𝑖=1 𝑘 𝑖 . The instantaneous ISI of the spike train from the 𝑛 th neuron is [43]
To estimate the impact of de-stress operation on ISI, we compute two statistics for each neuronthe instantaneous ISI (Equation 9) and the average ISI, which is computed as the average of all ISIs for a neuron. Using 𝑡𝐷𝑆𝐶 as the time to de-stress a tile, the change in instantaneous and average ISI of the 𝑛 th neuron are Δ𝐼𝑆𝐼 𝑛 inst (𝑖) = 𝐼𝑆𝐼 𝑛 inst (𝑖) + 𝑡𝐷𝑆𝐶 and Δ𝐼𝑆𝐼 𝑛 avg = 𝑡𝐷𝑆𝐶/𝑘 𝑁 (10)
Figure 13 illustrates our simulation framework. An SNN-based application is simulated using CARLsim [20], a GPU accelerated SNN simulator used to train and test SNN models. CARLsim reports spike times for every synapse in the SNN. The spike times are used to perform mapping explorations optimizing some objective, such as performance (PyCARL [8]) and reliability (RENEU [88]). We use NeuroXplorer [12], a cycle-accurate simulator of neuromorphic hardware such as DYNAP-SE [64].
The neuron and synapse mapping obtained using the mapping exploration framework is applied to NeuroXplorer to perform cycle-accurate simulation of the application on the hardware model, using current data. NCRTM is implemented inside NeuroXplorer to estimate the circuit aging and control it by de-stressing the circuit when the aging exceeds a threshold. The change in spike latency due to the de-stress operation can be precisely modeled in NeuroXplorer, as shown in [12]. All simulations are conducted on a system with 8 CPUs, 32GB RAM, and NVIDIA Tesla GPU, running Ubuntu 16.04.
We evaluated 10 machine learning applications that are representative of three most commonly used neural network classes -convolutional neural network (CNN), multi-layer perceptron (MLP), and recurrent neural network (RNN). These applications are 1) LeNet [56] based handwritten digit recognition with 28 × 28 images of handwritten digits from the MNIST dataset [37]; 2) AlexNet [52] for Imagenet classification [36]; 3) VGG16 [82], also for Imagenet classification [36]; 4) ECGbased heart-beat classification (HeartClass) [5,31] using electrocardiogram (ECG) data from the Physionet database [63]; 5) multi-layer perceptron (MLP)-based handwritten digit recognition (MLP-MNIST) [38] using the MNIST database; 6) image smoothing (ImgSmooth) [20] on 64 × 64 images; 7) edge detection (EdgeDet) [20] on 64 × 64 images using difference-of-Gaussian; 8) heart-rate estimation (HeartEstm) [22] using ECG data; 9) gender classification using speech data (SpeechRecog) [39]; and 10) RNN-based predictive visual pursuit (VisualPursuit) [49]. The former 7 are supervised applications, while the latter 3 are unsupervised applications. Table 1 summarizes the topology, the number of neurons and synapses of these applications, and their baseline accuracy on DYNAP-SE using PyCARL [8]. 6Table 1. Applications used to evaluate our approach NCRTM.
In our cycle-accurate simulator, we model the architecture of the DYNAP-SE neuromorphic hardware [64] with the following configurations.
• A tiled array, with each tile accommodating 128 input and 128 output neurons. There are 65,536 crosspoints in each tile. • Spikes are digitized and communicated between cores through a mesh routing network using the Address Event Representation (AER) protocol. The DYNAP-SE platform uses static random access memory (SRAM) to implement the synaptic cells in each crossbar. However, in our simulator, we use Phase-Change Memory (PCM) as the synaptic element. Table 2 reports the major hardware parameters.
In the future, we will demonstrate NCRTM on a real NVM-based silicon neuromorphic system.
We evaluate the following three approaches.
• PyCARL [8]: This is a performance-oriented approach to map SNN-based applications to neuromorphic hardware. This approach first generates clusters of neurons and synapses, where each cluster can fit on to the resources of a tile in the hardware. Then it uses an optimization algorithm to place these clusters to the hardware, maximizing performance of the learning application on the hardware. CMOS circuits are not de-stressed at run-time.
• RENEU [88]: This is a reliability-oriented approach to map SNN-based applications to neuromorphic hardware. This approach also generates clusters of neurons and synapses from an application, but maps the clusters to the hardware minimizing the maximum aging while considering only the training data. CMOS circuits are not de-stressed at run-time.
• DTRO [2]: This is a reliability-oriented approach where neuron and synapse circuits of the neuromorphic hardware are de-stressed at run-time at fixed interval. This interval is decided based on analysis performed at design-time using training data.
We evaluate the following metrics.
• Aging: This is the maximum circuit aging in DYNAP-SE for each machine learning workload.
• ISI: This is average ISI of each machine learning workload.
• Application Performance (accuracy): The performance, e.g., accuracy is defined in terms of misclassification rate for image-based CNN and MLP applications. For RNN applications that use time-series data, performance is measured in terms of error rate [22]. • Aging per unit ISI distortion: This is an unified metric reporting the aging per unit ISI distortion for each workload, defined as
In formulating the optimization objective of Equation 11, NCRTM aims to optimize (i.e., minimize) circuit aging A for a given constraint on the ISI distortion. In our earlier works [9,32], we have shown the dependency of application performance, e.g., accuracy on ISI due to inter-spike intervalbased information encoding in SNNs. Therefore, any distortion in ISI may lead to a reduction in performance [9,32]. Correspondingly, the above optimization problem essentially reduces to minimizing the aging A for a given constraint on SNN accuracy.
To compute aging, the slope parameter of Weibull distribution is set to 𝛽 = 2, and the operating temperature is set to 300𝐾. Other fitting parameters are adjusted to achieve an MTTF of 2 years in the baseline system (PyCARL), corresponding to a threshold voltage shift of 10%. This is what Dynamic Reliability Management in Neuromorphic Computing 1:17 is typically accepted by technologists as the maximum allowed degradation before timing errors begin to appear.
Table 3 summarizes the key results.
Figure 14 plots the aging of the neuron and synapse circuits in DYNAP-SE during the execution of the machine learning applications for each evaluated approach, normalized to PyCARL. We make the following three key observations. First, the aging due to RENEU is lower than PyCARL by an average of 2.5%. This improvement is due to the aging-aware neuron and synapse mapping policy of RENEU, which balances the aging of all tiles in the hardware. PyCARL, which balances the utilization of the tiles in the hardware, has higher aging. However, both PyCARL and RENEU are design-time based policies, i.e., they do not make any run-time decisions. Second, DTRO is a hybrid approach, which uses the neuron and synapse mapping of RENEU. Compared to PyCARL and RENEU, DTRO de-stresses all circuits in the hardware periodically at run-time. The de-stress interval is determined at design-time by analyzing the training data. The aging of DTRO is therefore lower than both PyCARL (average 35% lower) and RENEU (average 33.5% lower). Third, NCRTM, which is a run-time approach, has the lowest aging of all. The average aging of NCRTM is 74% lower than PyCARL, 73% lower than RENEU, and 60% lower than DTRO. The improvement of NCRTM is due to the precise tracking of aging at run-time using current data, to achieve a target MTTF.
Circuit aging manifests as shift in threshold voltage. Figure 15 plots the shift in threshold voltage (Δ𝑉 th ) in DYNAP-SE after executing each machine learning application continuously till the end of its lifetime of 2 years. We normalize the results so that the threshold voltage shift due to PyCARL is 10%. We make the following key observations. We observe that, compared to PyCARL, the average threshold voltage shift when using RENEU is 9.75%, DTRO is 7.0%, and NCRTM is only 4.8%. The threshold voltage shift is the lowest in NCRTM because the aging of NCRTM is lowest of all the approaches, which we reported in Section 7.2. Increase in threshold voltage results in the reduction in drive current, which in turn results in temporal performance degradation of neuron and synapse circuits in the neuromorphic hardware.
Figure 16 plots the ISI of the machine learning workloads on DYNAP-SE for each evaluated approach, normalized to PyCARL. We make the following five key observations. First, the ISI obtained with RENEU is similar to PyCARL. This is because RENEU generates a mapping of the workload to the hardware, which improves reliability without significantly hurting the performance. Since no run-time decisions are made in both these approaches, their performance at run-time are therefore similar. Second, the ISI obtained with DTRO is higher than RENEU by an average of 10%. This increase is because DTRO make run-time decision of de-stressing the neuron and synapse circuits periodically to control their aging. This leads to increase in latency, which increases the average ISI (see Equation 10). The advantage in this case is lower aging, which we analyzed in Section 7.2, leading to a lower drift of the threshold voltage (see Section 7.3). Third, the ISI using NCRTM is higher than RENEU by an average of 12%. This increase is due to the run-time de-stresses in NCRTM (similar to DTRO), which introduces latency, impacting the ISI. The ISI using NCRTM is only 2% higher than DTRO. This increase is because NCRTM never allows the aging to reach critical levels and therefore, schedules more de-stresses by precisely tracking it at run-time. However, due to NCRTM's policy to schedule the de-stresses by tracking their latency impact on ISI, NCRTM ensures only marginal change in ISI. ISI change may lead to accuracy impact, which is discussed in Section 7.5. Fourth, the ISI of NCRTM is lower than DTRO for MLP-MNIST. This is because for this application, the circuit aging is generally lower due to the sparsity of spike generation in the workload. So, the BTI stress is recovered in the idle period. DTRO cannot track this recovery and therefore, applies a conservative control, unnecessarily constraining the performance. Finally, for the three unsupervised applications (HeartEstm, SpeechRecog, and VisualPursuit), the ISI using NCRTM is on average 10% lower than DTRO. This is because in the absence of training data for these applications, DTRO applies a conservative policy to de-stress neuron and synapse circuits frequently to prevent their aging from reaching a critical value. NCRTM, on the other hand, tracks the aging at run-time based on the data that these models encounter and de-stress the circuits, only when needed.
We conclude that for machine learning workloads with sparse activation, NCRTM is significantly better than design-time based approaches both in terms of reliability and performance. For dense activation, NCRTM improves reliability significantly compared to these approaches, with marginal impact on performance. Furthermore, NCRTM outperforms any design-time based policy, when the availability of training data is limited.
Change in ISI manifests as loss in accucay of a machine learning workload. Table 4 reports the accuracy of each machine learning workload on DYNAP-SE using the evaluated approaches. We make the following three key observations. First, the accuracy of RENEU and PyCARL are the same. This is because RENEU maps neurons and synapses of an SNN workload to the hardware resources statically to minimize the aging. It does so, ensuring that the spike communication latency on the interconnect does not induce any change in ISI compared to that obtained using PyCARL. Second, the accuracy of NCRTM is on average 4.52% lower than PyCARL and RENEU, and 0.76% lower than DTRO. This reduction in accuracy is a direct result of the change in ISI, which we analyzed in Section 7.4. Third, although NCRTM results in 4.8% lower accuracy than Baseline for AlexNet (71.7% Baseline accuracy compared to 68.2% accuracy using NCRTM), it reduces circuit aging by 62% compared to Baseline (See Sec. 7.2).
Figure 17 illustrates the reliability impact of increasing the number of tiles in a neuromorphic hardware. The figure plots the aging results of NCRTM on DYNAP-SE with 16 and 32 tiles, normalized to the aging on DYNAP-SE with the baseline configuration of 12 tiles. We observe that the average aging with 16 and 32 tiles is 18% and 51% lower than the aging with baseline configuration of 12 tiles, respectively. Circuit aging is lower with more number of tiles. This is because with more tiles in the hardware, fewer neurons and synapses are mapped to each tile. Therefore, each tile of the hardware generates fewer spikes, which lowers the aging of its neurons and synapse circuits.
Figure 18 illustrates the temperature dependency of the aging in a neuromorphic hardware. We report the aging results of NCRTM at two elevated temperatures, 320K and 340K, for each of our machine learning applications. Aging results are normalized to NCRTM at 300K. We observe that aging increases with an increase in temperature. Aging at 320K and 340K is higher than the aging at 300K by an average of 7% and 30%, respectively. This is due to the exponential dependency of circuit aging on temperature (Equation 6). We also observe from this equation that aging also depends on the voltage needed to operate the neurons and synapses in the hardware when generating and propagating spikes. Therefore, VGG16, ImgSmooth, and VisualPursuit, which have more spikes, have higher aging at the elevated temperatures than all other applications. Higher aging leads to higher threshold voltage shift in the transistors.
This paper introduces NCRTM, a run-time reliability manager for neuromorphic computing. We observe that neurons and synapses in neuromorphic hardware are exposed to high voltages and/or currents because of the operating requirements of the Non-Volatile Memory, which are used for high density and low energy synaptic storage in the hardware. When exposed to these elevated conditions for too often, the CMOS transistors in the neuron and synapse circuit suffer strong aging, leading to hard breakdown. But in strongly scaled sub-10nm technology nodes, even under normal workloads, parametric soft breakdown mechanisms will start drifting the transistor parameters from their nominal values. In contrast to long-term aging, which permanently damages the hardware, shortterm aging in scaled CMOS transistors is mostly due to BTI. The latter is heavily workload-dependent and more importantly, partially reversible. Based on these observations, NCRTM dynamically destresses neuron and synapse circuits in response to the short-term aging in their CMOS transistors during the execution of machine learning tasks, with the objective of meeting a reliability target. NCRTM de-stresses these circuits only when it is absolutely necessary to do so, otherwise reducing the performance impact by scheduling de-stress operations off the critical path. We evaluate NCRTM with supervised and unsupervised machine learning applications on a neuromorphic hardware. Our results demonstrate that that for machine learning workloads with sparse activation, NCRTM is significantly better than design-time based approaches both in terms of reliability and performance. For dense activation, NCRTM improves reliability significantly compared to these approaches, with only marginal impact on performance. We conclude that NCRTM can be easily extended to incorporate other failure mechanisms. In the future, we plan on implementing NCRTM on NVM-based DYNAP-SE, when such board will be made publicly available.
This work is supported by the National Science Foundation Faculty Early Career Development Award CCF-1942697 (CAREER: Facilitating Dependable Neuromorphic Computing: Vision, Architecture, and Impact on Programmability).
Spiking Neural Networks (SNNs) [60] are regarded as the third generation of neural networks (see Figure 20a). SNNs consist of spiking neurons, which are implemented using integrate and fire [13] model. In this model, a neuron fires a spike when its membrane voltage exceeds a threshold and subsequently the membrane voltage is reset. The moment of threshold crossing in a neuron defines its firing time. Post firing, the neuron goes into a refractory state, where the neuron cannot be excited to generate a second action potential (no matter how intense the input stimulus be) (see Figure 20b). Spiking neurons are interconnected via synapses as shown in Figure 20a.
Information Encoding in SNNs: Information in SNNs can be encoded using different techniques [80], prominent among which are rate coding [96] and temporal coding [75]. Rate coding encodes information as number of spikes within an encoding window without considering the temporal characteristics of the signal. Temporal coding encodes information as inter-spike interval (ISI), capturing the spatio-temopral structure of the input signal.
Machine Learning Approaches using SNNs: SNNs can be used to implement many machine learning approaches. One example is the supervised approach, where an SNN is first trained with examples from the field and then used for inference with current data. SNNs can also implement unsupervised, semi-supervised, and reinforcement learning-based machine learning approaches.
Learning Algorithms in SNNs: Currently, spike-based learning rules are limited, compared to the wide range of learning rules available for analog or rate-based artificial neural networks (ANNs). Most learning rules are based on unsupervised correlational learning rules, such as spike timing dependent plasticity (STDP) [21], short-term plasticity (STP), and long-term plasticity (LTP). Other modifications include a localized version of backpropagation suitable for SNNs. However, these variants are supervisory and takes a while to converge. Attempts have been made to add a reinforcer to STDP based on the idea that dopamine in the brain carries a reward prediction error signal. In practice, dopamine modulated STDP (DA-STDP) takes a long time before the network has a strong enough signal to drive behavior [19]. Recently, a reward-modulated STDP (R-STDP) learning is developed to train SNN controllers for obstacle avoiding behavior in mobile robots [58].
To illustrate how ISI distortion and spike disorder impact accuracy, we consider a small SNN example where three input neurons are connected to an output neuron. In Figure 21, we illustrate the impact of ISI distortion on the output spike. In the top sub-figure, we observe that a spike is generated at the output neuron at 22ms due to spikes from the input neurons. In the bottom sub-figure, we observe that the second spike from input 3 is delayed, i.e., has ISI distortion. As a result of this distortion, there is no output spike. Missing spikes can impact application accuracy, as spike timings encode information in SNNs. of crossbars in the hardware, latency, ISI, and spike disorder increases. This is because with increase in the number of crossbars, spike traffic on the shared interconnect increases, which increases the congestion, and delays some spikes more than others. When we use a hardware with 36 small crossbars arranged in a 6x6 mesh, we observe a significant increase of latency (average 3.2x), ISI distortion (average 6x), and spike disorder (average 1.5x) compared to the baseline configuration of using 4 large crossbars.
From this analysis, we conclude that when more crossbars are used for an application, latency, ISI distortion, and spike disorder increases, which jointly impacts accuracy, as described next.
Accuracy Impact: To illustrate how ISI distortion and spike disorder impact accuracy, we consider a small SNN example where three input neurons are connected to an output neuron. In Figure 3a, we illustrate the impact of ISI distortion on the output spike. In the top sub-figure, we observe that a spike is generated at the output neuron at 22ms due to spikes from the input neurons. In the bottom sub-figure, we observe that the second spike from input 3 is delayed, i.e., has ISI distortion. As a result of this distortion, there is no output spike. Missing spikes can impact application accuracy, as spikes encode information in SNNs.
In Figure 3b, we illustrate the impact of spike disorder on the output spike. In the top sub-figure, we observe that the spike A from input 2 is generated before the spike B from input 3, causing an output spike to be generated at 21ms. In the bottom sub-figure, we observe that the spike order of inputs 2 and 3 is reversed, i.e., the spike B is generated before the spike A. This spike disorder results in no spike being generated at In Figure 5a, we illustrate is conventionally mapped to of these conventional approa neural networks, and custom NEUTRAMS [25] and Eyeri approach PACMAN [28] f large SNNs, (2) customize
We believe these architectures are similar for other designs like TrueNorth[35] and Loihi[34].
Beside neuromorphic computing, NVMs are also used as main memory for conventional computing[54,57,71,85,87,89,90].
BTI issues are also a reliability concern for standard DRAM and SRAM memories[67]. However, due to the use of transistors as access devices, the peripheral circuits in DRAM and SRAM can use lower operating voltages ≈ 1.2V. BTI-related reliability issues in DRAM and SRAM are therefore less severe than in NVM contexts[87,90].
There are works that address run-time management for conventional multi-core systems[81].
ACM J. Emerg. Technol. Comput. Syst., Vol. 1, No. 1, Article 1. Publication date: January 2021.
The aging threshold 𝑡ℎ 𝑎 is a user-defined parameter used to achieve a given reliability target. ACM J. Emerg. Technol. Comput. Syst., Vol. 1, No. 1, Article 1. Publication date: January 2021.
The CNN models LeNet, AlexNet, and VGG16 are converted to spiking domain using our previously proposed converter[5]. For the inference performance of the original model, readers are referred to[74].