An official website of the United States government
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.
more »
« less
Nonlinear Ion Dynamics Enable Spike Timing Dependent Plasticity of Electrochemical Ionic Synapses
Abstract Programmable synaptic devices that can achieve timing‐dependent weight updates are key components to implementing energy‐efficient spiking neural networks (SNNs). Electrochemical ionic synapses (EIS) enable the programming of weight updates with very low energy consumption and low variability. Here, the strongly nonlinear kinetics of EIS, arising from nonlinear dynamics of ions and charge transfer reactions in solids, are leveraged to implement various forms of spike‐timing‐dependent plasticity (STDP). In particular, protons are used as the working ion. Different forms of the STDP function are deterministically predicted and emulated by a linear superposition of appropriately designed pre‐ and post‐synaptic neuron signals. Heterogeneous STDP is also demonstrated within the array to capture different learning rules in the same system. STDP timescales are controllable, ranging from milliseconds to nanoseconds. The STDP resulting from EIS has lower variability than other hardware STDP implementations, due to the deterministic and uniform insertion of charge in the tunable channel material. The results indicate that the ion and charge transfer dynamics in EIS can enable bio‐plausible synapses for SNN hardware with high energy efficiency, reliability, and throughput.
more »
« less
- Award ID(s):
- 2235462
- PAR ID:
- 10578115
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Advanced Materials
- Volume:
- 37
- Issue:
- 10
- ISSN:
- 0935-9648
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.more » « less
-
The Liquid State Machine (LSM) is a promising model of recurrent spiking neural networks. It consists of a fixed recurrent network, or the reservoir, which projects to a readout layer through plastic readout synapses. The classification performance is highly dependent on the training of readout synapses which tend to be very dense and contribute significantly to the overall network complexity. We present a unifying biologically inspired calcium-modulated supervised spike-timing dependent plasticity (STDP) approach to training and sparsification of readout synapses, where supervised temporal learning is modulated by the post-synaptic firing level characterized by the post-synaptic calcium concentration. The proposed approach prevents synaptic weight saturation, boosts learning performance, and sparsifies the connectivity between the reservoir and readout layer. Using the recognition rate of spoken English letters adopted from the TI46 speech corpus as a measure of performance, we demonstrate that the proposed approach outperforms a baseline supervised STDP mechanism by up to 25%, and a competitive non-STDP spike-dependent training algorithm by up to 2.7%. Furthermore, it can prune out up to 30% of readout synapses without causing significant performance degradation.more » « less
-
Abstract Memristive devices have demonstrated rich switching behaviors that closely resemble synaptic functions and provide a building block to construct efficient neuromorphic systems. It is demonstrated that resistive switching effects are controlled not only by the external field, but also by the dynamics of various internal state variables that facilitate the ionic processes. The internal temperature, for example, works as a second‐state variable to regulate the ion motion and provides the internal timing mechanism for the native implementation of timing‐ and rate‐based learning rules such as spike timing dependent plasticity (STDP). In this work, it is shown that the 2nd state‐variable in a Ta2O5‐based memristor, its internal temperature, can be systematically engineered by adjusting the material properties and device structure, leading to tunable STDP characteristics with different time constants. When combined with an artificial post‐synaptic neuron, the 2nd‐order memristor synapses can spontaneously capture the temporal correlation in the input streaming events.more » « less
-
Neuromorphic computing has the great potential to enable faster and more energy‐efficient computing by overcoming the von Neumann bottleneck. However, most emerging nonvolatile memory (NVM)‐based artificial synapses suffer from insufficient precision, nonlinear synaptic weight update, high write voltage, and high switching latency. Moreover, the spatiotemporal dynamics, an important temporal component for cognitive computing in spiking neural networks (SNNs), are hard to generate with existing complementary metal–oxide–semiconductor (CMOS) devices or emerging NVM. Herein, a three‐terminal, LixWO3‐based electrochemical synapse (LiWES) is developed with low programming voltage (0.2 V), fast programming speed (500 ns), and high precision (1024 states) that is ideal for artificial neural networks applications. Time‐dependent synaptic functions such as paired‐pulse facilitation (PPF) and temporal filtering that are critical for SNNs are also demonstrated. In addition, by leveraging the spike‐encoded timing information extracted from the short‐term plasticity (STP) behavior in the LiWES, an SNNs model is built to benchmark the pattern classification performance of the LiWES, and the result indicates a large boost in classification performance (up to 128×), compared with those NO‐STP synapses.more » « less
