An official website of the United States government
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.
more »
« less
Nonlinear Ion Dynamics Enable Spike Timing Dependent Plasticity of Electrochemical Ionic Synapses
Abstract Programmable synaptic devices that can achieve timing‐dependent weight updates are key components to implementing energy‐efficient spiking neural networks (SNNs). Electrochemical ionic synapses (EIS) enable the programming of weight updates with very low energy consumption and low variability. Here, the strongly nonlinear kinetics of EIS, arising from nonlinear dynamics of ions and charge transfer reactions in solids, are leveraged to implement various forms of spike‐timing‐dependent plasticity (STDP). In particular, protons are used as the working ion. Different forms of the STDP function are deterministically predicted and emulated by a linear superposition of appropriately designed pre‐ and post‐synaptic neuron signals. Heterogeneous STDP is also demonstrated within the array to capture different learning rules in the same system. STDP timescales are controllable, ranging from milliseconds to nanoseconds. The STDP resulting from EIS has lower variability than other hardware STDP implementations, due to the deterministic and uniform insertion of charge in the tunable channel material. The results indicate that the ion and charge transfer dynamics in EIS can enable bio‐plausible synapses for SNN hardware with high energy efficiency, reliability, and throughput.
more »
« less
- Award ID(s):
- 2235462
- PAR ID:
- 10580069
- Publisher / Repository:
- Wiley Advanced
- Date Published:
- Journal Name:
- Advanced Materials
- Volume:
- 37
- Issue:
- 10
- ISSN:
- 0935-9648
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.more » « less
-
The Liquid State Machine (LSM) is a promising model of recurrent spiking neural networks. It consists of a fixed recurrent network, or the reservoir, which projects to a readout layer through plastic readout synapses. The classification performance is highly dependent on the training of readout synapses which tend to be very dense and contribute significantly to the overall network complexity. We present a unifying biologically inspired calcium-modulated supervised spike-timing dependent plasticity (STDP) approach to training and sparsification of readout synapses, where supervised temporal learning is modulated by the post-synaptic firing level characterized by the post-synaptic calcium concentration. The proposed approach prevents synaptic weight saturation, boosts learning performance, and sparsifies the connectivity between the reservoir and readout layer. Using the recognition rate of spoken English letters adopted from the TI46 speech corpus as a measure of performance, we demonstrate that the proposed approach outperforms a baseline supervised STDP mechanism by up to 25%, and a competitive non-STDP spike-dependent training algorithm by up to 2.7%. Furthermore, it can prune out up to 30% of readout synapses without causing significant performance degradation.more » « less
-
How do sensory systems optimize detection of behaviorally relevant stimuli when the sensory environment is constantly changing? We addressed the role of spike timing-dependent plasticity (STDP) in driving changes in synaptic strength in a sensory pathway and whether those changes in synaptic strength could alter sensory tuning. It is challenging to precisely control temporal patterns of synaptic activity in vivo and replicate those patterns in vitro in behaviorally relevant ways. This makes it difficult to make connections between STDP-induced changes in synaptic physiology and plasticity in sensory systems. Using the mormyrid species Brevimyrus niger and Brienomyrus brachyistius, which produce electric organ discharges for electrolocation and communication, we can precisely control the timing of synaptic input in vivo and replicate these same temporal patterns of synaptic input in vitro. In central electrosensory neurons in the electric communication pathway, using whole cell intracellular recordings in vitro, we paired presynaptic input with postsynaptic spiking at different delays. Using whole cell intracellular recordings in awake, behaving fish, we paired sensory stimulation with postsynaptic spiking using the same delays. We found that Hebbian STDP predictably alters sensory tuning in vitro and is mediated by NMDA receptors. However, the change in synaptic responses induced by sensory stimulation in vivo did not adhere to the direction predicted by the STDP observed in vitro. Further analysis suggests that this difference is influenced by polysynaptic activity, including inhibitory interneurons. Our findings suggest that STDP rules operating at identified synapses may not drive predictable changes in sensory responses at the circuit level. NEW & NOTEWORTHY We replicated behaviorally relevant temporal patterns of synaptic activity in vitro and used the same patterns during sensory stimulation in vivo. There was a Hebbian spike timing-dependent plasticity (STDP) pattern in vitro, but sensory responses in vivo did not shift according to STDP predictions. Analysis suggests that this disparity is influenced by differences in polysynaptic activity, including inhibitory interneurons. These results suggest that STDP rules at synapses in vitro do not necessarily apply to circuits in vivo.more » « less
-
In spike-timing-dependent plasticity (STDP), synap-tic weights are modified according to the relative time difference between pre and post-synaptic spikes of spiking neural network (SNN). A triplet STDP model was proposed since this model can better take account of a series of spikes and thus more closely mimic the activity in biological neural systems. Circuit that can switch between different STDP rules was also introduced to improve the range of STDP applications. To apply the advantages of triplet STDP to various tasks, a mixed-signal triplet reconfigurable STDP circuit and its hardware prototype are proposed in this paper. The performance analysis of the STDP training algorithm is carried out with a hardware testbench as well as Pytorch-based SNN. This triplet STDP design achieves 3.28% and 3.63% higher accuracy than the pair STDP learning rule through datasets such as MNIST and CIFAR-10. Our design shows one of the best reconfigurability while keeping a relatively low energy per spike operation (SOP) through the performance comparison with the state of the arts.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1002/adma.202418484
