Many controlled
LODeNNS: A Linearly-approximated and Optimized Dendrocentric Nearest Neighbor STDP
Realizing Hebbian plasticity in large-scale neuromorphic systems is essential for reconfiguring them for recognition tasks. Spike-timing-dependent plasticity, as a tool to this effect, has received a lot of attention in recent times. This phenomenon encodes weight update information as correlations between the presynaptic and postsynaptic event times, as such, it is imperative for each synapse in a silicon neural network to somehow keep its own time. We present a biologically plausible and optimized Register Transfer Level (RTL) and algorithmic approach to the Nearest-Neighbor STDP with time management handled by the postsynaptic dendrite. We adopt a time-constant based ramp approximation for ease of RTL implementation and incorporation in large-scale digital neuromorphic systems.
more »
« less
- Award ID(s):
- 1824198
- NSF-PAR ID:
- 10376848
- Date Published:
- Journal Name:
- ICONS '22: Proceedings of the International Conference on Neuromorphic Systems 2022
- Page Range / eLocation ID:
- 1 to 8
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
in vitro studies have demonstrated how postsynaptic responses to presynaptic spikes are not constant but depend on short-term synaptic plasticity (STP) and the detailed timing of presynaptic spikes. However, the effects of short-term plasticity (depression and facilitation) are not limited to short, subsecond timescales. The effects of STP appear on long timescales as changes in presynaptic firing rates lead to changes in steady-state synaptic transmission. Here, we examine the relationship between natural variations in the presynaptic firing rates and spike transmissionin vivo . Using large-scale spike recordings in awake male and female mice from the Allen Institute Neuropixels dataset, we first detect putative excitatory synaptic connections based on cross-correlations between the spike trains of millions of pairs of neurons. For the subset of pairs where a transient, excitatory effect was detected, we use a model-based approach to track fluctuations in synaptic efficacy and find that efficacy varies substantially on slow (∼1 min) timescales over the course of these recordings. For many connections, the efficacy fluctuations are correlated with fluctuations in the presynaptic firing rate. To understand the potential mechanisms underlying this relationship, we then model the detailed probability of postsynaptic spiking on a millisecond timescale, including both slow changes in postsynaptic excitability and monosynaptic inputs with short-term plasticity. The detailed model reproduces the slow efficacy fluctuations observed with many putative excitatory connections, suggesting that these fluctuations can be both directly predicted based on the time-varying presynaptic firing rate and, at least partly, explained by the cumulative effects of STP.SIGNIFICANCE STATEMENT The firing rates of individual neurons naturally vary because of stimuli, movement, and brain state. Models of synaptic transmission predict that these variations in firing rates should be accompanied by slow fluctuations in synaptic strength because of short-term depression and facilitation. Here, we characterize the magnitude and predictability of fluctuations in synaptic strengthin vivo using large-scale spike recordings. For putative excitatory connections from a wide range of brain areas, we find that typical synaptic efficacy varies as much as ∼70%, and in many cases the fluctuations are well described by models of short-term synaptic plasticity. These results highlight the dynamic nature ofin vivo synaptic transmission and the interplay between synaptic strength and firing rates in awake animals. -
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.more » « less
-
The Artificial Intelligence (AI) disruption continues unabated, albeit at extreme compute requirements. Neuromorphic circuits and systems offer a panacea for this extravagance. To this effect, event-based learning such as spike-timing-dependent plasticity (STDP) in spiking neural networks (SNNs) is an active area of research. Hebbian learning in SNNs fundamentally involves synaptic weight updates based on temporal correlations between pre- and post- synaptic neural activities. While there are broadly two approaches of realizing STDP, i.e. All-to-All versus Nearest Neighbor (NN), there exist strong arguments favoring the NN approach on the biologically plausibility front. In this paper, we present a novel current-mode implementation of a postsynaptic event-based NN STDP-based synapse. We leverage transistor subthreshold dynamics to generate exponential STDP traces using repurposed log-domain low-pass filter circuits. Synaptic weight operations involving addition and multiplications are achieved by the Kirchoff current law and the translinear principle respectively. Simulation results from the NCSU TSMC 180 nm technology are presented. Finally, the ideas presented here hold implications for engineering efficient hardware to meet the growing AI training and inference demands.more » « less
-
more » « less
Abstract Spike-Timing-Dependent Plasticity (STDP) is an unsupervised learning mechanism for Spiking Neural Networks (SNNs) that has received significant attention from the neuromorphic hardware community. However, scaling such local learning techniques to deeper networks and large-scale tasks has remained elusive. In this work, we investigate a Deep-STDP framework where a rate-based convolutional network, that can be deployed in a neuromorphic setting, is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs. We achieve $24.56\%$ higher accuracy and $3.5\times$ faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset in contrast to a $k$-means clustering approach.
-
How do sensory systems optimize detection of behaviorally relevant stimuli when the sensory environment is constantly changing? We addressed the role of spike timing-dependent plasticity (STDP) in driving changes in synaptic strength in a sensory pathway and whether those changes in synaptic strength could alter sensory tuning. It is challenging to precisely control temporal patterns of synaptic activity in vivo and replicate those patterns in vitro in behaviorally relevant ways. This makes it difficult to make connections between STDP-induced changes in synaptic physiology and plasticity in sensory systems. Using the mormyrid species Brevimyrus niger and Brienomyrus brachyistius, which produce electric organ discharges for electrolocation and communication, we can precisely control the timing of synaptic input in vivo and replicate these same temporal patterns of synaptic input in vitro. In central electrosensory neurons in the electric communication pathway, using whole cell intracellular recordings in vitro, we paired presynaptic input with postsynaptic spiking at different delays. Using whole cell intracellular recordings in awake, behaving fish, we paired sensory stimulation with postsynaptic spiking using the same delays. We found that Hebbian STDP predictably alters sensory tuning in vitro and is mediated by NMDA receptors. However, the change in synaptic responses induced by sensory stimulation in vivo did not adhere to the direction predicted by the STDP observed in vitro. Further analysis suggests that this difference is influenced by polysynaptic activity, including inhibitory interneurons. Our findings suggest that STDP rules operating at identified synapses may not drive predictable changes in sensory responses at the circuit level. NEW & NOTEWORTHY We replicated behaviorally relevant temporal patterns of synaptic activity in vitro and used the same patterns during sensory stimulation in vivo. There was a Hebbian spike timing-dependent plasticity (STDP) pattern in vitro, but sensory responses in vivo did not shift according to STDP predictions. Analysis suggests that this disparity is influenced by differences in polysynaptic activity, including inhibitory interneurons. These results suggest that STDP rules at synapses in vitro do not necessarily apply to circuits in vivo.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3546790.3546793
