More Like this

1. Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks

   Zhang, Wenrui; Li, Peng (December 2020, Advances in neural information processing systems)

   Spiking neural networks (SNNs) are well suited for spatio-temporal learning and implementations on energy-efficient event-driven neuromorphic processors. However, existing SNN error backpropagation (BP) methods lack proper handling of spiking discontinuities and suffer from low performance compared with the BP methods for traditional artificial neural networks. In addition, a large number of time steps are typically required to achieve decent performance, leading to high latency and rendering spike based computation unscalable to deep architectures. We present a novel Temporal Spike Sequence Learning Backpropagation (TSSL-BP) method for training deep SNNs, which breaks down error backpropagation across two types of inter-neuron and intra-neuron dependencies and leads to improved temporal learning precision. It captures inter-neuron dependencies through presynaptic firing times by considering the all-or-none characteristics of firing activities, and captures intra-neuron dependencies by handling the internal evolution of each neuronal state in time. TSSL-BP efficiently trains deep SNNs within a much shortened temporal window of a few steps while improving the accuracy for various image classification datasets including CIFAR10. 

   more » « less
2. Causal Spike Timing-Dependent Plasticity Prevents Assembly Fusion in Recurrent Networks

   https://doi.org/10.1103/9nv2-y63v  

   Yang, Xinruo; Doiron, Brent (January 2026, PRX Life)

   The organization of neurons into functionally related assemblies is a fundamental feature of cortical networks, yet our understanding of how these assemblies maintain distinct identities while sharing members remains limited. Here we analyze how spike timing-dependent plasticity (STDP) shapes the formation and stability of overlapping neuronal assemblies in recurrently coupled networks of spiking neuron models. Using numerical simulations and an associated mean-field theory, we demonstrate that the temporal structure of the STDP rule, specifically its degree of causality, critically determines whether assemblies that share neurons maintain segregation or merge together after training is completed. We find that causal STDP rules, where potentiation or depression occurs strictly when presynaptic spikes precede or follow postsynaptic spikes, allow assemblies to remain distinct even with substantial overlap in membership. This stability arises because causal STDP effectively cancels the symmetric correlations introduced by common inputs from shared neurons. In contrast, acausal STDP rules lead to assembly fusion when overlap exceeds a critical threshold, due to unchecked growth of common input correlations. Our results provide theoretical insight into how spike timing-dependent learning rules can support distributed representations where individual neurons participate in multiple assemblies while maintaining functional specificity. 

   more » « less
3. Robust point-process Granger causality analysis in presence of exogenous temporal modulations and trial-by-trial variability in spike trains

   https://doi.org/10.1371/journal.pcbi.1007675  

   Casile, Antonino; Faghih, Rose T.; Brown, Emery N. (January 2021, PLOS Computational Biology)

   Morrison, Abigail (Ed.)

   Assessing directional influences between neurons is instrumental to understand how brain circuits process information. To this end, Granger causality, a technique originally developed for time-continuous signals, has been extended to discrete spike trains. A fundamental assumption of this technique is that the temporal evolution of neuronal responses must be due only to endogenous interactions between recorded units, including self-interactions. This assumption is however rarely met in neurophysiological studies, where the response of each neuron is modulated by other exogenous causes such as, for example, other unobserved units or slow adaptation processes. Here, we propose a novel point-process Granger causality technique that is robust with respect to the two most common exogenous modulations observed in real neuronal responses: within-trial temporal variations in spiking rate and between-trial variability in their magnitudes. This novel method works by explicitly including both types of modulations into the generalized linear model of the neuronal conditional intensity function (CIF). We then assess the causal influence of neuron i onto neuron j by measuring the relative reduction of neuron j ’s point process likelihood obtained considering or removing neuron i . CIF’s hyper-parameters are set on a per-neuron basis by minimizing Akaike’s information criterion. In synthetic data sets, generated by means of random processes or networks of integrate-and-fire units, the proposed method recovered with high accuracy, sensitivity and robustness the underlying ground-truth connectivity pattern. Application of presently available point-process Granger causality techniques produced instead a significant number of false positive connections. In real spiking responses recorded from neurons in the monkey pre-motor cortex (area F5), our method revealed many causal relationships between neurons as well as the temporal structure of their interactions. Given its robustness our method can be effectively applied to real neuronal data. Furthermore, its explicit estimate of the effects of unobserved causes on the recorded neuronal firing patterns can help decomposing their temporal variations into endogenous and exogenous components. 

   more » « less
4. Robust computation with rhythmic spike patterns

   https://doi.org/10.1073/pnas.1902653116  

   Frady, E. Paxon; Sommer, Friedrich T. (August 2019, Proceedings of the National Academy of Sciences)

   Information coding by precise timing of spikes can be faster and more energy efficient than traditional rate coding. However, spike-timing codes are often brittle, which has limited their use in theoretical neuroscience and computing applications. Here, we propose a type of attractor neural network in complex state space and show how it can be leveraged to construct spiking neural networks with robust computational properties through a phase-to-timing mapping. Building on Hebbian neural associative memories, like Hopfield networks, we first propose threshold phasor associative memory (TPAM) networks. Complex phasor patterns whose components can assume continuous-valued phase angles and binary magnitudes can be stored and retrieved as stable fixed points in the network dynamics. TPAM achieves high memory capacity when storing sparse phasor patterns, and we derive the energy function that governs its fixed-point attractor dynamics. Second, we construct 2 spiking neural networks to approximate the complex algebraic computations in TPAM, a reductionist model with resonate-and-fire neurons and a biologically plausible network of integrate-and-fire neurons with synaptic delays and recurrently connected inhibitory interneurons. The fixed points of TPAM correspond to stable periodic states of precisely timed spiking activity that are robust to perturbation. The link established between rhythmic firing patterns and complex attractor dynamics has implications for the interpretation of spike patterns seen in neuroscience and can serve as a framework for computation in emerging neuromorphic devices. 

   more » « less
5. Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network

   https://doi.org/10.24963/ijcai.2020/388  

   Fang, Haowen; Shrestha, Amar; Zhao, Ziyi; Qiu, Qinru (July 2020, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence)

   The recently discovered spatial-temporal information processing capability of bio-inspired Spiking neural networks (SNN) has enabled some interesting models and applications. However designing large-scale and high-performance model is yet a challenge due to the lack of robust training algorithms. A bio-plausible SNN model with spatial-temporal property is a complex dynamic system. Synapses and neurons behave as filters capable of preserving temporal information. As such neuron dynamics and filter effects are ignored in existing training algorithms, the SNN downgrades into a memoryless system and loses the ability of temporal signal processing. Furthermore, spike timing plays an important role in information representation, but conventional rate-based spike coding models only consider spike trains statistically, and discard information carried by its temporal structures. To address the above issues, and exploit the temporal dynamics of SNNs, we formulate SNN as a network of infinite impulse response (IIR) filters with neuron nonlinearity. We proposed a training algorithm that is capable to learn spatial-temporal patterns by searching for the optimal synapse filter kernels and weights. The proposed model and training algorithm are applied to construct associative memories and classifiers for synthetic and public datasets including MNIST, NMNIST, DVS 128 etc. Their accuracy outperforms state-of-the-art approaches. 

   more » « less