More Like this

1. Spiking Neural Networks with Laterally-Inhibited Self-Recurrent Units

   Zheng, Wenrui; Li, Peng (July 2021, 2021 International Joint Conference on Neural Networks)

   null (Ed.)

   In biological brains, recurrent connections play a crucial role in cortical computation, modulation of network dynamics, and communication. However, in recurrent spiking neural networks (SNNs), recurrence is mostly constructed by random connections. How excitatory and inhibitory recurrent connections affect network responses and what kinds of connectivity benefit learning performance is still obscure. In this work, we propose a novel recurrent structure called the Laterally-Inhibited Self-Recurrent Unit (LISR), which consists of one excitatory neuron with a self-recurrent connection wired together with an inhibitory neuron through excitatory and inhibitory synapses. The self-recurrent connection of the excitatory neuron mitigates the information loss caused by the firing-and-resetting mechanism and maintains the long-term neuronal memory. The lateral inhibition from the inhibitory neuron to the corresponding excitatory neuron, on the one hand, adjusts the firing activity of the latter. On the other hand, it plays as a forget gate to clear the memory of the excitatory neuron. Based on speech and image datasets commonly used in neuromorphic computing, RSNNs based on the proposed LISR improve performance significantly by up to 9.26% over feedforward SNNs trained by a state-of-the-art backpropagation method with similar computational costs. 

   more » « less
2. Skip-Connected Self-Recurrent Spiking Neural Networks With Joint Intrinsic Parameter and Synaptic Weight Training

   https://doi.org/10.1162/neco_a_01393  

   Zhang, Wenrui; Li, Peng (June 2021, Neural Computation)

   null (Ed.)

   Abstract As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. First, due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. Second, training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNN, skip-connected self-recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs, while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method, backpropagated intrinsic plasticity (BIP), to boost the performance of ScSr-SNNs further by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Based on challenging speech, neuromorphic speech, and neuromorphic image data sets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods. 

   more » « less
3. Robust point-process Granger causality analysis in presence of exogenous temporal modulations and trial-by-trial variability in spike trains

   https://doi.org/10.1371/journal.pcbi.1007675  

   Casile, Antonino; Faghih, Rose T.; Brown, Emery N. (January 2021, PLOS Computational Biology)

   Morrison, Abigail (Ed.)

   Assessing directional influences between neurons is instrumental to understand how brain circuits process information. To this end, Granger causality, a technique originally developed for time-continuous signals, has been extended to discrete spike trains. A fundamental assumption of this technique is that the temporal evolution of neuronal responses must be due only to endogenous interactions between recorded units, including self-interactions. This assumption is however rarely met in neurophysiological studies, where the response of each neuron is modulated by other exogenous causes such as, for example, other unobserved units or slow adaptation processes. Here, we propose a novel point-process Granger causality technique that is robust with respect to the two most common exogenous modulations observed in real neuronal responses: within-trial temporal variations in spiking rate and between-trial variability in their magnitudes. This novel method works by explicitly including both types of modulations into the generalized linear model of the neuronal conditional intensity function (CIF). We then assess the causal influence of neuron i onto neuron j by measuring the relative reduction of neuron j ’s point process likelihood obtained considering or removing neuron i . CIF’s hyper-parameters are set on a per-neuron basis by minimizing Akaike’s information criterion. In synthetic data sets, generated by means of random processes or networks of integrate-and-fire units, the proposed method recovered with high accuracy, sensitivity and robustness the underlying ground-truth connectivity pattern. Application of presently available point-process Granger causality techniques produced instead a significant number of false positive connections. In real spiking responses recorded from neurons in the monkey pre-motor cortex (area F5), our method revealed many causal relationships between neurons as well as the temporal structure of their interactions. Given its robustness our method can be effectively applied to real neuronal data. Furthermore, its explicit estimate of the effects of unobserved causes on the recorded neuronal firing patterns can help decomposing their temporal variations into endogenous and exogenous components. 

   more » « less
4. Alpha modulation of spiking activity across multiple brain regions in mice performing a tactile selective detection task

   https://doi.org/10.1101/2025.03.07.642074  

   Kelley, Craig; Slater, Cody; Sorrentino, Marc; Noone, Dillon; Hung, Jocelyn; Sajda, Paul; Wang, Qi (March 2025, bioRxiv)

   Abstract Many cognitive and sensory processes are characterized by strong relationships between the timing of neuronal spiking and the phase of ongoing local field potential oscillations. The coupling of neuronal spiking in neocortex to the phase of alpha oscillations (8-12 Hz) has been well studied in nonhuman primates but remains largely unexplored in other mammals. How this alpha modulation of spiking differs between brain areas and cell types, as well as its role in sensory processing and decision making, are not well understood. We used Neuropixels 1.0 probes to chronically record neural activity from somatosensory cortex, prefrontal cortex, striatum, and amygdala in mice performing a whisker-based selective detection task. We observed strong spontaneous alpha modulation of single-neuron spiking activity during inter-trial intervals while mice performed the task. The prevalence and strength of alpha phase modulation differed significantly across regions and between cell types. Phase modulated neurons exhibited stronger responses to both go and no-go stimuli, as well as stronger motor- and reward-related changes in firing rate, than their unmodulated counterparts. The increased responsiveness of phase modulated neurons suggests they are innervated by more diverse populations. Alpha modulation of neuronal spiking during baseline activity also correlated with task performance. In particular, many neurons exhibited strong alpha modulation before correct trials, but not before incorrect trials. These data suggest that dysregulation of spiking activity with respect to alpha oscillations may characterize lapses in attention. 

   more » « less
5. An Ultra-Low Power Spintronic Stochastic Spiking Neuron with Self-Adaptive Discrete Sampling

   Sheikhfaal, Shadi; Pyle, Steven D.; Salehi, Soheil; DeMara, Ronald F. (August 2019, IEEE International Midwest Symposium on Circuits and Systems (MWSCAS))

   State-of-the-art machine learning models have achieved impressive feats of narrow intelligence, but have yet to realize the computational generality, adaptability, and power efficiency of biological brains. Thus, this work aims to improve current neural network models by leveraging the principle that the cortex consists of noisy and imprecise components in order to realize an ultra-low-power stochastic spiking neural circuit that resembles biological neuronal behavior. By utilizing probabilistic spintronics to provide true stochasticity in a compact CMOS-compatible device, an Adaptive Ring Oscillator for as-needed discrete sampling, and a homeostasis mechanism to reduce power consumption, provide additional biological characteristics, and improve process variation resilience, this subthreshold circuit is able to generate sub-nanosecond spiking behavior with biological characteristics at 200mV, using less than 80nW, along with behavioral robustness to process variation. 

   more » « less