More Like this

1. Detection Through Deep Neural Networks: A Reservoir Computing Approach for MIMO-OFDM Symbol Detection

   Kangjun Bai, Lingjia Liu (November 2020, IEEE International Conference on Computer-Aided Design)

   null (Ed.)

   The Reservoir Computing, a neural computing framework suited for temporal information processing, utilizes a dynamic reservoir layer for high-dimensional encoding, enhancing the separability of the network. In this paper, we exploit a Deep Learning (DL)-based detection strategy for Multiple-input, Multiple-output Orthogonal Frequency-Division Multiplexing (MIMO-OFDM) symbol detection. To be specific, we introduce a Deep Echo State Network (DESN), a unique hierarchical processing structure with multiple time intervals, to enhance the memory capacity and accelerate the detection efficiency. The resulting hardware prototype with the hybrid memristor-CMOS co-design provides in-memory computing and parallel processing capabilities, significantly reducing the hardware and power overhead. With the standard 180nm CMOS process and memristive synapses, the introduced DESN consumes merely 105mW of power consumption, exhibiting 16.7% power reduction compared to shallow ESN designs even with more dynamic layers and associated neurons. Furthermore, numerical evaluations demonstrate the advantages of the DESN over state-of-the-art detection techniques in the literate for MIMO-OFDM systems even with a very limited training set, yielding a 47.8% improvement against conventional symbol detection techniques. 

   more » « less
2. Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients

   https://doi.org/10.1109/SPAWC.2019.8815546  

   Rosenfeld, Bleema; Simeone, Osvaldo; Rajendran, Bipin (July 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC))

   Artificial Neural Networks (ANNs) are currently being used as function approximators in many state-of-the-art Reinforcement Learning (RL) algorithms. Spiking Neural Networks (SNNs) have been shown to drastically reduce the energy consumption of ANNs by encoding information in sparse temporal binary spike streams, hence emulating the communication mechanism of biological neurons. Due to their low energy consumption, SNNs are considered to be important candidates as co-processors to be implemented in mobile devices. In this work, the use of SNNs as stochastic policies is explored under an energy-efficient first-to-spike action rule, whereby the action taken by the RL agent is determined by the occurrence of the first spike among the output neurons. A policy gradient-based algorithm is derived considering a Generalized Linear Model (GLM) for spiking neurons. Experimental results demonstrate the capability of online trained SNNs as stochastic policies to gracefully trade energy consumption, as measured by the number of spikes, and control performance. Significant gains are shown as compared to the standard approach of converting an offline trained ANN into an SNN. 

   more » « less
3. A Sparse and Spike‐Timing‐Based Adaptive Photoencoder for Augmenting Machine Vision for Spiking Neural Networks

   https://doi.org/10.1002/adma.202202535  

   Subbulakshmi_Radhakrishnan, Shiva; Chakrabarti, Shakya; Sen, Dipanjan; Das, Mayukh; Schranghamer, Thomas_F; Sebastian, Amritanand; Das, Saptarshi (July 2022, Advanced Materials)

   Abstract The representation of external stimuli in the form of action potentials or spikes constitutes the basis of energy efficient neural computation that emerging spiking neural networks (SNNs) aspire to imitate. With recent evidence suggesting that information in the brain is more often represented by explicit firing times of the neurons rather than mean firing rates, it is imperative to develop novel hardware that can accelerate sparse and spike‐timing‐based encoding. Here a medium‐scale integrated circuit composed of two cascaded three‐stage inverters and one XOR logic gate fabricated using a total of 21 memtransistors based on photosensitive 2D monolayer MoS₂ for spike‐timing‐based encoding of visual information, is introduced. It is shown that different illumination intensities can be encoded into sparse spiking with time‐to‐first‐spike representing the illumination information, that is, higher intensities invoke earlier spikes and vice versa. In addition, non‐volatile and analog programmability in the photoencoder is exploited for adaptive photoencoding that allows expedited spiking under scotopic (low‐light) and deferred spiking under photopic (bright‐light) conditions, respectively. Finally, low energy expenditure of less than 1 µJ by the 2D‐memtransistor‐based photoencoder highlights the benefits of in‐sensor and bioinspired design that can be transformative for the acceleration of SNNs. 

   more » « less
4. Reservoir based spiking models for univariate Time Series Classification

   https://doi.org/10.3389/fncom.2023.1148284  

   Gaurav, Ramashish; Stewart, Terrence C.; Yi, Yang (June 2023, Frontiers in Computational Neuroscience)

   A variety of advanced machine learning and deep learning algorithms achieve state-of-the-art performance on various temporal processing tasks. However, these methods are heavily energy inefficient—they run mainly on the power hungry CPUs and GPUs. Computing with Spiking Networks, on the other hand, has shown to be energy efficient on specialized neuromorphic hardware, e.g., Loihi, TrueNorth, SpiNNaker, etc. In this work, we present two architectures of spiking models, inspired from the theory of Reservoir Computing and Legendre Memory Units, for the Time Series Classification (TSC) task. Our first spiking architecture is closer to the general Reservoir Computing architecture and we successfully deploy it on Loihi; the second spiking architecture differs from the first by the inclusion of non-linearity in the readout layer. Our second model (trained with Surrogate Gradient Descent method) shows that non-linear decoding of the linearly extracted temporal features through spiking neurons not only achieves promising results, but also offers low computation-overhead by significantly reducing the number of neurons compared to the popular LSM based models—more than 40x reduction with respect to the recent spiking model we compare with. We experiment on five TSC datasets and achieve new SoTA spiking results (—as much as 28.607% accuracy improvement on one of the datasets), thereby showing the potential of our models to address the TSC tasks in a green energy-efficient manner. In addition, we also do energy profiling and comparison on Loihi and CPU to support our claims. 

   more » « less
5. Spiking Neural Networks and Their Applications: A Review

   https://doi.org/10.3390/brainsci12070863  

   Yamazaki, Kashu; Vo-Ho, Viet-Khoa; Bulsara, Darshan; Le, Ngan (July 2022, Brain Sciences)

   The past decade has witnessed the great success of deep neural networks in various domains. However, deep neural networks are very resource-intensive in terms of energy consumption, data requirements, and high computational costs. With the recent increasing need for the autonomy of machines in the real world, e.g., self-driving vehicles, drones, and collaborative robots, exploitation of deep neural networks in those applications has been actively investigated. In those applications, energy and computational efficiencies are especially important because of the need for real-time responses and the limited energy supply. A promising solution to these previously infeasible applications has recently been given by biologically plausible spiking neural networks. Spiking neural networks aim to bridge the gap between neuroscience and machine learning, using biologically realistic models of neurons to carry out the computation. Due to their functional similarity to the biological neural network, spiking neural networks can embrace the sparsity found in biology and are highly compatible with temporal code. Our contributions in this work are: (i) we give a comprehensive review of theories of biological neurons; (ii) we present various existing spike-based neuron models, which have been studied in neuroscience; (iii) we detail synapse models; (iv) we provide a review of artificial neural networks; (v) we provide detailed guidance on how to train spike-based neuron models; (vi) we revise available spike-based neuron frameworks that have been developed to support implementing spiking neural networks; (vii) finally, we cover existing spiking neural network applications in computer vision and robotics domains. The paper concludes with discussions of future perspectives. 

   more » « less