Combinatorial optimization problems prevail in engineering and industry. Some are NP-hard and thus become difficult to solve on edge devices due to limited power and computing resources. Quadratic Unconstrained Binary Optimization (QUBO) problem is a valuable emerging model that can formulate numerous combinatorial problems, such as Max-Cut, traveling salesman problems, and graphic coloring. QUBO model also reconciles with two emerging computation models, quantum computing and neuromorphic computing, which can potentially boost the speed and energy efficiency in solving combinatorial problems. In this work, we design a neuromorphic QUBO solver composed of a swarm of spiking neural networks (SNN) that conduct a population-based meta-heuristic search for solutions. The proposed model can achieve about x20 40 speedup on large QUBO problems in terms of time steps compared to a traditional neural network solver. As a codesign, we evaluate the neuromorphic swarm solver on a 40nm 25mW Resistive RAM (RRAM) Compute-in-Memory (CIM) SoC with a 2.25MB RRAM-based accelerator and an embedded Cortex M3 core. The collaborative SNN swarm can fully exploit the specialty of CIM accelerator in matrix and vector multiplications. Compared to previous works, such an algorithm-hardware synergized solver exhibits advantageous speed and energy efficiency for edge devices.
more »
« less
A Spiking Neuromorphic Architecture Using Gated-RRAM for Associative Memory
This work reports a spiking neuromorphic architecture for associative memory simulated in a SPICE environment using recently reported gated-RRAM (resistive random-access memory) devices as synapses alongside neurons based on complementary metal-oxide semiconductors (CMOSs). The network utilizes a Verilog A model to capture the behavior of the gated-RRAM devices within the architecture. The model uses parameters obtained from experimental gated-RRAM devices that were fabricated and tested in this work. Using these devices in tandem with CMOS neuron circuitry, our results indicate that the proposed architecture can learn an association in real time and retrieve the learned association when incomplete information is provided. These results show the promise for gated-RRAM devices for associative memory tasks within a spiking neuromorphic architecture framework.
more »
« less
- PAR ID:
- 10386171
- Date Published:
- Journal Name:
- ACM Journal on Emerging Technologies in Computing Systems
- Volume:
- 18
- Issue:
- 2
- ISSN:
- 1550-4832
- Page Range / eLocation ID:
- 1 to 22
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Resistive random-access memory (RRAM) devices have been widely studied for neuromorphic, in-memory computing. One of the most studied RRAM structures consists of a titanium capping layer and a HfOx adaptive oxide. Although these devices show promise in improving neuromorphic circuits, high variability, non-linearity, and asymmetric resistance changes limit their usefulness. Many studies have improved linearity by changing materials in or around the device, the circuitry, or the analog bias conditions. However, the impact of prior biasing conditions on the observed analog resistance change is not well understood. Experimental results in this study demonstrate that prior higher reset voltages used after forming cause a greater resistance change during subsequent identical analog pulsing. A multiphysics finite element model suggests that this greater analog resistance change is due to a higher concentration of oxygen ions stored in the titanium capping layer with increasing magnitude of the reset voltage. This work suggests that local ion concentration variations in the titanium capping layer of just tens of atoms cause significant resistance variation during analog operation.
-
more » « less
Abstract Spiking neural networks exploit spatiotemporal processing, spiking sparsity, and high interneuron bandwidth to maximize the energy efficiency of neuromorphic computing. While conventional silicon-based technology can be used in this context, the resulting neuron-synapse circuits require multiple transistors and complicated layouts that limit integration density. Here, we demonstrate unprecedented electrostatic control of dual-gated Gaussian heterojunction transistors for simplified spiking neuron implementation. These devices employ wafer-scale mixed-dimensional van der Waals heterojunctions consisting of chemical vapor deposited monolayer molybdenum disulfide and solution-processed semiconducting single-walled carbon nanotubes to emulate the spike-generating ion channels in biological neurons. Circuits based on these dual-gated Gaussian devices enable a variety of biological spiking responses including phasic spiking, delayed spiking, and tonic bursting. In addition to neuromorphic computing, the tunable Gaussian response has significant implications for a range of other applications including telecommunications, computer vision, and natural language processing.
-
Associative memory is a widespread self-learning method in biological livings, which enables the nervous system to remember the relationship between two concurrent events. The significance of rebuilding associative memory at a behavior level is not only to reveal a way of designing a brain-like self-learning neuromorphic system but also to explore a method of comprehending the learning mechanism of a nervous system. In this paper, an associative memory learning at a behavior level is realized that successfully associates concurrent visual and auditory information together (pronunciation and image of digits). The task is achieved by associating the large-scale artificial neural networks (ANNs) together instead of relating multiple analog signals. In this way, the information carried and preprocessed by these ANNs can be associated. A neuron has been designed, named signal intensity encoding neurons (SIENs), to encode the output data of the ANNs into the magnitude and frequency of the analog spiking signals. Then, the spiking signals are correlated together with an associative neural network, implemented with a three-dimensional (3-D) memristor array. Furthermore, the selector devices in the traditional memristor cells limiting the design area have been avoided by our novel memristor weight updating scheme. With the novel SIENs, the 3-D memristive synapse, and the proposed memristor weight updating scheme, the simulation results demonstrate that our proposed associative memory learning method and the corresponding circuit implementations successfully associate the pronunciation and image of digits together, which mimics a human-like associative memory learning behavior.more » « less
-
A variety of advanced machine learning and deep learning algorithms achieve state-of-the-art performance on various temporal processing tasks. However, these methods are heavily energy inefficient—they run mainly on the power hungry CPUs and GPUs. Computing with Spiking Networks, on the other hand, has shown to be energy efficient on specialized neuromorphic hardware, e.g., Loihi, TrueNorth, SpiNNaker, etc. In this work, we present two architectures of spiking models, inspired from the theory of Reservoir Computing and Legendre Memory Units, for the Time Series Classification (TSC) task. Our first spiking architecture is closer to the general Reservoir Computing architecture and we successfully deploy it on Loihi; the second spiking architecture differs from the first by the inclusion of non-linearity in the readout layer. Our second model (trained with Surrogate Gradient Descent method) shows that non-linear decoding of the linearly extracted temporal features through spiking neurons not only achieves promising results, but also offers low computation-overhead by significantly reducing the number of neurons compared to the popular LSM based models—more than 40x reduction with respect to the recent spiking model we compare with. We experiment on five TSC datasets and achieve new SoTA spiking results (—as much as 28.607% accuracy improvement on one of the datasets), thereby showing the potential of our models to address the TSC tasks in a green energy-efficient manner. In addition, we also do energy profiling and comparison on Loihi and CPU to support our claims.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1145/3461667
