An official website of the United States government
The spatiotemporal nature of neuronal behavior in spiking neural networks (SNNs) makes SNNs promising for edge applications that require high energy efficiency. To realize SNNs in hardware, spintronic neuron implementations can bring advantages of scalability and energy efficiency. Domain wall (DW)-based magnetic tunnel junction (MTJ) devices are well suited for probabilistic neural networks given their intrinsic integrate-and-fire behavior with tunable stochasticity. Here, we present a scaled DW-MTJ neuron with voltage-dependent firing probability. The measured behavior was used to simulate a SNN that attains accuracy during learning compared to an equivalent, but more complicated, multi-weight DW-MTJ device. The validation accuracy during training was also shown to be comparable to an ideal leaky integrate and fire device. However, during inference, the binary DW-MTJ neuron outperformed the other devices after Gaussian noise was introduced to the Fashion-MNIST classification task. This work shows that DW-MTJ devices can be used to construct noise-resilient networks suitable for neuromorphic computing on the edge.
more »
« less
Semi-supervised learning and inference in domain-wall magnetic tunnel junction (DW-MTJ) neural networks
Advances in machine intelligence have sparked interest in hardware accelerators to implement these algorithms, yet embedded electronics have stringent power, area budgets, and speed requirements that may limit nonvolatile memory (NVM) integration. In this context, the development of fast nanomagnetic neural networks using minimal training data is attractive. Here, we extend an inference-only proposal using the intrinsic physics of domain-wall MTJ (DW-MTJ) neurons for online learning to implement fully unsupervised pattern recognition operation, using winner-take-all networks that contain either random or plastic synapses (weights). Meanwhile, a read-out layer trains in a supervised fashion. We find our proposed design can approach state-of-the-art success on the task relative to competing memristive neural network proposals, while eliminating much of the area and energy overhead that would typically be required to build the neuronal layers with CMOS devices.
more »
« less
- Award ID(s):
- 1910800
- PAR ID:
- 10145308
- Date Published:
- Journal Name:
- SPIE Spintronics XII
- Volume:
- 11090
- Page Range / eLocation ID:
- 110903I
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. However, an inevitability of transferring neural networks onto hardware is the fact that nonidealities, such as device-to-device variations or poor device yield impact performance. Methods, such as hardware-aware training, where substrate nonidealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware-based neural networks consisting of 20 000 magnetic tunnel junction (MTJ) arrays integrated on CMOS chips in a form that closely resembles scalable and market-ready spin transfer-torque magnetoresistive random access memory (STT-MRAM) technology. Using 36 dies, each containing a MTJ-CMOS crossbar array with its own nonidealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2%. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.more » « less
-
null (Ed.)Brain-inspired cognitive computing has so far followed two major approaches - one uses multi-layered artificial neural networks (ANNs) to perform pattern-recognition-related tasks, whereas the other uses spiking neural networks (SNNs) to emulate biological neurons in an attempt to be as efficient and fault-tolerant as the brain. While there has been considerable progress in the former area due to a combination of effective training algorithms and acceleration platforms, the latter is still in its infancy due to the lack of both. SNNs have a distinct advantage over their ANN counterparts in that they are capable of operating in an event-driven manner, thus consuming very low power. Several recent efforts have proposed various SNN hardware design alternatives, however, these designs still incur considerable energy overheads.In this context, this paper proposes a comprehensive design spanning across the device, circuit, architecture and algorithm levels to build an ultra low-power architecture for SNN and ANN inference. For this, we use spintronics-based magnetic tunnel junction (MTJ) devices that have been shown to function as both neuro-synaptic crossbars as well as thresholding neurons and can operate at ultra low voltage and current levels. Using this MTJ-based neuron model and synaptic connections, we design a low power chip that has the flexibility to be deployed for inference of SNNs, ANNs as well as a combination of SNN-ANN hybrid networks - a distinct advantage compared to prior works. We demonstrate the competitive performance and energy efficiency of the SNNs as well as hybrid models on a suite of workloads. Our evaluations show that the proposed design, NEBULA, is up to 7.9× more energy efficient than a state-of-the-art design, ISAAC, in the ANN mode. In the SNN mode, our design is about 45× more energy-efficient than a contemporary SNN architecture, INXS. Power comparison between NEBULA ANN and SNN modes indicates that the latter is at least 6.25× more power-efficient for the observed benchmarks.more » « less
-
Using binarized neural network (BNN) as an alternative to the conventional convolutional neural network is a promising candidate to answer the demand of using human brain-inspired in applications with limited hardware and power resources, such as biomedical devices, IoT edge sensors, and other battery-operated devices. Using nonvolatile memory elements like MTJ devices in a LiM-based architecture can eliminate the need to access and use external memory which can significantly reduce the power consumption and area overhead. In addition, by using adiabatic-based designs, a significant part of the consumed power can be recovered to the power source which leads to a huge reduction in power consumption which is vital in applications with limited power and hardware resources. In this paper by using nonvolatile MTJ devices in a LiM architecture and using adiabatic-based circuits, an XNOR/XOR synapse and neuron is proposed. The proposed design offers 97% improvement in comparison with its state-of-the-art counterparts in case of power consumption. Also, it achieves at least 7% lower area compared to other counterparts which makes the proposed design a promising candidate for hardware implementation of BNNs.more » « less
-
Multistate memory systems have the ability to store and process more data in the same physical space as binary memory systems, making them a potential alternative to existing binary memory systems. In the past, it has been demonstrated that voltage-controlled magnetic anisotropy (VCMA) based writing is highly energy-efficient compared to other writing methods used in non-volatile nano-magnetic binary memory systems. In this study, we introduce a new, VCMA-based and skyrmion-mediated non-volatile ternary memory system using a perpendicular magnetic tunnel junction (p-MTJ) in the presence of room temperature thermal perturbation. We have also shown that ternary states {− 1, 0, + 1} can be implemented with three magnetoresistance values obtained from a p-MTJ corresponding to ferromagnetic up, down, and skyrmion state, with 99% switching probability in the presence of room temperature thermal noise in an energy-efficient way, requiring ~ 2 fJ energy on an average for each switching operation. Additionally, we show that our proposed ternary memory demonstrates an improvement in area and energy by at least 2X and ~ 10e4 X respectively, compared to state-of-the-art spin-transfer torque (STT)-based non-volatile magnetic multistate memories. Furthermore, these three states can be potentially utilized for energy-efficient, high-density in-memory quantized deep neural network implementation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.1117/12.2530308
