skip to main content

Title: Hardware implementation of Bayesian network building blocks with stochastic spintronic devices

Bayesian networks are powerful statistical models to understand causal relationships in real-world probabilistic problems such as diagnosis, forecasting, computer vision, etc. For systems that involve complex causal dependencies among many variables, the complexity of the associated Bayesian networks become computationally intractable. As a result, direct hardware implementation of these networks is one promising approach to reducing power consumption and execution time. However, the few hardware implementations of Bayesian networks presented in literature rely on deterministic CMOS devices that are not efficient in representing the stochastic variables in a Bayesian network that encode the probability of occurrence of the associated event. This work presents an experimental demonstration of a Bayesian network building block implemented with inherently stochastic spintronic devices based on the natural physics of nanomagnets. These devices are based on nanomagnets with perpendicular magnetic anisotropy, initialized to their hard axes by the spin orbit torque from a heavy metal under-layer utilizing the giant spin Hall effect, enabling stochastic behavior. We construct an electrically interconnected network of two stochastic devices and manipulate the correlations between their states by changing connection weights and biases. By mapping given conditional probability tables to the circuit hardware, we demonstrate that any two node Bayesian networks can be implemented by our stochastic network. We then present the stochastic simulation of an example case of a four node Bayesian network using our proposed device, with parameters taken from the experiment. We view this work as a first step towards the large scale hardware implementation of Bayesian networks.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Scientific Reports
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Employing the probabilistic nature of unstable nano-magnet switching has recently emerged as a path towards unconventional computational systems such as neuromorphic or Bayesian networks. In this letter, we demonstrate proof-of-concept stochastic binary operation using hard axis initialization of nano-magnets and control of their output state probability (activation function) by means of input currents. Our method provides a natural path towards addition of weighted inputs from various sources, mimicking the integration function of neurons. In our experiment, spin orbit torque (SOT) is employed to “drive” nano-magnets with perpendicular magnetic anisotropy (PMA) -to their metastable state, i.e. in-plane hard axis. Next, the probability of relaxing into one magnetization state (+mi) or the other (−mi) is controlled using an Oersted field generated by an electrically isolated current loop, which acts as a “charge” input to the device. The final state of the magnet is read out by the anomalous Hall effect (AHE), demonstrating that the magnetization can be probabilistically manipulated and output through charge currents, closing the loop from charge-to-spin and spin-to-charge conversion. Based on these building blocks, a two-node directed network is successfully demonstrated where the status of the second node is determined by the probabilistic output of the previous node and a weighted connection between them. We have also studied the effects of various magnetic properties, such as magnet size and anisotropic field on the stochastic operation of individual devices through Monte Carlo simulations of Landau Lifshitz Gilbert (LLG) equation. The three-terminal stochastic devices demonstrated here are a critical step towards building energy efficient spin based neural networks and show the potential for a new application space.

    more » « less
  2. Abstract

    Execution of probabilistic computing algorithms require electrically programmable stochasticity to encode arbitrary probability functions and controlled stochastic interaction or correlation between probabilistic (p-) bits. The latter is implemented with complex electronic components leaving a large footprint on a chip and dissipating excessive amount of energy. Here, we show an elegant implementation with just two dipole-coupled magneto-tunneling junctions (MTJ), with magnetostrictive soft layers, fabricated on a piezoelectric film. The resistance states of the two MTJs (high or low) encode the p-bit values (1 or 0) in the two streams. The first MTJ is driven to a resistance state with desired probability via a current or voltage that generates spin transfer torque, while the second MTJ’s resistance state is determined by dipole coupling with the first, thus correlating the second p-bit stream with the first. The effect of dipole coupling can be varied by generating local strain in the soft layer of the second MTJ with a local voltage (~ 0.2 V) and that varies the degree of anti-correlation between the resistance states of the two MTJs and hence between the two streams (from 0 to 100%). This paradigm generates the anti-correlation with “wireless” dipole coupling that consumes no footprint on a chip and dissipates no energy, and it controls the degree of anti-correlation with electrically generated strain that consumes minimal footprint and is extremely frugal in its use of energy. It can be extended to arbitrary number of bit streams. This realizes an “all-magnetic” platform for generating correlations or anti-correlations for probabilistic computing. It also implements a simple 2-node Bayesian network.

    more » « less
  3. A low-energy hardware implementation of deep belief network (DBN) architecture is developed using near-zero energy barrier probabilistic spin logic devices (p-bits), which are modeled to real- ize an intrinsic sigmoidal activation function. A CMOS/spin based weighted array structure is designed to implement a restricted Boltzmann machine (RBM). Device-level simulations based on precise physics relations are used to validate the sigmoidal relation between the output probability of a p-bit and its input currents. Characteristics of the resistive networks and p-bits are modeled in SPICE to perform a circuit-level simulation investigating the performance, area, and power consumption tradeoffs of the weighted array. In the application-level simulation, a DBN is implemented in MATLAB for digit recognition using the extracted device and circuit behavioral models. The MNIST data set is used to assess the accuracy of the DBN using 5,000 training images for five distinct network topologies. The results indicate that a baseline error rate of 36.8% for a 784x10 DBN trained by 100 samples can be reduced to only 3.7% using a 784x800x800x10 DBN trained by 5,000 input samples. Finally, Power dissipation and accuracy tradeoffs for probabilistic computing mechanisms using resistive devices are identified. 
    more » « less
  4. Abstract Bayesian networks (BNs) find widespread application in many real-world probabilistic problems including diagnostics, forecasting, computer vision, etc. The basic computing primitive for BNs is a stochastic bit (s-bit) generator that can control the probability of obtaining ‘1’ in a binary bit-stream. While silicon-based complementary metal-oxide-semiconductor (CMOS) technology can be used for hardware implementation of BNs, the lack of inherent stochasticity makes it area and energy inefficient. On the other hand, memristors and spintronic devices offer inherent stochasticity but lack computing ability beyond simple vector matrix multiplication due to their two-terminal nature and rely on extensive CMOS peripherals for BN implementation, which limits area and energy efficiency. Here, we circumvent these challenges by introducing a hardware platform based on 2D memtransistors. First, we experimentally demonstrate a low-power and compact s-bit generator circuit that exploits cycle-to-cycle fluctuation in the post-programmed conductance state of 2D memtransistors. Next, the s-bit generators are monolithically integrated with 2D memtransistor-based logic gates to implement BNs. Our findings highlight the potential for 2D memtransistor-based integrated circuits for non-von Neumann computing applications. 
    more » « less
  5. Large-scale deep neural networks are both memory and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated. Specific forms of binary neural networks (BNNs) and stochastic computing-based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper, we prove that the ”ideal” SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior), which is a new angle from the original approximation property. The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a “bridge” to prove for BNNs. Besides the universal approximation property, we also derive an appropriate bound for bit length M in order to provide insights for the actual neural network implementations. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growth of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable. 
    more » « less