designs, enfolding the progress in memristor devices, neuron and synaptic circuits, and neuromorphic systems. Specialized design for machine learning acceleration in neuromorphic systems, in-memory computing, is also discussed to envision a further higher leap of energy efficiency in computing hardware for the future AI.

Artificial intelligence (AI) has permeated the real world, showing unprecedented success. On the one hand, numberless applications exploit machine learning (ML) technologies of Big Data and compute-intensive algorithms. On the other hand, the aspiration of authentic machine intelligence moves computing toward the edge to handle complex tasks conventionally tailored for human beings. While edge computing with AI (edge AI) presents the advantages of the local process in terms of efficiency and security, the need to process a myriad of data locally onto edge devices demands highly efficient computing [9]. The gap between the increasing resource requirements in ML and the restricted environments of edge engenders urgent attention to the challenges in efficiency. To reconcile the gap, efficient hardware is needed beyond algorithm development.

Unfortunately, hardware development falls far behind when compared to the huge success of deep learning models at algorithm and software levels. While the sensational advance of ML algorithms is a game-change of computing paradigms, conventional hardware unfits new paradigms due to fundamental limitations in its architecture and technology. The traditional architecture dividing storage and computation is dreadfully inefficient for innumerable data processing and computing in algorithms, showing high power consumption and low performance. The realization of the fundamental limitations motivates non-conventional hardware designs for efficiency.

The incredible efficiency of the human brain has been an ideal exemplar of computing mechanisms, and deep neural network (DNN) is the representative paradigm inspired by brain mechanisms of selective concentration and dynamic allocation. As neural network models verify biological excellence in capturing relevant information efficiently, there has been a persistent trend in hardware computing to mimic the brain process. Neuromorphic computing is an emerging hardware field, indicating the neuro-inspired hardware design. By fusing brain-inspired selective processing capabilities, neuromorphic systems boost energy efficiency to an unprecedented level while preserving biological plausibility.

In this review, we discuss the development of neuromorphic hardware designs toward ultimate efficiency in computing for edge AI, introducing an emerging technology at the device level-memristor-and its application to neuromorphic circuit and system designs along with the counterpart designs on conventional memory technologies. Fortunately, the found memristor at the device level has expedited the development of neuromorphic designs. Memristor presents the capability to imitate the synapse activities of the brain, and accordingly, it has been leveraged with neuron circuits as a synaptic device. CMOS-and memristorbased neuron circuits with bio-plausibility imitate the neuron activities of efficient communications. Stimulated pre-neurons send signals to post-neurons through synapses once the accumulated stimuli exceed the threshold of synapses. The implementation of threshold and spike functions is mainly researched in neuromorphic hardware design. While array structures are dominantly exploited in neuro-inspired designs, where synaptic devices are placed at cross-points, the computing design with the arrays turns out to be particularly efficient in specialized domains, such as DNNs. As such, in-memory computing (IMC) emerges from neuromorphic designs, aiming to accelerate the specific ML domains at circuit, architecture, and system levels. With state-of-the-art hardware designs, this review provides insights into neuromorphic designs, including IMC, for edge AI in the future.

Computer engineers have dreamed of replicating human beings, especially their brains, due to extraordinary efficiency-20 W for 86 billion neurons. Correspondingly, bio-inspired neural networks for ML have a history as long as computers themselves have existed, starting with von Neumann and Turing. Several decades ago, researchers already attempted to attain machine learning through neural networks. The first neural network was proposed in 1943 [23] and a convolutional neural network (CNN) was found in the 1980s [17]. Nevertheless, their meager success and stagnant progress buried the concept of neural networks. The ML research went through a dark period because of severe

In this review, we discuss the development of neuromorphic hardware designs toward ultimate efficiency in computing for edge AI, introducing an emerging technology at the device level-memristor-and its application to neuromorphic circuit and system designs along with the counterpart designs on conventional memory technologies. Fortunately, the found memristor at the device level has expedited the development of neuromorphic designs.

issues, such as vanishing gradients and the lack of high-performance computing devices. There was also insufficient evidence to support the effect of neural networks with multiple layers. However, the success of LeNet-5 [18] in 1998 rekindled the ML algorithm development from the depression period because it manifested the feasibility of artificial learning by leveraging convolution. The five-layer CNN effectively extracted spatial features and successfully classified handwritten images, called Modified National Institute of Standards and Technology (MNIST). Since then, neural networks, particularly CNNs, have dominated with fast evolution in visual applications. Various CNN architectures stacked more layers, forming deep networks, and their excellence indicated the power of deep layers. Consequently, deeper architectures have further appeared and competed with a standard 256×256-sized image dataset, ImageNet, in the ImageNet Large Scale Visual Recognition Challenge (ILS-VRC) contest, boasting higher accuracy. In 2017, ResNet [4] even surpassed the human-level performance of the 5% error rate by reporting a 3.6% result. Deep CNNs (DCNN) have reached almost 100% accuracy and rapidly increased the number of parameters and operations.

The DNN model size became not affordable in many devices, particularly edges of resource restraints. Many decent algorithms near 100% accuracy with remarkable burdens turned the research trend to designing tiny models. Diverse techniques have emerged to reduce the model size, e.g., variants of operations composing networks, fewer bit data through quantization, and sparsification by pruning. Regular convolution causes severe date and computation overheads due to operations required across channels and kernels. Depthwise and pointwise (a.k.a. 1×1) convolution has appeared to resolve the overheads by regular convolution and enable the smooth implementation of DNNs. Also, rather than single-or doubleprecision floating-point format, deep learning models reduced the number of bits by quantizing the data into 16-, 8-, or even 1-bit (binarization) fixed-point numbers to compress the model size. At the same time, ML developers noticed that DNNs have numerous zeros, which rarely provide significant meanings in models, and as a result, focused on this sparsity (the ratio of zeros in total data) for compressed DNNs. While devising methodologies to skip unnecessary operations for zeros, algorithm designers started pruning insignificant data into zeros (sparsification). These efforts have had fruitful outcomes for tiny ML.

FIGURE 1 We are facing the end of Moore's law (top) and Dennard scaling (bottom) [15].

In contrast to the techniques for tiny ML, deep learning models are booming again over recent years because of the huge success of large language models (LLMs) in natural language processing (NLP). After Transformer, LLMs for sequential data raise the attention to the effectiveness of deep learning in NLP. Open AI of ChatGPT expedites the model size increase, designing GPT models with trillions of parameters, and the large models become applied to other domains beyond languages.

In contrast to the techniques for tiny ML, deep learning models are booming again over recent years because of the huge success of large language models (LLMs) in natural language processing (NLP). After Transformer [30], LLMs for sequential data raise the attention to the effectiveness of deep learning in NLP. Open AI of ChatGPT expedites the model size increase, designing GPT models with trillions of parameters, and the large models become applied to other domains beyond languages. Rather than expecting smaller models at the algorithm level, the renaissance of deep learning needs hardware solutions for extremely repeated operations and produced data.

In 1965, a co-founder of Intel, Gordon E. Moore, claimed that the number of transistors on a microchip doubled yearly. In a decade, Moore revised his first prediction and projected a doubling in transistor density every two years, since he noticed the transistor density of quadratic growth but the speed with a linear increase. This revised prediction was dubbed Moore's law owing to its strong match with the remarkable technology node down-scaling. Standing firm for many years, Moore's law has been evidence of rapid technological development.

However, as shown in Figure 1(a), experts in the field found a gap between the projection by Moore and industry capabilities at the time around 2000. Since then, the transistor density increase has deviated from the expectation of Moore's law. As a result, these days, a consensus view is that Moore's law is approaching its end because of the difficulty of shrinking the transistor node after it has entered 10 nm. The industry development hardly follows the prediction Moore made.

A mismatch of another projection, namely Dennard scaling, is also revealing the limitation of hardware technology development. Rober Dennard expected that power consumption per transistor would decrease as transistor density increased. Consequently, he projected that power per unit area of silicon would remain relatively constant. Until the early 2000 s, Dennard scaling had been well-matched like Figure 1(b) and used to predict future development. In 2007, however, Den-nard scaling began fading after a significant surge in power per unit area. As technology development relies on multi-core approaches in recent years, the end of Dennard scaling implies that increasing cores will escalate the power consumption almost linearly as Figure 1(b) proves. This implication evokes other accompanying concerns, such as heat dissipation. As we are living in the post-Moore era with the end of Dennard scaling, we need to prepare solutions to continue hardware development, rather than leaning on the fabrication technology development.

Furthermore, the fundamentals of predominant computer architecture show the architectural limitation, i.e., the Von Neumann Bottleneck. The modern processor architecture was designed by John von Neumann for general-purpose computing. He stores a program as instructions in memory and leverages a separate arithmetic/log unit (ALU), control unit, and on-chip buffer to execute the stored instructions sequentially, as illustrated in Figure 2. The von Neumann architecture was called after the inventor's name and became a representative computing paradigm underlying most hardware systems. Differentiated storage and computing units necessitate passing data between the two heterogeneous modules.

The heterogeneity was acceptable until the computing speed was slower than or equal to the memory access and data transfer time. However, a challenge in the heterogeneity has stood out due to limited bandwidth and a noticeable speed mismatch in communication. The rapidly developed technology enables fast computing in processors but the data delivery or fetch from/save to memory take long. Since the bandwidth is finite, the computingcompleted data are piled up on the bus and cause a bottleneck, degrading the hardware performance.

The terminology, the von Neumann bottleneck or the memory wall, was coined to indicate this problem by data transfer and limited bandwidth in the heterogeneity. This von Neumann architecture limitation is easily observed in

The term neuromorphic was coined by Carver Mead at the California Institute of Technology to indicate the field of brain-inspired circuit design [24]. Computer engineers have discussed neuroinspired machines to replicate neural systems as long as a history of neural networks based on the motivation toward efficiency of the human brain. Neuromorphic chips have proven their potential for energy efficiency with the successful application of memristor to neuromorphic circuits.

For intuitive understanding, the connections in the brain and the neuromorphic hardware design are compared in Figure 3. Figure 3(a) illustrates the brain's connections between pre-and post-neurons by synapses. Once preneurons are stimulated and the accumulated stimulus exceeds a threshold, a spike is emitted which the synapses carry to post-neurons. The sparse spiking signals transfer through multiple synapses in parallel, achieving both high efficiency and high throughput. A connectivity matrix is produced by placing an synaptic device at each crosspoint of a crossbar that connects neurons in one layer to neurons in the next, as displayed in Figure 3(b).

While CMOS-based memories and circuits could function as electronic synapses and neurons, the found memristor promises further efficiency. In the following sections, we provide details on device-, circuit-, and system-level implementation of neuromorphic computing. The memristor device is introduced as an emerging nonvolatile memory (eNVM) technology realizing neuromorphic designs. CMOS designs along with eNVM present neuron functionality of thresholding effect. Furthermore, domain-specific neuromorphic designs for ML acceleration, i.e., in-memoy computing (IMC), are also discussed as follows.

Neu romor ph ic designs necessitate random-access memor y (R A M) to freely read and write any cell of a memory array, rather than read-only memory (ROM) including flash memory. There have been two types of RAM, static and dynamic R AM (SR AM and DR AM), in the conventional memory taxonomy. Intel designed both memory types of volatility in 1970, which have been widely used as cache and main memory in modern computer architecture. SR AM of six transistors shows fast speed for operations; however, some issues, such as large area, high power consumption, and signal margin, hinder its practical use for future hardware. On the other hand, DR AM has a one-transistor-one-capacitor (1T1C) cell structure, consuming little power and area. But charging and discharging the capacitor result in slow operations and demand regular refresh, which are challenges in DR AM-based systems.

Unlike the previous R AMs above, emerging memories under the name of memristor have appeared as promising future devices in the post-Moore era. The existence of the memristor was first discovered theoretically in 1971. Leon Chua at University of California, Berkeley discovered the existence of a fourth element through the relations with the three circuit elements-resistor, capacitor, and inductor. Because it was expected that the element would remember data as resistance based on the historical value, he coined the term "memristor" by compounding the words, memory and resistor. A notable advantage of the memristor distinguished from the existing R AM is nonvolatility. While SR AM and DR AM lose the stored data without power, the data in the memristor are not affected by the power supply. Correspondingly, the memristor became called emerging nonvolatile memory (eNVM), which is regarded as a promising solution for the next generation of hardware because of its low power consumption without static power. However, the memristor had been missing and existed theoretically due to the lack of fabrication technology for the element.

Meanwhile, HP Labs successfully realized a memristor device with a nano-scale thin-film structure for the first time in 2008 [27]. The "found" memristor incited fast development of memristor devices with different materials and properties. Diverse advantages of eNVMs have been discovered and applied to hardware designs, which are being developed with different materials and mechanisms. For instance, [20].

latency <10 ns 0.3 ns 10 ns 50 ns 20 ns Write energy ~5fJ ~0.7fJ ~0.1pJ ~10pJ ~0.1pJ Endurance > 10 16 > 10 16 < 10 12 < 10 9 > 10 15 As summarized in Table 1, memristor devices offer several benefits, including fast switching, small area, and low power consumption, while there exist some concerns regarding non-ideal properties of the devices like nonlinearity, variations, sneak-path currents, etc. [9], [13]. Besides, the eNVMs have multi-bit storage capability according to different resistive levels, which could revolutionize hardware design. This capability enables imitating synapse functionality in the brain and therefore the eNVMs are also applicable for bio-inspired neuromorphic designs as synaptic devices. These attractive features of eNVMs highlight the importance of memristor development for an energy-efficient future and explain the considerable attention to the devices. The table also ascertains that the memristor devices have a better nature than the memory circuits like SR AM and DR AM as early explained. In this review, RR AM is mainly adopted for its better properties than the other memristor devices.

Biological neurons have diverse spikes to communicate with other neurons. While CMOS-based circuits can implement the neuron spikes, RR AM devices are also applicable to neuron circuits for spikes and thresholding effects. For example, Mott memristor is utilized in the Pearson-Anson relaxation oscillation circuit to realize chaotic oscillation-which means the random difference in amplitude and frequency of the oscillation over time. Mott memristor shows the current-controlled negative differential resistance (NDR) due to a thermal feedback mechanism. Mott memristors randomly disturb the I-V characteristics, which results in chaotic characteristics. The device also has a reversible metalinsulator phase transition at a specific temperature and therefore presents a history-related resistance. Based on the features of Mott memristor, Yan et al. [33] present a neuron circuit in addition to the Pearson-Anson relaxation circuit, which is given in Figure 4. The authors append an output buffer to amplify the output voltage from the Mott memristor (V t x ( ) ) and reshape the voltage pulse to digital spikes. A combination of a differential amplifier and an inverter is utilized to reduce area occupation. The randomness from the proposed circuit is advantageous for fast training, avoiding "training plateau" that indicates the stuck-in local minimums. As a result, the proposed design accelerates single-layer and two-layer perceptions by 1.893× averagely and saves 27.68% and 43.32% power consumption with 36.67% and 26.75% less area, respectively.

As Figure 5(a)(top) depicts, the conventional DNN spiking neuromorphic designs adopt the clock-driven dataflow. The excitatory and inhibitory synapses are mapped to differently, and correspondingly, the positive and negative Authorized licensed use limited to: University of Texas at San Antonio. Downloaded on May 16,2025 at 22:48:59 UTC from IEEE Xplore. Restrictions apply. 3D RR AM can be leveraged for further efficiency in the bio-inspired neuromorphic design [8], [14], [28], [29]. The fully analog design in [13] is based on the 3D VRR AM with the even/odd WL structure, which is depicted in Figure 7(a), and considers the structural features for computing. The split even/odd WLs are used to represent the synapses and connected to the positive/negative lines of current-sense amplifiers, which generate a corresponding result after computing. A balanced structure to avoid elongating the sneak leakage path is attained by arranging inputs/outputs in view of 3D architecture. Instead of including additional devices to alleviate the sneak path issue, we leverage the high nonlinearity of Ta O 2 5 RR AM by modeling as described in [12] (Figure 7(b)). Furthermore, we define a weight adjustment method to harmonize with the use of even/odd lines and devise a sequential operation to ensure to program only intended cells. The proposed design enables a modified reinforcement learning algorithm to be directly executed on the 3D hardware system. In the paper, we present a design example of classifying 8 8

× -pixel images into three alphabet characters on the structure, and Figure 7(c) presents the synaptic weight changes during the training. Compared to implementation based on the 2D crossbar array, our design achieves 6 × improvement in area density. To evaluate the design approaches, the impact of the sneak path currents is analyzed and the analysis verifies the effectiveness of the even/odd WL structure for neuromorphic system implementation.

Neuromorphic architectures have broadened to incorporate designs based on non-von Neumann architectures like inmemory computing (IMC), as known as computing(processing)-in-memory (CIM, IMC). IMC is a specialized design in neuromorphic computing for machine learning acceleration, including systematic innovations. Figure 8 illustrates IMC with a conceptualized diagram; by fusing the memorization and processing capabilities, IMC designs reduce expensive data movement in hardware. IMC is capable of massively parallel computation with the same unit of storing and computing data built on the same crossbar structure.

Along with eNVM-based crossbar designs, conventional memory technology like SR AM is developed for IMC designs, computing vector-matrix multiplications (VMM) under 1 ns read and write latency. However, SR AM CIM devices need peripheral circuitry like analog-to-digital and digital-to-analog converters (ADC and DAC) for smooth communication between the heterogeneous units, consuming undeniably prodigious resources. Therefore, IMC with conventional memory can demonstrate its effectiveness by removing the need for converters, such as with the ADCless SR AM computing design [34]. This work fabricated a 32Kb SR AM CIM macro based on 28 nm technology with three main features that are illustrated in Figure 9(c) and (d). First, the ADC-less vector-hadamard-product (VHP) architecture is implemented for reconfigurable local process units (RLPU) in the bit cell array, supporting reconfigurable bitwise operations like AND, OR, and XOR. Dynamic logic compute circuits (DCC) follow six-transistor (6 T) cells and RLPUs, replacing the conventional ADC and CMOS logic circuits with a compact design. Lastly, flexible VHP/ VMM is supported by post-sum circuits, boasting an almost full utilization rate (over 98 % ).

Nonetheless, eNVMs of a nano-film metal-insulator-metal (MIM) structure, where a metal oxide layer is sandwiched between a top electrode (TE) and a bottom electrode (BE), are actively researched for IMC. Due to the MIM structure, RR AM is compatible with 2D crossbar, as illustrated in Figure 10. Arrays are structured by placing devices at every cross-point of perpendicular lines. Voltages with proper pulse width and magnitude over a threshold program the device resistance (memristance). The resistance level programmed accordingly represents data in a cell. Figure 10 shows the acceleration mechanism. For the multiplication between inputs and weights, the weight matrix is programmed to the RR AM array as memristance with its unrolled-vector form. The input matrix is also unrolled as vector data, which are supplied to the RR AM array as voltages. The two operands are multiplied at each cell and can be measured as currents by Ohm's law, and currents from each wordline are accumulated along with the electrical wire by Kirchhoff's law. In such a way, the VMM results can be obtained through a one-shot operation of the array. Therefore, the use of the RR AM crossbar is advantageous for high-parallel computing of ML algorithms.

Previous accelerators demonstrate the potential of RR AM and its array design in ML. In early exploration, Hu et al. demonstrated the analogy of RR AM to the synaptic function [5] in the implementation of the brain-state-in-a-box (BSB) model. Since RR AM devices have intrinsic positiveness, real numbers to compute the BSB model are expressed with the subtraction between two memristive arrays. Parallel computing through two RR AM arrays accelerates VMM. The calculated results are passed to the peripheral circuits to process the subsequent computations in the BSB model. Beyond relatively simple computations in traditional ML models, many RR AM-based IMC accelerators also aim for DNN models along with the prosperity of DNN.

Inference (testing):

The successful examples of DNN implementation led to diverse RR AM-based IMC accelerator designs for various ML techniques and operations. As CNNs become one of the most dominant algorithm designs since the convolution operations of LeNet-5 successfully classified the hand-written dataset, IMC designs for CNN acceleration in inference (testing) and training have appeared in various ways. PRIME [2] and ISAAC [25] are representative accelerators for the inference of CNNs. After unrolling weight parameters in 2D of CNNs, RR AM arrays efficiently execute convolutions with parallel operating. Other required operations, like nonlinear functions, are implemented by proper peripheries in both accelerators. In PRIME, a fullstack solution is proposed, encompassing hardware architecture and software interface, to support in-situ computation of neural networks in main memory. PRIME achieves four orders of magnitude improvement over CPU in speedup and energy saving. In the case of ISAAC, implemented peripheral circuits, i.e., the read-out, sample and hold, analog-digital conversion, and post-processing, are pipelined in each computing chunk for higher throughput. ISAAC outperforms its counterpart GPU design, reporting 450× speedup and 150× lower energy than NVIDIA K20M GPU. Also, ISSAC demonstrates a 14.8× higher throughput on average.

Training: Unlike inference (testing) of the feed-forward process, training includes additional complicated processes like backpropagation and weight update procedures. Correspondingly, more compute-intensive training than inference calls for accelerators, along with the efforts to simplify training hardware [7]. PipeLayer [26] presents an accelerator design for training. Pipe-Layer analyzes essential computations during the training and then introduces the pipelining scheme for efficient training process, as displayed in Figure 11; it adds weight update into the computation pipeline and resolves the dependencies in training. For fast batch computing of multiple input samples, PipeLayer performs an intra-pipeline-fashioned process with the samples. Since weight update is executed at the end of each batch process and the samples during the batch process show no dependence, the intra-pipelining is realizable The RR AM arrays in PipeLayer are utilized for two main purposes: morphable and memory subarrays to perform computation and storage, respectively. Memory mode subarrays save the activations for calculating the errors in the backpropagation. Pipe-Layer shows 42.45× speedup and 7.17× energy saving compared with a GPU platform.

GAN training: From the competitive results from DNN models, generative adversarial networks (GANs) have emerged as a strengthened training methodology through unsupervised learning. As illustrated in Figure 12(a), GAN is based on an adversarial game between a generator (G) and a discriminator (D). G with high-quality samples from noise simulates the distribution of true data samples, while D is an optimized binary classifier trained by distinguishing between authentic and generated samples. GANs are powerful, but the two DNNs in one mechanism demand high computing resources and frequent data movement. Furthermore, GANs involve with more computing phases through highly complex operations than traditional supervised deep learning. From the challenges, ReGAN [1] exploits RR AM-based IMC effectiveness in VMM operations. ReGAN has three partitioned RR AM memory arrays: pure memory subarrays for data storage, full-function subarrays for on-site VMM and data saving, and buffer subarrays for intermediates between layers. ReGAN connects the buffer and full-function arrays by highthroughput private local data ports to avoid limited bandwidth between memories and buffers. Also, the spikebased scheme presents smooth communications in ReGAN, minimizing the energy overhead by power-hungry analog-digital converters (ADC). The nonlinear activation function is conducted by utilizing a look-up table, and other peripheral units, such as subtractors and shift-and-adds, are implemented to support various computations in training. ReGAN shows 240× speedup and 94× energy saving than GPU on average.

Transformer: Large language models (LLMs) prove their superiority with GPT and BERT model algorithms. LLMs are based on the multi-head-attention model (Figure 12(b)), and ReTransformer [36] proposes a design to accelerate the transformer in a pipeline fashion. The inference starts with the initialization of two RR AM arrays with W Q and W K , respectively. Following the original flow, two weight matrices (W Q and W K ) are computed with input matrices to generate Q and K , which will multiply the Q and K T intermediate matrices. RR AM crossbars, therefore, hold one of the results Q and K for the dot product operation between Q and K .

In ReTransformer, the data generation and the computing process are reordered in the scaled dot-product attention layer between W Q and W K with two cascaded steps. As shown in Figure 12(b), the Q matrix is computed first, Then it is multiplied with the weight matrix (W K ), rather than computing the K matrix separately, and the generated matrix is multiplied by the input matrix. To prevent frequent writing of intermediate values into RR AM arrays, ReTansformer performs the optimized computing flow in a pipeline fashion. Initialization of RR AM arrays with weight matrices at the beginning is followed by two stages, to multiply weights and inputs for Q and to compute intermediate results by multiplying the generated Q and weights. In the meantime, other RR AM crossbar arrays are initialized with inputs so that the intermediate results can be performed with the input matrix to shorten the computing time. ReTransformer improves efficiency through the proposed reordering and pipelining scheme by up to 80%.

Compressed ML: As the model size is drastically increasing, algorithm techniques for compression have been advanced by utilizing sparsification [32] and quantization [21]. DNNs inherently include high sparsity (the ratio of zeros) since activations and weights have zero values due to the results from non-linear functions, etc. However, the unpredictability of the irregular distribution of the zeros hinders the application of sparsity to the learning acceleration. Consequently, intended sparsification through pruning has been proposed to get rid of insignificant values.

Sparsification: ReCom [6] leverages the sparsity of weights and activations to improve the resource utilization in compressed in NN based on efficient RR AM IMC design. Structured sparsification compresses the weights parameters. Then non-zero activations corresponding to the only non-zero weights are extracted by the proposed structurally weightoriented fetching (SWOF) unit. In-layer pipeline for memory and computation (IMPC) further speeds up the process, while skipping redundant computations in case of all-zero activations. ReCom shortens the computation time by up to 10.66× and increases energy efficiency by up to 9.43×, compared to PipeLayer. SNrram is designed as an RR AM-based efficient computation IMC architecture by leveraging the sparsity in weights and activations. [31]. Zero-value multiplications can be skipped by SNrram because only significant activations and weights are distinguished as a dense matrix and used for multiplication. Experiments verify that SNrram can reduce RR AM resources by 69.8%, improve the power consumption by 35.9%, and shorten the computation time by 2.49× on widely used DNN benchmarks, compared to the early introduced accelerator, PRIME.

Quantization: In addition, the quantization of activations/parameters reduces the number of bits, that is, the in hardware for learning. Consequently, quantized DNNs with advanced techniques preserve accuracy with decreased overheads. Binarized neural networks (BNNs) are an outcome from an extreme quantization of weight parameters. To further expedite the learning process in BNN, Kim et al. proposed a 3D RRAM design for BNN (3D-BNN) [10]. BNN maintains high accuracy with great savings in parameters at the algorithm level. Moreover, adopting binarized weights eliminates the requirement of multiplication and necessitates only additions. However, the remaining logic-based addition still demands numerous intermediates and corresponding devices, consuming high resources. Driven by motivation, 3D-BNN presents a newly devised simplified logic-based BNN computation and its implementation to 3D RRAM in an efficient way. The simplified logic-based full-adder scheme shortens the computing latency and device consumption, recognizing that the outcome of a logic-based full-adder depends on a specific term in the equation and obtaining the term in a prompt manner. After that, a 3D RRAM structure is designed for efficient parallel computing with a competing scheme. Half of the operands are saved into stacked layers with different assumptions regarding carry-in. The corresponding layer to the real carry-out is chosen for the following computation propagation. In such a design, 3D-BNN makes no sacrifice on accuracy from the original BNN result but also saves computing time and RRAM devices by up to 6×.

Dataflow: Dataflow is one of the critical factors determining hardware efficiency because of the different expenses of data transfer according to the components. Interestingly, the previous RR AM-based IMC accelerators have implemented one dataflow, weightstationary (WS) dataflow, by maintaining weights in IMC RR AM arrays and reading/writing inputs (activations) [37].

to external memories. This is because weight parameters are static values and activations are dynamic, and accordingly weights stay in arrays but inputs are moved. However, INCA [16] raised a question about the efficiency of WS for the first time, observing four fundamental limitations in WS. First, as shown in Figure 13 Specific Applications: While diverse applications are supported by machine learning algorithms, they have different requirements in computing. Some applications, such as biomedical applications [3], need high accuracy with small power consumption to be executed at edges in the real world and take advantage of IMC technology. A fabricated chip [11] built on TSMC 65 nm verifies the effectiveness of the systemic finding on IS for efficiency in diagnosis automation. The chip is implementable to epileptic seizure patients for their seizure-free lives. Whereas the status quo for epilepsy diagnosis is its dependence on wearable devices, wearable devices are 1) inaccurate because they use easily accessible signals like heart rate and 2) inconvenient since patients should always wear them. Instead of wearable devices, implementable chips could address the inconvenience with high accuracy.

Previously, seizure-classifying chips had adopted outdated algorithms that gave binary answers because of the extremely restricted resources in a small chip. Also, they are patient-specific for the sake of algorithm simplicity; that is, one chip functions for only one patient. But the classification for accurate diagnosis is a multi-class problem in practice according to its status, occurrence point, etc. Rather than patient-specific algorithms, patient independence is also pursued to apply the developed chip in general for efficiency.

Driven by this motivation, we design and fabricate an efficient chip, MulPi [37], after establishing a new hardwareoptimized 1D-CNN algorithm with cutting-edge techniques (MulPiCNN). We implemented MulPiCNN into an SR AM-CIM chip by designing, simulating, and performing the layout from SR AM to peripheries. As an architecture design, we first leveraged IS for the area-saving effect among its advantages in [16]. As a result, IS reduced the required SR AM capacity by 42 × more than WS. At the circuit level, we designed a Hadamard product unit with only two transistors to minimize resource consumption. In such a way, the chip outperforms state-of-the-art chips in performance and area with comparable energy efficiency, as described in Figure 14.

AI on the ML basis toward edge presents inconceivable convenience but simultaneously demands high efficiency as never before. The demand comes from the gap between the algorithm and hardware requirements. While the data volume and model size of DNN are ever-increasing, available resources are diminished in edge AI. Although efficient hardware development is needed more than ever to satisfy the requirements of the algorithm and environment, it faces fundamental limitations, such as the end of Moore's law and the von Neumann bottleneck. Domainspecific hardware designs are vigorously investigated as a breakthrough for the crisis of technology development in the coming AI generation. Neuromorphic designs are expected to be the most efficient, especially when they meet eNVMs, as the bio-inspired designs have been proved by the efficiency of the human brain and the success of NN.

While the CMOS technology is available in pursuing further efficiency, memrsitor has emerged as a promising solution at the device level in neuromorphic computing due to its device merits and compatibility with CMOS technology and neuromorphic array designs. Diverse experiments have demonstrated its high capability as a synaptic device. This review discusses the neuron circuit designs, which implement the threshold effect and spikes in the biological neuron's mechanisms for efficient communications. The neurons can be realized by CMOS technology but also by specific memristor devices, such as the Mott memristor. In the meantime, the array structure of pre-and post-neurons with the synaptic devices have proved the high efficiency in parallel computing of compute operations. Correspondingly, the overwhelmed MAC operations in ML exploited the array design to save and compute data simultaneously. As such, the neuromorphic designs have been developed with specialization for ML acceleration, which is called in-memory computing. IMC specifically aims to accelerate the specific ML domains at circuit, architecture, and system levels. Beyond the conventional hardware, neuromorphic to IMC designs will bring efficiency to a different level.