More Like this

1. FP-IMC: A 28nm All-Digital Configurable Floating-Point In-Memory Computing Macro

   Jyotishman Saikia, Amitesh Sridharan (September 2023, Proceedings of ESSCIRC)

   In-memory computing (IMC) provides energy- efficient solutions to deep neural networks (DNN). Most IMC de- signs for DNNs employ fixed-point precisions. However, floating- point precision is still required for DNN training and complex inference models to maintain high accuracy. There have not been float-point precision based IMC works in the literature where the float-point computation is immersed into the weight memory storage. In this work, we propose a novel floating-point precision IMC macro with a configurable architecture that supports both normal 8-bit floating point (FP8) and 8-bit block floating point (BF8) with a shared exponent. The proposed FP-IMC macro implemented in 28nm CMOS demonstrates 12.1 TOPS/W for FP8 precision and 66.6 TOPS/W for BF8 precision, improving energy-efficiency beyond the state-of-the-art FP IMC macros. 

   more » « less
2. Efficient Memory Integration: MRAM-SRAM Hybrid Accelerator for Sparse On-Device Learning

   https://doi.org/10.1145/3649329.3657390  

   Zhang, Fan; Sridharan, Amitesh; Tsai, Wilman; Chen, Yiran; Wang, Shan X; Fan, Deliang (June 2024, ACM)

   With the prosperous development of Deep Neural Network (DNNs), numerous Process-In-Memory (PIM) designs have emerged to accelerate DNN models with exceptional throughput and energy-efficiency. PIM accelerators based on Non-Volatile Memory (NVM) or volatile memory offer distinct advantages for computational efficiency and performance. NVM based PIM accelerators, demonstrated success in DNN inference, face limitations in on-device learning due to high write energy, latency, and instability. Conversely, fast volatile memories, like SRAM, offer rapid read/write operations for DNN training, but suffer from significant leakage currents and large memory footprints. In this paper, for the first time, we present a fully-digital sparse processing in hybrid NVM-SRAM design, synergistically combines the strengths of NVM and SRAM, tailored for on-device continual learning. Our designed NVM and SRAM based PIM circuit macros could support both storage and processing of N:M structured sparsity pattern, significantly improving the storage and computing efficiency. Exhaustive experiments demonstrate that our hybrid system effectively reduces area and power consumption while maintaining high accuracy, offering a scalable and versatile solution for on-device continual learning. 

   more » « less
3. CANNON: C ommunication- A ware Sparse N eural N etwork O ptimizatio n

   https://doi.org/10.1109/TETC.2023.3289778  

   Goksoy, A. Alper; Li, Guihong; Mandal, Sumit K.; Ogras, Umit Y.; Marculescu, Radu (January 2023, IEEE Transactions on Emerging Topics in Computing)

   Sparse deep neural networks (DNNs) have the potential to deliver compelling performance and energy efficiency without significant accuracy loss. However, their benefits can quickly diminish if their training is oblivious to the target hardware. For example, fewer critical connections can have a significant overhead if they translate into long-distance communication on the target hardware. Therefore, hardware-aware sparse training is needed to leverage the full potential of sparse DNNs. To this end, we propose a novel and comprehensive communication-aware sparse DNN optimization framework for tile-based in-memory computing (IMC) architectures. The proposed technique, CANNON first maps the DNN layers onto the tiles of the target architecture. Then, it replaces the fully connected and convolutional layers with communication-aware sparse connections. After that, CANNON optimizes the communication cost with minimal impact on the DNN accuracy. Extensive experimental evaluations with a wide range of DNNs and datasets show up to 3.0× lower communication energy, 3.1× lower communication latency, and 6.8× lower energy-delay product compared to state-of-the-art pruning approaches with a negligible impact on the classification accuracy on IMC-based machine learning accelerators. 

   more » « less
4. C ir CNN: accelerating and compressing deep neural networks using block-circulant weight matrices

   https://doi.org/10.1145/3123939.3124552  

   Ding, Caiwen; Yuan, Geng; Ma, Xiaolong; Zhang, Yipeng; Tang, Jian; Qiu, Qinru; Lin, Xue; Yuan, Bo; Liao, Siyu; Wang, Yanzhi; et al (January 2017, Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture)

   Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning, which affects performance and throughput; 2) the increased training complexity; and 3) the lack of rigirous guarantee of compression ratio and inference accuracy. To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from O(n2) to O(n log n) and the storage complexity from O(n2) to O(n), with negligible accuracy loss. Compared to other approaches, CirCNN is distinct due to its mathematical rigor: the DNNs based on CirCNN can converge to the same "effectiveness" as DNNs without compression. We propose the CirCNN architecture, a universal DNN inference engine that can be implemented in various hardware/software platforms with configurable network architecture (e.g., layer type, size, scales, etc.). In CirCNN architecture: 1) Due to the recursive property, FFT can be used as the key computing kernel, which ensures universal and small-footprint implementations. 2) The compressed but regular network structure avoids the pitfalls of the network pruning and facilitates high performance and throughput with highly pipelined and parallel design. To demonstrate the performance and energy efficiency, we test CirCNN in FPGA, ASIC and embedded processors. Our results show that CirCNN architecture achieves very high energy efficiency and performance with a small hardware footprint. Based on the FPGA implementation and ASIC synthesis results, CirCNN achieves 6 - 102X energy efficiency improvements compared with the best state-of-the-art results. 

   more » « less
5. ScRRAMBLe: block-sparse deep learning architecture for analog in-memory computing accelerators

   https://doi.org/10.1038/s44335-026-00071-7  

   Jaltare, Vikrant; Mondal, Rommani; Gibb, Leif; Song, Jiahao; Leugering, Johannes; Cauwenberghs, Gert (December 2026, npj Unconventional Computing)

   Analog compute-in-memory combines compute and storage using crossbar arrays of non-volatile memory, thus promising to reduce the energy demand for artificial intelligence workloads. Yet, significant challenges limit their scaling. Implementing signed weights using positive conductances requires a differential encoding scheme that substantially increases area. Furthermore, large fully-connected networks must be decomposed and distributed over multiple fan-in limited CIM modules, incurring a large inter-core communication traffic. Consequently, the partial sum of intermediate logits needs to be computed, requiring high-precision analog-to-digital converters, which have area and power consumption. We propose ScRRAMBLe, a block-sparse neural network architecture with input-balanced random routing between the cores to address these challenges. By imposing an input balancing constraint, we implement each signed weight using a single positive conductance, avoiding the area penalties of a fully differential encoding. We introduce block-sparsity in the network by defining connectivity between neural subpopulations and weight sharing across feature dimensions, improving the communication overhead of unstructured sparsity by reducing the number of address bits. Replacing fully-connected layers, we show the ScRRAMBLe achieves 98.95% accuracy (MNIST) with 94.5% inter-core traffic reduction. Integrated with ResNet, ScRRAMBLe yields 90.46% (CIFAR-10) and 70.36% (CIFAR-100) accuracy with >97% traffic reduction. 

   more » « less