Not AvailableModern Artificial Intelligence (AI) workloads demand computing systems with large silicon area to sustain throughput and competitive performance. However, prohibitive manufacturing costs and yield limitations at advanced tech nodes and die-size reaching the reticle limit restrain us from achieving this. With the recent innovations in advanced packaging technologies, chiplet-based architectures have gained significant attention in the AI hardware domain. However, the vast design space of chiplet-based AI accelerator design and the absence of system and package-level co-design methodology make it difficult for the designer to find the optimum design point regarding Power, Performance, Area, and manufacturing Cost (PPAC). This paper presents Chiplet-Gym, a Reinforcement Learning (RL)-based optimization framework to explore the vast design space of chiplet-based AI accelerators, encompassing the resource allocation, placement, and packaging architecture. We analytically model the PPAC of the chiplet-based AI accelerator and integrate it into an OpenAI gym environment to evaluate the design points. We also explore non-RL-based optimization approaches and combine these two approaches to ensure the robustness of the optimizer. The optimizer-suggested design point achieves 1.52× throughput, 0.27× energy, and 0.89× cost of its monolithic counterpart at iso-area.
more »
« less
System and Design Technology Co-optimization of Chiplet-based AI Accelerator with Machine Learning
With the availability of advanced packaging technology and its attractive features, the chiplet-based architecture has gained traction among chip designers. The large design space and the lack of system and package-level co-design methods make it difficult for the designers to create the optimum design choices. In this research, considering the colossal design space of advanced packaging technologies, resource allocation, and chiplet placement, we design an optimizer that looks for the design choices that maximize the Power, Performance, and Area (PPA) and minimize the cost of the chiplet-based AI accelerator. Inspired by the Bayesian approach for black-box function optimization, our optimizer guides the search space toward global maxima instead of randomly traversing through the search space. We analytically synthesize a dataset from the search space and train an ML model to predict the target value of our defined cost function at the optimizer-suggested points. The optimizer locates the optimum design choices from the specified search space (≥ 1M data points) with minimal iterations (≤ 200 iterations) and trivial run time.
more »
« less
- Award ID(s):
- 2153394
- PAR ID:
- 10428292
- Date Published:
- Journal Name:
- Proceedings of the Great Lakes Symposium on VLSI
- Page Range / eLocation ID:
- 697 to 702
- Subject(s) / Keyword(s):
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Parametric optimization techniques allow building designers to pursue multiple performance objectives, which can benefit the overall design. However, the strategies used by architecture and engineering graduate students when working with optimization tools are unclear, and ineffective computational design procedures may limit their success as future designers. In response, this re-search identifies several designerly behaviors of graduate students when responding to a conceptual building design optimization task. It uses eye-tracking, screen recording, and empirical methods to code their behaviors following the situated FBS framework. From these data streams, three different types of design iterations emerge: one by the designer alone, one by the optimizer alone, and one by the designer incorporating feedback from the optimizer. Based on the timing and frequency of these loops, student participants were characterized as completing partial, crude, or complete optimization cycles while developing their designs. This organization of optimization techniques establishes reoccurring strategies employed by developing designers, which can encourage future pedagogical approaches that empower students to incorporate complete optimization cycles while improving their designs. It can also be used in future research studies to establish clear links between types of design optimization behavior and design quality.more » « less
-
Fast-evolving artificial intelligence (AI) algorithms such as large language models have been driving the ever increasing computing demands in today’s data centers. Heterogeneous computing with domain-specific architectures (DSAs) brings many opportunities when scaling up and scaling out the computing system. In particular, heterogeneous chiplet architecture is favored to keep scaling up and scaling out the system as well as to reduce the design complexity and the cost stemming from the traditional monolithic chip design. However, how to interconnect computing resources and orchestrate heterogeneous chiplets is the key to success. In this paper, we first discuss the diversity and evolving demands of different AI workloads. We discuss how chiplet brings better cost efficiency and shorter time to market. Then we discuss the challenges in establishing chiplet interface standards, packaging, and security issues. We further discuss the software programming challenges in chiplet systems.more » « less
-
2.5D packaging has become a popular alternative to integrate advanced logic and memory chiplets for high-performance computing and artificial intelligence systems. In the conventional design flow, chiplets and packages are independently designed and then integrated at the assembly stage. To bridge the gap between chiplet designs and package designs, existing chiplet-package co-design methods iteratively optimize chiplet layouts to improve the performance of the entire system. However, Redistribution Layer (RDL) routing, which finishes the interconnections between chiplets at the package level and significantly affects the system performance, is neglected in the existing co-design flows. Therefore, this article proposes an effective chiplet-package co-design flow focusing on the RDL routing to optimize the package system performance dynamically. The proposed co-design flow can fill in the missing link, package-level co-optimization, of previous design flows. In the proposed co-design flow, we propose an efficient RDL routing algorithm to iteratively optimize the substrate layout based on the cross-boundary timing context extracted from both chiplets and the package. The proposed RDL routing algorithm has two critical techniques, including (1) a Maximal Independent Set-based (MIS-based) pin assignment method to dynamically optimize the pin positions of nets and (2) a network-flow-based router to generate routing layouts. Experimental results show that the proposed design flow can gradually improve the maximum frequency of a real design to the target performance, 400 MHz.more » « less
-
Multi-chiplet architectures enabled by glass interposer offer superior electrical performance, enable higher bus widths due to reduced crosstalk, and have lower capacitance in the redistribution layer than current silicon interposer-based systems. These advantages result in lower energy per bit, higher communication frequencies, and extended interconnect range. However, deformation of the package (warpage) in glass interposer-based systems becomes a critical challenge as system size increases, leading to severe mechanical stress and reliability concerns. Beyond a certain size, conventional packaging techniques fail to manage warpage effectively, necessitating new approaches to mitigate warpage induced bending with scalable performance for glass interposer based multi-chiplet systems. To address these inter-twined challenges, we propose a thermal-, warpage-, and performance-aware design framework that employsarchitecture and packaging co-optimization. The proposed framework disintegrates the surface and embedded chiplets to balance conflicting design objectives, ensuring optimal tradeoffs between performance, power, and structural reliability. Our experiments demonstrate that optimized multi-chiplet architectures from our design framework achieve up to64.7% performance improvementand40% power reductioncompared to traditional 2.5D systems to execute deep neural network workloads with lower fabrication costs.more » « less
An official website of the United States government

