skip to main content

Title: HyScale-GNN: A Scalable Hybrid GNN Training System on Single-Node Heterogeneous Architecture
Award ID(s):
Author(s) / Creator(s):
Date Published:
Journal Name:
2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Page Range / eLocation ID:
557 to 567
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Recently, graph neural networks (GNNs), as the backbone of graph-based machine learning, demonstrate great success in various domains (e.g., e-commerce). However, the performance of GNNs is usually unsatisfactory due to the highly sparse and irregular graph-based operations. To this end, we propose TC-GNN, the first GNN acceleration framework based on GPU Tensor Core Units (TCUs). The core idea is to reconcile the "Sparse" GNN computation with the high-performance "Dense" TCUs. Specifically, we conduct an in-depth analysis of the sparse operations in mainstream GNN computing frameworks. We introduce a novel sparse graph translation technique to facilitate TCU processing of the sparse GNN workload. We implement an effective CUDA core and TCU collaboration design to fully utilize GPU resources. We integrate MGG with the PyTorch framework for high programmability. Rigorous experiments show an average of 1.70× speedup over the state-of-the-art DGL framework across various models and datasets. 
    more » « less