skip to main content

Title: Sparse Tensor Core: Algorithm and Hardware Co-Design for Vector-wise Sparse Neural Networks on Modern GPUs
; ; ;
Award ID(s):
1725447 1730309 1817037
Publication Date:
Journal Name:
Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture
Page Range or eLocation-ID:
359 to 371
Sponsoring Org:
National Science Foundation
More Like this
  1. Sparse principal component analysis and sparse canonical correlation analysis are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data. Both problems can be formulated as an optimization problem with nonsmooth objective and nonconvex constraints. Because nonsmoothness and nonconvexity bring numerical difficulties, most algorithms suggested in the literature either solve some relaxations of them or are heuristic and lack convergence guarantees. In this paper, we propose a new alternating manifold proximal gradient method to solve these two high-dimensional problems and provide a unified convergence analysis. Numerical experimental results are reported to demonstrate the advantages of our algorithm.