Accelerating Large-Scale Graph Neural Network Training on Crossbar Diet

TitleAccelerating Large-Scale Graph Neural Network Training on Crossbar Diet
Publication TypeJournal Article
Year of Publication2022
AuthorsC Ogbogu, AI Arka, BK Joardar,, H Li, K Chakrabarty, and PP Pande
JournalIeee Transactions on Computer Aided Design of Integrated Circuits and Systems
Volume41
Issue11
Start Page3626
Pagination3626 - 3637
Date Published11/2022
Abstract

Resistive random-access memory (ReRAM)-based manycore architectures enable acceleration of graph neural network (GNN) inference and training. GNNs exhibit characteristics of both DNNs and graph analytics. Hence, GNN training/inferencing on ReRAM-based manycore architectures give rise to both computation and on-chip communication challenges. In this work, we leverage model pruning and efficient graph storage to reduce the computation and communication bottlenecks associated with GNN training on ReRAM-based manycore accelerators. However, traditional pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. In this work, we propose a GNN pruning technique called DietGNN. DietGNN is a crossbar-aware pruning technique that achieves high accuracy training and enables energy, area, and storage efficient computing on ReRAM-based manycore platforms. The DietGNN pruned model can be trained from scratch without any noticeable accuracy loss. Our experimental results show that when mapped on to a ReRAM-based manycore architecture, DietGNN can reduce the number of crossbars by over 90% and accelerate GNN training by ${\sim }{2}.{7}{\times }$ compared to its unpruned counterpart. In addition, DietGNN reduces energy consumption by more than ${\sim }{3}.{5}{\times }$ compared to the unpruned counterpart.

DOI10.1109/TCAD.2022.3197342
Short TitleIeee Transactions on Computer Aided Design of Integrated Circuits and Systems