Pre-training of heterogeneous graph neural networks

Speaker: Yuxiao Dong, Microsoft Research

Abstract

Recent years have witnessed the emergent success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous networks, in which all nodes or links have the same feature space and representation distribution, making them infeasible for representing real-world evolving heterogeneous graphs, such as knowledge graphs. In this talk, I will introduce GNN architectures that can model billion-scale heterogeneous graphs with dynamics. The focus will be on how we design the graph attention and temporal encoding mechanisms to capture the heterogeneous and dynamic natures of real-world graphs. With this, I will further discuss the strategies of pre-training such GNNs for general graph mining tasks. Finally, to handle Web-scale data, I will introduce the heterogeneous mini-batch graph sampling algorithm for efficient and scalable training. Extensive experiments show the promise of GNN pre-training for billion-scale (knowledge) graphs in practice.

The slides are available here.

Bio

The bio is available here.