Recently, several extensions of Graph Convolutional Networks for heterogeneous graphs have been proposed [1]}, [2]}, [3]}, [4]}, [5]}. These proposed models often operate on moderate-size to large-scale graphs. One such heterogeneous graph model, HGT [5]}, showed promising results on Open Academic Graph [7]} consisting of 179 million nodes and 2 billion edges. However, the proposed version of HGT is trained on a single machine and requires all the training data (heterogeneous graph and node features) to be present on that single system. At Pinterest, in an arbitrary version of our graph, we have 2+ billion pins, 2+ billion boards, 7+ billion pin-board edges and 400+ million users [8]}.
The storage of graph and node features itself requires more than 3TB of space. Hence, given the scale of the data at Pinterest, training existing heterogeneous graph models on a single machine is not feasible. There also exist a few frameworks such as Distributed Deep Graph Library (DistDGL) [9]} and Aligraph [10]} that support distributed training of heterogeneous GNNs models. DistDGL and Aligraph trained distributed GCNs models on Ogbn-papers100m [11]} (\(\sim \) 111 million nodes) and Taobao-large (\(\sim \) 483 million nodes) graphs, respectively. Both DistDGL and Aligraph rely on multi-processing for "on-the-fly" sampling of node's neighbors. On a 4+ billion node graph, the "on-the-fly" sampling of neighbors in every epoch or after a few epochs, results in a high training time of the model. The running time of sampling can be decreased by increasing the number of workers but that results in additional costs.
Hence, it is challenging to train the heterogeneous graph models on web-scale graphs.
