The distributed training frameworks for heterogeneous graphs in general addresses three main scalability challenges related to storage, sampling and GNN operations. The AliGraph [1]} system address these challenges by proposing three layers: i) Storage layer partitions the graphs using four graph partition algorithms and introduces indexing and caching operations for retrieval of node and its neighbors features, ii) Sampling layer identify the node's neighbors which are required by GNN. Sampling is efficient as each worker sample neighbors on the assigned graph partition, and iii) Operations layers that implement aggregation and combination operation of GNN in an optimized manner. DistDGL [2]} stores the graphs on multiple devices by partitioning the graph using METIS [3]} algorithm. The node features are stored using a distributed key-value store. For sampling, DistDGL introduced a distributed sampler that follows client-server model to samples the node neighbors in parallel. DistDGL perform random-walks on each partition and a single random-walk does not move across different partitions. We estimate that the migration to these new frameworks in the Pinterest software ecosystem would also incur significant infrastructure costs. P3 [4]} proposed a novel distributed training approach that eliminates partitioning overheads and high communication among workers. P3 introduced a pipelined push-pull parallelism based execution strategy for the distributed training of GNNs.
