Some recent works also introduce transformers to the field of graph machine learning.
GTN [1]} is performed on heterogeneous graphs with transformers by meta-path-based relation learning.
HGT [2]} is a transformer model for the representation learning on Web-scale heterogeneous graphs, which reaches state-of-the-art results on various downstream tasks.
GROVER [3]} integrates the message passing mechanism into the transformer architecture to learn representation for molecule graph data.
Graph-BERT [4]} constructs a BERT-like network model for static graph learning and introduces various well-designed tasks for self-supervised model pre-training [5]}.
