A Bird's-Eye Tutorial of Graph Attention Architectures
Graph Neural Networks (GNNs) have shown tremendous strides in performance for
graph-structured problems especially in the domains of natural language
processing, computer vision and recommender systems. Inspired by the success of
the transformer architecture, there has been an ever-growing body of work on
attention variants of GNNs attempting to advance the state of the art in many
of these problems. Incorporating "attention" into graph mining has been viewed
as a way to overcome the noisiness, heterogenity and complexity associated with
graph-structured data as well as to encode soft-inductive bias. It is hence
crucial and advantageous to study these variants from a bird's-eye view to
assess their strengths and weaknesses. We provide a systematic and focused
tutorial centered around attention based GNNs in a hope to benefit researchers
dealing with graph-structured problems. Our tutorial looks at GNN variants from
the point of view of the attention function and iteratively builds the reader's
understanding of different graph attention variants.