Higher-Order Attention Networks
This paper introduces higher-order attention networks (HOANs), a novel class
of attention-based neural networks defined on a generalized higher-order domain
called a combinatorial complex (CC). Similar to hypergraphs, CCs admit
arbitrary set-like relations between a collection of abstract entities.
Simultaneously, CCs permit the construction of hierarchical higher-order
relations analogous to those supported by cell complexes. Thus, CCs effectively
generalize both hypergraphs and cell complexes and combine their desirable
characteristics. By exploiting the rich combinatorial nature of CCs, HOANs
define a new class of message-passing attention-based networks that unifies
higher-order neural networks. Our evaluation on tasks related to mesh shape
analysis and graph learning demonstrates that HOANs attain competitive, and in
some examples superior, predictive performance in comparison to
state-of-the-art neural networks.