We further dissect the relationship between graph representations and normalization in Appendix , confirming that poor performance on large graphs can be explained by unstable representations due to the choice of aggregation and normalization schemes.
Using Max aggregators and BatchNorm with batch statistics are temporary hacks to overcome the failure of the current architectural components.
Overall, our results suggest that inference beyond training sizes will require the development of mechanisms that are both expressive as well as invariant to distribution shifts [1]}, [2]}.
