We can further visualize this phenomenon through 2D Principal Component Analysis (PCA) plots of graph embedding spaces for GNN-Max and GNN-Max + learnt BN models, see Figures REF  and REF .
In both cases, the graph embeddings at larger sizes have very similar magnitudes and are extremely close to each other, indicating that the model is unable to differentiate among different graphs.
Thus, decoders currently lack good global structural context.
Investigating better graph embeddings through attention [1]} or hierarchical methods [2]} could be an interesting approach towards representing global graph structure beyond training sizes.
<FIGURE><FIGURE>