
where \(\mathbf {{\rm MLP}}(\cdot )\)  is a 1-hidden-layer MLP with ReLU activation and layer normalization [1]} applied before the activation.
For aggregating the message, the GNS model used either mean or sum, so we tried both in our experiments. We found sum aggregation to perform better, and report results of sum aggregation in this paper.
After the messages are aggregated, GNS uses a learnable linear function to transform the node embeddings.
Similar to ForceNet, we additionally apply a batch normalization on the node embeddings, which alleviates training instability and significantly improves performance.
The GNS model uses a residual connection, where the computed node embeddings are added into the node embeddings from the previous layer. For the decoder, the GNS model uses a 1-hidden-layer MLP with ReLU activation. All the node embeddings and hidden units in the MLPs have the same dimensionality.
