For training, we use the Adam optimizer [1]}, with an initial learning rate of 0.0005.
We train ForceNet and the GNS model for 500K iterations with the batch size of 256, which is equivalent to 1 epoch for the entire datasetWe do not observe much gain by training models longer than 1 epoch. This is probably because of the redundancy in data, i.e., out of 130M data points, there are 650k unique atom configurations (ignoring the positional differences)..
For ForceNet-large, we use the batch size of 512.
All the parameters of the force-centric models are initialized with Xavier uniform initialization [2]}. The learning rate is kept constant for the first 250K iterations, after which it is halved every 50K iterations.
We use the checkpoint with the best validation ID performance, and evaluate the saved model over all four validation sets. MAE over forces is used as the training loss. We will evaluate our models on the hidden test sets once the test server is ready.
