In our experiments, all training process is completed on a work station with an AMD 7742 2.25G CPU, 256G memory, a single RTX 3090 GPU, and Ubuntu 20.04.
For the pre-trained BERT, we reuse the base cased English model released by Huggingfacehttps://huggingface.co/bert-base-cased, which contains 12 Transformer blocks and the hidden size \(d\)  is 768.
We tune our model on the valid set and use grid search to adjust important hyper-parameters.
Specifically,
the batch size is set to 8/6 on NYT/WebNLG, and all parameters are optimized by Adam algorithm [1]} with a learning rate of 1e-5.
The dimension of the hidden layer \(d_e\)  is set to \(3 \times d\) , the dropout probability in equation (REF ) is 0.1, the max sequence length is set to 100.
