For the weight predcition model in Figure REF , Robert-Basehttps://huggingface.co/roberta-base is used as the pre-trained language model. Multi-head attention [1]} is used as the cross-Attention module, and we set \(head = 6\)  and \(dropout = 0.3\)  in this work. A single linear layer is used as the classifier, with a \(dropout = 0.5\) . Cross-Entropy is used to calculate the loss between each predicted logits and true labels. AdamW is used as the optimizer, with \(learning rate = 3e-5\)  and \(eps = 1e-8\) . Training epoch is set as 20, and an early stopping mechanism is applied on the validation step to select the best model.
