With 64-dimensional embedding as default, we train our policy network, Q function network, graph encoder, \(\alpha \)  in SAC, and priority predictor using the Adam optimizer [1]},
an initial learning rate of 1e-3, weight decay of 1e-4, and a batch size of 256. Learning rates are reduced with ReduceLROnPlateau on PyTorch, with reduce factor 0.1 and patience steps 768 for policy and Q function, and 500 for priority predictor.
In case of SAC, initial soft actor-critic \(\alpha \)  values are set to 1 and [min, max] value set to [0.05, 20]. The learning rate of \(\alpha \)  in soft actor-critic was initialized with 5e-4.
Hyper-parameters for PER follow original setting in [2]}, with \(\alpha \)  in () set to 0.6, and \(\beta \)  in (REF ) set at \(min(1.0, \beta _{init} + idx * (1.0 - \beta _{init}) / \beta _{frames})\) , where \(\beta _{init} = 0.4\)  and \(\beta _{frames} = 1e+5\) .
Every trainable parameter is initialized with Xavier uniform initializer [3]}.
