The method is implemented in Python 3.6 using Tensorflow 1.7. A server with Intel Xeon Platinum 8168 CPU and 2 Tesla V100 GPUS are used to conduct all experiments. We trained the model using Adam optimizer with 0.01 as learning rate which decreases every 10 epochs after the 40th iteration. We used a total of \(C=2\)  attention heads with an embedding size of \(F=16\) . In addition we used a 2 layer GRU, with 64 units, a batch size of 64, and the scheduled sampling technique as described in [1]}.
