We also use the copy gate from<cite> (Makarov et al., 2017)</cite> : since the neural network copies the vast majority of its symbols, the output distribution p i is obtained as a weighted sum of singleton distribution which outputs current input symbol and the preliminary distribution p i specified above.