Weights α i are computed with attention mechanism implemented as a two-layer neural network<cite> [16]</cite> : the first layer produces a hidden representation u i = tanh(W u q i + b u ) and the second layer outputs unnormalized importance a i = W a u i + b a .