In <cite>transformer</cite>, both Q and K are from the source sequence, while in the conventional RNN-based attention model [19] , Q is from the decoder hidden state, and K is from the encoder hidden state.