Specifically, Ashish et.al<cite> [34]</cite> compute the attention function on a set of queries simultaneously, packed together into a matrix Q, while the keys and values are also packed together into matrices K and V , respectively.