einsum ( " hv , hdv−>d " , o , P_o) r e t u r n y Note: <cite>[Vaswani et al., 2017]</cite> include a constant scaling factor on the logits. We omit this in our code, as it can be folded into the linear projections P q or P k .