[1]} showed that using relative position was important for achieving good accuracies. We find the same outcome with HaloNet. Using absolute factorized abosolute position encodings, which are added to the activations before local self-attention in every layer, drops accuracy from to \(78.6\%\)  (the first row in Table REF ) to \(77.5\%\) 
