
where an input token sequence \(X=\lbrace x_1,...,x_i,...,x_N\rbrace \) , and \(x_i\)  is a sub-word (e.g., a BPE token [1]}), encoded by a sequence of contextual dense vectors \(\mathbf {H}=\lbrace \mathbf {h}_1,...,\mathbf {h}_i,...,\mathbf {h}_N\rbrace \) .
