The above observation inspires us to connect <cite>Transformer's</cite> attention to kernel learning (Scholkopf and Smola, 2001) : they both concurrently and order-agnostically process all inputs by calculating the similarity between the inputs.