It is a legitimate question whether it would be possible to devise a fully-quantum Transformer, i.e. a quantum circuit acting upon qubits in a fashion similar to a combination self-attention and positional embedding. While we cannot offer a definite answer here, we argue that most of the building blocks of the basic Transformer have a corresponding quantum operation, as arithmetic operations and exponentiation can be implemented as quantum circuits [1]}, or that they can be cast in form of variational quantum circuits. It is the sheer size of the computation that is likely beyond the limits of current quantum computers, although we foresee that this situation may change rapidly over the course of this decade. The end of this paper is a good place to circle back to the original question of what a Transformer actually is, and why it is so successful at interpreting human language. While a number of interpretations have been given in terms of computations on graphs [2]}, [3]}, in due course quantum computing may offer alternative ways to apply attention to sequential data, or provide some completely different conceptual framework altogether [4]}. Only time will tell.
