Consider an example of translating “I like this cell phone” to German “Ich mag dieses Handy” using a classical machine translation model (an RNN, LSTM, or GRU) and Transformer as illustrated in Figure REF . The input words must be first processed using an embedding layer to convert raw words into vectors of size \(d\) . The concept of embedding a discrete word into a continuous space of real numbers is a common practice in NLP [1]}, [2]}, [3]}, [4]}. In classical language translation models, each embedded word in a sentence corresponds to a specific RNN/LSTM/GRU cell. The operations in the subsequent cells depend on the output from the previous cell. Therefore, each embedded word in the input is processed sequentially after processing the preceding word. In a model based on the Transformer architecture, the entire input sequence “I like this cell phone” is fed into the model simultaneously, eliminating the need for sequential processing of the data. The order of the sequence is tracked using positional encoding.
<FIGURE>