Considering the relatively long length of sequential radial spokes, the application of the Transformer network is expected to learn the long-range dependencies between the spokes. As shown in Fig. REF (e), the Transformer has an encoder-decoder structure. The encoder maps an input of sequential radial spokes to sequential representations, which was used by the decoder to generate sequential output one spoke at a time. Each spoke is treated as a token in the network. At each step the decoder is auto-regressive, using the previously generated spokes to generate the next. The encoder consists of \(N=6\)  identical stacks, each having a multi-head self-attention sub-layer and a fully connected feed-forward sub-layer. After each sub-layer, a residual connection [1]} is implemented followed by layer normalization [2]}. The decoder also has \(N=6\)  identical stacks. In addition to the two sub-layers in encoder, the decoder has a third sub-layer, which learns the multi-head attention from the outputs. The output is offset by one position and is masked for each iteration so that the prediction for position \(pos\)  depends only on the known outputs at positions less than \(pos\) .
