• Reduction layer: a new layer design that fits the pipeline proposed by Vaswani et al. <cite>[2]</cite> and compresses the input embedding size for subsequent layers (this is especially beneficial when employing pre-trained embeddings).