In this paper, we leverage a new type of Transformer, the Synthesizer Transformer [1]}. To properly understand our architecture, we first provide an overview of the Vanilla Transformer architecture upon which our proposed model is based.
