The Transformer network<cite> (Vaswani et al., 2017)</cite> avoids the recurrence completely and uses only self-attention. We propose a modified Transformer network wherein the multi-head attention layer is replaced by a branched self-attention layer.