Given the encoded sequence from the front-end, a pooling layer is adopted to obtain an utterance-level representation. During the last few years, there are several studies addressing different types of pooling strategies [1]}, [2]}, [3]}. X-vector originally uses statistical pooling [4]}. Self attention mechanisms have been used to improve statistical pooling, such as [5]}. In works like [6]}, attention is used to extract better order features statistics. A wide set of pooling layers based on self attention have been proposed improving this vanilla self attention mechanism. In [5]} several attentions are applied over the same encoded sequence, producing multiple context vectors. In our previous work [8]}, the encoded sequence is split into different heads and a different attention model is applied over each head sub-sequence. Non self attention mechanisms have also been proposed like [9]}, where a mutual attention network is fed with the pair of utterances aimed to compare.
