Our proposed system architecture is illustrated in Figure REF . It utilizes a cnn-based front-end which takes in a set of variable length mel-spectrogram features and outputs a sequence of speaker representations. These speaker representations are further subject to a Double mha pooling which is the main contribution of this work. The Double mha layer comprises a Self mha pooling and an additional Self Attention layer that summarizes the information of each head context vector into a unique speaker embedding. The combination of Self mha pooling together with this Self Head Attention layer provides us with a deeper self-attention pooling mechanism (Figure REF ). The speaker embedding obtained from the pooling layer is sent through a set of fc layers to predict the speaker posteriors. This network architecture is trained with ams loss [1]} as a speaker classifier so as to have a speaker embedding extractor.
