Attention-based conditioning methods using variable frame rate for style-robust speaker verification
We propose an approach to extract speaker embeddings that are robust to
speaking style variations in text-independent speaker verification. Typically,
speaker embedding extraction includes training a DNN for speaker classification
and using the bottleneck features as speaker representations. Such a network
has a pooling layer to transform frame-level to utterance-level features by
calculating statistics over all utterance frames, with equal weighting.
However, self-attentive embeddings perform weighted pooling such that the
weights correspond to the importance of the frames in a speaker classification
task. Entropy can capture acoustic variability due to speaking style
variations. Hence, an entropy-based variable frame rate vector is proposed as
an external conditioning vector for the self-attention layer to provide the
network with information that can address style effects. This work explores
five different approaches to conditioning. The best conditioning approach,
concatenation with gating, provided statistically significant improvements over
the x-vector baseline in 12/23 tasks and was the same as the baseline in 11/23
tasks when using the UCLA speaker variability database. It also significantly
outperformed self-attention without conditioning in 9/23 tasks and was worse in
1/23. The method also showed significant improvements in multi-speaker
scenarios of SITW.