Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting
In this paper, we propose a novel end-to-end user-defined keyword spotting
method that utilizes linguistically corresponding patterns between speech and
text sequences. Unlike previous approaches requiring speech keyword enrollment,
our method compares input queries with an enrolled text keyword sequence. To
place the audio and text representations within a common latent space, we adopt
an attention-based cross-modal matching approach that is trained in an
end-to-end manner with monotonic matching loss and keyword classification loss.
We also utilize a de-noising loss for the acoustic embedding network to improve
robustness in noisy environments. Additionally, we introduce the LibriPhrase
dataset, a new short-phrase dataset based on LibriSpeech for efficiently
training keyword spotting models. Our proposed method achieves competitive
results on various evaluation sets compared to other single-modal and
cross-modal baselines.