Personalized Keyword Spotting through Multi-task Learning
Keyword spotting (KWS) plays an essential role in enabling speech-based user
interaction on smart devices, and conventional KWS (C-KWS) approaches have
concentrated on detecting user-agnostic pre-defined keywords. However, in
practice, most user interactions come from target users enrolled in the device
which motivates to construct personalized keyword spotting. We design two
personalized KWS tasks; (1) Target user Biased KWS (TB-KWS) and (2) Target user
Only KWS (TO-KWS). To solve the tasks, we propose personalized keyword spotting
through multi-task learning (PK-MTL) that consists of multi-task learning and
task-adaptation. First, we introduce applying multi-task learning on keyword
spotting and speaker verification to leverage user information to the keyword
spotting system. Next, we design task-specific scoring functions to adapt to
the personalized KWS tasks thoroughly. We evaluate our framework on
conventional and personalized scenarios, and the results show that PK-MTL can
dramatically reduce the false alarm rate, especially in various practical
scenarios.