Prior work on personalizing end-to-end models proposes finetuning pre-trained ASR models using small amounts of speaker-specific utterances [1]}, [2]}. While these works propose different finetuning schedules that would be most beneficial given the limited amount of target data, the question of which speaker-specific utterances should be chosen in order to be transcribed and further used for finetuning has received far less attention. [3]} presents a method to select sentences within a fixed budget that are most likely to induce ASR errors and result in higher-quality personalized ASR models for target accents compared to random selection. However, they assume access to a small seed set of labeled utterances from the target speaker. We address the more realistic setting where we have access only to a limited number of unlabeled utterances from the target setting.
