We apply the method described in [1]} to find more positive pairs with k-Neareast-Neighbors search (step 1 and 2 in Figure REF ).
We extract sub-sequences of frames within each VA ensuring they are all shifted by the average phoneme length (\(80ms\) ). Next, we apply the SSE model pretrained according to our initialisation method (section REF ) to turn sampled sub-sequences of frames into embeddings, which are indexed for k-NN search using the FAISS library [2]}.
For each embedding \(s\) , the search returns \(N\)  closest neighbors according to the cosine distance. We trim down this number to construct pairs that are more likely to have the same phonetic content as follows:
First, all neighbors that are temporally overlapping with \(s\)  are removed. Second, when two neighbors are temporally overlapping with each other, we keep only the one that has the smallest cosine distance with \(s\)  (Non-Maximal-Suppression [1]}).
Finally, we apply a distance threshold above which all pairs are discarded. We set it such that half of speech sequences indexed in the k-NN graph have at least one selected pair. This was estimated on a sample of LibriSpeech (which functions as our dev set) where we found out that the proportion of speech sequence (excluding silences) with a unique transcription was around \(50\%\) . All selected pairs were used to form batches of positive pairs to retrain a new SSE model from scratch (step 3 in Figure REF ). We iterate the procedure by re-embedding the speech sequences into a new k-NN index, re-sampling positive pairs and retraining another SSE model.
<FIGURE>