Speech Sequence Embeddings. One straightforward method to obtain an SSE is to perform a max pooling along the time axis of the frame-wise speech features. This method (combined with a PCA) have been used in [1]} to represent speech sequences obtained by unsupervised phoneme segmentation. Another method involves downsampling the sequence into a fixed number of frames and concatenate them into a single vector [2]}. Other methods use bottleneck auto-encoders [2]}.
More advanced methods use Siamese loss [4]} or contrastive learning [5]} and rely on unsupervised ways to obtain positive pairs. Similarly, auto-encoders with a correspondence objective [6]} can encode a member of a pair and reconstruct the other member of the pair. Correspondance and Siamese losses can be combined effectively in a model called CAE-Siamese that is on par with the current state-of-the-art in SSE models [7]}, [8]}.
