Wav2vec [1]} is a pre-trained model through self-supervision: it is learned to predict future samples from a current window analysis. This model is composed of two distinct convolutional neural networks. A first `encoder network' converts the audio signal into a new representation that is given to the second network, the `context network', which takes care of the context by aggregating multiple time step representation in order to use a receptive field of about 210 ms. Both are then used to compute a contrastive loss function. The resulting embedding is a 512-dimensional feature vector.
