The the success of BERT (Devlin et al., 2018) and Word2Vec (Mikolov et al., 2013) for NLP tasks motivated more research on self-supervised approaches for acoustic word embedding and unsupervised acoustic feature representation (Bengio and Heigold; Levin et al.; Chung et al., b; He et al.; van den Oord et al., 2018;<cite> Baevski et al., 2019b)</cite> , either by predicting masked discrete or continuous input, or by contrastive prediction of neighboring or similarly sounding segments using distant supervision or proximity in the audio signal as an indication of similarity.