Despite the early discovery of predictive coding in neuroscience, the idea of predictive coding is at the inception level in machine learning until recently. Contrastive Predictive Coding [1]}, [2]} is a self-supervised representational learning technique inspired by Predictive Coding. In a CPC, the loss is applied in the representational space instead of the output space. To this end, the main objective of a CPC model is to distinguish represenations, instead of optimizing output loss function. CPC has produced impressive results due to its ability to find generalized representations suitable for various downstream tasks.
