Contrast learning has been successfully applied in the area of self-supervision learning ([1]}, [2]}), especially on computer vision tasks ([3]}).
Many extensions and improvements have been made, including w.r.t. the learning loss function ([4]}), data augmentation techniques ([5]}), network architectures ([2]}) and computational efficiency ([7]}). Our proposed method is partially based on this technique, with two major differences. First, our technique addresses CDE problem and not density estimation or self-supervised learning. Second, the distribution we choose is unknown, potentially intractable, and/or with a large set of highly dependent components, which violates the usual restrictions for performing noise contrastive density estimation but allows us to taylor the noise distribution precisely to the estimated distribution.
This allows us to fit the noise distribution precisely to the estimated distribution. To the best of our knowledge, the technique closest to our method is designed to evaluate the deviation from the independence setting, \(i.e.\)  when all random features \(\lbrace X_j\rbrace _{j=1,\cdots ,p}\)  are independent, in an unsupervised framework. It has been briefly described in the second edition of [8]} (pages 495-497) where, based on the noise contrastive reformulation given above with \(g(\cdot ) = \Pi _{j=1}^{p} \,{\textit {f}}_{X_j}\,(\cdot )\) . Note that \(g\)  is unknown and corresponds to what we will call later the noise distribution. Nevertheless, noise samples can be generated by applying a random permutation on the feature columns of the dataset, which is sufficient to discover association rules between the \(X\)  features but not to estimate the density function \(\,{\textit {f}}_{X}\,\) . On the other hand, although each component of \(X\)  is one-dimensional by definition, the errors made when estimating the marginal densities \(p\)  will compound when estimating \(g\) , which means that the dimensionality of \(X\)  is again a limiting factor.
