To the best of knowledge the first universal feed-forward style transfer method was proposed by Chen et al. [1]}. After replacing each content feature with it's nearest neighbor style feature a decoder can be trained to reconstruct pixels from the resulting representation (As an alternative to the optimization based algorithm also proposed in [1]}). In contrast to this non-parametric approach Huang et al. [3]} proposed extending the parametric approach of [4]}, replacing a bias and scaling term learned for each style with a bias and scaling terms computed on the fly based on the statistics of features extracted from the style image. This technique, called Adaptive Instance Normalization (AdaIN), was extremely influential and has been applied to many image synthesis tasks [5]}, [6]}, [7]}. To better capture the second order statistics of the style features Li et al. [8]} propose replacing the scaling vector of [3]} with a 'whitening' matrix derived by applying zero-phase component analysis (ZCA) to the covariance matrix of content features and a 'coloring' matrix similarly derived from the covariance of style features. This pair of affine maps has been termed the whitening and coloring transfrom (WCT). Chiu [10]} theoretically analyzed the WCT and showed that in comparison to transforms derived from PCA and the Cholesky decomposition of the covariance matrix, using the WCT (derived from ZCA) will create features with a lower upper bound on the content loss of Gatys. Lu et al. [11]} went a step further and derives a modified affine transform inspired by optimal transport that matches second order statistics while minimizing distance from the original content features. In a similar spirit Chiu et al. [12]} proposes using gradient descent directly on the neural content features to trade off between the content loss and style loss, while this is more expensive than the closed form solution proposed by [11]} it allows non-linear modifications to the features. An et al. [14]} demonstrates that the feature transforms proposed by (WCT, AdaIn) can be applied to the representations extracted by 'flow models' (invertible neural networks that learn to map a data distribution to gaussian which can easily be sample from). This solves the problems of style transfer being unstable (changing after multiple rounds of stylization) and the uncertainty of whether a trained decoder is perfect (because the flow model is inverible).
