From this point, research began to emerge that focused on style transfer as an independent task, primarily taking inspiration from [1]} and refining the framework of 'image analogies'. Rosales et al. [2]} relax the 'supervised' requirement of 'image analogies' (requiring an 'unfiltered style image'). They did so by replace the greedy synthesis algorithms common in texture synthesis (which iteratively chose a pixel/patch from the texture/style image to place at each location) with minimizing the energy of a Markov Random Field (MRF) over patch assignments via loopy belief propagation (where unary potentials describe the style patches' distances to a content patch, and binary potentials describe the distance between a selected style patch and a neighboring selected style patch in regions of overlap). This more sophisticated matching algorithm made patch assignments robust enough that the 'unfiltered style' was no longer needed as guidance. Cheng et al. [3]} used the MRF formulation of [2]}, but remained the the regime of 'supervised' style transfer [5]} where access to an unfiltered style image was assumed. They improve patch assignments by using metric learning to find a smooth distance function which minimized the distance between matching patches in the filtered and unfiltered style images, then using this data-driven metric to choose which patches to use when stylizing the content image. This work also proposed an elegant implementation detail which I have not seen elsewhere, recommending matching octagonal patches instead of square ones, ensuring that each region of overlap need only harmonize two patches (as opposed to square patches which must harmonize 4 patches at the corners). In addition the overlap of octagonal patches tapers to a point at either end, making them well suited to being joined using the minimum error boundary cut of [6]}. Barnes et al. [7]} focused on improving the computational efficiency of [5]}, and proposed a pre-computed data structure that supports efficiently querying similarity with a predetermined library of patches. Zhang et al. [9]} use feature engineering to improve the unsupervised algorithm of [2]} by decomposing the style image into 'segmentation', 'paint', and 'edge' components (essentially low, middle, and high frequencies), then matching patches from these components independently. Frigola et al. [11]} also build upon the framework of [2]}, but do so by extending it to handle a MRF over patches of varying sizes, noting that often flat regions of a photograph (such as sky) can be matched to large regions of the style images (the large swirls of Starry Night), but details must be matched to small patches (individual brushstrokes).
<FIGURE>