A related area of research is data-driven 3d rendering, where a 2d rendering of a 3d model undergoes stylization. Unlike style transfer for arbitrary 2d images, in this case additional information, namely ground truth geometry and lighting, is available because the content image is produced using 3d modeling software. The most influential paradigm in this problem is the 'lit sphere' method proposed by Sloan et al. [1]}. In this framework an artist draws or paints over a 3d rendering of a sphere. Because a sphere contains all surface angles visible to a viewer the artist's stylized rendering can be propagated to an arbitrary 3d model under similar lighting conditions. This approach is spiritually similar to the 'image analogies' paradigm of [2]}, in that the original depth and lighting of the sphere is used as an 'unfiltered' source of features to match patches between the 3d model the artist's rendering. Later work by Fiser et al. [3]} exploited this similarity and improved the method by incorporating improvements in patch assignment using Markov Random Fields similar to those used in style transfer [4]}. A follow up paper by Fiser et al. [5]} speeds up the algorithm of [3]} by sparsely searching for matches between the 3d model and the sphere, then filling in the gaps by growing the patch of sphere used until it deviates from the target 3d model's geometry by a significant margin.
