The highly influential work of Gatys et al. [1]} demonstrated that, despite being learned from photographs, these features also capture the texture or 'style' of non-photorealistic artwork. In the spirit of Julesz the sufficient statistics of features taken from the early layers can be used to compute a 'style similarity' loss, and an output optimized to minimize this loss takes on many attributes of the desired style. Beyond this insight, other details of the style transfer algorithm proposed by [1]} have also had a huge impact. First, in addition to a 'style similarity' term the objective function also contains a 'content similarity' term based on deep layers of the network (which were assumed to capture semantics but not style) which is jointly minimized with the style loss, allowing trading off between content preservation and stylization. Second, [1]} used a variant of gradient descent to directly optimize the pixels of an output image to minimize the objective function, rather than the previously dominant approach of stitching together small patches from the target style. This paper sparked the sub-field of 'Neural Style Transfer', an umbrella which covers all of the work I will present as part of my thesis.
