Until shortly before 'Deformable Style Transfer' was published, style transfer methods could not transfer geometric style at all and were limited to transferring color and texture. However, several recent works had tackled geometric style transfer for faces. Cole et al. [1]} focused on generating frontal, uniformly lit, views of faces from normal photographs. One component of their method was estimating a face 'texture' by predicting a sparse set of face landmarks, then using these as keypoints to warp the input face onto a canonical set of front facing landmarks using thin-plate cubic splines (which interpolate the sparse offsets defined by keypoint pairs to form a dense warping field which defines the new location in the output of every pixel in the input). Their proposed warping module was differentiable and thus easily added to a system trained end-to-end through backpropagation. Subsequent works combined image warping with feed-forward style transfer techniques to learn both textural and geometric stylizations of faces. CariGAN [2]} translates a photo to a caricature by training two GANs. One is a standard texture transfer GAN, which alters the texture of a photograph to resemble a caricature (similar to the stylization loss of [3]} except the style is caricatures instead of Van Gogh or Cezanne); The second GAN learns to predict offsets to facial landmarks extracted from the input photo which results in transformed landmarks that match the distribution of caricature landmarks. WarpGAN  [4]}, tackle the same task but with a more flexible warping module, learning to predict both the locations of keypoints (rather than using a canonical set of landmarks) and their displacements. Both CariGAN and WarpGAN rely on a dataset of caricatures with manually labeled keypoints. Face of Art (FoA) [5]} trains a neural network model to automatically detect 68 canonical facial landmarks in artistic portraits (supervised using a labeled dataset of landmarks in various artists' portraits). These landmarks can then be used as the keypoints of a warp field to match the geometry of an input face to a target artistic portrait.
