Deformable Style Transfer relied on finding correspondences (i.e. paired keypoints) between two input images in an arbitrary domain. Several recent works had developed techniques leveraging CNN-derived features to find such correspondences. Fully Convolutional Self-Similarity  [1]} is a descriptor for dense semantic correspondence that uses local self-similarity to match keypoints among different instances within the same object class. Neural Best-Buddies (NBB) [2]} is a more robust method (which can operate across object classes) for finding a set of sparse correspondences by finding mutual nearest neighbors in the deep layers of a pretrained network, then spatially refining these correspondences by examining activations in earlier layers of the network. We used NBB as the basis for generating correspondences between the content and style images, and give a more detailed description of NBB and our modifications in next section.
