To protect privacy in social media photos without obtruding on user experience [1]}, face in-painting approaches have been proposed. Mahajan et al. provided the SwapItUp framework to swap in a reference face [2]}. Using the detected facial landmarks, pose and feature estimation is performed to query a nearest-matching face from a reference dataset. This face is aligned, color matched and blended over the input.
Sun et al. provided a GAN-based framework [3]} for facial in-painting in social media photos. Given a partial or full-body image, facial landmarks are detected or generated using an adversarial auto-encoder network. They then perform a structure-guided image generation. Their method improves privacy protection against context-aware models, which are shown to fixate on the "fake" face. When context-aware models predict blurred or masked faces, body cues are prioritized and the facial region is largely ignored. However, these approaches lack control over the resulting identity or temporal consistency when applied to video.
