@inproceedings{
lu2024d,
title={3D Priors-Guided Diffusion for Blind Face Restoration},
author={Xiaobin Lu and Xiaobin Hu and Jun Luo and zhuben and paulruan and Wenqi Ren},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=k7T9PYFm0l}
}
Blind face restoration aims to restore a sharp face image from a degraded counterpart. Recent methods using GANs as priors have achieved many successful stories in this domain. However, these methods still struggle to balance realism and fidelity when facing complex degradation scenarios. In this paper, we propose a novel framework by embedding 3D facial priors into a denoising diffusion model, enabling the extraction of facial structure and identity information from 3D facial images. Specifically, the downgraded image undergoes initial processing through a pre-trained restoration network to obtain an incompletely restored face image. This image is then fed into the 3D Morphable Model (3DMM) to reconstruct a 3D facial image. During the denoising process, the structural and identity information is extracted from the 3D prior image using a multi-level feature extraction module. Given that the denoising process of the diffusion model primarily involves initial structure refinement followed by texture detail enhancement, we propose a time-aware fusion block (TAFB). This module can provide more effective fusion information for denoising as the time step changes. Extensive experiments demonstrate that our network performs favorably against state-of-the-art algorithms on synthetic and real-world datasets for blind face restoration.
3D Priors-Guided Diffusion for Blind Face Restoration
Xiaobin Lu Xiaobin Hu Jun Luo zhuben paulruan Wenqi Ren