@inproceedings{
du2024ldbfr,
title={{LD}-{BFR}: Vector-Quantization-Based Face Restoration Model with Latent Diffusion Enhancement},
author={Yuzhen Du and Teng Hu and Ran Yi and Lizhuang Ma},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=ogfSPe0ff1}
}
Blind Face Restoration (BFR) aims to restore high-quality face images from low-quality images with unknown degradation. Previous GAN-based or ViT-based methods have shown promising results, but have identity details loss once degradation is severe; while recent diffusion-based methods work on image level and take a lot of time to infer. To restore images in any degradation types with high quality and spend less time, we propose LD-BFR, a novel BFR framework that integrates both the strengths of vector quantization and latent diffusion. First, we employ a Dual Cross-Attention vector quantization to restore the degraded image in a global manner. Then we utilize the restored high-quality quantized feature as the guidance in our latent diffusion model to generate high-quality restored images with rich details. With the help of the proposed high-quality feature injection module, our LD-BFR effectively injects the high-quality feature as a condition to guide the generation of our latent diffusion model. Extensive experiments demonstrate the superior performance of our model over the state-of-the-art BFR methods.
LD-BFR: Vector-Quantization-Based Face Restoration Model with Latent Diffusion Enhancement
Yuzhen Du Teng Hu Ran Yi Lizhuang Ma