@inproceedings{
wang2024stdface,
title={S2{TD}-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketch},
author={Zidu Wang and Xiangyu Zhu and Jiang Yu and Tianshuo Zhang and Zhen Lei},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=OQ9dB9s7P2}
}
3D textured face reconstruction from sketches applicable in many scenarios such as animation, 3D avatars, artistic design, missing people search, etc., is a highly promising but underdeveloped research topic. On the one hand, the stylistic diversity of sketches leads to existing sketch-to-3D-face methods only being able to handle pose-limited and realistically shaded sketches. On the other hand, texture plays a vital role in representing facial appearance, yet sketches lack this information, necessitating additional texture control in the reconstruction process. This paper proposes a novel method for reconstructing controllable textured and detailed 3D faces from sketches, named S2TD-Face. S2TD-Face introduces a two-stage geometry reconstruction framework that directly reconstructs detailed geometry from the input sketch. To keep geometry consistent with the delicate strokes of the sketch, we propose a novel sketch-to-geometry loss that ensures the reconstruction accurately fits the input features like dimples and wrinkles. Our training strategies do not rely on hard-to-obtain 3D face scanning data or labor-intensive hand-drawn sketches. Furthermore, S2TD-Face introduces a texture control module utilizing text prompts to select the most suitable textures from a library and seamlessly integrate them into the geometry, resulting in a 3D detailed face with controllable texture. S2TD-Face surpasses existing state-of-the-art methods in extensive quantitative and qualitative experiments. The code will be publicly available.
S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single Sketch
Zidu Wang Xiangyu Zhu Jiang Yu Tianshuo Zhang Zhen Lei