@inproceedings{
wang2024uvmapid,
title={{UVM}ap-{ID}: A Controllable and Personalized {UV} Map Generative Model},
author={Weijie Wang and Jichao Zhang and Chang Liu and Xia Li and Xingqian Xu and Humphrey Shi and Nicu Sebe and Bruno Lepri},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=BJK6MBxIId}
}
Recently, diffusion models have made significant strides in synthesizing realistic 2D human images based on provided text prompts. Building upon this, researchers have extended 2D text-to-image diffusion models into the 3D domain for generating human textures (UV Maps). However, some important problems about UV Map Generative models are still not solved, i.e., how to generate personalized texture maps for any given face image, and how to define and evaluate the quality of these generated texture maps. To solve the above problems, we introduce a novel method, UVMap-ID, which is a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model which is integrated with a face fusion module for achieving ID-driven customized generation. To support the finetuning strategy, we introduce a small-scale attribute-balanced training dataset, including high-quality textures with labeled text and Face ID. Additionally, we introduce some metrics to evaluate the multiple aspects of the textures. Finally, both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation.
UVMap-ID: A Controllable and Personalized UV Map Generative Model
Weijie Wang Jichao Zhang Chang Liu Xia Li Xingqian Xu Humphrey Shi Nicu Sebe Bruno Lepri