Models for dynamic full-body humans have been widely studied because of their applications in the movie and gaming industries, as well as virtual and augmented reality. Human avatars are usually built to be animatable with joint angles and skinning to allow easy control of skeleton-level deformation. The classical approach for the modeling of full-body appearance is to use a fixed texture associated with the template mesh [1]}, [2]}, [3]}, which enables the rendering of the avatars by various graphics techniques such as direct shading or ray-tracing.
However, it is impossible for cheap rendering to account for the complicated appearance change under the variation of viewpoint, deformation, material and lighting.
Improved realism has been achieved in classic methods with techniques related to image-based warping [4]}, [5]}, [6]}, [7]}, but they do not enable body reposing.
To achieve photorealistic rendering of animatable full-body human appearance using classical techniques requires heavy computation and efforts from digital artists.
In recent years there has been a lot of interest in building photorealistic human avatars directly from captured images in a data-driven manner. In these approaches, a deep neural network takes as input a specific body configuration (for example pose or mesh vertices), and outputs a predicted image supervised by the ground truth capture from the same viewpoint [8]}, [9]}, [10]}, [11]}.
Data-driven approaches based on image translation networks have also attacked this problem purely from a monocular 2D perspective, using rendered skeletons [12]} or 2D body correspondences [13]} as input.
Some work further utilizes recent breakthrough in neural rendering, such as Neural Radiance Field [14]}, [15]}, [16]}, [17]}, [18]}, [19]}, [20]} and Deferred Neural Rendering [21]}, [22]}, [23]}. Notably, full-body Codec Avatars [24]}, [25]} have been proposed as a promising approach to achieving photorealistic telepresence. Most of these approaches consider clothing as rigidly attached to the human body, and the results are limited to tight clothing such as T-shirt and pants. One exception is the work by Habermann and colleagues habermann2021real, which can model the deformation of loose clothing such as skirts and dresses in a coarse-to-fine manner, but the predicted clothing animation does not deform in a physically realistic way because the approach lacks explicit physics priors.
