Geometry-aware Single-image Full-body Human Relighting

Single-image human relighting aims to relight a target human under new lighting conditions by decomposing the input image into albedo, shape and lighting. Although plausible relighting results can be achieved, previous methods suffer from both the entanglement between albedo and lighting and the lack of hard shadows, which significantly decrease the realism. To tackle these two problems, we propose a geometry-aware single-image human relighting framework that leverages single-image geometry reconstruction for joint deployment of traditional graphics rendering and neural rendering techniques. For the de-lighting, we explore the shortcomings of UNet architecture and propose a modified HRNet, achieving better disentanglement between albedo and lighting. For the relighting, we introduce a ray tracing-based per-pixel lighting representation that explicitly models high-frequency shadows and propose a learning-based shading refinement module to restore realistic shadows (including hard cast shadows) from the ray-traced shading maps. Our framework is able to generate photo-realistic high-frequency shadows such as cast shadows under challenging lighting conditions. Extensive experiments demonstrate that our proposed method outperforms previous methods on both synthetic and real images.


Introduction
Human relighting aims to relight a target human with correct shadow effects under a desired illumination. Relighting can realize seamless replacement of the background while ensuring the light consistency of the foreground and the background and has a huge application prospect in film-making, video chat and Virtual Reality. Single-image human relighting for a general human person is convenient and promising for amateur photographers but also more challenging because the difficulty of decoupling lighting, geometry, and surface material from a single image.
Most existing single-image human relighting methods focus on portrait relighting [40,50,53,37,5,16,18,33,38,43,44,47,51,32,46,61,60,22] and only a few works arXiv:2207.04750v2 [cs.CV] 12 Jul 2022 [28,25,11] focus on single-image full-body relighting. Since the style, texture and material of human clothes are widely varied and the geometry and poses of clothed humans are usually complex, decoupling the albedo and lighting is highly ill-posed. Moreover, mutual occlusion of limbs will produce self-shadows, which are not only difficult to remove for de-lighting tasks, but also challenging to generate under the target lighting conditions for relighting tasks. Previous fullbody relighting methods [28,25,11] have attempted to address these issues, but the results still have the following drawbacks: (i) inability to disambiguate albedo and lighting (ii) inability to model high-frequency shadows due to reliance on Spherical harmonics representation of lighting.
We propose a novel geometry-aware single-image human relighting framework that leverages SOTA single-image geometry reconstruction [42,19] for fully leveraging the traditional graphics rendering and neural rendering techniques simultaneously. First of all, accurate intrinsic decomposition for albedo estimation (de-lighting) is the cornerstone of producing high-quality relighting results. Previous methods [28,25,11] have used the UNet to infer albedo and suffer from severe entanglement between lighting and albedo. We discover that the skip connections in UNet is the culprit and found that HRNet [49] performs much better for de-lighting. In addition we further improve the de-lighting performance by: i) removing the aggressive down-sampling operations in the early stage, ii) eliminating skip connections and iii) fusing multi-scale features while maintaining high-resolution representations in the HRNet, and finally achieve better disentanglement between lighting and albedo on both synthetic and real images.
More importantly, even with high-quality albedo, we still need photo-realistic shadows to produce realistic relit results. Limited by the expressive ability of the spherical harmonics (SH) lighting model and the lack of 3D geometry information, previous methods [28,25,11] can only produce low-frequency shadows. Low-resolution environment maps [50,53] and pre-filtered environment maps [40] have also been explored and struggle to generate hard cast shadows. With the development of single-image 3D human reconstruction technology, obtaining highquality mannequins from a single image is possible, which can provide more 3D prior information for human relighting. Hence, we propose a geometry-aware relighting method, which consists of a ray tracing-based per-pixel lighting rep-resentation that explicitly models high-frequency shadows and a learning-based shading refinement module to restore realistic shadows. The key idea of our method is that with the estimated 3D human model, we can render photorealistic shading maps that encode full-band lighting information under target lighting conditions using ray tracing. This lighting representation is able to preserve high-frequency shadows and significantly improve the quality of the relit results which remains difficult for learning-based methods. However, the raytraced shadows still suffer from artifacts due to the errors in geometry estimation, which prevents direct composition of the relit results. Thus, we further propose a learning-based refinement module that utilizes the ray-traced shading maps and the inferred ambient occlusion map as input to restore photo-realistic high-frequency shadows with rich local details and clear shadow boundaries. The proposed framework dramatically improves relighting performance and produces a photo-realistic relit image with high-frequency self-shadowing effects under the target lighting conditions.
To conclude, our main contributions are the following: -A novel geometry-aware single-image human relighting framework that combines single-image 3D human reconstruction, ray tracing and neural rendering technologies. -We demonstrate the UNet with skip connections is not suitable for the delighting task and propose a modified HRNet for achieving better disentanglement between lighting and albedo on both synthetic images and real photographs. -We propose a ray tracing-based per-pixel lighting representation and a learningbased shading refinement module that utilizes the inferred ambient occlusion map as auxiliary input to restore photo-realistic shadows while preserving rich local details.

Person-Specific Human Relighting
Turner Whitted [56] first used recursive ray tracing to simulate global illumination and render realistic images under target lighting conditions. However, the forward rendering technique requires a high-precision 3d model and corresponding PBR textures of the target object, which are unavailable or costly for realworld objects. Debevec et al. [13] proposed collecting OLAT(one-look-at-a-time) images using LightStage to synthesize specific person's face under novel illuminations with no 3D model required. Subsequent works improved LightStage to capture higher-quality images and a larger range of human bodies [21,14,54,12,55,8,20,59]. Guo et al.[20] attempted to combine image-based rendering technique with geometry and material esrimated by LightStage and achieved unprecedented quality and photorealism for free viewpoint videos. However, collecting OLAT data, gradient data and object geometry requires professional equipment and it is a time-consuming and complicated process for person-specific relighting. Li et al.
[31] attempted to relight target human from multi-view video recorded under unknown illumination conditions and Imber et al. [23] extended this approach to scene relighting by introducing intrinsic textures. These two methods can produce high-quality relit results but multiview video must be collected for every target human or scene.

Inverse Rendering
Ramachandran et al. [41] proposed the shape-from-shading method to estimate shape from shading given an input image with known lighting conditions. Subsequent works [10,39,34] assume a simple light source model such as directional and point light sources. Based on the Retinex theory [29], intrinsic images [4,57,7,6,27,45,30,15] aims to decompose an input image into reflectance and shading. Recent single-image human relighting studies [25,28,11] have drawn on the idea of intrinsic images to decompose an input image into albedo, shape and illumination using UNet. We have further improved the intrinsic images decomposition results in the de-lighting stage utilizing the modified HRNet [49] and achieved better disentanglement between albedo and lighting.

Overview
Following current work [53], our framework consists of two stages: de-lighting and relighting. Fig. 2 shows the architecture of the whole framework. For the de-lighting stage, we first use two geometry networks (Geometry Module) to infer the per-pixel normal map N and ambient occlusion map AO separately. Similar to [40], the inferred normal map and the input image are concatenated as the input of Albedo Module to infer albedo A. For the relighting stage, we first render the 3D human models estimated by [42] and [19] (3D Recon Module) using ray tracing to obtain the coarse full-body shading map S body coarse and coarse face shading map S f ace coarse . Then the Refine Module takes these coarse shading maps and the inferred ambient occlusion map as input to produce the refined full-body shading map S body f ine and refined face shading map S f ace f ine , which are composited together to obtain the final shading map S. Details regarding the specific architecture and implementations of the networks are provided in the supplementary materials. Finally, the dot product of the inferred albedo and the final shading map is obtained to produce the relit image I relit using the following formula:

De-lighting
One of the most important factor for achieving better de-lighting results is the network architecture. Although the UNet architecture has been widely used in previous de-lighting networks [28,25,40,53,43], we find that it usually produces heavy lighting and albedo entangled results, possibly because of the fact that the lighting and albedo features remain coupled in the UNet architecture.
To guarantee that the lighting and albedo can be disentangled in the network architecture level, we modified the HRNet [49] architecture and achieved better delighting performance than UNet as shown in Fig. 3. Note that the vanilla HRNet can only produce oversmoothed albedo map due to the aggressive downsampling operations in the beginning stage. Even with skip connections between the input downsampling features and output upsampling features, the results of vanilla HRNet remain poor and even make unremoved shadows appear in the inferred albedo map, especially in outdoor scenarios. We assume that skip connections have a negative impact on the final de-lighting results because high-resolution features from the input still contain environmental lighting information. Based on this assumption, we modify the vanilla HRNet by removing the downsampling operations directly (which also avoid skip connections) to fuse multiscale features while maintaining high-resolution representations. Moreover, we modify HRNet to output lighting prediction at the layer with the lowest resolution of the final stage. This strategy guarantees a better decomposition between lighting and albedo at both the architecture and feature representation level. Details regarding the specific architecture and implementations are provided in the supplementary materials.
To estimate the albedo of an input human image, we employ losses as follows: where A is the inferred albedo and A is the ground truth albedo. Similar to portrait relighting [50], we use the weighted log-L 2 loss. I l and I l are latitudelongitude representation of environmental illumination and are used to describe the estimated lighting and ground truth lighting, respectively. The ground truth lighting map is downsampled to 16 × 32 × 3 using Gaussian pyramid. w is the solid angle of each "pixel". The L DL is the total loss for the de-lighting stage, and λ input , λ vgg and λ light are the weight factors. Empirically, we find λ input = 500, λ vgg = 100, λ light = 0.025 achieves the best performance.

Relighting
To render photorealistic shadows, we divide the relighting stage into two substages: ray tracing and shading refinement. We first estimate 3D models of the target human because the classical ray tracing algorithm requires a 3D model of target object. For geometry estimation, given a single human image, we use PIFuHD [42] to estimate a complete 3D human model. Then we crop the human face from the input image and align it with the dense 3D face model following [19] by regressing the 3DMM parameters. Both the full-body 3D model and face 3D model have no texture because we only need to render shading maps of the target human.

Ray Tracing
For photorealistic rendering, we use the Cycles rendering engine in Blender [1] and a Principled BSDF shader. We set the camera mode to orthographic to ensure that the rendered shading map is pixel-aligned with the input image. Due to the limitations of the PIFuHD [42] network's generation capacity and memory space, the surface of the estimated full-body 3D model is not smooth enough, which may produce checkered artifacts after raytracing. The schematic diagram of the artifacts is provided in the supplementary materials. To solve the problem, we adopt Laplacian smoothing [48] for the full-body 3D model and set smoothing steps to 10 with cotangent weight. Then the smoothed full-body model is sent to the renderer to render the full-body shading map under the target lighting environment using ray tracing. In addition, the generated 3D face model was directly sent to the renderer without smoothing for ray tracing.
In a word, we render a coarse full-body shading map S body coarse and coarse face shading map S f ace coarse under the target lighting environment using ray tracing. The rendered coarse shading maps are pixel-aligned with the input image.

Shading Refinement
After ray tracing under the target lighting environment, we obtained roughly correct shadows (especially for self-occlusions and hard shadows such as cast shadows) of the target human. However, the full-body 3D model reconstructed from a single image is not completely accurate and thus the ray-traced shading maps contain unnatural shadows and obvious geometry errors. To enhance the realism of the ray-traced full-body shading map, we introduce two refinement networks to compensate for shadow-rendering errors and restore a high-quality shading map. The first refinement network is designed to paint in the ray-traced full-body shading map and improve the overall quality of the shading details. The second refinement network is designed to refine facial shading details to ensure that we can fully leverage the geometry priors of human faces. Fig. 4 shows the entire shading refinement process.
Full-body Shading Refinement The full-body refinement network takes the coarse full-body shading S body coarse and the inferred ambient occlusion map AO as input and outputs the refined full-body shading residual. We choose an ambient occlusion map instead of a normal map for the following three reasons. First, the ambient occlusion map can supplement part of the self-shadows lost due to geometry prediction errors. Second, compared with inferring a normal map, inferring an ambient occlusion map is more robust under various lighting environments even with extreme lighting distributions. Third, existing 3D human reconstruction methods such as [42,24] are highly dependent on the surface normal to predict 3D geometry surface details, which means that the normal prediction errors are consistent with the geometry errors and S body coarse cannot obtain extra correct geometry information from the normal map to compensate for existing shading errors. Fig. 4 shows the refined results with the normal and ambient occlusion maps as auxiliary inputs, respectively.
The architecture of full-body refinement network is similar to MIMO-UNet [9], and we deepen the network to 4 downsampling operations. The coarse fullbody shading map S body coarse and the inferred ambient occlusion map AO are concatenated together as the original scale input and the downsampled AO is used as multiscale input. We use multiscale output for supervision. We add adversarial loss in the final highest-resolution output layer to help the network generate plausible shading effects and use PatchGAN as the discriminator. The training loss consists of content loss, fft loss and PatchGAN loss and is defined as follows: where K is the number of levels and K = 5; S body k f ine is the k th level output and S k is the k th downsampled ground truth shading map. P is the PatchGAN discriminator, and F () denotes the fast Fourier transform (FFT). λ content and λ f f t are weight factors and we empirically set λ content = 10 and λ f f t =0.001.
Face Shading Refinement Although the above-mentioned refinement network produces realistic shading with rich details, the human eye is very sensitive to the details of the face and is able to distinguish small geometry and shadow errors. Therefore, we continue to refine the face region on the basis of the refined full-body shading map S body f ine . The face refinement network takes S f ace crop cropped from S body f ine and S f ace coarse as the input and outputs the refined face shading residual. The training loss can be expressed as follows: where L is the LSGAN discriminator, λ f ace is the weight factor and is set to be λ f ace = 5. S f ace is the ground truth face shading map. We use UNet architecture as the backbone of face refinement network and details regarding the architecture are provided in the supplementary materials.

Experiments
In this section, we first compare our method with previous state-of-the-art methods quantitatively and qualitatively to show that our method performs better on de-lighting task and produces more photorealistic relit results under challenging lighting conditions. Then we evaluate the key contributions of our proposed method and prove the effectiveness of the entire framework and each module. For quantitative comparisons, we adopt the metrics MSE, PSNR and SSIM to compare the inferred albedo, relit shading and relit images with the corresponding ground truth images in our testing dataset. For qualitative comparisons, we show some intrinsic images decomposition results and relit results under target lighting conditions using both real images and synthetic images.

Comparisons with SOTA Methods
We compare our method with the state-of-the-art single-image human relighting methods RH [25], SFHR [28] and RHW [11]. All of them are based on intrinsic images decomposition and require only a single human image and target lighting for relighting as in our approach. Details regarding the specific comparison settings are provided in the supplementary materials. Comparison on Synthetic Data We first perform quantitative and qualitative comparisons on the testing dataset where we have ground truth images as a comparison. Tab. 1 shows quantitative comparison of intrinsic images decomposition performance and single-image human relighting quality. Our method outperforms the competitive methods on every metric for both tasks. To limit the comparison to decomposition and relighting quality only, all metrics are computed only on the foreground region for all methods. Qualitative comparisons for relighting are shown in Fig. 5. To improve the visual effects, we use MODNet [26] to infer alpha channel of the input image and change the background of the relit images to the corresponding part of the environment lighting map. Benefiting from the combination of classical forward rendering and deep learning, our method can produce photorealistic shadows under arbitrary lighting conditions. As a comparison, RH [25], SFHR [28] and RHW [11] can only produce low-frequency shadows. Moreover, they also fail to relight images under outdoor lighting conditions and produce overly bright or dark relit results. This is due to the fact that their models require target illuminations to have a reference radiance in the range of [0.7,0.9], which greatly restricts their performance and broad applications. As shown in Fig. 6, our method can produce plausible hard cast shadows and achieve better disentanglement between albedo and shading, which remain challenging for previous methods. Since all methods rely on albedo estimation at first for relighting, we also compare the de-lighting performance. As shown in Fig. 7, compared with RH [25], SFHR [28] and RHW [11], our method achieves better disentanglement between lighting and albedo and is able to remove large areas of shadows (even including self-shadows from concave regions on the real-world 3D shape of the human body, e.g., armpits, crotch, neck under the chin or folds on the clothing.
Comparison on Real-world Images Although our method is trained on synthetic datasets, it is generalizable to real data. The second column of Fig. 6 and Fig. 7 show shading estimation and de-lighting results on real images, respectively, and the second column of Fig. 5 shows the relit results of images photographed in the real world under arbitrary and complex illumination con-   [11] still suffer from the entanglement of lighting and albedo and struggle to produce high-frequency shadows. By comparison, our method performs better at removing detailed shadows from the original images and generating photorealistic shadows on the relit images.

Ablation study
To demonstrate the effectiveness of our delighting network, full-body refinement network and face refinement network, we conduct comprehensive ablation studies both quantitatively and qualitatively. First, we compare our delighting network with vanilla UNet, vanilla HR-Net and vanilla HRNet with skip connections. Tab. 3 shows the quantitative results of different networks and Fig. 3 presents the de-lighting results on a real image. The vanilla HRNet is HRNet-W32 [49] has two transposed convolution layers with stride 2 to ensure that the output size and input size are the same. Based on the vanilla HRNet, the vanilla HRNet with skip connections further adds skip connections between the downsampled features of the first stage and the transposed convolution features of the output. "HRNet(w extra params)" means vanilla HRNet with skip connections, extra stages, modules and blocks. "Ours(5 stages)" indicates that the architecture of network is similar to that of our de-lighting network but only contains 5 stages. The proposed network  "Ours" outperforms other networks on all metrics and achieves better disentanglement between lighting and albedo. By contrast, the vanilla HRNet fails to produce high-resolution results and the removal of partial self-shadows such as those caused by clothing folds remains difficult, even with skip connections and extra parameters. Compared with UNet, our method greatly improves PSNR (increasing by 3) and MSE (dropping to half of the UNet's). Second, we verify the effectiveness of the ambient occlusion map used by the full-body refinement network. "Refinement Net(w/o ambient)" means that the refinement network only takes coarse full-body shading as input and removes SCM and FAM modules of MIMO-UNet [9]. "Refinement Net(w normal)" means that the refinement network takes the normal map as the auxiliary input rather than the ambient occlusion map. Tab. 2 shows quantitative results and Fig. 4 shows qualitative results. Without an ambient occlusion map, the refinement work cannot fill in missing geometry details and shadows. Moreover, when the input image is under extreme lighting conditions, the inference of normal map may fail around the boundary of the shadows. By contrast, the inferred ambient occlusion map is unaffected by shadows and able to recover more geometry and occlusion details, thus restoring better shading maps.
Finally, we evaluate the face refinement network. To highlight the role of this module, we present the cropped face shading for comparison. Fig. 8 shows the qualitative results for face refinement. The face without refinement contains unnatural facial details such as a twisted nose and asymmetric eyes. By contrast, thanks to the geometry priors provided by 3DMM templates, the refined face owns clearer and more natural facial features. Compared with DSPR [61] and SMFR [22], our method can generate plausible hard cast shadows, especially around the nose and neck, whereas SMFR [22] may produces patchy shadows and DSPR [61] may produces overexposed results.

Discussion
Conclusion. We propose a geometry-aware single-image human relighting framework that leverages 3D geometry prior information to produce higher-quality relit results. Our framework contains two stages: de-lighting and relighting. For the de-lighting stage, we use a modified HRNet as the de-lighting network and achieve better disentanglement between lighting and albedo. For the relighting stage, we use ray tracing to render the shading map of the target human, and further refine it utilizing learning-based refinement networks. The extensive results demonstrate that our framework can produce photorealistic high-frequency shadows with clear boundaries under challenging lighting conditions and outperforms the existing SOTA method on both synthetic images and real images. Limitations. Due to the limitation of the dataset, we adopt the assumption of Lambertian materials for the clothed humans, which fails to produce specular reflectance in the relit results. For the same reason, our delighting network struggles to remove the highlights on the face. Moreover, inaccurate single-image geometry reconstruction may generate unnatural refined shading results. 1 Network Architecture

Geometry Module
Given an input image I, our geometry module performs image-to-image translation of the input image to its corresponding normal map N and ambient occlusion map AO using a U-Net architecture similar to pix2pixHD [52]. We empirically found that training geometry networks separately produces more accurate results than joint training. The losses for geometry estimation can be expressed as follows: where L N is the normal loss, L AO is the ambient occlusion loss and V gg is the Vgg loss. AO is the ground truth of ambient occlusion map and N is the ground truth of normal map. λ geo is the weight factor and λ geo = 5. Note that we add skip connections for the ambient occlusion map inference network to restore more details. Fig. 1 shows the architecture of the de-lighting network. We remove the the downsampling operations of the first stage in HRNet [49] directly (which also avoid skip connections) to fuse multiscale features while maintaining high-resolution representations. Our de-lighting network contains 6 stages and the number modules of last five stages are [1,2,2,2,2]. The "NUM BLOCKS" is set to 2 for the last five stages and the last layer is followed by an extra 1 × 1 convolution layer to generate output. The convolution stride of the first stage is set to 1 and the upsample mode is set to bilinear. "Ours(5 stages)" only contains 5 stages and the rest of the network is the same as "Ours". "HRNet(w extra parameters)" is adpated from HRNet-W32 [49] and owns 6 stages (the number modules of last five stages are [1,4,3,2,1]). The "NUM BLOCKS" is set to 4.  Fig. 2 shows the architecture of the full-body shading refinement network. The network is modified from MIMO-Unet [9] and is deepen to 4 downsampling operations. It takes multi-scale input images as input and outputs multi-scale refined full-body shading maps. We concatenate the ray-traced shading map S body coarse and inferred ambient occlusion map AO as the input to the highestresolution convolution layer and the downsampled ambient occlusion maps are used as the inputs to low-resolution convolution layers. The FAM, SCM and AFF modules are the same with the MIMO-UNet [9]. Fig. 3 shows the architecture of the face shading refinement network. The network takes the cropped face from the refined full-body shading map S body f ine and ray-traced face shading map S f ace coarse as input, and outputs the refined face shading map. The input is resized to 128 × 128. The network has 8 encoder-decoder layers and skip connections and each layer is run through 3 × 3 convolutions followed by LeakyReLU activations and BatchNorm (7 × 7 convolutions for the first and last layer, and the last layer is followed by an extra 3 × 3 convolution layer to generate output). The number of filters are 64,128,256,512 for the encoder,512 for the bottleneck, and 256,128,64,64 for the decoder respectively. Each convolution layer except the last layer is followed by a residual block and there are 3 residual blocks in the bottleneck.    Fig. 4, each model can produce 216 sets of images where each set consists of the rendered image, albedo map, normal map, ambient occlusion map and corresponding mask. The human models are collected with well-behaved lighting conditions and the color of the texture is treated as ground truth albedo. To obtain the ambient occlusion map, we add extra output node in Cycles engine [1] and enable "AO pass" during rendering. In total, we generate 150k sets of images for training and 20k sets of images for testing. The resolution of the rendered images is 512 × 512. All training data are augmented using random blur, noise and color enhancement. The Twindom dataset we used is restricted to make public and we will release pretrained model based on the THUman 2.0 dataset [58] for research purposes.

Training and Testing Details
We train geometry networks, de-lighting nerwork and shading refinement networks separately in PyTorch. For all models that require training, we use the Adam optimizer and set learning rate to 1e-5. The learning rate for all discriminators is set to 1e-6. The size of inputs for geometry networks, de-lighting network and full-body shading refinement network is 512 × 512. For the face refinement network, all inputs are resized to 128 × 128.
All the training processes are conducted on a 4-RTX2080Tis-Server, which consists of: 1) 3D reconstruction using PIFuHD [42], 3DMM fitting [19] and ray-tracing take 2 days in total and 2) Training of Geometry Module, Albedo Module and Shading refinement module (body/face) takes 2 days, 4 days and 2 days respectively. The testing (on a single RTX2080Ti PC) efficiency is 17s per image which includes: 120 ms for estimating albedo, 14s for 3D reconstruction, 3s for ray-tracing and 50 ms for shading refinement.
For every 3D human model, there are two sets of rendered images under different lighting conditions for every pose. When testing, we use one set of images as input and predict the relit image under the lighting condition of another set. Therefore, we can compare the difference between the inferred relit result and the ground truth. The Fig. 5 shows the checkered artifacts mentioned in Section 5.1. The resolution of the estimated 3D model by PIFuHD [42] is low, and direct ray tracing may produce grid-like artifacts. We use Laplacian smoothing to smooth the surface of the estimated full-body model and improve the quality of the ray-traced shading map. The ray-traced 3DMM face shading map is shown in Fig. 5 (g). Compared with cropped faces from the ray-traced full-body shading maps, it owns clear facial geometry details and shadows. [25,11,28] All input images are resized to 512 × 512. Since all three methods use spherical harmonic lighting representation (but with different orders), we extract 25 SH coefficients for every HDR environmental lighting map, and use the first 9 SH coefficients for testing RH [25] and RHW [11] and all 25 SH coefficients for testing SFHR [28].

Details of Comparisons with SOTA Methods
For the comparison with RH [25], we re-render our dataset using spherical harmonics lighting and pre-computed radiance transfer(PRT) to ensure that all the 3D models and lighting are the same as in our dataset. Experimentally we find that the model of RH [25] retrained on our larger dataset demonstrates superior performance, as shown in Tab.1. For a fair comparison, the qualitative results of RH [25] come from both the retrained model and the official pretrained model. We directly test the released models of RHW [11] and SFHR [28] on our testing dataset. Since these three methods require the brightness of target lighting to lie within [0.7,0.9], the relit results will fail under harsh illuminations and large areas of white pixels appear in the relit shading map. Therefore, we