We present a new color photometric stereo (CPS) method that can recover high quality, detailed 3D face geometry in a single shot. Our system uses three uncalibrated near point lights of different colors and a single camera. We first utilize 3D morphable model (3DMM) and semantic segmentation of facial parts to achieve robust self-calibration of light sources. We then address the spectral ambiguity problem by incorporating albedo consensus, albedo similarity, and proxy prior into a unified framework. We avoid the need for spatial constancy of albedo and use a new measure for albedo similarity that is based on the albedo norm profile. Experiments show that our new approach produces state-of-the-art results in single image with high-fidelity geometry that includes details such as wrinkles.
Fyffe @cite_63 extended the usual three color channels to six by using two RGB cameras and a pair of Dolby dichroic filters. An extension of their work @cite_14 employed polarized color gradient illumination but requires an ultra complex setup with 2040 LEDs light sources. Chakrabarti and Sunkavalli @cite_4 observe that the reflectance and normal within a uniform color region can be uniquely recovered from spectrally demultiplexed image by assuming piecewise constant albedo. Ozawa @cite_0 densely discretize albedo chromaticity and utilize consensus on albedo norms to reconstruct objects with spatially varying albedo. Most of these approaches assume directional lighting and require pre-calibrating them. It is possible to use near light sources @cite_48 but still require pre-calibration. In contrast, our technique assumes unknown light positions and spatially varying albedo. The former enables more feasible capture and the latter fulfils the physical property of real faces.
@cite_14 @cite_4 @cite_48 @cite_0 @cite_63
1890563702 2016462422 2520788639 2403721818
We present a method for acquiring the per-pixel diffuse albedo, specular albedo, and surface normal maps of a subject at a single instant in time. The method is single-shot, requiring no optical flow, and per-pixel, making no assumptions regarding albedo statistics or surface connectivity. We photograph the subject inside a spherical illumination device emitting a static lighting pattern of vertically polarized RGB color gradients aligned with the XYZ axes, and horizontally polarized RGB color gradients inversely aligned with the XYZ axes. We capture simultaneous photographs using one of two possible setups: a single-view setup using a coaxially aligned camera pair with a polarizing beam splitter, and a multi-view stereo setup with different orientations of linear polarizing filters placed on the cameras, enabling high- quality geometry reconstruction. From this lighting we derive full-color diffuse albedo, single-channel specular albedo suitable for dielectric materials, and polarization-preserving surface normals which are free of corruption from subsurface scattering. We provide simple formulae to estimate the diffuse albedo, specular albedo, and surface normal maps in the single-view and multi-view cases and show error bounds which are small for many common subjects including faces. We present a novel process for acquiring detailed facial geometry with high resolution diffuse and specular photometric information from multiple viewpoints using polarized spherical gradient illumination. Key to our method is a new pair of linearly polarized lighting patterns which enables multiview diffuse-specular separation under a given spherical illumination condition from just two photographs. The patterns -- one following lines of latitude and one following lines of longitude -- allow the use of fixed linear polarizers in front of the cameras, enabling more efficient acquisition of diffuse and specular albedo and normal maps from multiple viewpoints. In a second step, we employ these albedo and normal maps as input to a novel multi-resolution adaptive domain message passing stereo reconstruction algorithm to create high resolution facial geometry. To do this, we formulate the stereo reconstruction from multiple cameras in a commonly parameterized domain for multiview reconstruction. We show competitive results consisting of high-resolution facial geometry with relightable reflectance maps using five DSLR cameras. Our technique scales well for multiview acquisition without requiring specialized camera systems for sensing multiple polarization states. We present a single-shot system to recover surface geometry of objects with spatially-varying albedos, from images captured under a calibrated RGB photometric stereo setup—with three light directions multiplexed across different color channels in the observed RGB image. Since the problem is ill-posed point-wise, we assume that the albedo map can be modeled as piece-wise constant with a restricted number of distinct albedo values. We show that under ideal conditions, the shape of a non-degenerate local constant albedo surface patch can theoretically be recovered exactly. Moreover, we present a practical and efficient algorithm that uses this model to robustly recover shape from real images. Our method first reasons about shape locally in a dense set of patches in the observed image, producing shape distributions for every patch. These local distributions are then combined to produce a single consistent surface normal map. We demonstrate the efficacy of the approach through experiments on both synthetic renderings as well as real captured images. This paper presents a novel approach for recovering the shape of non-Lambertian, multicolored objects using two input images. We show that a ring light source with complementary colored lights has the potential to be effectively utilized for this purpose. Under this lighting, the brightness of an object surface varies with respect to different reflections. Therefore, analyzing how brightness is modulated by illumination color gives us distinct cues to recover shape. Moreover, the use of complementary colored illumination enables the color photometric stereo to be applicable to multicolored surfaces. Here, we propose a color correction method based on the addition principle of complementary colors to remove the effect of illumination from the observed color. This allows the inclusion of surfaces with any number of chromaticities. Therefore, our method offers significant advantages over previous methods, which often assume constant object albedo and Lambertian reflectance. To the best of our knowledge, this is the first attempt to employ complementary colors on a ring light source to compute shape while considering both non-Lambertian reflection and spatially varying albedo. To show the efficacy of our method, we present results on synthetic and real world images and compare against photometric stereo methods elsewhere in the literature.