Real-Time 3D Shape of Micro-Details

Motivated by the growing demand for interactive environments, we propose an accurate real-time 3D shape reconstruction technique. To provide a reliable 3D reconstruction which is still a challenging task when dealing with real-world applications, we integrate several components including (i) Photometric Stereo (PS), (ii) perspective Cook-Torrance reflectance model that enables PS to deal with a broad range of possible real-world object reflections, (iii) realistic lightening situation, (iv) a Recurrent Optimization Network (RON) and finally (v) heuristic Dijkstra Gaussian Mean Curvature (DGMC) initialization approach. We demonstrate the potential benefits of our hybrid model by providing 3D shape with highly-detailed information from micro-prints for the first time. All real-world images are taken by a mobile phone camera under a simple setup as a consumer-level equipment. In addition, complementary synthetic experiments confirm the beneficial properties of our novel method and its superiority over the state-of-the-art approaches.


Introduction
3D reconstruction can be considered as one of the major areas for computer vision which provides not only 3D interpretation, but also the surface feature extraction. Photometric Stereo (PS) is a potential technique for this purpose. In PS, several input images taken from a fixed view point under different illumination directions are used for 3D reconstruction. The pioneers of the problem are Woodham [66] and Horn et al. [21]. As it is shown by Woodham [67], the orientation of a Lambertian surface can be uniquely determined when at least three input images are given. During last decades, PS has attracted wide attention in different practical domains such as industrial quality control [11,74,42,25], face recognition [72,75,14,73], Figure 1. A successful 3D reconstruction from very fine details of micro-prints on a 50 euro banknote shown in gray color rectangular parts of (A) and (B). As can be seen, these details are even invisible to human eyes, while our real-time 3D reconstruction from these regions is able to reveal hidden information and features in high amount of details. This information recovery can be used in various applied areas such as detecting security items on financial documents for the fraud detection and also the quality control of any industrial productions that include delicate details such as printed circuits. medical science [44,63], texture classification [51,57,34] and estimation of weather conditions [55]. Another benefit of PS is to generate invariant features utilized in different kinds of recognition tasks [61,58]. However, most of the later works on PS kept two basic building blocks of Woodham's approach, namely Lambertian surface reflection and orthographic camera projection. For this reason, Lambert's law [29] is commonly used in classic computer vision models, which implies that the surface reflects light equally in all directions and ignores the specular component. However, it may easily fails for surfaces whose reflection is non-Lambertian. Corresponding effects (e.g. specularities as occurring on shiny objects) are quite readily observable in real-world images and surfaces with not pure isotropic luminance. So, two major light reflection components in-cluding specular reflection as the high frequency variations and diffuse reflection as the low frequency component should be combined to provide a successful reflectance model [47,40,65]. In order to circumvent difficulties of specularities, commercial instruments for 3D reconstruction use sophisticated techniques such as white light interferometry or scanning focal microscopy, while these laboratory-based devices tend to be large, slow, or at the expense of $100,000 or more. In many works, non-Lambertian effects are considered as outliers. In this context, Miyazaki et al. [38] suggested a median filtering to evade the influence of specular reflection and a preprocessing technique is presented by Yang et al. [70] to remove specularities. A maximum-likelihood estimation [64] and an expectation maximization [69] are other approaches based on assuming specularities as the outliers. While Mukaigawa et al. [39] suggested a random sample consensus based approach in which only diffuse reflection is employed as the underlying model, Tang et al. [60] proposed a coupled Markov random field treating specularities as noise. An approach relying on a maximum feasible subsystem method is introduced by Yu et al. [72], where input images satisfying a Lambertian constraint are selected for reconstruction. In addition, a convex optimization technique is developed by Wu et al. [68] in which specularities are considered as deviations from the Lambertian assumption in the objective function. Zickler et al. [23] provides an analyze to a subspace which is free from highlights for image transformation. Finally, Ikehata et al. [23] suggested a regression procedure that deals with the specular component as a sparse error of an underlying purely diffuse reflectance equation. Mallick et al. [32] considered a transformation of the RGB color space to avoid specularity using the light source color which is not always available. An inpainting technique for the highlights correction is presented by Tan et al. [49] using the color of illuminant and the diffuse component. In [59] and [54] the pixel chromaticity direct analysis and specular-free image are used respectively to specularity correction in which the diffuse component is adjusted to a criterion. This criterion is an indicative of smooth transition of color between diffuse and specular regions and evaluated using a nonlinear shifting of specular pixels intensity and chromaticity. Johnson et al. [24] presented an elastomer using a sensor skin constructed with a metal-flake pigment which is attached to the surface to change its BRDF. However, in all these works, the shape information that is evidently contained in non-Lambertian effects (e.g. highlights) is discarded. The orthographic projection as the second mentioned ingredient is realistic when objects are far away from the camera, but not if they are close since then perspective effects grow to be important. In addition, when a surface is projected orthographically, it can be reconstructed only up to a generalized bas-relief trans-formation. However, the perspective projection as proved in [43], can remove the bas-relief ambiguity and provide the uniqueness of the reconstructed surface. There is only a small number of previous papers dealing with perspective models in PS. To our knowledge, the first of such models has been developed by Lee and Kuo [30]. They approximated the depth function as a linear combination of individual element basis functions. Other perspective methods are applied by Galo and Tozzi [13], Tankus and Kiryati [62] and Mecca et al. [37], while the Lambertian reflectance is applied in all these researches. Concerning more general non Lambertian reflectance models based on perspective PS, Khanian et al. proposed the application of perspective Blinn-Phong model [5,46] for the signal reconstruction and corresponding lightening conditions in [26] and for the 3D reconstruction with different perspective techniques in [27]. An empirical approach is also considered in [36,35]. However, for the derivation of the final applied model, purely diffuse and purely specular reflections should be decoupled in [36,35] . Moreover, the proposed approach in [36] relies on the Fast Marching method [52] where the unknown depth value of the center point should be provided in advance which makes difficulty for real-world applications. Some state-of-the-art in PS methods in which the information of depth should be provided in advance, can be pointed as [7,50] which require the depth to be known on the boundary, and [35] that needs the mean value of depth. In contrast, we do not need any information of depth to be given in advance in our approach as also proved in Figure  1.

Our contributions
In this paper, we provide the following innovations: 1. We proposed an accurate hybrid real-time 3D reconstruction approach which merges the benefits of perspective projection and non-Lambertian reflectance model motivated by Cook-Torrance reflectance [8] to handle complex reflecting characteristics. To the best of our knowledge, this is the first work for the real-time PS which also regarded the application of perspective Cook-Torrance model.

Introducing a Recurrent Optimization Network (RON)
which provides a robust PS framework to obtain a highlydetailed 3D shape from even micro-print texts where their visibility is hard for the human visual system as can be seen in Figure 1. In this Figure, we could reconstruct 3D shape of two regions of A and B with high details, while these regions are very tiny microtexts. The information hidden in these areas became completely clear by our approach. As a result, our technique can be used in different applied areas. 3. Since the performance of many optimization techniques is highly-dependent on the initialization process, we provided a Dijkstra Gaussian Mean Curvature (DGMC) technique to find anchor points which can be applied as a Figure 2. A complex real scene including several objects with different sizes captured by a mobile phone with a simple setup which is used for our 3D shape reconstructions. key point to offer more effective optimization approach. 4. Furthermore, we extended our proposed approach to deal with realistic lightening situation using spatially dependent lighting. 5. We show that our approach is able to provide the realtime 3D shape of objects as captured by an ordinary mobile phone camera without employing controlled laboratory conditions (cf. Figure 2). So, our PS process is conducted by the consumer-level equipment where specialized equipment and professional laboratory settings are not required. Our instruments are easily available for general consumers as should be expected for an ideal solution. Consequently, our method can be readily employed in real-world situations. We demonstrate that this paper provides the basis for a potentially useful 3D reconstruction technique. This improvement breaks inhibitor limitations of a controlled setup and the necessity of working with specific scenes (e.g. Lambertian materials) and turns mobile phone into powerful 3D shape reconstruction tools. 6. Finally, our approach also provides non-uniform colorful albedo from images with diverse color intensities as shown in Figure 12. All these attempts greatly advance the applicability of photometric stereo to various applications, especially for reconstructing 3D surfaces from very tiny structures like microprints.

Cook-Torrance reflectance model
The Cook-Torrance reflectance as a physically based model designed to account for the spectral composition of specularity and cope with the problem of modeling reflections of realistic materials [8]. This physically based model agrees well with actual measured BRDFs [28]. Like recent popular theoretical models, this reflectance model considers this assumption that the large scale BRDF is a collection of microfacets as shiny V-grooves with random sizes and orientations [28]. This hypothesis leads to the microfacet theory for non-perfectly-smooth surfaces. These surfaces are essentially height fields, where the distribution of facet is described statistically. So, the main component of these models is an expression for the distribution of microfacet normals. This property enables them to effectively model many real-world surfaces [45] in which, mirror-like facets are modeled by the Cook-Torrance reflectance to retrieve the local shape from specularity. The Cook-Torrance image irradiance equation is defined as follows: and In this notation, I(x, y) and N (x, y) are the intensity and surface normal at pixel (x, y), respectively. The parameter k d is the diffuse material parameter (albedo) and m is the root mean square slope of the surface microfacets (i.e. the surface roughness).
It is worth to mention that in expression (8), H is used instead of N as suggested in [18]. Let us emphasize that as opposed to an orthogonal viewer direction (0, 0, 1) which is commonly used, we model here a pinhole camera. Therefore, the viewing vector V depends explicitly on (x, y) and also on the depth z. This model is powerful enough to capture the geometrical properties of the scene as recorded by the camera. Moreover, its images resemble what would be seen by human visual system. Hence, pinhole camera is also referred to as perspective camera model in graphics and vision literatures [71].
The vector H is defined to be the halfway vector between the viewing vector V and the light vector L, as the direction to which the microfacet normals need to be oriented to reflect L into V . The light source intensity is denoted by l d . F is the Fresnel term including f λ which is the reflection coefficient (the reflectance of the surface at normal incidence). Here, the distribution of facets is described by D. It should be noticed that in PS, it is necessary to apply a set of non-coplanar light vectors L i (corresponding to the input images I i ) as also investigated in [67,20,19,9]. It is worthwhile noting that in all our experiments (including synthetic and real-world scenes and even for specular images), we will still follow Woodham [66] and use only three input images I i , i = 1, 2, 3. We consider it an indication of the robustness and potential practical usefulness of our method that it works effectively with such a low number of input images. Considering (1) for multiple images leads to a highly non-linear system of equations for the unknown surface normal vectors that should be solved.

Perspective projection and intrinsic parameters of pinhole camera model
Under the perspective projection model, the surface parametrization is performed as follows: The normal vector as the cross product of the partial derivatives of the surface is computed as: where f is the focal length and ∇z(x, y) = (z x , z y ) is the gradient field. On the other hand, to implement the perspective projection of the pinhole camera model, the intrinsic camera parameters should be involved which leads to the CCD camera [17]. To this aim, we apply the following transformation to convert the homogeneous representations pixel coordinate X = (x, y, 1) to image coordinate χ = (µ, ν, f ): where Γ is the matrix of intrinsic parameters of the camera expressed as: Here, ψ x and ψ y are the focal length in x and y directions. ξ describes the skew effect and (δ x , δ y ) denotes the principal point or focal point. We neglect the skew parameter (ξ = 0) as it is zero for most of normal cameras [17].

Lightening sensitivity analysis
Different types of light sources can be defined based on the energy emission from a single point in space with some angularly varying distribution of outgoing lights. At first, we considered the distant light, also known as the directional light or sparse light. This source of energy is an emitter which deposits illumination from the same direction at every point in space. It can be considered also as a light at infinity (e.g. sun). Furthermore, in this kind of light sources, energy strength does not depend on the distance between the light source and the object [45]. In addition, we also equipped our perspective Cook-Torrance model with spatially varying lightening in which the point-wise light source implies how the light varies from one point to another. This assumption facilitates the independent control of intensity. To this aim, a three dimensional vector field of the light L should be imposed instead of a directional light vector. In this case, the rate at which the light falls off w.r.t. the distance should be induced to make the model more consistent with the actual physical aspects. Integrating all the mentioned points, the new spatially dependent light vector L is defined as: with the light attenuation Π: where Υ is the light source position. Since we developed the perspective projection, the surface parametrization as in (9) is imposed in S(x, y) for spatially point light source. As a result, the light direction depends not only on the point (x,y), but also on the depth of the surface itself. So, in this case, an initial guess for the depth of the surface is required.

Optimization process
Using equation (1) for three input images (as the minimum number of input images in PS), we constitute a system of equation , h = 1, 2, 3 and formed by equations given by (1) corresponding to each input image I h (x, y), h = 1, 2, 3 as: To estimate the normal field, we should solve the optimization problem as follows: In optimization process, we update the solution using an iterative procedure: Here, d k is the search direction in iteration k. The main difference between various optimization approaches is caused by the definition of d k .

Quasi-Newton with BFGS updating
In this section, we investigate various procedures that we applied for the nonlinear system of equations described by F (x) = 0. Among many iterative methods, Newton's method is one of the popular approaches. But this method has some drawbacks which can be removed by its modified versions. Here, we used the updated formula known as Quasi-Newton with BFGS suggested independently by [6,12,15,53], which as explained in [48], has much better performance than other modified versions of Newton's method. In this approach, the Quasi-Newton condition (secant condition) is used to find the search direction d k : where B and J(X) are the Hessian matrix approximation and the Jacobian matrix of F (X), respectively. The following expression is known as Quasi-Newton condition or the BFGS formula for updating B: with

Levenberg-Marquardt
Another strategy that we considered is the Levenberg-Marquardt method [31,33] with the search direction determined as: where λ k > 0 and the (J(X k )J(X k ) T +λ k I) −1 is positive definite.

Powell's Dog Leg
In this technique, a strategy is proposed to choose optimization steps based on a parameter called radius of trust region ∆. Three search directions are applied as follows: where Here, we have also used the strategy applied in [41] to choose β and ∆. The general process of mentioned algorithms is illustrated in the last section.

Dijkstra Gaussian Mean Curvature (DGMC) technique
As a heuristic search to find a proper initial value for the normal field, we have devised a technique that consists of two phases. At the first stage, we employ the Dijkstra algorithm [10] to find the nearest point in the outer highlight boundary S to each specular pixel p . This point is denoted as q.
In the second step, we compute the Gaussian Curvature (GC) and the Mean Curvature (MC) properties of p and q, separately: where K 1 is the smallest and K 2 is the biggest eigenvalues of the Hessian matrix of the pixel local neighboring area.
Having those parameter for both points allows us to define their local geometric similarity conditions as follows: In case that q does not meet similarity conditions, the next nearest member of S to p will be nominated.

Recurrent Optimization Network (RON)
We will introduce an intermittent optimization strategy which allows updating not only the surface normal but also the albedo and roughness parameter. At first, we consider a constant albedo and also a constant roughness so that we can obtain the surface normal. After obtaining the normal field, we update the albedo values. Furthermore, the roughness parameter is updated as well. To this aim, we use the following formulation proposed in [16] to update the roughness parameter m, where we already obtained the depth map z by integrating the normal field.
Here,z(x, y) is the mean surface depth, and n is the number of pixels. The whole algorithm of mentioned RON procedure is illustrated in Algorithm 1. Thanks to this concise optimization network with backtracing steps and DGMC technique which are integrated into the photometric stereo algorithm, our method enables PS to work in diverse applications by removing restrictions such as diffuse objects, orthographic projection, laboratory set up and professional equipment.

Integration
As the integrator of the normal field, the energy minimization of the following functional should be considered: which leads to the Euiler-Lagrange equation ∆z = ∇.N . In order to solve the mentioned minimization problem, we applied Generalized Minimal RESidual (GMRES) with the initial solution of Simchony [56].

Experiments
In the case of synthetic experiments, to perform a quantitative evaluation, we will make use of Mean Angular Error of Normal vectors (MAEN) in degrees and also Mean Squared Error (MSE) of depth.

Tests of accuracy on synthetic data
In the following, we discuss our results obtained for the synthetic test images as displayed in Figures 6, 7 and 8. All images are rendered using Blender software. The 3D model of Oldman and Woman images are obtained from [1] and [2] respectively. 3D models of the other synthetic images are publicly available at [3]. All images except Coping (256 × 200) are in the size of 256 × 256. All implementations are performed on an Intel Core i7 processor with 8 GB of RAM.

Comparison with other BRDFs:
In Figure 3 documenting the first set of experiments, we represent the results of comparison between the recent approach suggested by Khanian et al. [27] and also the results of Lambertian reflectance model (as the most common model applied in PS). The model presented in [27] is based on the Blinn-Phong reflectance. We extended our experiments to a wide range of the specular material parameter k s = 1 − k d to evaluate how the varying k s influence the error of 3D reconstruction. It can be noticed that the output of our method achieves higher rate of accuracy over [27] and Lambertian model as k s increases. Whereas, there are some fluctuations when k d k s or k d k s . As illustrated, error rates of presented models are almost similar for low values of k s . However, as k s increases, the superiority of our model over Lambertian and [27] is significant. This observation agrees well with our motivation for focusing on the high frequency variations and model these components explicitly. One particular interpretation for the sharp ascending trend of the error rate for Coping input image (in the case of higher values of k s parameter) is the presence of the flat background region which leads to high amount of shiny highlighted areas in input images. As for the difference between [27] and Lambertian model, we find that the Lambertian still performs a little better on very low values of k s ; however, when k s is larger than 0.3, [27] improves accuracy much more than Lambertian model. As indicated, the MSE comparing our 3D reconstruction and ground truth data, demonstrates that our method is capable of producing accurate 3D reconstructions for the indicated images with specularity. The qualitative evaluations can be also seen as recovered surfaces shown in Figure 6. Our faithful 3D reconstructions with a high amount of details even for tiny surfaces containing low-depth details (e.g. Coping) represent the proficiency of our approach.

Optimization procedures:
As the next experiment shown in Figure 5, we compared various optimization procedures (applied in first and second steps of our RON technique) for several surfaces with diverse k s . It turns out that Dog Leg method achieves higher accuracy in all cases. Although, BFGS and Levenberg approaches follow the same trend for lower values of k s (Levenberg method can slightly outperform BFGS technique for low values of k s ) BFGS performs better as k s increases.

Lightening analysis:
To investigate the 3D reconstruction sensitivity w.r.t. the lightening model, we applied different PS frameworks.  Quantitative and qualitative results of this investigation are shown in Table 3 and Figure 7, respectively. To this end, we evaluated our approach regarding different available lightening conditions. In the first experiment, we consider the distant light source leads to perform directional-light-based PS referred as DLPS. The output images of this simulation resemble the real-world images that are illuminated by sun. The second evaluation is conducted to simulate spatially dependent light resulting in point-wise-light-based PS denoted as PLPS. We observed that the higher accuracy is achieved by applying the point-wise light source and adjusting lightings based on spatially dependent light as presented in (13) and (14). Figure 7 reveals that DLPS has problem to recover some areas since capturing images by directional-light leads to more shadows as shown in Figure  7. On the other hand, using the point-wise light source in the scene and formulating PLPS can provide the higher amount of accuracy (cf.   Table 1.
As it can be seen in the graph, the proposed method can be also favored due to its higher accuracy w.r.t both obtained normal vectors and depth map. To illustrate, our technique can provide accuracy improvement in normal field extraction over [22]

Comparison with several PS methods:
As another evaluation shown in Table 1, we compared our method from different aspects on the set of images with the state-of-the-art approaches: constrained bivariate regression [22], differential ratios [36], dimensionality reduc-tion based on PCA [4] and also least squares-regression [67]. For a fair comparison, normal map are obtained by the finite-difference of depth map for the methods suggested in [22,36] (these schemes provide depth, but not normal vectors) and the depth map is obtained by our integrator (section 5.6.) for [4,67] (these techniques obtain normal vectors, but not depth map). All methods show worse performances than our approach. One reason is that highfrequency variations are mis-classified and leads to the unpredictable errors. As can be seen in Table 1, the proposed strategy dramatically reduces error in computing normal    Table 3. MSE of our 3D shape reconstruction with perspective Cook-Torrance reflectance model in the presence of specularities using different lightening models.
in Table 1 is shown in Figure 4. For example, we could improve the accuracy in extracted normals by 93.15% over [36] and 90.43 % over [67] and provide higher accuracy in depth reconstruction by 73.29% over [36] and 70.46 % over [67]. Furthermore, the first row in Table 1 demonstrates the efficiency of our proposed method. Our method is the first one with significant high speed providing all information of normal vectors, colorful albedo, roughness value and depth map. Our speed improvement over mentioned techniques is represented in Table 2. The running time of [36,67] are comparable, yet their results appear both quantitatively and qualitatively different. The qualitative comparison of our approach with the mentioned methods is shown in Figure  8. The performance ranking of methods is consistent with qualitative results. Our results demonstrate the ability of the proposed scheme to provide a complete reconstruction without any deviation at specularities which outperforms other comparing approaches.

Test of applicability on real-world images
Finally, we evaluated the capability of our approach on a set of real-world images shown in Figures 9,10,11,12 as a test of the practical applicability of our method. All the images used in these experiments are captured by the camera of a Samsung Galaxy S5 smartphone and using a flash. For image acquisition the photographed figurine is standing on a table covered by a black tablecloth as shown in Figure 2. The light directions are measured by considering the camera as the origin of the coordinate system. The recovered 3D shape for the first real-world image is presented in Figure 9. In this case, we created a complex scene composed of numerous small statues with different sizes as a challenging set. However, we could reconstruct the surface with correct geometry where relevant details and main features are recovered faithfully. Let us stress that our realworld images are taken without any laboratory equipment or controlled setups. Nevertheless, the reconstruction result is in our opinion very reasonable. This test demonstrates the robustness of our method and its potential for real-world applications even using consumer-level equipment. Moreover, again for this real-world scene, we applied different PS-based lightnings models for a same light source of a flash. Improved quality of the 3D reconstruction with spatially dependent lights (PLPS) shown in Figure 9 (in terms of the height field of depth map and details) can be an advantage offered by this model. More real-world experiments are presented in Figures 10 and 11. As can be seen, our approach succeeds in producing accurate 3D reconstruction results even for high frequency details of tiny structure of Berlin souvenir statues. The colors of these images are extracted in Figure 12 where the diversity of colors is recovered as well in our non-uniform colorful albedo reconstruction. Let us also point out that our results show the capability of reliable 3D reconstruction for specular materials by using a minimum number of input images (for all synthetic and real-world experiments, we used three input images). Thus we document here that our method works under reasonable practical conditions with inexpensive instruments so that it can be used for many potential applications.

Conclusion
We presented a real-time robust PS which is also benefiting from perspective Cook-Torrance reflectance model to explicitly handle specularities and remove the limitations of working with diffuse materials or orthographic projection. A RON optimization technique based on DGMC approach is introduced to obtain accurate information of normal vectors, depth, roughness and colorful albedo. Our proposed RON and DGMC techniques can be applied in any highly non-linear optimization framework as a key point for providing multi-variable solutions and proper initialization. These innovations provide important steps towards a reliable PS. Furthermore, we equipped our model with the spatially dependent lightening offering more reasonable reconstructions. We have demonstrated the applicability of our method by applying minimum number of input images without any laboratory conditions or facilities. Furthermore, we stretch our approach to its full potential by extracting 3D reconstruction of micro-prints. Our method has the potential to be useful as the basis of future developments.
Coping Horseman Soldier Fighter Figure 6. Left: input images including specularity. Right: our 3D reconstruction results using perspective Cook-Torrance reflectance model. These results illustrate the capability of the proposed method for providing faithful reconstructions with high frequency details even for the fine details of a tiny surface (e.g. first input image).

Carpenter
Oldman Woman Figure 7. First row: input specular images with directional light. Second row: our 3D output using DLPS. Third row: input images with pointwise light. Last row: our 3D output with PLPS. Images produced by directional light are imposed by more specularity and shadows in comparison with images with pointwise light. These shadow parts lead to losing some reconstructed area. Figure 8. Comparison of several PS methods. From left: 1) input image (Militaryman) including high specularities, and the 3D results of 2) our reconstruction applying perspective Cook-Torrance reflectance model without any deviation or artifacts, 3) reconstruction of [67] that could not handle specularity areas, 4) reconstruction of [36] which shows deviations specially around the highly specular areas 5) [4] shows problem in producing complete 3D reconstruction in presence of spcularity and 5) [22] which still includes distortions. Figure 9. From left: 1) input real-world image including several tiny statues captured without laboratory setting cf. Figure 2, 2) our 3D reconstruction with Cook-Torrance reflectance model, 3) our 3D reconstruction using DLPS for the face of the smallest statue in the scene and 4) our 3D reconstruction using PLPS for the same statue. Figure 10. More real-world experiments with specularity, first row) input images including Einstein statue, two Berlin souvenir statues with comprehensive structure and a metallic coin, second row) our 3D reconstruction using DLPS and last row) our 3D reconstruction using PLPS. High quality details and structures recovered by the proposed approach confirm offered advantages and agrees well with them. Figure 11. Some qualitative evaluations for the proposed method: from left) close-up views of our reconstruction results corresponding to the red, blue, green and magenta rectangular areas in Coping as synthetic input and Einstein, Coin and first Berlin statue as the real scenes. It can be seen that our method achieves desirable reconstruction quality even for very fine details. Figure 12. Reconstructed albedo of two color input images shown in Figure 10. As can be seen, our RON approach can produce nonuniform albedo from images with a variety of colors. Since the albedo is not same across the entire real-world surfaces, our variant albedo extraction is more adjusted for real-world situations. It should be noticed that these statues are very tiny surfaces.

Algorithm 2: Quasi-Newton with BFGS updating
Primary input : X 0 that should be computed using our proposed "DGMC" technique explained in section 5.4. of the paper Secondary inputs: ϑ = 0, k = 0 output : Normal vectors for photometric stereo while J(X k ) > ε, and k < k max and ϑ < ϑ max do Compute d k := −(B(X k ) −1 )J(X k ) [X k+1 , dϑ] ← line search (X k , d k ) as applied in reference [33] of the paper Where is the computer accuracy and ε = 1e -12 is considered.