Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called observation map that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.
Diverse appearances of real world objects can be encoded by a BRDF @math , which relates the observed intensity @math to the associated surface normal @math , the @math -th incoming lighting direction @math , its intensity @math , and the outgoing viewing direction @math via where @math accounts for attached shadows and @math is an additive error to the model. eq:img_form1 is generally called image formation model . Most photometric stereo algorithms assumed the specific shape of @math and recovered the surface normals of a scene by inversely solving eq:img_form1 from a collection of observations under @math different lighting conditions @math . All the effects that are not represented by a BRDF (image noises, cast shadows, inter-reflections and so on) are typically put together in @math . Note that when the BRDF is Lambertian and the additive error is removed, it is simplified to the traditional Lambertian image formation model @cite_2 .
@cite_2
2889383273
Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called observation map that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.