We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch. This produces a mid-level scene descriptor, comprised of local shape distributions that are inferred separately at every image patch across multiple scales. The framework is based on a quadratic representation of local shape that, in the absence of noise, has guarantees on recovering accurate local shape and lighting. And when noise is present, the inferred local shape distributions provide useful shape information without over-committing to any particular image explanation. These local shape distributions naturally encode the fact that some smooth diffuse regions are more informative than others, and they enable efficient and robust reconstruction of object-scale shape. Experimental results show that this approach to surface reconstruction compares well against the state-of-art on both synthetic images and captured photographs.
In independent work, Kunsberg and Zucker @cite_13 have recently derived local uniqueness results that are related to, and consistent with, our results in Sec. . Their elegant analysis, which uses differential geometry and applies to continuous images, is complimentary to the discrete and algebraic approach employed in this paper. Kunsberg and Zucker also observe that the analysis of shading in patches instead of at isolated points is consistent with early processing in the visual cortex, and they discuss the possibility of local shading distributions being computed there. Indeed, the notion of such distributions is compatible with evidence that humans perceive shape in some diffuse regions more accurately than others @cite_14 .
@cite_14 @cite_13
2008840450 1830639927
The shading cue is supposed to be a major factor in monocular stereopsis. However, the hypothesis is hardly corroborated by available data. For instance, the conventional stimulus used in perception research, which involves a circular disk with monotonic luminance gradient on a uniform surround, is theoretically ‘explained’ by any quadric surface, including spherical caps or cups (the conventional response categories), cylindrical ruts or ridges, and saddle surfaces. Whereas cylindrical ruts or ridges are reported when the outline is changed from circular to square, saddle surfaces are never reported. We introduce a method that allows us to differentiate between such possible responses. We report observations on a number of variations of the conventional stimulus, including variations of shape and quality of the boundary, and contexts that allow the observer to infer illumination direction. We find strong and expected influences of outline shape, but, perhaps surprisingly, we fail to find any influence of context, and only partial influence of outline quality. Moreover, we report appreciable differences within the generic population. We trace some of the idiosyncrasies (as compared to shape from shading algorithms) of the human observer to generic properties of the environment, in particular the fact that many objects are limited in size and elliptically convex over most of their boundaries. Shape from shading is a classical inverse problem in computer vision. This shape reconstruction problem is inherently ill-dened; it depends on the assumed light source direction. We introduce a novel mathematical formulation for calculating local surface shape based on covariant derivatives of the shading ow eld, rather than the customary integral minimization or P.D.E approaches. On smooth surfaces, we show second derivatives of brightness are independent of the light sources and can be directly related to surface properties. We use these measurements to dene the matching local family of surfaces that can result from any given shading patch, changing the emphasis to characterizing ambiguity in the problem. We give an example of how these local surface ambiguities collapse along certain image contours and how this can be used for the reconstruction problem.