We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch. This produces a mid-level scene descriptor, comprised of local shape distributions that are inferred separately at every image patch across multiple scales. The framework is based on a quadratic representation of local shape that, in the absence of noise, has guarantees on recovering accurate local shape and lighting. And when noise is present, the inferred local shape distributions provide useful shape information without over-committing to any particular image explanation. These local shape distributions naturally encode the fact that some smooth diffuse regions are more informative than others, and they enable efficient and robust reconstruction of object-scale shape. Experimental results show that this approach to surface reconstruction compares well against the state-of-art on both synthetic images and captured photographs.
The shading cue is supposed to be a major factor in monocular stereopsis. However, the hypothesis is hardly corroborated by available data. For instance, the conventional stimulus used in perception research, which involves a circular disk with monotonic luminance gradient on a uniform surround, is theoretically ‘explained’ by any quadric surface, including spherical caps or cups (the conventional response categories), cylindrical ruts or ridges, and saddle surfaces. Whereas cylindrical ruts or ridges are reported when the outline is changed from circular to square, saddle surfaces are never reported. We introduce a method that allows us to differentiate between such possible responses. We report observations on a number of variations of the conventional stimulus, including variations of shape and quality of the boundary, and contexts that allow the observer to infer illumination direction. We find strong and expected influences of outline shape, but, perhaps surprisingly, we fail to find any influence of context, and only partial influence of outline quality. Moreover, we report appreciable differences within the generic population. We trace some of the idiosyncrasies (as compared to shape from shading algorithms) of the human observer to generic properties of the environment, in particular the fact that many objects are limited in size and elliptically convex over most of their boundaries.  For general images of smooth objects wholly contained in the field of view, and for illumination symmetric around the viewing direction, it is proven that shape is uniquely determined by shading. Thus, shape from shading is a well-posed problem under these illumination conditions; and regularization is unnecessary for surface reconstruction and should be avoided. Generic properties of surfaces and images are established. Questions of existence are also discussed. Under the conditions above, it is argued that most images are effectively impossible, with no corresponding physically reasonable surface, and that any image can be rendered effectively impossible by a small perturbation of its intensities. This is explicitly illustrated for a synthetic image. The proofs are based on ideas of dynamical systems theory and global analysis. Since the first shape-from-shading (SFS) technique was developed by Horn in the early 1970s, many different approaches have emerged. In this paper, six well-known SFS algorithms are implemented and compared. The performance of the algorithms was analyzed on synthetic images using mean and standard deviation of depth (Z) error, mean of surface gradient (p, q) error, and CPU timing. Each algorithm works well for certain images, but performs poorly for others. In general, minimization approaches are more robust, while the other approaches are faster. Local analysis of image shading, in the absence of prior knowledge about the viewed scene, may be used to provide information about the scene. The following has been proved. Every image point has the same image intensity and first and second derivatives as the image of some point on a Lambertian surface with principal curvatures of equal magnitude. Further, if the principal curvatures are assumed to be equal there is a unique combination of image formation parameters (up to a mirror reversal) that will produce a particular set of image intensity and first and second derivatives. A solution for the unique combination of surface orientation, etc., is presented. This solution has been extended to natural imagery by using general position and regional constraints to obtain estimates of the following: ? surface orientation at each image point; ? the qualitative type of the surface, i.e., whether the surface is planar, cylindrical, convex, concave, or saddle; ? the illuminant direction within a region. Algorithms to recover illuminant direction and estimate surface orientation have been evaluated on both natural and synthesized images, and have been found to produce useful information about the scene. Many algorithms have been suggested for the shape-from-shading problem, and some years have passed since the publication of the survey paper by [R. Zhang, P.-S. Tsai, J.E. Cryer, M. Shah, Shape from shading: a survey, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (8) (1999) 690-706]. In this new survey paper, we try to update their presentation including some recent methods which seem to be particularly representative of three classes of methods: methods based on partial differential equations, methods using optimization and methods approximating the image irradiance equation. One of the goals of this paper is to set the comparison of these methods on a firm basis. To this end, we provide a brief description of each method, highlighting its basic assumptions and mathematical properties. Moreover, we propose some numerical benchmarks in order to compare the methods in terms of their efficiency and accuracy in the reconstruction of surfaces corresponding to synthetic, as well as to real images. This article proposes a solution of the Lambertian Shape From Shading (SFS) problem by designing a new mathematical framework based on the notion of viscosity solutions. The power of our approach is twofolds: 1) it defines a notion of weak solutions (in the viscosity sense) which does not necessarily require boundary data. Note that, in the previous SFS work of [23,15], [8], [22,20], the characterization of a viscosity solution and its computation require the knowledge of its values on the boundary of the image. This was quite unrealistic because in practice such values are not known. 2) it unifies the work of [23,15], [8], [22,20], based on the notion of viscosity solutions and the work of Dupuis and Oliensis [6] dealing with classical (C 1) solutions. Also, we generalize their work to the “perspective SFS” problem recently introduced by Prados and Faugeras 20. We describe a mathematical and algorithmic study of the Lambertian "Shape-From-Shading" problem for orthographic and pinhole cameras. Our approach is based upon the notion of viscosity solutions of Hamilton-Jacobi equations. This approach provides a mathematical framework in which we can show that the problem is well-posed (we prove the existence of a solution and we characterize all the solutions). Our contribution is threefold. First, we model the camera both as orthographic and as perspective (pinhole), whereas most authors assume an orthographic projection (see Horn and Brooks (1989) for a survey of the SFS problem up to 1989 and (1999), Kozera (1998), (2004) for more recent ones); thus we extend the applicability of shape from shading methods to more realistic acquisition models. In particular it extends the work of (2002a) and Rouy and Tourin (1992). We provide some novel mathematical formulations of this problem yielding new partial differential equations. Results about the existence and uniqueness of their solutions are also obtained. Second, by introducing a "generic" Hamiltonian, we define a general framework allowing to deal with both models (orthographic and perspective), thereby simplifying the formalization of the problem. Thanks to this unification, each algorithm we propose can compute numerical solutions corresponding to all the modeling. Third, our work allows us to come up with two new generic algorithms for computing numerical approximations of the "continuous solution of the "Shape-From-Shading" problem as well as a proof of their convergence toward that solution. Moreover, our two generic algorithms are able to deal with discontinuous images as well as images containing black shadows. Understanding how the shape of a three dimensional object may be recovered from shading in a two-dimensional image of the object is one of the most important - and still unresolved - problems in machine vision. Although this important subfield is now in its second decade, this book is the first to provide a comprehensive review of shape from shading. It brings together all of the seminal papers on the subject, shows how recent work relates to more traditional approaches, and provides a comprehensive annotated bibliography.The book's 17 chapters cover: Surface Descriptions from Stereo and Shading. Shape and Source from Shading. The Eikonal Equation: some Results Applicable to Computer Vision. A Method for Enforcing Integrability in Shape from Shading Algorithms. Obtaining Shape from Shading Information. The Variational Approach to Shape from Shading. Calculating the Reflectance Map. Numerical Shape from Shading and Occluding Boundaries. Photometric Invariants Related to Solid Shape. Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System. A Provably Convergent Algorithm for Shape from Shading. Recovering Three Dimensional Shape from a Single Image of Curved Objects. Perception of Solid Shape from Shading. Local Shading Analysis Pentland. Radarclinometry for the Venus Radar Mapper. Photometric Method for Determining Surface Orientation from Multiple Images.Berthold K. P. Horn is Professor of Electrical Engineering and Computer Science at MIT. He has presided over the field of machine vision for more than a decade and is the author of "Robot Vision. "Michael Brooks is Reader in Computer Science at The Flinders University of South Australia. "Shape from Shading" is included in the Artificial Intelligence series, edited by Michael Brady, Daniel Bobrow, and Randall Davis. For general objects, and for illumination from a general direction, the constraints on shape imposed by shading are studied. It is argued that, for a typical image, shading determines shape essentially up to a finite ambiguity. Thus regularization is often unnecessary and should be avoided. For some images, shape from shading is a partially well-constrained problem: the surface is uniquely determined over most of the image, but infinitely ambiguous in small regions bordering the image boundary, even though the image contains singular points. The main result is that, contrary to previous belief, the image of the occluding boundary does not strongly constrain the surface solution. It is shown that characteristic strips are curves of steepest ascent on the imaged surface. A theorem characterizing the properties of generic images is presented. >