We present a generalized method for reconstructing the shape of an object from measured gradient data. A certain class of optical sensors does not measure the shape of an object but rather its local slope. These sensors display several advantages, including high information efficiency, sensitivity, and robustness. For many applications, however, it is necessary to acquire the shape, which must be calculated from the slopes by numerical integration. Existing integration techniques show drawbacks that render them unusable in many cases. Our method is based on an approximation employing radial basis functions. It can be applied to irregularly sampled, noisy, and incomplete data, and it reconstructs surfaces both locally and globally with high accuracy.
In general, it is crucial to note that the reconstruction method depends on the slope-measuring sensor and the properties of the acquired data. For example, slope data acquired by Shape from Shading is rather noisy, exhibits curl, and is usually located on a full grid of millions of points. Here, a fast subspace approximation method like the one proposed by Frankot and Chellappa @cite_13 is appropriate. On the other hand, wavefront reconstruction deals with much smaller data sets, and the surface is known to be rather smooth and flat. In this case, a direct finite-difference solver can be applied @cite_11 . Deflectometric sensors deliver a third type of data: It consists of very large data sets with rather small noise and curl, but the data may not be complete, depending on the local reflectance of the measured surface. Furthermore, the measuring field may have an unknown, irregularly shaped boundary. These properties render most of the aforementioned methods unusable for deflectometric data. In the following sections, we will describe a surface reconstruction method which is especially able to deal with slope data acquired by sensors such as Phase-measuring Deflectometry.
@cite_13 @cite_11
2138127543 2003292041
An approach for enforcing integrability, a particular implementation of the approach, an example of its application to extending an existing shape-from-shading algorithm, and experimental results showing the improvement that results from enforcing integrability are presented. A possibly nonintegrable estimate of surface slopes is represented by a finite set of basis functions, and integrability is enforced by calculating the orthogonal projection onto a vector subspace spanning the set of integrable slopes. The integrability projection constraint was applied to extending an iterative shape-from-shading algorithm of M.J. Brooks and B.K.P. Horn (1985). Experimental results show that the extended algorithm converges faster and with less error than the original version. Good surface reconstructions were obtained with and without known boundary conditions and for fairly complicated surfaces. > This thesis describes a general method for automatic reconstruction of accurate, concise, piecewise smooth surfaces from unorganized 3D points. Instances of surface reconstruction arise in numerous scientific and engineering applications, including reverse-engineering--the automatic generation of CAD models from physical objects. Previous surface reconstruction methods have typically required additional knowledge, such as structure in the data, known surface genus, or orientation information. In contrast, the method outlined in this thesis requires only the 3D coordinates of the data points. From the data, the method is able to automatically infer the topological type of the surface, its geometry, and the presence and location of features such as boundaries, creases, and corners. The reconstruction method has three major phases: (1) initial surface estimation, (2) mesh optimization, and (3) piecewise smooth surface optimization. A key ingredient in phase 3, and another principal contribution of this thesis, is the introduction of a new class of piecewise smooth representations based on subdivision. The effectiveness of the three-phase reconstruction method is demonstrated on a number of examples using both simulated and real data. Phases 2 and 3 of the surface reconstruction method can also be used to approximate existing surface models. By casting surface approximation as a global optimization problem with an energy function that directly measures deviation of the approximation from the original surface, models are obtained that exhibit excellent accuracy to conciseness trade-offs. Examples of piecewise linear and piecewise smooth approximations are generated for various surfaces, including meshes, NURBS surfaces, CSG models, and implicit surfaces. We present a novel method to obtain fine-scale detail in 3D reconstructions generated with low-budget RGB-D cameras or other commodity scanning devices. As the depth data of these sensors is noisy, truncated signed distance fields are typically used to regularize out the noise, which unfortunately leads to over-smoothed results. In our approach, we leverage RGB data to refine these reconstructions through shading cues, as color input is typically of much higher resolution than the depth data. As a result, we obtain reconstructions with high geometric detail, far beyond the depth resolution of the camera itself. Our core contribution is shading-based refinement directly on the implicit surface representation, which is generated from globally-aligned RGB-D images. We formulate the inverse shading problem on the volumetric distance field, and present a novel objective function which jointly optimizes for fine-scale surface geometry and spatially-varying surface reflectance. In order to enable the efficient reconstruction of sub-millimeter detail, we store and process our surface using a sparse voxel hashing scheme which we augment by introducing a grid hierarchy. A tailored GPU-based Gauss-Newton solver enables us to refine large shape models to previously unseen resolution within only a few seconds. We present a Bayesian technique for the reconstruction and subsequent decimation of 3D surface models from noisy sensor data. The method uses oriented probabilistic models of the measurement noise and combines them with feature-enhancing prior probabilities over 3D surfaces. When applied to surface reconstruction, the method simultaneously smooths noisy regions while enhancing features such as corners. When applied to surface decimation, it finds models that closely approximate the original mesh when rendered. The method is applied in the context of computer animation where it finds decimations that minimize the visual error even under nonrigid deformations. This paper presents a novel method for 3D surface reconstruction that uses polarization and shading information from two views. The method relies on the polarization data acquired using a standard digital camera and a linear polarizer. Fresnel theory is used to process the raw images and to obtain initial estimates of surface normals, assuming that the reflection type is diffuse. Based on this idea, the paper presents two novel contributions to the problem of surface reconstruction. The first is a technique to enhance the surface normal estimates by incorporating shading information into the method. This is done using robust statistics to estimate how the measured pixel brightnesses depend on the surface orientation. This gives an estimate of the object material reflectance function, which is used to refine the estimates of the surface normals. The second contribution is to use the refined estimates to establish correspondence between two views of an object. To do this, surface patches are extracted from each view, which are then aligned by minimising an energy functional based on the surface normal estimates and local topographic properties. The optimum alignment parameters for different patch pairs are then used to establish stereo correspondence. This process results in an unambiguous field of surface normals, which can be integrated to recover the surface depth. Our technique is most suited to smooth nonmet allic surfaces. It complements existing stereo algorithms since it does not require salient surface features to obtain correspondences. An extensive set of experiments, yielding reconstructed objects and reflectance functions, are presented and compared to ground truth. We consider the case in which a robot has to navigate in an unknown environment but does not have enough on-board power or payload to carry a traditional depth sensor (e.g., a 3D lidar) and thus can only acquire a few (point-wise) depth measurements. We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements? Reconstruction from incomplete data is not possible in general, but when the robot operates in man-made environments, the depth exhibits some regularity (e.g., many planar surfaces with only a few edges); we leverage this regularity to infer depth from a small number of measurements. Our first contribution is a formulation of the depth reconstruction problem that bridges robot perception with the compressive sensing literature in signal processing. The second contribution includes a set of formal results that ascertain the exactness and stability of the depth reconstruction in 2D and 3D problems, and completely characterize the geometry of the profiles that we can reconstruct. Our third contribution is a set of practical algorithms for depth reconstruction: our formulation directly translates into algorithms for depth estimation based on convex programming. In real-world problems, these convex programs are very large and general-purpose solvers are relatively slow. For this reason, we discuss ad-hoc solvers that enable fast depth reconstruction in real problems. The last contribution is an extensive experimental evaluation in 2D and 3D problems, including Monte Carlo runs on simulated instances and testing on multiple real datasets. Empirical results confirm that the proposed approach ensures accurate depth reconstruction, outperforms interpolation-based strategies, and performs well even when the assumption of structured environment is violated. Shape from shading is a classical inverse problem in computer vision. This shape reconstruction problem is inherently ill-dened; it depends on the assumed light source direction. We introduce a novel mathematical formulation for calculating local surface shape based on covariant derivatives of the shading ow eld, rather than the customary integral minimization or P.D.E approaches. On smooth surfaces, we show second derivatives of brightness are independent of the light sources and can be directly related to surface properties. We use these measurements to dene the matching local family of surfaces that can result from any given shading patch, changing the emphasis to characterizing ambiguity in the problem. We give an example of how these local surface ambiguities collapse along certain image contours and how this can be used for the reconstruction problem. The estimation of surface integrals on the boundary of an unknown body is a challenge for nonparametric methods in statistics, with powerful applications to physics and image analysis, among other fields. Provided that one can determine whether random shots hit the body, [Ann. Statist. 35 (2007) 1031―1051] estimate the boundary measure (the boundary length for planar sets and the surface area for 3-dimensional objects) via the consideration of shots at a box containing the body. The statistics considered by these authors, as well as those in subsequent papers, are based on the estimation of Minkowski content and depend on a smoothing parameter which must be carefully chosen. For the same sampling scheme, we introduce a new approach which bypasses this issue, providing strongly consistent estimators of both the boundary measure and the surface integrals of scalar functions, provided one can collect the function values at the sample points. Examples arise in experiments in which the density of the body can be measured by physical properties of the impacts, or in situations where such quantities as temperature and humidity are observed by randomly distributed sensors. Our method is based on random Delaunay triangulations and involves a simple procedure for surface reconstruction from a dense cloud of points inside and outside the body. We obtain basic asymptotics of the estimator, perform simulations and discuss, via Google Earth's data, an application to the image analysis of the Aral Sea coast and its cliffs. We present a benchmark for the evaluation and comparison of algorithms which reconstruct a surface from point cloud data. Although a substantial amount of effort has been dedicated to the problem of surface reconstruction, a comprehensive means of evaluating this class of algorithms is noticeably absent. We propose a simple pipeline for measuring surface reconstruction algorithms, consisting of three main phases: surface modeling, sampling, and evaluation. We use implicit surfaces for modeling shapes which are capable of representing details of varying size and sharp features. From these implicit surfaces, we produce point clouds by synthetically generating range scans which resemble realistic scan data produced by an optical triangulation scanner. We validate our synthetic sampling scheme by comparing against scan data produced by a commercial optical laser scanner, where we scan a 3D-printed version of the original surface. Last, we perform evaluation by comparing the output reconstructed surface to a dense uniformly distributed sampling of the implicit surface. We decompose our benchmark into two distinct sets of experiments. The first set of experiments measures reconstruction against point clouds of complex shapes sampled under a wide variety of conditions. Although these experiments are quite useful for comparison, they lack a fine-grain analysis. To complement this, the second set of experiments measures specific properties of surface reconstruction, in terms of sampling characteristics and surface features. Together, these experiments depict a detailed examination of the state of surface reconstruction algorithms. This paper provides an introduction to photometric methods for image-based 3D shape reconstruction and a survey of photometric stereo techniques. We begin with taxonomy of active and passive shape acquisition techniques. Then we describe the methodical background of photometric 3D reconstruction, define the canonical setting of photometric stereo (Lambertian surface reflectance, parallel incident light, known illumination direction, known surface albedo, absence of cast shadows), discuss the 3D reconstruction of surfaces from local gradients, summarize the concept of the bidirectional reflectance distribution function (BRDF), and outline several important empirically and physically motivated reflectance models. We provide a detailed treatment of several generalizations of the canonical setting of photometric stereo, namely non-distant light sources, unknown illumination directions, and, in some detail, non-Lambertian surface reflectance functions. An important special case is purely specular reflections, where an extended light source allows capturing a surface that consists of perfectly specular surface patches. Linear combinations of purely Lambertian and purely specular reflectance components are favorably used for reconstructing smooth surfaces and also human skin. Nonuniform surface reflectance properties are estimated based on a simultaneous 3D reconstruction and determination of the locally variable parameters of the reflectance function based on a multitude of images. Assuming faceted surfaces, the effective resolution of the 3D reconstruction result can be increased to some extent beyond that of the underlying images. Other approaches separate specular and diffuse reflectance components based on polarization data or color information. The specular reflections can be used additionally to estimate the direction from which the surface is illuminated. Finally, we describe methods to combine photometric 3D reconstruction techniques with active and passive triangulation-based approaches. Non-line-of-sight (NLOS) imaging utilizes the full 5D light transient measurements to reconstruct scenes beyond the cameras field of view. Mathematically, this requires solving an elliptical tomography problem that unmixes the shape and albedo from spatially-multiplexed measurements of the NLOS scene. In this paper, we propose a new approach for NLOS imaging by studying the properties of first-returning photons from three-bounce light paths. We show that the times of flight of first-returning photons are dependent only on the geometry of the NLOS scene and each observation is almost always generated from a single NLOS scene point. Exploiting these properties, we derive a space carving algorithm for NLOS scenes. In addition, by assuming local planarity, we derive an algorithm to localize NLOS scene points in 3D and estimate their surface normals. Our methods do not require either the full transient measurements or solving the hard elliptical tomography problem. We demonstrate the effectiveness of our methods through simulations as well as real data captured from a SPAD sensor. We describe a novel device that can be used as a 2.5D “scanner” for acquiring surface texture and shape. The device consists of a slab of clear elastomer covered with a reflective skin. When an object presses on the skin, the skin distorts to take on the shape of the object's surface. When viewed from behind (through the elastomer slab), the skin appears as a relief replica of the surface. A camera records an image of this relief, using illumination from red, green, and blue light sources at three different positions. A photometric stereo algorithm that is tailored to the device is then used to reconstruct the surface. There is no problem dealing with transparent or specular materials because the skin supplies its own BRDF. Complete information is recorded in a single frame; therefore we can record video of the changing deformation of the skin, and then generate an animation of the changing surface. Our sensor has no moving parts (other than the elastomer slab), uses inexpensive materials, and can be made into a portable device that can be used “in the field” to record surface shape and texture. In this paper we present a method for fast surface reconstruction from large noisy datasets. Given an unorganized 3D point cloud, our algorithm recreates the underlying surface's geometrical properties using data resampling and a robust triangulation algorithm in near realtime. For resulting smooth surfaces, the data is resampled with variable densities according to previously estimated surface curvatures. Incremental scans are easily incorporated into an existing surface mesh, by determining the respective overlapping area and reconstructing only the updated part of the surface mesh. The proposed framework is flexible enough to be integrated with additional point label information, where groups of points sharing the same label are clustered together and can be reconstructed separately, thus allowing fast updates via triangular mesh decoupling. To validate our approach, we present results obtained from laser scans acquired in both indoor and outdoor environments. This paper presents a method for acquiring dense nonrigid shape and deformation from a single monocular depth sensor. We focus on modeling the human hand, and assume that a single rough template model is available. We combine and extend existing work on model-based tracking, subdivision surface fitting, and mesh deformation to acquire detailed hand models from as few as 15 frames of depth data. We propose an objective that measures the error of fit between each sampled data point and a continuous model surface defined by a rigged control mesh, and uses as-rigid-as-possible (ARAP) regularizers to cleanly separate the model and template geometries. A key contribution is our use of a smooth model based on subdivision surfaces that allows simultaneous optimization over both correspondences and model parameters. This avoids the use of iterated closest point (ICP) algorithms which often lead to slow convergence. Automatic initialization is obtained using a regression forest trained to infer approximate correspondences. Experiments show that the resulting meshes model the user's hand shape more accurately than just adapting the shape parameters of the skeleton, and that the retargeted skeleton accurately models the user's articulations. We investigate the effect of various modeling choices, and show the benefits of using subdivision surfaces and ARAP regularization. We consider sensor networks that measure spatio-temporal correlated processes. An important task in such settings is the reconstruction at a certain node, called the sink, of the data at all points of the field. We consider scenarios where data is time critical, so delay results in distortion, or suboptimal estimation and control. For the reconstruction, the only data available to the sink are the values measured at the nodes of the sensor network, and knowledge of the correlation structure: this results in spatial distortion of reconstruction. Also, for the sake of power efficiency, sensor nodes need to transmit their data by relaying through the other network nodes: this results in delay, and thus temporal distortion of reconstruction if time critical data is concerned. We study data gathering for the case of Gaussian processes in one- and two-dimensional grid scenarios, where we are able to write explicit expressions for the spatial and time distortion, and combine them into a single total distortion measure. We prove that, for various standard correlation structures, there is an optimal finite density of the sensor network for which the total distortion is minimized. Thus, when power efficiency and delay are both considered in data gathering, it is useless from the point of view of accuracy of the reconstruction to increase the number of sensors above a certain threshold that depends on the correlation structure characteristics. This paper presents a technique for acquiring the shape of real-world objects with complex isotropic and anisotropic reflectance. Our method estimates the local normal and tangent vectors at each pixel in a reference view from a sequence of images taken under varying point lighting. We show that for many real-world materials and a restricted set of light positions, the 2D slice of the BRDF obtained by fixing the local view direction is symmetric under reflections of the halfway vector across the normal-tangent and normal-binormal planes. Based on this analysis, we develop an optimization that estimates the local surface frame by identifying these planes of symmetry in the measured BRDF. As with other photometric methods, a key benefit of our approach is that the input is easy to acquire and is less sensitive to calibration errors than stereo or multi-view techniques. Unlike prior work, our approach allows estimating the surface tangent in the case of anisotropic reflectance. We confirm the accuracy and reliability of our approach with analytic and measured data, present several normal and tangent fields acquired with our technique, and demonstrate applications to appearance editing. We present a technique for estimating intrinsic images from image+depth video, such as that acquired from a Kinect camera. Intrinsic image decomposition in this context has importance in applications like object modeling, in which surface colors need to be recovered without illumination effects. The proposed method is based on two new types of decomposition constraints derived from the multiple viewpoints and reconstructed 3D scene geometry of the video data. The first type provides shading constraints that enforce relationships among the shading components of different surface points according to their similarity in surface orientation. The second type imposes temporal constraints that favor consistency in the intrinsic color of a surface point seen in different video frames, which improves decomposition in cases of view-dependent non-Lambertian reflections. Local and non-local variants of the two constraints are employed in a manner complementary to local and non-local reflectance constraints used in previous works. Together they are formulated within a linear system that allows for efficient optimization. Experimental results demonstrate that each of the new constraints appreciably elevates the quality of intrinsic image estimation, and that they jointly yield decompositions that compare favorably to current techniques. This paper describes a depth image enhancement method for consumer RGB-D cameras. Most existing methods use the pixel-coordinates of the aligned color image. Because the image plane generally has no relationship to the measured surfaces, the global coordinate system is not suitable to handle their local geometries. To improve enhancement accuracy, we use local tangent planes as local coordinates for the measured surfaces. Our method is composed of two steps, a calculation of the local tangents and surface reconstruction. To accurately estimate the local tangents, we propose a color heuristic calculation and an orientation correction using their positional relationships. Additionally, we propose a surface reconstruction method by ray-tracing to local tangents. In our method, accurate depth image enhancement is achieved by using the local geometries approximated by the local tangents. We demonstrate the effectiveness of our method using synthetic and real sensor data. Our method has a high completion rate and achieves the lowest errors in noisy cases when compared with existing techniques. We are developing a total-internal-reflection-based tactile sensor in which the shape is reconstructed using an optical reflection. This sensor consists of silicone rubber, an image pattern, and a camera. It reconstructs the shape of the sensor surface from an image of a pattern reflected at the inner sensor surface by total internal reflection. In this study, we propose precise real-time reconstruction by employing an optimization method. Furthermore, we propose to use active patterns. Deformation of the reflection image causes reconstruction errors. By controlling the image pattern, the sensor reconstructs the surface deformation more precisely. We implement the proposed optimization and active-pattern-based reconstruction methods in a reflection-based tactile sensor, and perform reconstruction experiments using the system. A precise deformation experiment confirms the linearity and precision of the reconstruction. One. Introduction: On the Absence of Geometry from Morphometrics.- First Part. The Measurement of Biological Shape.- Two. Shapes and Measures of Shape.- A. Properties of the Euclidean Plane and Euclidean Space.- B. Outlines and Homologous Landmarks.- C. Definitions of Shape, Shape Change, Shape Measurement.- D. Shapes as Data.- Three. Critique of an Applied Field: Conventional Cephalometrics.- A. Landmarks, Curvature, and Growth.- B. Registration.- Four. New Statistical Methods for Shape.- A. Analysis of the Tangent Angle Function.- 1. History.- 2. Sampling from the tangent angle function.- 3. Conic replacement curves and their estimation Fit of a conic to a point - Geometric interpretation - Estimator for a circle - Estimation for the general conic - Computation of the extremum -Linear constraints.- 4. Conic splining Joint conic fitting under constraint -An example - Application - Analysis of parameters.- B. Extension to Three Dimensions: A Sketch.- C. Skeletons Definition - A multivariate statistical method - Bibliographic note.- Second Part. The Measurement of Shape Change Using Biorthogonal Grids.- Five. The Study of Shape Transformation after D'Arcy Thompson.- A. The Original Method.- 1. Thompson's own work.- 2. Later examples.- 3. Difficulties.- B. Analysis of Growth Gradients.- C. Simulations.- D. Other Morphometric Schemes Vector displacements - Multivariate morphometrics.- Six. The Method of Biorthogonal Grids.- A. Representation of Affine Transformations.- B. General Lines of Growth and Biorthogonal Grids.- C. Summarizing the Grids.- Technical Note 1. Existence and Form of Biorthogonal Grids.- Technical Note 2. Interpolation from Landmark Locations and Arcs The measure of roughness - The vector space and its associated functions - Interpolation from boundary values - Interpolation from boundary values and interior points - Note on computation.- Technical Note 3. Construction of Integral Curves.- Technical Note 4. On Homologous Points.- Seven. Examples of Biorthogonal Analysis.- A. Comparison of Square and Biorthogonal Grids: Thompson's Diodon Figure.- B. Phylogeny and Ontogeny of Primate Crania.- 1. Functional craniology and craniometrics.- 2. Data for this exercise.- 3. Two types of transformations.- 4. Comparative ontogeny of the apes and man.- Eight. Future Directions for Transformation Analysis.- A. Statistical Methods The symmetric tensor field - Concordance -Linear methods.- B. Computation Other kinds of information about homology -Three dimensions.- C. Likely Applications Computed tomography - Orthodontics - Developmental biology.- Nine. Envoi.- Literature Cited.