9201508
Structure from Motion Causally Integrated Over Time
We describe an algorithm for reconstructing three-dimensional structure and motion causally, in real time from monocular sequences of images. We prove that the algorithm is minimal and stable, in the sense that the estimation error remains bounded with probability one throughout a sequence of arbitrary length. We discuss a scheme for handling occlusions (point features appearing and disappearing) and drift in the scale factor. These issues are crucial for the algorithm to operate in real time on real scenes. We describe in detail the implementation of the algorithm, which runs on a personal computer and has been made available to the community. We report the performance of our implementation on a few representative long sequences of real and synthetic images. The algorithm, which has been tested extensively over the course of the past few years, exhibits honest performance when the scene contains at least 20-40 points with high contrast, when the relative motion is "slow" compared to the sampling frequency of the frame grabber (30 Hz), and the lens aperture is "large enough" (typically more than 30/spl deg/ of visual field).
15692724
Region-Based Segmentation on Evolving Surfaces with Application to 3D Reconstruction of Shape and Piecewise Constant Radiance
We consider the problem of estimating the shape and radiance of a scene from a calibrated set of images under the assumption that the scene is Lambertian and its radiance is piecewise constant. We model the radiance segmentation explicitly using smooth curves on the surface that bound regions of constant radiance. We pose the scene reconstruction problem in a variational framework, where the unknowns are the surface, the radiance values and the segmenting curves. We propose an iterative procedure to minimize a global cost functional that combines geometric priors on both the surface and the curves with a data fitness score. We carry out the numerical implementation in the level set framework.
2282922
Estimation of 3D Surface Shape and Smooth Radiance from 2D Images: A Level Set Approach
We cast the problem of shape reconstruction of a scene as the global region segmentation of a collection of calibrated images. We assume that the scene is composed of a number of smooth surfaces and a background, both of which support smooth Lambertian radiance functions. We formulate the problem in a variational framework, where the solution (both the shape and radiance of the scene) is a minimizer of a global cost functional which combines a geometric prior on shape, a smoothness prior on radiance and a data fitness score. We estimate the shape and radiance via an alternating minimization: The radiance is computed as the solutions of partial differential equations defined on the surface and the background. The shape is estimated using a gradient descent flow, which is implemented using the level set method. Our algorithm works for scenes with smooth radiances as well as fine homogeneous textures, which are known challenges to traditional stereo algorithms based on local correspondence.
7863218
A semi-direct approach to structure from motion
The problem of structure from motion is often decomposed into two steps: feature correspondence and three-dimensional reconstruction. This separation often causes gross errors when establishing correspondence fails. Therefore, we advocate the necessity to integrate visual information not only in time (i.e. across different views), but also in space, by matching regions – rather than points – using explicit photometric deformation models. We present an algorithm that integrates image-feature tracking and three-dimensional motion estimation into a closed loop, while detecting and rejecting outlier regions that do not fit the model. Due to occlusions and the causal nature of our algorithm, a drift in the estimates accumulates over time. We describe a method to perform global registration of local estimates of motion and structure by matching the appearance of feature regions stored over long time periods. We use image intensities to construct a score function that takes into account changes in brightness and contrast. Our algorithm is recursive and suitable for real-time implementation.
38046402
A semi-direct approach to structure from motion
Reconstructing three-dimensional structure and motion is often decomposed into two steps: point feature correspondence and three-dimensional reconstruction. This separation often causes gross errors since correspondence relies on the brightness constancy constraint that is local in space and time. Therefore, we advocate the necessity to integrate visual information not only in time (i.e. across different views), but also in space, by matching regions-rather than points-using explicit photometric deformation models. We present an algorithm that integrates 2D region tracking and 3D motion estimation into a closed loop based on an explicit geometric and photometric model, while detecting and rejecting outlier regions that do not fit the model. Our algorithm is recursive and suitable for real-time implementation. Our experiments show that it far exceeds the accuracy and robustness of point feature-based SFM algorithms.
8032236
First story detection in TDT is hard
✁✄✂✆☎✞✝✠✟☛✡✌☞✞✟☛✟✎✍✑✏✓✒✄✔✕✒ ✗✖✘✝✠✡✚✙✛✂✜✍ ✢✂✜✡✜✍✢✝✠✒✤✣✦✥✗✣✧☎★✔✪✩✫✥✬✡✮✭✯✝✠✣✘✰★✱✲✔ ✳✙✴✔✶✵ ✂✌✷✸✂✜✣✯✍☛✹✻✺✘✥✗✟☛✂ ✼☎✽✝✾✣❀✿❁✒✬✩✢❂❃✥✬✍✢✝✠✒✤✣❄✒✗✩✢✰✤✥✗✣✞✝✾❅✌✥✬✍✢✝✠✒✤✣ ❆✍✫✥✗✟☛✭✯✟✌❇❈✍☛✩✫✥✗✡✫✭❀✝✠✣✞✰ ✥✗✣✘☎ ❊❉✞✩✢✟ ❋✍❃✟❋✍✢✒✗✩☛●★☎❀✂✜✍✢✂✌✡✜✍✢✝✠✒✤✣■❍❏✁ ❑✂✆✟☛▲✘✒✼✏ ▼✍✢▲✧✥ ◆✍ ❖✏P▲✘✂✜✣◗✥❘❉✞✩✢✟❋✍ ✟❋✍✢✒✗✩☛●❙☎✞✂✜✍✢✂✌✡✮✍✢✝✾✒✗✣ ❚✟❋●❀✟❋✍✢✂✌❂❯✝✠✟❱✺✘✥✗✟☛✂✼☎❲☞✘✖❳✒✗✣✚✍☛✩✫✥✗✡✫✭❀✝✠✣✞✰❨✍✢✂✌✡✫▲✘✣✞✒✤❩❬✹ ✒✤✰✬●❪❭✤✏✓✂✶✡✌✥✗✣❖✂✜❫❀✖❳✂✌✡✮✍ ❴✍✢▲✘✥✬✍❴✖❳✂✜✩☛✿❵✒✗✩✢❂❃✥✗✣✞✡✌✂✳✏P✝✠❩✠❩✧✺❳✂❛✖❳✒❪✒✗✩✼❍❴✔✳▲✘✥✬✍ ✖✞✩✢✂✌☎✞✝✠✡✜✍✢✝✠✒✤✣❜✝✠✟✕✡✌✒✗✣✘✟☛✝✠✟❋✍✢✂✌✣❪✍ ❝✏P✝❬✍✢▲✎✥✬✡✜✍✢☞✧✥✬❩❪✖❳✂✜✩☛✿❁✒✬✩✢❂❃✥✗✣✞✡✌✂✓✝✠✣❜✔✳✙✴✔ ✂✌✷✗✥✗❩✠☞✧✥◆✍✢✝✠✒✤✣✘✟✌❍❞✁❑✂
✍✢▲✘✂✜✣❢✟☛▲✞✒◆✏ ❣✍✢▲✘✥✬✍❛✍✢✒✎✥✗✡✫▲✘✝✠✂✌✷✸✂
▲✘✝✠✰✤▲❀✹✑❤❪☞✘✥✗❩✠✝❬✍✑● ❉✘✩✢✟❋✍✪✟❋✍✢✒✗✩☛●✴☎✞✂✜✍✢✂✌✡✮✍✢✝✾✒✗✣✪❭◆✍☛✩✫✥✗✡✫✭✯✝✠✣✘✰❛✂✜✐❥✂✌✡✮✍✢✝✾✷✤✂✌✣✘✂✌✟☛✟✪❂✎☞✞✟❋✍✕✝✠❂❦✖✞✩✢✒◆✷✸✂ ✍✢✒✛✥
☎✞✂✌✰✬✩✢✂✌✂✓✍✢▲✘✥✬✍❧✂✜❫❀✖❳✂✜✩✢✝✠✂✌✣✞✡✌✂✳✟☛☞✞✰✤✰✤✂✜✟❋✍✢✟❴✝✠✟❧☞✞✣✘❩✠✝✠✭✸✂✌❩❬●❪❍✪✁❑✂✳✡✌✒✗✣✞✹ ✡✌❩✠☞✧☎❀✂✶✍✢▲✘✥✬✍✓✂✜✐❥✂✌✡✜✍✢✝✠✷✸✂✶❉✘✩✢✟❋✍✳✟❋✍✢✒✬✩☛●❃☎✞✂✮✍✢✂✌✡✜✍✢✝✠✒✤✣❨✝✠✟✓✂✌✝❬✍✢▲✘✂✮✩❛✝✠❂❦✖❳✒✤✟❋✹ ✟☛✝✠✺✘❩✠✂✴✒✗✩P✩✢✂✼❤❪☞✞✝✠✩✢✂✜✟✶✟☛☞✞✺✘✟❋✍✫✥✬✣✯✍✢✝✾✥✬❩✾❩❬●✆☎✞✝❬✐❥✂✜✩✢✂✌✣❪✍✶✥✬✖✘✖✞✩✢✒✤✥✗✡✫▲✘✂✌✟✌❍