7863218
A semi-direct approach to structure from motion
The problem of structure from motion is often decomposed into two steps: feature correspondence and three-dimensional reconstruction. This separation often causes gross errors when establishing correspondence fails. Therefore, we advocate the necessity to integrate visual information not only in time (i.e. across different views), but also in space, by matching regions – rather than points – using explicit photometric deformation models. We present an algorithm that integrates image-feature tracking and three-dimensional motion estimation into a closed loop, while detecting and rejecting outlier regions that do not fit the model. Due to occlusions and the causal nature of our algorithm, a drift in the estimates accumulates over time. We describe a method to perform global registration of local estimates of motion and structure by matching the appearance of feature regions stored over long time periods. We use image intensities to construct a score function that takes into account changes in brightness and contrast. Our algorithm is recursive and suitable for real-time implementation.
15692724
Region-Based Segmentation on Evolving Surfaces with Application to 3D Reconstruction of Shape and Piecewise Constant Radiance
We consider the problem of estimating the shape and radiance of a scene from a calibrated set of images under the assumption that the scene is Lambertian and its radiance is piecewise constant. We model the radiance segmentation explicitly using smooth curves on the surface that bound regions of constant radiance. We pose the scene reconstruction problem in a variational framework, where the unknowns are the surface, the radiance values and the segmenting curves. We propose an iterative procedure to minimize a global cost functional that combines geometric priors on both the surface and the curves with a data fitness score. We carry out the numerical implementation in the level set framework.
6346856
Real-Time Feature Tracking and Outlier Rejection with Changes in Illumination
We develop an efficient algorithm to track point features supported by image patches undergoing aflne deformations and changes in illumination. The algorithm is based on a combined model of geonietty and photometry that is used to track features as well as to detect outliers in a hypothesis testing framework. The algorithm runs in real time on a personal computer, and is available to the public.
2282922
Estimation of 3D Surface Shape and Smooth Radiance from 2D Images: A Level Set Approach
We cast the problem of shape reconstruction of a scene as the global region segmentation of a collection of calibrated images. We assume that the scene is composed of a number of smooth surfaces and a background, both of which support smooth Lambertian radiance functions. We formulate the problem in a variational framework, where the solution (both the shape and radiance of the scene) is a minimizer of a global cost functional which combines a geometric prior on shape, a smoothness prior on radiance and a data fitness score. We estimate the shape and radiance via an alternating minimization: The radiance is computed as the solutions of partial differential equations defined on the surface and the background. The shape is estimated using a gradient descent flow, which is implemented using the level set method. Our algorithm works for scenes with smooth radiances as well as fine homogeneous textures, which are known challenges to traditional stereo algorithms based on local correspondence.
2927787
Real-time virtual object insertion
We present a system to insert virtual objects into real image sequences in real time. The system consists of offthe- shelf hardware (a camera connected to a Pentium PC) and software to (a) automatically select and track region features despite changes in illumination, (b) estimate threedimensional position and orientation of surface patches relative to an inertial reference frame despite individual pointfeatures appearing and disappearing, (c) insert a texturemapped virtual object into the scene so as to make it appear to be part of the scene and moving with it. This is all done in real time. The multi-thread C++ code, which is readily interfaced with a frame grabber as well as Matlab for development, will be made available to the public at the demonstration.
14241332
Visual tracking in the presence of motion blur
We consider the problem of visual tracking of regions of interest in a sequence of motion blurred images. Traditional methods couple tracking with deblurring in order to correctly account for the effects of motion blur. Such coupling is usually appropriate, but computationally wasteful when visual tracking is the lone objective. Instead of deblurring images, we propose to match regions by blurring them. The matching score for two image regions is governed by a cost function that only involves the region deformation parameters and two motion blur vectors. We present an efficient algorithm to minimize the proposed cost function and demonstrate it on sequences of real blurred images.
8032236
First story detection in TDT is hard
✁✄✂✆☎✞✝✠✟☛✡✌☞✞✟☛✟✎✍✑✏✓✒✄✔✕✒ ✗✖✘✝✠✡✚✙✛✂✜✍ ✢✂✜✡✜✍✢✝✠✒✤✣✦✥✗✣✧☎★✔✪✩✫✥✬✡✮✭✯✝✠✣✘✰★✱✲✔ ✳✙✴✔✶✵ ✂✌✷✸✂✜✣✯✍☛✹✻✺✘✥✗✟☛✂ ✼☎✽✝✾✣❀✿❁✒✬✩✢❂❃✥✬✍✢✝✠✒✤✣❄✒✗✩✢✰✤✥✗✣✞✝✾❅✌✥✬✍✢✝✠✒✤✣ ❆✍✫✥✗✟☛✭✯✟✌❇❈✍☛✩✫✥✗✡✫✭❀✝✠✣✞✰ ✥✗✣✘☎ ❊❉✞✩✢✟ ❋✍❃✟❋✍✢✒✗✩☛●★☎❀✂✜✍✢✂✌✡✜✍✢✝✠✒✤✣■❍❏✁ ❑✂✆✟☛▲✘✒✼✏ ▼✍✢▲✧✥ ◆✍ ❖✏P▲✘✂✜✣◗✥❘❉✞✩✢✟❋✍ ✟❋✍✢✒✗✩☛●❙☎✞✂✜✍✢✂✌✡✮✍✢✝✾✒✗✣ ❚✟❋●❀✟❋✍✢✂✌❂❯✝✠✟❱✺✘✥✗✟☛✂✼☎❲☞✘✖❳✒✗✣✚✍☛✩✫✥✗✡✫✭❀✝✠✣✞✰❨✍✢✂✌✡✫▲✘✣✞✒✤❩❬✹ ✒✤✰✬●❪❭✤✏✓✂✶✡✌✥✗✣❖✂✜❫❀✖❳✂✌✡✮✍ ❴✍✢▲✘✥✬✍❴✖❳✂✜✩☛✿❵✒✗✩✢❂❃✥✗✣✞✡✌✂✳✏P✝✠❩✠❩✧✺❳✂❛✖❳✒❪✒✗✩✼❍❴✔✳▲✘✥✬✍ ✖✞✩✢✂✌☎✞✝✠✡✜✍✢✝✠✒✤✣❜✝✠✟✕✡✌✒✗✣✘✟☛✝✠✟❋✍✢✂✌✣❪✍ ❝✏P✝❬✍✢▲✎✥✬✡✜✍✢☞✧✥✬❩❪✖❳✂✜✩☛✿❁✒✬✩✢❂❃✥✗✣✞✡✌✂✓✝✠✣❜✔✳✙✴✔ ✂✌✷✗✥✗❩✠☞✧✥◆✍✢✝✠✒✤✣✘✟✌❍❞✁❑✂
✍✢▲✘✂✜✣❢✟☛▲✞✒◆✏ ❣✍✢▲✘✥✬✍❛✍✢✒✎✥✗✡✫▲✘✝✠✂✌✷✸✂
▲✘✝✠✰✤▲❀✹✑❤❪☞✘✥✗❩✠✝❬✍✑● ❉✘✩✢✟❋✍✪✟❋✍✢✒✗✩☛●✴☎✞✂✜✍✢✂✌✡✮✍✢✝✾✒✗✣✪❭◆✍☛✩✫✥✗✡✫✭✯✝✠✣✘✰❛✂✜✐❥✂✌✡✮✍✢✝✾✷✤✂✌✣✘✂✌✟☛✟✪❂✎☞✞✟❋✍✕✝✠❂❦✖✞✩✢✒◆✷✸✂ ✍✢✒✛✥
☎✞✂✌✰✬✩✢✂✌✂✓✍✢▲✘✥✬✍❧✂✜❫❀✖❳✂✜✩✢✝✠✂✌✣✞✡✌✂✳✟☛☞✞✰✤✰✤✂✜✟❋✍✢✟❴✝✠✟❧☞✞✣✘❩✠✝✠✭✸✂✌❩❬●❪❍✪✁❑✂✳✡✌✒✗✣✞✹ ✡✌❩✠☞✧☎❀✂✶✍✢▲✘✥✬✍✓✂✜✐❥✂✌✡✜✍✢✝✠✷✸✂✶❉✘✩✢✟❋✍✳✟❋✍✢✒✬✩☛●❃☎✞✂✮✍✢✂✌✡✜✍✢✝✠✒✤✣❨✝✠✟✓✂✌✝❬✍✢▲✘✂✮✩❛✝✠❂❦✖❳✒✤✟❋✹ ✟☛✝✠✺✘❩✠✂✴✒✗✩P✩✢✂✼❤❪☞✞✝✠✩✢✂✜✟✶✟☛☞✞✺✘✟❋✍✫✥✬✣✯✍✢✝✾✥✬❩✾❩❬●✆☎✞✝❬✐❥✂✜✩✢✂✌✣❪✍✶✥✬✖✘✖✞✩✢✒✤✥✗✡✫▲✘✂✌✟✌❍