In this work we propose a novel approach to remove undesired objects from RGB-D sequences captured with freely moving cameras, which enables static 3D reconstruction. Our method jointly uses existing information from multiple frames as well as generates new one via inpainting techniques. We use balanced rules to select source frames; local homography based image warping method for alignment and Markov random field (MRF) based approach for combining existing information. For the left holes, we employ exemplar based multi-view inpainting method to deal with the color image and coherently use it as guidance to complete the depth correspondence. Experiments show that our approach is qualified for removing the undesired objects and inpainting the holes.
Multi-view inpainting techniques leverage information from multiple source frames. Hays and Efros @cite_21 gather photos from Internet as a huge database to help with image completion. Similarly, Whyte @cite_24 cover an undesired region on the query image with Internet photographs of the same scene, in which multi-homography and photometric registration are used to achieve geometric registration between the query image and the source ones. Also a Markov random field optimization @cite_37 is employed for selecting the optimal proposals. This kind of methods are pretty unstable since the masked objects are easy to be reintroduced as a result of lacking necessary means to filer the source information.
@cite_24 @cite_37 @cite_21
1997903019 2001933992 2171011251
Often when we review our holiday photos, we notice things we wish we could have avoided, such as vehicles, construction work, or simply other tourists. We cannot go back and retake the photo, so what can we do if we want to remove these things from our photos? We want to replace these sections of the image in a convincing way, preferably with what would really have been seen there without the occlusions. Previous work on this problem, often referred to as “inpainting”, are mainly applicable to small image regions, and rely largely on models of the local behaviour of natural images, e.g. [3, 5]. Recently, replacing large occlusions in photographs has been approached using images from the Internet [2, 7] or by combining several images captured at approximately the same time [1, 9]. In this paper we leverage recent advances in viewpoint invariant image search [8] to find other images of the same scene on the Internet. Beginning with a query image containing a target region to be replaced, we first use an online image search engine to retrieve images of the same scene, and take these to be a set of oracles. Since these images may have significant variations in viewpoint and lighting, we register each oracle to the query image using multiple homographies and a simple global photometric correction. We then use each oracle to propose a solution, by copying image data into the target region using Poisson blending. Finally, we use a Markov random field (MRF) formulation to combine the proposals into a single, occlusion-free result. Figure 1 shows the main stages of our system, and compares our result to those of two other methods. We describe an interactive, computer-assisted framework for combining parts of a set of photographs into a single composite picture, a process we call "digital photomontage." Our framework makes use of two techniques primarily: graph-cut optimization, to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion, a process based on Poisson equations, to further reduce any remaining visible artifacts in the composite. Also central to the framework is a suite of interactive tools that allow the user to specify a variety of high-level image objectives, either globally across the image, or locally through a painting-style interface. Image objectives are applied independently at each pixel location and generally involve a function of the pixel values (such as "maximum contrast") drawn from that same location in the set of source images. Typically, a user applies a series of image objectives iteratively in order to create a finished composite. The power of this framework lies in its generality; we show how it can be used for a wide variety of applications, including "selective composites" (for instance, group photos in which everyone looks their best), relighting, extended depth of field, panoramic stitching, clean-plate production, stroboscopic visualization of movement, and time-lapse mosaics. What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.