A light field records numerous light rays from a real-world scene. However, capturing a dense light field by existing devices is a time-consuming process. Besides, reconstructing a large amount of light rays equivalent to multiple light fields using sparse sampling arises a severe challenge for existing methods. In this paper, we present a learning-based method to reconstruct multiple novel light fields between two mutually independent light fields. We indicate that light rays distributed in different light fields have the same consistent constraints under a certain condition. The most significant constraint is a depth related correlation between angular and spatial dimensions. Our method avoids working out the error-sensitive constraint by employing a deep neural network. We predict residual values of pixels on epipolar plane image (EPI) to reconstruct novel light fields. Our method is able to reconstruct 2 to 4 novel light fields between two mutually independent input light fields. We also compare our results with those yielded by a number of alternatives elsewhere in the literature, which shows our reconstructed light fields have better structure similarity and occlusion.
Many methods focus on light field view synthesis, such as @cite_1 @cite_20 @cite_10 @cite_13 @cite_19 @cite_12 @cite_16 @cite_22 @cite_11 . The most important insight is that all these methods synthesize novel views in internal of a light field. Due to the narrow baseline among views of existing commercial light field camera, all the input view images in these methods have high overlapping ratio between each other. These methods use redundant information between input view images. Therefore, they are only able to synthesize novel views within input views. On the contrast, we propose a novel method that can reconstruct multiple light fields between two input light fields instead of reconstructing views inside a single light field. Therefore, our proposed method is capable of reconstructing a dense light field by using a sparse set of input light fields. Besides, the method is able to reconstruct a mass of occlusions in reconstructed light fields without requiring depth estimation.
@cite_22 @cite_10 @cite_1 @cite_19 @cite_16 @cite_13 @cite_12 @cite_20 @cite_11
1951771206 2794141660 2758175304 2745172755
We present a novel phase-based approach for reconstructing 4D light field from a micro-baseline stereo pair. Our approach takes advantage of the unique property of complex steerable pyramid filters in micro-baseline stereo. We first introduce a Disparity Assisted Phase based Synthesis (DAPS) strategy that can integrate disparity information into the phase term of a reference image to warp it to its close neighbor views. Based on the DAPS, an “analysis by synthesis” approach is proposed to warp from one of the input binocular images to the other, and iteratively optimize the disparity map to minimize the phase differences between the warped one and the ground truth input. Finally, the densely and regularly spaced, high quality light field images can be reconstructed using the proposed DAPS according to the refined disparity map. Our approach also solves the problems of disparity inconsistency and ringing artifact in available phase-based view synthesis methods. Experimental results demonstrate that our approach substantially improves both the quality of disparity map and light field, compared with the state-of-the-art stereo matching and image based rendering approaches. In this paper, we explore the structure of light field (LF) and efficiently improve the performance of the pseudo sequence based lenslet image compression, by optimized sub-view rearrangement, enhanced illumination compensation and adaptive reconstruction filtering. First, the decomposed sub-view images are rearranged into a pseudo sequence according to our optimized scan order based on sub-view correlation. Second, the generated pseudo sequence is compressed with JEM codec in which we enhance the illumination compensation by adaptively selecting reference pixels used in parameter derivation. Finally, to reduce the distortions in lenslet image decomposition and reconstruction, we propose an lenslet reconstruction method by applying adaptive filters to the reconstructed lenslet images to compensate the reconstruction errors. Each filter is derived by minimizing the distortions between the original and reconstructed pixels with same geolocation in the lenslet image. Extensive experimental results show that the proposed method achieves up to 53.7 bit rate reduction over HEVC intra coding and 34.8 over JEM intra coding in terms of BDBR. We present a novel algorithm for view synthesis that utilizes a soft 3D reconstruction to improve quality, continuity and robustness. Our main contribution is the formulation of a soft 3D representation that preserves depth uncertainty through each stage of 3D reconstruction and rendering. We show that this representation is beneficial throughout the view synthesis pipeline. During view synthesis, it provides a soft model of scene geometry that provides continuity across synthesized views and robustness to depth uncertainty. During 3D reconstruction, the same robust estimates of scene visibility can be applied iteratively to improve depth estimation around object edges. Our algorithm is based entirely on O(1) filters, making it conducive to acceleration and it works with structured or unstructured sets of input views. We compare with recent classical and learning-based algorithms on plenoptic lightfields, wide baseline captures, and lightfield videos produced from camera arrays. We present a machine learning algorithm that takes as input a 2D RGB image and synthesizes a 4D RGBD light field (color and depth of the scene in each ray direction). For training, we introduce the largest public light field dataset, consisting of over 3300 plenoptic camera light fields of scenes containing flowers and plants. Our synthesis pipeline consists of a convolutional neural network (CNN) that estimates scene geometry, a stage that renders a Lambertian light field using that geometry, and a second CNN that predicts occluded rays and non-Lambertian effects. Our algorithm builds on recent view synthesis methods, but is unique in predicting RGBD for each light field ray and improving unsupervised single image depth estimation by enforcing consistency of ray depths that should intersect the same scene point. Please see our supplementary video at this https URL