A light field records numerous light rays from a real-world scene. However, capturing a dense light field by existing devices is a time-consuming process. Besides, reconstructing a large amount of light rays equivalent to multiple light fields using sparse sampling arises a severe challenge for existing methods. In this paper, we present a learning-based method to reconstruct multiple novel light fields between two mutually independent light fields. We indicate that light rays distributed in different light fields have the same consistent constraints under a certain condition. The most significant constraint is a depth related correlation between angular and spatial dimensions. Our method avoids working out the error-sensitive constraint by employing a deep neural network. We predict residual values of pixels on epipolar plane image (EPI) to reconstruct novel light fields. Our method is able to reconstruct 2 to 4 novel light fields between two mutually independent input light fields. We also compare our results with those yielded by a number of alternatives elsewhere in the literature, which shows our reconstructed light fields have better structure similarity and occlusion.
Apparently, EPI has a strong characteristic of linearity. Many methods explored light field processing based on EPI. However, there are fewer work focusing on angular interpolation of light field. @cite_13 trained a residual-based network to increase angular resolution of EPI. They employ a 1D blur kernel to remove high spatial frequency in the sparsely sampled EPI before feeding to the CNN. Then, they carry out a non-blind de-blur to restore high spatial frequency details which are removed by the blur kernel. However, due to the limitation of blur kernel's size and interpolation algorithm in the preprocessing, the method also fails when the baseline of input views is wide.
@cite_13
2744570394 2257183329 2578266682 2794141660
In this paper, we take advantage of the clear texture structure of the epipolar plane image (EPI) in the light field data and model the problem of light field reconstruction from a sparse set of views as a CNN-based angular detail restoration on EPI. We indicate that one of the main challenges in sparsely sampled light field reconstruction is the information asymmetry between the spatial and angular domain, where the detail portion in the angular domain is damaged by undersampling. To balance the spatial and angular information, the spatial high frequency components of an EPI is removed using EPI blur, before feeding to the network. Finally, a non-blind deblur operation is used to recover the spatial detail suppressed by the EPI blur. We evaluate our approach on several datasets including synthetic scenes, real-world scenes and challenging microscope light field data. We demonstrate the high performance and robustness of the proposed framework compared with the state-of-the-arts algorithms. We also show a further application for depth enhancement by using the reconstructed light field. In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects. We present a method to separate a dichromatic reflection component from diffuse object colors for the set of rays in a 4D light field such that the separation is consistent across all subaperture views. The separation model is based on explaining the observed light field as a sparse linear combination of a constant-color specular term and a small finite set of albedos. Consistency across the light field is achieved by embedding the ray-wise separation into a global optimization framework. On each individual epipolar plane image (EPI), the diffuse coefficients need to be constant along lines which are the projections of the same scene point, while the specular coefficient needs to be constant along the direction of the specular flow within the epipolar volume. We handle both constraints with depth-dependent anisotropic regularizers, and demonstrate promising performance on a number of real-world light fields captured with a Lytro Illum plenoptic camera. In this paper, we explore the structure of light field (LF) and efficiently improve the performance of the pseudo sequence based lenslet image compression, by optimized sub-view rearrangement, enhanced illumination compensation and adaptive reconstruction filtering. First, the decomposed sub-view images are rearranged into a pseudo sequence according to our optimized scan order based on sub-view correlation. Second, the generated pseudo sequence is compressed with JEM codec in which we enhance the illumination compensation by adaptively selecting reference pixels used in parameter derivation. Finally, to reduce the distortions in lenslet image decomposition and reconstruction, we propose an lenslet reconstruction method by applying adaptive filters to the reconstructed lenslet images to compensate the reconstruction errors. Each filter is derived by minimizing the distortions between the original and reconstructed pixels with same geolocation in the lenslet image. Extensive experimental results show that the proposed method achieves up to 53.7 bit rate reduction over HEVC intra coding and 34.8 over JEM intra coding in terms of BDBR.