The plenoptic camera can capture both angular and spatial information of the rays, enabling 3D reconstruction by single exposure. The geometry of the recovered scene structure is affected by the calibration of the plenoptic camera significantly. In this paper, we propose a novel unconstrained two-parallel-plane (TPP) model with 7 parameters to describe a 4D light field. By reconstructing scene points from ray-ray association, a 3D projective transformation is deduced to establish the relationship between the scene structure and the TPP parameters. Based on the transformation, we simplify the focused plenoptic camera as a TPP model and calibrate its intrinsic parameters. Our calibration method includes a close-form solution and a nonlinear optimization by minimizing re-projection error. Experiments on both simulated data and real scene data verify the performance of the calibration on the focused plenoptic camera.
To acquire light field, there are various imaging systems developed from the traditional camera. @cite_25 presented a camera array to obtain light field with high spatial and angular resolution. Prior work dealt with the calibration of the camera arrays @cite_4 . Unfortunately, applications on camera arrays are limited by its high cost and complex control. In contrast, a MLA enables a single camera to record 4D light field more conveniently and efficiently, though the baseline and spatial resolution is smaller than the camera array. Recent work devoted to calibrate the intrinsic parameters of the plenoptic cameras in two designs @cite_0 @cite_3 , which are quite different according to the image structure of the micro lenses. Moreover, in traditional multi-view geometry, multiple cameras in different poses are defined as a set of unconstrained rays, which is known as as Generalized Camera Model (GCM) @cite_16 . The ambiguity of the reconstructed scene was discussed in traditional topics. For a plenoptic camera, the different views of the same scene point are obtained, and the calibration of a plenoptic camera can use the theory on traditional multi-view for reference.
@cite_4 @cite_3 @cite_0 @cite_16 @cite_25
2133515844 2897594148 1131710387 2622195611
Plenoptic cameras, constructed with internal microlens arrays, focus those microlenses at infinity in order to sample the 4D radiance directly at the microlenses. The consequent assumption is that each microlens image is completely defocused with respect to to the image created by the main camera lens and the outside object. As a result, only a single pixel in the final image can be rendered from it, resulting in disappointingly low resolution. In this paper, we present a new approach to lightfield capture and image rendering that interprets the microlens array as an imaging system focused on the focal plane of the main camera lens. This approach captures a lightfield with significantly higher spatial resolution than the traditional approach, allowing us to render high resolution images that meet the expectations of modern photographers. Although the new approach samples the lightfield with reduced angular density, analysis and experimental results demonstrate that there is sufficient parallax to completely support lightfield manipulation algorithms such as refocusing and novel views Limited angular resolution has become the main bottleneck of microlens-based plenoptic cameras towards practical vision applications. Existing view synthesis methods mainly break the task into two steps, i.e. depth estimating and view warping, which are usually inefficient and produce artifacts over depth ambiguities. In this paper, an end-to-end deep learning framework is proposed to solve these problems by exploring Pseudo 4DCNN. Specifically, 2D strided convolutions operated on stacked EPIs and detail-restoration 3D CNNs connected with angular conversion are assembled to build the Pseudo 4DCNN. The key advantage is to efficiently synthesize dense 4D light fields from a sparse set of input views. The learning framework is well formulated as an entirely trainable problem, and all the weights can be recursively updated with standard backpropagation. The proposed framework is compared with state-of-the-art approaches on both genuine and synthetic light field databases, which achieves significant improvements of both image quality (+2 dB higher) and computational efficiency (over 10X faster). Furthermore, the proposed framework shows good performances in real-world applications such as biometrics and depth estimation. The use of a multi-camera system enables a robot to obtain a surround view, and thus, maximize its perceptual awareness of its environment. If vision-based simultaneous localization and mapping (vSLAM) is expected to provide reliable pose estimates for a micro aerial vehicle (MAV) with a multi-camera system, an accurate calibration of the multi-camera system is a necessary prerequisite. We propose a novel vSLAM-based self-calibration method for a multi-camera system that includes at least one calibrated stereo camera, and an arbitrary number of monocular cameras. We assume overlapping fields of view to only exist within stereo cameras. Our self-calibration estimates the inter-camera transforms with metric scale; metric scale is inferred from calibrated stereo. On our MAV, we set up each camera pair in a stereo configuration which facilitates the estimation of the MAV's pose with metric scale. Once the MAV is calibrated, the MAV is able to estimate its global pose via a multi-camera vSLAM implementation based on the generalized camera model. We propose a novel minimal and linear 3-point algorithm that uses relative rotation angle measurements from a 3-axis gyroscope to recover the relative motion of the MAV with metric scale and from 2D-2D feature correspondences. This relative motion estimation does not involve scene point triangulation. Our constant-time vSLAM implementation with loop closures runs on-board the MAV in real-time. To the best of our knowledge, no published work has demonstrated real-time on-board vSLAM with loop closures. We show experimental results from simulation experiments, and real-world experiments in both indoor and outdoor environments. Over the last decade, advancements in optical devices have made it possible for new novel image acquisition technologies to appear. Angular information for each spatial point is acquired in addition to the spatial information of the scene that enables 3D scene reconstruction and various post-processing effects. Current generation of plenoptic cameras spatially multiplex the angular information, which implies an increase in image resolution to retain the level of spatial information gathered by conventional cameras. In this work, the resulting plenoptic image is interpreted as a multi-view sequence that is efficiently compressed using the multi-view extension of high efficiency video coding (MV-HEVC). A novel two-dimensional weighted prediction and rate allocation scheme is proposed to adopt the HEVC compression structure to the plenoptic image properties. The proposed coding approach is a response to ICIP 2017 Grand Challenge: Light field Image Coding. The proposed scheme outperforms all ICME-contestants, and improves on the JPEG-anchor of ICME with an average PSNR gain of 7.5 dB and the HEVC-anchor of ICIP 2017 Grand Challenge with an average PSNR gain of 2.4 dB.