Human visual system relies on both binocular stereo cues and monocular focusness cues to gain effective 3D perception. In computer vision, the two problems are traditionally solved in separate tracks. In this paper, we present a unified learning-based technique that simultaneously uses both types of cues for depth inference. Specifically, we use a pair of focal stacks as input to emulate human perception. We first construct a comprehensive focal stack training dataset synthesized by depth-guided light field rendering. We then construct three individual networks: a FocusNet to extract depth from a single focal stack, a EDoFNet to obtain the extended depth of field (EDoF) image from the focal stack, and a StereoNet to conduct stereo matching. We then integrate them into a unified solution to obtain high quality depth maps. Comprehensive experiments show that our approach outperforms the state-of-the-art in both accuracy and speed and effectively emulates human vision systems.
Blur carries information about the object's distance. Depth from Focus Defocus (DfF DfD) recovers scene depth from a collection of images captured under varying focus settings. In general, DfF @cite_18 @cite_25 @cite_1 determines the depth by analyzing the most in-focus slice in the focal stack, while DfD @cite_36 @cite_15 infers depth based on the amount of the spatially varying blur at each pixel. To avoid ambiguity in textureless region, Moreno-Noguer @cite_52 used active illumination to project a sparse set of dots onto the scene. The defocus of the dots offers depth cue, which could be further used for realistic refocusing. @cite_53 combined focal stack with varying aperture to recover scene geometry. Moeller @cite_6 applied an efficient nonconvex minimization technique to solve DfD in a variational framework. Suwajanakorn @cite_17 proposed the DfF with mobile phone under uncalibrated setting. They first aligned the focal stack, then jointly optimized the camera parameters and depth map, and further refined the depth map using anisotropic regularization.
@cite_18 @cite_36 @cite_53 @cite_1 @cite_52 @cite_6 @cite_15 @cite_25 @cite_17
2511026390 1964313029 2089129849 1923779427
Scene depth variation is an important factor that leads to spatially-varying camera motion blur. Most of the previous methods require auxiliary cameras or user interaction to make depth-aware deblurring tractable. In this work, we propose to use a noisy blurred noisy image sequence and simultaneously recorded inertial measurements to jointly estimate scene depth and remove spatially-varying blur caused by depth variation and camera in-plane motion. The inertial data could provide initialization of camera motion parameters, while the noisy image pair preserve large-scale sharp edges from which a coarse disparity map can be generated. However, this initial estimate is not accurate enough to produce a high-quality clean image. Therefore, we develop an optimization scheme to refine depth, motion parameters and latent image alternately. A Markov Random Field (MRF) framework is formulated to solve for the depth map by exploiting both stereo cues and motion blur cues and the residual Richardson-Lucy algorithm is used to effectively suppress deconvolution ringing artifacts. Experimental results demonstrate that our approach can address both depth estimation as well as image deblurring. The classical approach to depth from defocus (DFD) uses lenses with circular apertures for image capturing. We show in this paper that the use of a circular aperture severely restricts the accuracy of DFD. We derive a criterion for evaluating a pair of apertures with respect to the precision of depth recovery. This criterion is optimized using a genetic algorithm and gradient descent search to arrive at a pair of high resolution apertures. These two coded apertures are found to complement each other in the scene frequencies they preserve. This property enables them to not only recover depth with greater fidelity but also obtain a high quality all-focused image from the two captured images. Extensive simulations as well as experiments on a variety of real scenes demonstrate the benefits of using the coded apertures over conventional circular apertures. In this paper we propose a novel depth measurement method by fusing depth from defocus (DFD) and stereo. One of the problems of passive stereo method is the difficulty of finding correct correspondence between images when an object has a repetitive pattern or edges parallel to the epipolar line. On the other hand, the accuracy of DFD method is inherently limited by the effective diameter of the lens. Therefore, we propose the fusion of stereo method and DFD by giving different focus distances for left and right cameras of a stereo camera with coded apertures. Two types of depth cues, defocus and disparity, are naturally integrated by the magnification and phase shift of a single point spread function (PSF) per camera. In this paper we give the proof of the proportional relationship between the diameter of defocus and disparity which makes the calibration easy. We also show the outstanding performance of our method which has both advantages of two depth cues through simulation and actual experiments. While prior depth from focus and defocus techniques operated on laboratory scenes, we introduce the first depth from focus (DfF) method capable of handling images from mobile phones and other hand-held cameras. Achieving this goal requires solving a novel uncalibrated DfF problem and aligning the frames to account for scene parallax. Our approach is demonstrated on a range of challenging cases and produces high quality results.