Time-of-Flight (ToF) cameras require active illumination to obtain depth information thus the power of illumination directly affects the performance of ToF cameras. Traditional ToF imaging algorithms is very sensitive to illumination and the depth accuracy degenerates rapidly with the power of it. Therefore, the design of a power efficient ToF camera always creates a painful dilemma for the illumination and the performance trade-off. In this paper, we show that despite the weak signals in many areas under extreme short exposure setting, these signals as a whole can be well utilized through a learning process which directly translates the weak and noisy ToF camera raw to depth map. This creates an opportunity to tackle the aforementioned dilemma and make a very power efficient ToF camera possible. To enable the learning, we collect a comprehensive dataset under a variety of scenes and photographic conditions by a specialized ToF camera. Experiments show that our method is able to robustly process ToF camera raw with the exposure time of one order of magnitude shorter than that used in conventional ToF cameras. In addition to evaluating our approach both quantitatively and qualitatively, we also discuss its implication to designing the next generation power efficient ToF cameras. We will make our dataset and code publicly available.
Image enhancement under low light. For conventional RGB cameras, photography in low light is challenging. Several techniques have been proposed to increase the SNR of the recovered image @cite_17 @cite_9 @cite_3 @cite_14 @cite_22 . Chen at el. @cite_21 established a pipeline by training a fully convolutional neural network which directly translate the very noise and dark Bayer pattern camera raw to high quality color images. Though impressive results from the aforementioned studies, deep learning and data-driven approaches have not yet been adopted to recover high quality depth information from weak and noisy ToF raw. It remains unclear if such methodology is effective for ToF imaging. The aim of this paper is to disclose its feasibility.
@cite_14 @cite_22 @cite_9 @cite_21 @cite_3 @cite_17
2793381735 2951408185 2799265886 2593264379
RGB-D images captured by consumer camera can provide pair-wise color and depth information but depth image usually contains strong noise and large holes. Due to different modalities of RGB-D, the intensity-guided depth enhancement easily causes artifacts and blurred edges. To solve this problem, we propose the multi-scale mutual feature convolutional neural network (MSMF-CNN) to learn essential mutual features of RGB-D. This end-to-end framework has two stages, i.e., mutual feature learning and depth regeneration. Firstly, we design two parallel separated subnetworks as intensity and depth mutual feature generator. Specifically, we design a multi-scale mutual feature generator to enforce the depth mutual feature learning and incorporate various structure characteristic. Secondly, depth transformation sub-network combines two mutual features to recover clean depth image. We tested our model on two well-known datasets in terms of PSNR, visual effect and computing speed. Compared with state-of-art algorithms, our method provides better performance of all three criteria. Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure low-light images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work. The results are shown in the supplementary video at this https URL Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure low-light images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work. In low light condition, the signal-to-noise ratio (SNR) is low and thus the captured images are seriously degraded by noise. Since low light images contain much noise in flat and dark regions, contrast enhancement without considering noise characteristics causes serious noise amplification. In this paper, we propose low light image enhancement based on two-step noise suppression. First, we perform noise aware contrast enhancement using noise level function (NLF). NLF is used to get a noise aware histogram which prevents noise amplification, and we use the noise aware histogram in contrast enhancement. However, the increase of intensity by contrast enhancement reduces the visibility threshold, which makes noise visible by human eyes. Second, we utilize a just noticeable difference (JND) model from luminance adaptation to suppress noise based on human visual perception. Experimental results show that the proposed method successfully enhances contrast in low light images while minimizing noise amplification.