Abstract Computational color constancy refers to the problem of computing the illuminant color so that the images of a scene under varying illumination can be normalized to an image under the canonical illumination. In this paper, we adopt a deep learning framework for the illumination estimation problem. The proposed method works under the assumption of uniform illumination over the scene and aims for the accurate illuminant color computation. Specifically, we trained the convolutional neural network to solve the problem by casting the color constancy problem as an illumination classification problem. We designed the deep learning architecture so that the output of the network can be directly used for computing the color of the illumination. Experimental results show that our deep network is able to extract useful features for the illumination estimation and our method outperforms all previous color constancy methods on multiple test datasets.
Direct methods build their own estimation model and estimate the illumination by learning the model from the training data. Gamut-based methods @cite_8 @cite_60 @cite_4 find the canonical gamut from the training data and estimate the illumination by mapping the gamut of the input image into the canonical gamut. Distributions of the pixel intensity and the chromaticity are used as the key features for estimating the illumination in the correlation framework @cite_39 , the neural network based method @cite_12 , the support vector regression @cite_53 , and the Bayesian framework @cite_14 @cite_31 . @cite_50 @cite_4 , the derivative structure and the spatial distribution of the image are used for the illumination estimation.
@cite_14 @cite_4 @cite_60 @cite_8 @cite_53 @cite_39 @cite_50 @cite_31 @cite_12
2748157832 2104487652 2110635295 2736596523
In this paper, we present a method that estimates reflectance and illumination information from a single image depicting a single-material specular object from a given class under natural illumination. We follow a data-driven, learning-based approach trained on a very large dataset, but in contrast to earlier work we do not assume one or more components (shape, reflectance, or illumination) to be known. We propose a two-step approach, where we first estimate the object’s reflectance map, and then further decompose it into reflectance and illumination. For the first step, we introduce a Convolutional Neural Network (CNN) that directly predicts a reflectance map from the input image itself, as well as an indirect scheme that uses additional supervision, first estimating surface orientation and afterwards inferring the reflectance map using a learning-based sparse data interpolation technique. For the second step, we suggest a CNN architecture to reconstruct both Phong reflectance parameters and high-resolution spherical illumination maps from the reflectance map. We also propose new datasets to train these CNNs. We demonstrate the effectiveness of our approach for both steps by extensive quantitative and qualitative evaluation in both synthetic and real data as well as through numerous applications, that show improvements over the state-of-the-art. This paper describes a new method for estimating the illumination distribution of a real scene from a radiance distribution inside shadows cast by an object in the scene. First, the illumination distribution of the scene is approximated by discrete sampling of an extended light source. Then the illumination distribution of the scene is estimated from a radiance distribution inside shadows cast by an object of known shape onto another object in the scene. Instead of assuming any particular reflectance properties of the surface inside the shadows, both the illumination distribution of the scene and the reflectance properties of the surface are estimated simultaneously, based on iterative optimization framework. In addition, this paper introduces an adaptive sampling of the illumination distribution of a scene. Rather than using a uniform discretization of the overall illumination distribution, we adaptively increase sampling directions of the illumination distribution based on the estimation at the previous iteration. Using the adaptive sampling framework, we are able to estimate overall illumination more efficiently by using fewer sampling directions. The proposed method is effective for estimating an illumination distribution even under a complex illumination environment. Illumination estimation is an important component of color constancy and automatic white balancing. A number of methods of combining illumination estimates obtained from multiple subordinate illumination estimation methods now appear in the literature. These combinational methods aim to provide better illumination estimates by fusing the information embedded in the subordinate solutions. The existing combinational methods are surveyed and analyzed here with the goals of determining: 1) the effectiveness of fusing illumination estimates from multiple subordinate methods; 2) the best method of combination; 3) the underlying factors that affect the performance of a combinational method; and 4) the effectiveness of combination for illumination estimation in multiple-illuminant scenes. The various combinational methods are categorized in terms of whether or not they require supervised training and whether or not they rely on high-level scene content cues (e.g., indoor versus outdoor). Extensive tests and enhanced analyzes using three data sets of real-world images are conducted. For consistency in testing, the images were labeled according to their high-level features (3D stages, indoor outdoor) and this label data is made available on-line. The tests reveal that the trained combinational methods (direct combination by support vector regression in particular) clearly outperform both the non-combinational methods and those combinational methods based on scene content cues. We present a convolutional neural network (CNN) based solution for modeling physically plausible spatially varying surface reflectance functions (SVBRDF) from a single photograph of a planar material sample under unknown natural illumination. Gathering a sufficiently large set of labeled training pairs consisting of photographs of SVBRDF samples and corresponding reflectance parameters, is a difficult and arduous process. To reduce the amount of required labeled training data, we propose to leverage the appearance information embedded in unlabeled images of spatially varying materials to self-augment the training process. Starting from an initial approximative network obtained from a small set of labeled training pairs, we estimate provisional model parameters for each unlabeled training exemplar. Given this provisional reflectance estimate, we then synthesize a novel temporary labeled training pair by rendering the exact corresponding image under a new lighting condition. After refining the network using these additional training samples, we re-estimate the provisional model parameters for the unlabeled data and repeat the self-augmentation process until convergence. We demonstrate the efficacy of the proposed network structure on spatially varying wood, met als, and plastics, as well as thoroughly validate the effectiveness of the self-augmentation training process.