We present a Deep Convolutional Neural Network architecture which serves as a generic image-to-image regressor that can be trained end-to-end without any further machinery. Our proposed architecture: the Recursively Branched Deconvolutional Network (RBDN) develops a cheap multi-context image representation very early on using an efficient recursive branching scheme with extensive parameter sharing and learnable upsampling. This multi-context representation is subjected to a highly non-linear locality preserving transformation by the remainder of our network comprising of a series of convolutions deconvolutions without any spatial downsampling. The RBDN architecture is fully convolutional and can handle variable sized images during inference. We provide qualitative quantitative results on @math diverse tasks: relighting, denoising and colorization and show that our proposed RBDN architecture obtains comparable results to the state-of-the-art on each of these tasks when used off-the-shelf without any post processing or task-specific architectural modifications.
In the field of Face Recognition Verification, while most research focuses on extracting illumination-invariant features, is the relatively less explored alternative @cite_37 of directly making illumination corrections normalizations to an image. Traditional face relighting approaches used the Retinex @cite_47 Lambertian Reflectance @cite_10 theory and used spherical @cite_62 @cite_10 hemispherical @cite_41 harmonics, subspace-based @cite_22 @cite_29 or dictionary-based @cite_63 @cite_8 @cite_61 @cite_64 @cite_46 @cite_27 illumination corrections. Deep Lambertian Networks @cite_12 encoded lambertian models illumination corrections directly into their network architecture. This however limited the expressive power of the network, particularly due to the strong lambertian assumptions on isotropicity and absence of specular highlights, which seldom hold true for face images. In section , we show that it is possible to train a well-performing relighting model without making any lambertian assumptions using our generic RBDN architecture.
@cite_61 @cite_37 @cite_62 @cite_64 @cite_22 @cite_8 @cite_41 @cite_46 @cite_29 @cite_27 @cite_63 @cite_47 @cite_10 @cite_12
56688174 2092179111 2053955554 2153690423
This paper focuses on enhancing Sparse Representation based Classifier (SRC) in single-sample face recognition tasks under varying illumination conditions. The major contribution is two-fold: firstly, we present an interesting observation based on Lambertian reflectance model: the identity information will be canceled out by the pair-wise difference images from the same subject in logarithmic domain, and only the subject-independent illumination variation retains. Secondly, inspired from this observation, we propose to “borrow” illumination variations from any generic subject by constructing an illumination variation dictionary composed of pair-wise difference images of generic subjects in logarithmic domain to cover the possible illumination variations between test and gallery samples. Experimental results on Extended Yale B and FERET face databases demonstrate the superiority of our method. As part of the face recognition task in a robust security system, we propose a novel approach for the illumination recovery of faces with cast shadows and specularities. Given a single 2D face image, we relight the face object by extracting the nine spherical harmonic bases and the face spherical illumination coefficients by using the face spherical spaces properties. First, an illumination training database is generated by computing the properties of the spherical spaces out of face albedo and normal values estimated from 2D training images. The training database is then discriminately divided into two directions in terms of the illumination quality and light direction of each image. Based on the generated multi-level illumination discriminative training space, we analyze the target face pixels and compare them with the appropriate training subspace using pre-generated tiles. When designing the framework, practical real-time processing speed and small image size were considered. In contrast to other approaches, our technique requires neither 3D face models nor restricted illumination conditions for the training process. Furthermore, the proposed approach uses one single face image to estimate the face albedo and face spherical spaces. In this work, we also provide the results of a series of experiments performed on publicly available databases to show the significant improvements in the face recognition rates. Face recognition under varying pose is a challenging problem, especially when illumination variations are also present. In this paper, we propose to address one of the most challenging scenarios in face recognition. That is, to identify a subject from a test image that is acquired under dierent pose and illumination condition from only one training sample (also known as a gallery image) of this subject in the database. For example, the test image could be semifrontal and illuminated by multiple lighting sources while the corresponding training image is frontal under a single lighting source. Under the assumption of Lambertian reflectance, the spherical harmonics representation has proved to be effective in modeling illumination variations for a fixed pose. In this paper, we extend the spherical harmonics representation to encode pose information. More specifically, we utilize the fact that 2D harmonic basis images at different poses are related by close-form linear transformations, and give a more convenient transformation matrix to be directly used for basis images. An immediate application is that we can easily synthesize a different view of a subject under arbitrary lighting conditions by changing the coefficients of the spherical harmonics representation. A more important result is an efficient face recognition method, based on the orthonormality of the linear transformations, for solving the above-mentioned challenging scenario. Thus, we directly project a nonfrontal view test image onto the space of frontal view harmonic basis images. The impact of some empirical factors due to the projection is embedded in a sparse warping matrix; for most cases, we show that the recognition performance does not deteriorate after warping the test image to the frontal view. Very good recognition results are obtained using this method for both synthetic and challenging real images. To deal with image variations due to illumination problem, recently Ramamoorthi and Basri have independently derived a spherical harmonic analysis for the Lambertian reflectance and linear subspace. Their theoretical work provided a new approach for face representation, however both of them had the assumption that the 3D surface normal and albedo are known. This assumption limits this algorithm's application. In this paper, we present a novel method for modeling 3D face shape and albedo from only three images with unknown light directions and this work well fills the blank, which Ramamoorthi and Basri left. By taking the advantage of similar 3D shape of all human faces, the highlight of the new method is that it circumambulates the linear ambiguity by 3D alignment. The experiment results show that our estimated model can be perfectly employed to face recognition and 3D reconstruction.