Learning-based color enhancement approaches typically learn to map from input images to retouched images. Most of existing methods require expensive pairs of input-retouched images or produce results in a non-interpretable way. In this paper, we present a deep reinforcement learning (DRL) based method for color enhancement to explicitly model the step-wise nature of human retouching process. We cast a color enhancement process as a Markov Decision Process where actions are defined as global color adjustment operations. Then we train our agent to learn the optimal global enhancement sequence of the actions. In addition, we present a 'distort-and-recover' training scheme which only requires high-quality reference images for training instead of input and retouched image pairs. Given high-quality reference images, we distort the images' color distribution and form distorted-reference image pairs for training. Through extensive experiments, we show that our method produces decent enhancement results and our DRL approach is more suitable for the 'distort-and-recover' training scheme than previous supervised approaches. Supplementary material and code are available at this https URL
One traditional approach for color enhancement is transferring the color of an example image to a given input image. It is originated from @cite_19 in which the global color distribution of an input image is warped to mimic an example style. There are many subsequent works to improve this technique @cite_0 . While this approach can provide expressive enhancement and diverse stylizations, the results highly depend on example images while providing proper exemplars is challenging. Recent works @cite_14 @cite_13 @cite_3 (semi-)automate exemplar selection by image retrieval methods. Liu al @cite_14 used a keyword-based image search to choose example images. Lee al @cite_13 learn a content-specific style ranking using a large photo collection and select the best exemplar images for color enhancement. For pixel-wise local enhancement, Hwang al @cite_3 find candidate images from a database then search local color enhancement operators.
@cite_14 @cite_3 @cite_0 @cite_19 @cite_13
1984127314 2464766066 2471440592 2113636985 1994796550 2395531316 2247537918 2103069163 2611605760 2951924128 2946864864 2412926690 1976416062 2764975447 2162342922 2964024144 2564591810 2160530465 2154294191 2149049515
Abstract Conventional enhancements for the color display of multispectral images are based on independent contrast modifications or “stretches” of three input images. This approach is not effective if the image channels are highly correlated or if the image histograms are strongly bimodal or more complex. Any of several procedures that tend to “stretch” color saturation while leaving hue unchanged may better utilize the full range of colors for the display of image information. Two conceptually different enhancements are discussed: the “decorrelation stretch”, based on principal-component (PC) analysis, and the “stretch” of “hue”-“saturation”-intensity (HSI) transformed data. The PC transformation is scene-dependent, but the HSI transformation is invariant. Examples of images enhanced by conventional linear stretches, decorrelation stretch, and by stretches of HSI transformed data are compared. Schematic variation diagrams or two- and three-dimensional histograms are used to illustrate the “decorrelation stretch” method and the effect of the different enhancements. The aim of this paper is twofold. First, we propose a new method for enhancing the contrast of gray-value images. We use the difference of the average local contrast measures between the original and the enhanced images within a variational framework. This enables the user to intuitively control the contrast level and the scale of the enhanced details. Moreover, our model avoids large modifications of the original image histogram. Thereby it preserves the global illumination of the scene and it can cope with large areas having similar gray values. The minimizer of the proposed functional is computed by a gradient descent algorithm in connection with a polynomial approximation of the average local contrast measure. The polynomial approximation is computed via Bernstein polynomials. In the second part, the approach is extended to a variational enhancement method for color images. The model approximately preserves the hue of the original image and additionally includes a total variation term to correct the possible noise. The method requires no post- or preprocessing. The minimization problem is solved with a hybrid primal---dual algorithm. Experiments demonstrate the efficiency and the flexibility of the proposed approaches in comparison with state-of-the-art methods. This paper presents a novel unsupervised method to transfer the style of an example image to a source image. The complex notion of image style is here considered as a local texture transfer, eventually coupled with a global color transfer. For the local texture transfer, we propose a new method based on an adaptive patch partition that captures the style of the example image and preserves the structure of the source image. More precisely, this example-based partition predicts how well a source patch matches an example patch. Results on various images show that our method outperforms the most recent techniques. We present a machine-learned ranking approach for automatically enhancing the color of a photograph. Unlike previous techniques that train on pairs of images before and after adjustment by a human user, our method takes into account the intermediate steps taken in the enhancement process, which provide detailed information on the person's color preferences. To make use of this data, we formulate the color enhancement task as a learning-to-rank problem in which ordered pairs of images are used for training, and then various color enhancements of a novel input image can be evaluated from their corresponding rank values. From the parallels between the decision tree structures we use for ranking and the decisions made by a human during the editing process, we posit that breaking a full enhancement sequence into individual steps can facilitate training. Our experiments show that this approach compares well to existing methods for automatic color enhancement. Color image enhancement is a complex and challenging task in digital imaging with abundant applications. Preserving the hue of the input image is crucial in a wide range of situations. We propose simple image enhancement algorithms, which conserve the hue and preserve the range (gamut) of the R, G, B channels in an optimal way. In our setup, the intensity input image is transformed into a target intensity image whose histogram matches a specified, well-behaved histogram. We derive a new color assignment methodology where the resulting enhanced image fits the target intensity image. We analyze the obtained algorithms in terms of chromaticity improvement and compare them with the unique and quite popular histogram-based hue and range preserving algorithm of Naik and Murthy. Numerical tests confirm our theoretical results and show that our algorithms perform much better than the Naik-Murthy algorithm. In spite of their simplicity, they compete with well-established alternative methods for images where hue-preservation is desired. Color enhancement is a very important aspect of photo editing. Even when photographers have tens of or hundreds of photographs, they must enhance each photo one by one by manually tweaking sliders in software such as brightness and contrast, because automatic color enhancement is not always satisfactory for them. To support this repetitive manual task, we present self-reinforcing color enhancement, where the system implicitly and progressively learns the user's preferences by training on their photo editing history. The more photos the user enhances, the more effectively the system supports the user. We present a working prototype system called SelPh, and then describe the algorithms used to perform the self-reinforcement. We conduct a user study to investigate how photographers would use a self-reinforcing system to enhance a collection of photos. The results indicate that the participants were satisfied with the proposed system and strongly agreed that the self-reinforcing approach is preferable to the traditional workflow. We introduce a new technique that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner. We achieve this by learning style ranking for a given input using a large photo collection and selecting a diverse subset of matching styles for final style transfer. We also propose a novel technique that transfers the global color and tone of the chosen exemplars to the input photograph while avoiding the common visual artifacts produced by the existing style transfer methods. Together, our style selection and transfer techniques produce compelling, artifact-free results on a wide range of input photographs, and a user study shows that our results are preferred over other techniques. Extensive studies have been carried out for detecting image forgery such as copy-move, re-sampling, blurring, and contrast enhancement. Although color modification is a common forgery technique, there is no reported forensic method for detecting this type of manipulation. In this paper, we propose a novel algorithm for estimating color modification in images acquired from digital cameras when the images are modified. Most commercial digital cameras are equipped with a color filter array (CFA) for acquiring the color information of each pixel. As a result, the images acquired from such digital cameras include a trace from the CFA pattern. This pattern is composed of the basic red green blue (RGB) colors, and it is changed when color modification is carried out on the image. We designed an advanced intermediate value counting method for measuring the change in the CFA pattern and estimating the extent of color modification. The proposed method is verified experimentally by using 10,366 test images. The results confirmed the ability of the proposed method to estimate color modification with high accuracy. We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene. Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" [ 2001] with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique deep image analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style texture transfer, color style swap, sketch painting to photo, and time lapse. We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene. Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique Deep Image Analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style texture transfer, color style swap, sketch painting to photo, and time lapse. Contrast enhancement algorithms have been evolved through last decades to meet the requirement of its objectives. Actually, there are two main objectives while enhancing the contrast of an image: (i) improve its appearance for visual interpretation and (ii) facilitate increase the performance of subsequent tasks (e.g., image analysis, object detection, and image segmentation). Most of the contrast enhancement techniques are based on histogram modifications, which can be performed globally or locally. The Contrast Limited Adaptive Histogram Equalization (CLAHE) is a method which can overcome the limitations of global approaches by performing local contrast enhancement. However, this method relies on two essential hyperparameters: the number of tiles and the clip limit. An improper hyperparameter selection may heavily decrease the image quality toward its degradation. Considering the lack of methods to efficiently determine these hyperparameters, this article presents a learning-based hyperparameter selection method for the CLAHE technique. The proposed supervised method was built and evaluated using contrast distortions from well-known image quality assessment datasets. Also, we introduce a more challenging dataset containing over 6200 images with a large range of contrast and intensity variations. The results show the efficiency of the proposed approach in predicting CLAHE hyperparameters with up to 0.014 RMSE and 0.935 R2 values. Also, our method overcomes both experimented baselines by enhancing image contrast while keeping its natural aspect. We propose a straightforward and efficient fusion-based method for enhancing weakly illumination images that uses several mature image processing techniques. First, we employ an illumination estimating algorithm based on morphological closing to decompose an observed image into a reflectance image and an illumination image. We then derive two inputs that represent luminance-improved and contrast-enhanced versions of the first decomposed illumination using the sigmoid function and adaptive histogram equalization. Designing two weights based on these inputs, we produce an adjusted illumination by fusing the derived inputs with the corresponding weights in a multi-scale fashion. Through a proper weighting and fusion strategy, we blend the advantages of different techniques to produce the adjusted illumination. The final enhanced image is obtained by compensating the adjusted illumination back to the reflectance. Through this synthesis, the enhanced image represents a trade-off among detail enhancement, local contrast improvement and preserving the natural feel of the image. In the proposed fusion-based framework, images under different weak illumination conditions such as backlighting, non-uniform illumination and nighttime can be enhanced. HighlightsA fusion-based method for enhancing various weakly illuminated images is proposed.The proposed method requires only one input to obtain the enhanced image.Different mature image processing techniques can be blended in our framework.Our method has an efficient computation time for practical applications. We propose a new high-quality and efficient single-image upscaling technique that extends existing example-based super-resolution frameworks. In our approach we do not rely on an external example database or use the whole input image as a source for example patches. Instead, we follow a local self-similarity assumption on natural images and extract patches from extremely localized regions in the input image. This allows us to reduce considerably the nearest-patch search time without compromising quality in most images. Tests, that we perform and report, show that the local self-similarity assumption holds better for small scaling factors where there are more example patches of greater relevance. We implement these small scalings using dedicated novel nondyadic filter banks, that we derive based on principles that model the upscaling process. Moreover, the new filters are nearly biorthogonal and hence produce high-resolution images that are highly consistent with the input image without solving implicit back-projection equations. The local and explicit nature of our algorithm makes it simple, efficient, and allows a trivial parallel implementation on a GPU. We demonstrate the new method ability to produce high-quality resolution enhancement, its application to video sequences with no algorithmic modification, and its efficiency to perform real-time enhancement of low-resolution video standard into recent high-definition formats. This paper presents a low-light image enhancement method using the variational-optimization-based Retinex algorithm. The proposed enhancement method first estimates the initial illumination and uses its gamma corrected version to constrain the illumination component. Next, the variational-based minimization is iteratively performed to separate the reflectance and illumination components. The color assignment of the estimated reflectance component is then performed to restore the color component using the input RGB color channels. Experimental results show that the proposed method can provide better enhanced result without saturation, noise amplification or color distortion. The first step in many techniques for processing intensity and saturation in color images keeping hue unaltered is the transformation of the image data from RGB space to other color spaces such as LHS, HSI, YIQ, HSV, etc. Transforming from one space to another and processing in these spaces usually generate a gamut problem, i.e., the values of the variables may not be in their respective intervals. We study enhancement techniques for color images theoretically in a generalized setup. A principle is suggested to make the transformations gamut-problem free. Using the same principle, a class of hue-preserving, contrast-enhancing transformations is proposed; they generalize existing grey scale contrast intensification techniques to color images. These transformations are also seen to bypass the above mentioned color coordinate transformations for image enhancement. The developed principle is used to generalize the histogram equalization scheme for grey scale images to color images. Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing textto- image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256.256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions. Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256x256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions. We address the problem of regional color transfer between two natural images by probabilistic segmentation. We use a new expectation-maximization (EM) scheme to impose both spatial and color smoothness to infer natural connectivity among pixels. Unlike previous work, our method takes local color information into consideration, and segment image with soft region boundaries for seamless color transfer and compositing. Our modified EM method has two advantages in color manipulation: first, subject to different levels of color smoothness in image space, our algorithm produces an optimal number of regions upon convergence, where the color statistics in each region can be adequately characterized by a component of a Gaussian mixture model (GMM). Second, we allow a pixel to fall in several regions according to our estimated probability distribution in the EM step, resulting in a transparency-like ratio for compositing different regions seamlessly. Hence, natural color transition across regions can be achieved, where the necessary intra-region and inter-region smoothness are enforced without losing original details. We demonstrate results on a variety of applications including image deblurring, enhanced color transfer, and colorizing gray scale images. Comparisons with previous methods are also presented. Degradation of images by the atmosphere often restricts imaging applications to good visibility conditions. For example, when imaging the terrain from a forward-looking airborne camera, the atmospheric degradation causes both a loss in contrast and color information. Enhancement of such images is a difficult task due to the complexity in restoring both the luminance and chrominance while maintaining good color fidelity. One particular problem is the fact that the level of contrast loss depends strongly on the wavelength; shorter wavelengths i.e., blue are more effected. In this paper, a novel method is presented for the enhancement of color images. This method is based on the underlying physics of the degradation and the parameters required for enhancement are estimated from the image itself. The proposed method is tested using synthetic images to explore the limitations and reliability of the method under different visibility conditions. Enhancement is performed on real images taken using an airborne camera at a height of approximately 1000 meters in hazy conditions for which the visibility is approximately 10 kilometers. Significant improvements in terms of contrast, visible range and color fidelity are evident when compared to existing methods. In this paper, we describe a technique to automatically enhance the perceptual quality of an image. Unlike previous techniques, where global statistics of the image are used to determine enhancement operation, our method is local and relies on local scene descriptors and context in addition to high-level image statistics. We cast the problem of image enhancement as searching for the best transformation for each pixel in the given image and then discovering the enhanced image using a formulation based on Gaussian Random Fields. The search is done in a coarse-to-fine manner, namely by finding the best candidate images, followed by pixels. Our experiments indicate that such context-based local enhancement is better than global enhancement schemes. A user study using Mechanical Turk shows that the subjects prefer contextual and local enhancements over the ones provided by existing schemes.