The typical content-based image retrieval problem is to find images within a database that are similar to a given query image. This paper presents a solution to a different problem, namely that of content based sub-image retrieval, i.e., finding images from a database that contains another image. Note that this is different from finding a region in a (segmented) image that is similar to another image region given as a query. We present a technique for CBsIR that explores relevance feedback, i.e., the user's input on intermediary results, in order to improve retrieval efficiency. Upon modeling images as a set of overlapping and recursive tiles, we use a tile re-weighting scheme that assigns penalties to each tile of the database images and updates the tile penalties for all relevant images retrieved at each iteration using both the relevant and irrelevant images identified by the user. Each tile is modeled by means of its color content using a compact but very efficient method which can, indirectly, capture some notion of texture as well, despite the fact that only color information is maintained. Performance evaluation on a largely heterogeneous dataset of over 10,000 images shows that the system can achieve a stable average recall value of 70 within the top 20 retrieved (and presented) images after only 5 iterations, with each such iteration taking about 2 seconds on an off-the-shelf desktop computer.
An straigthforward way to model images is to use its average color. This is obviously not effective in any non-trivial situation. Another simple, and in many situations cost-effective means is to use a global color histogram (GCH) (c.f., @cite_14 ). A common critique to GCHs is that it is unable to capture any notion of spatial distribution. To address this several other approaches have been proposed A comprehensive survey thereof is beyond the scope of this paper. , but they add complexity as a trade-off in order to gain effectiveness. Nevertheless, the use of color only, without any notion of spatial distribution, may be effective, if one is able to capture other features of the images, e.g., texture. That is exactly the advantage of the BIC technique proposed in @cite_0 and which we re-use within our proposal.
@cite_0 @cite_14
1841033533 2565651106 1626352492 2145268773 1982616653 2464766066 2029731618 2138467836 2080955644 2416791088 1513768190 2128059302 2163352848 2039388884 2101789519 2088828956 2616247523 1753871439 1762020163 2129099426
We present new methods which transfer the color style of a source image into an arbitrary given target image having a different 3D color distribution. The color transfer has a high importance ensuring a wide area of applications from artistic transformation of the color atmosphere of images until different scientific visualizations using special gamut mappings. Our technique use a permissive, or optionally strict, 3D histogram matching, similarly to the sampling of multivariable functions applying a sequential chain of conditional probability density functions. We work by order of hue, hue dependent lightness and from both dependent saturation histograms of source and target images, respectively. We apply different histogram transformations, like smoothing or contrast limitation, in order to avoid some unexpected high gradients and other artifacts. Furthermore, we use dominance suppression optionally, by applying sub-linear functions for the histograms in order to get well balanced color distributions, or an overall appearance reflecting the memory color distribution better. Forward and inverse operations on the corresponding cumulative histograms ensure a continuous mapping of perceptual attributes correlating to limited derivatives. Sampling an appropriate fraction of the pixels and using perceptually accurate and continuous histograms with minimal size as well as other gems make this method robust and fast. In this paper, we present a color consistency technique in order to make images in the same collection share the same color style and to avoid gamut problems. Some previous methods define simple global parameter-based models and use optimizing algorithms to obtain the unknown parameters, which usually cause gamut problems in bright and dark regions. Our method is based on the range-preserving histogram specification and can enforce images to share the same color style, without resulting in gamut problems. We divide the input images into two sets having respectively high visual quality and low visual quality. The high visual quality images are used to make color balance. And then the low visual quality images are color transferred using the previous corrected high quality images. Our experiments indicate that such histogram-based color correction method is better than the compared algorithm. We proposed a computational visual saliency modeling technique. The proposed technique makes use of a color co-occurrence histogram (CCH) that captures not only "how many" but also "where and how" image pixels are composed into a visually perceivable image. Hence the CCH encodes image saliency information that is usually perceived as the discontinuity between an image region or object and its surrounding. The proposed technique has a number of distinctive characteristics: It is fast, discriminative, tolerant to image scale variation, and involves minimal parameter tuning. Experiments over benchmarking datasets show that it predicts fixational eye tracking points accurately and a superior AUC of 71.25 is obtained. A novel color image histogram equalization approach is proposed that exploits the correlation between color components and it is enhanced by a multi-level smoothing technique borrowed from statistical language engineering. Multi-level smoothing aims at dealing efficiently with the problem of unseen color values, either considered independently or in combination with others. It is applied here to the HSI color space for the probability of intensity and the probability of saturation given the intensity, while the hue is left unchanged. Moreover, the proposed approach is extended by an empirical technique, which is based on a hue preserving non-linear transformation, in order to eliminate the gamut problem. This is the second method proposed in the paper. The equalized images by the two methods are compared to those produced by other well-known methods. The better quality of the images equalized by the proposed methods is judged in terms of their visual appeal and objective figures of merit, such as the entropy and the Kullback-Leibler divergence estimates between the resulting color histogram and the multivariate uniform probability density function. Color histograms lack spatial information and are sensitive to intensity variation, color distortion and cropping. As a result, images with similar histograms may have totally different semantics. The region-based approaches are introduced to overcome the above limitations, but due to the inaccurate segmentation, these systems may partition an object into several regions that may have confused users in selecting the proper regions. In this paper, we present a robust image retrieval based on color histogram of local feature regions (LFR). Firstly, the steady image feature points are extracted by using multi-scale Harris-Laplace detector. Then, the significant local feature regions are ascertained adaptively according to the feature scale theory. Finally, the color histogram of local feature regions is constructed, and the similarity between color images is computed by using the color histogram of LFRs. Experimental results show that the proposed color image retrieval is more accurate and efficient in retrieving the user-interested images. Especially, it is robust to some classic transformations (additive noise, affine transformation including translation, rotation and scale effects, partial visibility, etc.). The aim of this paper is twofold. First, we propose a new method for enhancing the contrast of gray-value images. We use the difference of the average local contrast measures between the original and the enhanced images within a variational framework. This enables the user to intuitively control the contrast level and the scale of the enhanced details. Moreover, our model avoids large modifications of the original image histogram. Thereby it preserves the global illumination of the scene and it can cope with large areas having similar gray values. The minimizer of the proposed functional is computed by a gradient descent algorithm in connection with a polynomial approximation of the average local contrast measure. The polynomial approximation is computed via Bernstein polynomials. In the second part, the approach is extended to a variational enhancement method for color images. The model approximately preserves the hue of the original image and additionally includes a total variation term to correct the possible noise. The method requires no post- or preprocessing. The minimization problem is solved with a hybrid primal---dual algorithm. Experiments demonstrate the efficiency and the flexibility of the proposed approaches in comparison with state-of-the-art methods. We address the task of learning a semantic segmentation from weakly supervised data. Our aim is to devise a system that predicts an object label for each pixel by making use of only image level labels during training – the information whether a certain object is present or not in the image. Such coarse tagging of images is faster and easier to obtain as opposed to the tedious task of pixelwise labeling required in state of the art systems. We cast this task naturally as a multiple instance learning (MIL) problem. We use Semantic Texton Forest (STF) as the basic framework and extend it for the MIL setting. We make use of multitask learning (MTL) to regularize our solution. Here, an external task of geometric context estimation is used to improve on the task of semantic segmentation. We report experimental results on the MSRC21 and the very challenging VOC2007 datasets. On MSRC21 dataset we are able, by using 276 weakly labeled images, to achieve the performance of a supervised STF trained on pixelwise labeled training set of 56 images, which is a significant reduction in supervision needed. In this paper we present a process called color transfer which can borrow one image's color characteristics from another. Recently Reinhard and his colleagues reported a pioneering work of color transfer. Their technology can produce very believable results, but has to transform pixel values from RGB to lαβ. Inspired by their work, we advise an approach which can directly deal with the color transfer in any 3D space.From the view of statistics, we consider pixel's value as a three-dimension stochastic variable and an image as a set of samples, so the correlations between three components can be measured by covariance. Our method imports covariance between three components of pixel values while calculate the mean along each of the three axes. Then we decompose the covariance matrix using SVD algorithm and get a rotation matrix. Finally we can scale, rotate and shift pixel data of target image to fit data points' cluster of source image in the current color space and get resultant image which takes on source image's look and feel. Besides the global processing, a swatch-based method is introduced in order to manipulate images' color more elaborately. Experimental results confirm the validity and usefulness of our method. This paper presents a novel image feature representation method, namely color difference histograms (CDH), for image retrieval. This method is entirely different from the existing histograms; most of the existing histogram techniques merely count the number or frequency of pixels. However, the unique characteristic of CDHs is that they count the perceptually uniform color difference between two points under different backgrounds with regard to colors and edge orientations in L*a*b* color space. This method pays more attention to color, edge orientation and perceptually uniform color differences, and encodes color, orientation and perceptually uniform color difference via feature representation in a similar manner to the human visual system. The method can be considered as a novel visual attribute descriptor combining edge orientation, color and perceptually uniform color difference, as well as taking the spatial layout into account without any image segmentation, learning processes or clustering implementation. Experimental results demonstrate that it is much more efficient than the existing image feature descriptors that were originally developed for content-based image retrieval, such as MPEG-7 edge histogram descriptors, color autocorrelograms and multi-texton histograms. It has a strong discriminative power using the color, texture and shape features while accounting for spatial layout. In this paper, we explore various aspects of fusing LIDAR and color imagery for pedestrian detection in the context of convolutional neural networks (CNNs), which have recently become state-of-art for many vision problems. We incorporate LIDAR by up-sampling the point cloud to a dense depth map and then extracting three features representing different aspects of the 3D scene. We then use those features as extra image channels. Specifically, we leverage recent work on HHA [9] (horizontal disparity, height above ground, and angle) representations, adapting the code to work on up-sampled LIDAR rather than Microsoft Kinect depth maps. We show, for the first time, that such a representation is applicable to up-sampled LIDAR data, despite its sparsity. Since CNNs learn a deep hierarchy of feature representations, we then explore the question: At what level of representation should we fuse this additional information with the original RGB image channels? We use the KITTI pedestrian detection dataset for our exploration. We first replicate the finding that region-CNNs (R-CNNs) [8] can outperform the original proposal mechanism using only RGB images, but only if fine-tuning is employed. Then, we show that: 1) using HHA features and RGB images performs better than RGB-only, even without any fine-tuning using large RGB web data, 2) fusing RGB and HHA achieves the strongest results if done late, but, under a parameter or computational budget, is best done at the early to middle layers of the hierarchical representation, which tend to represent midlevel features rather than low (e.g. edges) or high (e.g. object class decision) level features, 3) some of the less successful methods have the most parameters, indicating that increased classification accuracy is not simply a function of increased capacity in the neural network. Color-based trackers recently proposed in [3,4,5] have been proved robust and versatile for a modest computational cost. They are especially appealing for tracking tasks where the spatial structure of the tracked objects exhibits such a dramatic variability that trackers based on a space-dependent appearance reference would break down very fast. Trackers in [3,4,5] rely on the deterministic search of a window whose color content matches a reference histogram color model.Relying on the same principle of color histogram distance, but within a probabilistic framework, we introduce a new Monte Carlo tracking technique. The use of a particle filter allows us to better handle color clutter in the background, as well as complete occlusion of the tracked entities over a few frames.This probabilistic approach is very flexible and can be extended in a number of useful ways. In particular, we introduce the following ingredients: multi-part color modeling to capture a rough spatial layout ignored by global histograms, incorporation of a background color model when relevant, and extension to multiple objects. We introduce an efficient maximum likelihood approach for one part of the color constancy problem: removing from an image the color cast caused by the spectral distribution of the dominating scene illuminant. We do this by developing a statistical model for the spatial distribution of colors in white balanced images (i.e., those that have no color cast), and then using this model to infer illumination parameters as those being most likely under our model. The key observation is that by applying spatial band-pass filters to color images one unveils color distributions that are unimodal, symmetric, and well represented by a simple parametric form. Once these distributions are fit to training data, they enable efficient maximum likelihood estimation of the dominant illuminant in a new image, and they can be combined with statistical prior information about the illuminant in a very natural manner. Experimental evaluation on standard data sets suggests that the approach performs well. Presents a theoretically very simple, yet efficient, multiresolution approach to gray-scale and rotation invariant texture classification based on local binary patterns and nonparametric discrimination of sample and prototype distributions. The method is based on recognizing that certain local binary patterns, termed "uniform," are fundamental properties of local image texture and their occurrence histogram is proven to be a very powerful texture feature. We derive a generalized gray-scale and rotation invariant operator presentation that allows for detecting the "uniform" patterns for any quantization of the angular space and for any spatial resolution and presents a method for combining multiple operators for multiresolution analysis. The proposed approach is very robust in terms of gray-scale variations since the operator is, by definition, invariant against any monotonic transformation of the gray scale. Another advantage is computational simplicity as the operator can be realized with a few operations in a small neighborhood and a lookup table. Experimental results demonstrate that good discrimination can be achieved with the occurrence statistics of simple rotation invariant local binary patterns. This paper presents (Border Interior pixel Classification), a compact and efficient CBIR approach suitable for broad image domains. It has three main components: (1) a simple and powerful image analysis algorithm that classifies image pixels as either border or interior, (2) a new logarithmic distance (dLog) for comparing histograms, and (3) a compact representation for the visual features extracted from images. Experimental results show that the BIC approach is consistently more compact, more efficient and more effective than state-of-the-art CBIR approaches based on sophisticated image analysis algorithms and complex distance functions. It was also observed that the dLog distance function has two main advantages over vectorial distances (e.g., L 1 ): (1) it is able to increase substantially the effectiveness of (several) histogram-based CBIR approaches and, at the same time, (2) it reduces by 50 the space requirement to represent a histogram. In this paper, we present an efficient approach for unsupervised segmenta- tion of natural and textural images based on the extraction o fi mage features and a fast active contour segmentation model. We address the problem of textures where neither the gray-level information nor the boundary information is adequate for object extraction. This is often the case of natural images compose do f both homogeneous and textured regions. Because these images cannot be in generaldirectly processed by the gray-level information, we propose a new texture descripto rw hich intrinsically def ines the geometry of textures using semi-local image informatio na nd tools from differen- tial geometry. Then, we use the popular Kullback-Leibler distance to design an active contour model which distinguishes the background and textures of interest. The exis- tence of a minimizing solution to the proposed segmentationmodel is proven. Finally, at exture segmentation algorithm based on the Split-Bregma nm ethod is introduced to extract meaningful objects in a fast way. Promising synthetic and real-world results for gray-scale and color images are presented. Color constancy is a well-studied topic in color vision. Methods are generally categorized as (1) low-level statistical methods, (2) gamut-based methods, and (3) learning-based methods. In this work, we distinguish methods depending on whether they work directly from color values (i.e., color domain) or from values obtained from the image’s spatial information (e.g., image gradients frequencies). We show that spatial information does not provide any additional information that cannot be obtained directly from the color distribution and that the indirect aim of spatial-domain methods is to obtain large color differences for estimating the illumination direction. This finding allows us to develop a simple and efficient illumination estimation method that chooses bright and dark pixels using a projection distance in the color distribution and then applies principal component analysis to estimate the illumination direction. Our method gives state-of-the-art results on existing public color constancy datasets as well as on our newly collected dataset (NUS dataset) containing 1736 images from eight different high-end consumer cameras. We propose a technique for producing "visual explanations" for decisions from a large class of CNN-based models, making them more transparent. Our approach - Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept, flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, GradCAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multimodal inputs (e.g. VQA) or reinforcement learning, without any architectural changes or re-training. We combine GradCAM with fine-grained visualizations to create a high-resolution class-discriminative visualization and apply it to off-the-shelf image classification, captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into their failure modes (showing that seemingly unreasonable predictions have reasonable explanations), (b) are robust to adversarial images, (c) outperform previous methods on weakly-supervised localization, (d) are more faithful to the underlying model and (e) help achieve generalization by identifying dataset bias. For captioning and VQA, our visualizations show that even non-attention based models can localize inputs. Finally, we conduct human studies to measure if GradCAM explanations help users establish trust in predictions from deep networks and show that GradCAM helps untrained users successfully discern a "stronger" deep network from a "weaker" one. Our code is available at this https URL A demo and a video of the demo can be found at this http URL and youtu.be COjUB9Izk6E. The idea of using statistical inference for analyzing and understanding images has been used for at least 20 years, going back, for instance, to the work of Grenander [Gr] and Cooper [Co]. To apply these techniques, one needs, of course, a probabilistic model for some class of images or some class of structures present in images. Many models of this type have been introduced. There are stochastic models for image textures [GGGD], [ZMW], for contours in images [Mu], [GCK], for the decomposition of an image into regions [G-G], [M-S], for disparity maps, for grammatical parsing of shapes [Fu], for template matching, for speci c tasks such as face recognition [HGYGM]. The common framework for all these studies is to describe some class of images I(x; y) by means of a set of auxiliary variables fx g representing the salient structures in the images, e.g. edges, texture statistics, inferred depth values or relations, illumination features, medial axes or shape features, locations of key points such as eyes in a face, labels (as in character recognition), etc. Then i) a prior probability model for the hidden' variables p(fx g) and ii) an imagingmodel p(Ijfx g) for I, given the hidden variables, are de ned. Finally, an image is analyzed using Bayes's rule p(fx gjI) p(Ijfx g)p(fx g) In addition to the problem of which image analysis models to use in digital libraries, e.g. wavelet, Wold, color histograms, is the problem of how to combine these models with their different strengths. Most present systems place the burden of combination on the user, e.g. the user specifies 50 texture features, 20 color features, etc. This is a problem since most users do not know how to best pick the settings for the given data and search problem. The paper addresses this problem, describing research in progress for a system that: (1) automatically infers which combination of models best represents the data of interest to the user; and (2) learns continuously during interaction with each user. In particular, these two components-inference and learning-provide a solution that adapts to the subjective and hard to predict behaviors frequently seen when people query or browse image libraries. This paper presents a novel approach to face recognition, named Multi-resolution Histograms of Local Variation Patterns (MHLVP), in which face images are represented as the concatenation of the local spatial histogram of local variation patterns computed from the multi-resolution Gabor features. For a face image with abundant texture and shape information, aGabor feature map(GFM) is computed by convolving the image with each of the forty multi-scale and multi-orientation Gabor filters. Each GFM is then divided into small non-overlapping regions to enhance its shape information, and then Local Binary Pattern (LBP) histograms are extracted for each region and concatenated into a feature histogram to enhance the texture information in the specific GFM. Further more, all of the feature histograms extracted from the forty GFMs are further concatenated into a single feature histogram as the final representation of the given face image. Eventually, the identification is achieved by histogram intersection operation. Our experimental results on FERET face databases show that the proposed method performs terrifically better than the performance of some classical results including the best results in FERET'97.