Real-world images usually contain vivid contents and rich textural details, which will complicate the manipulation on them. In this paper, we design a new framework based on content-aware synthesis to enhance content-aware image retargeting. By detecting the textural regions in an image, the textural image content can be synthesized rather than simply distorted or cropped. This method enables the manipulation of textural & non-textural regions with different strategy since they have different natures. We propose to retarget the textural regions by content-aware synthesis and non-textural regions by fast multi-operators. To achieve practical retargeting applications for general images, we develop an automatic and fast texture detection method that can detect multiple disjoint textural regions. We adjust the saliency of the image according to the features of the textural regions. To validate the proposed method, comparisons with state-of-the-art image targeting techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.
The adaptive integration of the color and texture attributes in image segmentation is one of the most investigated topics of research in computer vision (surveyed in @cite_39 ). However, most of the image segmentation algorithm do not clearly illustrate the type of each region (textural or non-textural) in the result. @cite_9 present a fast texture descriptor based on LU transform, but how to determine if a pixel is texture or non-texture according to the feature values is not discussed. @cite_5 present an intuitive texture detection method which is based on contrast and disorganization measurements of image blocks. The method is not effective on noisy images which tend to have decreasing contrast and often generate many disjoint areas. Todorovic and Ahuja @cite_38 formulate the detection of texture subimages as identifying modes of the pdf of region descriptors. However, the method is not efficient (5 minutes for a @math image) for practical image retargeting applications.
@cite_5 @cite_9 @cite_38 @cite_39
2024212834 1482080565 1909952827 2101789519 2119943055 2043331342 2017695766 2154020982 2085958611 2140233853 2157121948 2141376824 2144436897 2137276306 2125480112 2159693038 2067912884 2146228873 1931273051 2108976589
The adaptive integration of the colour and texture attributes in the development of complex image descriptors is one of the most investigated topics of research in computer vision. The substantial interest shown by the research community in colour-texture-based segmentation is mainly motivated by two factors. The first is related to the observation that the imaged objects are often described at perceptual level by distinctive colour and texture characteristics, while the second is motivated by the large spectrum of possible applications that can be addressed by the colour-texture integration in the segmentation process. Over the past three decades a substantial number of techniques in the field of colour-texture segmentation have been reported and it is the aim of this article to thoroughly evaluate and categorise the most relevant algorithms with respect to the modality behind the integration of these two fundamental image attributes. In this paper we also provide a detailed discussion about data collections, evaluation metrics and we review the performance attained by state of the art implementations. We conclude with a discussion that samples our views on the field of colour-texture image segmentation and this is complemented with an examination of the potential future directions of research. Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture at- tributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN substantially improves the state-of-the-art in texture, mate- rial and scene recognition. Our approach achieves 82.3 accuracy on Flickr material dataset and 81.1 accuracy on MIT indoor scenes, providing absolute gains of more than 10 over existing approaches. D-CNN easily trans- fers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at lo- calizing stuff categories and obtains state-of-the-art re- sults on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset. Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture attributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, FV-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. FV-CNN substantially improves the state-of-the-art in texture, material and scene recognition. Our approach achieves 79.8 accuracy on Flickr material dataset and 81 accuracy on MIT indoor scenes, providing absolute gains of more than 10 over existing approaches. FV-CNN easily transfers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, FV-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at localizing “stuff” categories and obtains state-of-the-art results on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset. In this paper, we present an efficient approach for unsupervised segmenta- tion of natural and textural images based on the extraction o fi mage features and a fast active contour segmentation model. We address the problem of textures where neither the gray-level information nor the boundary information is adequate for object extraction. This is often the case of natural images compose do f both homogeneous and textured regions. Because these images cannot be in generaldirectly processed by the gray-level information, we propose a new texture descripto rw hich intrinsically def ines the geometry of textures using semi-local image informatio na nd tools from differen- tial geometry. Then, we use the popular Kullback-Leibler distance to design an active contour model which distinguishes the background and textures of interest. The exis- tence of a minimizing solution to the proposed segmentationmodel is proven. Finally, at exture segmentation algorithm based on the Split-Bregma nm ethod is introduced to extract meaningful objects in a fast way. Promising synthetic and real-world results for gray-scale and color images are presented. In this work we approach the analysis and segmentation of natural textured images by combining ideas from image analysis and probabilistic modeling. We rely on AM-FM texture models and specifically on the Dominant Component Analysis (DCA) paradigm for feature extraction. This method provides a low-dimensional, dense and smooth descriptor, capturing essential aspects of texture, namely scale, orientation, and contrast. Our contributions are at three levels of the texture analysis and segmentation problems: First, at the feature extraction stage we propose a regularized demodulation algorithm that provides more robust texture features and explore the merits of modifying the channel selection criterion of DCA. Second, we propose a probabilistic interpretation of DCA and Gabor filtering in general, in terms of Local Generative Models. Extending this point of view to edge detection facilitates the estimation of posterior probabilities for the edge and texture classes. Third, we propose the weighted curve evolution scheme that enhances the Region Competition Geodesic Active Regions methods by allowing for the locally adaptive fusion of heterogeneous cues. Our segmentation results are evaluated on the Berkeley Segmentation Benchmark, and compare favorably to current state-of-the-art methods. Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments. We show that combining multiple texture description methods significantly improves the performance compared to using the single best texture method alone.We further propose to use information theoretic compression approach to compress high-dimensional multi-texture features into a compact heterogeneous texture representation.We perform a comprehensive evaluation of color features, popular in object recognition, for the task of texture classification.We show that late fusion of our compact texture descriptor with discriminative color feature outperforms state-of-the-art results on challenging texture recognition datasets. Describing textures is a challenging problem in computer vision and pattern recognition. The classification problem involves assigning a category label to the texture class it belongs to. Several factors such as variations in scale, illumination and viewpoint make the problem of texture description extremely challenging. A variety of histogram based texture representations exists in literature. However, combining multiple texture descriptors and assessing their complementarity is still an open research problem. In this paper, we first show that combining multiple local texture descriptors significantly improves the recognition performance compared to using a single best method alone. This gain in performance is achieved at the cost of high-dimensional final image representation. To counter this problem, we propose to use an information-theoretic compression technique to obtain a compact texture description without any significant loss in accuracy. In addition, we perform a comprehensive evaluation of pure color descriptors, popular in object recognition, for the problem of texture classification. Experiments are performed on four challenging texture datasets namely, KTH-TIPS-2a, KTH-TIPS-2b, FMD and Texture-10. The experiments clearly demonstrate that our proposed compact multi-texture approach outperforms the single best texture method alone. In all cases, discriminative color names outperforms other color features for texture classification. Finally, we show that combining discriminative color names with compact texture representation outperforms state-of-the-art methods by 7.8 , 4.3 and 5.0 on KTH-TIPS-2a, KTH-TIPS-2b and Texture-10 datasets respectively. This paper describes an approach to perceptual segmentation of color image textures. A multiscale representation of the texture image, generated by a multiband smoothing algorithm based on human psychophysical measurements of color appearance is used as the input. Initial segmentation is achieved by applying a clustering algorithm to the image at the coarsest level of smoothing. The segmented clusters are then restructured in order to isolate core clusters, i.e., patches in which the pixels are definitely associated with the same region. The image pixels representing the core clusters are used to form 3D color histograms which are then used for probabilistic assignment of all other pixels to the core clusters to form larger clusters and categorise the rest of the image. The process of setting up color histograms and probabilistic reassignment of the pixels to the clusters is then propagated through finer levels of smoothing until a full segmentation is achieved at the highest level of resolution. This study presents a method for resampling detection. By combining texture analysis with resampling detection, the task of resampling detection is considered as a texture classification problem. In other words, the influence of resampling operations on a raw single-sampled image is viewed as an alteration of the image texture in a fine scale. First, local linear transform is used to obtain textural detail sub-bands. A 36-D feature vector is then extracted from the normalized characteristic function moments of textural detail sub-bands to train a support vector machine classifier. Finally, experimental results are reported on three databases, with each having almost 10,000 images. Comparison with the previous study reveals that the proposed method is effective for resampling detection. In addition, extensive experiments on cover and stego bitmap images illustrate that the proposed method is essential for constructing accurate targeted and blind steganalysis methods for heterogeneous images, raw single-sampled images, and images resampled at different scales. A robust and efficient anomaly detection technique is proposed, capable of dealing with crowded scenes where traditional tracking based approaches tend to fail. Initial foreground segmentation of the input frames confines the analysis to foreground objects and effectively ignores irrelevant background dynamics. Input frames are split into non-overlapping cells, followed by extracting features based on motion, size and texture from each cell. Each feature type is independently analysed for the presence of an anomaly. Unlike most methods, a refined estimate of object motion is achieved by computing the optical flow of only the foreground pixels. The motion and size features are modelled by an approximated version of kernel density estimation, which is computationally efficient even for large training datasets. Texture features are modelled by an adaptively grown code-book, with the number of entries in the codebook selected in an online fashion. Experiments on the recently published UCSD Anomaly Detection dataset show that the proposed method obtains considerably better results than three recent approaches: MPPCA, social force, and mixture of dynamic textures (MDT). The proposed method is also several orders of magnitude faster than MDT, the next best performing method. This paper investigates variational region-level criterion for supervised and unsupervised texture-based image segmentation. The focus is given to the demonstration of the effectiveness and robustness of this region-based formulation compared to most common variational approaches. The main contributions of this global criterion are twofold. First, the proposed methods circumvent a major problem related to classical texture based segmentation approaches. Existing methods, even if they use different and various texture features, are mainly stated as the optimization of a criterion evaluating punctual pixel likelihoods or similarity measure computed within a local neighborhood. These approaches require sufficient dissimilarity between the considered texture features. An additional limitation is the choice of the neighborhood size and shape. These two parameters and especially the neighborhood size significantly influence the classification performances: the neighborhood must be large enough to capture texture structures and small enough to guarantee segmentation accuracy. These parameters are often set experimentally. These limitations are mitigated with the proposed variational methods stated at the region-level. It resorts to an energy criterion defined on image where regions are characterized by nonparametric distributions of their responses to a set of filters. In the supervised case, the segmentation algorithm consists in the minimization of a similarity measure between region-level statistics and texture prototypes and a boundary based functional that imposes smoothness and regularity on region boundaries. In the unsupervised case, the data-driven term involves the maximization of the dissimilarity between regions. The proposed similarity measure is generic and permits optimally fusing various types of texture features. It is defined as a weighted sum of Kullback-Leibler divergences between feature distributions. The optimization of the proposed variational criteria is carried out using a level-set formulation. The effectiveness and the robustness of this formulation at region-level, compared to classical active contour methods, are evaluated for various Brodatz and natural images. This paper provides an algorithm for partitioning grayscale images into disjoint regions of coherent brightness and texture. Natural images contain both textured and untextured regions, so the cues of contour and texture differences are exploited simultaneously. Contours are treated in the intervening contour framework, while texture is analyzed using textons. Each of these cues has a domain of applicability, so to facilitate cue combination we introduce a gating operator based on the texturedness of the neighborhood at a pixel. Having obtained a local measure of how likely two nearby pixels are to belong to the same region, we use the spectral graph theoretic framework of normalized cuts to find partitions of the image into regions of coherent texture and brightness. Experimental results on a wide range of images are shown. High correlation among the neighboring pixels both spatially and spectrally in a multispectral image makes it necessary to use an efficient data transformation approach before performing pan-sharpening. Wavelets and principal component analysis (PCA) methods have been a popular choice for spatial and spectral transformations, respectively. Current PCA-based pan-sharpening methods make an assumption that the first principal component (PC) of high variance is an ideal choice for replacing or injecting it with high spatial details from the high-resolution histogram-matched panchromatic (PAN) image. This paper presents a combined adaptive PCA-contourlet approach for pan-sharpening, where the adaptive PCA is used to reduce the spectral distortion and the use of nonsubsampled contourlets for spatial transformation in pan-sharpening is incorporated to overcome the limitation of the wavelets in representing the directional information efficiently and capturing intrinsic geometrical structures of the objects. The efficiency of the presented method is tested by performing pan-sharpening of the high-resolution (IKONOS and QuickBird) and the medium-resolution (Landsat-7 Enhanced Thematic Mapper Plus) datasets. The evaluation of the pan-sharpened images using global validation indexes reveal that the adaptive PCA approach helps reducing the spectral distortion, and its merger with contourlets provides better fusion results. The paper studies the problem of combining region and boundary cues for natural image segmentation. We employ a large database of manually segmented images in order to learn an optimal affinity function between pairs of pixels. These pairwise affinities can then be used to cluster the pixels into visually coherent groups. Region cues are computed as the similarity in brightness, color, and texture between image patches. Boundary cues are incorporated by looking for the presence of an "intervening contour", a large gradient along a straight line connecting two pixels. We first use the dataset of human segmentations to individually optimize parameters of the patch and gradient features for brightness, color, and texture cues. We then quantitatively measure the power of different feature combinations by computing the precision and recall of classifiers trained using those features. The mutual information between the output of the classifiers and the same-segment indicator function provides an alternative evaluation technique that yields identical conclusions. As expected, the best classifier makes use of brightness, color, and texture features, in both patch and gradient forms. We find that for brightness, the gradient cue outperforms the patch similarity. In contrast, using color patch similarity yields better results than using color gradients. Texture is the most powerful of the three channels, with both patches and gradients carrying significant independent information. Interestingly, the proximity of the two pixels does not add any information beyond that provided by the similarity cues. We also find that the convexity assumptions made by the intervening contour approach are supported by the ecological statistics of the dataset. This paper presents an algorithm capable of real-time separation of foreground from background in monocular video sequences. Automatic segmentation of layers from colour contrast or from motion alone is known to be error-prone. Here motion, colour and contrast cues are probabilistically fused together with spatial and temporal priors to infer layers accurately and efficiently. Central to our algorithm is the fact that pixel velocities are not needed, thus removing the need for optical flow estimation, with its tendency to error and computational expense. Instead, an efficient motion vs nonmotion classifier is trained to operate directly and jointly on intensity-change and contrast. Its output is then fused with colour information. The prior on segmentation is represented by a second order, temporal, Hidden Markov Model, together with a spatial MRF favouring coherence except where contrast is high. Finally, accurate layer segmentation and explicit occlusion detection are efficiently achieved by binary graph cut. The segmentation accuracy of the proposed algorithm is quantitatively evaluated with respect to existing groundtruth data and found to be comparable to the accuracy of a state of the art stereo segmentation algorithm. Foreground background segmentation is demonstrated in the application of live background substitution and shown to generate convincingly good quality composite video. We present a robust image synthesis method to automatically infer missing information from a damaged 2D image by tensor voting. Our method translates image color and texture information into an adaptive ND tensor, followed by a voting process that infers non-iteratively the optimal color values in the ND texture space for each defective pixel. ND tensor voting can be applied to images consisting of roughly homogeneous and periodic textures (e.g. a brick wall), as well as difficult images of natural scenes, which contain complex color and texture information. To effectively tackle the latter type of difficult images, a two-step method is proposed. First, we perform texture-based segmentation in the input image, and extrapolate partitioning curves to generate a complete segmentation for the image. Then, missing colors are synthesized using ND tensor voting. Automatic tensor scale analysis is used to adapt to different feature scales inherent in the input. We demonstrate the effectiveness of our approach using a difficult set of real images. We address the problems of contour detection, bottom-up grouping and semantic segmentation using RGB-D data. We focus on the challenging setting of cluttered indoor scenes, and evaluate our approach on the recently introduced NYU-Depth V2 (NYUD2) dataset [27]. We propose algorithms for object boundary detection and hierarchical segmentation that generalize the gPb-ucm approach of [2] by making effective use of depth information. We show that our system can label each contour with its type (depth, normal or albedo). We also propose a generic method for long-range amodal completion of surfaces and show its effectiveness in grouping. We then turn to the problem of semantic segmentation and propose a simple approach that classifies super pixels into the 40 dominant object categories in NYUD2. We use both generic and class-specific features to encode the appearance and geometry of objects. We also show how our approach can be used for scene classification, and how this contextual information in turn improves object recognition. In all of these tasks, we report significant improvements over the state-of-the-art. A robust synthesis method is proposed to automatically infer missing color and texture information from a damaged 2D image by ND tensor voting (N > 3). The same approach is generalized to range and 3D data in the presence of occlusion, missing data and noise. Our method translates texture information into an adaptive ND tensor, followed by a voting process that infers noniteratively the optimal color values in the ND texture space. A two-step method is proposed. First, we perform segmentation based on insufficient geometry, color, and texture information in the input, and extrapolate partitioning boundaries by either 2D or 3D tensor voting to generate a complete segmentation for the input. Missing colors are synthesized using ND tensor voting in each segment. Different feature scales in the input are automatically adapted by our tensor scale analysis. Results on a variety of difficult inputs demonstrate the effectiveness of our tensor voting approach. We propose new dense descriptors for texture segmentation. Given a region of arbitrary shape in an image, these descriptors are formed from shape-dependent scale spaces of oriented gradients. These scale spaces are defined by Poisson-like partial differential equations. A key property of our new descriptors is that they do not aggregate image data across the boundary of the region, in contrast to existing descriptors based on aggregation of oriented gradients. As an example, we show how the descriptor can be incorporated in a Mumford-Shah energy for texture segmentation. We test our method on several challenging datasets for texture segmentation and textured object tracking. Experiments indicate that our descriptors lead to more accurate segmentation than non-shape dependent descriptors and the state-of-the-art in texture segmentation. Texture segmentation is a difficult problem, as is apparent from camouflage pictures. A textured region can contain texture elements of various sizes, each of which can itself be textured. We approach this problem using a bottom-up aggregation framework that combines structural characteristics of texture elements with filter responses. Our process adaptively identifies the shape of texture elements and characterize them by their size, aspect ratio, orientation, brightness, etc., and then uses various statistics of these properties to distinguish between different textures. At the same time our process uses the statistics of filter responses to characterize textures. In our process the shape measures and the filter responses crosstalk extensively. In addition, a top-down cleaning process is applied to avoid mixing the statistics of neighboring segments. We tested our algorithm on real images and demonstrate that it can accurately segment regions that contain challenging textures.