Real-world images usually contain vivid contents and rich textural details, which will complicate the manipulation on them. In this paper, we design a new framework based on content-aware synthesis to enhance content-aware image retargeting. By detecting the textural regions in an image, the textural image content can be synthesized rather than simply distorted or cropped. This method enables the manipulation of textural & non-textural regions with different strategy since they have different natures. We propose to retarget the textural regions by content-aware synthesis and non-textural regions by fast multi-operators. To achieve practical retargeting applications for general images, we develop an automatic and fast texture detection method that can detect multiple disjoint textural regions. We adjust the saliency of the image according to the features of the textural regions. To validate the proposed method, comparisons with state-of-the-art image targeting techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.
Numerous content-aware image retargeting techniques have recently been proposed. Cropping has been widely used to eliminate the unimportant information from the image periphery or improve the overall composition @cite_35 @cite_40 @cite_21 . Seam carving methods iteratively remove a seam in the input image to preserve visually salient content @cite_13 @cite_3 . A seam is a continuous path with minimum significance. Multi-operator algorithms combine seam carving, homogeneous scaling and cropping to optimally resize images @cite_43 @cite_14 @cite_28 . @cite_48 introduced Shift-Map that removed or added band regions instead of scaling or stretching images. For many cases these approaches can generate pleasing results, however, the seam removal may cause discontinuous artifacts, and cropping is unsuitable for the case when there are visually salient contents near the borders of images.
@cite_35 @cite_14 @cite_28 @cite_48 @cite_21 @cite_3 @cite_43 @cite_40 @cite_13
2167322721 2132610780 2021448594 74030613 2005619762 2018377917 2043331342 1606751047 2091089676 1972735298 2749488618 2127079338 1977904760 2790280905 2009874829 2407857922 2129688096 2015038178 2518979500 1573739146
We present a novel method for content-aware image resizing based on optimization of a well-defined image distance function, which preserves both the important regions and the global visual effect (the background or other decorative objects) of an image. The method operates by joint use of seam carving and image scaling. The principle behind our method is the use of a bidirectional similarity function of image Euclidean distance (IMED), while cooperating with a dominant color descriptor (DCD) similarity and seam energy variation. The function is suitable for the quantitative evaluation of the resizing result and the determination of the best seam carving number. Different from the previous simplex-mode approaches, our method takes the advantages of both discrete and continuous methods. The technique is useful in image resizing for both reduction retargeting and enlarging. We also show that this approach can be extended to indirect image resizing. Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes. In the new formulation, a seam is given by a minimal cut in the graph and we show how to construct a graph such that the resulting cut is a valid seam. That is, the cut is monotonic and connected. In addition, we present a novel energy criterion that improves the visual quality of the retargeted images and videos. The original seam carving operator is focused on removing seams with the least amount of energy, ignoring energy that is introduced into the images and video by applying the operator. To counter this, the new criterion is looking forward in time - removing seams that introduce the least amount of energy into the retargeted result. We show how to encode the improved criterion into graph cuts (for images and video) as well as dynamic programming (for images). We apply our technique to images and videos and present results of various applications. Traditional image resizing techniques are oblivious to the content of the image when changing its width or height. In contrast, media (i.e., image and video) retargeting take s content into account. For example, one would like to change the aspect ratio of a video without making human figures look too fat or too skinny, or change the size of an image by automatically removing "unnecessary" portions while keeping the "important" features intact. We propose a simple operator; we term seam carving to support image and video retargeting. A seam is an optimal 1D path of pixels in an image, or a 2D manifold in a video cube, going from top to bottom, or left to right. Optimality is defined by minimizing an energy function that assigns costs to pixels. We show that computing a seam reduces to a dynamic programming problem for images and a graph min-cut search for video. We demonstrate that several image and video operations, such as aspect ratio correction, size change, and object removal, can be recast as a successive operation of the seam carving operator. Image retargeting algorithms aim to adapt the image to the display screen with the goal of preserving the image content as much as possible. However, existing methods and research efforts have mostly been directed towards retargeting algorithms that retarget images to rectangular domains. This significantly hampers its application to broader classes of display devices and platforms for which the display area can be of any origins and shapes. For example, seam carving-based methods retarget images by carving out seams that run from the top to the bottom of the images, and this results in changing the width and therefore aspect ratio of the image without changing the shape of the image boundary in any essential way. However, by carving out appropriately-chosen seam segments, seams that are not required to cut across the entire image, it is then possible to retarget the images to a broader array of image domains with non-rectangular boundaries. Based on this simple idea of carving out the seam segments, the main contribution of this paper is a novel image retargeting algorithm that is capable of retargeting images to non-rectangular domains. We evaluate the effectiveness of the proposed method on a number of challenging indoor and outdoor scene images, and the results demonstrate that the proposed algorithm is both efficient and effective, and it is capable of providing good-quality retargeted images for a variety of interesting boundary shapes. In this paper, we propose a novel image retargeting algorithm to resize images based on the extracted saliency information from the compressed domain. Firstly, we utilize DCT coefficients in JPEG bitstream to perform saliency detection with the consideration of the human visual sensitivity. The obtained saliency information is used to determine the relative visual importance of each 8 x 8 block for the image. Furthermore, we propose a new adaptive block-level seam removal operation for connected blocks to resize the image. Thanks to the directly derived saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the objects of attention, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the careful analysis and in the extensive experiments. Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction we can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create multi-size images, that are able to continuously change in real time to fit a given size. Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments. We describe an algorithm for generating panoramic video from unstructured camera arrays. Artifact-free panorama stitching is impeded by parallax between input views. Common strategies such as multi-level blending or minimum energy seams produce seamless results on quasi-static input. However, on video input these approaches introduce noticeable visual artifacts due to lack of global temporal and spatial coherence. In this paper we extend the basic concept of local warping for parallax removal. Firstly, we introduce an error measure with increased sensitivity to stitching artifacts in regions with pronounced structure. Using this measure, our method efficiently finds an optimal ordering of pair-wise warps for robust stitching with minimal parallax artifacts. Weighted extrapolation of warps in non-overlap regions ensures temporal stability, while at the same time avoiding visual discontinuities around transitions between views. Remaining global deformation introduced by the warps is spread over the entire panorama domain using constrained relaxation, while staying as close as possible to the original input views. In combination, these contributions form the first system for spatiotemporally stable panoramic video stitching from unstructured camera array input. Current multi-operator image resizing methods succeeding in generating impressive results by using image similarity measure to guide the resizing process. An optimal operation path is found in the resizing space. However, their slow resizing speed caused by inefficient computation strategy of the bidirectional patch matching becomes a drawback in practical use. In this paper, we present a novel method to address this problem. By combining seam carving with scaling and cropping, our method can realize content-aware image resizing very fast. We define cost functions combing image energy and dominant color descriptor for all the operators to evaluate the damage to both local image content and global visual effect. Therefore our algorithm can automatically find an optimal sequence of operations to resize the image by using dynamic programming or greedy algorithm. We also extend our algorithm to indirect image resizing which can protect the aspect ratio of the dominant object in an image. We present a "scale-and-stretch" warping method that allows resizing images into arbitrary aspect ratios while preserving visually prominent features. The method operates by iteratively computing optimal local scaling factors for each local region and updating a warped image that matches these scaling factors as closely as possible. The amount of deformation of the image content is guided by a significance map that characterizes the visual attractiveness of each pixel; this significance map is computed automatically using a novel combination of gradient and salience-based measures. Our technique allows diverting the distortion due to resizing to image regions with homogeneous content, such that the impact on perceptually important features is minimized. Unlike previous approaches, our method distributes the distortion in all spatial directions, even when the resizing operation is only applied horizontally or vertically, thus fully utilizing the available homogeneous regions to absorb the distortion. We develop an efficient formulation for the nonlinear optimization involved in the warping function computation, allowing interactive image resizing. In this paper, a novel content-preserving video stitching method is proposed. Video stitching has to deal with moving objects in the overlapped area (OA) which often cause problems of structural misalignment or ghost effect. To this end, the proposed method first finds an optimal seam that does not pass through the moving objects, by using the extended dynamic programing (DP) technique based on the energy minimization. Then the content-aware adaptive blending is performed which effectively reduces the color discontinuity while restricting the ghost effect caused by moving objects in the OA. In addition, to reduce the computational complexity, a partial seam-update (PSU) scheme is proposed, in which the seam is re-calculated only for a part of the seam passing through the moving object. Experimental results demonstrate that the proposed method is superior to the conventional ones in terms of both subjective quality and computational complexity. In addition, this method achieves real-time performance on mobile platform, making it applicable to consumer electronic devices. The aim of this paper is to achieve seamless image stitching without producing visual artifact caused by severe intensity discrepancy and structure misalignment, given that the input images are roughly aligned or globally registered. Our new approach is based on structure deformation and propagation for achieving the overall consistency in image structure and intensity. The new stitching algorithm, which has found applications in image compositing, image blending, and intensity correction, consists of the following main processes. Depending on the compatibility and distinctiveness of the 2D features detected in the image plane, single or double optimal partitions are computed subject to the constraints of intensity coherence and structure continuity. Afterwards, specific 1D features are detected along the computed optimal partitions from which a set of sparse deformation vectors is derived to encode 1D feature matching between the partitions. These sparse deformation cues are robustly propagated into the input images by solving the associated minimization problem in gradient domain, thus providing a uniform framework for the simultaneous alignment of image structure and intensity. We present results in general image compositing and blending in order to show the effectiveness of our method in producing seamless stitching results from complex input images. Image retargeting adjusts images to arbitrary sizes such that they can be viewed on different displays. Content-aware image retargeting has been receiving increased attention. In particular, researchers have improved a patch-wise scaling method for image retargeting at the object level. The scaling partitions the image into rectangular patches of adaptive sizes, which are comparable to the sizes of the salient objects in the image. This partitioning is based on a visual-saliency map; accordingly, the method labels the patches as important or unimportant. Then, the method scales the important patches as uniformly as possible and stretches or squeezes the unimportant patches to fit the target size. A patch-based image-similarity measure finds the optimal set of scaling factors. In experiments, the improved method performed well for three image types: lines and edges, foreground objects, and geometric structures. Image stitching is still challenging in consumer-level photography due to imperfect image captures. Recent works show that seam-cutting approaches can effectively relieve the artifacts generated by local misalignment. Normally, the seam-cutting approach is described in terms of energy minimization. However, few of existing methods consider the human perception in their energy functions, which sometimes causes that there exists another seam that is perceptually better than the one with the minimum energy. In this paper, we propose a novel perception-based seam-cutting approach that considers the nonlinearity and the nonuniformity of human perception into the energy minimization. Our method uses a sigmoid metric to characterize the perception of color discrimination and a saliency weight to simulate that the human eye inclines to pay more attention to the salient objects. In addition, our approach can be easily integrated into other stitching pipelines. Representative experiments demonstrate substantial improvements over the conventional seam-cutting approach. In this paper, we propose a technique for video object segmentation using patch seams across frames. Typically, seams, which are connected paths of low energy, are utilised for retargeting, where the primary aim is to reduce the image size while preserving the salient image contents. Here, we adapt the formulation of seams for temporal label propagation. The energy function associated with the proposed video seams provides temporal linking of patches across frames, to accurately segment the object. The proposed energy function takes into account the similarity of patches along the seam, temporal consistency of motion and spatial coherency of seams. Label propagation is achieved with high fidelity in the critical boundary regions, utilising the proposed patch seams. To achieve this without additional overheads, we curtail the error propagation by formulating boundary regions as rough-sets. The proposed approach out-perform state-of-the-art supervised and unsupervised algorithms, on benchmark datasets. Content-aware image retargeting has attracted substantial research interests in the related research community. However, so far there is still no method can preserve important image contents and structure well without introducing deformation. To address this problem, we propose a Saliency & Structure Preserving Multi-operator (SSPM) method. SSPM classifies images into three categories utilizing SIFT density to improve performance of saliency preservation, helping to mitigate negative influence from center-bias property of most existing saliency detection models. SSPM also employs different principles to improve structure preservation performance, including Earth Mover's Distance (EMD) and Gray-Level Cooccurrence Matrix (GLCM) to get optimal operator sequences for smart content-aware image retargeting. SSPM method not only can well preserve salient contents and structure, but also can greatly improve deformation resilience. Experimental results demonstrated that our method outperforms state-of-art image retargeting methods. This letter proposes an improved seam carving approach for content-aware image retargeting. The proposed algorithm extends upon the backward and forward energy cost functionals used in previous seam carving methods by incorporating an energy gradient cost functional in the optimization process. This combined absolute energy cost functional penalizes seam candidates that cross areas of local extrema, which characterizes regions with high concentrations of “important” image detail. Experiments show superior visual quality results using the proposed absolute energy cost functional over existing methods on a variety of images characterized by high image detail concentration. Image resizing can be more effectively achieved with a better understanding of image semantics. In this paper, similar patterns that exist in many real-world images. are analyzed. By interactively detecting similar objects in an image, the image content can be summarized rather than simply distorted or cropped. This method enables the manipulation of image pixels or patches as well as semantic objects in the scene during image resizing process. Given the special nature of similar objects in a general image, the integration of a novel object carving operator with the multi-operator framework is proposed for summarizing similar objects. The object removal sequence in the summarization strategy directly affects resizing quality. The method by which to evaluate the visual importance of the object as well as to optimally select the candidates for object carving is demonstrated. To achieve practical resizing applications for general images, a template matching-based method is developed. This method can detect similar objects even when they are of various colors, transformed in terms of perspective, or partially occluded. To validate the proposed method, comparisons with state-of-the-art resizing techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method. Most existing image dehazing methods tend to boost local image contrast for regions with heavy haze. Without special treatment, these methods may significantly amplify existing image artifacts such as noise, color aliasing and blocking, which are mostly invisible in the input images but are visually intruding in the results. This is especially the case for low quality cellphone shots or compressed video frames. The recent work of (2014) addresses blocking artifacts for dehazing, but is insufficient to handle other artifacts. In this paper, we propose a new method for reliable suppression of different types of visual artifacts in image and video dehazing. Our method makes contributions in both the haze estimation step and the image recovery step. Firstly, an image-guided, depth-edge-aware smoothing algorithm is proposed to refine the initial atmosphere transmission map generated by local priors. In the image recovery process, we propose Gradient Residual Minimization (GRM) for jointly recovering the haze-free image while explicitly minimizing possible visual artifacts in it. Our evaluation suggests that the proposed method can generate results with much less visual artifacts than previous approaches for lower quality inputs such as compressed video clips. Image retargeting algorithms often create visually disturbing distortion. We introduce the property of scene consistency, which is held by images which contain no object distortion and have the correct object depth ordering. We present two new image retargeting algorithms that preserve scene consistency. These algorithms make use of a user-provided relative depth map, which can be created easily using a simple GrabCut-style interface. Our algorithms generalize seam carving. We decompose the image retargeting procedure into (a) removing image content with minimal distortion and (b) re-arrangement of known objects within the scene to maximize their visibility. Our algorithms optimize objectives (a) and (b) jointly. However, they differ considerably in how they achieve this. We discuss this in detail and present examples illustrating the rationale of preserving scene consistency in retargeting.