Seam-cutting and seam-driven techniques have been proven effective for handling imperfect image series in image stitching. Generally, seam-driven is to utilize seam-cutting to find a best seam from one or finite alignment hypotheses based on a predefined seam quality metric. However, the quality metrics in most methods are defined to measure the average performance of the pixels on the seam without considering the relevance and variance among them. This may cause that the seam with the minimal measure is not optimal (perception-inconsistent) in human perception. In this paper, we propose a novel coarse-to-fine seam estimation method which applies the evaluation in a different way. For pixels on the seam, we develop a patch-point evaluation algorithm concentrating more on the correlation and variation of them. The evaluations are then used to recalculate the difference map of the overlapping region and reestimate a stitching seam. This evaluation-reestimation procedure iterates until the current seam changes negligibly comparing with the previous seams. Experiments show that our proposed method can finally find a nearly perception-consistent seam after several iterations, which outperforms the conventional seam-cutting and other seam-driven methods.
In recent years, many efforts have been devoted to seam-cutting or seam-driven to address the complex scenes and issues in image stitching. Seam-cutting is proposed to handle the imperfect image series which aims to estimate an invisible seam between overlapping images such that the images can be seamlessly blended together. Most seam-cutting methods formulate the seam estimation as the energy minimization of a labeling problem, and minimize the energy via graph cuts @cite_4 . Various energy functions were defined in their work to handle specific issues @cite_11 @cite_13 @cite_14 @cite_3 @cite_19 @cite_0 . Our method takes the conventional seam-cutting as the initial seam estimation method.
@cite_14 @cite_4 @cite_3 @cite_0 @cite_19 @cite_13 @cite_11
2790280905 2122022994 2518764509 2101309634
Image stitching is still challenging in consumer-level photography due to imperfect image captures. Recent works show that seam-cutting approaches can effectively relieve the artifacts generated by local misalignment. Normally, the seam-cutting approach is described in terms of energy minimization. However, few of existing methods consider the human perception in their energy functions, which sometimes causes that there exists another seam that is perceptually better than the one with the minimum energy. In this paper, we propose a novel perception-based seam-cutting approach that considers the nonlinearity and the nonuniformity of human perception into the energy minimization. Our method uses a sigmoid metric to characterize the perception of color discrimination and a saliency weight to simulate that the human eye inclines to pay more attention to the salient objects. In addition, our approach can be easily integrated into other stitching pipelines. Representative experiments demonstrate substantial improvements over the conventional seam-cutting approach. We propose a new energy function for seam carving based on forward gradient differences to preserve regular structures in images. The energy function measures the curvature inconsistency between the pixels that become adjacent after seam removal, and involves the difference of gradient orientation and magnitude of the pixels. Our objective is to minimize the differences induced by the removed seam, and the optimization is performed by dynamic programming based on multiple cumulative energy maps, each of which corresponds to the seam pattern associated with a pixel. The proposed technique preserves straight lines and regular shapes better than the original and improved seam carving, and can be easily combined with other types of energy functions within the seam carving framework. We evaluated the performance of our algorithm by comparing with the original and improved seam carving algorithms using public data. Image stitching with large parallax is a challenging problem. Global alignment usually introduces noticeable artifacts. A common strategy is to perform partial alignment to facilitate the search for a good seam for stitching. Different from existing approaches where the seam estimation process is performed sequentially after alignment, we explicitly use the estimated seam to guide the process of optimizing local alignment so that the seam quality gets improved over each iteration. Furthermore, a novel structure-preserving warping method is introduced to preserve salient curve and line structures during the warping. These measures substantially improve the effectiveness of our method in dealing with a wide range of challenging images with large parallax. In the last few years, several new algorithms based on graph cuts have been developed to solve energy minimization problems in computer vision. Each of these techniques constructs a graph such that the minimum cut on the graph also minimizes the energy. Yet, because these graph constructions are complex and highly specific to a particular energy function, graph cuts have seen limited application to date. In this paper, we give a characterization of the energy functions that can be minimized by graph cuts. Our results are restricted to functions of binary variables. However, our work generalizes many previous constructions and is easily applicable to vision problems that involve large numbers of labels, such as stereo, motion, image restoration, and scene reconstruction. We give a precise characterization of what energy functions can be minimized using graph cuts, among the energy functions that can be written as a sum of terms containing three or fewer binary variables. We also provide a general-purpose construction to minimize such an energy function. Finally, we give a necessary condition for any energy function of binary variables to be minimized by graph cuts. Researchers who are considering the use of graph cuts to optimize a particular energy function can use our results to determine if this is possible and then follow our construction to create the appropriate graph. A software implementation is freely available.