Many telepresence robots are equipped with a forward-facing camera for video communication and a downward-facing camera for navigation. In this paper, we propose to stitch videos from the FF-camera with a wide-angle lens and the DF-camera with a fisheye lens for telepresence robots. We aim at providing more compact and efficient visual feedback for the user interface of telepresence robots with user-friendly interactive experiences. To this end, we present a multi-homography-based video stitching method which stitches videos from a wide-angle camera and a fisheye camera. The method consists of video image alignment, seam cutting, and image blending. We directly align the wide-angle video image and the fisheye video image based on the multi-homography alignment without calibration, distortion correction, and unwarping procedures. Thus, we can obtain a stitched video with shape preservation in the non-overlapping regions and alignment in the overlapping area for telepresence. To alleviate ghosting effects caused by moving objects and or moving cameras during telepresence robot driving, an optimal seam is found for aligned video composition, and the optimal seam will be updated in subsequent frames, considering spatial and temporal coherence. The final stitched video is created by image blending based on the optimal seam. We conducted a user study to demonstrate the effectiveness of our method and the superiority of telepresence robots with a stitched video as visual feedback.
There are several commercial video stitching softwares, such as VideoStitch Studio https: www.orah.co software videostitch-studio and AutoPano http: www.kolor.com autopano . These softwares usually compute a 2D transformation relating two cameras, and then bring all pairs of video images into alignment for the post-production of stitched videos. To improve the quality of the stitched video, @cite_6 found double-seam to eliminate intensity misalignment, and similar work has been presented for designing a content-aware adaptive blending @cite_49 . Some works were presented to obtain better alignment results. Lee and Sim @cite_15 stitched videos by projecting the background plane and the foreground objects separately, while Jiang and Gu @cite_54 stitched videos using spatial-temporal content-preserving warps. For videos captured by handheld cameras, @cite_4 @cite_51 and @cite_19 combined the stitching and stabilization techniques together into a unified optimization framework for video stitching, whereas @cite_44 stitched videos by reconstructing the 3D scene using the recovered 3D camera paths and the 3D scene points. These methods stitched videos in an iterative manner with low computational efficiency.
@cite_4 @cite_54 @cite_6 @cite_44 @cite_19 @cite_49 @cite_15 @cite_51
2516719699 2404729542 1606751047 2559571756
In this paper, we extend image stitching to video stitching for videos that are captured for the same scene simultaneously by multiple moving cameras. In practice, videos captured under this circumstance often appear shaky. Directly applying image stitching methods for shaking videos often suffers from strong spatial and temporal artifacts. To solve this problem, we propose a unified framework in which video stitching and stabilization are performed jointly. Specifically, our system takes several overlapping videos as inputs. We estimate both inter motions (between different videos) and intra motions (between neighboring frames within a video). Then, we solve an optimal virtual 2D camera path from all original paths. An enlarged field of view along the virtual path is finally obtained by a space-temporal optimization that takes both inter and intra motions into consideration. Two important components of this optimization are that: 1) a grid-based tracking method is designed for an improved robustness, which produces features that are distributed evenly within and across multiple views and 2) a mesh-based motion model is adopted for the handling of the scene parallax. Some experimental results are provided to demonstrate the effectiveness of our approach on various consumer-level videos and a Plugin, named “Video Stitcher” is developed at Adobe After Effects CC2015 to show the processed videos. Images videos captured by portable devices (e.g., cellphones, DV cameras) often have limited fields of view. Image stitching, also referred to as mosaics or panorama, can produce a wide angle image by compositing several photographs together. Although various methods have been developed for image stitching in recent years, few works address the video stitching problem. In this paper, we present the first system to stitch videos captured by hand-held cameras. We first recover the 3D camera paths and a sparse set of 3D scene points using CoSLAM system, and densely reconstruct the 3D scene in the overlapping regions. Then, we generate a smooth virtual camera path, which stays in the middle of the original paths. Finally, the stitched video is synthesized along the virtual path as if it was taken from this new trajectory. The warping required for the stitching is obtained by optimizing over both temporal stability and alignment quality, while leveraging on 3D information at our disposal. The experiments show that our method can produce high quality stitching results for various challenging scenarios. We describe an algorithm for generating panoramic video from unstructured camera arrays. Artifact-free panorama stitching is impeded by parallax between input views. Common strategies such as multi-level blending or minimum energy seams produce seamless results on quasi-static input. However, on video input these approaches introduce noticeable visual artifacts due to lack of global temporal and spatial coherence. In this paper we extend the basic concept of local warping for parallax removal. Firstly, we introduce an error measure with increased sensitivity to stitching artifacts in regions with pronounced structure. Using this measure, our method efficiently finds an optimal ordering of pair-wise warps for robust stitching with minimal parallax artifacts. Weighted extrapolation of warps in non-overlap regions ensures temporal stability, while at the same time avoiding visual discontinuities around transitions between views. Remaining global deformation introduced by the warps is spread over the entire panorama domain using constrained relaxation, while staying as close as possible to the original input views. In combination, these contributions form the first system for spatiotemporally stable panoramic video stitching from unstructured camera array input. Stitching videos captured by handheld devices is very useful, but also very challenging due to the heavy and independent shakiness in the videos. In this paper, we propose a hand-taken video stitching method which combines the techniques of video stitching and stabilization together into a unified optimization framework. In this way, our method can compute the most optimal stabilization and stitching results with respect to each other, which outperforms previous methods that take stabilization and stitching as separate operations. Our method is based on the framework of bundled camera paths [ 2013]. We present a novel unified camera paths optimization formulation which consists of two stabilization terms and one stitching term. We also present a corresponding iterative solver that finds best stitching and stabilization solutions numerically. We compare our method with previous methods, and the experiments demonstrate the effectiveness of our method.