What is the Truth: A Survey of Video Compositing Techniques <s> C. Compositing Of Virtual And Real Footages <s> We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on virtual monitors which can be freely positioned about a user in space. Users can collaboratively view and interact with virtual objects using a shared virtual whiteboard. This is possible through precise virtual image registration using fast and accurate computer vision techniques and head mounted display (HMD) calibration. We propose a method for tracking fiducial markers and a calibration method for optical see-through HMD based on the marker tracking. <s> BIB001 </s> What is the Truth: A Survey of Video Compositing Techniques <s> C. Compositing Of Virtual And Real Footages <s> Using generic interpolation machinery based on solving Poisson equations, a variety of novel tools are introduced for seamless editing of image regions. The first set of tools permits the seamless importation of both opaque and transparent source image regions into a destination region. The second set is based on similar mathematical ideas and allows the user to modify the appearance of the image seamlessly, within a selected region. These changes can be arranged to affect the texture, the illumination, and the color of objects lying in the region, or to make tileable a rectangular selection. <s> BIB002 </s> What is the Truth: A Survey of Video Compositing Techniques <s> C. Compositing Of Virtual And Real Footages <s> The use of optical motion capture systems to produce high quality humanoid motion sequences has recently become increasingly popular in many areas such as high-end gaming, cinema movie and animation production, and for educational purposes. Motion capture studios are expensive to employ or are unavailable to many people and locales. For this reason, the reuse of existing motion capture data and the resulting high-quality animation is an important area of research. Strict motion capture data restricts users to pre-recorded movement that does not allow the addition of dynamic behaviors required in advanced games and interactive environment. Our procedural animation method offers animators high quality animations produced from an optical motion capture session, without incurring the cost of running their own sessions. This method utilizes a database of common animations sequences, derived from several motion capture sessions, which animators can manipulate and apply to their own existing characters through the use of our procedural animation toolkit. The basic animation types include walk, run, and jump sequences that can be applied with personality variations that correspond to a character's gender, age, and energy levels. <s> BIB003 </s> What is the Truth: A Survey of Video Compositing Techniques <s> C. Compositing Of Virtual And Real Footages <s> Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches. <s> BIB004 </s> What is the Truth: A Survey of Video Compositing Techniques <s> C. Compositing Of Virtual And Real Footages <s> This paper presents a system for performance-based character animation that enables any user to control the facial expressions of a digital avatar in realtime. The user is recorded in a natural environment using a non-intrusive, commercially available 3D sensor. The simplicity of this acquisition device comes at the cost of high noise levels in the acquired data. To effectively map low-quality 2D images and 3D depth maps to realistic facial expressions, we introduce a novel face tracking algorithm that combines geometry and texture registration with pre-recorded animation priors in a single optimization. Formulated as a maximum a posteriori estimation in a reduced parameter space, our method implicitly exploits temporal coherence to stabilize the tracking. We demonstrate that compelling 3D facial dynamics can be reconstructed in realtime without the use of face markers, intrusive lighting, or complex scanning hardware. This makes our system easy to deploy and facilitates a range of new applications, e.g. in digital gameplay or social interactions. <s> BIB005
In this subsection, virtual objects are integrated into real scenes. In the first experiment, Boujou software was used in the match moving process, where the free camera's movements were estimated. The estimated camera movements were applied for a virtual camera. Virtual objects that were generated using the particle system that is presented by 3ds Max software were rendered using Vray render. Virtual shadows were generated using a virtual plane with the 3ds max Matten/Shadow material, see Fig. 15 . The rendering process used the virtual camera for applying the same movements of the real camera to the generated footages of the virtual objects, see Fig. 16 . In the following experiment, a virtual head is integrated into a real actor for changing the appearance of the actor. This example is inspired from the virtual actor system that is presented in  . Microsoft Kinect sensor was used for tracking the real head of the actor using T. Weise et al. technique BIB005 . Because of the low-resolution camera of Microsoft Kinect sensor, three printed markers and a high-resolution camera were used. The highresolution camera was synchronized with Microsoft Kinect sensor during recording the footages of the actor. The first one of the markers was attached to the face of the actor. The other markers were attached to the neck of the actor. Those markers were used by M-estimator SAmple Consensus algorithm (MSAC)  for aligning the virtual head with the real body, where the virtual head was rendered with similar three markers in the draft version. The actor wore a green cloth that covers his head except his face. Color difference matte technique  is used for separating green pixels from the footages. A virtual head was aligned with the real body of the actor. After replacing the real head with a virtual one, the real facial features of the actor is composited with the virtual head using MPB technique  . For replacing the body of an actor with a virtual one, H. Kato et al. BIB001 was used for tracking the real body of the actor. One of the visual markers that are designed for the tracking technique was used. The actor wore a green Tshirt for separating his body using chroma-keying techniques. Dynamic virtual clothes were designed using CLO 3D/Marvelous Designer software. The rendering process was performed using 3ds Max. The final result is an actor with virtual clothes that follow his movements, as shown in Fig. 18 . In the last experiment, virtual actors are integrated into a real scene. For simplicity, a static camera was used for recording the real scene. Virtual animations were generated using Naturalpoint Inc. OptiTrack MoCap system and Endorphin software (http://www.naturalmotion.com/) which is a dynamic motion generation software package that generates a realistic simulation of the human animation using realworld parameters BIB003 . Fig. 19 shows a virtual actor after integrating him into a real scene. Fig. 20 shows another example of integrating a virtual actor into a real scene, where an interaction between a real car and the virtual actor is presented. For making the virtual actor interacts with the real car, a virtual model of the car was modeled using 3ds Max. This model is used by Endorphin software for estimating the realistic human animations of the virtual actor after being hit by the car. BIB002 , error-tolerant image compositing BIB004 , and MPB technique  in seconds. The first column represents the size of the dataset  . Under each technique there are three columns. Each column represents the average required time in seconds by each technique for blending source images that have different sizes with the corresponding size of the dataset. L refers to a large source image (400X320 pixels). M refers to a medium source image (200X251 pixels). S refers to a small source image (100X76 pixels). 