What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> Image registration finds a variety of applications in computer vision. Unfortunately, traditional image registration techniques tend to be costly. We present a new image registration technique that makes use of the spatial intensity gradient of the images to find a good match using a type of Newton-Raphson iteration. Our technique is taster because it examines far fewer potential matches between the images than existing techniques Furthermore, this registration technique can be generalized to handle rotation, scaling and shearing. We show how our technique can be adapted tor use in a stereo vision system. <s> BIB001 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments. > <s> BIB002 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on virtual monitors which can be freely positioned about a user in space. Users can collaboratively view and interact with virtual objects using a shared virtual whiteboard. This is possible through precise virtual image registration using fast and accurate computer vision techniques and head mounted display (HMD) calibration. We propose a method for tracking fiducial markers and a calibration method for optical see-through HMD based on the marker tracking. <s> BIB003 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> We address the problems of virtual object interaction and user tracking in a table-top augmented reality (AR) interface. In this setting there is a need for very accurate tracking and registration techniques and an intuitive and useful interface. This is especially true in AR interfaces for supporting face to face collaboration where users need to be able to easily cooperate with each other. We describe an accurate vision-based tracking method for table-top AR environments and tangible user interface (TUI) techniques based on this method that allow users to manipulate virtual objects in a natural and intuitive manner. Our approach is robust, allowing users to cover some of the tracking markers while still returning camera viewpoint information, overcoming one of the limitations of traditional computer vision based systems. After describing this technique we describe its use in prototype AR applications. <s> BIB004 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> Merging virtual objects with human video sequences is an important technique for many applications such as special effects in movies and augmented reality. In the traditional method, the operator manually fits a 3D body model onto the human video sequence, and generates virtual objects at the current 3D body pose. However, the manual fitting is a time consuming task, and automatic registration is required. We propose a new method for merging virtual objects onto the human video sequence. First, we track the current 3D pose of the human figure by using spatio-temporal analysis and structural knowledge of the human body. Then we generate a CG object and merge it with the human figure in video. We demonstrate examples of merging virtual cloth with the video captured images. <s> BIB005 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> Preface.List of Contributors.List of Figures.List of Tables.1. An Overview of Virtual Humans (Nadia Magnenat Thalmann and Daniel Thalmann).2. Face Cloning and Face Motion Capture (Wonsook Lee, Taro Goto, Sumedha Kshirsagar, Tom Molet).3. Body Cloning and Body Motion Capture (Pascal Fua, Ralf Plaenkers, WonSook Lee, Tom Molet).4. Anthropometric Body Modeling (Hyewon Seo).5. Body Motion Control (Ronan Boulic, Paolo Baerlocher).6. Facial Deformation Models (Prem Kalra, Stephane Garchery, Sumedha Kshirsagar).7. Body Deformations (Amaury Aubel).8. Hair Simulation (Sunil Hadap).9. Cloth Simulation (Pascal Volino, Frederic Cordier).10. Expressive Speech Animation and Facial Communication (Sumedha Kshirsagar, Arjan Egges, Stephane Garchery).11. Behavioral Animation (Jean-Sebastien Monzani, Anthony Guye-Vuilleme, Etienne de Sevin).12. Body Gesture Recognition and Action Response (Luc Emering, Bruno Herbelin).13. Interaction with 3-D Objects (Marcello Kallmann).14. Groups and Crowd Simulation (Soraia Raupp Musse, Branislav Ulicny, Amaury Aubel).15. Rendering of Skin and Clothes (Neeharika Adabala).16. Standards for Virtual Humans (Stephane Garchery, Ronan Boulic, Tolga Capin, Prem Kalra).Appendix A: Damped Least Square Pseudo-Inverse J+A.Appendix B: H-Anim Joint and Segment Topology.Appendix C: Facial Animation Parameter Set .References.Index. <s> BIB006 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> This paper describes an Augmented Reality (AR) Videoconferencing System, which is a novel remote collaboration tool combining a desktop-based AR system and a videoconference module. The novelty of our system is the combination of these tools with AR applications superimposed on live video background displaying the conference parties' real environment, merging the advantages of the natural face-to-face communication of videoconferencing and AR's interaction capabilities with distributed virtual objects using tangible physical artifacts. The simplicity of the system makes it affordable for everyday use. We explain our system design based on concurrent video streaming, optical tracking and 3D application sharing, and provide experimental proof that it yields superior quality compared to pure video streaming with successive optical tracking from the compressed streams. We demonstrate the system's collaborative features with a volume rendering application that allows users to display and examine volumetric data simultaneously and to highlight or explore slices of the volume by manipulating an optical marker as a cutting plane interaction device. <s> BIB007 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> The goal of this article is to review the state-of-the-art tracking methods, classify them into different categories, and identify new trends. Object tracking, in general, is a challenging problem. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of both the object and the scene, nonrigid object structures, object-to-object and object-to-scene occlusions, and camera motion. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. Typically, assumptions are made to constrain the tracking problem in the context of a particular application. In this survey, we categorize the tracking methods on the basis of the object and motion representations used, provide detailed descriptions of representative methods in each category, and examine their pros and cons. Moreover, we discuss the important issues related to tracking including the use of appropriate image features, selection of motion models, and detection of objects. <s> BIB008 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> Abstract Human motion tracking for rehabilitation has been an active research topic since the 1980s. It has been motivated by the increased number of patients who have suffered a stroke, or some other motor function disability. Rehabilitation is a dynamic process which allows patients to restore their functional capability to normal. To reach this target, a patients’ activities need to be continuously monitored, and subsequently corrected. This paper reviews recent progress in human movement detection/tracking systems in general, and existing or potential application for stroke rehabilitation in particular. Major achievements in these systems are summarised, and their merits and limitations individually presented. In addition, bottleneck problems in these tracking systems that remain open are highlighted, along with possible solutions. <s> BIB009 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> This chapter presents an overview of the Mixed Reality (MR) paradigm, which proposes to overlay our real-world environment with digital, computer-generated objects. It presents example applications and outlines limitations and solutions for their technical implementation. In MR systems, users perceive both the physical environment around them and digital elements presented through, for example, the use of semitransparent displays. By its very nature, MR is a highly interdisciplinary field engaging signal processing, computer vision, computer graphics, user interfaces, human factors, wearable computing, mobile computing, information visualization, and the design of displays and sensors. This chapter presents potential MR applications, technical challenges in realizing MR systems, as well as issues related to usability and collaboration in MR. It separately presents a section offering a selection of MR projects which have either been partly or fully undertaken at Swiss universities and rounds off with a section on current challenges and trends. <s> BIB010 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> This paper presents a system for performance-based character animation that enables any user to control the facial expressions of a digital avatar in realtime. The user is recorded in a natural environment using a non-intrusive, commercially available 3D sensor. The simplicity of this acquisition device comes at the cost of high noise levels in the acquired data. To effectively map low-quality 2D images and 3D depth maps to realistic facial expressions, we introduce a novel face tracking algorithm that combines geometry and texture registration with pre-recorded animation priors in a single optimization. Formulated as a maximum a posteriori estimation in a reduced parameter space, our method implicitly exploits temporal coherence to stabilize the tracking. We demonstrate that compelling 3D facial dynamics can be reconstructed in realtime without the use of face markers, intrusive lighting, or complex scanning hardware. This makes our system easy to deploy and facilitates a range of new applications, e.g. in digital gameplay or social interactions. <s> BIB011 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> A more robust mean shift tracker using the joint of color and Completed Local Ternary Pattern (CLTP) histogram is proposed. CLTP is a generalization of Local Binary Pattern (LBP) which can be applied to obtain texture features that are more discriminant and less sensitive to noise. The joint of color and CLTP histogram based target representation can exploit the target structural information efficiently. To reduce the interference of background in target localization, a corrected background-weighted histogram and background update mechanism are adapted to decrease the weights of both prominent background color and texture features similar to the target object. Comparative experimental results on various challenging videos demonstrate that the proposed tracker performs favorably against several variants of state-of-the-art mean shift tracker when heavy occlusions and complex background changes exist. <s> BIB012 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> The Microsoft Kinect camera is becoming increasingly popular in many areas aside from entertainment, including human activity monitoring and rehabilitation. Many people, however, fail to consider the reliability and accuracy of the Kinect human pose estimation when they depend on it as a measuring system. In this paper we compare the Kinect pose estimation (skeletonization) with more established techniques for pose estimation from motion capture data, examining the accuracy of joint localization and robustness of pose estimation with respect to the orientation and occlusions. We have evaluated six physical exercises aimed at coaching of elderly population. Experimental results present pose estimation accuracy rates and corresponding error bounds for the Kinect system. <s> BIB013 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> This paper gives the survey of the existing developments of Visual object target tracking using particle filter from the last decade and discusses the advantage and disadvantages of various particle filters. A variety of different approaches and algorithms have been proposed in literature. At present most of the work in Visual Object Target Tracking is focusing on using particle filter. The particle filters has the advantage that they deal with nonlinear models and non-Gaussian innovations, and they focus sequentially on the higher density regions of the state space, mostly parallelizable and easy to implement, so it gives a robust tracking framework, as it models the uncertainty and showing good improvement in the recognition performance compared to the kalman filter and other filters like Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF).Various features and classifiers that are used with particle filter are given in this survey. <s> BIB014 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> Analysis of human behaviour through visual information has been a highly active research topic in the computer vision community. This was previously achieved via images from a conventional camera, however recently depth sensors have made a new type of data available. This survey starts by explaining the advantages of depth imagery, then describes the new sensors that are available to obtain it. In particular, the Microsoft Kinect has made high-resolution real-time depth cheaply available. The main published research on the use of depth imagery for analysing human activity is reviewed. Much of the existing work focuses on body part detection and pose estimation. A growing research area addresses the recognition of human actions. The publicly available datasets that include depth imagery are listed, as are the software libraries that can acquire it from a sensor. This survey concludes by summarising the current state of work on this topic, and pointing out promising future research directions. For both researchers and practitioners who are familiar with this topic and those who are new to this field, the review will aid in the selection, and development, of algorithms using depth data. <s> BIB015 </s> What is the Truth: A Survey of Video Compositing Techniques <s> III. TRACKING <s> matchmoving the invisible art of camera tracking is available in our book collection an online access to it is set as public so you can get it instantly. Our digital library hosts in multiple locations, allowing you to get the most less latency time to download any of our books like this one. Merely said, the matchmoving the invisible art of camera tracking is universally compatible with any devices to read. <s> BIB016
The tracking process is performed because of that, if one of the layers that are used in the blending stage is captured using a dynamic camera, the extracted object from the original footage appears, after blending it with a new background, as a skating object. The camera movements in the scene, that are captured using a dynamic camera, must be applied to the footages that are captured using a static camera. In the case of using dynamic cameras for recording both of the two layers, the stabilizing process is performed to make one of them appears as footages that were recorded using a static camera. The dynamic camera may be a tilt/pan camera or a free camera. For the tilt/pan camera, the 2D tracking is sufficient for getting the movements of the camera. In the free camera case, match moving techniques are used for estimating the 3D position of the camera BIB016 . Another reason for the tracking process is that if there is an extracted object that is required to be attached with a dynamic object in other footage. The tracking process is performed to estimate the 2D or 3D position of this object for achieving the spatial/temporal coherence of the extracted object and the dynamic object. There are many 2D tracking techniques were presented BIB014 , such as Kanade-Lucas-Tomasi (KLT) algorithm BIB001  BIB002 , mean-shift  BIB012 , and appearance tracking  . According to A. Yilmaz et al. BIB008 , KLT handles affine transformations for partial occlusion cases without training. In addition, there is a set of commercial software packages that facilitate the 2D tracking process, such as mocha software (http://www.imagineersystems.com/) that uses a planar tracking image engine. The 2D tracking process is used for estimating the movements of tilt/pan cameras. In this case, a good marker for tracking is recommended to be used. The marker must be a visual marker that has large details in the texture or a set of corners  . In the case of green screens, the suitable solution for tracking the movements of the camera is a green marker. The usage of green visual markers for tracking the movements of the tilt/pan camera facilitates the marker removal process using the same chromakeying technique that is used for removing the background. In the case of 2D movements of a dynamic object, 2D tracking is efficient for attached other static footage with the moving object. While in the case of 3D movements of a dynamic object, there are other techniques that are used, such as Motion Capture (MoCap) systems. MoCap systems are considered the best solution for tracking 3D transformations of an animated object BIB006 . Magnetic MoCap technologies are used to capture the object's animations using wired transmitters that generate magnetic fields BIB006 . Optical MoCap systems provide a suitable environment rather than magnetic MoCap systems. The capturing process in optical MoCap systems depends on an optical movement analysis of the locations of markers that are attached to the dynamic object, see Fig. 6 . Recently, depth sensors are used to analysis the captured depth frames from the scene for the 3D tracking of objects and human skeletal movements BIB009 . Microsoft Kinect sensor tracks the human body using a set of 3D joints that represent the skeleton of the actor  . In spite of the low-cost of the depth sensors and its efficiency on many applications, there are problems in the accuracy on tracking occluding body parts BIB013 BIB015 . In 2011, T. Weise et al. BIB011 presented a real time facial animation system that is based on the depth sensor technology. The technique uses Microsoft Kinect to track the facial performance of the user using a predefined expression model of him. Those models were reconstructed as blend-shapes in an off-line preprocessing step. The tracking data is used to assign weights to the blend-shapes for generating the reconstructed head that mimics the user's facial performance. Fig. 7 shows the usage of T. Weise et al. technique BIB011 for tracking movements of a real head. The extracted animations were applied to a virtual head of the user that was generated using FaceGen software (http://www.facegen.com/). Traditional computer vision tracking techniques usually require manual initializations and a close camera to the interesting object. However, most 3D tracking techniques are considered fragile techniques  . H. Kato et al. BIB003 presented a robust tracking technique with low CPU requirements for the 3D tracking purpose without constraining the camera position  BIB010 . This technique is presented as a computer tracking library called AR-Toolkit that is used in the video-based Augmented Reality (AR) system that is presented by H. Kato et al. BIB004 . H. Kato et al. BIB003 presented an AR technique for estimating 3D transformations of a visual marker, see Fig 8. AR-Toolkit requires only one visual marker for estimating the movements of an object in the real scene. The visual marker is a specific printed graphical symbol that is designed for a recognition technique, see Fig. 9 . Detecting the printed marker is the first step of H. Kato et al. technique BIB003 . The detection process is performed by converting the color frame to a binary one. After generating the binary image of the captured frame, the connected component labeling is performed to label each 8-connected neighborhood under the same number. For each one of the connected components, the contour is detected. Those contours are used to detect rectangular shapes on the captured marker. The final output of this process is an outer rectangle of the marker. For detecting the target symbol inside the marker, the sub-image within the region of the marker is normalized to be analyzed from the front view. The analysis is performed by matching the symbol inside the marker with predefined templates for the detection technique. The detected marker is used for estimating the Model-view matrix (M). M specifies the 3D transformations of the marker with respect to the camera coordinates. In which, M transforms vertices from the coordinates of the marker to the coordinates of the camera, as shown in the following equation Fig. 9 is considered the most effective marker to be used in the tracking process BIB007 . In the case of free camera, match moving techniques are used BIB016 . Match moving process estimates the 3D movements of the objects from the 2D footages. From the estimated data, the camera movements can be estimated and applied to a virtual camera for integrating virtual objects into the real scene BIB016 . Another task for match moving process is to integrate a real object that is recorded using a free camera into a virtual world  . The real scene must contain heavy details to be used as features for the match moving process  . The features are extracted from the real scene. After that, a camera solver technique uses the extracted features for generating the 3D trajectories of those features, see Fig. 10 . Many techniques use match moving process for integrating virtual objects into a real scene, such as H. Saito et al. BIB005 who integrated a virtual clothes into a real human. In addition, there are several commercial software products provide the match moving process, such as Boujou After tracking the interesting animated object and the camera movements, the blending stage is performed for generating the pre-final result of the video compositing process.