Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> 5) CAMERA MOTION DECOMPOSITION METHODS <s> Motion segmentation plays a central role in video analysis, such as the surveillance, human-computer interaction, action recognition, etc. Extensive studies have been done on the stationary camera scenarios. Recently, more attentions are focusing on dynamic backgrounds with several moving objects in the scene. In many applications, the background motion is of much less interest, and solely the local object motion is expected. Several approaches have been proposed for global motion estimation and motion segmentation (the global motion and the object motion are also named as inlier and outlier, respectively). The work in [6] introduced a parametric form which assumed the global motion model from simple translation to general perspective transformation using different parameters. A joint global motion estimation and segmentation method was proposed in [2]. It iteratively updates the inlier model by segmenting the outlier out. A regression scheme, using gradient descent (GD) [6] or least squares (LS) [5], is also applied to refine the inlier model. An outlier rejection filter in [1] explicit filters motion vectors by checking their similarity in a pre-defined window. RANSAC in [3] is a statistical method which estimates the inlier model by iteratively updating the probability of inlier. All above methods are 2D based methods. They require the multiple motions to be independent. But for interdependent motions, they may fail to deal with. Considering this, it is better to place different motions on different layers in higher dimensional space. To this end, our method transforms 2D motion field into 3D surfaces. Local extremes on the surface such as peaks, ridges and valleys depict local motions while smoothing places represent the global motion. The 3D surface is calculated using Helmoholtz decomposition. In [4], a particle filter based on Helmholtz decomposition was proposed for flow estimation. Following is its fundamental theory. For an arbitrary flow field ξ, it is decomposed into two components: curl-free (divergence) component ∇E and divergence-free (curl) component ∇ × W , where E and W are what we want to obtain the 3D potential surfaces. The <s> BIB001 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> 5) CAMERA MOTION DECOMPOSITION METHODS <s> Motion segmentation in moving camera videos is a very challenging task because of the motion dependence between the camera and moving objects. Camera motion compensation is recognized as an effective approach. However, existing work depends on prior-knowledge on the camera motion and scene structure for model selection. This is not always available in practice. Moreover, the image plane motion suffers from depth variations, which leads to depth-dependent motion segmentation in 3D scenes. To solve these problems, this paper develops a prior-free dependent motion segmentation algorithm by introducing a modified Helmholtz-Hodge decomposition (HHD) based object-motion oriented map (OOM). By decomposing the image motion (optical flow) into a curl-free and a divergence-free component, all kinds of camera-induced image motions can be represented by these two components in an invariant way. HHD identifies the camera-induced image motion as one segment irrespective of depth variations with the help of OOM. To segment object motions from the scene, we deploy a novel spatio-temporal constrained quadtree labeling. Extensive experimental results on benchmarks demonstrate that our method improves the performance of the state-of-the-art by 10%~20% even over challenging scenes with complex background. <s> BIB002
The aforementioned camera motion compensation algorithms such as feature based parametric methods or dense motion based joint estimation and segmentation methods mainly treat different camera motions as a whole without considering different camera motions according to camera motion models. This results in the inaccurate camera motion estimation and thus unconvincing foreground segmentation. Moreover, existing methods mainly performed in the 2D image motion field, where the camera motion and object motions mixed and influence with each other making the segmentation difficult. To solve these problems, one kind of camera motion decomposition methods was proposed. This kind of methods decomposes an arbitrary optical flow field into two components: curl component and divergence component using a computational fluid dynamics algorithm named as Helmholtz-Hodge decomposition (HHD) BIB001 - BIB002 . The camera radial motion (including expansion and contraction), which is irrotational, exists only in the divergence component; the camera rotation, which is incompressible, exits only in the curl component; the camera translation which is irrotational and incompressible, can exist in both the curl component and divergence component equally. To deal with high mixture between the camera motion and object motions in the 2D image motion field, HHD transforms the 2D dense motion flow field to 3D potential surfaces and develops a Quad-tree based object motion segmentation framework. This framework (as shown in Fig. 2 ) makes camera motion estimation and object motion segmentation more robust and accurate. 