Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> Fast and accurate segmentation of moving objects in video sequences is a basic task in many computer vision and video analysis applications. It has a critical impact on the performance of object tracking and classification and activity analysis. This paper presents effective methods for solving this problem. Firstly, a fast and efficient algorithm is presented for background update to handle various sources of scene changes, including ghosts, left objects, camera shaking, and abrupt illumination changes. This is done by analyzing properties of object motion in image pixels and temporal frames, and combining both levels of constraints. Moreover, the algorithm does not need training sequence. Secondly, a real-time and accurate moving object segmentation algorithm is presented for moving object localization. Here, a novel filtering method is presented based on multiple scale and fast connected blob extraction. An intelligent video surveillance system is developed to test the performance of the algorithms. Experiments are performed using long video sequences under different conditions indoor and outdoor. The results show that the proposed algorithm is effective and efficient in real-time and accurate background update and moving object segmentation. <s> BIB001 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> In background subtraction, it is challenging to detect foreground objects in the presence of dynamic background motions. The paper proposes two new algorithms to this problem by improving the codebook model with the incorporation of the spatial and temporal context of each pixel. The spatial context involves the local spatial dependency between neighboring pixels, and the temporal context involves the preceding detection result. Only the spatial context is incorporated into the first algorithm which makes the background representation more compact than the standard codebook. The second algorithm explicitly models the spatio-temporal context with a Markov random field model, thus achieving more accurate foreground detection. Extensive experiments on several dynamic scenes are conducted to compare the two proposed algorithms with each other and with the standard codebook algorithm. (C) 2009 Elsevier GmbH. All rights reserved. <s> BIB002 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> With the increasing amount of surveillance data, moving object segmentation in the compressed domain has drawn broad attention from both academy and industry. In this paper, we propose a novel moving object segmentation method towards H.264 compressed surveillance videos. First, the motion vectors (MV) are accumulated and filtered to achieve reliable motion information. Second, considering the spatial and temporal correlations among adjacent blocks, spatio-temporal Local Binary Pattern (LBP) features of MVs are extracted to obtain coarse and initial object regions. Finally, a coarse-to-fine segmentation algorithm of boundary modification is conducted based on the DCT coefficients. The experimental results validate that the proposed method not only can extract fairly accurate objects in compressed video, but also has a relatively low computational complexity. <s> BIB003 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> According to the characteristics of embedded system which is severely restricted in computational resource and memory, we propose a novel real-time detection algorithm based on Mixture of Gaussian fusing spatial-temporal information. The algorithm dynamically adjusts the number of Gaussian distributions according to the complexity of background, which can greatly improve the computational efficiency and reduce the memory consumption on the basis of proper detection. In addition, we propose a spatial-temporal histogram algorithm based on block to correct or verify the extracted object pixel, which can reduce the error detection caused by dynamic background. Eventually, we implement our algorithm on multi-DSP system based on TI's TMS320DM642. The experimental results show that the proposed algorithm achieves good performances in real time and proper rate of detection (recall and precision). In conclusion, our method is able to accomplish the detection of moving object in complex scenes stably and reliably. <s> BIB004 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> Detection of moving foreground in video is very important for many applications, such as visual surveillance, objectbased video coding, etc. When objects move with different speeds and under illumination changes, the robustness of moving object detection methods proposed so far is still not satisfactory. In this paper, we use the semantic information to adjust the pixel-wise learning rate adaptively for more robust detection performance, which are obtained by spatial saliency map based on Gaussian mixture model (GMM) in luma space and temporal saliency map obtained by background subtraction. In addition, we design a two-pass background estimation framework, in which the initial estimation is used for temporal saliency estimation, and the other is to detect foreground and update model parameters. The experimental results show that our method can achieve better moving object extraction performance than the existing background subtraction method based on GMM. <s> BIB005 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> Background subtraction is the classical approach to differentiate moving objects in a scene from the static background when the camera is fixed. If the fixed camera assumption does not hold, a frame registration step is followed by the background subtraction. However, this registration step cannot perfectly compensate camera motion, thus errors like translations of pixels from their true registered position occur. In this paper, we overcome these errors with a simple, but effective background subtraction algorithm that combines Temporal and Spatio-Temporal approaches. The former models the temporal intensity distribution of each individual pixel. The latter classifies foreground and background pixels, taking into account the intensity distribution of each pixels' neighborhood. The experimental results show that our algorithm outperforms the state-of-the-art systems in the presence of jitter, in spite of its simplicity. <s> BIB006 </s> Dependent Motion Segmentation in Moving Camera Videos: A Survey <s> b: SPATIO-TEMPORAL METHODS <s> Moving object detection in video streams is the first step of many computer vision applications. Background modeling and subtraction for moving detection is the most common technique for detecting, while how to detect moving objects correctly is still a challenge. Some methods initialize the background model at each pixel in the first N frames. However, it cannot perform well in dynamic background scenes since the background model only contains temporal features. Herein, a novel pixelwise and nonparametric moving object detection method is proposed, which contains both spatial and temporal features. The proposed method can accurately detect the dynamic background. Additionally, several new mechanisms are also proposed to maintain and update the background model. The experimental results based on image sequences in public datasets show that the proposed method provides the robustness and effectiveness in dynamic background scenes compared with the existing methods. <s> BIB007
Compared with spatial methods, spatio-temporal methods add more spatio-temporal constraints either to the background model or to the segmentation results to make the segmentation smooth. They require that the computed background or foreground have accurate and temporally consistent boundaries in multiple frames. Representative spatio-temporal background subtraction methods include Yang et al. BIB001 , Wu and Peng BIB002 , Yang et al. BIB003 , Li et al. BIB004 , Xia et al. BIB005 , Romanoni et al. BIB006 , and Yang et al. BIB007 . For example, Yang et al. BIB001 introduced a real-time moving object segmentation method based on a spatio-temporal background update algorithm. The background update fusing of pixel-level motion detection and frame-level motion detection is of high capability to handle various scene changes, such as camera shaking, left objects, ghosts, and abrupt illumination changes BIB001 . Wu and Peng BIB002 developed a codebook-based dynamic background subtraction algorithm based on spatio-temporal context, where the spatial context involves the local spatial dependency among neighboring pixels and the temporal context involves the previous segmentation result. Yang et al. BIB003 proposed a spatio-temporal LBP based moving object segmentation algorithm for the compressed videos such as H.264 compressed surveillance videos. Li et al. BIB004 constructed a real-time moving object detection algorithm using Mixture of Gaussian fusing spatial-temporal information. Xia et al. BIB005 design a moving object detection algorithm based on the spatio-temporal saliency, where the spatial saliency map is generated using Gaussian mixture model (GMM) in luma space and the temporal saliency map is obtained by background subtraction. Romanoni et al. BIB006 developed a robust background subtraction algorithm by combining temporal and spatio-temporal histograms of pixel intensities for moving camera videos. These two histograms all represent temporal distributions, but the former one only deals with a single pixel, while the latter one deals with the temporal distribution between its neighboring pixels BIB006 . Yang et al. BIB007 proposed a moving object segmentation algorithm for dynamic background scenarios using a spatiotemporal model, which can accurately model the dynamic background.