Video Segmentation by Non Local Consensus voting
We address the problem of Foreground/Background segmentation of "unconstrained" video. By "unconstrained" we mean that the moving objects and the background scene may be highly non rigid (e.g. waves in the sea) the camera may undergo a complex motion with 3D parallax; moving objects may suffer from motion blur, large scale and illumination changes, etc. Most existing segmentation methods fail on such unconstrained videos, especially in the presence of highly non rigid motion and low resolution. We propose a computationally efficient algorithm which is able to produce accurate results on a large variety of unconstrained videos. This is obtained by casting the video segmentation problem as a voting scheme on the graph of similar 're occurring' regions in the video sequence. We start from crude saliency votes at each pixel, and iteratively correct those votes by 'consensus voting' of re occurring regions across the video sequence. The power of our consensus voting comes from the non locality of the region re occurrence, both in space and in time enabling propagation of diverse and rich information across the entire video sequence. Qualitative and quantitative experiments indicate that our approach outperforms current state of the art methods.
BMVC