1. A method for updating a scene model in a video which is composed of a plurality of visual elements, comprising: a classifying step for classifying the visual elements in a scene into stationary visual elements and moving visual elements according to their appearance change rates; a border determining step for determining borders from the scene according to a spatial distribution information of the stationary visual elements and the moving visual elements elements wherein the spatial distribution information is a sum of a first density of the stationary visual elements at a side of a position and a second density of the moving visual elements at another side of the position and the position as the border is determined based on the sum of the first density and the second density; and an updating step for updating the scene model according to the determined borders in said scene.