1. An image background maintenance system for processing an image sequence having a plurality of frames, each frame containing a pixel, comprising: a computation module that provides previous values of the pixel; a prediction module that predicts a predicted future value for the pixel using the previous pixel values; and a declaration module that classifies the pixel by comparing the predicted future value to an actual future value of the pixel.