1. A video tracking system, comprising: a processor designed to execute layered processing comprising: a master peripheral tracker operative to interact with frames of image data, said master peripheral tracker including logic to monitor a scene and to detect an object within said frames of image data using a motion history image algorithm utilizing a windowed sequence of consecutive frame differences; and a first tunnel tracker initiated by said master peripheral tracker, said first tunnel tracker including logic dedicated to track one said detected object, located within a first buffer area, utilizing a first portion of said image data frame-by-frame by building correspondences for said tracked object based on its attributes, said first buffer area being formed by said master peripheral tracker based on said tracked object's current trajectory, size and location.