1. An object tracking device, comprising: a light source configured to emit light at a lighting frequency; an image sensor configured to output an operating image when the light source is being turned on and output a background image when the light source is being turned off; and a processing unit configured to obtain background information from the background image, wherein the background information is a background object position of at least one background object in the background image, calculate a differential image of the operating image and the background image, identify at least one object in the differential image, and remove an invalid object, which has a distance from the background object position smaller than a distance threshold and has a size smaller than a size threshold, from the differential image.