1. An image processing apparatus comprising: a background image holding unit configured to hold a background image; a threshold table holding unit configured to hold a threshold; a previous frame image holding unit configured to hold a previous frame image which is a frame image temporally prior to the latest image; and an object extraction unit configured to perform a background subtraction process of calculating difference values between the background image and latest image and detecting a pixel whose calculated difference value is equal to or larger than the threshold as an extracted object and a process of judging the magnitude of difference between the background image and latest image, wherein the object extraction unit is configured to when determining that the magnitude of difference between the background image and latest image is less than a predetermined value, update the background image by adding the latest image to the background image in a predetermined ratio, when determining that the magnitude of difference between the background and latest image is not less than the predetermined value, update the threshold by calculating a difference average which is the average of the difference values between the previous frame image and the latest image and adding the absolute value of the calculated difference average to the threshold, update the background image by replacing the pixels of the latest image which are not recognized as the extracted object as the result of execution of the background subtraction process using the updated threshold, for the corresponding pixels of the background image as pixels of a new background image and replacing pixels of the background image corresponding to the pixels which are recognized as the extracted object with the pixels obtained by adding the difference average to the background image as pixels of the new background image, the background image holding unit holds the background image updated by the object extraction unit instead of the background image that the background image holding unit has held until that time, and the threshold table holding unit holds the threshold updated by the object extraction unit instead of the threshold that the threshold table holding unit has held until that time.