1. A brightness sensing system comprising: an image pickup unit for taking a previously taken reference image and a currently taken target image of a target space; and a calculation unit, wherein each of the reference image and the target image contains a predetermined number of pixel blocks each being formed of one or more pixels, and wherein the calculation unit is configured to: calculate a threshold value based on an overall brightness change between the predetermined number of pixel blocks of the reference image and those of the target image, obtain a pixel block based brightness change between each pixel block in the target image and its corresponding pixel block in the reference image, and compare a brightness of the target image with that of the reference image based on remaining pixel blocks other than each pixel block whose pixel block based brightness change is not less than the threshold value to thereby determine whether the target image is brighter than the reference image or not.