1. Field of the Invention
The present invention relates to an image capturing apparatus that captures images with high dynamic-range, and a method of controlling the same.
2. Description of the Related Art
As is generally known, the dynamic range of an image sensor used in an image capturing apparatus such as a digital single-lens reflex camera, compact digital camera, or digital video camera is narrower than that of nature. For this reason, methods to widen the dynamic range of an image sensor have been conventionally examined. There are, for example, the following three methods.
Sampling a plurality of images by multi-shooting
One-shot sampling using an image sensor having pixels of different sensitivities arranged in a fixed pattern
Controlling the exposure period of each pixel in accordance with the object luminance
The three conventional methods that widen the dynamic range (to be referred to as DR hereinafter) will be explained below.
In the method of sampling a plurality of images by multi-shooting, a plurality of images of a single object are captured with various exposure periods. Based on the exposure period ratio, gain correction is performed for the pixel values of each image, thereby compositing the plurality of images. Information with a high DR (high dynamic range (HDR) image) is thus generated from the plurality of images. For example, according to a method described in Japanese Patent Laid-Open No. 2004-363666 (corresponding to U.S. Pre-Granted Publication No. 2007-103569), high-speed reading is performed in all pixels after long-time exposure. At this time, high-speed reading is performed in a time half of a frame cycle, and short-period exposure is performed in a time half of the remaining time. Additionally, during reading of the short-period exposure result, ¼ short-period exposure is performed. As a consequence, exposure is performed in a total of three different exposure periods, and the images are composited. In Japanese Patent Laid-Open No. 2004-159274 as well, a low illuminance signal by long-time accumulation, a high illuminance signal by short-time accumulation, and an ultra-high illuminance signal by very-short-time accumulation are obtained from the pixel unit of the image sensor. Exposure is thus performed in three different exposure periods, and the images are composited.
That is, sampling a plurality of images by multi-shooting allows to obtain a very wide dynamic range. However, due to misalignment caused by the time lag between shots, degradation such as contour blur or pseudo-contour occurs in the resultant composite image.
Second, the one-shot sampling method by fixed-pattern sensitivity arrangement is known. This is a method that eliminates the time lag between different exposure shots and, for example, misalignment of a moving object. In this method, a plurality of different sensitivity pixels are provided on the sensor so as to acquire a plurality of kinds of exposure information by image capturing of one time. To specifically set the sensitivities, the fill factor or filter transmittance of each pixel is changed to make the different sensitivities, and the pixels with the different sensitivities are arranged in a fixed pattern. This method can improve misalignment caused by the time lag between a high sensitivity signal and a low sensitivity signal. However, since the high and low sensitivities are fixed, if the luminance range of a scene is wider than the dynamic range of image capturing at the low sensitivity, highlight detail loss may occur. That is, the dynamic range widening effect cannot be obtained in some cases. As another method, the values of the high and low sensitivities are set depending on the length of the exposure period, as described in Japanese Patent Laid-Open No. 2006-253876. In this case, the sensitivities can be set in accordance with the scene.
However, in the one-shot sampling method, a sensitivity difference of a fixed pattern is set on the sensor. Hence, the number of sampling points is smaller than in a conventional RGB sensor for both the high and low sensitivities. This lowers the resolution. Additionally, the fixed pattern is used independently of the object luminance. For this reason, noise increases in pixels corresponding to the low sensitivity.
The third method controls the exposure period in accordance with the object luminance. For example, as described in Japanese Patent Laid-Open 2007-532025 (corresponding to U.S. Pat. No. 7,023,369), an A/D converter and a comparator which compares a converted digital value with an external digital value are arranged for each pixel. This enables to detect the appropriate exposure amounts of the pixels without a single charge reading.
In this method, however, the charge amounts need to be compared every time in all pixels. It is therefore difficult to increase the number of pixels and the number of bits. In addition, charges generated in a photodiode always flow as a floating diffusion. Since noise generated there cannot always be removed, resistance against noise is poor.
As described above, it is difficult to obtain a high-quality HDR image by image capturing of one time, even with the first to third conventional methods. More specifically, to obtain a high-quality HDR image, it is necessary to acquire pieces of image capturing information by exposure of a plurality of levels at once, and suppress noise and a decrease in resolution by changing the sensitivity in accordance with the object luminance.
Controlling the exposure amount for each pixel may enable to obtain a high-quality image by image capturing of one time. In this case, however, the image quality may degrade at a boundary portion where the exposure amount changes. For example, if the exposure amount control value contains such an error that increases the exposure amount, the pixel will be saturated. This will cause information loss, for example, highlight detail loss or degradation such as pseudo-contour.