The present invention relates to an image sensing apparatus suitable for using a sequential scanning type solid-state image sensing device having a color filter of so-called Bayer arrangement, and the like.
Recently, an image sensing device, such as CCD, capable of sequentially reading signals of all the pixels (referred as xe2x80x9cnon-interlace scanning type image sensing devicexe2x80x9d, hereinafter) has been developed with the progress of semiconductor manufacturing technique. The non-interlace scanning type image sensing device has an advantage in that a higher resolution image can be obtained with less blurring than an image sensed by using a conventional interlace scanning type image sensing device even when sensing a moving object. In the interlace scanning type image sensing device, a frame image is composed of two field images which are sensed at different times, usually at a field period interval. Accordingly, there is a problem in which, when sensing a moving object, there are notches on edges of the object and perhaps of the background in a frame image because of the time gap between the two field images composing a frame image. If a frame image is made of image data of a single field image to overcome the aforesaid problem, there would not be notches on edges, however, the vertical resolution of the obtained frame image is halved. In contrast, with a non-interlace scanning type image sensing device, it is possible to sense a frame image in a field period, thus, the non-interlace scanning type image sensing device is expected to be applied to a still image camera and a camera for EDTVII, for example.
A general image sensing apparatus using an image sensing device which outputs signals after adding two vertically adjacent pixel charges will be explained with reference to FIG. 22.
Referring to FIG. 22, an image sensing device 901 output signals after adding two vertically adjacent pixel charges in accordance with timing signals t1 and t2 generated by a timing signal generator (TG) 909. The output image signals are inputted to a correlated double sampling (CDS) circuit 903 via a buffer 902, and reset noises of the image sensing device 901 are removed from the output image signals by the CDS circuit 903., then the image signals enter an automatic gain controller (AGC) 904. In the AGC 904, the image signals are amplified by a gain set in accordance with a control signal c2 from a microcomputer 908 (gain control). The gain-controlled image signals are converted into digital signals by an analog-digital (A/D) converter 905, then transmitted to a camera processing circuit 906, where predetermined processes are applied to the digital image signals and a luminance signal Y and a color difference signal C are outputted. Further, the microcomputer 908 generates a control signal c2 for controlling the gain in the AGC 904 in accordance with the gain information c1 detected by a camera processing circuit 906.
Most of the non-interlace scanning type image sensing devices used at the present time are provided with R, G and B filter chips arranged in so-called Bayer arrangement as shown in FIG. 23. In this color arrangement, G signal is used as a luminance signal. Further, the non-interlace scanning type image sensing devices output image data of one frame in a field period, thus the speed for transferring charges is two times faster than the transferring speed of an image sensing device, as shown in FIG. 22, which outputs image signals obtained by adding two vertically adjacent pixel charges. Accordingly, it is preferred to design an image sensing device to have two horizontal registers which respectively transfer image signals of odd and even lines simultaneously to be first and second output signals, instead of transferring by one line through a single horizontal register.
In this case, the buffer 902, the CDS circuit 903, the AGC 904, and the A/D converter 905 shown in FIG. 22 are needed for each of the two horizontal registers. In a case where image signals are obtained from an image sensing device provided with a color filter of Bayer arrangement as shown in FIG. 23, the G signals are obtained from all the corresponding pixels by horizontally interpolating by using signals obtained at G filter chips (i.e., G signals) as shown in FIG. 24A. The G signals obtained as above are converted into luminance signals to be displayed. However, if image signals to be displayed are obtained in this manner, difference in characteristic between the two AGCs will affect the quality of an image.
The AGCs change gains to be applied to image signals when a gain is provided, and the two AGCs have different characteristics from each other in general. Therefore, even though the same gain is provided to the two AGCs, the levels of amplified image signals may differ from each other. If this occurs, when an object of a uniform color (e.g., a white paper) is sensed, a variation in output signal level of the two AGCs appears in a stripe pattern of odd and even lines. Therefore, when an image of the object is displayed, the variation in output signal level appears as the difference in output signal level between odd and even line fields on a display as shown in FIG. 24B, which causes field flicker. This noticeably deteriorates the quality of the image.
To overcome this problem, a method for interpolating an average of pixel values of the G signals in a vertical row can be considered. However, in this method, when an object of a uniform color (e.g., a white paper) is sensed, a variation in output signal level of the two AGCs may cause vertical stripes which alternatively have different output levels on a display as shown in FIG. 24C. The difference in output level in the vertical stripes also noticeably deteriorates the quality of an image.
Further, in order to compensate a variation in output signal level of the two AGCs, feed-back control is considered. Feed-back control can be performed in the following sequence, for example. First, a test signal is inputted to the two horizontal registers of the image sensing device at predetermined timing, then the test signals outputted from the two horizontal registers are processed in the same manner of processing image signals. Then, the difference in output level between the two AGCs is detected. On the basis of the detected difference in output level, new gains to be provided to the AGCs are adjusted. Thereafter, a test signal is inputted to the horizontal registers again so as to confirm that the difference in output level between two AGCs are corrected.
As described above, if there is a variation in characteristic between two AGCs, it is possible to compensate a variation in output signal level between the two AGCs by performing feed-back control.
However, in the aforesaid method, a problem in which the size of a hardware increases is posed in providing a feed-back control function for compensating a variation in output signal level between two AGCs, since a circuit necessary for performing the feed-back control has to be added.
Further, because of an arrangement of pixels for luminance signals (i.e., pixels with xe2x80x9cGxe2x80x9d filter chips) as shown in FIG. 23, the following problem arises. FIG. 25 is a graph showing a spatial frequency plane. In FIG. 25, the abscissa denotes a spatial frequency in the horizontal direction, the ordinate denotes a spatial frequency in the vertical direction, and the oblique axis denotes the spatial frequency in the oblique direction. In FIG. 25, the spatial frequencies at ∘ marks are the spatial sampling frequency decided in accordance with the number of pixels, and so on, of CCD (the spatial sampling frequencies in the horizontal, vertical and oblique directions are respectively referred as xe2x80x9cfs-hxe2x80x9d, xe2x80x9cfs-vxe2x80x9d, and xe2x80x9cfs-oxe2x80x9d, hereinafter). When signal components having these sampling frequencies are sampled, the obtained signals appear as a direct current component signal, which gives bad effects on an image. Therefore, it is necessary to remove signal components of the sampling frequencies from the image.
As a method for removing such the frequency components from the image, an optical crystal low-pass filter (referred as xe2x80x9cO-LPFxe2x80x9d, hereinafter) is generally used. More specifically, with an O-LPF designed to have null points at spatial frequencies shown by three ∘ in FIG. 25 in the oblique direction, it is possible to effectively remove sampling frequency components not only in the oblique direction but also in the horizontal and vertical directions.
However, moire due to signal components, shown by ⊚ in FIG. 25, having Nyquist frequencies which are half of the sampling frequencies (i.e., xc2xd(fs-h), xc2xd(fs-v), and xc2xd(fs-o)) also appear on an image, which deteriorates the image. Especially, the moirxc3xa9 which appears in the oblique direction has very large component signals, and it appears even more clearly when displaying a moving image and an image obtained while panning a camera. However, the aforesaid O-LPF can not remove such moirxc3xa9.
In order to overcome this problem, it is possible to use an O-LPF designed to have null points at spatial frequencies shown by ⊚ in FIG. 25 in the oblique direction, similarly to the O-LPF for removing signal components of sampling frequencies, to remove signal components of Nyquist frequencies in the horizontal, vertical and oblique directions. However, in this methods, a bandwidth of luminance signals obtained when an image is formed on the non-interlace type image sensing device becomes narrow. Furthermore, when another filter is used, response in the bandwidth drops dramatically. Thus, a resultant image will not be expressed sharply.
In addition, an image sensed by the aforesaid non-interlace scanning type image sensing device can be displayed both by a non-interlace method and by an interlace method. Accordingly, as an image sensing apparatus having a non-interlace scanning type image sensing device, one which can switch between outputting an image for a interlace type display and outputting an image for a non-interlace type display is demanded.
Further, there is a demand for inputting photograph data to a personal computer. Especially, there is a demand for taking an image into the personal computer from a negative film since images sensed by using a camera are often stored in negative films.
The present invention has been made in consideration of above situation, and has as its object to effectively prevent deterioration of the quality of an image due to a variation in amplification characteristic in each line upon reading image signals by a plurality of lines and due to an arrangement of a color filter.
Further, it is another object to prevent deterioration of the quality of an image due to a variation in amplification characteristic in each line upon reading image signals by a plurality of lines and due to an arrangement of a color filter by using a simple circuit configuration without adding a feed-back type correction circuit.
According to a preferred configuration of the present invention, each color signals obtained by separating image signals which are read from an image sensing device are processed by a properly designed two-dimensional filter. With this two-dimensional filter, it is possible to remove signal components of Nyquist frequencies in the horizontal, vertical and oblique directions, thereby harmful signals due to a variation in amplification characteristic (gain), which appears as a variation of luminance signals having Nyquist frequencies, corresponding to each of a plurality of horizontal registers of the image sensing device and due to an arrangement of a color filter are effectively removed. As a result, the quality of an image is improved.
It is still another object of the present invention to enable to switch between image signal outputs conforming to a plurality of display methods in an image sensing apparatus.
Further, it is still another object of the present invention to enable to change characteristics of image signals to be outputted depending upon the state of an object and the type of output device. More specifically, it is an object to provide an image sensing apparatus capable of setting characteristics of edge enhancement signal frequencies in accordance with a state of an output device.
Furthermore, it is still another object of the present invention to reduce the size of a circuit for negative-positive inversion as well as to provide an image signal control circuit for achieving a sufficient white balance adjustment.
Further, it is still another object of the present invention to realize a circuit which reads and processes image signals from a non-interlace scanning type image sensing device by inputting two alternate scan lines so as to be able to switch between different signal output formats (e.g., interlace output and non-interlace output) with a simple circuit configuration.