Integrated circuit technology has revolutionized various fields, including computers, control systems, telecommunications, and imaging. One field in which integrated circuitry is widely used is video imaging. Different types of semiconductor imagers include: charge coupled devices, photodiode arrays, charge injection devices, and hybrid focal plane arrays. Many of these devices include pixels that are arranged in sensor arrays to convert light images into electrical signals.
Examples of MOS imaging devices are detailed in “A ¼ Inch Format 250K Pixel Amplified MOS Image Sensor Using CMOS Process” by Kawashima et al., IEDM 93–575 (1993), and “A Low Noise Line-Amplified MOS Imaging Devices” by Ozaki et al., IEEE Transactions on Electron Devices, Vol. 38, No. 5, May 1991. In addition, U.S. Pat. No. 5,345,266 to Denyer, titled “Matrix Array Image Sensor Chip,” describes a MOS image sensor. The devices disclosed in these publications provide a general design approach to MOS imaging devices. In addition, MOS approaches to color imaging devices are described in “Color Filters and Processing Alternatives for One-Chip Cameras,” by Paruiski, IEEE Transactions on Electron Devices, Vol. ED-32, No. 8, August 1985, and “Single-Chip Color Cameras With Reduced Aliasing” by Imaide et al., Journal of Imaging Technology, Vol. 12, No. 5, October 1986, pp. 258–260.
Image sensor circuitry generally includes circuits for performing black level calibration and automatic gain control. Black level calibration attempts to eliminate the portion of the image signal that exists when no light is being detected, thus allowing for a truer zero reference signal so that the later signal processing is improved. After the black level calibration has been performed, automatic gain control amplifies the video signal at a controlled level so as to utilize more of the available signal amplification range. The combination of the black level calibration and automatic gain control can be said to form one type of an “auto black expansion” method. One prior art circuit that addresses issues related to these processes is shown in U.S. Pat. No. 4,187,519 to Vitols et al. FIGS. 1, 2, 3, 4, 5, 6 and 7 of Vitols et al. have been reproduced herein as FIGS. 1, 2A, 2B, 2C, 2D, 2E, and 2F, respectively.
Vitols et al. disclose a circuit that is designed to provide a video contrast expansion system. As illustrated in FIG. 1, a sequence of video pixel data representative of an image is developed by a signal source 11, which may be, for example, a radar, an IR (infrared) sensor or a TV (television) camera. This pixel data is applied to an I-getter (intensity getter) 13 and to a delay circuit 19. An initialize command signal from the source 11 initializes the operation of the I-getter 13 before or during the time that the sequence of pixels is being generated by the source 11.
The I-getter 13 searches the input sequence of pixels (designated as new data or ND) to determine bias and gain intensity parameters. A bias function generator 15 is automatically adjusted by the bias parameters to develop pixel bias correction signals. In a similar manner a gain function generator 17, which is similar in structure and operation to the generator 15, is automatically adjusted by the gain parameters to develop pixel correction signals.
In its operation the I-getter 13 reduces and transforms the input sequence of pixels into a reduced number of bias parameters and a reduced number of gain parameters. Each of the function generators 15 and 17 smoothly fills in or shades in its associated widely spaced apart parameters (bias or gain) by double linear interpolations to produce its associated correction signals (bias or gain). Each sequence of correction signals is equal to the number of pixels in the input sequence of pixels.
It takes time for each of the function generators 15 and 17 to get sufficient parameter data in before it can start computing its associated correction signals. Furthermore, it takes additional time before each of the function generators 15 and 17 compute its associated correction signals. This combined delay time is offset by the delay circuit 19 which synchronizes the time of occurrence of the sequence of pixels at the output of the delay circuit 19 with the time of generation of the pixel bias correction signals and pixel gain correction signals.
The computed pixel bias correction signals are respectively subtracted in a combiner or subtractor 21 from the delayed or synchronized sequence of pixels to selectively lower the minimum values in associated groups of pixels in the delayed sequence of pixels to very close to zero. The minimized output of the combiner 21 is respectively multiplied in a multiplier or AGC (automatic gain control) circuit 23 to selectively expand the maximum amplitudes in the associated groups of pixels at the output of the combiner 21 to near the saturation level of the electronics of the system of FIG. 1. The output of the AGC circuit 23 can be applied to a display generator (not shown) to generate an enhanced picture of the image in which the video contrast of the image has been substantially expanded in two directions (minimum and maximum values of contrast).
The pixel data at the output of the signal source 11 may have a very narrow dynamic range of contrast, making it extremely difficult to discern objects in areas of lower contrast if directly viewed at the point. However, by the operation just explained, the pixel data at the output of the AGC circuit 23 has its dynamic range of contrast selectively expanded or stretched from near minimum to near maximum, making objects originally in low areas of contrast now clearly discernible when displayed.
FIGS. 2A–2F illustrate some of the possible operations that the circuit of FIG. 1 can perform on a one-dimensional signal. Such operations are analogous to those performed on a two-dimensional signal since a two-dimensional signal is essentially comprised of a vertical plurality of horizontal one-dimensional signals.
FIG. 2A illustrates an exemplary one-dimensional signal I(x) comprised of a sequence of 256 pixels. Although I(x) is shown as an analog signal it is a sequence of 256 intensity (I) points of varying amplitudes. Note that I(x) varies over a fairly wide intensity range from an amplitude of 630 (Isat or the intensity saturation level of the system) to an amplitude of about 50 at the low end.
It may be desired in some applications to look for small, rather than large, amplitude deviations from nominal. For example, if the intensity variations in the incoming signals I(x) of FIG. 2 represent pixels or intensity points of a scene, very high intensity portions of I(x) would ultimately develop very bright portions in a picture, and very low intensity portions of I(x) would ultimately develop very dark portions of the picture. However, a human observer or operator may not necessarily be interested in very high or very low intensity levels in the incoming signal I(x). Rather, an observer may only be interested in small variations of intensity in I(x), whether contained in very high, very low and/or intermediate levels of intensity in I(x). This is due to the fact that the small variations of intensity in I(x) can define the details of a scene or image sufficiently to possibly enable an observer to identify what is happening or contained in the picture.
By using a conventional contrast enhancement technique, an observer may be unable to discern what is contained in picture areas of small intensity variations. It is to the correction of this problem that the circuit of FIG. 1 is directed. To illustrate, assume that the low intensity region of pixels between 100 and 200 in FIG. 2 represents the signal that is coming from the inside of a dark cave. The brightest pixel in the 100–200 region of pixels is dark in comparison with the other high intensity or bright pixels in FIG. 2. An observer may be looking for the glint of light on a gun barrel inside that cave. If the dynamics of the signal in that 100–200 pixel region remain unchanged, an observer would not be able to see that glint of light on that gun barrel. However, if the gain of the signal in the 100–200 pixel region (where the amplitude of the signal is very low) could be increased so that variations of the signal show up to substantially the fullest extent possible without saturating the electronics of FIG. 1, the human observer could more readily determine what was inside the cave. It should be noted that a resultant picture would not look like a normal picture since the inside of the cave would be just as bright as what surrounds the cave on the outside. However, the intent of the circuit of FIG. 1 is to brighten everything to the fullest dynamic range, so that an observer can see what is happening or contained in a resultant picture.
FIG. 2B illustrates a possible guide as to what may be done to increase the dynamic range of I(x) of FIG. 2A. As shown, FIG. 2B illustrates a plurality of eight local data regions, each 32 pixels in length, which respectively encompass contiguous 32-pixel long portions of I(x). In each data region the brightest (or largest intensity) and darkest (or smallest intensity) parts of the associated portion of I(x) are determined. The largest and smallest intensity values found in each data region are used to form horizontal ceiling (C) and floor (F) values for each pixel across that region. Thus, the sequence of ceiling values found in the respective data regions forms a ceiling function of X values, designated C(x), while the sequence of floor values found in the respective data regions forms a floor function of X values, designated F(x), for the included portions of I(x). FIG. 2C shows just F(x) the floor function of FIG. 2B. As shown, F(x) is comprised of the segmented sequence of minimum values across the respective regions of FIG. 2B.
As described, a first possible operation that can be performed to maximize the dynamic range of I(x) is to subtract F(x) from I(x) to produce the wave form shown in FIG. 2D. It can be seen in FIG. 2D that the amplitude of the signal I(x) of FIG. 2A is substantially reduced, while still substantially retaining the intensity variations of FIG. 2A.
As also described, the next possible operation that can be performed is to expand the amplitude of the signal I(x)–F(x) of FIG. 2D by adjusting the gain in the first data region (0–32), second data region (32–64), third data region (64–96) etc.—each gain adjustment being independent of the others-so that the maximum value of the signal in each data region go all the way up to the maximum permissible level or saturation level (Isat) FIG. 2E illustrates a piecewise constant gain function G(x) that would independently adjust the amplitude in each segment or data region up to the saturation level. This piecewise constant gain function would be determined by the value of the saturation intensity (Isat) divided by the difference between the C(x) and F(x) functions.
FIG. 2F illustrates the result of multiplying the function of FIG. 2D by the gain function of FIG. 2E. This multiplication raises the maximum value in each data region up to the saturation level. However, Vitols et al. describe FIG. 2F as being very unsatisfactory because of the discontinuities or, places where the amplitude of the signal rises or falls vertically. Although the full dynamic range of the signal is obtained in each data region, the resultant signal shown in FIG. 2F is very bumpy. So the overall information that an operator may be looking for in the signal may be totally lost in the discontinuities. If the one-dimensional wave form of FIG. 2F were applied to a two-dimensional picture, a very strong checkerboard pattern would result, which would distort the operator's perception of a picture or image to the point where he probably could not discern what he was looking at. Vitols et al. go on to explain how the discontinuities shown in FIG. 2F can be eliminated by smoothing the signal functions F(x) and G(x) from one data region to another. This smoothing operation is described further in the Vitols et al. patent, but will not be discussed further herein.
One of the drawbacks of the Vitols et al. circuit is that it requires a complex sequence of operations to determine the desired adjustment functions for the system. As described, the I-getter (as illustrated in FIG. 12 of Vitols et al.) requires two shift registers for every data cell used in an output picture. In addition, a large number of switching points capable of switching out each of the data cells for separate comparator operations must be used. In addition, during the comparator operations the comparator voltage level to which the incoming pixel signals are compared may be constantly shifting. In addition, as further described in Vitols et al., it takes time for each of the function generators to get sufficient parameter data in before the function generators can start computing the associated correction signals. Furthermore, it is also stated that it takes additional time before each of the function generators can compute its associated correction signals. This combined delay time must be offset by a delay circuit which synchronizes the time of occurrence of the sequence of pixels with the time of generation of the pixel bias correction signals and pixel gain correction signals.
The present invention is directed to a circuit that overcomes the foregoing and other problems in the prior art. More specifically, the present invention is directed to an auto black expansion method and apparatus for an image sensor that uses a simplified digital control system and does not require additional shift registers for the pixel signals or continual shifting of input comparator levels during a given field.