The present invention relates to a method and apparatus for interpolating missing lines of pixels from a video signal.
Broadcast signals today are sent and received as interlaced signals comprised of alternating fields of odd and even lines. An interlaced screen displays only one field at a time with a high (60 Hz. NTSC, 50 Hz. PAL/SECAM) temporal frequency. Due to its signal decay characteristics and the persistence of vision, the field perceived at the current time instant is overlaid onto the field perceived at the previous time instant in the human brain. These two fields, however, do not match temporally, and signal elements of the two signals may have moved between frames.
Interlaced to progressive conversion avoids this problem by interpolating the missing lines at each time instant from the lines of the received fields. Received and interpolated lines are simultaneously displayed on a progressive display screen. One method of interpolating a pixel of a missing line is to use the signal elements of the pixels vertically local to that pixel. Such a method of (intrafield) interpolation by lowpass estimation operates as a one-dimensional ("1-D") low-pass filter. Unless the vertical frequency of the signal is very low, this method severely limits the vertical resolution of the signal, and may cause undesirable spatial blurring of the vertical edges or temporal flicker.
Another solution would be to temporally interpolate the values of the pixels of missing lines. The simple method of temporal interpolation by adjacent field pixel substitution would use the same pixel elements from the preceding and following fields to interpolate the value of the missing pixel. Here, unless the temporal change in the signal is low, this method severely limits the resolution of the movement of the signal, but does not help increase the vertical resolution.
A combination of these two methods such as averaging their outputs can also be used. However, both the resolution of movement in the signal and the vertical resolution of the signal are deteriorated by this method.
Known motion adaptive conversion methods provide a means for hard or soft switching between intrafield low-pass estimation and adjacent field pixel substitution. This solution generally works well since pixel substitution preserves vertical detail for stationary regions in the video signal, and loss of fine vertical detail due to intrafield estimation is not noticeable for fast moving regions. However, neither intrafield low-pass estimation nor adjacent pixel substitution is suitable for the de-interlacing of slow moving regions in the video where the eye can detect the blur caused by the intrafield low-pass estimation.
Another method of temporal interpolation is referred to as motion compensated filtering which makes use of pixels in the preceding or succeeding frames and estimates the missing pixels by applying a low-pass filter along the direction of motion. The motion vector estimates used for determining the direction of motion need to be consistently accurate and those decoded in the systems of conventional coding standards, for transmission of digital video, are of limited use due to their lack of robustness in the presence of motion estimation errors.
U.S. Pat. No. 4,789,893 to Weston introduced a combined spatio-temporal interpolation technique. Weston interpolates a given missing pixel by combining the above mentioned intrafield low-pass estimate with high-pass estimates from temporally adjacent fields. For each missing pixel, there is a directly preceding pixel, and a directly following pixel in the opposite polarity adjacent field. High-pass estimates are obtained from the adjacent fields by vertically operating a 1-D high-pass filter centered at the directly preceding and directly following pixels. This leads to some loss in vertical high frequency detail for moving regions and some gain in vertical high frequency detail for stationary regions with respect to the intrafield low-pass estimate. One drawback with this method is that for those moving objects which the eye can track, there will be a discernable loss in vertical detail for those objects that have fine vertical high frequency detail.