1. Field of the Invention
The present invention relates to the structure of three-dimensional image data, a method of recording three-dimensional image data, and a method of displaying and reproducing three-dimensional image data, and in particular, to the structure of three-dimensional image data, a method of recording three-dimensional image data, and a method of displaying and reproducing three-dimensional image data, wherein three-dimensional display images suitable for compression are recorded.
2. Description of the Related Art
Various schemes are known for three-dimensional image display apparatuses that are able to display motion pictures three-dimensionally, that is, three-dimensional displays. In recent years, in particular, three-dimensional image display apparatuses of a flat panel type have been highly demanded which are based on a scheme not requiring any dedicated type of glass or the like. For flat panel type display apparatuses such as direct-vision or projection type liquid crystal display apparatuses and plasma display apparatuses whose pixels have fixed position in display surfaces, three-dimensional image display apparatuses can be easily implemented on the basis of the following scheme: a parallax barrier is installed immediately in front of the display panel to controllably direct rays from the display panel toward a viewer.
Light rays are controlled so that even when observing the same position on the parallax barrier, the viewer views different images depending on the angle. Specifically, a slit array or lenticular sheet, that is, a cylindrical lens array, is used to provide only lateral parallaxes, that is, horizontal parallaxes, or a pinhole array or a lens array is used to also provide vertical parallaxes. The scheme using the parallax barrier is further classified into a two-view (binocular) scheme, a multiview scheme, a super-multiview scheme that is a multiview scheme provided with super-multiview conditions, and integral photography (hereinafter simply referred to as IP). The basic principle of these schemes is substantially the same as that which was invented about 100 years ago and has been used for three-dimensional photography since then.
With either the IP scheme or multiview scheme, viewing distance is normally finite. Consequently, a display image is created so that a perspective image is actually viewed at that viewing distance. An IP scheme providing only horizontal parallaxes and no vertical parallaxes is disclosed in SID04 Digest 1438 (2004). This one-dimensional IP scheme generates a set of parallel rays if the horizontal pitch of the parallax barrier is set equal to an integral multiple (n times) of the horizontal pitch of the pixels (the one-dimensional IP scheme is hereinafter also referred to as parallel-ray one-dimensional IP). Accordingly, a parallax component image in which pixel columns constituting a set of parallel rays are accumulated is obtained by perspective projection for a given viewing distance in the vertical distance and by orthographic projection in the horizontal direction. A parallax synthetic image (an elemental image array) is created by dividing each parallax component image obtained by the perspective projection in the vertical distance and by the orthographic projection in the horizontal direction, into pixel columns and interleaving and synthesizing. When the parallax image synthetic image is displayed on a display surface and observed through the parallax barrier, the correct projection, that is, a three-dimensional image based on perspective projection in both horizontal and vertical directions, is obtained. Specific methods are disclosed in SID04 Digest 1438 (2004), Jpn. Pat. Appln. KOKAI Publication Nos. 2003-90738 and 2003-315356, and the like. The multiview scheme divides an image obtained by a simple perspective projection, into pixel columns and interleaving and synthesizing to obtain a three-dimensional image based on the correct projection.
It is difficult to implement an image pickup apparatus that varies the projection method or a projection center distance between the vertical direction and the horizontal direction. This is because such an apparatus requires a camera or lens of the same size as that of a subject for orthographic projection. Accordingly, a practical method of obtaining orthographic projection data by image pickup is to convert perspective projection image pickup data into orthographic projection data. A known practical method is a ray space method based on interpolation and using EPI (epipolar surface).
The parallel-ray one-dimensional IP scheme is advantageous in that an image obtained by this scheme is easier to see than one obtained by the binocular scheme. However, the parallel-ray one-dimensional IP scheme requires complex image formats for the projection method and the dividing and arranging method. The binocular and multiview schemes are three-dimensional image display schemes that display the simplest three-dimensional images, and thus use simple image formats. With these schemes, each parallax image is formed of the same number of vertical pixels and the same number of horizontal pixels. For example, two parallax component images for the binocular scheme or nine parallax component images for the nine view scheme may be divided into pixel columns, which may then be synthesized into a parallax synthetic image in an image format in which the image can be displayed on the display surface. However, the parallel-ray one-dimensional IP scheme requires a larger number of parallax component images than the multiview scheme, which achieves substantially the same resolution. With the parallel-ray one-dimensional IP scheme, the number of horizontal pixels (horizontal range used to display three-dimensional images) in each parallax component image varies depending on parallax direction, resulting in complex image formats.
Both the multiview scheme and parallel-ray one-dimensional IP scheme pose a problem if each piece of parallax information is divided into subpixels assigned to a parallax synthetic image. When a parallax synthetic image is irreversibly compressed by an encoding method such as JPEG or MPEG, parallax information may be mixed together, and decompression may degrade image quality. In particular, if the color arrangement of color filters or the like are mosaic arrangement in order to avoid a moire effect, color information may also be mixed together, and decompression may further degrade image quality. Reversible compression avoids the degradation but offers a compression rate considerably lower than that achieved by the irreversible compression. A method exists which allows each parallax component image to be irreversibly compressed and decompressed. This method can be used with the multiview scheme but is inefficient with the parallel-ray one-dimensional IP scheme because it involves a large number of parallax component images and a varying number of horizontal pixels.
As described above, the conventional three-dimensional image recording method based on the parallel-ray one-dimensional IP scheme is disadvantageous in that it achieves a considerably low compression rate and may involve degradation of image quality as a result of irreversible compression and decompression.