FIELD

The present disclosure generally relates to methods and apparatus for efficiently storing image data and more particularly to compressing image data that contains depth information.

BACKGROUND

For computer-generated imagery or computer-enhanced imagery, whether individual images or video sequences of images sometimes referred to as frames, might be stored in computer memory such that computer systems, such as animation creation systems, renderers, projectors, etc. can access that imagery and operate on it or present it. In a basic representation, an image is stored in a form of a two-dimensional (2D) pixel array where each element or sample of the pixel array corresponds to a position on the image and has a pixel value. The value is intended to capture a quantity of interest at such a location, for example, a color or other pixel component value.

An image management system, as might be part of an animation creation system, might generate a pixel array with more information per pixel than just an ultimate pixel color value that might be displayed for a given pixel, or a pixel array might even omit color values and store other values on a per pixel basis. A “deep image” might refer to a stored representation of an image that includes pixel values for different depths of a pixel. With such a deep image representation, instead of just a pixel value for a pixel area of the camera viewing frame, the deep image representation might include a plurality of pixel values, with different ones of the plurality representing pixel values at different depths.

An image might be computer-generated from a scene description that describes virtual objects, lighting, effects, a camera position, a camera orientation, a camera viewing frame, etc. in a three-dimensional (3D) virtual space. An image generator, such as a renderer, might compute pixel color values based on which objects are intersected by rays from the camera position through pixel areas of the camera viewing frame. In an example, a scene description might specify the positions, shapes, texture, and colors of various objects and a renderer could generate an image of what the scene would look like from a specified camera position through a specified camera viewing frame. Such a generated image might not show a far object if that far object is further from the camera position than a near object and both the far object and the near object are intersected by a viewing ray from the camera position through a pixel or pixels of the viewing frame.

A deep image representing a scene might occupy a large amount of computer memory if there are a large number of pixels, a large number of possible pixel values, and a large number of objects present in a scene description for which the deep image is generated. This can be addressed by flattening the deep image to preserve only a resulting pixel value from contributions of objects at different depths along each pixel ray. However, once flattened, the image might not contain sufficient information to allow for editing the image to remove or add objects without introducing undesirable artifacts.

A method and apparatus for compressing a deep image while retaining structure that allows for easy editing of the deep image is desirable.

SUMMARY

A computer-implemented method might be provided for compressing image data that includes depth information, the method comprising under the control of one or more computer systems configured with executable instructions, obtaining an image dataset in computer-readable form, wherein image data in the image dataset comprises a plurality of pixel image value arrays, wherein a first pixel image value array having a first number of image value array samples each having an image value, a depth value, and an association with an associated pixel position, determining, for the first number of image value array samples, a compressed image, determine, for the first number of image value array samples, a compressed image value array comprising a second number of compressed image value array samples, wherein the second number is less than or equal to the first number and wherein compressed image value array samples are computed from the first number of image value array samples and combining criteria, wherein the combining criteria are based on object data and/or depth variations of depth values in the first pixel image value array, and generating a modified image dataset wherein the first pixel image value array represented in the image dataset by the first number of image value array samples is represented in the modified image dataset by the second number of compressed image value array samples.

An image value array sample might include an object identifier, wherein an object identifier is an indication of an object that is associated with the pixel image value array sample and that provides the image value and depth value for the image value array sample. Having object identifiers can allow for a compressor or a decompressor to process data based on object identifiers, such as combining samples during compression when their object identifiers indicate that all of the samples are from one object. For example, the combining criteria for combining a first plurality of uncompressed samples of the first pixel image value array by a compressed sample in the compressed image value array might include whether each of the first plurality of uncompressed samples share an object identifier in common.

Where object identifiers are present, the combining criteria may be configured to combine adjacent samples if they have the same object identifier. However, in some cases, that might over-compress an image. If object identifiers are present, and required, the criteria might be to combine adjacent samples that are within an error threshold and share a common object identifier.

Image values of samples of the first number of image value array samples might comprise pixel values, wherein the compressed image value array samples comprise pixel values, and wherein the image dataset and the modified image dataset comprise pixel values for pixels over an image. A pixel value might comprise one or more pixel color value.

The compressed image value array might be computed from the first number of image value array samples based on depth values associated with the pixel image value array samples, and wherein the combining criteria for combining a first plurality of uncompressed samples of the first pixel image value array by a compressed sample in the compressed image value array includes whether each of the first plurality of uncompressed samples have a depth value within a predefined limited depth range threshold.

The combining criteria might further comprise determining that each object represented by samples within the predefined limited depth range threshold all share the object identifier in common.

The image dataset might comprise data for a two-dimensional pixel array of R rows and C columns, R and C being positive integers, wherein the image data comprises R times C pixel image value arrays, one per pixel, wherein a number of image value array samples in image value arrays may vary, wherein each image value array sample comprises a pixel component value, an alpha value, a depth value or depth range, and each image value array sample is associated with one of the R times C pixels of the image data, and wherein the modified image dataset comprises R times C (or more or fewer compressed image value arrays), wherein at least some of the pixel image value arrays are stored in a compressed version with fewer image value array samples relative to an uncompressed version.

The method might further comprise omitting at least one compressed image value array corresponding to a pixel not intersected by any objects.

The first pixel image value array might comprise array samples each having depth information that indicates, for its associated pixel, one or more contributing object that contribute to a color value of the associated pixel, wherein the contributing object is represented by a depth or a depth range.

The contributing object might be contributing to the first pixel color when a position of the contributing object intersects or is within a ray or volume defined by bounds rays from a camera position through a corresponding pixel.

The method might further comprise determining an error threshold between a second pixel value function derived from the compressed image value array and the first pixel image value array, adding an additional sample to the compressed image value array to reduce an error below the error threshold, determining whether modifying a depth range of an object in the first pixel image value array would reduce an error between a second pixel value function derived from the compressed image value array and the first pixel image value array, determining whether the error exceeds an error threshold, modifying the depth range to extend a boundary of adjacent objects to reduce a gap between the adjacent objects that when combined separately from the gap would reduce the error, and/or truncating the compressed image value array at a truncation sample representing an object that fully occludes farther objects at a pixel location of the compressed image value array.

A computer system comprising one or more processors and a storage medium storing instructions, which when executed by the at least one processor, might cause the system to implement methods described herein.

A non-transitory computer-readable storage medium storing instructions, which when executed by at least one processor of a computer system, might cause the computer system to carry out methods described herein.

A computer-readable medium carrying instructions, which when executed by at least one processor of a computer system, might cause the computer system to carry out methods described herein.

A carrier medium might carry image data that includes depth information compressed according to methods described herein.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A more extensive presentation of features, details, utilities, and advantages of the surface computation method, as defined in the claims, is provided in the following written description of various embodiments of the disclosure and illustrated in the accompanying drawings.

