COPYRIGHT

A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.

BACKGROUND OF THE DISCLOSURE

Field of the Disclosure

The present disclosure relates generally to video image processing and in one exemplary aspect, to methods and apparatus for providing a frame packing arrangement for panoramic, 360° or virtual reality (VR) images.

Description of Related Art

Panoramic images (e.g., spherical images) are typically obtained by capturing multiple images with overlapping fields of view from different cameras and combining (“stitching”) these images together in order to provide, for example, a two-dimensional projection for use with modern display devices. Converting a panoramic image to a two-dimensional projection format can introduce some amount of distortion and/or affect the subsequent imaging data. However, two-dimensional projections are desirable for compatibility with existing image processing techniques and also for most user applications. In particular, many encoders and compression techniques assume traditional rectangular image formats.

Additionally, many extant image capturing devices that are configured to capture panoramic content have on-board stitching capabilities. For example, a twin-lens image capture device arranged in a so-called Janus configuration may output a fully stitched panoramic image. However, these image capturing devices are often times portable and may have limited processing resources. Accordingly, the stitching operations performed by these image capturing devices may output sub-optimally stitched images.

While techniques exist that enable the encoding/decoding of this so-called panoramic content, extant frame packing arrangement techniques for these panoramic images may not provide for the ability to improve upon on-board (i.e., on-camera) stitching. Accordingly, methods and apparatus are needed which allow for improved post-processed image stitching capabilities.

SUMMARY

The present disclosure satisfies the foregoing needs by providing, inter alia, methods and apparatus for providing a frame packing arrangement for panoramic images.

In one aspect, an encoder apparatus is disclosed. In one embodiment, the encoder apparatus includes a processor apparatus and a non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by the processor apparatus: receive a panoramic image that has been captured by two or more image capture devices, the panoramic image including an overlapping portion that has been captured by at least two of the two or more image capture devices; arrange non-overlapping portions of the panoramic image into a transport format; arrange the overlapping portion of the panoramic image within the transport format; and encode the arranged non-overlapping portions and the arranged overlapping portion of the panoramic image in order to produce an encoded image.

In one variant, the transport format includes a cubemap projection and the arrangement of the overlapping portion of the panoramic image includes arrangement of the overlapping portion of the panoramic image within a separate facet of the cubemap projection.

In another variant, the transport format includes a cubemap projection, the cubemap projection including a stitch line that bisects four imaging facets of the cubemap projection and the arrangement of the overlapping portion of the panoramic image includes an arrangement of the overlapping portion of the panoramic image between the bisected four imaging facets of the cubemap projection.

In yet another variant, the transport format includes a cubemap projection, the cubemap projection including a stitch line that bisects four imaging facets of the cubemap projection and the arrangement of the overlapping portion of the panoramic image includes an arrangement of the overlapping portion as a watermark within the four bisected imaging facets.

In yet another variant, the computer-readable instructions are further configured to, when executed by the processor apparatus: arrange the transport format into a 4×3 aspect ratio, the 4×3 aspect ratio including four facets in a width dimension and three facets in a vertical dimension.

In yet another variant, the arrangement of the overlapping portion of the panoramic image further includes an arrangement of the overlapping portion of the panoramic image within a single facet of the 4×3 aspect ratio.

In yet another variant, the arrangement of the overlapping portion of the panoramic image further includes an arrangement for the overlapping portion of the panoramic image such that the overlapping portion is arranged in a substantially contiguous manner.

In a second aspect, a decoder apparatus is disclosed. In one embodiment, the decoder apparatus includes a processor apparatus and a non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by the processor apparatus: decode faces with a stitch line and overlapping portions of a transport format; apply a full-stitching algorithm to the decoded faces and overlapping portions; decode untouched faces; and render an output for display.

In another embodiment, the decoder apparatus includes a processor apparatus and a non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by the processor apparatus: receive an encoded transport format associated with a panoramic image that includes overlap data; determine whether to discard the overlap data; if the determination is yes, render the transport format as a cut along a stitch line; and if the determination is no, fully-stitch the panoramic image using decoded overlap data.

In a third aspect, an encoding/decoding apparatus is disclosed. In one embodiment, the encoding/decoding apparatus is configured to encode and decode panoramic content that includes overlap imaging data.

In a fourth aspect, a method for encoding imaging data is disclosed. In one embodiment, the method includes receiving a panoramic image that has been captured by two or more image capture devices, the panoramic image including an overlapping portion that has been captured by at least two of the two or more image capture devices; arranging non-overlapping portions of the panoramic image into a transport format; arranging the overlapping portion of the panoramic image within the transport format; and encoding the arranged non-overlapping portions and the arranged overlapping portion of the panoramic image.

In one variant, the transport format includes a cubemap projection and the arranging of the overlapping portion of the panoramic image includes arranging the overlapping portion of the panoramic image within a separate facet of the cubemap projection.

In another variant, the transport format includes a cubemap projection, the cubemap projection including a stitch line that bisects four imaging facets of the cubemap projection and the arranging of the overlapping portion of the panoramic image includes arranging the overlapping portion of the panoramic image between the bisected four imaging facets of the cubemap projection.

In yet another variant, the transport format includes a cubemap projection, the cubemap projection including a stitch line that bisects four imaging facets of the cubemap projection and the arranging of the overlapping portion of the panoramic image includes arranging the overlapping portion as a watermark within the four bisected imaging facets.

In yet another variant, the method includes arranging the transport format in a 4×3 aspect ratio, the 4×3 aspect ratio including four facets in a width dimension and three facets in a vertical dimension.

In yet another variant, the method includes arranging the transport format in a 4×2 aspect ratio, the 4×2 aspect ratio including four facets in a width dimension and two facets in a vertical dimension.

In yet another variant, the arranging of the overlapping portion of the panoramic image further includes arranging the overlapping portion of the panoramic image within two facets of the 4×2 aspect ratio.

In yet another variant, the arranging of the overlapping portion of the panoramic image further includes arranging the overlapping portion of the panoramic image such that the overlapping portion is arranged in a substantially contiguous manner.

In a fifth aspect, a method for decoding imaging data is disclosed. In one embodiment, the method includes decoding faces with a stitch line and decode overlapping portions of a transport format; applying a full-stitching algorithm to the decoded faces and the decoded overlapping portions of the transport format; and rendering an output for display.

In a sixth aspect, a computer-readable storage apparatus is disclosed. In one embodiment, the computer-readable storage apparatus includes a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by a processor apparatus, to perform at least a portion of the aforementioned methodologies described herein.

In a seventh aspect, an integrated circuit (IC) apparatus is disclosed. In one embodiment, the IC apparatus includes logic configured to: receive a panoramic image that has been captured by two or more image capture devices, the panoramic image including an overlapping portion that has been captured by at least two of the two or more image capture devices; arrange non-overlapping portions of the panoramic image into a transport format; arrange the overlapping portion of the panoramic image within the transport format; and encode the arranged non-overlapping portions and the arranged overlapping portion of the panoramic image in order to produce an encoded image.

In an eighth aspect, a computing device is disclosed. In one embodiment, the computing device includes a processor apparatus and a non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by the processor apparatus: receive a panoramic image that has been captured by two or more image capture devices, the panoramic image including an overlapping portion that has been captured by at least two of the two or more image capture devices; arrange non-overlapping portions of the panoramic image into a transport format; and arrange the overlapping portion of the panoramic image within the transport format.

Other features and advantages of the present disclosure will immediately be recognized by persons of ordinary skill in the art with reference to the attached drawings and detailed description of exemplary implementations as given below.

