Internet streaming is the art of transmitting digital media in such a way that the media can be played at the receiver side as it is being received, without the requirement of saving a media file to disk. Streaming enables live broadcast in the digital world, similar to radio and television broadcasts in the analog world.
Conventionally, streaming has been used for time-based media, including digital audio and digital video. A user can listen to music generated from audio data and watch movies generated from video data, as the data is being received by his client computer from a server computer, without the need to save an audio file or a video file on the client computer. As additional data is received at the client, the music and video being played advance in time.
Image streaming is a new paradigm for non-time based media, useful for images and other media that can be interactively viewed and that require large amounts of data for storage. Although an image is a single frame, and not a time-ordered sequence, nevertheless there is great advantage in being able to interactively view the image as image data is being received, without the need to save the image data into a file.
A high-quality raster image generally cannot be displayed at its full resolution within a computer display screen. Instead, either a low-resolution version of the image can be displayed in full, or else a high-resolution version can be displayed in part. A user can interactively view different portions of the image by zooming in to a smaller portion of the image at a higher resolution, by zooming out to a larger portion of the image at a lower resolution, or by panning left, right, up and down within a fixed resolution.
Image streaming involves sending image data from a server computer to a client computer in response to a user's interactive viewing requests, in such a way that the client has the necessary data to display the currently requested portion of the image at the currently requested resolution. In distinction from audio and video streaming, which generally push data to the client in a time sequential order, image streaming generally pulls data from the server in an “on demand” order, based on individual user navigation. Thus, while a first user may navigate through the image in his own sequential order, a second user may navigate in a different order. In each case, “just enough data” is streamed to each user in order to satisfy each user's interactive requests.
For raster images, image streaming is implemented by generating versions of a raster image at various resolutions, and partitioning the image into rectangular local image tile regions at each generated resolution. A user request for a specific portion of the image at a specific resolution is satisfied by identifying and transmitting one or more appropriate local image tile regions.
Image streaming can be implemented using various architectures, including server-side rendering, client-side rendering, and hybrid server-side and client-side rendering. Server-side rendering requires the server to assemble together the appropriate local image tile regions, and crop and re-scale the assembled image as appropriate, to generate a single JPEG image, for each interactive user request. An advantage of server-side rendering is that image streaming is enabled for any client computer that has a conventional web browser running; i.e., it does not require special purpose software on the client side. Client-side rendering shifts the image processing burden from the server to the client. The appropriate local image tile regions are transmitted from the server to the client as raw unprocessed data, and the client does the processing work to assemble the local image tile regions. An advantage of client-side rendering is faster performance at the client.
A significant distinction between conventional time-based audio and video streaming and between non-time based image streaming, is recurrence of data that is rendered. Specifically, audio and video sequences are generally rendered in a one-pass forward play order, and the same data is generally not rendered more than once. Image navigation, however, is characteristically recurrent. While zooming in and out, and panning left, right, up and down, a user often returns to the same data over and over again. Typically, the same local image tile regions are rendered multiple times while a user is navigating through an image. As such, image streaming performance benefits greatly from tile caching.
U.S. Pat. No. 5,968,120 to Guedalia describes image streaming by transmitting local image tiles. U.S. Pat. Nos. 6,121,970 and 6,356,283 to Guedalia describe embodiments of server-side image streaming that operate by modifying references to image files within HTML pages, using the Internet Imaging Protocol.
U.S. Pat. No. 6,536,043 to Guedalia and U.S. Pat. No. 6,721,952 to Guedalia et al. describe progressive image streaming, in which successive chunks of image data are transmitted, each successive chunk serving to upgrade the quality of the rendered image.
U.S. Pat. No. 6,745,226 to Guedalia describes a push-pull method and system for transmitting image data, in which data for a low quality version of an image is pushed to a user's desktop, and data for upgrading the quality of the image is pulled on demand by a user at will.
With the advent of wireless devices and the cost of transmission time for large digital files, it has become more practical to distribute vector images for large graphics, instead of raster images. Vector images are generally comprised of vector primitives, also referred to as vector objects, such as points, lines and areas, and raster images are generally comprised of arrays of pixel color values. Vector objects are typically specified by attributes, including inter alia attributes for a color, a style and one or more control points. As such, for large graphics, vector image files are typically much smaller than their counterpart raster image files.
Moreover, vector images are significant for maps that can be viewed at a variety of scales, such as maps displayed by GPS systems.
Vector images are also significant for rich text format documents, since true type fonts are generally represented as vector objects. Many of today's wireless devices, such as cellular phones, do not have the capability to render rich text documents, such as WORD and PDF documents. Instead, the documents are converted to plain ASCII text. For example, if a user receives e-mail with a WORD document attachment on his cell phone, the WORD document is generally displayed in plain text.
The lack of capability to render rich text documents in wireless devices is not due to inherent limitations in wireless communication or wireless device processors. Instead, the primary reason for not rendering rich text documents is the drawback of the download time required to receive such a document over the air and save it to memory. Clearly image streaming should be used to mitigate such drawback. Image streaming should enable a client device to render portions of a document on demand, as data for the document is being received at the client, without the need to save the entire document to memory.
However, conventional image streaming applications only stream raster images, and not vector images. Using conventional technology, vector images have to be first converted to raster images before they can be streamed from a server computer to a client computer. Such conversion clearly eliminates the size advantage of vector images over raster images.
There is thus a need for an application that streams vector images without conversion to raster images, so that vector images can be interactively viewed by a user on demand.