The International Organization for Standardization is currently developing a standard specifying the coded representation of video for digital storage media, supporting a continuous data transfer rate of 1.5 Mbits/sec., which standard is described in the document ISO-IEC JTC1/SC2/WG11; CODING OF MOVING PICTURES AND ASSOCIATED AUDIO; MPEG90/176 Rev. 2, Dec. 18, 1990. This format has become known as MPEG. According to this format sequences of frames are divided into groups, and respective frames within each group are encoded according to one of a plurality of coding modes. Typically the coding modes include intraframe coding, (I frames) and two types of interframe predictive coding (P and B frames). In all modes only odd fields are encoded, the even fields being discarded.
The Advanced Television Research Consortium (ATRC) in the United States has modified the MPEG format for transmission of high definition television (HDTV) signals in digital form. Generally, the initial signal coding of this HDTV signal is similar to MPEG except that the pixel resolution, and data rates are increased, and both odd and even frames of each field are coded. In the HDTV system the coded signal is prioritized between a higher and a lower priority channel for transmission. Coded data having apparent greater importance to picture reproduction is transmitted with a given power level and coded data of lesser importance is transmitted with a lesser power level, to minimize cochannel interference.
FIG. 1 is a pictorial representation of the coding format prior to prioritization. The frame sequence is merely representative. The letters I, P, and B above respective frames indicate the coding mode for the respective frame. The frame sequence is divided into groups of frames (GOF) each of which includes the same coding sequence. Each frame of coded data is divided into slices representing, for example, 16 image lines. Each slice is divided into macroblocks each of which represents, for example, a 16.times.16 matrix of pixels. Each macroblock is divided into 6 blocks including four blocks of information relating to luminance signal and two blocks of information relating to chrominance signal. The luminance and chrominance information are coded separately and then combined for transmission. The luminance blocks include data relating to respective 8.times.8 matrices of pixels. Each chrominance block comprises and 8.times.8 matrix of data relating to the entire 16.times.16 matrix of pixels represented by the macroblock.
Blocks of data, encoded according to intraframe coding, consist of matrices of Discrete Cosine Coefficients. That is, respective 8.times.8 blocks of pixels are subjected to a Discrete Cosine Transform (DCT) to provide coded signal. The coefficients are subjected to adaptive quantization, and before being applied to the priority processor are run-length and variable-length encoded. Hence respective blocks of transmitted data may include fewer than an 8.times.8 matrix of codewords. Macro blocks of intraframe encoded data, will include, in addition to the DCT coefficients, information such as the level of quantization employed, a macroblock address or location indicator, and a macroblock type.
Blocks of data encoded according to P or B interframe coding also consist of matrices of Discrete Cosine Coefficients. In this instance however the coefficients represent residues or differences between a predicted 8.times.8 pixel matrix and the actual 8.times.8 pixel matrix. These coefficients are also subjected to quantization and run- and variable-length coding. In the frame sequence I and P frames are designated anchor frames. Each P frame is predicted from the lastmost occurring anchor frame. Each B frame is predicted from one or both of the anchor frames between which it is disposed. The predictive coding process involves generating displacement vectors which indicate which block of an anchor frame most closely matches the block of the predicted frame currently being coded. The pixel data of the matched block in the anchor frame is subtracted, on a pixel-by-pixel basis, from the block of the frame being encoded, to develop the residues. The transformed residues and the vectors comprise the coded data for the predictive frames. As for intraframe coded frames the macroblocks include quantization, address and type information. Note that even though a frame is predictive encoded, if no reasonable block matches can be found, a particular block or macroblock in the predictive frame may be intraframe coded. In addition certain ones of the macroblocks may not be encoded. Macroblocks are skipped by increasing the address of the next coded macroblock.
After the video data is coded, it is arranged according to an MPEG-like protocol. The MPEG hierarchical format includes a plurality of layers each with respective header information as shown in FIG. 2. Nominally each header includes a start code, data related to the respective layer and provision for adding header extensions. Much of the header information (as indicated in the referenced MPEG document) is required for synchronization purposes in an MPEG systems environment. For purposes of providing a compressed video signal for a digital HDTV simulcast system, only descriptive header information is required, that is start codes and optional extensions may be excluded.
When referring to the MPEG-like signal produced by the present system what is meant is that a) successive fields/frames of video signal are encoded according to an I, P, B coding sequence, and b) coded data at the picture level is encoded in MPEG-like slices or group of blocks albeit that the number of slices per field/frame may differ and the number of macro blocks per slice may differ.
The coded output signal of the present system is segmented in groups of fields/frames (GOF) illustrated by the row of boxes L1 (FIG. 2). Each GOF (L2) includes a header followed by segments of picture data. The GOF header includes data related to the horizontal and vertical picture size, the aspect ratio, the field/frame rate, the bit rate, etc.
The picture data (L3) corresponding to respective fields/frames includes a header followed by slice data (L4). The picture header includes a field/frame number and a picture code type. Each slice (L4) includes a header followed by a plurality of blocks of data MBi. The slice header includes a group number and a quantization parameter.
Each block MBi (L5) represents a macroblock and includes a header followed by motion vectors and coded coefficients. The MBi headers include a macroblock address, a macroblock type and a quantization parameter. The coded coefficients are illustrated in layer L6. Note each macroblock is comprised of 6 blocks, including four luminance blocks, one U chrominance block and one V chrominance block.
The block coefficients are provided one block at a time with the DCT, DC coefficient occurring first followed by respective DCT AC coefficients in the order of their relative importance. An end of block code EOB is appended at the end of each successively occurring block of data.
Compressed video data hierarchically formatted as indicated in FIG. 2 is applied to a priority processor, wherein the coded data is parsed between a high priority channel HP and a low priority channel LP. High priority information is that information, the loss or corruption of which, would create the greatest degradation in reproduced images. Stated conversely, it is the least data needed to create an image, albeit less than a perfect image. Low priority information is the remaining information. The high priority information includes substantially all of the header information included in the different hierarchical levels plus the DC coefficients of the respective blocks and a portion of the AC coefficients of the respective blocks (level 6, FIG. 2).
For priority processing purposes, the respective types of encoded data are assigned priority classes or types. For example all information above slice header information (including the slice identifier, slice quantization parameter etc.) are assigned priority type "0". Macroblock header data is assigned priority type "1". Motion vectors are assigned priority type "2". Priority type "3" may be reserved. The coded block pattern is assigned priority type "4". DC DCT coefficients are assigned priority type "5" and successive codewords representing higher order DCT coefficients are assigned priority types "6" to "68". The priority processor determines, according to the relative amounts of higher and lower priority data, the priority types which will be allocated to the high and low priority channels. Note that the priority classification is indicative of the relative importance of the particular types of data with priority type "0" being the most important. The processor in effect determines a priority break point (PBP) which corresponds to the class or type number above which all data is designated to the low priority channel. The remaining type data is allocated to the high priority channel. Refer to FIG. 2 and assume that for a particular macroblock the PBP is determined to be "5", so that the DC coefficients and all hierarchically higher data is to be allocated to the HP channel, and all AC coefficients and the EOB codes are assigned to the LP channel. For transmission purposes all the HP codewords are concatenated in bit-serial form without demarcation of data from respective blocks. In addition, the codewords are variable length encoded and there are no separations between codewords (in order to realize the greatest effective bandwidth in a limited bandwidth channel). The PBP for corresponding macroblocks is transmitted so that the receiver has the requisite information for separating the HP data amongst the respective blocks. In the LP channel, data from respective blocks is separated by EOB codes.
The HP and LP compressed video data are applied to a transport processor which a) segments the HP and LP data streams into respective HP and LP transport blocks, b) performs a parity or cyclic redundancy check on each transport block and appends the appropriate parity check bits thereto, and c) multiplexes the auxiliary data with the HP or LP video data. The parity check bits are utilized by the receiver for isolating errors in conjunction with synchronizing header information and for providing error concealment in the event of uncorrectable bit errors in the received data.
FIG. 3 illustrates the format of the signal provided by the transport processor. Respective transport blocks may include more or less than a slice of data. Thus a particular transport block may include data from the end of one slice and data from the beginning of the next subsequent slice. Transport blocks including video data may be interleaved with transport blocks containing other data, e.g., audio. Each transport block includes a service type header ST which indicates the type of information included in the respective transport block. In this example the ST header is an 8-bit word which indicates whether the data is HP or LP, and whether the information is audio, video or auxiliary data.
Each transport block includes a transport header TH immediately following the ST header. For the LP channel the transport header includes a 7-bit macroblock pointer, an 18-bit identifier and a 7-bit record header (RH) pointer. The transport header of the HP channel includes only an 8-bit record header (RH) pointer. The macroblock pointer is used for segmented macroblock or record header components, and points to the start of the next decodable component. For example, if the particular transport block includes macroblock data associated with the end of slice n and the beginning of slice n+1, the data from slice n is placed adjacent the transport header and the pointer indicates that the next decodable data is adjacent the transport header TH. Conversely, if a record header RH is adjacent the TH, the first pointer indicates the byte position following the record header RH. A zero valued macroblock pointer indicates that the transport block has no macroblock entry point.
The transport block may include none, one or more than one record header. A record header occurs at the beginning of each slice of macroblock data in the HP and LP channel. No record headers are included in transport blocks that include only video data header information. The record header (RH) pointer points to the byte position containing the start of the first record header in the transport block. A zero valued RH pointer indicates that there are no record headers in the transport block. If both the record header pointer and the macroblock pointer are zero valued, this state indicates that the transport block includes only video data header information.
The 18-bit identifier in the LP transport header identifies the current frame type, the frame number (modulo 32), the current slice number, and the first macroblock contained in the transport block.
Following the transport header is either a record header, RH, or data. As indicated in FIG. 3 the record header for the video data in the HP channel includes the following information: A 1-bit FLAG which indicates if a header extension, EXTEND, is present. Following the FLAG is an identifier IDENTITY, which indicates a) the field/frame type I, B or P; b) a field/frame number (modulo 32) FRAME ID; and c) a slice number (modulo 64) SLICE IDENTITY. Following the identifier the record header includes a macroblock priority break point indicator, PBP. The PBP indicates the codeword class, developed by the analyzer 152 of the priority selector, for dividing the codewords between the HP and LP channels. Lastly, an optional header extension may be included in the HP record header.
The record header incorporated in the LP channel includes only an identifier, IDENTITY, similar to the identifier implemented in the HP channel.
Each transport block is terminated with a 16-bit frame check sequence, FCS, which is calculated over all bits in the transport block. The FCS may be generated using a cyclic redundancy code.
The transport blocks of information are applied to respective forward error encoding elements which a) perform REED SOLOMON forward error correction encoding independently to the respective data streams; b) interleave blocks of data to preclude large error bursts from corrupting a large contiguous area of a reproduced image; and c) appends, e.g., Barker codes to the data for synchronizing the data stream at the receiver.
A receiver, responsive to transmitted signals which are formatted as indicated above, includes apparatus for performing inverse prioritization and inverse coding. Inverse prioritization, or recombining of the HP and LP data must be performed before decoding can be accomplished, because the decoder expects to see data in a predetermined format (similar to that shown in FIG. 2). It should readily be appreciated that at least a portion of the received signal will be corrupted by the transmission process. Consider that the PBP code in a HP transport block is lost. Without this PBP code, information corresponding to the respective blocks of a macroblock cannot be separated. As a result a considerable portion of the information contained in the HP transport block may be rendered useless. In addition information in the LP transport block, corresponding to blocks contained in the HP transport block, is also rendered unusable. In fact, the loss of a single PBP codeword contained in a HP transport block can render otherwise valid data for an entire slice useless. A second example is the loss of, for example, the codeword in a picture header which designates the frame coding type. Without this codeword an entire frame of coded data is rendered unusable or at least unreliable.