before presenting a description of the particular implementations of the present invention , it should be explained that the wide range of applications in which digital video is expected to play a role , imposes many conflicting requirements for video compression algorithms . these conflicts are manifested in terms of standards compatibility , encoder and decoder implementation complexity , functionality , image quality , etc . it is believed that these conflicting requirements cannot be satisfied by a single coding algorithm , but instead require a flexible architecture of algorithms which can be matched to the requirements of the specific application . such flexibility may be used to satisfy many of these conflicting requirements , while still preserving a great deal of compatibility amongst the different manifestations of the architecture . it is believed , for example , that while different encoder implementations may be needed for different applications , it should be possible to implement a single decoding device which can decode all bitstreams that conform to the flexible architecture . it is also believed that such a generic decoder should not be excessively complex . one object of this invention is to provide such a generic decoder . several applications will benefit from resolution and bitstream scalability features as described above . the methods and apparatus of the invention support resolution and bitstream scaling by hierarchical coding of 8 × 8 dct components . although dcts of other sizes could be used , the preferred embodiment starts with 8 × 8 dcts because that is the choice of several standard compression algorithms . for resolution scaling , the invention may be used to provide , for example , up to four levels of resolution . the lowest possible resolution is attained by coding the equivalent of the upper left component of an 8 × 8 dct block ; this resolution is 1 / 64 of the original resolution . next a resolution of 1 / 16 of the original resolution may be obtained by coding of the equivalent of the upper left 2 × 2 coefficients of the dct block . coding of the equivalent of the upper left 4 × 4 coefficients leads to a resolution of 1 / 4 of the original resolution . finally , the coding of all 8 × 8 coefficients leads to the full resolution video . given a fixed resolution , the invention supports bitstream scaling by coding multiple hierarchical layers at the same resolution . but , with progressively finer quantization factors , these layers would result in video of the same spatial resolution and of incresing quality . in this case , the first and lowest layer of the hierarchy is coded with coarse quantization and the higher layers are coded with quantization of increased precision . the architecture is flexible : one or more of these hierarchical layers can be stacked up in order of increasing resolution or precision , such that the reconstructed coefficients at one level of resolution are used to predict the corresponding coefficients at the next level of resolution . except for the lowest hierarchical layer , transform coefficients in any layer are coded differentially with respect to their prediction . an encoder might be chosen , for example , to generate a bitstream that only contains data for the full and the 1 / 16 resolution scales . in such case , the coefficients at the 1 / 16 resolution layer would be used to predict the coefficients at the full resolution layer . alternatively , for bitstream scalability , a coarsely quantized set of 8 × 8 dct coefficients in one layer can serve as prediction for the corresponding set of 8 × 8 coefficients in the next layer , which are more finely quantized . the basic characteristic of the architecture of the invention is that the identity of the macroblock ( mb ) structure of mpeg - 1 is preserved across all resolution and precision layers . fig6 shows this feature for the case of coding with four levels of resolution . it is important to preserve this identity because , as noted in the description of the mpeg - 1 algorithm , an mb is associated with a series of attributes which contribute to the amount of overhead data incorporated in a compressed data stream . preserving the mb identity permits the reuse of this overhead data for all hierarchical layers . for example , the motion vector data corresponding to any resolution scale can be derived from the highest resolution motion vector data by appropriate scaling . although not the subject of this invention , it is important to note that , before transmission or storage , the data for the various scales must be multiplexed by an encoding device . there are many options for implementing this multiplexing . for example , the data for a full picture at each level of hierarchy could be catenated in the order of increased resolution and precision . it will be understood that the data contains signals representative of information related to the picture elements or pixels of the video images being handled . these signals are typically electrical signals that are processed in appropriate electronic devices , such as video cameras , computers , and ancilliary equipment as will be thoroughly familiar to and understood by those of skill in the art . in the preferred embodiment , the quantization of each hierarchical layer uses the same matrices specified in mpeg - 1 . thus , the dct data for the various resolution scales is derived from the full resolution 8 × 8 dct matrix . if the mpeg - 1 quantization matrix of weights is denoted by q8 , the matrix of weights for lower resolution dcts are derived as follows : ______________________________________quantizer dct factor______________________________________q1 1 × 1 1 / 8 q8q2 2 × 2 1 / 4 q8q4 4 × 4 1 / 2 q8______________________________________ it should be noted that the present invention also covers the case where generic quantization matrices are used at each scale , but this only makes the decoding device more complex . in the preferred embodiment , dct coefficients in a hierarchical layer are used to predict the corresponding coefficients in the next layer of the hierarchy . an example is illustrated in fig7 wherein a hierarchy of 4 resolution layers is shown . the prediction algorithm is a simple one - to - one mapping of the properly scaled coefficients . however , it should be noted that other prediction algorithms could be used again at the cost of increased complexity . in the preferred embodiment the quantization parameter of mpeg - 1 , qp , is used at the lowest layer of the hierarchy . qp parameters at other layers of the hierarchy are specified with reference to this lower layer qp . for example , a high layer qp parameter could be specified to be twice the lower layer qp . the invention involves an architecture with a flexible number of hierarchical layers . however , for ease of explanation , two three - layer decoders that exemplify the resolution anti bitstream scalability features will be described . the decoder apparatus , which is shown in fig8 supports 2 × 2 ( low ), 4 × 4 ( medium ), and 8 × 8 ( high ) resolution scales . decoders with only one target resolution scale can be implemented by eliminating the boxes in fig8 that are not necessary to achieve the target . after entropy decoding and demultiplexing the compressed data for the three resolution scales , there will be for every 8 × 8 block data , corresponding 2 × 2 and 4 × 4 block data , all of which are necessary to build the final 8 × 8 matrix of dct coefficients . in the preferred embodiment , the following steps are followed to arrive at the full resolution 8 × 8 dct coefficients . after dequantizing by the qp 2 quantization parameter of the 2 × 2 layer , the low resolution 2 × 2 blocks are used as a prediction to the four lowest order coefficients of the corresponding 4 × 4 blocks . these predictions are summed to the dequantized 4 × 4 coefficients , where dequatization of the 4 × 4 coefficients is performed by the qp 4 quantization parameter . the results of the previous sum are similarly used as prediction to the 16 lowest order coefficients of the corresponding 8 × 8 blocks . these predictions are summed to the dequantized 8 × 8 coefficients , where dequantization of the 8 × 8 coefficients is performed by the qp 8 quantization parameter . note that the dct coefficients are only dequantized by quantization parameters as the final matrix of coefficients is rebuilt . dequantization by the quantization matrix of weights is only needed once the final resolution is reached . this feature is possible because , in the preferred embodiment , the matrix of weights at the various scales are proportionally related as explained above . the final 8 × 8 matrix of coefficients can now be used to reconstruct the full resolution picture using mpeg - 1 techniques , including motion compensated prediction . referring to fig8 in this regard , it will be appreciated that at the full resolution level the 16 × 16 mcp unit represents a generic mpeg - 1 motion compensation prediction unit operating on an mb ; the idct box is a unit for performing a standard mpeg - 1 8 × 8 inverse transformation ; and the q8 - 1 box represents a unit for performing inverse quantization by the corresponding mpeg - 1 matrix of weights . to decode video pictures at other resolution scales a similar procedure is followed , except that the process of summing predictions to dequantized coefficient data stops at the resolution at which it is desired to decode . the operation of dequantization by a matrix of weights , represented by q4 - 1 , and q2 - 1 , uses the scaled matrices of the preferred embodiment . the operation of inverse transformation is performed by using a transform of the appropriate size . thus for decoding at 1 / 4 resolution a 4 × 4 inverse dct should be used . the one - dimensional dct matrices appropriate for decoding at the three &# 34 ; scaled &# 34 ; resolutions supported are : ## equ1 ## both the dct ( 1 × 1 ) and the dct ( 2 × 2 ) are trivial and should be easy to implement , even in software . it should be noted that mcp units at other resolutions share the same motion vector data , mv , to generate motion compensated predictions for the scaled mbs at the various resolution scales . of special note is that when using motion compensation techniques , the full resolution motion vector should be scaled appropriately to match the decoder resolution . this has been described previously . a decoder that implements bitstream scalability is shown in fig9 . the operation of this decoder is very similar to that of fig8 except that only 8 × 8 operations are used to produce output pictures of increasing levels of quality . for this reason the various units ( e . g ., q8 - 1 , idcts , and mcp , etc ), can be physically implemented by single hardware and shared by the various hierarchical layers . finally , an alternative decoder implementation that is not part of the preferred embodiment , but in keeping with the invention , is shown in fig1 . this decoder operates in the same manner as that in fig8 . however , it permits any quantization weight matrices to be used in the various hierarchical layers . it also implements generic means of predicting dct coefficients from a lower hierarchical layer . in this sense p2 is used to predict 4 × 4 coefficients from 2 × 2 coefficients , and p4 is used to predict 8 × 8 coefficients from 4 × 4 coefficients . there are many prediction algorithms that may be used , including the one - to - one mapping of the preferred embodiment . there are many possible implementations of encoders compatible with the decoders of the present invention described above , however , by way of example , two such encoders , each of which is only designed for resolution scaling , will be described . the general structure of a three resolution layer encoder is shown in fig1 . the encoder is divided into three parts . the first part is a transform unit which takes the digital video input and outputs dct data for the three resolution layers : d ( 8 × 8 ), d ( 4 × 4 ), and d ( 2 × 2 ). the second part is an hierarchical prediction unit which takes the tranform unit &# 39 ; s dct output , and outputs quantized differential dct data at all resolution layers : q ( 8 × 8 ), q ( 4 × 4 ), and q ( 2 × 2 ). this output is multiplexed and entropy coded in the third unit to generate the final compressed video . the prediction unit also generates reconstructed dct data : b ( 8 × 8 ), b ( 4 × 4 ), and b ( 2 × 2 ), which is fed back to the transform unit to complete the loop of typical hybrid transform codes . fig1 a shows a simple implementation of the transform unit . in this implementation the 8 × 8 layer contains elements which are also part of mpeg - 1 encoders and more generically , part of motion compensated hybrid transform encoders . the upper branch contains : a summer ( σ ) to generate a motion compensated prediction difference ; a forward 8 × 8 dct transform to generate the 8 × 8 transform coefficients of said prediction difference ; and a unit for quantization by a matrix of weights ( q8 ). the output is a set of partially quantized dct coefficients , d ( 8 × 8 ). the return or feedback branch receives a set of partially reconstructed dct coefficients , b ( 8 × 8 ), and then processes them with the following units : an inverse quantizer for the matrix of weights ( q8 - 1 ) and inverse dct transformer ( idct 8 × 8 ) that reconstructs the prediction differences ; a summer to add the reconstructed prediction differences to the motion compensated prediction thus reconstructing the original picture data ; and finally , a motion compensation prediction ( mcp ) unit to generate a prediction for the next picture . in mpeg - 1 this mcp unit operates on 16 × 16 mbs as shown in the figure , but other block sizes could be used . in this implementation of the transform unit , the d ( 2 × 2 ) and d ( 4 × 4 ) coefficients are simply extracted from the corresponding 8 × 8 coefficients . it should be noted that it is also possible to derive the d ( 2 × 2 ) and d ( 4 × 4 ) coefficients through other reduction or weighting algorithms applied to the 8 × 8 coefficients . note that because there is no feedback loop in the lower resolution scales , this encoder will result in accumulation of quantization and motion compensation errors at these resolution scales . the error , however , will be naturally reset back to zero whenever a new group of pictures starts . while the quality of the lower resolution layers will be limited by this accumulation of errors , the simplicity of the encoder makes this approach attractive . in particular , if what is required is only bitstream scalability , this approach is all that is needed . another implementation of the transform unit is shown in fig1 b . this version is similar to the one in fig1 a , however , d ( 2 × 2 ) and d ( 4 × 4 ) are generated by similar , but completely independent loops operating at each resolution scale . in this sense , the h 4 and h 2 refits are used to filter and reduce the resolution of the input video by a factor of 1 / 4 in each case . in this manner each layer takes an input of the appropriate resolution . all operations such as dct , quantization , and mcp are also scaled according to the resolution of the layer . at a cost of increased complexity , this version would generally produce better quality low - resolution pictures than those produced by the unit in fig1 a . in this case , coding errors will not accumulate more than one picture period . note , however , that the results of the 16 × 16 motion estimation can be shared by all resolution loops since , in the invention , motion vectors are one of the attributes shared by mbs at all scales . this implementation is more appropriate for applications where the quality of the low resolution video is important . fig1 a shows one implementation of the hierarchical prediction unit . first the hierarchical prediction differences for the 4 × 4 and 8 × 8 layers are generated in the summers . all layers are then quantized by their respective quantizer parameters and the results are output as q ( 2 × 2 ), q ( 4 × 4 ), and q ( 8 × 8 ). these results are also inverse quantized by the corresponding quantizer parameters and then added in the two other summers to generate the partially reconstructed b ( 2 × 2 ), b ( 4 × 4 ), and b ( 8 × 8 ) data , which is fed back to the transform unit . fig1 b shows a rearrangement of the same elements which can be used as an alternative implementation for the hierarchical prediction unit . it should be noted that one skilled in the art can design further encoding alternatives that are compatible with the decoding methodology and architecture of the present invention .