1. (after amendment) A moving image encoding apparatus which separates one moving image sequence to a lower layer having a low frame rate and an upper layer having a high frame rate, encodes an area of a part of said lower layer as a parts image in said upper layer, encodes a shape of a parts area for synthesizing a background of the parts image of said upper layer, on the condition that there is no frame of the lower layer corresponding to the upper layer, and encodes the upper layer by prediction, wherein when the parts area of the lower layer appearing as a background is larger than a predetermined threshold, the moving image encoding apparatus encodes the shape of the parts area, and synthesizes a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer to generate image information of the background of said parts image, and when said parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the moving image encoding apparatus does not encode the shape of the parts area and generates image information by the weighted average of the lower layer.