1. A video decoding system in which video objects are recognized from video data, wherein instances of a video object at given times are coded as video object planes (VOPs) and VOPs are assigned to one or more video object layers, the video decoder system comprising a decoder structure, the decoder structure further comprising: a base layer decoder having an input for VOP data associated with a first video object layer of the video object; a processor coupled to an output of the base layer decoder; and an enhancement layer decoder, having a first input for VOP data associated with a second video object layer of the video object and a second input coupled to the processor and responsive to predictive coded VOP (P-VOP) data including a ref_select_code included therein, the enhancement layer decoder decoding the P-VOP data with reference to one of: data of a VOP most recently decoded by the enhancement layer decoder; data of a most recent VOP in a display order decoded by the base layer decoder; data of a next VOP in a display order decoded by the base layer decoder; and data of a temporally coincident VOP by the base layer decoder.