1. A method of modifying video data to encode metadata, comprising: receiving in a video processor video in the form of a sequence of images, each image having metadata associated with it, the metadata associated with each of the images differing from the metadata associated with at least one other of the images, each image comprising pixel data defining pixel values of that image; for a current image in the sequence, collecting in the video processor the metadata associated with that image and the metadata associated with N of the images preceding the current image in the sequence of images, where N is equal to or greater than four; and modifying in the video processor the pixel data of the current image to encode the metadata associated with the current image and to encode the metadata associated respectively with N of the images preceding the current image in the sequence of images.