1. A method of modifying video data to encode metadata, comprising: receiving in a video processor a sequence of video images, each image having metadata associated with it, the metadata associated with each of the video images differing from the metadata associated with at least one other of the video images, each video image comprising pixel data defining pixel values of that image; for an image, collecting in the video processor the metadata associated with that image and the metadata associated respectively with N of the images preceding that image in the sequence of images, where N is equal to or greater than four; and modifying in the video processor the pixel data of an image to encode the metadata associated with that image and to encode the metadata associated respectively with N of the images preceding that image in the sequence of images.