1. A method for processing keypoint trajectories in a video, wherein the keypoint trajectories describe motion of a plurality of keypoints across pictures of the video over time, comprising steps of: acquiring the video of a scene using a camera; detecting keypoints and associated feature descriptors in each picture; matching the keypoints and associated features descriptors between neighboring pictures to generate keypoint trajectories; coding predictively the keypoint trajectories into a bitstream; and outputting the bitstream, wherein the steps are performed in a processor.