1. A method of generating a three dimensional representation of a scene, which includes a plurality of objects disposed on a plane, the three dimensional representation being generated from one or more video images of the scene, which video images include the objects on the plane produced from a view of the scene by a video camera, the method comprising: processing the captured video images so as to extract one or more image features from each object, comparing the one or more image features with sample image features from a predetermined set of possible example objects which the video images may contain, identifying the objects from the comparison of the image features with the stored image features of the possible example objects, generating object path data, which includes object identification data for each object which identifies the respective object; and provides a position of the object on the plane in the video images with respect to time; calculating a projection matrix for projecting the position of each of the objects according to the object path data from the plane into a three dimensional model of the plane, for generating the three dimensional representation of the scene, by projecting the position of the objects according to the object path data into the plane of the three dimensional model of the scene using the projection matrix.