1. A method for generating a 3D representation of a dynamically changing 3D scene including human players, comprising the steps of: a) acquiring at least two video streams from at least two cameras located at different locations, observing the same 3D scene, and outputting texture data of said at least two video streams for all frames, wherein each of the at least two cameras are side view cameras which do not look down at a surface of a playing field in a vertical direction; b) determining camera parameters, which comprise the position, orientation and internal parameters, for said at least two cameras; c) tracking the 2D movement of players and determining 2D shapes of the players in the at least two video streams based only on the texture data of said at least two video streams, wherein the 2D shapes of players is determined by evaluating the texture data and defining an area that includes at least one of a player's position and size; d) determining the 3D position of the players by combining information from the at least two video streams to generate a 3D computational model of the 3D scene which includes data for rendering a synthesized view of the 3D scene from a virtual viewpoint that is distinct from viewpoints associated with the locations of the at least two cameras; and e) computing extrapolated 2D positions of the players in the at least two video streams from the 3D positions of the players; wherein the step of tracking the 2D movement of the players in the at least two video streams and determining the 2D shapes of the players in the at least two video streams uses said computed extrapolated 2D positions of the players in the at least two video streams that stem from one or more earlier instants in time.