1. A computer-implemented method of real-time encoding of a multi-view video stream of a scene being captured by a plurality of video cameras, method comprising: receiving the multi-view video stream, the multi-view video stream comprising a plurality of video streams, each stream comprising a series of video frames of a scene captured by a corresponding different video camera, and a macro-frame comprising a single frame from each video stream, the frames of a macro-frame having been captured substantially concurrently; estimating a probability distribution of viewpoints; compressing the multi-view video based on the probability distribution; and transmitting the compressed multi-view video to a remote device that displays the multi-view video.