1. A video decoder comprising: a predictor for forming a prediction from reconstructed residuals; and an adder for adding the obtained prediction to differential residual samples to obtain reconstructed residual samples, wherein differential residuals are formed from residuals by performing sample-based prediction according to one of the following modes: no prediction, prediction along horizontal direction and prediction along a vertical direction, wherein for inter coded blocks the use of the sample-based residual prediction is decoded from the bit-stream via two binary flags: one to indicate the use of sample-based residual prediction and the other to indicate the direction to perform sample-based residual prediction.