1. A method of encoding video data, the method comprising: determining a candidate motion vector for a motion vector prediction process for each of one or more candidate portions of a current video frame, wherein the one or more candidate portions are causal to a current portion of the current video frame; determining a current motion vector for the current portion; determining whether each of the candidate motion vectors and the current motion vector refer to the same reference video frame; and performing the motion vector prediction process in the case that each of the candidate motion vectors and the current motion vector are determined to refer to the same reference video frame, whereby signaling of a reference index indicating the reference video frame is avoided by inferring the reference index from the candidate motion vectors of the one or more candidate portions at a video decoder, the motion vector prediction process comprising: calculating a motion vector difference between the current motion vector and each of the candidate motion vectors; selecting one of the candidate motion vectors based on the calculated motion vector differences; not signaling the reference index; signaling an index identifying the candidate portion having the selected one of the candidate motion vectors; and signaling the motion vector difference calculated with respect to the selected one of the candidate motion vectors.