1. A method of encoding video data, the method comprising: (i) selecting a first set of prediction modes; (ii) calculating an approximated cost of each of the first set of prediction modes; (iii) creating a second set of prediction modes including an prediction mode in the first set of prediction modes having a minimum approximate cost compared to other prediction modes in the first set of prediction modes; (iv) for each of other prediction modes in the first set of prediction modes: adding the one of the other prediction modes in the first set of prediction modes to the second set of prediction modes provided that it is not probable enough that, based on its approximate cost value, it will have a higher RDO (Rate Distortion Optimization) cost than a mode having a minimum approximate cost; (v) calculating an RDO cost for each prediction mode in the second set of prediction modes; and (vi) selecting a prediction mode from the second set of prediction modes having a minimum RDO cost compared to other prediction modes in the second set of prediction modes.