1. A method for encoding or decoding video content, the method comprising: determining a mapping between interpolation filters in a set of interpolation filters and sizes of different prediction units within the video content, the set of interpolation filters for use in interpolating sub-pel pixel values; receiving a first prediction unit of the video content; determining a size of the first prediction unit; determining an interpolation filter in the set of interpolation filters to interpolate a sub-pel pixel value for use in a temporal prediction process for the first prediction unit by comparing the size of the first prediction unit to the mapping to select the interpolation filter in the set of interpolation filters associated with the size of the first prediction unit in the mapping; when the one or more computer processors includes an encoder, transmitting the mapping between the interpolation filters and the sizes from the encoder to a decoder and not signaling, from the encoder to the decoder, which interpolation filter was determined to interpolate the sub-pel pixel value during the temporal prediction process; and when the one or more computer processors includes the decoder, determining the mapping between the interpolation filters and the sizes by receiving the mapping between the interpolation filters and the sizes at the decoder from the encoder and not receiving a signal at the decoder from the encoder regarding which interpolation filter was determined to interpolate the sub-pel pixel value during the temporal prediction process.