1. A method for encoding a video signal having a plurality of frames, comprising: selecting a filter set of frames from the plurality of frames, the filter set including at least three frames of the plurality of frames, one of the at least three frames being an anchor frame; generating an alternate reference frame for at least some of the plurality of frames by, for each block of a plurality of blocks of the anchor frame: determining a first weighting factor, for each corresponding block of a respective frame of the filter set, that represents a temporal correlation of the block with the corresponding block; determining a second weighting factor, for each pixel for each corresponding block of the respective frame of the filter set, that represents a temporal correlation of the pixel to a spatially-correspondent pixel in the block; determining a filter weight for each pixel in the block and for each spatially-correspondent pixel is each corresponding block based on the first weighting factor and the second weighting factor; and generating a weighted average pixel value for each pixel position in the block to form a block of the alternate reference frame based on the filter weights; using the alternate reference frame to encode at least one block within at least one frame of the plurality of frames for transmission in a bitstream; and transmitting the alternate reference frame in the bitstream.