1. A method of encoding video data, the method comprising: processing a plurality of source video frames having non-overlapping input blocks to be encoded; and for each input block to be encoded, applying continuous block tracking (CBT) for initial motion estimation within a model-based inter-prediction process to produce CBT motion vector candidates by: providing frame-to-frame tracking of each input block in a current frame referenced to a source video frame resulting in a set of frame-to-frame motion vectors configured to specify, for each input block, a location of a matching region in a temporally-adjacent source video frame; and providing continuous tracking over multiple reference frames by relating frame-to-frame motion vectors over the multiple reference frames, the continuous tracking resulting in a set of continuous tracking motion vectors configured to specify, for each input block, a location of a matching region in a temporally non-adjacent source video frame; and combining the CBT motion vector candidates with enhanced predictive zonal search (EPZS) motion vector candidates to form an aggregate set of initial CBT/EPZS motion vector candidates in a unified motion estimation framework.