1. A method of encoding raw video data, comprising: receiving multiple frames of raw video data; encoding the multiple frames of the raw video data to make an H.264 macroblock encoding; identifying, in the H.264 macroblock encoding, a groups of pels in close proximity to each other exhibiting encoding complexity, such that the group of pels of the H.264 macroblock encoding use a disproportionate amount of bandwidth computationally relative to other regions in one or more of the multiple frames of raw video; responding to the identified group of pels by forming tracking information including: detecting, in the identified group of pels, at least one of a feature or an object in a region of interest of at least one frame of the raw video data, the region of interest of the detected at least one feature not being aligned with the underlying macroblock grid; modeling the detected at least one of the feature and the object using a set of parameters; and associating any instances of the detected and modeled at least one of the feature or the object across plural frames of the raw video data providing at least one feature or object track of the associated instances, each feature or object track providing tracking information of respective associated instances; relating the at least one feature or object track to at least one macroblock of the raw video data to be encoded; producing an indirect model-based prediction of the at least one macroblock of the raw video data using the tracking information of the at least one related feature or object track, by using offsets between (i) the at least one macroblock of the raw video data and (ii) respective instances from the at least one related feature or object track to generate indirect predictions for the at least one macroblock of the raw video data, such that the feature or object track information is used indirectly to predict macroblocks instead of directly to predict the at least one feature or object, the indirect model-based prediction having model-based motion vectors; comparing the compression efficiency of a standards-compliant encoding derived from the model-based motion vectors with the compression efficiency of the H.264 macroblock encoding of the groups of pels in close proximity to each other exhibiting encoding complexity; caching the model-based motion vectors if it is determined that the standards-compliant encoding derived from the model-based motion vectors provides improved compression efficiency relative to the H.264 macroblock encoding of the groups of pels in close proximity to each other exhibiting encoding complexity; and incorporating the model-based motion vectors into a standards-compliant bit stream such that the model-based prediction is stored as standards-compliant encoded video data.