1. A computerized method comprising: receiving motion prediction errors corresponding to motion blocks in an input frame that occurs within a video sequence of frames, wherein the received motion prediction errors are associated with a single-motion model; selecting a motion block if the corresponding received motion prediction error satisfies a predetermined criterion; calculating a plurality of motion prediction errors for the selected motion block relative to a first reference frame and relative to a second reference frame, wherein the input frame, the first reference frame, and the second reference frame are different frames in the video sequence; partitioning, by a computer, the selected motion block into an arrangement that minimizes a value of each of the plurality of the motion prediction errors, the arrangement comprising a first region of pixels that are predicted relative to the first reference frame and a second region of pixels that are predicted relative to the second reference frame, wherein the first reference frame is a past frame and the second reference frame is a future frame; deriving a multiple-motion model for the selected motion block from the arrangement; and encoding the input frame by using both the multiple-motion model and the single-motion model.