1. An image processing device comprising: a temporal prediction control unit configured to control, in a prediction of a motion vector, whether a temporal prediction is to be performed to predict the motion vector by using a motion vector of a neighbor region temporally adjacent to a current region being processed; and a motion vector decoding unit configured to predict the motion vector of the current region by performing only a spatial prediction to predict the motion vector by using a motion vector of a neighbor region spatially adjacent to the current region or performing both the spatial prediction and the temporal prediction under the control of the temporal prediction control unit, and reconstruct the motion vector of the current region by using the predicted value, wherein the temporal prediction control unit controls whether the temporal prediction is to be performed in accordance with a number of times a temporally-predicted motion vector that is the motion vector of the current region predicted through the temporal prediction has been adopted as a predicted value of the motion vector of the current value, and wherein the temporal prediction control unit and the motion vector decoding unit are each implemented via at least one processor.