1. A motion vector detecting apparatus comprising: (a) a memory for holding video signals which are coded and composed of frame units, (b) a horizontal differentiating filter for reading out luminance data from the memory and differentiating in a horizontal direction of arbitrary image coordinates, (c) a vertical differentiating filter for reading out luminance data from the memory and differentiating in a vertical direction of the image coordinates, (d) a time differentiating filter for reading out the luminance data of preceding and succeeding frames from the memory, and differentiating in a time direction, (e) feature vector combining means for combining a luminance differential vector (Ix, Iy, It), including results from the horizontal differentiating filter, the vertical differentiating filter and the time differentiating filter, with a position vector (x, y) of the image coordinates, to obtain a sample vector, (f) a memory for holding a plurality of estimations of a motion vector (u, v) and a position vector mean (x, y) of the sample vector as class data, (g) nearest class determining means for determining a distance between the class data and the sample vector from a formula (x-x, y,-y) and a formula (u*Ix+v*Iy+It), and achieving a correspondence between the sample vector and the nearest class data, and (h) class data changing means for changing the class data in a manner that a distance sum between the class data and one or more sample vectors becomes smaller, from one or more sample vectors and the class data of which correspondence has been achieved by the nearest class determining means.