1. A distance measurement system, comprising: a plurality of reference planes; a light source module, provided a light beam having a speckle pattern to the reference planes and an object, so that a plurality of images having the speckle pattern are shown on the reference planes and a surface of the object respectively, wherein the speckle pattern has a plurality of speckles; an image capture device, capturing the images having the speckle pattern on each reference plane to generate a plurality of reference image information, and capturing the image having the speckle pattern on the surface of the object to generate an object image information; and a processing module, coupled to the image capture device so as to compare brightness relationship between each of the speckles and the other speckles rounding the each of the speckles to constitute a brightness information of each of speckles, compute the position of each of the speckles on each of the reference image information based on the brightness information, and compare the position of each of the speckles on the object image information with the position of each of the speckles on each of the reference image information to obtain a plurality of similarity scores; wherein when the highest similarity score is less than a threshold value, the processing module ignores the distance measurement result, and when the highest similarity score is greater than or equal to the threshold value, the processing module computes the position of the object according to the position of corresponding reference plane with the highest similarity score.