This is because the viewers attention is typically directed to the center of the image and not to the edges of the image.
The calculation of the motion vectors for these blocks is less important than for blocks near the central portion of the image.