There is tradeoff between using fewer image blocks and degradation in accuracy.
The number of selected image blocks should not be less than four in order to get reliable results, especially when a perspective model, such as the one shown in.