1. An image processing apparatus comprising: a processor configured to execute: a detection unit configured to detect segments from a first image; a sorting unit configured sort and prioritize the detected segments based on an order of length, wherein a longer segment has a higher priority than a shorter segment; a selection unit configured to select pairs of segments formed of a predetermined number of segments counted from a higher priority in the order of the length among the sorted and prioritized detected segments and segments in a second image, the second image being a model of the first image; a calculation unit configured to calculate distortion of the first image with respect to the second image by using the pairs of segments; and a correction unit configured to correct layout information by using an affine transformation amount determined based on a degree of coincidence between the first image and the second image, the degree of coincidence being a number of pairs of the closest relationship arranged within a predetermined distance among feature points detected from the first image and feature points in the second image.